Driving data correction method and device, equipment and storage medium
By identifying the error types in driving data detected by vehicle-mounted vision sensors and making targeted corrections, and by utilizing vehicle-mounted radar sensors and distributed correction methods, the problem of low accuracy in driving data was solved, thereby improving the accuracy and safety of driving route planning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ZHEJIANG LEAPMOTOR TECH CO LTD
- Filing Date
- 2022-12-21
- Publication Date
- 2026-05-15
AI Technical Summary
Driving data detected by vehicle-mounted vision sensors has low accuracy, affecting driving route planning and even leading to safety accidents.
By acquiring the first state data of the target object, the error type is determined, and the corresponding data correction method is selected for correction, including correction of jump error, global error, local error and feedback error. Calibration is performed using data from the vehicle radar sensor, and data correction is performed using normal distribution and histogram distribution.
It improves the accuracy of driving data, ensures the precision of driving route planning, and reduces safety risks.
Smart Images
Figure CN116012809B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of visual inspection technology, and in particular to a method, apparatus, device, and storage medium for correcting driving data. Background Technology
[0002] During vehicle operation, it is usually necessary to acquire some driving data, such as the distance between the vehicle and surrounding target objects, and the heading angle of the target vehicle, in order to plan the driving path.
[0003] However, when the above driving data is obtained based on images detected by vehicle-mounted vision sensors, the accuracy of the driving data is low due to factors such as camera manufacturing process, deviation in manual camera placement, calibration of internal and external parameters, distortion correction, uneven road surface, and inaccurate detection frame. This is not conducive to driving route planning and may even lead to vehicle safety accidents.
[0004] Therefore, improving the accuracy of driving data obtained from vehicle-mounted vision sensors has become an urgent technical problem to be solved. Summary of the Invention
[0005] The main technical problem solved by this invention is to provide a method, apparatus, device and computer-readable storage medium for correcting driving data, which can improve the accuracy of driving data obtained based on vehicle vision sensors.
[0006] To address the aforementioned technical problems, this application provides a method for correcting driving data. The method includes: acquiring at least one first state data of the target object based on images of target objects around the vehicle detected by an onboard vision sensor; determining a target error type corresponding to the first state data in response to an error in the first state data; selecting a data correction method corresponding to the target error type from a correction correspondence as the target correction method; wherein the correction correspondence includes several preset error types and their corresponding data correction methods, and the target error type belongs to several preset error types; and correcting the first state data of the target object based on the target correction method corresponding to the first state data.
[0007] Among them, several preset error types include at least one of the following: jump error, global error, local error, and feedback error.
[0008] The first state data of the target object includes the first state values of the target object at different times during its lifecycle. Determining the target error type corresponding to the first state data includes: obtaining the first proportion of the number of first state values in the first state data that meet the first preset condition in the total number of first state values in the first state data; wherein the first preset condition includes: the absolute value of the difference between the first state value and the first state value at the previous time is greater than the difference threshold; and determining the target error type corresponding to the first state data based on the first proportion and the first proportion threshold.
[0009] Specifically, based on the first proportion and the first proportion threshold, the target error type corresponding to the first state data is determined, including at least one of the following: in response to the first proportion being less than the first proportion threshold, the target error type corresponding to the first state data is determined to be a jump error; in response to the first proportion being not less than the first proportion threshold, the target error type corresponding to the first state data is determined to be a feedback error.
[0010] Before determining the target error type corresponding to the first state data, the method further includes: acquiring second state data that matches the first state data of the target object based on the radar information of the target object detected by the vehicle-mounted radar sensor; determining the target error type corresponding to the first state data includes: determining the target error type corresponding to the first state data based on the second state data that matches the first state data.
[0011] The first state data includes the first state values of the target object at different times during its lifecycle, and the second state data includes the second state values of the target object at different times during its lifecycle. Based on the second state data that matches the first state data, the target error type corresponding to the first state data is determined, including: obtaining the second proportion of the number of first state values in the first state data that meet a second preset condition to the total number of first state values in the first state data; wherein the second preset condition includes: the error between the first state value and its second state value at the same time is greater than an error threshold; and the target error type corresponding to the first state data is determined based on the second proportion and the second proportion threshold.
[0012] Specifically, based on the second proportion and the second proportion threshold, the target error type corresponding to the first state data is determined, including at least one of the following: in response to the second proportion being equal to 1, the target error type corresponding to the first state data is determined to be a global error; in response to the second proportion not being greater than the second proportion threshold, the target error type corresponding to the first state data is determined to be a local error; in response to the second proportion being greater than the second proportion threshold and less than 1, the target error type corresponding to the first state data is determined to be a feedback error.
[0013] Specifically, when the target error type corresponding to the first state data is a jump error or a local error, the first state data of the target object is corrected based on the target correction method corresponding to the first state data, including: obtaining at least one of the normal distribution and histogram distribution corresponding to the first state data as correction reference data; and correcting the first state data based on the correction reference data.
[0014] Wherein, when a normal distribution is obtained as the correction reference data, the first state data is corrected based on the correction reference data, including: obtaining the average value of the data in the first state data that are distributed within a set interval of the normal distribution, and correcting the data in the first state data that are distributed outside the set interval of the normal distribution based on the average value, wherein the set interval is determined based on the mean and standard deviation of the normal distribution; and / or, when a histogram distribution is obtained as the correction reference data, the first state data is corrected based on the correction reference data, including: obtaining the average value of the data in the first state data whose proportion in the histogram distribution is greater than a first threshold, and correcting the data in the first state data whose proportion in the histogram distribution is less than a second threshold based on the average value; wherein the first threshold is greater than the second threshold, and the sum of the first threshold and the second threshold is 1.
[0015] Specifically, when the target error type corresponding to the first state data is a global error, the first state data of the target object is corrected based on the target correction method corresponding to the first state data, including: correcting the first state data of the target object based on the size of the target object.
[0016] After determining that there is an error in the first state data, the method further includes: outputting at least one of the following: the object identifier of the target object to which the first state data belongs, and the timestamp of the error data in the first state data.
[0017] To solve the above-mentioned technical problems, another technical solution adopted in this application is: providing a vehicle data correction device, the device comprising: an acquisition module, used to acquire at least one first state data of the target object based on an image of a target object detected by an onboard vision sensor around the vehicle; a determination module, used to determine the target error type corresponding to each of the various first state data; a selection module, used to select a data correction method corresponding to the target error type in a correction correspondence as the target correction method; wherein the correction correspondence includes several preset error types corresponding to several preset error types, and the target error type belongs to several preset error types; and a correction module, used to correct the first state data of the target object based on the target correction method corresponding to the first state data.
[0018] To solve the above-mentioned technical problems, another technical solution adopted in this application is: to provide a processing device, including a memory and a processor coupled to each other, wherein the memory stores program instructions; and the processor is used to execute the program instructions stored in the memory to implement the above-mentioned method for correcting driving data.
[0019] To solve the above-mentioned technical problems, another technical solution adopted in this application is to provide a computer-readable storage medium for storing program instructions that can be executed to implement the above-mentioned method for correcting driving data.
[0020] The above solution first acquires at least one first state data of the target object based on images of target objects detected by the vehicle's onboard vision sensor. Then, for various first state data, when errors exist in the first state data, the target error type corresponding to the first state data is determined, and a data correction method corresponding to the target error type is selected from the correction correspondence. Finally, the first state data is corrected based on the target correction method corresponding to the first state data. That is, when errors exist in the first state data of the target object, the first state data can be corrected based on the data correction method corresponding to the error type of the first state data, improving the accuracy of the first state data of the target object, and thus improving the accuracy of the vehicle's driving data. Attached Figure Description
[0021] Figure 1 This is a flowchart illustrating an embodiment of the method for correcting driving data provided in this application;
[0022] Figure 2 This is a schematic diagram of the uncorrected heading angle distribution of the target object provided in this application during its life cycle;
[0023] Figure 3 This is a schematic diagram of the normal distribution and histogram distribution of the heading angle of the target object provided in this application during its life cycle without correction;
[0024] Figure 4 This is a schematic diagram of the corrected heading angle distribution of the target object provided in this application during its lifecycle;
[0025] Figure 5 This is a schematic diagram of the uncorrected lateral distance distribution of the target object during its lifecycle, as provided in this application.
[0026] Figure 6 This is a schematic diagram of the corrected lateral distance distribution of the target object during its lifecycle, provided in this application;
[0027] Figure 7 This is a schematic diagram showing the vertical distance distribution of the target object within its lifecycle, as provided in this application.
[0028] Figure 8 This is a schematic diagram of the movement trajectory of the target object during its life cycle, as provided in this application;
[0029] Figure 9 This is a schematic diagram of a framework of an embodiment of the vehicle data correction device provided in this application;
[0030] Figure 10 This is a schematic diagram of the framework of an embodiment of the processing device provided in this application;
[0031] Figure 11 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. Detailed Implementation
[0032] To make the purpose, technical solution and effects of this application clearer and more explicit, the following describes this application in further detail with reference to the accompanying drawings and embodiments.
[0033] It should be noted that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship. Furthermore, "many" in this article means two or more. Moreover, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of elements. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C.
[0034] Furthermore, if the embodiments of this application involve descriptions such as "first" or "second," these descriptions are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, features defined with "first" or "second" may explicitly or implicitly include at least one of those features. Additionally, the technical solutions of various embodiments can be combined with each other, but this must be based on the ability of those skilled in the art to implement them. If the combination of technical solutions is contradictory or impossible to implement, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application.
[0035] Please see Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the driving data correction method provided in this application. It should be noted that if substantially the same result is achieved, the method of this invention is not necessarily identical. Figure 1 The illustrated process sequence is limited. For example... Figure 1 As shown, the method includes the following steps:
[0036] S101: Based on the image of the target object around the vehicle detected by the vehicle-mounted vision sensor, acquire at least one first state data of the target object.
[0037] In this embodiment, the vehicle-mounted vision sensor can be a monocular camera, a binocular camera, or a surround-view camera installed on the vehicle. This embodiment does not specifically limit the specific type of vehicle-mounted vision sensor.
[0038] The target object can be a static obstacle or a dynamic obstacle existing around the vehicle. For example, a static obstacle can be a fence, barrier, sign, roadblock, or building. A dynamic obstacle can be a pedestrian, animal, or vehicle. There must be at least one target object around the vehicle.
[0039] When the target object is a static obstacle or a dynamic obstacle other than a vehicle, at least one first state data of the target object includes at least one of the following: the Euclidean distance between the target object and the vehicle, the lateral distance between the target object and the vehicle, the longitudinal distance between the target object and the vehicle, and the target object's trajectory. When the target object is a vehicle, at least one first state data of the target object includes at least one of the following: the Euclidean distance between the target object and the vehicle, the lateral distance between the target object and the vehicle, the longitudinal distance between the target object and the vehicle, the heading angle of the target object, the trajectory of the target object, and the size of the target object. The lateral distance and longitudinal distance between the target object and the vehicle are the lateral and longitudinal distances between the target object and the vehicle in the vehicle's coordinate system. The x-axis of the vehicle coordinate system represents the length direction of the vehicle, the y-axis represents the width direction of the vehicle, and the z-axis represents the height direction of the vehicle. The size of the target object includes its length and width.
[0040] In this embodiment, the vehicle needs to acquire driving data for route planning during driving. The driving data includes at least one of the aforementioned first state data. Specifically, images of target objects around the vehicle are first acquired through an onboard vision sensor, and then the images of the target objects are processed by relevant visual algorithms to obtain at least one first state data of the target objects.
[0041] For example, the Euclidean distance between the target object and the vehicle can be calculated from the image of the target object using a visual distance algorithm. The lateral distance and longitudinal distance between the target object and the vehicle can be calculated based on the Euclidean distance between them and the vehicle coordinate system. For example, when the target object is a vehicle, by performing image detection and recognition on the image of the target object, the heading angle and size of the target object can be obtained. The heading angle of the target object can be calculated based on the detection box type of the target object and the coordinates of the front and rear ground contact points of the target object. The detection box type is used to determine the range of values for the heading angle of the target object. Detection box types include front frame, rear frame, left side frame, right side frame, etc. The motion trajectory of the target object can be obtained based on the position of the target object at different times.
[0042] S102: For various first state data, in response to the existence of errors in the first state data, determine the target error type corresponding to the first state data.
[0043] In this embodiment, it is first determined whether there is an error in the first state data of the target object. When it is determined that there is an error in the first state data of the target object, the target error type corresponding to the first state data with error is then determined.
[0044] In one embodiment, the presence of error in the first state data of the target object can be determined based on whether there are any jumps in the first state data. The first state data includes the first state values of the target object at different times within its lifecycle. The lifecycle refers to the time from the appearance of the target object to its disappearance. For example, the time interval between the first state values at different times is 20m, 30ms, etc., and this embodiment does not impose a specific limitation.
[0045] Specifically, when a first state value in the first state data satisfies a first preset condition, it is determined that the first state data has an error; when no first state value in the first state data satisfies the first preset condition, it is determined that the first state data has no error. The first preset condition includes: the absolute value of the difference between the first state value and its previous first state value is greater than a difference threshold. The difference threshold is specifically set according to the type of first state data. For example, when the first state data is the lateral distance between the target object and the vehicle, the difference threshold can be set to 1m; when the first state data is the longitudinal distance between the target object and the vehicle, the difference threshold can be set to 2m; when the first state is the heading angle of the target object, the difference threshold can be set to 10°.
[0046] In another embodiment, when a matching second state data of the target object detected by an onboard radar sensor exists for the first state data of the target object, the difference between the first and second state data of the target object can be used to determine whether there is an error in the first state data of the target object. Since the onboard radar sensor has relatively accurate distance detection, in this embodiment, the second state data includes at least one of the following: the Euclidean distance between the target object and the vehicle, the lateral distance between the target object and the vehicle, the longitudinal distance between the target object and the vehicle, and the motion trajectory of the target object. The onboard radar sensor can be a lidar, millimeter-wave radar, etc., and this embodiment does not specifically limit it. The second state data includes the second state values of the target object at different times during its lifecycle.
[0047] Specifically, when a first state value in the first state data satisfies a second preset condition, it is determined that the first state data has an error; when no first state value in the first state data satisfies the second preset condition, it is determined that the first state data has no error. The second preset condition includes: the error between the first state value and its second state value at the same time is greater than an error threshold. The error between the first state value and its second state value at the same time is the absolute value of the difference between the first state value and its second state value at the same time, or the error between the first state value and its second state value at the same time is the error rate between the first state value and its second state value at the same time. The error threshold is set according to actual needs, and this embodiment does not specifically limit it. For example, when the error between the first state value and its second state value at the same time is the error rate between the first state value and its second state value at the same time, the error threshold is 10%. The error rate between the first state value and its second state value at the same time can be calculated using the following formula:
[0048]
[0049] In formula (1), c represents the error rate, and d t D represents one of the first state values of the target object at different times during its lifecycle. t Indicates the relationship with d t The second state value at the same time.
[0050] Optionally, in this embodiment, after determining that there is an error in the first state data, at least one of the following is output: the object identifier of the target object to which the first state data belongs, and the timestamp of the error data in the first state data.
[0051] In one example, the object identifier of the target object can be a number or ID (IdentityDocument) corresponding to the target object. The number corresponding to the target object can be numbers, letters, etc., and this embodiment does not impose specific limitations on it.
[0052] In one example, the timestamp of the error data in the first state data can be the time or frame number corresponding to the first state value with the error. When the first state data has an error, feeding back the object identifier of the target object can help relevant personnel quickly identify which target object's first state data has an error. When the first state data has an error, feeding back the timestamp of the error data in the first state data can help relevant personnel quickly identify the location of the specific error data in the first state data.
[0053] Furthermore, the object identifier of the target object to which the first state data belongs and the timestamp of the error data in the first state data can be output to the display interface of the device for display, so that relevant personnel can intuitively obtain the object identifier of the target object to which the first state data belongs and the error data in the first state data.
[0054] In this embodiment, the target error type corresponding to the first state data of the target object includes one of the following: jump error, global error, local error, and feedback error.
[0055] In one embodiment, when there is no matching second state data of the target object detected by an onboard radar sensor for the first state data of the target object, the first state data of the target object has a jump error and a feedback error. The jump error refers to a jump in the first state value at some different times within the first state data. When the first state data has an error and the corresponding target error type is not a jump error, the unknown error type corresponding to the first state data is called a feedback error.
[0056] In this embodiment, determining the target error type corresponding to the first state data includes: obtaining the first proportion of the number of first state values satisfying the first preset condition in the first state data to the total number of first state values in the first state data; and determining the target error type corresponding to the first state data based on the first proportion and the first proportion threshold. The relevant content of the first preset condition is described above and will not be repeated here.
[0057] Specifically, in response to a first proportion being less than a first proportion threshold, the target error type corresponding to the first state data is determined to be a jump error. In response to a first proportion not being less than the first proportion threshold, the target error type corresponding to the first state data is determined to be a feedback error. The first proportion threshold is set according to actual needs, and this embodiment does not specifically limit it. For example, the first proportion threshold is 20%. For instance, the first state data of the target object includes the heading angles of the target object at different times during its life cycle. The first proportion of the number of heading angles that meet the first preset condition in the total number of heading angles in the first state data is 18%. Since the first proportion of 18% is less than the first proportion threshold of 20%, the target error type corresponding to the first state data is a jump error.
[0058] In the above embodiments, when there is no matching second state data of the target object detected by the vehicle-mounted radar sensor for the first state data of the target object, the presence or absence of error in the first state data of the target object can be accurately determined based on the jump situation of the first state data of the target object.
[0059] In another embodiment, when the first state data of the target object has matching second state data detected by an onboard radar sensor, the first state data of the target object contains global error, local error, and feedback error. Global error refers to the existence of error between all first state values in the first state data and the second state values in the second state data at the same time. Local error refers to the existence of error between some first state values in the first state data and the second state values in the second state data at the same time. When the first state data contains error and the corresponding target error type is not global error or local error, the unknown error type corresponding to the first state data is called feedback error.
[0060] In this embodiment, determining the target error type corresponding to the first state data includes: acquiring second state data that matches the first state data of the target object based on the radar information of the target object detected by the vehicle-mounted radar sensor; and determining the target error type corresponding to the first state data based on the second state data that matches the first state data. The first state data includes first state values of the target object at different times within its lifecycle, and the second state data includes second state values of the target object at different times within its lifecycle. Matching the first state data with the second state data means that for each time point in the first state data, there exists a second state value in the second state data that matches the first state value at the same time point. Further, to improve the accuracy of determining the target error type of the first state data of the target object, the number of matches between the first state value in the first state data and the second state value in the second state data is greater than a set threshold. The set threshold is set according to actual needs; for example, the set threshold is 50.
[0061] In this embodiment, determining the target error type corresponding to the first state data based on second state data that matches the first state data includes: obtaining a second proportion of the number of first state values in the first state data that satisfy a second preset condition to the total number of first state values in the first state data; and determining the target error type corresponding to the first state data based on the second proportion and a second proportion threshold. The relevant content of the second preset condition is described above and will not be repeated here.
[0062] Specifically, in response to the second proportion being equal to 1, the target error type corresponding to the first state data is determined to be a global error. In response to the second proportion not being greater than the second proportion threshold, the target error type corresponding to the first state data is determined to be a local error. In response to the second proportion being greater than the second proportion threshold and less than 1, the target error type corresponding to the first state data is determined to be a feedback error. The second proportion threshold is set according to actual needs, and this embodiment does not specifically limit it. For example, the second proportion threshold is 20%. For instance, the first state data of the target object includes the lateral distance between the target object and the vehicle at different times during its life cycle. If the second proportion of the number of lateral distances that meet the second preset condition in the total number of lateral distances in the first state data is 100%, the target error type corresponding to this first state data is a global error.
[0063] In the above embodiments, when there is a matching second state data of the target object detected by the vehicle-mounted radar sensor for the first state data of the target object, since the distance measurement by the vehicle-mounted radar sensor is relatively accurate, the difference between the first state data and the second state data of the target object can accurately determine whether there is an error in the first state data of the target object.
[0064] Optionally, in the above embodiments, when there are multiple target objects, before acquiring the first state data of the target objects and the second state data matching the first state data, the method further includes: performing target matching on the multiple target objects detected by the vehicle-mounted vision sensor and the multiple target objects detected by the vehicle-mounted radar sensor, so that the multiple target objects detected by the vehicle-mounted vision sensor and the multiple target objects detected by the vehicle-mounted radar sensor are matched one by one. For example, the target matching algorithm can be the Hungarian matching algorithm.
[0065] S103: Select the data correction method corresponding to the target error type in the correction correspondence as the target correction method.
[0066] In this embodiment, the correction correspondence includes several preset error types and their corresponding data correction methods, and the target error type belongs to several preset error types. In the correction correspondence, the data correction methods corresponding to different preset error types may be the same or different. The several preset error types include at least one of: jump error, global error, local error, and feedback error. For details on jump error, global error, local error, and feedback error, please refer to the aforementioned step S103, which will not be repeated here. After determining the target error type corresponding to the first state data of the target object, the target correction data correction method corresponding to the target error type can be directly determined from the correction correspondence.
[0067] S104: Based on the target correction method corresponding to the first state data, correct the first state data of the target object.
[0068] In one embodiment, if the target error type corresponding to the first state data is a jump error or a local error, at least one of the normal distribution and histogram distribution corresponding to the first state data is obtained as correction reference data; the first state data is corrected based on the correction reference data.
[0069] In one example, when a normal distribution is used as the correction reference data, the first state data is corrected based on the correction reference data. This includes: obtaining the average value of the data in the first state data that falls within a defined interval of the normal distribution, and correcting the data in the first state data that falls outside the defined interval of the normal distribution based on the average value of the data within the defined interval. The defined interval is determined based on the mean and standard deviation of the normal distribution. For example, the defined interval is the probability interval corresponding to P(μ-σ≤X≤μ+σ), or the defined interval is the probability interval corresponding to P(μ-3σ≤X≤μ+3σ). μ is the mean of the normal distribution, and σ is the standard deviation of the normal distribution.
[0070] In the first state data, data distributed within a set interval correspond to first state values in the first state data that have no error, while data distributed outside the set interval correspond to first state values in the first state data that have error. Specifically, correcting the data distributed outside the set interval in the first state data based on the average value of the data distributed within the set interval of the normal distribution includes: modifying the first state values distributed outside the set interval in the first state data to the average value of the data distributed within the set interval in the first state data.
[0071] In another example, when the histogram distribution is used as correction reference data, the first state data is corrected based on the correction reference data. This includes: obtaining the average value of the data in the first state data whose proportion in the histogram distribution is greater than a first threshold, and correcting the data in the first state data whose proportion in the histogram distribution is less than a second threshold based on the average value of the data whose proportion in the histogram distribution is greater than the first threshold. The first threshold is greater than the second threshold, and the sum of the first threshold and the second threshold is 1. The first threshold and the second threshold are determined according to actual conditions, and this embodiment does not specifically limit them. For example, the first threshold is 0.8, and the second threshold is 0.2.
[0072] In the first-state data, data whose proportion in the histogram distribution is greater than a first threshold corresponds to first-state values in the first-state data that have no error. Data whose proportion in the histogram distribution is less than a second threshold corresponds to first-state values in the first-state data that have error. Specifically, correcting data in the first-state data whose proportion in the histogram distribution is less than the second threshold based on the average of data whose proportion in the histogram distribution is greater than the first threshold includes: modifying the first-state values in the first-state data whose proportion in the histogram distribution is less than the second threshold to the average of the data in the first-state data whose proportion in the histogram distribution is greater than the first threshold.
[0073] In another example, since the data in the first state data that falls within the defined interval of the normal distribution corresponds to the data in the first state data whose proportion in the histogram distribution is greater than a first threshold, and the data in the first state data that falls outside the defined interval of the normal distribution corresponds to the data in the first state data whose proportion in the histogram distribution is less than a second threshold, when the normal distribution and histogram distribution are used as correction reference data, the first state data is corrected based on the correction reference data. This includes: obtaining the average value of the data in the first state data that falls within the defined interval of the normal distribution, and correcting the data in the first state data whose proportion in the histogram distribution is less than the second threshold based on the average value of the data in the defined interval of the normal distribution; or, obtaining the average value of the data in the first state data whose proportion in the histogram distribution is greater than the first threshold, and correcting the data in the first state data that falls outside the defined interval of the normal distribution based on the average value of the data in the histogram distribution whose proportion is greater than the first threshold.
[0074] The following example illustrates how to correct the first-state data based on the normal distribution and histogram distribution corresponding to the first-state data.
[0075] Please see Figure 2 , Figure 2 This is a schematic diagram of the uncorrected heading angle distribution of the target object provided in this application throughout its lifecycle. For example... Figure 2As shown, the heading angle distribution from frame 400 to frame 560 is relatively stable without any jumps. The heading angle from frame 570 to frame 580 jumps between -3.14° and 3.14°.
[0076] Please see Figure 3 , Figure 3 This is a schematic diagram of the normal distribution and histogram distribution of the heading angle of the target object provided in this application during its lifecycle, without correction. For example... Figure 3 As shown, the heading angles in the range of 2.14° to 3.14° account for more than 0.8% of the histogram distribution, while the heading angles in the range of -3.14° to -2.14° account for less than 0.2%. Furthermore, the heading angles in the range of -3.14° to -2.14° fall outside the defined interval of the normal distribution. Therefore, the average value of the heading angles in the range of 2.14° to 3.14° is used to correct for the heading angles in the range of -3.14° to -2.14°.
[0077] Please see Figure 4 , Figure 4 This is a schematic diagram showing the distribution of the corrected heading angles of the target object provided in this application over its lifecycle. For example... Figure 4 As shown, the heading angle distribution from frame 570 to frame 580 is normal after correction, and there are no jumps.
[0078] In the above embodiments, the normal distribution and / or histogram distribution corresponding to the first state data can reflect the data distribution with errors and the data distribution without errors in the first state data. By using the normal distribution and / or histogram distribution corresponding to the first state data as correction reference data to correct the first state data, the data with errors in the first state data can be corrected to the normal range, thereby improving the accuracy and stability of the first state data.
[0079] In another embodiment, when the target error type corresponding to the first state data is a global error, the first state data of the target object is corrected based on the target correction method corresponding to the first state data. This includes correcting the first state data of the target object based on the size of the target object. Specifically, the first state values at different times in the first state data are added to or subtracted from the size correction value of the target object to correct the first state data of the target object. The size correction value is obtained based on the length or width of the target object. For example, when the target object is a vehicle, the size of the target object is the vehicle length and the vehicle width.
[0080] In one example, when the initial state data of the target object includes the lateral distance of the target object at different times during its lifecycle, the size correction value can be calculated based on the following formula:
[0081]
[0082] In formula (2), A represents the size correction value corresponding to the first state data including the lateral distance of the target object at different times. x represents the average error of the lateral distance of the target object at different times, which can be obtained based on the lateral distance of the target object detected by the vehicle vision sensor at different times and the lateral distance of the target object detected by the vehicle radar sensor at different times. W represents the width of the target object. "[]" represents the rounding symbol.
[0083] In another example, when the initial state data of the target object is the longitudinal distance of the target object at different times, the size correction value can be calculated based on the following formula:
[0084]
[0085] In formula (3), B represents the size correction value corresponding to the first state data including the longitudinal distance of the target object at different times. y represents the average error of the longitudinal distance of the target object at different times, which can be obtained based on the longitudinal distance of the target object detected by the vehicle vision sensor at different times and the longitudinal distance of the target object detected by the vehicle radar sensor at different times. L represents the length of the target object. "[]" represents the rounding symbol.
[0086] The following example illustrates how to correct the first-state data of a target object when the target error type corresponding to the first-state data is a global error.
[0087] Please see Figure 5 , Figure 5 This is a schematic diagram of the uncorrected lateral distance distribution of the target object during its lifecycle, as provided in this application. Figure 5 As shown, curve 1 represents the lateral distance between the target object and the vehicle obtained from the onboard vision sensor, while curve 2 represents the lateral distance between the target object and the vehicle obtained from the radar vision sensor. The lateral distance in each frame corresponding to curve 1 differs significantly from the lateral distance in the corresponding frame in curve 2. This indicates a global error in the longitudinal distance between the target object and the vehicle obtained from the onboard vision sensor.
[0088] Please see Figure 6 , Figure 6 This is a schematic diagram illustrating the corrected lateral distance distribution of the target object over its lifecycle, as provided in this application. Figure 6 As shown, curve 3 passes through... Figure 5 Curve 1 in the figure is obtained by correcting it using the aforementioned formula (2), and curve 2 is... Figure 5 Curve 2 in the original text. The corrected curve 3 basically meets the accuracy requirements.
[0089] In the above embodiments, when the target error type corresponding to the first state data is determined to be a global error, the method of correcting the first state data of the target object based on the size of the target object can correct the first state data by consuming only the time cost of running the code once, which can improve the correction efficiency and shorten the development cycle.
[0090] In another embodiment, when the target error type corresponding to the first state data is a feedback error, at least one of the following is output: the object identifier of the target object to which the first state data belongs, and the timestamp of the error data in the first state data, so that relevant personnel can further analyze the error type corresponding to the first state data and correct the first state data based on the feedback information. The object identifier of the target object and the timestamp of the error data in the first state data are described in step S102 above and will not be repeated here.
[0091] When the target error type corresponding to the first-state data is a feedback error, the specific error type and correction method for the first-state data are unclear. However, when the target error type is a feedback error, the object identifier of the feedback target object allows relevant personnel to quickly identify which target object's first-state data has an error; the timestamp of the error data in the feedback first-state data allows relevant personnel to quickly identify the location of the specific error data within the first-state data. This reduces the time spent by relevant personnel in finding the target object to which the erroneous first-state data belongs and in locating the error data within the first-state data.
[0092] Furthermore, when multiple target objects exist around the vehicle, considering the potential for mismatches when matching target objects detected by the vehicle's vision sensors and radar sensors, if the target error type corresponding to the first state data is a feedback error and there is a matching second state data for a target object detected by the vehicle's radar sensor, the object identifier of the target object to which the second state data belongs can also be fed back to facilitate relevant personnel in verifying whether the target match is correct.
[0093] Optionally, in this embodiment, when there is a matching second state data of the target object detected by the vehicle-mounted radar sensor for the first state data, a distribution map of the first state data and the second state data in the same coordinate system can be output so that relevant personnel can intuitively compare and determine the error type and correction method corresponding to the first state data based on the first state data and the second state data in the same coordinate system.
[0094] Please see Figure 7 , Figure 7This is a schematic diagram illustrating the vertical distance distribution of the target object within its lifecycle, as provided in this application. Figure 7 As shown, curve 4 represents the longitudinal distance between the target object and the vehicle, obtained from the onboard vision sensor, while curve 5 represents the longitudinal distance between the target object and the vehicle, obtained from the radar vision sensor. Figure 7 As can be seen, curve 4 is basically consistent with curve 5. Therefore, it can be determined that there is no error in the longitudinal distance between the target object and the vehicle obtained based on the onboard vision sensor.
[0095] Please see Figure 8 , Figure 8 This is a schematic diagram of the movement trajectory of the target object provided in this application during its lifecycle. For example... Figure 8 As shown, curve 6 represents the motion trajectory of the target object obtained based on the vehicle-mounted vision sensor, and curve 7 represents the motion trajectory of the target object obtained based on the vehicle-mounted radar sensor. Figure 8 It can be seen that the distance between curve 6 and curve 7 on the y-axis is quite different, which indicates that there is an error in the lateral distance between the target object and the vehicle obtained based on the vehicle vision sensor.
[0096] Optionally, in this embodiment, after correcting the first state data of the target object based on the target correction method corresponding to the first state data, the method further includes: outputting at least one of the following: the target error type, the cause of the error, and the target correction method corresponding to the first state data. This is to allow relevant personnel to know the target error type, the cause of the error, and the corresponding target correction method corresponding to the first state data. For example, when the first state data is the lateral distance between the target object and the vehicle and the target error type corresponding to the first state data is determined to be a local error, the output information includes: local error, lateral distance error, and correction using histogram distribution. As another example, when the first state data is the lateral distance between the target object and the vehicle and the target error type corresponding to the first state data is a feedback error, the output information includes: feedback error, lateral distance error unknown, and manual correction.
[0097] In this embodiment, firstly, based on images of target objects around the vehicle detected by the vehicle-mounted vision sensor, at least one first state data of the target object is acquired. Then, for various first state data, when errors exist in the first state data, the target error type corresponding to the first state data is determined, and a data correction method corresponding to the target error type is selected from the correction correspondence. Finally, the first state data is corrected based on the target correction method corresponding to the first state data. That is, when errors exist in the first state data of the target object, the first state data can be corrected based on the data correction method corresponding to the error type of the first state data, improving the accuracy of the first state data of the target object, and thus improving the accuracy of the vehicle's driving data.
[0098] Please see Figure 9 , Figure 9 This is a schematic diagram of a framework of an embodiment of the vehicle data correction device provided in this application. In this embodiment, the vehicle data correction device 90 includes: an acquisition module 91, a determination module 92, a selection module 93, and a correction module 94.
[0099] The acquisition module 91 is used to acquire at least one first state data of the target object based on images of target objects detected by the vehicle-mounted vision sensor around the vehicle. The determination module 92 is used to determine the target error type corresponding to each of the various first state data. The selection module 93 is used to select a data correction method corresponding to the target error type in the correction correspondence as the target correction method. The correction correspondence includes several preset error types and their corresponding data correction methods, and the target error type belongs to several preset error types. The correction module 94 is used to correct the first state data of the target object based on the target correction method corresponding to the first state data.
[0100] Optionally, several preset error types include at least one of the following: jump error, global error, local error, and feedback error.
[0101] Optionally, the first state data of the target object includes the first state values of the target object at different times during its life cycle. The determining module 92 is used to obtain the first proportion of the number of first state values in the first state data that meet the first preset condition in the total number of first state values in the first state data. The first preset condition includes: the absolute value of the difference between the first state value and the first state value at the previous time is greater than the difference threshold. Based on the first proportion and the first proportion threshold, the target error type corresponding to the first state data is determined.
[0102] Optionally, the determining module 92 is used to determine the target error type corresponding to the first state data as a jump error in response to the first proportion being less than the first proportion threshold; and to determine the target error type corresponding to the first state data as a feedback error in response to the first proportion being not less than the first proportion threshold.
[0103] Optionally, the acquisition module 91 is further configured to acquire second state data that matches the first state data of the target object based on the radar information of the target object detected by the vehicle radar sensor before determining the target error type corresponding to the first state data; the determination module 92 is configured to determine the target error type corresponding to the first state data based on the second state data that matches the first state data.
[0104] Optionally, the first state data includes the first state values of the target object at different times during its lifecycle, and the second state data includes the second state values of the target object at different times during its lifecycle. The determining module 92 is used to obtain the second proportion of the number of first state values in the first state data that meet the second preset condition to the total number of first state values in the first state data. The second preset condition includes: the error between the first state value and the second state value at the same time is greater than the error threshold. Based on the second proportion and the second proportion threshold, the target error type corresponding to the first state data is determined.
[0105] Optionally, the determining module 92 is configured to determine the target error type corresponding to the first state data as a global error in response to the second proportion being equal to 1; to determine the target error type corresponding to the first state data as a local error in response to the second proportion not being greater than the second proportion threshold; and to determine the target error type corresponding to the first state data as a feedback error in response to the second proportion being greater than the second proportion threshold and less than 1.
[0106] Optionally, if the target error type corresponding to the first state data is a jump error or a local error, the correction module 94 is used to obtain at least one of the normal distribution and histogram distribution corresponding to the first state data as correction reference data; and to correct the first state data based on the correction reference data.
[0107] Optionally, when a normal distribution is obtained as the correction reference data, the correction module 94 is used to obtain the average value of the data in the first state data that are distributed within the normal distribution set interval, and correct the data in the first state data that are distributed outside the normal distribution set interval based on the average value, wherein the set interval is determined based on the mean and standard deviation of the normal distribution; and / or, when a histogram distribution is obtained as the correction reference data, the correction module 94 is used to obtain the average value of the data in the first state data whose proportion in the histogram distribution is greater than a first threshold, and correct the data in the first state data whose proportion in the histogram distribution is less than a second threshold based on the average value; wherein the first threshold is greater than the second threshold, and the sum of the first threshold and the second threshold is 1.
[0108] Optionally, if the target error type corresponding to the first state data is a global error, the correction module 94 is used to correct the first state data of the target object based on the size of the target object.
[0109] Optionally, the vehicle data correction device further includes an output module 95. After the determining module 92 determines that there is an error in the first state data, the output module 95 is used to output at least one of the following: the object identifier of the target object to which the first state data belongs, and the timestamp of the error data in the first state data.
[0110] It should be noted that the apparatus of this embodiment can perform the steps in the above method. For detailed descriptions of the relevant content, please refer to the method section above, which will not be repeated here.
[0111] Please see Figure 10 , Figure 10 This is a schematic diagram of a framework of an embodiment of the processing device provided in this application. In this embodiment, the processing device 100 includes a memory 101 and a processor 102.
[0112] Processor 102 can also be referred to as CPU (Central Processing Unit). Processor 102 may be an integrated circuit chip with signal processing capabilities. Processor 102 can also be a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component. The general-purpose processor can be a microprocessor, or processor 102 can be any conventional processor 102, etc.
[0113] The memory 101 in the processing device 100 is used to store the program instructions required for the processor 102 to run.
[0114] The processor 102 is used to execute program instructions to implement the method for correcting driving data in this application.
[0115] Please see Figure 11 , Figure 11 This is a schematic diagram of a framework of an embodiment of the computer-readable storage medium provided in this application. The computer-readable storage medium 110 of this application embodiment stores program instructions 111, which, when executed, implement the vehicle data correction method provided in this application. The program instructions 111 can be formed into a program file and stored in the aforementioned computer-readable storage medium 110 in the form of a software product, so that a computer device (which may be a personal computer, server, or network device, etc.) can execute all or part of the steps of the methods of various embodiments of this application. The aforementioned computer-readable storage medium 110 includes various media capable of storing program code, such as a USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, or terminal devices such as computers, servers, mobile phones, and tablets.
[0116] The above solution first acquires at least one first state data of the target object based on images of target objects detected by the vehicle's onboard vision sensor. Then, for various first state data, when errors exist in the first state data, the target error type corresponding to the first state data is determined, and a data correction method corresponding to the target error type is selected from the correction correspondence. Finally, the first state data is corrected based on the target correction method corresponding to the first state data. That is, when errors exist in the first state data of the target object, the first state data can be corrected based on the data correction method corresponding to the error type of the first state data, improving the accuracy of the first state data of the target object, and thus improving the accuracy of the vehicle's driving data.
[0117] In some embodiments, the functions or modules of the apparatus provided in this disclosure can be used to perform the methods described in the above method embodiments. The specific implementation can be referred to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.
[0118] The description of the various embodiments above tends to emphasize the differences between the various embodiments. The similarities or similarities between them can be referred to, and for the sake of brevity, they will not be repeated here.
[0119] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus implementations described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or units may be electrical, mechanical, or other forms.
[0120] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.
[0121] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0122] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods of various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.
[0123] The above description is merely an embodiment of this application and does not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.
Claims
1. A method for correcting driving data, characterized in that, The method includes: Based on images of target objects around the vehicle detected by an onboard vision sensor, at least one first state data of the target object is obtained, and based on radar information of the target object detected by an onboard radar sensor, second state data matching the first state data of the target object is obtained. The first state data includes at least one of the Euclidean distance, lateral distance, and longitudinal distance between the target object and the vehicle, as well as the target object's motion trajectory, heading angle, and size. The second state data includes at least one of the Euclidean distance, lateral distance, and longitudinal distance between the target object and the vehicle, as well as the target object's motion trajectory. For various types of the first state data, in response to the existence of errors in the first state data, the target error type corresponding to the first state data is determined; Select the data correction method corresponding to the target error type in the correction correspondence as the target correction method; wherein, the correction correspondence includes data correction methods corresponding to several preset error types respectively, the target error type belongs to the several preset error types, the several preset error types include at least one of jump error, global error, local error and feedback error, and, the first state data also includes the first state value of the target object at different times in the life cycle, the second state data also includes the second state value of the target object at different times in the life cycle, the jump error indicates that there is a jump in some first state values at different times in the first state data, the global error indicates that there is an error between all first state values in the first state data and the second state values at the same time in the second state data, the local error indicates that there is an error between some first state values in the first state data and the second state values at the same time in the second state data, and the feedback error indicates that there is an error in the first state data and the corresponding target error type is not the jump error, the global error and the local error; Based on the target correction method corresponding to the first state data, the first state data of the target object is corrected.
2. The method according to claim 1, characterized in that, Determining the target error type corresponding to the first state data includes: Obtain the first percentage of the number of first state values in the first state data that satisfy the first preset condition in the total number of first state values in the first state data; wherein, the first preset condition includes: the absolute value of the difference between the first state value and the first state value at the previous moment is greater than the difference threshold. Based on the first proportion and the first proportion threshold, the target error type corresponding to the first state data is determined.
3. The method according to claim 2, characterized in that, The determination of the target error type corresponding to the first state data based on the first proportion and the first proportion threshold includes at least one of the following: In response to the first proportion being less than the first proportion threshold, the target error type corresponding to the first state data is determined to be the jump error; In response to the first proportion being not less than the first proportion threshold, the target error type corresponding to the first state data is determined to be the feedback error.
4. The method according to claim 1, characterized in that, Determining the target error type corresponding to the first state data includes: Based on the second state data that matches the first state data, the target error type corresponding to the first state data is determined.
5. The method according to claim 4, characterized in that, The step of determining the target error type corresponding to the first state data based on the second state data that matches the first state data includes: Obtain the second percentage of the number of first state values in the first state data that satisfy the second preset condition in the total number of first state values in the first state data; wherein, the second preset condition includes: the error between the first state value and the second state value at the same time is greater than the error threshold; Based on the second proportion and the second proportion threshold, the target error type corresponding to the first state data is determined.
6. The method according to claim 5, characterized in that, The determination of the target error type corresponding to the first state data based on the second proportion and the second proportion threshold includes at least one of the following: In response to the second proportion being equal to 1, the target error type corresponding to the first state data is determined to be a global error; In response to the second proportion not being greater than the second proportion threshold, the target error type corresponding to the first state data is determined to be the local error; In response to the second proportion being greater than the second proportion threshold and less than 1, the target error type corresponding to the first state data is determined to be the feedback error.
7. The method according to claim 6, characterized in that, When the target error type corresponding to the first state data is a jump error or a local error, the step of correcting the first state data of the target object based on the target correction method corresponding to the first state data includes: Obtain at least one of the normal distribution and histogram distribution corresponding to the first state data as correction reference data; The first state data is corrected based on the corrected reference data.
8. The method according to claim 7, characterized in that, When the normal distribution is obtained as the correction reference data, the step of correcting the first state data based on the correction reference data includes: The average value of the data in the first state data distributed within the set interval of the normal distribution is obtained, and the data in the first state data distributed outside the set interval of the normal distribution is corrected based on the average value, wherein the set interval is determined based on the mean and standard deviation of the normal distribution; And / or, when the histogram distribution is obtained as the correction reference data, the step of correcting the first state data based on the correction reference data includes: The average value of the data in the first state data whose proportion in the histogram distribution is greater than a first threshold is obtained, and the data in the first state data whose proportion in the histogram distribution is less than a second threshold is corrected based on the average value; wherein, the first threshold is greater than the second threshold, and the sum of the first threshold and the second threshold is 1.
9. The method according to claim 1, characterized in that, When the target error type corresponding to the first state data is the global error, the step of correcting the first state data of the target object based on the target correction method corresponding to the first state data includes: Based on the size of the target object, the first state data of the target object is corrected.
10. The method according to claim 1, characterized in that, After determining that the first state data contains an error, the method further includes: Output at least one of the following: the object identifier of the target object to which the first state data belongs, and the timestamp of the error data in the first state data.
11. A device for correcting driving data, characterized in that, The device includes: The acquisition module is used to acquire at least one first state data of the target object based on images of target objects around the vehicle detected by the vehicle-mounted vision sensor, and is also used to acquire second state data matching the first state data of the target object based on radar information of the target object detected by the vehicle-mounted radar sensor. The first state data includes at least one of the Euclidean distance, lateral distance, and longitudinal distance between the target object and the vehicle, as well as the target object's motion trajectory, heading angle, and size. The second state data includes at least one of the Euclidean distance, lateral distance, and longitudinal distance between the target object and the vehicle, as well as the target object's motion trajectory. The determination module is used to determine the target error type corresponding to each of the various first state data; The selection module is used to select a data correction method corresponding to the target error type in the correction correspondence as the target correction method; wherein, the correction correspondence includes data correction methods corresponding to several preset error types respectively, the target error type belongs to the several preset error types, the several preset error types include at least one of jump error, global error, local error and feedback error, and, the first state data also includes the first state value of the target object at different times in its life cycle, the second state data also includes the second state value of the target object at different times in its life cycle, the jump error indicates that there is a jump in some of the first state values at different times in the first state data, the global error indicates that there is an error between all the first state values in the first state data and the second state values at the same time in the second state data, the local error indicates that there is an error between some of the first state values in the first state data and the second state values at the same time in the second state data, and the feedback error indicates that there is an error in the first state data and the corresponding target error type is not the jump error, the global error and the local error; The correction module is used to correct the first state data of the target object based on the target correction method corresponding to the first state data.
12. A processing apparatus, characterized in that, Including interconnected memory and processor, The memory stores program instructions; The processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1-10.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store program instructions that can be executed to implement the method of any one of claims 1-10.