Data processing methods, devices and vehicles

Abstract

This invention discloses a data processing method, apparatus, and vehicle. The method includes: acquiring multiple frames of images to be monitored during vehicle operation; identifying the images to be monitored to obtain identification results, and determining the confidence level of a monitored object in the identification results, wherein the confidence level represents the degree of matching of the monitored object; and verifying the images to be monitored based on the confidence level to obtain verification results, wherein the verification results characterize the accuracy of the identification of the images to be monitored. This invention solves the technical problem of poor accuracy in evaluating the positioning effect of monitored objects.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically, to a data processing method, apparatus, and vehicle. Background Technology

[0002] In related technologies, the method for evaluating the accuracy of the identification results of the monitored object is usually based on the ratio of the area of ​​the intersection of the perceived real area and the predicted area of ​​the monitored object. However, the above method is not intuitive enough and has the technical problem of poor accuracy in evaluating the positioning effect of the monitored object. Summary of the Invention

[0003] This invention provides a data processing method, apparatus, and vehicle to at least address the technical problem of poor accuracy in assessing the effectiveness of obstacle identification.

[0004] According to one aspect of the present invention, a data processing method is provided, comprising: acquiring multiple frames of images to be monitored during the driving process of a vehicle; identifying the images to be monitored to obtain an identification result, and determining the confidence level of a monitored object in the identification result, wherein the confidence level is used to represent the degree of matching of the monitored object; and verifying the images to be monitored based on the confidence level to obtain a verification result, wherein the verification result is used to characterize the accuracy of the identification of the images to be monitored.

[0005] Optionally, based on confidence level, the monitoring images are verified to obtain verification results, including: classifying the correctly identified monitoring objects in multiple frames to obtain at least one monitoring object group; classifying the at least one monitoring object group based on confidence level to obtain at least one sub-monitoring object group; determining the sub-verification results of the at least one sub-monitoring object group to obtain at least one sub-verification result, wherein the sub-verification results are used to characterize the accuracy of identifying the same type of monitoring objects in multiple monitoring images; and averaging the sub-verification results to determine the verification result.

[0006] Optionally, classifying at least one monitoring object group based on confidence level to obtain at least one sub-monitoring object group includes: dividing at least one confidence level of at least one monitoring object group into at least one confidence level group; smoothing each of the at least one confidence level group to obtain at least one equally divided confidence level threshold group, wherein the confidence level threshold group includes the confidence level in the corresponding confidence level group; and classifying the corresponding monitoring object group through the at least one confidence level threshold group to obtain at least one sub-monitoring object group, wherein the number of confidence level thresholds corresponds one-to-one with the number of sub-monitoring object groups.

[0007] Optionally, the sub-verification result for each sub-monitoring object group is determined, including: determining the intersection-union ratio (IUU) between the detection bounding boxes in the identification results of the sub-monitoring object group and the actual detection bounding boxes of the monitored objects; determining the number of instances in the sub-monitoring object group where the identity information of the same monitored object differs from the identity information of adjacent frames; and determining the sub-verification result based on the number of instances of identity information that differs from adjacent frames, the IUU, and the number of correct verifications.

[0008] Optionally, the minimum value of the sub-verification results in at least one monitoring object group is determined to obtain at least one minimum value. The minimum value of the at least one sub-verification result is determined as the target verification result of at least one monitoring object group, wherein the number of target verification results corresponds one-to-one with the number of monitoring object groups. The verification result is obtained by averaging all target verification results.

[0009] Optionally, based on the identity information in the identification results and the real identity information of the monitored object, the number of times the monitored object was correctly identified is determined; based on the number of correct identifications, the verification result is determined.

[0010] Optionally, based on the identity information and real identity information in the identification results, the number of correct identifications of the monitored object is determined, including: in response to a successful match between the identity information and the real identity information, determining that the identification result of the monitored object is correct, and incrementing the number of correct identifications by one.

[0011] Optionally, in response to a failure to match the identity information with the real identity information, the location information in the monitoring result is matched with the real location information of the monitored object; in response to a successful match between the location information and the real location information, the identification result of the monitored object is determined to be correct, and the number of correct matches is incremented by one.

[0012] According to another aspect of the present invention, a data processing apparatus is also provided, comprising: an acquisition unit for acquiring multiple frames of images to be monitored during the driving process of a vehicle; an identification unit for identifying the images to be monitored, obtaining identification results, and determining the confidence level of a monitored object in the identification results, wherein the confidence level is used to represent the degree of matching of the monitored object; and a verification unit for verifying the images to be monitored based on the confidence level, obtaining verification results, wherein the verification results are used to characterize the accuracy of the identification of the images to be monitored.

[0013] According to another aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium includes a stored program, wherein, when the program is executed, it controls the device where the computer-readable storage medium is located to perform the data processing method of the present invention.

[0014] According to another aspect of the present invention, a processor is also provided. The processor is used to run a program, wherein the program executes the data processing method of the embodiments of the present invention during runtime.

[0015] According to another aspect of the present invention, a vehicle is also provided. This vehicle is used to execute the data processing method of the embodiments of the present invention.

[0016] In this embodiment of the invention, multiple frames of images to be monitored are acquired during the vehicle's driving process; the images to be monitored are identified to obtain identification results, and the confidence level of the monitored object in the identification results is determined, whereby the confidence level is used to represent the matching degree of the monitored object; based on the confidence level, the images to be monitored are verified to obtain verification results, whereby the verification results are used to characterize the accuracy of the identification of the monitored images. In other words, this embodiment of the invention, by identifying multiple frames of images to be monitored during the driving process, obtaining identification results, determining the confidence level of the identification results, and then verifying the images to be monitored based on the confidence level, can obtain verification results. By indexing the verification results of the monitored object, the technical problem of poor accuracy in evaluating the positioning effect of the monitored object is solved, and the technical effect of improving the accuracy of the positioning effect evaluation of the monitored object is achieved. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention;

[0019] Figure 2 This is a schematic diagram of obstacle classification results according to an embodiment of the present invention;

[0020] Figure 3 This is a flowchart of an obstacle size error assessment method according to an embodiment of the present invention;

[0021] Figure 4 This is a schematic diagram of a data processing apparatus according to an embodiment of the present invention. Detailed Implementation

[0022] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0023] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0024] Example 1

[0025] According to an embodiment of the present invention, an embodiment of a data processing method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0026] Figure 1 This is a flowchart of a data processing method according to an embodiment of the present invention, such as... Figure 1 As shown, the method may include the following steps:

[0027] Step S102: Acquire multiple frames of images to be monitored during the vehicle's driving process.

[0028] In the technical solution provided by step S102 of the present invention, multiple frames of images to be monitored during the vehicle's driving process are acquired. The images to be monitored may include images of the vehicle's surroundings during the vehicle's driving process, such as images captured by a camera, point cloud data acquired by a lidar, etc. No specific restrictions are placed on the form and method of the acquired images to be monitored. Images that can be used to characterize obstacles to be monitored around the vehicle or images containing information such as vectors in the three-dimensional coordinate system of the obstacles can be the images to be monitored in the embodiments of the present invention. The obstacles in the images to be monitored can be moving vehicles, trees, houses, etc., and no specific restrictions are placed here.

[0029] For example, millimeter-wave radar can be used to simultaneously identify multiple monitoring objects around a vehicle and collect images of the vehicle during its movement to obtain the image to be monitored. Alternatively, lidar can be used to identify cloud points of monitoring objects around the vehicle to obtain the three-dimensional coordinates of the monitoring objects, thereby obtaining the image to be monitored.

[0030] For another example, millimeter-wave radar identifies obstacles around the vehicle, a camera captures images of the obstacles around the vehicle, and lidar determines the three-dimensional coordinates of the obstacles around the vehicle. Through the combined action of these three methods, multiple frames of images to be monitored in this embodiment of the invention are obtained.

[0031] It should be noted that the millimeter-wave radar and lidar used here and their functions are only illustrative examples, and no specific restrictions are placed on the information contained in the acquired images to be monitored or the required sensors.

[0032] Step S104: Identify the image to be monitored, obtain the identification result, and determine the confidence level of the monitored object in the identification result.

[0033] In the technical solution provided in step S104 of the present invention, by recognizing the image to be monitored, the recognition result of the monitored object can be obtained, and the confidence level of the monitored object in the image to be monitored can be determined from the recognition result. The monitored object can be an obstacle in the image to be monitored. The confidence level can be used to characterize the degree of matching between the obstacle in the image to be monitored and the actual obstacle. For example, it can be used to characterize the degree of matching between the actual detection box of the obstacle and the detected detection box. It can be in the form of numbers, Chinese characters, etc. For example, it can be represented by numbers between 0 and 1. The larger the number, the higher the degree of matching, etc. This is only an example and no specific limitation is made on the form of confidence. The recognition result can include information such as the category, size, distance, speed, detection box and confidence level of the obstacle in the image to be monitored. The content of the recognition result is only an example and no specific limitation is made here.

[0034] Step S106: Based on the confidence level, verify the image to be monitored to obtain the verification result.

[0035] In the technical solution provided in step S106 of the present invention, the image to be monitored is identified to obtain a confidence level. Based on the confidence level, the image to be monitored is verified to obtain a verification result. The verification result can be used to characterize the accuracy of the image to be monitored, which can be represented by the average multiple object tracking scale error (amotse). The lower the index, the higher the accuracy of the image to be monitored.

[0036] For example, the recognition results of all frames can be obtained. Based on the confidence level and the type of monitored object in the recognition results, the results are classified. This determines the number of correct associations, the number of incorrect associations, and the cross-union ratio (CUI) between the detected and actual bounding boxes for each type of obstacle at each confidence level. Based on this, the multi-object tracking scale error of the obstacles is determined. Then, the average multi-object tracking scale error is calculated to obtain the average multi-object tracking scale error, thus yielding the verification result. It should be noted that the method for determining the multi-object tracking error and the required data are only illustrative examples. All methods for determining the multi-object tracking scale error should be within the protection scope of this invention, and no specific limitations are imposed here.

[0037] In this embodiment of the invention, multiple frames of images to be monitored are acquired during the vehicle's driving process; the images to be monitored are identified to obtain identification results, and the confidence level of the monitored object in the identification results is determined, whereby the confidence level is used to represent the matching degree of the monitored object; based on the confidence level, the images to be monitored are verified to obtain verification results, whereby the verification results are used to characterize the accuracy of the identification of the monitored images. In other words, this embodiment of the invention, by identifying multiple frames of images to be monitored during the driving process, obtaining identification results, determining the confidence level of the identification results, and then verifying the images to be monitored based on the confidence level, can obtain verification results. By indexing the verification results of the monitored object, the technical problem of poor accuracy in evaluating the positioning effect of the monitored object is solved, and the technical effect of improving the accuracy of the positioning effect evaluation of the monitored object is achieved.

[0038] The method described in this embodiment will be further described below.

[0039] As an optional embodiment, step S106, based on confidence level, verifies the image to be monitored to obtain a verification result, including: classifying the correctly identified multi-frame monitoring objects to obtain at least one monitoring object group; classifying the at least one monitoring object group based on confidence level to obtain at least one sub-monitoring object group; determining the sub-verification result of the at least one sub-monitoring object group to obtain at least one sub-verification result, wherein the sub-verification result is used to characterize the accuracy of identifying the same type of monitoring objects in the multi-frame monitoring images; and averaging the sub-verification results to determine the verification result.

[0040] In this embodiment, when the real identity information and the identity information are successfully matched or the location information and the real location information are successfully matched, it can be determined that the identification result of the monitored object is correct. The monitored objects in the correctly identified multiple frames can be classified to obtain at least one monitoring object group. The above monitoring object group can be further classified based on different confidence levels to obtain at least one sub-monitoring object group. The sub-verification result of the sub-monitoring object group is determined to obtain at least one sub-verification result. Finally, the sub-verification results are averaged to determine the verification result. The sub-verification result corresponds one-to-one with the sub-monitoring object group and can be used to characterize the accuracy of the identification of the same type of monitored object in multiple frames of images to be monitored. It can be used as the multiple object tracking scale error (MOTE).

[0041] For example, when identifying multiple frames of images to be monitored, if the identification result is that the monitored objects in the images are pedestrians and vehicles, the monitored objects can be divided into monitoring object groups containing pedestrians and monitoring object groups containing vehicles based on the category of the monitored objects. Based on the confidence level corresponding to each monitoring object group, the monitored objects in each monitoring object group can be further classified to obtain sub-monitoring object groups under different confidence levels, and the multi-target tracking scale error of each sub-monitoring object group can be determined.

[0042] As an optional embodiment, step S106, classifying at least one monitoring object group based on confidence level to obtain at least one sub-monitoring object group, includes: dividing at least one confidence level of at least one monitoring object group into at least one confidence level group; smoothing each of the at least one confidence level group to obtain at least one equally divided confidence level threshold group, wherein the confidence level threshold group includes the confidence level in the corresponding confidence level group; classifying the corresponding monitoring object group through the at least one confidence level threshold group to obtain at least one sub-monitoring object group, wherein the number of confidence level thresholds corresponds one-to-one with the number of sub-monitoring object groups.

[0043] In this embodiment, the confidence level of each monitoring object in the monitoring object group is obtained to obtain a confidence level group. The confidence level group is then smoothed to obtain an evenly divided confidence threshold group. The corresponding monitoring object group can be classified by the confidence threshold group to obtain a sub-monitoring object group. The confidence threshold can be a range of values ​​including a maximum value and a minimum value. The confidence threshold group can include at least one confidence threshold, and the number of confidence thresholds corresponds one-to-one with the number of sub-monitoring object groups.

[0044] For example, if five confidence levels are obtained from a group of monitored objects, such as 0.2, 0.3, 0.4, 0.6, and 0.7, a confidence level group containing these five levels can be obtained. This confidence level group can then be smoothed, for example, by inserting a confidence level of 0.5. The interpolated confidence level group can then be divided into three confidence thresholds: [0.2, 0.3], [0.4, 0.5], and [0.6, 0.7]. The confidence threshold group contains the three confidence thresholds mentioned above, and each confidence threshold includes the confidence level within that confidence threshold range. The corresponding monitored object group can be further classified using the confidence threshold group to obtain sub-monitored object groups. One confidence threshold corresponds to one sub-monitored object group.

[0045] As an optional embodiment, step S106, determining the sub-verification result for each sub-monitoring object group, includes: determining the intersection-union ratio (IUU) between the detection bounding boxes in the identification results of the sub-monitoring object group and the actual detection bounding boxes of the monitored objects; determining the number of instances in the sub-monitoring object group where the identity information of the same monitored object differs from the identity information of adjacent frames; and determining the sub-verification result based on the number of instances where the identity information differs from that of adjacent frames, the IUU, and the number of correct verifications.

[0046] In this embodiment, the intersection-union ratio (IUU) of the detection boxes in the identification results of each sub-monitoring object group with the actual detection boxes of the monitored objects can be determined. The number of times the identity information of the same monitored object in the sub-monitoring object group differs from the identity information of adjacent frames can be determined. Finally, the sub-verification result can be determined by the IUU, the number of correct results, and the number of times the identity information differs from that of adjacent frames.

[0047] For example, a polygon clipping algorithm can be used to perform a polygon clipping algorithm on the detection boxes in the identification results of the sub-monitoring object group and the actual detection boxes of the monitored objects to determine the intersection and union regions of the two detection boxes and calculate the intersection-union ratio. It should be noted that the calculation method of the intersection-union ratio is only for illustration. The method of determining the intersection-union ratio of the two detection boxes in the three-dimensional space based on various data of the obstacles should be within the protection scope of the embodiments of the present invention, and no specific restrictions are made here.

[0048] For another example, obstacles can be classified (filtered) based on different confidence levels, thereby obtaining the recall rate corresponding to different confidence levels. Therefore, by acquiring the image to be monitored, the recognition result of a certain obstacle can be determined, and it can be matched with the actual information to determine the recall rate. Then, by using the mapping relationship between confidence and recall rate, the confidence level of the obstacle can be determined. It should be noted that the method and data for determining confidence are only illustrative examples. The method of determining confidence based on various information of the obstacle should be within the protection scope of the embodiments of this invention, and no specific limitations are made here.

[0049] As an optional example, it can be determined whether the identity information of the monitored objects in adjacent frames of the monitored image matches. If the match fails, it can be determined that the identity information of the adjacent frames is different, and the number of differences is incremented by one to obtain the number of differences in the identity information of the adjacent frames. If the match succeeds, it can be determined that the identity information of the adjacent frames is the same.

[0050] Optionally, the intersection-union ratio (CUI) between the detected bounding boxes and the actual detected bounding boxes in the recognition results can be determined through the above examples. Based on the determined CUI, the number of correct detections, and the number of differences in identity information between adjacent frames, the multi-object tracking scale error can be calculated using the following formula:

[0051]

[0052] Among them, motse r It can be used to represent the multi-object tracking size error when the recall is r, TP t IDS is the number of correctly associated frames at the t-th frame when the recall rate is r. t Let r be the number of erroneous associations in frame t. Let be the stereo crossover ratio between the true value i of a successful match in frame t and the hypothesis.

[0053] As an optional embodiment, in step S106, the minimum value of the sub-verification results in at least one monitoring object group is determined to obtain at least one minimum value, and the minimum value of the at least one sub-verification result is determined as the target verification result of at least one monitoring object group, wherein the number of target verification results corresponds one-to-one with the number of monitoring object groups; the average of all target verification results is calculated to obtain the verification result.

[0054] In this embodiment, the minimum value of the sub-verification result in each monitoring object group can be determined, and at least one minimum value among all minimum values ​​can be obtained. The minimum value of the sub-verification result is determined as the target verification result of the monitoring object group. The verification result can be determined by averaging all target verification results. Each monitoring object group will correspond to a number of target verification results. That is, the number of target verification results corresponds one-to-one with the number of monitoring object groups.

[0055] Alternatively, the average multi-target tracking size error can be calculated using the following formula:

[0056]

[0057] Where amotse is the average multi-target tracking size error, which is the average effective multi-target tracking size error under different thresholds for the same type of obstacle.

[0058] As an optional embodiment, in step S104, based on the identity information in the identification result and the real identity information of the monitored object, the correct number of times the monitored object was identified is determined; based on the correct number of times, the verification result is determined.

[0059] In this embodiment, the image to be monitored is identified to obtain the identification result. Based on the identity information in the identification result and the real identity information of the monitored object, it can be determined whether the monitored object is correctly identified. Furthermore, the number of times the monitored object is correctly identified can be determined. Based on the number of correct identifications, the verification result can be determined. The identity information can be used to characterize the identity of the monitored object and can be a tracking identifier (Identity Document, or ID for short). The real identity information can be the real identity information corresponding to each monitored object that is predetermined.

[0060] Optionally, it can be determined whether the identity information in the recognition result is consistent with the real identity information of the monitored object. If they are consistent, it can be determined that the monitored object is correctly identified, and the number of times the image to be monitored is correctly identified can be further determined. If they are inconsistent, it can be determined that the monitored object is incorrectly identified.

[0061] Optionally, during vehicle operation, multiple frames of images to be monitored are acquired for identification. The tracking ID in the identification result of a certain object can be matched with the real identity information of the monitored object. If the match is successful, it means that the identity information of the monitored object is correct. Based on the above method, the number of times the monitored object is correctly identified can be determined, and the verification result can be determined based on the number of correct identifications.

[0062] For example, a tracking ID of 1 can indicate that the monitored object is a vehicle, and a tracking ID of 0 can indicate that the monitored object is a pedestrian. The identity information of monitored objects at the same location can be confirmed. If the tracking ID in the identification result is 1, but the actual tracking ID is 0, it can be determined that the monitored object has been identified incorrectly.

[0063] As an optional embodiment, step S104, based on the identity information and real identity information in the identification result, determines the correct number of times the monitored object is identified, including: in response to the successful matching of identity information and real identity information, determining that the identification result of the monitored object is correct, and incrementing the correct number by one.

[0064] In this embodiment, the image to be monitored is identified to obtain the identification result. It is then determined whether the identity information in the identification result can be successfully matched with the identity information of the monitored object. If the match is successful, the identification result of the monitored object can be determined to be correct, and the correct count is incremented by one.

[0065] For example, before identifying the monitored object, the number of correct identification results can be 0. When the monitored object is identified, the identity information in the identification result is determined to be 1, and the real identity information of the monitored object is also 1. The two identity information are successfully matched, so the identification result can be determined to be correct, and the number of correct results is increased by 1 from 0, so the number of correct results is 1.

[0066] As an optional embodiment, in step S104, in response to the failure to match the identity information and the real identity information, the location information in the monitoring result is matched with the real location information of the monitored object; in response to the successful matching of the location information and the real location information, the identification result of the monitored object is determined to be correct, and the correct count is incremented by one.

[0067] In this embodiment, the image to be monitored is identified to obtain the identification result. Based on the identity information in the identification result, it can be determined whether the identity information in the identification result can be matched with the identity information of the monitored object. If the match fails, the location information in the monitoring result can be further matched with the real location information of the monitored object to determine whether the match is successful. If the match is successful, it can be determined that the identification result of the monitored object is correct, and the correct count is incremented by one. The location information can be used to characterize the location of the monitored object and can be a distance threshold. The real location information can be the real location information corresponding to each monitored object that is predetermined.

[0068] Optionally, based on matching the identity information in the identification results with the identity information of the monitored object, errors may occur during the implementation of this step, causing the identity information that should have been matched to fail to match. Therefore, in order to avoid such errors, for the monitored object whose identity information failed to match, the location information in the identification results can be further matched with the real information of the monitored object. If the match is successful, the identification result at this time can be determined to be correct, and the correct count is incremented by one.

[0069] In this embodiment of the invention, multiple frames of images to be monitored are acquired during the vehicle's driving process; the images to be monitored are identified to obtain identification results, and the confidence level of the monitored object in the identification results is determined, whereby the confidence level is used to represent the matching degree of the monitored object; based on the confidence level, the images to be monitored are verified to obtain verification results, whereby the verification results are used to characterize the accuracy of the identification of the monitored images. In other words, this embodiment of the invention, by identifying multiple frames of images to be monitored during the driving process, obtaining identification results, determining the confidence level of the identification results, and then verifying the images to be monitored based on the confidence level, can obtain verification results. By indexing the verification results of the monitored object, the technical problem of poor accuracy in evaluating the positioning effect of the monitored object is solved, and the technical effect of improving the accuracy of the positioning effect evaluation of the monitored object is achieved.

[0070] Example 2

[0071] The technical solutions of the embodiments of the present invention will be illustrated below with reference to preferred embodiments.

[0072] Currently, the post-fusion scale accuracy assessment of multi-sensor target data is based on the fusion of given obstacle information through strategies, and the original raw data cannot be obtained. Therefore, there is no particularly good method for assessing obstacle scale.

[0073] In the evaluation of obstacle monitoring results in related technologies, the evaluation results are not quantified. Therefore, it is only possible to evaluate the obstacle monitoring results, but the accuracy of the evaluation cannot be guaranteed. Thus, the technical problem of poor accuracy in evaluating obstacle positioning effectiveness still exists.

[0074] In one related technology, a method for evaluating obstacle detection results of an autonomous vehicle is disclosed. This method involves acquiring a perceived image of an obstacle and a real image of the obstacle from the autonomous vehicle; calculating the minimum bounding rectangle of the perceived and real images to obtain the perceived and real regions of the obstacle; calculating the overlap rate between the perceived and real regions based on the areas of their intersection and union; calculating the distance between the center points of the perceived and real regions based on their center point positions; calculating the graphic similarity between the perceived and real regions based on their areas and aspect ratios; and calculating the matching degree between the perceived and real regions based on the overlap rate, center point distance, and graphic similarity, thus achieving a more accurate evaluation of obstacle detection results.

[0075] Another related technology discloses a method for monitoring spatial obstacles. This method monitors the presence of spatial obstacles and the distance to those obstacles in one or more directions. For each of the one or more directions, the following operations are performed: comparing the monitoring results at a first moment and a second moment after the first moment; and determining that the monitoring result in the direction is valid if the distance to the obstacle at the second moment is less than the distance at the first moment. This invention allows for the judgment of the validity of monitoring results, filtering out valid monitoring results, and enabling timely obstacle avoidance measures based on these valid monitoring results to prevent collisions.

[0076] However, none of the above methods take into account the quantification of the evaluation results. Therefore, there is still a technical problem of poor accuracy in evaluating obstacle localization effectiveness.

[0077] Based on the above problems, this invention proposes a data processing method. By recognizing multiple frames of images to be monitored during the driving process, the method obtains recognition results and determines the confidence level of the recognition results. Then, based on the confidence level, the images to be monitored are verified to obtain verification results. Since the verification results are indexed, the technical problem of poor accuracy in obstacle positioning effect evaluation is solved, and the technical effect of improving the accuracy of obstacle positioning effect evaluation is achieved.

[0078] This invention embodiment is based on the obstacle result format required by the evaluation kit of the open-source autonomous driving dataset (Nuscenes) on the Internet. The required obstacle results may include specific content such as obstacle size, translation, velocity, tracking identifier, and tracking score. It should be noted that this is only an example and no specific restrictions are imposed on the required obstacle results.

[0079] The next step is to further introduce the obstacle size error assessment method of the post-fusion algorithm in this invention.

[0080] As an optional embodiment, a large number of images to be monitored can be acquired during the vehicle's operation. All images to be monitored can be stored according to a first category based on the scene. Within each scene, images can be stored according to a second category based on the frame segments. Based on the above classification and storage, multiple frames of images to be monitored can be obtained.

[0081] Optionally, the monitored objects can be classified based on the identification results. Figure 2 This is a schematic diagram of obstacle classification results according to an embodiment of the present invention, such as... Figure 2 As shown, the results of obstacle identification can include multiple scenes, each scene can include at least one frame segment, and each frame can include multiple types of monitored objects.

[0082] For example, the results of obstacle recognition can be divided into scene 1, scene 2 and scene 3, etc., according to the scene. In each scene, the image is divided into frame segments such as the first frame and the second frame. Each frame contains multiple types of monitoring objects 1 and monitoring objects 2, etc.

[0083] Figure 3 This is a flowchart of an obstacle size error assessment method according to an embodiment of the present invention, such as... Figure 3 As shown, the obstacle size error assessment method may include the following steps:

[0084] Step S302: The recall rate is determined by matching the tracking identifiers of obstacles with the distance matrix.

[0085] In the technical solution provided by step S302 of the present invention, the tracking identifier and distance matrix between the obstacle prediction result and the true result can be used to perform matching, determine whether the matching is correct, and divide the number of correct matches by the true value to obtain the recall rate.

[0086] Optionally, if the tracking identifier of the identified obstacle is successfully matched with the tracking identifier in the real result, the number of correct matches can be obtained, and the number of correct matches can be incremented by one for each correct match; if the match is unsuccessful, the distance matrix of the identified obstacle can be matched with the distance matrix in the real result. If the match is successful, the number of correct matches can be incremented by one, and the recall rate can be further determined.

[0087] Step S304: Determine the mapping relationship between confidence and recall.

[0088] In the technical solution provided by step S304 of the present invention, since different confidence levels will result in different recall rates for obstacle filtering, the corresponding confidence level can be obtained by interpolation, and the mapping relationship between confidence level and recall rate can be determined.

[0089] Optionally, the purpose of determining the mapping relationship between confidence and recall through interpolation is to smooth the corresponding confidence groups, thereby obtaining a confidence group that can be evenly divided. The result of the even division can be used to screen the monitored objects.

[0090] Step S306: Determine the confidence threshold.

[0091] In the technical solution provided by step S306 of the present invention, the threshold in each confidence group can be determined for each evenly divided confidence group obtained after smoothing.

[0092] For example, if we obtain the confidence levels of a group of monitored objects, such as 0.3, 0.4, 0.6, 0.7, and 0.8, we get a confidence level group that includes the above five confidence levels. Then, we smooth the confidence level group, for example, by inserting a confidence level of 0.5. We can then divide the interpolated confidence level group into two confidence level thresholds, [0.3, 0.5] and [0.6, 0.8]. The confidence level threshold group contains the above two confidence level thresholds, and the confidence level thresholds include the confidence levels within the confidence level threshold interval. By smoothing and dividing the confidence level group as described above, we can obtain the confidence level threshold group.

[0093] Step S308: Iterate through the confidence thresholds in each average score result.

[0094] In the technical solution provided by step S308 of the present invention, a list can be created to store the cumulative results of all scenarios under the confidence threshold in the current average score result. Then, each scenario is traversed to create a class object, wherein the class object is used to evaluate the cumulative quantity.

[0095] Optionally, each frame in each scene is traversed, and all results in the current evaluation category with a confidence level greater than the current confidence level threshold are selected as valid monitoring targets in the current frame. The stereo intersection-union ratio (IUU) between the identification results of the valid monitoring targets and the actual results is calculated. After the current frame is completed, the next frame is processed. The IUU value ranges from 0 to 1.

[0096] Step S310: Calculate the index value using the accumulated data mentioned above.

[0097] In the technical solution provided by step S310 of the present invention, the required data is extracted from the accumulated data, and the multi-target tracking scale error can be calculated using the following formula:

[0098]

[0099] Among them, motse r It can be used to represent the multi-object tracking size error when the recall is r, TP t IDS is the number of correctly associated frames at the t-th frame when the recall rate is r. t Let r be the number of erroneous associations in frame t. Let be the stereo intersection-union ratio between the true value i that was successfully matched in frame t and the hypothesis. The value of this index ranges from [0,1]. The lower the value, the higher the accuracy of the obstacle recognition result.

[0100] Optionally, all multi-target tracking scale errors in each frame are determined, and the minimum multi-target tracking scale error in all frames is selected. The average multi-target tracking scale error can be obtained by averaging all the minimum values.

[0101] For the effective multi-target tracking scale error under different thresholds for the same type of obstacle, the average multi-target tracking scale error can be calculated using the following formula:

[0102]

[0103] Among them, amotse is the average multi-object tracking scale error, which is the average effective multi-object tracking scale error under different thresholds for the same type of obstacle. The value range of this index is [0,1]. The lower the value, the higher the accuracy of the obstacle recognition result.

[0104] This invention addresses the technical problem of poor accuracy in obstacle positioning effect assessment by identifying multiple frames of images to be monitored during driving, determining the confidence level of the identification results, and then verifying the images to be monitored based on the confidence level. By indexing the verification results of obstacles, this invention improves the accuracy of obstacle positioning effect assessment.

[0105] Example 3

[0106] According to an embodiment of the present invention, a data processing apparatus is also provided. It should be noted that this data processing apparatus can be used to execute the data processing method in Embodiment 1.

[0107] Figure 4 This is a schematic diagram of a data processing apparatus according to an embodiment of the present invention. Figure 4 As shown, the data processing device 400 may include: an acquisition unit 402, an identification unit 404, and a verification unit 406.

[0108] The acquisition unit 402 is used to acquire multiple frames of images to be monitored during the vehicle's driving process.

[0109] The identification unit 404 is used to identify the image to be monitored, obtain the identification result, and determine the confidence level of the monitored object in the identification result in the image to be monitored, wherein the confidence level is used to identify the degree of matching of the monitored object.

[0110] The verification unit 406 is used to verify the image to be monitored based on the confidence level and obtain the verification result, wherein the verification result is used to characterize the accuracy of the image to be monitored.

[0111] Optionally, the verification unit 406 may include: a first classification module, which classifies the correctly identified multi-frame monitoring objects to obtain at least one monitoring object group; a first determination module, which classifies the at least one monitoring object group based on confidence level to obtain at least one sub-monitoring object group; a second determination module, which determines the sub-verification result of the at least one sub-monitoring object group to obtain at least one sub-verification result, wherein the sub-verification result is used to characterize the accuracy of identifying the same type of monitoring objects in the multi-frame images to be monitored; and a third determination module, which averages the sub-verification results to determine the verification result.

[0112] Optionally, the third determining module may include: a first dividing submodule, used to divide at least one confidence level in at least one monitoring object group into at least one confidence level group; a first determining submodule, used to smooth the at least one confidence level group respectively to obtain at least one equally divided confidence level threshold group, wherein the confidence level threshold group includes the confidence level of the corresponding confidence group; and a second determining submodule, used to classify the corresponding monitoring object group through the at least one confidence level threshold group to obtain at least one sub-monitoring object group, wherein the number of confidence level thresholds corresponds one-to-one with the number of sub-monitoring object groups.

[0113] Optionally, the second determining module may include: a third determining submodule, used to determine the interaction ratio between the detection box in the recognition result and the real detection box of the monitored object in the sub-monitoring object group; a fourth determining submodule, used to determine the number of differences between the identity information of the same monitored object and the identity information of adjacent frames in the sub-monitoring object group; and a fifth determining submodule, used to determine the sub-verification result based on the number of differences in identity information with adjacent frames, the interaction ratio, and the number of correct answers.

[0114] Optionally, the second determining module may include: a fifth determining submodule, used to determine the minimum value of the verification result in at least one monitoring object group respectively, to obtain at least one minimum value, and to determine the minimum value of the at least one sub-verification result as the target verification result of at least one monitoring object, wherein the number of target verification results corresponds one-to-one with the number of monitoring object groups; and a sixth determining submodule, used to perform an average calculation on all target verification results to obtain a verification result.

[0115] Optionally, the identification unit 404 may include: a fourth determining module, used to determine the correct number of times the monitoring object is identified based on the identity information in the identification result and the real identity information of the monitoring object; and a second determining module, used to determine the verification result based on the correct number of times.

[0116] Optionally, the fourth determining module may include: a seventh determining submodule, which, in response to a successful match between the identity information and the real identity information, can determine that the identification result of the monitored object is correct and increment the correct count by one.

[0117] Optionally, the fourth determining module may include: a first matching submodule, which, in response to a failure to match the identity information with the real identity information, matches the location information in the monitoring result with the real location information of the monitored object; and an eighth determining submodule, which, in response to a successful match between the location information and the real location information, determines that the identification result of the monitored object is correct and increments the correct count by one.

[0118] According to an embodiment of the present invention, an acquisition unit is used to acquire multiple frames of images to be monitored during the vehicle's driving process; an identification unit is used to identify the images to be monitored, obtain identification results, and determine the confidence level of the monitored object in the identification results, wherein the confidence level is used to represent the matching degree of the monitored object; and a verification unit is used to verify the images to be monitored based on the confidence level, obtain verification results, wherein the verification results are used to characterize the accuracy of the identification of the images to be monitored. By indexing the verification results of obstacles, the technical problem of poor accuracy in evaluating the positioning effect of monitored objects is solved, and the technical effect of improving the accuracy of evaluating the positioning effect of monitored objects is achieved.

[0119] Example 4

[0120] According to an embodiment of the present invention, a computer-readable storage medium is also provided, the storage medium including a stored program, wherein the program executes the data processing method described in Embodiment 1.

[0121] Example 5

[0122] According to an embodiment of the present invention, a processor is also provided for running a program, wherein the program executes the data processing method described in Embodiment 1 during runtime.

[0123] Example 6

[0124] According to an embodiment of the present invention, a vehicle is also provided, which is used to perform the data processing method of the embodiments of the present invention.

[0125] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0126] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0127] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0128] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0129] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0130] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0131] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data processing method, characterized in that, include: Acquire multiple frames of images of the vehicle during its driving process; The image to be monitored is identified to obtain an identification result, and the confidence level of the monitored object in the image to be monitored in the identification result is determined, wherein the confidence level is used to represent the degree of matching of the monitored object; Based on the confidence level, the image to be monitored is verified to obtain a verification result, wherein the verification result is used to characterize the accuracy of the recognition of the image to be monitored; Based on the confidence level, the image to be monitored is verified to obtain a verification result, including: classifying the correctly identified monitoring objects in multiple frames to obtain at least one monitoring object group; classifying the at least one monitoring object group based on the confidence level to obtain at least one sub-monitoring object group; determining the sub-verification result of the at least one sub-monitoring object group to obtain at least one sub-verification result, wherein the sub-verification result is used to characterize the accuracy of identifying the same type of monitoring object in multiple frames of the image to be monitored; and averaging the sub-verification results to determine the verification result.

2. The method according to claim 1, characterized in that, Based on the confidence level, at least one of the monitored object groups is classified to obtain at least one sub-monitored object group, including: Divide at least one confidence level in at least one of the monitored object groups into at least one confidence group; Smoothing is performed on at least one of the confidence groups to obtain at least one equally divided confidence threshold group, wherein the confidence threshold group includes the confidence in the corresponding confidence group; The corresponding monitoring object group is classified by at least one set of confidence thresholds to obtain at least one sub-monitoring object group, wherein the number of confidence thresholds corresponds one-to-one with the number of sub-monitoring object groups.

3. The method according to claim 1, characterized in that, Determine the sub-validation results for each of the sub-monitoring object groups, including: Determine the intersection-union ratio (IoU) between the detection bounding boxes in the identification results of the sub-monitoring object group and the actual detection bounding boxes of the monitored objects; Determine the number of instances in the sub-monitoring object group where the identity information of the same monitored object differs from that of adjacent frames; The sub-verification result is determined based on the number of identity information that differs from that of adjacent frames, the cross-union ratio, and the number of correct verifications.

4. The method according to claim 3, characterized in that, include: The minimum value of the sub-verification results in at least one of the monitoring object groups is determined to obtain at least one minimum value. The minimum value of the at least one sub-verification result is determined as the target verification result of at least one of the monitoring object groups, wherein the number of target verification results corresponds one-to-one with the number of monitoring object groups. The average of all the target verification results is used to obtain the verification result.

5. The method according to claim 1, characterized in that, The method further includes: Based on the identity information in the identification results and the real identity information of the monitored object, determine the correct number of times the monitored object is identified; The verification result is determined based on the number of correct responses.

6. The method according to claim 5, characterized in that, Based on the identity information in the identification result and the real identity information, the correct number of times the monitored object is identified is determined, including: In response to a successful match between the identity information and the real identity information, the identification result of the monitored object is determined to be correct, and the correct count is incremented by one.

7. The method according to claim 6, characterized in that, The method further includes: In response to the failure to match the identity information with the real identity information, the location information in the monitoring result is matched with the real location information of the monitored object; In response to a successful match between the location information and the actual location information, the identification result of the monitored object is determined to be correct, and the correct count is incremented by one.

8. A data processing apparatus, characterized in that, include: The acquisition unit is used to acquire multiple frames of images to be monitored during the vehicle's driving process; The identification unit is used to identify the image to be monitored, obtain an identification result, and determine the confidence level of the monitored object in the identification result, wherein the confidence level is used to represent the matching degree of the monitored object; A verification unit is used to verify the image to be monitored based on the confidence level and obtain a verification result, wherein the verification result is used to characterize the accuracy of the recognition of the image to be monitored; The device is further configured to: classify the correctly identified monitoring objects in multiple frames to obtain at least one monitoring object group; classify the at least one monitoring object group based on the confidence level to obtain at least one sub-monitoring object group; determine the sub-verification result of the at least one sub-monitoring object group to obtain at least one sub-verification result, wherein the sub-verification result is used to characterize the accuracy of identifying the same type of monitoring object in multiple frames of the image to be monitored; and average the sub-verification results to determine the verification result.

9. A vehicle, characterized in that, Used to perform the method according to any one of claims 1 to 7.

Images