Perception result processing method and apparatus, storage medium, electronic device, and vehicle

Abstract

This disclosure relates to a method, apparatus, storage medium, electronic device, and vehicle for processing perception results. The method includes: acquiring a ground truth perception result corresponding to each frame of an environmental image, wherein the environmental image is an image of the vehicle's surroundings collected during vehicle operation; acquiring a first perception result corresponding to each frame of the environmental image determined by a first perception system, and a second perception result corresponding to each frame of the environmental image determined by a second perception system; and aligning a first timestamp corresponding to a plurality of the first perception results with a second timestamp corresponding to a plurality of the second perception results based on the plurality of ground truth perception results. Thus, with timestamp matching, the first perception result and the second perception result can be compared, thereby enabling comparative evaluation of the first perception system and the second perception system.

Description

Technical Field

[0001] This disclosure relates to the field of autonomous driving technology, and in particular to a perception result processing method, device, storage medium, electronic device, and vehicle. Background Technology

[0002] With the development of artificial intelligence technology, autonomous driving technology for vehicles is also rapidly advancing. Sensors on autonomous vehicles typically include camera sensors, LiDAR sensors, and solid-state radar sensors. The perception system needs to output perception results based on the image data collected by the sensors, and the autonomous driving system controls the vehicle's autonomous driving based on these perception results. Therefore, the perception system is crucial in autonomous driving.

[0003] In related technologies, the performance of a vehicle's perception system is evaluated through extensive road tests or simulation testing. However, this method can only test its own performance and cannot determine the gap with other perception systems in the field. Therefore, how to achieve comparative testing of different perception systems has become an urgent problem to be solved. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this disclosure provides a method, apparatus, storage medium, electronic device, and vehicle for processing sensing results.

[0005] According to a first aspect of the present disclosure, a method for processing perception results is provided, comprising:

[0006] Obtain the truth perception result corresponding to each frame of the environmental image, wherein the environmental image is the image around the vehicle collected during the vehicle's movement;

[0007] Acquire the first perception result corresponding to each frame of the environmental image determined by the first perception system, and the second perception result corresponding to each frame of the environmental image determined by the second perception system;

[0008] Based on multiple truth-aware results, the first timestamps corresponding to the multiple first-aware results are aligned with the second timestamps corresponding to the multiple second-aware results.

[0009] Optionally, aligning the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results based on the multiple truth-aware results includes:

[0010] Based on multiple truth-aware results, the truth-aware trajectory of the detected target object is determined;

[0011] Based on multiple first perception results, a first detection trajectory of the detected target object is determined;

[0012] Based on multiple second perception results, a second detection trajectory of the detected target object is determined;

[0013] Based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

[0014] Optionally, aligning the first timestamps corresponding to the plurality of first perception results with the second timestamps corresponding to the plurality of second perception results based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory includes:

[0015] According to a first preset duration, based on the first detection trajectory and the second detection trajectory, a first true value detection trajectory to be matched is determined from the true value detection trajectory;

[0016] According to the second preset time, based on the first detection trajectory and the second detection trajectory, the matching target perception result is determined from the first true value detection trajectory, wherein the second preset time is less than the first preset time;

[0017] Based on the timestamps corresponding to the target perception results, update the first timestamps corresponding to multiple first perception results and the second timestamps corresponding to multiple second perception results.

[0018] Optionally, updating the first timestamps corresponding to multiple first perception results and the second timestamps corresponding to multiple second perception results based on the timestamps corresponding to the target perception results includes:

[0019] Based on the timestamp corresponding to the target perception result, adjust the first timestamp corresponding to the first target perception result that matches the target perception result in the first detection trajectory, and adjust the second timestamp corresponding to the second target perception result that matches the target perception result in the second detection trajectory;

[0020] Based on the first timestamp difference corresponding to the first target perception result, the first timestamp corresponding to the first remaining perception result is adjusted. The first remaining perception result includes multiple first perception results other than the first target perception result. The first timestamp difference is the difference between the updated first target timestamp and the first target timestamp before the update. The first target timestamp is the first timestamp corresponding to the first target perception result.

[0021] Based on the second timestamp difference corresponding to the second target perception result, the second timestamp corresponding to the second remaining perception result is adjusted. The second remaining perception result includes multiple second perception results other than the second target perception result. The second timestamp difference is the difference between the updated second target timestamp and the original second target timestamp. The second target timestamp is the second timestamp corresponding to the second target perception result.

[0022] Optionally, the method further includes:

[0023] Based on the position information of the target object in the truth detection trajectory, a second truth detection trajectory is determined from the truth detection trajectory;

[0024] The step of aligning the first timestamps corresponding to multiple first perception results with the second timestamps corresponding to multiple second perception results based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory includes:

[0025] Based on the second truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

[0026] Optionally, obtaining the truth-aware result corresponding to each frame of the environmental image includes:

[0027] For each frame of the environmental image, the environmental image is input into a pre-trained environmental perception model to obtain the ground truth perception result output by the environmental perception model.

[0028] Optionally, the method further includes:

[0029] Based on multiple truth perception results, the accuracy of the first perception system is determined according to the first timestamp after aligning the multiple first perception results.

[0030] Based on multiple truth-aware results, the accuracy of the second sensing system is determined according to a second timestamp aligned with the multiple second sensing results.

[0031] According to a second aspect of the present disclosure, a sensing result processing apparatus is provided, comprising:

[0032] The first acquisition module is configured to acquire the truth perception result corresponding to each frame of the environmental image, wherein the environmental image is an image of the vehicle's surroundings collected during the vehicle's driving process;

[0033] The second acquisition module is configured to acquire a first perception result corresponding to each frame of the environmental image determined by the first perception system, and a second perception result corresponding to each frame of the environmental image determined by the second perception system.

[0034] The alignment module is configured to align the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results based on the multiple truth-aware results.

[0035] Optionally, the alignment module is further configured to:

[0036] Based on multiple truth-aware results, the truth-aware trajectory of the detected target object is determined;

[0037] Based on multiple first perception results, a first detection trajectory of the detected target object is determined;

[0038] Based on multiple second perception results, a second detection trajectory of the detected target object is determined;

[0039] Based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

[0040] Optionally, the alignment module is further configured to:

[0041] According to a first preset duration, based on the first detection trajectory and the second detection trajectory, a first true value detection trajectory to be matched is determined from the true value detection trajectory;

[0042] According to the second preset time, based on the first detection trajectory and the second detection trajectory, the matching target perception result is determined from the first true value detection trajectory, wherein the second preset time is less than the first preset time;

[0043] Based on the timestamps corresponding to the target perception results, update the first timestamps corresponding to multiple first perception results and the second timestamps corresponding to multiple second perception results.

[0044] Optionally, the alignment module is further configured to:

[0045] Based on the timestamp corresponding to the target perception result, adjust the first timestamp corresponding to the first target perception result that matches the target perception result in the first detection trajectory, and adjust the second timestamp corresponding to the second target perception result that matches the target perception result in the second detection trajectory;

[0046] Based on the first timestamp difference corresponding to the first target perception result, the first timestamp corresponding to the first remaining perception result is adjusted. The first remaining perception result includes multiple first perception results other than the first target perception result. The first timestamp difference is the difference between the updated first target timestamp and the first target timestamp before the update. The first target timestamp is the first timestamp corresponding to the first target perception result.

[0047] Based on the second timestamp difference corresponding to the second target perception result, the second timestamp corresponding to the second remaining perception result is adjusted. The second remaining perception result includes multiple second perception results other than the second target perception result. The second timestamp difference is the difference between the updated second target timestamp and the original second target timestamp. The second target timestamp is the second timestamp corresponding to the second target perception result.

[0048] Optionally, the device further includes:

[0049] The first determining module is configured to determine a second truth detection trajectory from the truth detection trajectory based on the position information of the target object in the truth detection trajectory;

[0050] The alignment module is also configured to:

[0051] Based on the second truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

[0052] Optionally, the first acquisition module is further configured to:

[0053] For each frame of the environmental image, the environmental image is input into a pre-trained environmental perception model to obtain the ground truth perception result output by the environmental perception model.

[0054] Optionally, the device further includes:

[0055] The second determining module is configured to determine the accuracy of the first sensing system based on a plurality of truth-aware results and a first timestamp aligned with the plurality of first sensing results.

[0056] The third determining module is configured to determine the accuracy of the second sensing system based on a plurality of truth-aware results and a second timestamp aligned with the plurality of second sensing results.

[0057] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, having stored thereon computer program instructions that, when executed by a processor, implement the steps of the perception result processing method provided in the first aspect of the present disclosure.

[0058] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising:

[0059] A memory on which computer programs are stored;

[0060] A processor is configured to execute the computer program in the memory to implement the steps of the perception result processing method provided in the first aspect of this disclosure.

[0061] According to a fifth aspect of the present disclosure, a vehicle is provided, the vehicle including electronic equipment as described in the fourth aspect above; or, the vehicle and the electronic equipment described in the fourth aspect above are independent of each other and communicatively connected.

[0062] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects: acquiring a truth perception result corresponding to each frame of an environmental image, wherein the environmental image is an image of the vehicle's surroundings collected during vehicle operation; acquiring a first perception result corresponding to each frame of the environmental image determined by a first perception system, and a second perception result corresponding to each frame of the environmental image determined by a second perception system; and aligning a first timestamp corresponding to a plurality of the first perception results with a second timestamp corresponding to a plurality of the second perception results based on the plurality of truth perception results. In other words, this disclosure can align the timestamps of the first perception results determined by the first perception system and the timestamps of the second perception results determined by the second perception system based on the truth perception results. Thus, when the timestamps match, the first perception results and the second perception results can be compared, thereby enabling a comparative evaluation of the first perception system and the second perception system.

[0063] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0064] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0065] Figure 1 This is a flowchart illustrating a perception result processing method according to an exemplary embodiment of the present disclosure;

[0066] Figure 2 This is a flowchart illustrating another perception result processing method according to an exemplary embodiment of the present disclosure;

[0067] Figure 3 It is based on Figure 2 The illustrated embodiment shows a flowchart of another method for processing perception results;

[0068] Figure 4 This is a flowchart illustrating another perception result processing method according to an exemplary embodiment of the present disclosure;

[0069] Figure 5 This is a block diagram illustrating a perception result processing apparatus according to an exemplary embodiment of the present disclosure;

[0070] Figure 6 This is a block diagram of another perception result processing apparatus according to an exemplary embodiment of the present disclosure;

[0071] Figure 7 This is a block diagram of another perception result processing apparatus according to an exemplary embodiment of the present disclosure;

[0072] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. Detailed Implementation

[0073] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0074] The present disclosure will now be described in conjunction with specific embodiments.

[0075] Figure 1 This is a flowchart illustrating a perception result processing method according to an exemplary embodiment of the present disclosure, such as... Figure 1 As shown, the method may include:

[0076] S101. Obtain the truth perception result corresponding to each frame of the environment image.

[0077] The environmental image refers to the image of the vehicle's surroundings captured during vehicle movement. This image can be captured by cameras installed on the vehicle. The cameras used to capture the environmental image are determined based on the image parameters of the environmental images that the first and second sensing systems can process. These image parameters can be pixel values ​​or other image parameters, which are not limited in this disclosure. For example, if the image parameters of the environmental images that the first and second sensing systems can process are the same, the environmental image can be captured using the same set of cameras. If the image parameters of the environmental images that the first and second sensing systems can process are different, the environmental image can be captured using two separate sets of cameras, which can be installed at the same angle on the vehicle.

[0078] The perception results may include the size of the perceived target object, the position information of the target object relative to the vehicle, the speed information of the target object, and the target object may include vehicles, obstacles, etc.

[0079] In this step, after acquiring the environmental image, for each frame of the environmental image, the environmental image can be input into a pre-trained environmental perception model to obtain the ground truth perception result output by the environmental perception model. This environmental perception model can be trained using existing model training methods, which will not be elaborated here. It should be noted that if the acquired environmental images include two sets, the target environmental image can be determined from the two sets, and the corresponding ground truth perception result can be obtained. For example, either set of environmental images can be used as the target environmental image, or the set with the higher pixel value can be used as the target environmental image.

[0080] S102. Obtain the first perception result corresponding to each frame of the environment image determined by the first perception system, and the second perception result corresponding to each frame of the environment image determined by the second perception system.

[0081] In this step, if the image parameters of the environmental images that the first sensing system and the second sensing system can process are different, the first sensing system can acquire the environmental images acquired by a set of cameras corresponding to the first sensing system and determine the first sensing result corresponding to the environmental images. The second sensing system can acquire the environmental images acquired by a set of cameras corresponding to the second sensing system and determine the second sensing result corresponding to the environmental images.

[0082] S103. Based on the multiple truth perception results, align the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results.

[0083] In this step, after determining the truth perception result, the first perception result, and the second perception result, a target environment image with the same perception result for one of the target objects can be identified from these three results. Based on the timestamp of the truth perception result corresponding to the target environment image, the first timestamp of the first perception result and the second timestamp of the second perception result corresponding to the target environment image are adjusted to align the adjusted first and second timestamps. For example, if the timestamp of the truth perception result corresponding to the target environment image is 5.003s, the first timestamp of the first perception result is 5.001s, and the second timestamp of the second perception result is 5.116s, then the adjusted first timestamp could be 5.001s + 0.002s, and the adjusted second timestamp could be 5.116s - 0.113s. This aligns the adjusted first and second timestamps.

[0084] It should be noted that the execution order of steps S101 and S102 is not important and they can be executed simultaneously.

[0085] Using the above method, based on the truth perception results, the timestamps corresponding to the first perception results determined by the first perception system and the second perception results determined by the second perception system are aligned. In this way, when the timestamps match, the first perception results and the second perception results can be compared, thereby enabling the comparative evaluation of the first perception system and the second perception system.

[0086] Figure 2 This is a flowchart illustrating another perception result processing method according to an exemplary embodiment of the present disclosure, such as... Figure 2 As shown, the implementation of step S103 may include:

[0087] S1031. Based on multiple truth-sensing results, determine the truth detection trajectory of the detected target object.

[0088] In this step, after determining the truth perception result corresponding to each frame of the environmental image, each detected target object can be determined based on multiple truth perception results. For each target object, the truth detection trajectory of the target object can be determined based on multiple truth perception results.

[0089] S1032. Based on multiple first perception results, determine the first detection trajectory of the detected target object.

[0090] In this step, after determining the first perception result corresponding to each frame of the environmental image, each detected target object can be determined based on multiple first perception results. For each target object, the first detection trajectory of the target object is determined based on multiple first perception results.

[0091] S1033. Based on multiple second perception results, determine the second detection trajectory of the detected target object.

[0092] In this step, after determining the second perception result corresponding to each frame of the environmental image, each detected target object can be determined based on multiple second perception results. For each target object, the second detection trajectory of the target object is determined based on multiple second perception results.

[0093] S1034. Based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory, align the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results.

[0094] In this step, after determining the true value detection trajectory, the first detection trajectory, and the second detection trajectory, a first true value detection trajectory to be matched can be determined from the true value detection trajectory according to a first preset duration, based on the first and second detection trajectories; a matching target perception result can be determined from the first true value detection trajectory according to a second preset duration, based on the first and second detection trajectories, where the second preset duration is less than the first preset duration; and the first timestamps corresponding to multiple first perception results and the second timestamps corresponding to multiple second perception results are updated according to the timestamps corresponding to the target perception result. The first preset duration can be determined through a pre-set duration correlation, which can include the correspondence between different driving conditions and durations. For example, for high-speed driving conditions, the first preset duration can be set to a larger value, such as 10 seconds; for urban driving conditions, the first preset duration can be set to a smaller value, such as 1 second. The second preset duration can be preset based on experience; for example, the second preset duration can be 50 ms.

[0095] For example, the first preset duration can be used as the sliding window step size. Using existing sliding window positioning methods, multiple similar environmental images matching the first and second ground truth trajectories can be determined from the ground truth detection trajectory. These similar environmental images are those similar to the ground truth perception result, the first perception result, and the second perception result across multiple frames of environmental images. For example, if the positions of the detected target objects in the ground truth perception result, the first perception result, and the second perception result of the target environmental image are relatively close, then the target environmental image can be determined as the similar environmental image. After determining the similar environmental images, the detection trajectory composed of multiple similar environmental images is used as the first ground truth detection trajectory.

[0096] After determining the first ground truth detection trajectory, the second preset time duration can be used as the sliding window step size. Using existing sliding window methods, a target perception result matching the first ground truth detection trajectory and the second ground truth trajectory is determined from the first ground truth detection trajectory. This target perception result is the ground truth perception result corresponding to the target similar environment image that matches the ground truth perception result, the first perception result, and the second perception result among multiple similar environment images. Matching the ground truth perception result, the first perception result, and the second perception result can be understood as the perception results for the target object being the same in the ground truth perception result, the first perception result, and the second perception result. For example, the location information of the target vehicle determined in the ground truth perception result, the first perception result, and the second perception result are the same.

[0097] In one possible implementation, after determining the target perception result, the first timestamp corresponding to the first target perception result matching the target perception result in the first detection trajectory and the second timestamp corresponding to the second target perception result matching the target perception result in the second detection trajectory can be adjusted based on the timestamp corresponding to the target perception result. The first timestamp corresponding to the first remaining perception result can be adjusted based on the first timestamp difference, where the first remaining perception result includes multiple first perception results other than the first target perception result, and the first timestamp difference is the difference between the updated first target timestamp and the first target timestamp before the update; the first target timestamp is the first timestamp corresponding to the first target perception result. The second timestamp corresponding to the second remaining perception result can be adjusted based on the second timestamp difference, where the second remaining perception result includes multiple second perception results other than the second target perception result, and the second timestamp difference is the difference between the updated second target timestamp and the second target timestamp before the update; the second target timestamp is the second timestamp corresponding to the second target perception result.

[0098] For example, after determining the target perception result, the first target timestamp of the first target perception result matching the target perception result in the first detection trajectory and the second target timestamp of the second target perception result matching the target perception result in the second detection trajectory can be adjusted according to the timestamp corresponding to the target perception result. After adjusting the first target timestamp and the second target timestamp, the difference between the first timestamp and the difference between the second timestamp can be determined. Based on the difference between the first timestamps, the first timestamp corresponding to the first remaining perception result is adjusted, and based on the difference between the second timestamps, the second timestamp corresponding to the second remaining perception result is adjusted. For example, if the difference between the first timestamps is 0.001s, the first timestamp corresponding to each first perception result in the first remaining perception result can be increased by 0.001s; if the difference between the second timestamps is -0.011s, the second timestamp corresponding to each second perception result in the second remaining perception result can be decreased by 0.011s.

[0099] In another possible implementation, if the frame rates of the first sensing system and the second sensing system are different, the first timestamp corresponding to each first sensing result in the first remaining sensing results can be adjusted in the manner described above for adjusting the first target timestamp corresponding to the first target sensing result, and the second timestamp corresponding to each second sensing result in the second remaining sensing results can be adjusted in the manner described above for adjusting the second target timestamp corresponding to the second target sensing result.

[0100] Figure 3 It is based on Figure 2 The flowchart of another perception result processing method shown in the embodiment is as follows: Figure 3 As shown, the method may further include:

[0101] S1035. Based on the position information of the target object in the true value detection trajectory, determine the second true value detection trajectory from the true value detection trajectory.

[0102] Step S1034 can be implemented as follows:

[0103] S1036. Based on the second true value detection trajectory, the first detection trajectory, and the second detection trajectory, align the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results.

[0104] For example, after determining the ground truth detection trajectory corresponding to each target object, a second ground truth detection trajectory for the target object in the core region can be determined from multiple ground truth detection trajectories based on the target object's location information contained in each ground truth detection trajectory. This core region can be a rectangular area in front of the vehicle, with its lower edge representing the vehicle's location, its upper edge representing a location 30 meters away from the vehicle, its left edge representing the location of the second lane to the left of the vehicle's lane, and its right edge representing the location of the second lane to the right of the vehicle's lane. It should be noted that the above core region is merely an illustrative example, and this disclosure does not limit its scope.

[0105] After determining the second truth detection trajectory, the first timestamps corresponding to multiple first perception results can be aligned with the second timestamps corresponding to multiple second perception results based on the second truth detection trajectory, the first detection trajectory, and the second detection trajectory. Since the second truth detection trajectory is a filtered truth detection trajectory, it is more convenient to match the first detection trajectory and the second detection trajectory, thereby improving the comparison efficiency of perception results.

[0106] Figure 4 This is a flowchart illustrating another perception result processing method according to an exemplary embodiment of the present disclosure, such as... Figure 4 As shown, the method may further include:

[0107] S104. Based on multiple truth perception results, determine the accuracy of the first perception system according to the first timestamp after aligning the multiple first perception results.

[0108] S105. Based on multiple true-value perception results, determine the accuracy of the second perception system according to the second timestamp after aligning the multiple second perception results.

[0109] For example, after aligning the first timestamps corresponding to multiple first perception results with the second timestamps corresponding to multiple second perception results, the accuracy of the first perception system can be determined by comparing the first perception result with the truth perception result according to the aligned first timestamp. Similarly, the accuracy of the second perception system can be determined by comparing the second perception result with the truth perception result according to the aligned second timestamp. This disclosure can also use the first perception system as a reference, directly comparing the first perception result with the second perception result according to the aligned first timestamp and the aligned second timestamp to determine the difference between the second perception system and the first perception system.

[0110] Figure 5 This is a block diagram illustrating a perception result processing apparatus according to an exemplary embodiment of the present disclosure, such as... Figure 5As shown, the device may include:

[0111] The first acquisition module 501 is configured to acquire the truth perception result corresponding to each frame of the environmental image, wherein the environmental image is an image of the vehicle's surroundings collected during the vehicle's driving process;

[0112] The second acquisition module 502 is configured to acquire the first perception result corresponding to each frame of the environment image determined by the first perception system, and the second perception result corresponding to each frame of the environment image determined by the second perception system.

[0113] Alignment module 503 is configured to align the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results based on the multiple truth-value perception results.

[0114] Optionally, the alignment module 503 is also configured to:

[0115] Based on multiple truth-aware results, determine the truth-aware trajectory of the detected target object;

[0116] Based on multiple initial perception results, the first detection trajectory of the detected target object is determined;

[0117] Based on multiple second perception results, determine the second detection trajectory of the detected target object;

[0118] Based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

[0119] Optionally, the alignment module 503 is also configured to:

[0120] According to the first preset time, the first true value detection trajectory to be matched is determined from the true value detection trajectory based on the first detection trajectory and the second detection trajectory.

[0121] According to the second preset duration, based on the first detection trajectory and the second detection trajectory, the matching target perception result is determined from the first true value detection trajectory, wherein the second preset duration is less than the first preset duration;

[0122] Based on the timestamp corresponding to the target perception result, update the first timestamp corresponding to the first perception result and the second timestamp corresponding to the second perception result.

[0123] Optionally, the alignment module 503 is also configured to:

[0124] Based on the timestamp corresponding to the target perception result, adjust the first timestamp corresponding to the first target perception result that matches the target perception result in the first detection trajectory, and adjust the second timestamp corresponding to the second target perception result that matches the target perception result in the second detection trajectory;

[0125] Based on the first timestamp difference corresponding to the first target perception result, the first timestamp corresponding to the first remaining perception result is adjusted. The first remaining perception result includes multiple first perception results other than the first target perception result. The first timestamp difference is the difference between the updated first target timestamp and the first target timestamp before the update. The first target timestamp is the first timestamp corresponding to the first target perception result.

[0126] Based on the second timestamp difference corresponding to the second target perception result, the second timestamp corresponding to the second remaining perception result is adjusted. The second remaining perception result includes multiple second perception results other than the second target perception result. The second timestamp difference is the difference between the updated second target timestamp and the original second target timestamp. The second target timestamp is the second timestamp corresponding to the second target perception result.

[0127] Optionally, Figure 6 This is a block diagram illustrating another perception result processing apparatus according to an exemplary embodiment of the present disclosure, such as... Figure 6 As shown, the device also includes:

[0128] The first determining module 504 is configured to determine a second truth detection trajectory from the truth detection trajectory based on the position information of the target object in the truth detection trajectory.

[0129] The alignment module 503 is also configured as follows:

[0130] Based on the second true value detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

[0131] Optionally, the first acquisition module 501 is further configured to:

[0132] For each frame of the environment image, the environment image is input into a pre-trained environment perception model to obtain the ground truth perception result output by the environment perception model.

[0133] Optionally, Figure 7 This is a block diagram illustrating another perception result processing apparatus according to an exemplary embodiment of the present disclosure, such as... Figure 7 As shown, the device also includes:

[0134] The second determining module 505 is configured to determine the accuracy of the first sensing system based on multiple truth-value sensing results and a first timestamp aligned with the multiple first sensing results.

[0135] The third determining module 506 is configured to determine the accuracy of the second sensing system based on multiple truth-value sensing results and a second timestamp aligned with multiple second sensing results.

[0136] Using the aforementioned device, the timestamps corresponding to the first perception result determined by the first perception system and the second perception result determined by the second perception system are aligned based on the truth perception results. In this way, when the timestamps match, the first perception result and the second perception result can be compared, thereby enabling the comparative evaluation of the first perception system and the second perception system.

[0137] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0138] This disclosure also provides a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, implement the steps of the perception result processing method provided in this disclosure.

[0139] Figure 8 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment of the present disclosure. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0140] Reference Figure 8 The electronic device 800 may include one or more of the following components: processing component 802, memory 804, power supply component 806, multimedia component 808, audio component 810, input / output interface 812, sensor component 814, and communication component 816.

[0141] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the aforementioned perception result processing method. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0142] Memory 804 is configured to store various types of data to support the operation of electronic device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0143] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0144] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0145] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0146] Input / output interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0147] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 can detect the on / off state of electronic device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0148] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0149] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described sensing result processing method.

[0150] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to complete the aforementioned perception result processing method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0151] In another exemplary embodiment, a computer program product is also provided, which includes a computer program executable by a programmable device, the computer program having a code portion for performing the above-described perception result processing method when executed by the programmable device.

[0152] In another exemplary embodiment, a vehicle is also provided, which includes the aforementioned electronic equipment; or, the vehicle and the aforementioned electronic equipment are independent of each other but are communicatively connected.

[0153] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of this disclosure. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0154] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A method for processing perception results, characterized in that, include: Obtain the truth perception result corresponding to each frame of the environmental image, wherein the environmental image is the image around the vehicle collected during the vehicle's movement; Acquire the first perception result corresponding to each frame of the environmental image determined by the first perception system, and the second perception result corresponding to each frame of the environmental image determined by the second perception system; Based on the multiple truth-aware results, align the first timestamps corresponding to the multiple first-aware results with the second timestamps corresponding to the multiple second-aware results. The step of aligning the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results based on the multiple truth-aware results includes: Based on multiple truth-aware results, the truth-aware trajectory of the detected target object is determined; Based on multiple first perception results, a first detection trajectory of the detected target object is determined; Based on multiple second perception results, a second detection trajectory of the detected target object is determined; Based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

2. The method according to claim 1, characterized in that, The step of aligning the first timestamps corresponding to multiple first perception results with the second timestamps corresponding to multiple second perception results based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory includes: According to a first preset duration, based on the first detection trajectory and the second detection trajectory, a first true value detection trajectory to be matched is determined from the true value detection trajectory; According to the second preset time, based on the first detection trajectory and the second detection trajectory, the matching target perception result is determined from the first true value detection trajectory, wherein the second preset time is less than the first preset time; Based on the timestamps corresponding to the target perception results, update the first timestamps corresponding to multiple first perception results and the second timestamps corresponding to multiple second perception results.

3. The method according to claim 2, characterized in that, The step of updating the first timestamps corresponding to multiple first perception results and the second timestamps corresponding to multiple second perception results based on the timestamps corresponding to the target perception results includes: Based on the timestamp corresponding to the target perception result, adjust the first timestamp corresponding to the first target perception result that matches the target perception result in the first detection trajectory, and adjust the second timestamp corresponding to the second target perception result that matches the target perception result in the second detection trajectory; Based on the first timestamp difference corresponding to the first target perception result, the first timestamp corresponding to the first remaining perception result is adjusted. The first remaining perception result includes multiple first perception results other than the first target perception result. The first timestamp difference is the difference between the updated first target timestamp and the first target timestamp before the update. The first target timestamp is the first timestamp corresponding to the first target perception result. Based on the second timestamp difference corresponding to the second target perception result, the second timestamp corresponding to the second remaining perception result is adjusted. The second remaining perception result includes multiple second perception results other than the second target perception result. The second timestamp difference is the difference between the updated second target timestamp and the original second target timestamp. The second target timestamp is the second timestamp corresponding to the second target perception result.

4. The method according to claim 3, characterized in that, The method further includes: Based on the position information of the target object in the truth detection trajectory, a second truth detection trajectory is determined from the truth detection trajectory; The step of aligning the first timestamps corresponding to multiple first perception results with the second timestamps corresponding to multiple second perception results based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory includes: Based on the second truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

5. The method according to claim 1, characterized in that, The process of obtaining the truth-aware results corresponding to each frame of the environmental image includes: For each frame of the environmental image, the environmental image is input into a pre-trained environmental perception model to obtain the ground truth perception result output by the environmental perception model.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Based on multiple truth perception results, the accuracy of the first perception system is determined according to the first timestamp after aligning the multiple first perception results. Based on multiple truth-aware results, the accuracy of the second sensing system is determined according to a second timestamp aligned with the multiple second sensing results.

7. A sensing result processing device, characterized in that, include: The first acquisition module is configured to acquire the truth perception result corresponding to each frame of the environmental image, wherein the environmental image is an image of the vehicle's surroundings collected during the vehicle's driving process; The second acquisition module is configured to acquire a first perception result corresponding to each frame of the environmental image determined by the first perception system, and a second perception result corresponding to each frame of the environmental image determined by the second perception system. The alignment module is configured to align the first timestamps corresponding to the multiple first perception results with the second timestamps corresponding to the multiple second perception results based on the multiple truth perception results. The alignment module is also configured to: Based on multiple truth-aware results, the truth-aware trajectory of the detected target object is determined; Based on multiple first perception results, a first detection trajectory of the detected target object is determined; Based on multiple second perception results, a second detection trajectory of the detected target object is determined; Based on the truth detection trajectory, the first detection trajectory, and the second detection trajectory, the first timestamps corresponding to the multiple first perception results are aligned with the second timestamps corresponding to the multiple second perception results.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that, When executed by a processor, the program instructions implement the steps of the method described in any one of claims 1-6.

9. An electronic device, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-6.

10. A vehicle, characterized in that, The vehicle includes the electronic device as described in claim 9; or, the vehicle and the electronic device as described in claim 9 are independent of each other but are communicatively connected.

Images