Method and apparatus for determining actual values of vehicle three-dimensional coordinates, device, medium, and vehicle
Through the coordinated work of the main acquisition vehicle and the auxiliary acquisition vehicle, the coordinate conversion matrix is calculated and the position information of the auxiliary acquisition vehicle is projected, which solves the problem that the lidar cannot perceive the three-dimensional coordinates of ultra-long-distance vehicles, and realizes accurate three-dimensional coordinate perception.
Patent Information
- Application Number
- PCT/CN2023/110169
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-31
- Filing Date
- 2023-07-31
- Publication Date
- 2024-12-05
AI Technical Summary
The existing lidar can sense the vehicle's three-dimensional coordinates within a range of 60 meters to 120 meters, but cannot sense it at a long distance, resulting in the inability to obtain the vehicle's three-dimensional coordinate truth value.
Through the mutual cooperation between the main acquisition vehicle and the auxiliary acquisition vehicle, data is collected using inertia sensors and vehicle-end sensors, the coordinate conversion matrix is calculated, and the position information of the auxiliary acquisition vehicle is projected into the vehicle coordinate system of the main acquisition vehicle, and the ultra-far results are obtained The truth value of the three-dimensional coordinates of the vehicle.
It realizes the accurate three-dimensional coordinate truth value of ultra-long-distance vehicles without adding new sensors, and expands the perception range.
Smart Images

Figure CN2023110169_05122024_PF_FP_ABST
Abstract
Description
Method, device, equipment, medium and vehicle for determining true value of vehicle three-dimensional coordinates Technical Field
[0001] The present invention relates to the field of intelligent driving technology, and in particular to a method, device, equipment, medium and vehicle for determining the true value of the three-dimensional coordinates of a vehicle. Background Art
[0002] During the driving process of an unmanned vehicle, it needs to detect vehicles on the road so as to avoid accidents such as collisions and ensure its driving safety.
[0003] Specifically, vehicle-side sensors, such as lidar, can be installed on the unmanned vehicle. The lidar can sense the surrounding environment of the unmanned vehicle during its driving process to obtain the true three-dimensional coordinates of each sensed vehicle.
[0004] The perception distance of conventional lidar is only 60 meters to 120 meters. Once the distance exceeds the upper limit of 120 meters, the lidar cannot obtain any three-dimensional perception results. Therefore, it is currently impossible to obtain the true three-dimensional coordinates of vehicles at ultra-long distances.
[0005] Summary of the Invention
[0006] The present invention provides a method, device, equipment, medium, and vehicle for determining the true value of a vehicle's three-dimensional coordinates, capable of obtaining the true value of a vehicle's three-dimensional coordinates at an extremely long distance. The specific technical solution is as follows.
[0007] In a first aspect, an embodiment of the present invention provides a method for determining the true value of a vehicle's three-dimensional coordinates. In an initial state, a main collection vehicle and an auxiliary collection vehicle are both located at an initial point and the vehicle-side sensors of both vehicles are facing the front of the vehicle. When the auxiliary collection vehicle travels forward, the main collection vehicle is stationary. The vehicle-side sensors of both vehicles simultaneously collect perception data packets of a preset duration each time the auxiliary collection vehicle stops at a preset distance interval. When the number of stops reaches K, the auxiliary collection vehicle continues to travel to a preset distance position and stops. At this time, the vehicle-side sensors of both vehicles simultaneously collect perception data packets of the preset duration. The method includes:
[0008] Obtaining K sets of vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K sets of primary perception data packets collected by the primary collection vehicle, wherein K is greater than a preset number of times, each set of primary perception data packets is composed of multiple frames of timestamped perception data, and each frame of timestamped perception data includes three-dimensional coordinate information of a perception vehicle, and the perception vehicles include at least the auxiliary collection vehicle;
[0009] A coordinate conversion matrix between the startup vehicle coordinate system of the primary collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle is calculated based on the K groups of ego-vehicle position information and the K groups of primary perception data packets, wherein the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer when it is powered on as the coordinate system origin;
[0010] The projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle is calculated based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, and the projection coordinate information is used as the true value of the three-dimensional coordinate of the auxiliary collection vehicle, wherein the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold.
[0011] Optionally, the step of calculating a coordinate conversion matrix between the start-up vehicle coordinate system of the main collection vehicle and the start-up vehicle coordinate system of the auxiliary collection vehicle based on the K groups of ego-vehicle position information and the K groups of main perception data packets includes:
[0012] The first group of main perception data packets in the K groups of main perception data packets is used as the current group of main perception data packets. The three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets and the first ego vehicle position information in the first group of ego vehicle position information are traversed according to the nearest neighbor algorithm to calculate a first relative distance error. The three-dimensional coordinate information of the perception vehicle corresponding to the minimum first relative distance error is determined as the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets. The first ego vehicle position information is determined as the ego vehicle position information of the auxiliary collection vehicle corresponding to the first frame timestamp in the current group of main perception data packets.
[0013] For each frame timestamp in the current group of primary perception data packets except the first frame timestamp, the three-dimensional coordinate information of the perception vehicle corresponding to the frame timestamp is calculated with the first three-dimensional coordinate information to obtain a second relative distance error, the three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error less than a preset distance error threshold is determined as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the primary collection vehicle at the frame timestamp, the first group of ego vehicle position information and the second three-dimensional coordinate information are traversed according to a nearest neighbor algorithm to calculate a third relative distance error, and the ego vehicle position information corresponding to the smallest third relative distance error is determined as the ego vehicle position information of the auxiliary collection vehicle corresponding to the frame timestamp;
[0014] The conversion matrix corresponding to the current group of primary perception data packets is calculated based on the three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at each frame timestamp, the position information of the auxiliary collection vehicle corresponding to each frame timestamp, and a preset general graph optimization formula, and the conversion matrix is used as the current conversion matrix;
[0015] The next group of main perception data packets of the current group of main perception data packets is used as the current group of main perception data packets, and the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets is calculated based on the next group of ego vehicle position information of the first group of ego vehicle position information and the current transformation matrix. The step of calculating the second relative distance error by calculating the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp except the first frame timestamp in the current group of main perception data packets and the first three-dimensional coordinate information is returned to execute, until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained, and the transformation matrix corresponding to the last group of main perception data packets is used as the coordinate transformation matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle.
[0016] Optionally, the step of calculating the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position includes:
[0017] The coordinate conversion matrix is calculated and multiplied on the left by the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position to obtain the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle.
[0018] Optionally, after the step of calculating the projection coordinate information of the auxiliary vehicle in the vehicle coordinate system of the main vehicle based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary vehicle at the preset distance, the method for determining the true value of the vehicle three-dimensional coordinates further includes:
[0019] Receive the two-dimensional annotation box of the auxiliary collection vehicle manually annotated in the target main perception data packet, perform two-dimensional-three-dimensional matching on the two-dimensional annotation box and the three-dimensional bounding box corresponding to the projection coordinate information, and when the match is successful, execute the step of using the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the target main perception data packet is the main perception data packet collected by the vehicle-side sensor of the main collection vehicle when the auxiliary collection vehicle continues to travel to a preset distance position and stops.
[0020] Optionally, the step of performing 2D-3D matching on the 2D annotation box and the 3D bounding box corresponding to the projection coordinate information includes:
[0021] Projecting the three-dimensional bounding box corresponding to the projection coordinate information onto the image plane where the two-dimensional annotation box is located to obtain a two-dimensional projected bounding box;
[0022] A first area of the intersection and a second area of the union between the two-dimensional annotation box and the two-dimensional projection bounding box are calculated, a quotient of the first area and the second area is calculated, and when the quotient is greater than a preset value, it is determined that the match is successful.
[0023] Optionally, after the step of using the projection coordinate information as the three-dimensional coordinate true value of the auxiliary data collection vehicle, the method for determining the three-dimensional coordinate true value of the vehicle further includes:
[0024] An adjustment to the vehicle height in the three-dimensional coordinate true value of the auxiliary data collection vehicle is received to obtain the adjusted three-dimensional coordinate true value of the auxiliary data collection vehicle.
[0025] Optionally, after the step of using the projection coordinate information as the three-dimensional coordinate true value of the auxiliary data collection vehicle, the method for determining the three-dimensional coordinate true value of the vehicle further includes:
[0026] The three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle is calculated based on the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-side sensor of the auxiliary collection vehicle at the preset distance position.
[0027] In a second aspect, an embodiment of the present invention provides a device for determining the true value of a vehicle's three-dimensional coordinates. In an initial state, a main collection vehicle and an auxiliary collection vehicle are both located at an initial point and the vehicle-side sensors of both vehicles are facing the front of the vehicle. When the auxiliary collection vehicle travels forward, the main collection vehicle is stationary. The vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset duration each time the auxiliary collection vehicle stops at a preset distance interval. When the number of stops reaches K, the auxiliary collection vehicle continues to travel to a preset distance position and stops. At this time, the vehicle-side sensors of the two vehicles simultaneously collect perception data packets of the preset duration. The device includes:
[0028] an acquisition module, configured to obtain K sets of vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K sets of primary perception data packets collected by the primary collection vehicle, wherein K is greater than a preset number of times, each set of primary perception data packets is composed of multiple frames of timestamped perception data, each frame of timestamped perception data includes three-dimensional coordinate information of a perception vehicle, and the perception vehicles include at least the auxiliary collection vehicle;
[0029] A coordinate conversion matrix determination module is configured to calculate a coordinate conversion matrix between the startup vehicle coordinate system of the primary acquisition vehicle and the startup vehicle coordinate system of the auxiliary acquisition vehicle based on the K sets of ego-vehicle position information and the K sets of primary perception data packets, wherein the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer at startup as the coordinate system origin;
[0030] The first coordinate true value determination module is used to calculate the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, and use the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold.
[0031] Optionally, the coordinate transformation matrix determination module includes:
[0032] a first calculation submodule, configured to use a first group of main perception data packets among the K groups of main perception data packets as a current group of main perception data packets, traverse the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets and the first ego vehicle position information in the first group of ego vehicle position information according to a nearest neighbor algorithm to calculate a first relative distance error, determine the three-dimensional coordinate information of the perception vehicle corresponding to the minimum first relative distance error as the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets, and determine the first ego vehicle position information as the ego vehicle position information of the auxiliary collection vehicle corresponding to the first frame timestamp in the current group of main perception data packets;
[0033] a second calculation submodule, configured to calculate, for each frame timestamp in the current group of primary perception data packets except the first frame timestamp, the three-dimensional coordinate information of the perception vehicle corresponding to the frame timestamp and the first three-dimensional coordinate information to obtain a second relative distance error, determine the three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error less than a preset distance error threshold as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the primary collection vehicle at the frame timestamp, traverse the first group of ego vehicle position information and the second three-dimensional coordinate information according to a nearest neighbor algorithm to obtain a third relative distance error, and determine the ego vehicle position information corresponding to the smallest third relative distance error as the ego vehicle position information of the auxiliary collection vehicle corresponding to the frame timestamp;
[0034] A current conversion matrix determination submodule is configured to calculate the conversion matrix corresponding to the current group of primary perception data packets based on the three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at each frame timestamp, the position information of the auxiliary collection vehicle corresponding to each frame timestamp, and a preset general graph optimization formula, and use the conversion matrix as the current conversion matrix;
[0035] A third calculation submodule is configured to use the next group of main perception data packets of the current group of main perception data packets as the current group of main perception data packets, calculate the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets based on the next group of ego vehicle position information of the first group of ego vehicle position information and the current transformation matrix, return to execute the step of calculating the second relative distance error by calculating the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp other than the first frame timestamp in the current group of main perception data packets with the first three-dimensional coordinate information, until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained, and use the transformation matrix corresponding to the last group of main perception data packets as the coordinate transformation matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle.
[0036] Optionally, the first coordinate true value determination module is specifically configured to:
[0037] The coordinate conversion matrix is calculated and multiplied on the left by the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position to obtain the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle.
[0038] Optionally, the above-mentioned vehicle three-dimensional coordinate true value determination device further includes:
[0039] The receiving module is configured to receive the two-dimensional annotation box of the auxiliary collection vehicle manually annotated in the target main perception data packet after calculating the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, perform two-dimensional-three-dimensional matching on the two-dimensional annotation box and the three-dimensional bounding box corresponding to the projection coordinate information, and when the match is successful, trigger the use of the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the target main perception data packet is the main perception data packet collected by the vehicle-side sensor of the main collection vehicle when the auxiliary collection vehicle continues to travel to the preset distance position and stops.
[0040] Optionally, the receiving module includes:
[0041] A projection submodule, configured to project the three-dimensional bounding box corresponding to the projection coordinate information onto the image plane where the two-dimensional annotation box is located to obtain a two-dimensional projection bounding box;
[0042] The matching submodule is used to calculate a first area of the intersection and a second area of the union between the two-dimensional annotation box and the two-dimensional projection bounding box, calculate a quotient of the first area and the second area, and determine that the match is successful when the quotient is greater than a preset value.
[0043] Optionally, the above-mentioned vehicle three-dimensional coordinate true value determination device further includes:
[0044] The adjustment module is configured to receive an adjustment to the vehicle height in the three-dimensional coordinate true value of the auxiliary collection vehicle after the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle, and obtain the adjusted three-dimensional coordinate true value of the auxiliary collection vehicle.
[0045] Optionally, the above-mentioned vehicle three-dimensional coordinate true value determination device further includes:
[0046] The second coordinate true value determination module is used to calculate the three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle based on the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-end sensor of the auxiliary collection vehicle at the preset distance position after the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle.
[0047] In a third aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a method for determining the true value of a vehicle's three-dimensional coordinates as provided in any embodiment of the present invention.
[0048] In a fourth aspect, an embodiment of the present invention provides an electronic device, comprising:
[0049] one or more processors;
[0050] The processor is coupled to a storage device, wherein the storage device is configured to store one or more programs;
[0051] When the one or more programs are executed by the one or more processors, the electronic device implements the method for determining the true value of the three-dimensional coordinates of a vehicle as provided in any embodiment of the present invention.
[0052] In a fifth aspect, an embodiment of the present invention provides a vehicle, which includes a device for determining the true value of a vehicle's three-dimensional coordinates as provided in any embodiment of the present invention, or includes an electronic device as provided in any embodiment of the present invention.
[0053] As can be seen from the foregoing, embodiments of the present invention provide a method, apparatus, device, medium, and vehicle for determining a true value of a vehicle's three-dimensional coordinates. The method can obtain K sets of ego-vehicle position information collected by an inertial sensor of an auxiliary vehicle during K stops and K sets of primary perception data packets collected by a primary vehicle, where K is greater than a preset number of times, and each set of primary perception data packets consists of multiple frames of timestamped perception data, each frame of timestamped perception data including three-dimensional coordinate information of a perception vehicle, which includes at least an auxiliary vehicle. A coordinate conversion matrix between a startup vehicle coordinate system of the primary vehicle and a startup vehicle coordinate system of the auxiliary vehicle is calculated based on the K sets of ego-vehicle position information and the K sets of primary perception data packets, where the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer at startup as the coordinate system origin. Projected coordinate information of the auxiliary vehicle in the vehicle coordinate system of the primary vehicle is calculated based on the coordinate conversion matrix and ego-vehicle position information collected by the inertial sensor of the auxiliary vehicle at a preset distance, and the projected coordinate information is used as the true value of the auxiliary vehicle's three-dimensional coordinates, where the distance between the preset distance position and the primary vehicle is greater than a preset distance threshold. In an embodiment of the present invention, the main and auxiliary collection vehicles cooperate with each other to calculate a coordinate transformation matrix between the main collection vehicle's start-up vehicle coordinate system and the auxiliary collection vehicle's start-up vehicle coordinate system based on K sets of ego vehicle position information and K sets of main perception data packets. The ego vehicle position information of the auxiliary collection vehicle is then projected into the vehicle coordinate system of the main collection vehicle based on the coordinate transformation matrix to obtain the auxiliary collection vehicle's true three-dimensional coordinate value. Because the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold, the obtained true three-dimensional coordinate value of the auxiliary collection vehicle is an ultra-long-distance true three-dimensional coordinate value, thereby enabling the main collection vehicle to perceive the true three-dimensional coordinate value of vehicles at ultra-long distances. Of course, implementing any product or method of the present invention does not necessarily require achieving all of the advantages described above simultaneously.
[0054] The innovative features of the embodiments of the present invention include:
[0055] 1. Through the cooperation between the main collection vehicle and the auxiliary collection vehicle, the coordinate conversion matrix between the main collection vehicle's starting vehicle coordinate system and the auxiliary collection vehicle's starting vehicle coordinate system is calculated based on K groups of ego vehicle position information and K groups of main perception data packets. Then, according to the coordinate conversion matrix, the ego vehicle position information of the auxiliary collection vehicle is projected into the vehicle coordinate system of the main collection vehicle to obtain the three-dimensional coordinate true value of the auxiliary collection vehicle. Since the distance between the preset distance position and the main collection vehicle is greater than the preset distance threshold, the obtained three-dimensional coordinate true value of the auxiliary collection vehicle is the three-dimensional coordinate true value of the ultra-long distance, realizing the three-dimensional coordinate true value perception of the ultra-long distance vehicle by the main collection vehicle.
[0056] 2. In the embodiment of the present invention, only through the mutual cooperation between the main collection vehicle and the auxiliary collection vehicle, the accurate three-dimensional coordinate true value of the vehicle at an ultra-long distance can be obtained without adding new sensors.
[0057] 3. Based on K groups of ego-vehicle position information, K groups of main perception data packets and the nearest neighbor algorithm, the coordinate transformation matrix between the starting vehicle coordinate system of the main collection vehicle and the starting vehicle coordinate system of the auxiliary collection vehicle is calculated.
[0058] 4. By receiving the adjustment of the vehicle height in the three-dimensional coordinate true value of the auxiliary collection vehicle, the accuracy of the adjusted three-dimensional coordinate true value of the auxiliary collection vehicle in the height direction is improved.
[0059] 5. By means of coordinate conversion, the three-dimensional coordinate information of each sensing vehicle in the auxiliary sensing data packet collected by the auxiliary collection vehicle at the preset distance position is converted to the vehicle coordinate system of the main collection vehicle, and the three-dimensional coordinate true value of the three-dimensional coordinate information of each sensing vehicle sensed by the auxiliary collection vehicle at an ultra-long distance in the vehicle coordinate system of the main collection vehicle is obtained. BRIEF DESCRIPTION OF THE DRAWINGS
[0060] To more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely some embodiments of the present invention. Those skilled in the art can derive other drawings based on these drawings without inventive effort.
[0061] FIG1 is a flow chart of a method for determining the true value of a vehicle's three-dimensional coordinates according to an embodiment of the present invention.
[0062] FIG2 is a schematic diagram of vehicle-side sensors of a primary data collection vehicle and a secondary data collection vehicle respectively collecting perception data packets according to an embodiment of the present invention;
[0063] FIG3 is a schematic structural diagram of a device for determining a true value of a vehicle's three-dimensional coordinates according to an embodiment of the present invention;
[0064] FIG4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention;
[0065] FIG5 is a schematic structural diagram of a vehicle provided by an embodiment of the present invention.
[0066] In Figure 2, 1 is the main collection vehicle, 2 is the auxiliary collection vehicle, 3 is other vehicles, and h is the preset distance interval. DETAILED DESCRIPTION
[0067] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0068] It should be noted that the terms "including" and "having" and any variations thereof in the embodiments of the present invention and the accompanying drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or apparatus.
[0069] The embodiments of the present invention disclose a method, apparatus, device, medium, and vehicle for determining the true value of a vehicle's three-dimensional coordinates, which can obtain the true value of a vehicle's three-dimensional coordinates at an extremely long distance. The embodiments of the present invention are described in detail below.
[0070] Figure 1 is a flow chart of a method for determining the true value of a vehicle's three-dimensional coordinates according to an embodiment of the present invention. The method is applied to electronic equipment.
[0071] Figure 2 is a schematic diagram of the vehicle-side sensors of the main collection vehicle 1 and the auxiliary collection vehicle 2 respectively collecting perception data packets provided by an embodiment of the present invention. Referring to Figure 2, in the initial state, the main collection vehicle 1 and the auxiliary collection vehicle 2 are both located at the initial point and the vehicle-side sensors of the two vehicles are facing the front of the vehicle, that is, the direction of the arrow in Figure 2. When the auxiliary collection vehicle 2 is moving forward, that is, moving in the direction of the arrow, the main collection vehicle 1 is stationary.
[0072] The vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset duration every time the auxiliary collection vehicle 2 stops at a preset distance interval h. When the number of stops reaches K, the auxiliary collection vehicle 2 continues to travel to point A at a preset distance and stops. At this time, the vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset duration. When the auxiliary collection vehicle 2 stops at point A, another vehicle 3 passes by point A, and the main collection vehicle 1 is not in P gear when it is stationary.
[0073] On this basis, the above method specifically includes the following steps.
[0074] S110: Obtain K groups of vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K groups of main perception data packets collected by the main collection vehicle, where K is greater than a preset number of times, each group of main perception data packets consists of perception data with multiple frames of timestamps, and each frame of perception data with a timestamp includes three-dimensional coordinate information of the perception vehicle, and the perception vehicle includes at least the auxiliary collection vehicle.
[0075] As auxiliary vehicle 2 drives, its inertial sensors collect real-time ego vehicle position information. Because auxiliary vehicle 2 stops every preset distance h, and its onboard sensors collect perception data packets for a preset duration, its inertial sensors also collect a set of ego vehicle position information corresponding to the preset duration during each stop. Therefore, K sets of ego vehicle position information are obtained for K stops.
[0076] Since the vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset duration every time the auxiliary collection vehicle 2 stops at a preset distance interval h, although the main collection vehicle 1 has been stationary, the main collection vehicle 1 also obtains K groups of main perception data packets during K stops, where each group of main perception data packets consists of perception data with multiple frames of timestamps, and the perception data with each timestamp includes the three-dimensional coordinate information of the perceived vehicle. K is greater than a preset number of times, illustratively, the preset number of times is 3.
[0077] Since the auxiliary collection vehicle 2 is moving forward and the main collection vehicle 1 is always stationary, the vehicle-side sensor of the main collection vehicle 1 can always sense the auxiliary collection vehicle 2. Therefore, the sensed vehicles sensed by the main collection vehicle 1 include at least the auxiliary collection vehicle.
[0078] In order to obtain the true value of the three-dimensional coordinates of the vehicle at an ultra-long distance, the electronic equipment obtains K groups of vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K groups of main perception data packets collected by the main collection vehicle.
[0079] The main collection vehicle 1 is a vehicle that needs to obtain the true value of the three-dimensional coordinates of the vehicle at an ultra-long distance, and the auxiliary collection vehicle 2 is a vehicle that assists the main collection vehicle 1 in obtaining the true value of the three-dimensional coordinates of the vehicle at an ultra-long distance.
[0080] S120: Calculate the coordinate conversion matrix between the powered-on vehicle coordinate system of the main collection vehicle and the powered-on vehicle coordinate system of the auxiliary collection vehicle based on K groups of ego-vehicle position information and K groups of main perception data packets, where the powered-on vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer when it is powered on as the coordinate system origin.
[0081] After obtaining K groups of ego vehicle position information and K groups of main perception data packets, the electronic device needs to calculate the coordinate conversion matrix between the power-on vehicle coordinate system of the main collection vehicle 1 and the power-on vehicle coordinate system of the auxiliary collection vehicle 2 based on the K groups of ego vehicle position information and the K groups of main perception data packets. Among them, the power-on vehicle coordinate system refers to the vehicle coordinate system with the position of the vehicle's on-board computer when it is turned on as the coordinate system origin.
[0082] Specifically, step S120 may include:
[0083] The first group of main perception data packets in the K groups of main perception data packets is used as the current group of main perception data packets. The three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets and the first ego vehicle position information in the first group of ego vehicle position information are traversed according to the nearest neighbor algorithm to calculate a first relative distance error. The three-dimensional coordinate information of the perception vehicle corresponding to the smallest first relative distance error is determined as the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets. The first ego vehicle position information is determined as the ego vehicle position information of the auxiliary collection vehicle corresponding to the first frame timestamp in the current group of main perception data packets.
[0084] For each frame timestamp except the first frame timestamp in the current group of main perception data packets, the three-dimensional coordinate information of the perception vehicle corresponding to the frame timestamp is calculated with the first three-dimensional coordinate information to obtain a second relative distance error. The three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error less than the preset distance error threshold is determined as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the frame timestamp. The first group of ego vehicle position information and the second three-dimensional coordinate information are traversed according to the nearest neighbor algorithm to calculate a third relative distance error. The ego vehicle position information corresponding to the smallest third relative distance error is determined as the ego vehicle position information of the auxiliary collection vehicle corresponding to the frame timestamp.
[0085] The conversion matrix corresponding to the current group of main perception data packets is calculated based on the 3D coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at each frame time stamp, the position information of the auxiliary collection vehicle corresponding to each frame time stamp, and the preset general graph optimization formula, and the conversion matrix is used as the current conversion matrix;
[0086] The next group of main perception data packets of the current group of main perception data packets is used as the current group of main perception data packets. The first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets is calculated based on the next group of ego vehicle position information of the first group of ego vehicle position information and the current transformation matrix. The step of returning to execute, for each frame timestamp except the first frame timestamp in the current group of main perception data packets, calculating the three-dimensional coordinate information of the perception vehicle corresponding to the frame timestamp and the first three-dimensional coordinate information to obtain a second relative distance error is performed until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained. The transformation matrix corresponding to the last group of main perception data packets is used as the coordinate transformation matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle.
[0087] When the main collection vehicle 1 collects the first group of main perception data packets, it is relatively close to the auxiliary collection vehicle 2. At this time, in order to obtain the three-dimensional coordinate information of the auxiliary collection vehicle 2 in the main collection vehicle 1, the first group of main perception data packets in the K groups of main perception data packets is used as the current group of main perception data packets. According to the nearest neighbor algorithm, the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets and the first ego vehicle position information in the first group of ego vehicle position information are traversed to obtain the first relative distance error. The three-dimensional coordinate information of the perception vehicle corresponding to the smallest first relative distance error is determined as the first three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1 at the first frame timestamp in the current group of main perception data packets. Among them, the first group of ego vehicle position information is the group of ego vehicle position information that ranks first in the K groups of ego vehicle position information.
[0088] That is to say, the three-dimensional coordinate information corresponding to the minimum first relative distance error is the three-dimensional coordinate information collected by the auxiliary collection vehicle 2 at the first frame timestamp in the main collection vehicle 1, and then the first ego vehicle position information is determined as the ego vehicle position information of the auxiliary collection vehicle 2 corresponding to the first frame timestamp in the current group of main perception data packets.
[0089] At present, only the three-dimensional coordinate information of the auxiliary collection vehicle 2 at the first frame timestamp in the first group of main perception data packets is obtained. In order to obtain the three-dimensional coordinate information of the auxiliary collection vehicle 2 at other timestamps, it is necessary to calculate the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp except the first frame timestamp in the current group of main perception data packets with the first three-dimensional coordinate information to obtain a second relative distance error, and determine the three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error which is less than the preset distance error threshold as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the frame timestamp. In this way, the three-dimensional coordinate information of the auxiliary collection vehicle 2 at other timestamps in the first group of main perception data packets is obtained. For example, the preset distance error threshold can be 1m.
[0090] Then, according to the nearest neighbor algorithm, the first set of ego vehicle position information and the second three-dimensional coordinate information are traversed to calculate the third relative distance error. The ego vehicle position information corresponding to the smallest third relative distance error is determined as the ego vehicle position information of the auxiliary collection vehicle 2 corresponding to the frame timestamp. Finally, the three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1 at each frame timestamp and the ego vehicle position information of the auxiliary collection vehicle 2 corresponding to each frame timestamp are obtained.
[0091] After obtaining the three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1 at each frame timestamp and the vehicle position information of the auxiliary collection vehicle 2 corresponding to each frame timestamp, the transformation matrix corresponding to the current group of main perception data packets is calculated based on the three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1 at each frame timestamp, the vehicle position information of the auxiliary collection vehicle 2 corresponding to each frame timestamp and the preset general graph optimization formula, and the transformation matrix is used as the current transformation matrix. For example, the current transformation matrix is a 4x4 transformation matrix.
[0092] In this way, the first set of main perception data packets is processed, and the current conversion matrix representing the conversion relationship between the three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1 and the vehicle position information of the auxiliary collection vehicle 2 is obtained.
[0093] The three-dimensional coordinate information of the auxiliary collection vehicle 2 in the vehicle coordinate system of the main collection vehicle 1 and the vehicle position information of the auxiliary collection vehicle 2 satisfy the following formula:
[0094] Among them, ξ * is the general graph optimization error, ξ is the Lie algebra corresponding to the special Euclidean group of se3, B is the vehicle coordinate system of the auxiliary collection vehicle, A is the vehicle coordinate system of the main collection vehicle, is the 4x4 transformation matrix from A to B, For point p bi The three-dimensional coordinate information in the vehicle coordinate system of the main collection vehicle is the three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1. argmin is the value of the number that satisfies the minimum value of the formula. For point p bi The three-dimensional coordinate information in the vehicle coordinate system of the auxiliary collection vehicle is the vehicle position information of vehicle B, i is the counting point, b is vehicle B, is the i-th position of car B, is a skew-symmetric matrix, exp is the exponential mapping, and ||||2 is the two-norm.
[0095] Next, the main perception data packets of the other groups except the first group of main perception data packets in the K groups are processed, and the next group of main perception data packets of the current group is used as the current group of main perception data packets. When the main collection vehicle 1 collects the second group of main perception data packets, it is far away from the auxiliary collection vehicle 2. Therefore, there is an error between the second group of ego vehicle position information and the three-dimensional coordinate information of the auxiliary collection vehicle 2 in the main collection vehicle 1. Therefore, it is necessary to use the current transformation matrix calculated above to substitute the second group of ego vehicle position information to obtain the three-dimensional coordinate information of the auxiliary collection vehicle 2 in the main collection vehicle 1 at this time.
[0096] Specifically, the first three-dimensional coordinate information of the auxiliary collection vehicle 2 collected by the main collection vehicle 1 at the first frame timestamp in the current group of main perception data packets is calculated based on the next group of ego vehicle position information of the first group of ego vehicle position information and the current transformation matrix, and the step of calculating the second relative distance error by performing calculations on the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp except the first frame timestamp in the current group of main perception data packets and the first three-dimensional coordinate information is returned until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained, and the transformation matrix corresponding to the last group of main perception data packets is used as the coordinate transformation matrix between the startup vehicle coordinate system of the main collection vehicle 1 and the startup vehicle coordinate system of the auxiliary collection vehicle 2.
[0097] Therefore, the coordinate transformation matrix between the starting vehicle coordinate system of the main collection vehicle and the starting vehicle coordinate system of the auxiliary collection vehicle is calculated based on K groups of ego vehicle position information, K groups of main perception data packets and the nearest neighbor algorithm.
[0098] S130: Calculate the projection coordinate information of the auxiliary vehicle in the vehicle coordinate system of the main vehicle based on the coordinate conversion matrix and the position information of the auxiliary vehicle collected by the inertial sensor of the auxiliary vehicle at a preset distance, and use the projection coordinate information as the true value of the three-dimensional coordinates of the auxiliary vehicle, wherein the distance between the preset distance position and the main vehicle is greater than a preset distance threshold.
[0099] Since the main collection vehicle 1 remains stationary when the auxiliary collection vehicle 2 moves forward, the vehicle coordinate system of the main collection vehicle 1 is the starting vehicle coordinate system. Therefore, the coordinate conversion matrix can be considered as the transformation matrix of the starting vehicle coordinate system of the auxiliary collection vehicle 2 relative to the starting vehicle coordinate system of the main collection vehicle 1. Therefore, at this time, the three-dimensional coordinate information of the auxiliary collection vehicle 2 in the main collection vehicle 1 at the preset distance position is also the coordinate information of the auxiliary collection vehicle 2 in the starting vehicle coordinate system of the main collection vehicle 1.
[0100] Therefore, after obtaining the coordinate conversion matrix, the projection coordinate information of the auxiliary collection vehicle 2 in the vehicle coordinate system of the main collection vehicle 1 can be calculated based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle 2 at the preset distance position A, and the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle 2, wherein the distance between the preset distance position and the main collection vehicle 1 is greater than the preset distance threshold. For example, the preset distance threshold is 300m.
[0101] Since the distance between the preset distance position and the main collection vehicle 1 is greater than the preset distance threshold, it means that the preset distance position is far away from the main collection vehicle 1. Therefore, the obtained three-dimensional coordinate true value of the auxiliary collection vehicle 2 is the three-dimensional coordinate true value of the ultra-long distance.
[0102] Specifically, the projection coordinate information of the auxiliary collection vehicle 2 in the vehicle coordinate system of the main collection vehicle 1 is calculated based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle 2 at a preset distance, which may include:
[0103] The coordinate conversion matrix is calculated and multiplied on the left by the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle 2 at the preset distance position to obtain the projection coordinate information of the auxiliary collection vehicle 2 in the vehicle coordinate system of the main collection vehicle 1.
[0104] As can be seen from the above content, the embodiment of the present invention can obtain K groups of ego vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K groups of main perception data packets collected by the main collection vehicle, where K is greater than a preset number of times, each group of main perception data packets is composed of multiple frames of time-stamped perception data, and each frame of time-stamped perception data includes three-dimensional coordinate information of the perception vehicle, and the perception vehicle includes at least the auxiliary collection vehicle; based on the K groups of ego vehicle position information and the K groups of main perception data packets, a coordinate conversion matrix between the power-on vehicle coordinate system of the main collection vehicle and the power-on vehicle coordinate system of the auxiliary collection vehicle is calculated, where the power-on vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's on-board computer when it is powered on as the coordinate system origin; based on the coordinate conversion matrix and the ego vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at a preset distance position, the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle is calculated, and the projection coordinate information is used as the true value of the three-dimensional coordinate of the auxiliary collection vehicle, where the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold. In the embodiment of the present invention, the main collection vehicle and the auxiliary collection vehicle cooperate with each other, and the coordinate conversion matrix between the main collection vehicle's startup vehicle coordinate system and the auxiliary collection vehicle's startup vehicle coordinate system is calculated based on K groups of ego vehicle position information and K groups of main perception data packets. Then, according to the coordinate conversion matrix, the ego vehicle position information of the auxiliary collection vehicle is projected into the vehicle coordinate system of the main collection vehicle to obtain the three-dimensional coordinate true value of the auxiliary collection vehicle. Since the distance between the preset distance position and the main collection vehicle 1 is greater than the preset distance threshold, the obtained three-dimensional coordinate true value of the auxiliary collection vehicle 2 is the three-dimensional coordinate true value of the ultra-long distance, thereby realizing the three-dimensional coordinate true value perception of the ultra-long distance vehicle of the main collection vehicle.
[0105] Furthermore, in the embodiment of the present invention, accurate three-dimensional coordinate true values of vehicles at ultra-long distances can be obtained simply by the mutual cooperation between the main data collection vehicle and the auxiliary data collection vehicle without adding additional sensors.
[0106] In one implementation, after step S130, the above-mentioned method for determining the true value of the vehicle three-dimensional coordinate may further include:
[0107] An adjustment to the vehicle height in the three-dimensional coordinate true value of the auxiliary collection vehicle is received, and the adjusted three-dimensional coordinate true value of the auxiliary collection vehicle is obtained.
[0108] Since the road surface is not completely flat, the obtained three-dimensional coordinate true value of the auxiliary collection vehicle 2 has a certain offset in the height direction, and the height needs to be adjusted manually. The electronic device receives the adjustment of the vehicle height in the three-dimensional coordinate true value of the auxiliary collection vehicle 2 and obtains the adjusted three-dimensional coordinate true value of the auxiliary collection vehicle 2.
[0109] Therefore, by receiving the adjustment of the vehicle height in the three-dimensional coordinate true value of the auxiliary data collection vehicle, the accuracy of the adjusted three-dimensional coordinate true value of the auxiliary data collection vehicle in the height direction is improved.
[0110] In another implementation, after step S130, the above-mentioned method for determining the true value of the vehicle three-dimensional coordinate may further include:
[0111] The three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle is calculated based on the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-side sensor of the auxiliary collection vehicle at a preset distance position.
[0112] The aforementioned use of the projected coordinate information as the true 3D coordinate value of auxiliary vehicle 2 only provides the true 3D coordinate value of auxiliary vehicle 2 as perceived by the primary vehicle 1 at an extremely long distance. Furthermore, the 3D coordinate information of each perceived vehicle in the auxiliary perception data packet collected by auxiliary vehicle 2 at a preset distance can be converted to the vehicle coordinate system of the primary vehicle 1. Specifically, the true 3D coordinate value of each perceived vehicle in the vehicle coordinate system of the primary vehicle can be calculated based on the projected coordinate information and the 3D coordinate information of each perceived vehicle in the auxiliary perception data packet collected by the auxiliary vehicle's onboard sensor at a preset distance.
[0113] Specifically, the three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle is calculated based on the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-side sensor of the auxiliary collection vehicle at a preset distance position, which may include:
[0114] The three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-side sensor of the auxiliary collection vehicle at a preset distance position is calculated and multiplied by the projection coordinate information to obtain the three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle.
[0115] Therefore, through coordinate conversion, the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the auxiliary collection vehicle 2 at the preset distance position is converted into the vehicle coordinate system of the main collection vehicle 1, and the three-dimensional coordinate true value of the three-dimensional coordinate information of each perception vehicle perceived by the auxiliary collection vehicle 2 at an ultra-long distance in the vehicle coordinate system of the main collection vehicle 1 is obtained.
[0116] In another implementation, after calculating the projection coordinate information of the auxiliary vehicle in the vehicle coordinate system of the main vehicle based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary vehicle at a preset distance, the above-mentioned method for determining the true value of the vehicle three-dimensional coordinates may further include:
[0117] Receive the two-dimensional annotation box of the auxiliary collection vehicle manually annotated in the target main perception data packet, perform two-dimensional-three-dimensional matching on the two-dimensional annotation box and the three-dimensional bounding box corresponding to the projection coordinate information, and when the match is successful, execute the step of using the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the target main perception data packet is the main perception data packet collected by the vehicle-side sensor of the main collection vehicle when the auxiliary collection vehicle continues to travel to a preset distance position and stops.
[0118] Since the obtained projection coordinate information may not be accurate, it is necessary to determine whether the projection coordinate information is accurate. Specifically, the two-dimensional annotation box of the auxiliary collection vehicle 2 manually marked in the target main perception data packet is received, and the two-dimensional annotation box is matched with the three-dimensional bounding box corresponding to the projection coordinate information. When the match is successful, it means that the obtained projection coordinate information is accurate. At this time, the step of using the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle is executed, wherein the target main perception data packet is the main perception data packet collected by the vehicle-side sensor of the main collection vehicle 1 when the auxiliary collection vehicle 2 continues to travel to the preset distance position A and stops.
[0119] Specifically, performing 2D-3D matching on the 2D annotation box and the 3D bounding box corresponding to the projection coordinate information may include:
[0120] Projecting the three-dimensional bounding box corresponding to the projection coordinate information onto the image plane where the two-dimensional annotation box is located to obtain a two-dimensional projected bounding box;
[0121] A first area of the intersection and a second area of the union between the two-dimensional annotation box and the two-dimensional projection bounding box are calculated, and a quotient of the first area and the second area is calculated. When the quotient is greater than a preset value, it is determined that the match is successful.
[0122] The method of performing 2D-3D matching between the 2D annotation box and the 3D bounding box corresponding to the projection coordinate information is to first reduce the 3D to 2D, and then match it with the 2D annotation box, that is, project the 3D bounding box corresponding to the projection coordinate information onto the image plane where the 2D annotation box is located to obtain a 2D projected bounding box, calculate the first area of the intersection between the 2D annotation box and the 2D projected bounding box, and the second area of the union, and calculate the quotient of the first area and the second area. When the quotient is greater than a preset value, it means that the two are relatively similar, and the match is determined to be successful.
[0123] Therefore, whether the two-dimensional annotation box matches the three-dimensional bounding box corresponding to the projection coordinate information is determined by a two-dimensional-three-dimensional matching method.
[0124] FIG3 is a schematic diagram of a structure of a device for determining the true value of a vehicle's three-dimensional coordinates provided by an embodiment of the present invention. Referring to FIG3 , in an initial state, the main collection vehicle and the auxiliary collection vehicle are both located at an initial point and the vehicle-side sensors of the two vehicles are both facing the front of the vehicle. When the auxiliary collection vehicle is traveling forward, the main collection vehicle is stationary. The vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset duration each time the auxiliary collection vehicle stops at a preset distance interval. When the number of stops reaches K, the auxiliary collection vehicle continues to travel to a preset distance position and stops. At this time, the vehicle-side sensors of the two vehicles simultaneously collect perception data packets of the preset duration. The device may include:
[0125] An acquisition module 310 is configured to obtain K sets of vehicle position information collected by the inertial sensor of the auxiliary acquisition vehicle during K stops and K sets of primary perception data packets collected by the primary acquisition vehicle, wherein K is greater than a preset number of times, each set of primary perception data packets is composed of multiple frames of timestamped perception data, and each frame of timestamped perception data includes three-dimensional coordinate information of a perception vehicle, and the perception vehicles include at least the auxiliary acquisition vehicle;
[0126] A coordinate conversion matrix determination module 320 is configured to calculate a coordinate conversion matrix between the startup vehicle coordinate system of the primary acquisition vehicle and the startup vehicle coordinate system of the auxiliary acquisition vehicle based on the K sets of ego-vehicle position information and the K sets of primary perception data packets, wherein the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer at startup as the coordinate system origin;
[0127] The first coordinate true value determination module 330 is configured to calculate, based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle, and use the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold.
[0128] The device provided by the embodiment of the present invention can obtain K groups of ego vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K groups of main perception data packets collected by the main collection vehicle, where K is greater than a preset number of times, and each group of main perception data packets is composed of multiple frames of time-stamped perception data, and each frame of time-stamped perception data includes three-dimensional coordinate information of the perception vehicle, and the perception vehicle includes at least the auxiliary collection vehicle; based on the K groups of ego vehicle position information and the K groups of main perception data packets, a coordinate conversion matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle is calculated, where the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's on-board computer when it is powered on as the coordinate system origin; based on the coordinate conversion matrix and the ego vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at a preset distance position, the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle is calculated, and the projection coordinate information is used as the true value of the three-dimensional coordinate of the auxiliary collection vehicle, where the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold. In the embodiment of the present invention, the main collection vehicle and the auxiliary collection vehicle cooperate with each other, and the coordinate conversion matrix between the main collection vehicle's startup vehicle coordinate system and the auxiliary collection vehicle's startup vehicle coordinate system is calculated based on K groups of ego vehicle position information and K groups of main perception data packets. Then, according to the coordinate conversion matrix, the ego vehicle position information of the auxiliary collection vehicle is projected into the vehicle coordinate system of the main collection vehicle to obtain the three-dimensional coordinate true value of the auxiliary collection vehicle. Since the distance between the preset distance position and the main collection vehicle is greater than the preset distance threshold, the obtained three-dimensional coordinate true value of the auxiliary collection vehicle is an ultra-long-distance three-dimensional coordinate true value, thereby realizing the three-dimensional coordinate true value perception of the ultra-long-distance vehicle by the main collection vehicle.
[0129] In one implementation, the coordinate transformation matrix determination module 320 may include:
[0130] a first calculation submodule, configured to use a first group of main perception data packets among the K groups of main perception data packets as a current group of main perception data packets, traverse the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets and the first ego vehicle position information in the first group of ego vehicle position information according to a nearest neighbor algorithm to calculate a first relative distance error, determine the three-dimensional coordinate information of the perception vehicle corresponding to the minimum first relative distance error as the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets, and determine the first ego vehicle position information as the ego vehicle position information of the auxiliary collection vehicle corresponding to the first frame timestamp in the current group of main perception data packets;
[0131] a second calculation submodule, configured to calculate, for each frame timestamp in the current group of primary perception data packets except the first frame timestamp, the three-dimensional coordinate information of the perception vehicle corresponding to the frame timestamp and the first three-dimensional coordinate information to obtain a second relative distance error, determine the three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error less than a preset distance error threshold as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the primary collection vehicle at the frame timestamp, traverse the first group of ego vehicle position information and the second three-dimensional coordinate information according to a nearest neighbor algorithm to obtain a third relative distance error, and determine the ego vehicle position information corresponding to the smallest third relative distance error as the ego vehicle position information of the auxiliary collection vehicle corresponding to the frame timestamp;
[0132] A current conversion matrix determination submodule is configured to calculate the conversion matrix corresponding to the current group of primary perception data packets based on the three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at each frame timestamp, the position information of the auxiliary collection vehicle corresponding to each frame timestamp, and a preset general graph optimization formula, and use the conversion matrix as the current conversion matrix;
[0133] A third calculation submodule is configured to use the next group of main perception data packets of the current group of main perception data packets as the current group of main perception data packets, calculate the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets based on the next group of ego vehicle position information of the first group of ego vehicle position information and the current transformation matrix, return to execute the step of calculating the second relative distance error by calculating the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp other than the first frame timestamp in the current group of main perception data packets with the first three-dimensional coordinate information, until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained, and use the transformation matrix corresponding to the last group of main perception data packets as the coordinate transformation matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle.
[0134] Optionally, the first coordinate true value determining module 330 may be specifically configured to:
[0135] The coordinate conversion matrix is calculated and multiplied on the left by the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position to obtain the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle.
[0136] Optionally, the above-mentioned vehicle three-dimensional coordinate true value determination device further includes:
[0137] The receiving module is configured to receive the two-dimensional annotation box of the auxiliary collection vehicle manually annotated in the target main perception data packet after calculating the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, perform two-dimensional-three-dimensional matching on the two-dimensional annotation box and the three-dimensional bounding box corresponding to the projection coordinate information, and when the match is successful, trigger the use of the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the target main perception data packet is the main perception data packet collected by the vehicle-side sensor of the main collection vehicle when the auxiliary collection vehicle continues to travel to the preset distance position and stops.
[0138] Optionally, the receiving module may include:
[0139] A projection submodule, configured to project the three-dimensional bounding box corresponding to the projection coordinate information onto the image plane where the two-dimensional annotation box is located to obtain a two-dimensional projection bounding box;
[0140] The matching submodule is used to calculate a first area of the intersection and a second area of the union between the two-dimensional annotation box and the two-dimensional projection bounding box, calculate a quotient of the first area and the second area, and determine that the match is successful when the quotient is greater than a preset value.
[0141] Optionally, the above-mentioned vehicle three-dimensional coordinate true value determination device may further include:
[0142] The adjustment module is configured to receive an adjustment to the vehicle height in the three-dimensional coordinate true value of the auxiliary collection vehicle after the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle, and obtain the adjusted three-dimensional coordinate true value of the auxiliary collection vehicle.
[0143] Optionally, the above-mentioned vehicle three-dimensional coordinate true value determination device may further include:
[0144] The second coordinate true value determination module is used to calculate the three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle based on the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-end sensor of the auxiliary collection vehicle at the preset distance position after the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle.
[0145] The above device embodiment corresponds to the method embodiment and has the same technical effects as the method embodiment. For detailed description, please refer to the method embodiment. The device embodiment is obtained based on the method embodiment. For detailed description, please refer to the method embodiment part and will not be repeated here.
[0146] FIG4 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. As shown in FIG4 , the electronic device includes:
[0147] one or more processors 410;
[0148] The processor 410 is coupled to a storage device 420 , and the storage device 420 is used to store one or more programs;
[0149] When the one or more programs are executed by the one or more processors 410 , the electronic device implements the method for determining the true value of the three-dimensional coordinates of a vehicle as provided in any embodiment of the present invention.
[0150] Based on the above embodiments, another embodiment of the present invention provides a vehicle, which includes the vehicle three-dimensional coordinate true value determination device provided by any embodiment of the present invention, or includes the electronic device provided by any embodiment of the present invention.
[0151] Figure 5 is a schematic diagram of a vehicle provided by an embodiment of the present invention. As shown in Figure 5, the vehicle includes a speed sensor 51, an ECU (Electronic Control Unit) 52, a GPS (Global Positioning System) positioning device 53, and a T-Box (Telematics Box) 54. The speed sensor 51 is used to measure vehicle speed and use it as an empirical speed for model training; the GPS positioning device 53 is used to obtain the vehicle's current geographic location; the T-Box 54 can act as a gateway to communicate with a server; and the ECU 52 can execute the above-described method for determining the true value of the vehicle's three-dimensional coordinates.
[0152] In addition, the vehicle may also include a V2X (Vehicle-to-Everything) module 55, a radar 56, and a camera 57. The V2X module 55 is used to communicate with other vehicles and roadside equipment. The radar 56 or camera 57 is used to sense road environment information ahead and / or in other directions to obtain raw point cloud data. The radar 56 and / or camera 57 can be located at the front and / or rear of the vehicle.
[0153] Based on the above method embodiments, another embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for determining the true value of the three-dimensional coordinates of a vehicle as provided in any embodiment of the present invention.
[0154] Those skilled in the art will appreciate that the accompanying drawings are merely schematic diagrams of an embodiment, and the modules or processes in the accompanying drawings are not necessarily required to implement the present invention.
[0155] Those skilled in the art will appreciate that the modules in the apparatuses of the embodiments may be distributed in the apparatuses of the embodiments as described in the embodiments, or may be located in one or more apparatuses different from the embodiments with corresponding changes. The modules in the above embodiments may be combined into one module or further divided into multiple sub-modules.
[0156] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for determining the true value of a vehicle's three-dimensional coordinates, characterized in that: In the initial state, the main collection vehicle and the auxiliary collection vehicle are both located at the initial point and the vehicle-side sensors of the two vehicles are facing the front of the vehicle. When the auxiliary collection vehicle moves forward, the main collection vehicle is stationary. The vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset time length when the auxiliary collection vehicle stops at a preset distance interval. When the number of stops reaches K times, the auxiliary collection vehicle continues to travel to a preset distance position and stops. At this time, the vehicle-side sensors of the two vehicles simultaneously collect perception data packets of the preset time length. The method includes: Obtain K groups of vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K groups of main perception data packets collected by the main collection vehicle, wherein K is greater than a preset number of times, each group of main perception data packets is composed of perception data with multiple frames of timestamps, and each frame of perception data with timestamps includes three-dimensional coordinate information of the perception vehicle, and the perception vehicle at least includes the auxiliary collection vehicle; A coordinate conversion matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle is calculated based on the K groups of vehicle position information and the K groups of main perception data packets, wherein the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer when it is started as the origin of the coordinate system; The projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle is calculated based on the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, and the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold.
2. The method according to claim 1, characterized in that The step of calculating the coordinate conversion matrix between the start-up vehicle coordinate system of the main collection vehicle and the start-up vehicle coordinate system of the auxiliary collection vehicle according to the K groups of ego-vehicle position information and the K groups of main perception data packets comprises: The first group of main perception data packets in the K groups of main perception data packets is used as the current group of main perception data packets. The three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets is traversed according to the nearest neighbor algorithm to calculate the first relative distance error with the first ego vehicle position information in the first group of ego vehicle position information. The three-dimensional coordinate information of the perception vehicle corresponding to the smallest first relative distance error is determined as the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets. The first ego vehicle position information is determined as the ego vehicle position information of the auxiliary collection vehicle corresponding to the first frame timestamp in the current group of main perception data packets. Car location information; For each frame timestamp in the current group of main perception data packets except the first frame timestamp, the three-dimensional coordinate information of the perception vehicle corresponding to the frame timestamp is calculated with the first three-dimensional coordinate information to obtain a second relative distance error, the three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error less than a preset distance error threshold is determined as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the frame timestamp, the first group of ego vehicle position information and the second three-dimensional coordinate information are traversed according to the nearest neighbor algorithm to calculate a third relative distance error, and the ego vehicle position information corresponding to the smallest third relative distance error is determined as the ego vehicle position information of the auxiliary collection vehicle corresponding to the frame timestamp; The conversion matrix corresponding to the current group of main perception data packets is calculated according to the three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at each frame timestamp, the position information of the auxiliary collection vehicle corresponding to each frame timestamp, and a preset general graph optimization formula, and the conversion matrix is used as the current conversion matrix; The next group of main perception data packets of the current group of main perception data packets is used as the current group of main perception data packets, and the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets is calculated according to the next group of ego-vehicle position information of the first group of ego-vehicle position information and the current transformation matrix, and the step of calculating the second relative distance error by performing the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp except the first frame timestamp in the current group of main perception data packets and the first three-dimensional coordinate information is returned to be executed, until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained, and the transformation matrix corresponding to the last group of main perception data packets is used as the coordinate transformation matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle.
3. The method according to claim 1 or 2, characterized in that The step of calculating the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle according to the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position comprises: The coordinate conversion matrix is calculated and multiplied on the left by the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position to obtain the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle.
4. The method according to claim 1, characterized in that After the step of calculating the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle according to the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, the method further includes: Receive the two-dimensional annotation box of the auxiliary collection vehicle manually annotated in the target main perception data packet, perform two-dimensional-three-dimensional matching between the two-dimensional annotation box and the three-dimensional boundary box corresponding to the projection coordinate information, and when the match is successful, execute the step of using the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the target main perception data packet is the main perception data packet collected by the vehicle-side sensor of the main collection vehicle when the auxiliary collection vehicle continues to travel to a preset distance position and stops.
5. The method according to claim 4, characterized in that The step of performing two-dimensional-three-dimensional matching between the two-dimensional annotation box and the three-dimensional bounding box corresponding to the projection coordinate information comprises: Projecting the three-dimensional bounding box corresponding to the projection coordinate information onto the image plane where the two-dimensional annotation box is located to obtain a two-dimensional projection bounding box; A first area of the intersection and a second area of the union between the two-dimensional annotation box and the two-dimensional projection boundary box are calculated, and a quotient of the first area and the second area is calculated. When the quotient is greater than a preset value, it is determined that the match is successful.
6. The method according to claim 1, characterized in that After the step of using the projection coordinate information as the true value of the three-dimensional coordinates of the auxiliary data collection vehicle, the method further includes: An adjustment made to the vehicle height in the three-dimensional coordinate true value of the auxiliary data collection vehicle is received to obtain the adjusted three-dimensional coordinate true value of the auxiliary data collection vehicle.
7. The method according to claim 1, characterized in that After the step of using the projection coordinate information as the true value of the three-dimensional coordinates of the auxiliary data collection vehicle, the method further includes: The three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle is calculated based on the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-side sensor of the auxiliary collection vehicle at the preset distance position.
8. A device for determining the true value of a vehicle's three-dimensional coordinates, characterized in that: In the initial state, the main collection vehicle and the auxiliary collection vehicle are both located at the initial point and the vehicle-side sensors of the two vehicles are facing the front of the vehicle. When the auxiliary collection vehicle moves forward, the main collection vehicle is stationary. The vehicle-side sensors of the two vehicles simultaneously collect perception data packets of a preset time length when the auxiliary collection vehicle stops at a preset distance interval. When the number of stops reaches K times, the auxiliary collection vehicle continues to travel to a preset distance position and stops. At this time, the vehicle-side sensors of the two vehicles simultaneously collect perception data packets of the preset time length. The device includes: An acquisition module, used to obtain K groups of vehicle position information collected by the inertial sensor of the auxiliary collection vehicle during K stops and K groups of main perception data packets collected by the main collection vehicle, wherein K is greater than a preset number of times, each group of main perception data packets is composed of perception data of multiple frames of timestamps, and each frame of perception data of timestamps includes three-dimensional coordinate information of the perception vehicle, and the perception vehicle at least includes the auxiliary collection vehicle; A coordinate conversion matrix determination module is used to calculate the coordinate conversion matrix between the startup vehicle coordinate system of the main collection vehicle and the startup vehicle coordinate system of the auxiliary collection vehicle according to the K groups of ego vehicle position information and the K groups of main perception data packets, wherein the startup vehicle coordinate system refers to a vehicle coordinate system with the position of the vehicle's onboard computer when it is started as the origin of the coordinate system; The first coordinate true value determination module is used to calculate the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle according to the coordinate conversion matrix and the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, and use the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle, wherein the distance between the preset distance position and the main collection vehicle is greater than a preset distance threshold.
9. The device according to claim 8, characterized in that The coordinate transformation matrix determination module comprises: A first calculation submodule is used to use the first group of main perception data packets in the K groups of main perception data packets as the current group of main perception data packets, traverse the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp in the current group of main perception data packets according to the nearest neighbor algorithm and calculate the first self-vehicle position information in the first group of self-vehicle position information to obtain a first relative distance error, determine the three-dimensional coordinate information of the perception vehicle corresponding to the minimum first relative distance error as the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets, and determine the first self-vehicle position information as the self-vehicle position information of the auxiliary collection vehicle corresponding to the first frame timestamp in the current group of main perception data packets; The second calculation submodule is used to calculate the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp except the first frame timestamp in the current group of main perception data packets, and the first three-dimensional coordinate information to obtain a second relative distance error, and determine the three-dimensional coordinate information of the perception vehicle corresponding to the second relative distance error less than the preset distance error threshold as the second three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the frame timestamp, and calculate the second three-dimensional coordinate information of the auxiliary collection vehicle according to the nearest neighbor calculation. The method traverses the first set of ego vehicle position information and the second three-dimensional coordinate information to calculate and obtain a third relative distance error, and determines the ego vehicle position information corresponding to the smallest third relative distance error as the ego vehicle position information of the auxiliary collection vehicle corresponding to the frame timestamp; The current conversion matrix determination submodule is used to calculate the conversion matrix corresponding to the current group of main perception data packets according to the three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at each frame timestamp, the vehicle position information of the auxiliary collection vehicle corresponding to each frame timestamp and a preset general graph optimization formula, and use the conversion matrix as the current conversion matrix; The third calculation submodule is used to use the next group of main perception data packets of the current group of main perception data packets as the current group of main perception data packets, calculate the first three-dimensional coordinate information of the auxiliary collection vehicle collected by the main collection vehicle at the first frame timestamp in the current group of main perception data packets according to the next group of ego vehicle position information of the first group of ego vehicle position information and the current transformation matrix, return to execute the step of calculating the second relative distance error between the three-dimensional coordinate information of the perception vehicle corresponding to each frame timestamp except the first frame timestamp in the current group of main perception data packets and the first three-dimensional coordinate information, until the transformation matrix corresponding to the last group of main perception data packets in the K groups of main perception data packets is obtained, and the transformation matrix corresponding to the last group of main perception data packets is used as the coordinate transformation matrix between the start-up vehicle coordinate system of the main collection vehicle and the start-up vehicle coordinate system of the auxiliary collection vehicle.
10. The device according to claim 8 or 9, characterized in that The first coordinate true value determination module is specifically used for: The coordinate conversion matrix is calculated and multiplied on the left by the vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position to obtain the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle.
11. The device according to claim 8, characterized in that The device also includes: The receiving module is used for receiving the two-dimensional annotation frame of the auxiliary collection vehicle manually annotated in the target main perception data packet after calculating the projection coordinate information of the auxiliary collection vehicle in the vehicle coordinate system of the main collection vehicle according to the coordinate conversion matrix and the self-vehicle position information collected by the inertial sensor of the auxiliary collection vehicle at the preset distance position, performing two-dimensional-three-dimensional matching between the two-dimensional annotation frame and the three-dimensional boundary frame corresponding to the projection coordinate information, and triggering the use of the projection coordinate information as the three-dimensional coordinate true value of the auxiliary collection vehicle when the matching is successful, wherein the target main perception data packet is the vehicle-side sensor of the main collection vehicle when the auxiliary collection vehicle continues to drive to the preset distance The main perception data packet collected when the vehicle stops at a certain position.
12. The device according to claim 11, characterized in that The receiving module comprises: A projection submodule, configured to project a three-dimensional bounding box corresponding to the projection coordinate information onto an image plane where the two-dimensional annotation box is located to obtain a two-dimensional projection bounding box; The matching submodule is used to calculate the first area of the intersection and the second area of the union between the two-dimensional annotation box and the two-dimensional projection boundary box, calculate the quotient of the first area and the second area, and determine that the match is successful when the quotient is greater than a preset value.
13. The device according to claim 8, characterized in that The device also includes: The adjustment module is used to receive the adjustment of the vehicle height in the three-dimensional coordinate true value of the auxiliary collection vehicle after the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle, so as to obtain the adjusted three-dimensional coordinate true value of the auxiliary collection vehicle.
14. The device according to claim 1, characterized in that The device also includes: The second coordinate true value determination module is used to calculate the three-dimensional coordinate true value of each perception vehicle in the vehicle coordinate system of the main collection vehicle according to the projection coordinate information and the three-dimensional coordinate information of each perception vehicle in the auxiliary perception data packet collected by the vehicle-end sensor of the auxiliary collection vehicle at the preset distance position after the projection coordinate information is used as the three-dimensional coordinate true value of the auxiliary collection vehicle.
15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.
16. An electronic device, characterized in that: The electronic device comprises: one or more processors; The processor is coupled to a storage device, and the storage device is used to store one or more programs; When the one or more programs are executed by the one or more processors, the electronic device implements the method as described in any one of claims 1 to 7.
17. A vehicle, characterized in that: The vehicle comprises the apparatus according to any one of claims 8 to 14, or comprises the electronic device according to claim 16.
Citation Information
Patent Citations
Method and device for determining coordinate system conversion parameters of image sensor and medium
CN110135376A
Lane line coordinate true value obtaining method and device based on unmanned vehicle
CN112633035A
Vehicle-road cooperation three-dimensional target detection method and system based on post-fusion
CN114627442A
Automated driving-based vehicle speed control method and apparatus, and computer device
WO2021134441A1
