Road end sensor calibration method and road end equipment

Abstract

The embodiment of the invention provides a road end sensor calibration method and road end equipment, the method is executed based on a road end calculation unit, and the method comprises the steps that first position information of a vehicle is acquired, the vehicle is located in a first range, and the first position information corresponds to a first coordinate system; acquiring sensing data of a first range acquired by a road end sensor; second position information of the vehicle is determined based on the sensing data, and the second position information corresponds to a second coordinate system; and determining a conversion relation between the first coordinate system and the second coordinate system based on the first position information and the second position information.

Description

Technical Field

[0001] This specification relates to the technical field of traffic information collection and analysis, and particularly relates to a road-side sensor calibration method and a road-side device. Background Art

[0002] At present, vehicle-road cooperation has become one of the basic pillars in the fields of intelligent transportation, smart city, etc. The vehicle-road cooperation system can achieve effective cooperation among people, vehicles, and roads through real-time information interaction between vehicles and between vehicles and roads. After the installation of each road-side sensor in the vehicle-road cooperation system, external parameter calibration is required. During the use process, in case of damage and replacement, vibration, and insecure installation, etc., parameter calibration work needs to be carried out again. Therefore, in order to avoid perception failure caused by untimely update of parameters, resulting in decision-making errors and potential safety hazards, it is necessary to update the external parameters in a timely manner to ensure the accuracy of the external parameters. Summary of the Invention

[0003] One or more embodiments of this specification provide a road-side sensor calibration method. The method is executed based on a road-side computing unit and includes: obtaining first position information of a vehicle, where the vehicle is located in a first range, and the first position information corresponds to a first coordinate system; obtaining perception data of the first range collected by a road-side sensor; determining second position information of the vehicle based on the perception data, where the second position information corresponds to a second coordinate system; and determining a conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0004] One or more embodiments of this specification provide a road-side sensor online calibration method. The method is executed based on a vehicle and includes: obtaining first data within a second range collected by a vehicle-end sensor at a current time, where the first data corresponds to a first coordinate system; obtaining second data within a third range collected by a road-side sensor at the current time, where the second data corresponds to a second coordinate system, and there is an overlapping area between the second range and the third range; and converting the second data into third data based on the first data, where the third data corresponds to the first coordinate system.

[0005] One or more embodiments of this specification provide a roadside device. The roadside device includes a roadside computing unit, and the roadside computing unit includes an information determination module, a perception module, a position determination module, and a calibration module, where: The information determination module is configured to obtain first position information of a vehicle, the vehicle is located in a first range, and the first position information corresponds to a first coordinate system; The perception module is configured to obtain perception data of the first range collected by a roadside sensor; The position determination module is configured to determine second position information of the vehicle based on the perception data, and the second position information corresponds to a second coordinate system; The calibration module is configured to determine a conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0006] One or more embodiments of this specification provide a roadside device. The roadside device includes a roadside computing unit, and the roadside computing unit includes at least one memory and at least one processor. The at least one memory is configured to store computer instructions, and the at least one processor executes the computer instructions or part of the instructions to implement the roadside sensor calibration method described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] This specification will be further described by way of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. These embodiments are not restrictive. In these embodiments, the same numbers represent the same structures, where:

[0008] Figure 1 is a schematic diagram of an application scenario of a roadside sensor calibration method shown in some embodiments of this specification;

[0009] Figure 2 is an exemplary block diagram of a roadside computing unit shown in some embodiments of this specification;

[0010] Figure 3 is an exemplary flowchart of a roadside sensor calibration method shown in some embodiments of this specification;

[0011] Figure 4 is an exemplary schematic diagram of determining a conversion relationship shown in some embodiments of this specification;

[0012] Figure 5 is an exemplary flowchart of an online calibration method for a roadside sensor shown in some embodiments of this specification;

[0013] Figure 6 is an exemplary flowchart of converting second data into third data shown in some embodiments of this specification. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0014] To more clearly illustrate the technical solutions of the embodiments of this specification, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, this specification can also be applied to other similar scenarios based on these drawings. Unless obvious from the language context or otherwise stated, the same reference numerals in the figures represent the same structure or operation.

[0015] It should be understood that the "system", "unit" and / or "module" used herein is a way to distinguish different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the said words can be replaced by other expressions.

[0016] As shown in this specification and the claims, unless the context clearly indicates an exception, words such as "a", "an", "one" and / or "the" are not specifically singular and may also include plural. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0017] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the previous or subsequent operations are not necessarily executed precisely in sequence. On the contrary, they can be executed in reverse order or simultaneously. At the same time, other operations can also be added to these processes, or one or more steps can be removed from these processes.

[0018] The extrinsic parameter calibration in vehicle-road cooperation is essentially a process of fusing the data collected by multi-source sensors. In the current calibration methods, one is to register the point cloud data of the high-precision map with the point cloud data to be calibrated. However, due to problems such as imperfect map coverage and unupdated versions, it is difficult to ensure the accuracy of the extrinsic parameter calibration; the other is to use a calibration device to mark points to collect coordinates and extract the feature points of the calibration device on the point cloud data to be calibrated for relative coordinate calculation. However, this method has a high degree of manual participation and low calibration efficiency. Therefore, it is hoped to provide a road-end sensor calibration method and a road-end device that can directly complete data fusion at the road end to automatically realize extrinsic parameter calibration and update, which can reduce data transfer and storage, ensure data security and improve calibration efficiency at the same time.

[0019] Figure 1It is a schematic diagram of the application scenario of the roadside sensor calibration method shown in some embodiments of this specification. The roadside sensor calibration method and roadside device in this specification can be applied to scenarios of mobile control, such as, for example, the autonomous driving scenario. In some embodiments, the roadside device can implement the external parameter calibration of the roadside sensor by implementing the methods and / or processes disclosed in this specification, such as, for example, determining the conversion relationship between the first coordinate system and the second coordinate system, etc.

[0020] In some embodiments, as Figure 1 shown, the application scenario 100 may include a network 110, a vehicle 120, a roadside device 130, and a storage device 140.

[0021] The network 110 can facilitate the exchange of data and / or information. In some embodiments, one or more components of the application scenario 100 (such as, for example, the vehicle 120, the roadside device 130, the storage device 140) can send information and / or data to other components through the network 110. For example, the roadside device 130 can obtain basic safety messages from the vehicle 120 via the network 110. In some embodiments, the network 110 can be any one or more of a wired network or a wireless network. For example, the network 110 can include a fiber optic network, a wireless network. Among them, the wireless network can include the Internet, a vehicle-to-everything network (such as, for example, LTE-V, NR-V2X, etc.) or any combination thereof. In some embodiments, the network connection between the components of the application scenario 100 can be in one of the above ways or can be in multiple ways. In some embodiments, the network can be various topological structures such as point-to-point, shared, centralized, etc. or a combination of multiple topological structures.

[0022] Vehicle 120 refers to a mobile device that can be equipped with a vehicle-road collaborative system and travels on the road. In some embodiments, vehicle 120 may include taxis, private cars, carpooling vehicles, shared vehicles, etc. In some embodiments, vehicle 120 may include a vehicle-end computing unit, vehicle-end sensors, communication components (such as a GPS communication module, a short-range wireless communication module, etc.), and the like. Among them, the vehicle-end computing unit refers to a computer device installed in the vehicle. For example, an in-vehicle computer, an in-vehicle computer, etc. In some embodiments, the vehicle-end computing unit may include a processor. The processor may include one or more sub-processors (such as a single-core processing device or a multi-core multi-chip processing device). By way of example only, the processor may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), an application-specific instruction processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a programmable logic circuit (PLD), a controller, a microprocessor, etc. or any combination thereof. The vehicle-end sensors refer to sensing devices installed in the vehicle. For example, the vehicle-end sensors may include motion sensors, pressure sensors, speed sensors, crankshaft position sensors, vehicle-mounted lidar sensors, vehicle-mounted cameras, GPS positioning sensors, etc. In some embodiments, the vehicle-end sensors may collect relevant data of the vehicle in real time. For example, the GPS positioning sensor may obtain the positioning information of the vehicle.

[0023] The roadside device 130 refers to a device installed at the roadside or transportation hub. For example, a Road Side Unit (RSU). The roadside device can be used to communicate with vehicles and provide data exchange and information transfer between vehicles. In some embodiments, the roadside device 130 may include a roadside computing unit, roadside sensors, communication components, etc. The roadside computing unit refers to a computer device configured on roadside facilities. For example, a Road Side Computing Unit (RSCU), etc. In some embodiments, the roadside computing unit may include a processor. The processor can perform real-time processing and analysis on the received data. For example, the roadside computing unit can receive the basic safety message sent by the vehicle; parse the basic safety message to obtain the first position information of the vehicle. In some embodiments, the roadside computing unit can execute the roadside sensor calibration method. For example, the roadside computing unit can obtain the first position information of the vehicle and the perception data within the first range collected by the roadside sensors. For another example, the roadside computing unit can determine the second position information of the vehicle based on the perception data. For another example, the roadside computing unit can determine the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information. The roadside sensors refer to sensing devices fixedly installed at the roadside or transportation hub. For example, the roadside sensors may include lidar sensors, camera sensors, etc. In some embodiments, the roadside sensors can collect the perception data within the first range of the vehicle in real time. For more information about the first range and the perception data, reference can be made to Figure 3 and its related descriptions.

[0024] The storage device 140 can be used to store data, instructions, and / or any other information. For example, the storage device can store perception data, first position information, second position information, etc. In some embodiments, the storage device 140 may include a mass storage device, a removable storage device, a volatile read-write memory (e.g., random access memory RAM), a read-only memory (ROM), etc., or any combination of the above. In some embodiments, the storage device 140 can be integrated or included in one or more other components (e.g., the vehicle 120, the roadside device 130) in the application scenario 100.

[0025] Figure 2 is a schematic diagram of the modules of the roadside computing unit shown in some embodiments of this specification.

[0026] In some embodiments, the roadside device may include a roadside computing unit.

[0027] In some embodiments, as Figure 2 shown, the roadside computing unit 200 may include an information determination module 210, a perception module 220, a position determination module 230, and a calibration module 240.

[0028] The information determination module 210 refers to a module for determining the first position information of a vehicle. In some embodiments, the information determination module 210 may be used to obtain the first position information of the vehicle, where the vehicle is located in a first range, and the first position information corresponds to a first coordinate system.

[0029] The perception module 220 refers to a module for obtaining perception data. In some embodiments, the perception module 220 may be used to obtain the perception data of the first range collected by roadside sensors.

[0030] The position determination module 230 refers to a module for determining the second position information of the vehicle. In some embodiments, the position determination module 230 may be used to determine the second position information of the vehicle based on the perception data, and the second position information corresponds to a second coordinate system.

[0031] The calibration module 240 refers to a module for determining a conversion relationship. In some embodiments, the calibration module 240 may be used to determine the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0032] For more information about the information determination module 210, the perception module 220, the position determination module 230, and the calibration module 240, see Figures 3 - 6 and its related descriptions.

[0033] It should be noted that the above description of the roadside computing unit and its modules is only for convenience of description and does not limit this specification to the scope of the examples given. It can be understood that for those skilled in the art, after understanding the principle of the system, they may, without departing from this principle, make any combination of the modules, or form a subsystem and connect it with other modules. In some embodiments, Figure 2 the information determination module 210, the perception module 220, the position determination module 230, and the calibration module 240 disclosed in

[0034] Figure 3 is an exemplary flowchart of a roadside sensor calibration method according to some embodiments of this specification. As Figure 3 shown, process 300 includes the following steps. In some embodiments, process 300 may be executed by a roadside computing unit.

[0035] Step 310, obtain the first position information of the vehicle.

[0036] In some embodiments, the vehicle is located within a first range. For more information about the vehicle, reference can be made to Figure 1 and its related description.

[0037] The first range refers to the range of the detection distance that the roadside sensor can sense. In some embodiments, different roadside sensors may correspond to different first ranges. For example, for a lidar sensor, the first range may be a range of 200 m from the lidar sensor. Another example is that for a camera sensor, the first range may be a range of 150 m from the camera sensor.

[0038] The first position information refers to the information that can reflect the position of the vehicle in the first coordinate system. In some embodiments, the first position information corresponds to the first coordinate system. For example, the first position information may be represented by coordinates in the first coordinate system with a timestamp.

[0039] The first coordinate system refers to the coordinate system used to represent the position of the vehicle in the basic safety message. For example, the first coordinate system may be a three-dimensional coordinate system with an arbitrary point as the origin. For more information about the basic safety message, reference can be made to the related description below.

[0040] In some embodiments, the first coordinate system may include a world coordinate system.

[0041] The world coordinate system refers to an absolute coordinate system that is fixed after being specified. In some embodiments, the world coordinate system may be a three-dimensional coordinate system and / or a planar coordinate system. For example, the world coordinate system may be an existing coordinate system such as the Universal Transverse Mercator (UTM) coordinate system, the geocentric inertial coordinate system, etc., or a fixed coordinate system constructed with an arbitrary point in space as the origin and three mutually perpendicular directions as the X-axis, Y-axis, and Z-axis, or other coordinate systems pre-constructed by humans or systems.

[0042] In some embodiments, the roadside computing unit may receive the basic safety message (BSM) sent by the vehicle; parse the basic safety message to obtain the first position information of the vehicle.

[0043] The basic safety message refers to the basic information related to the vehicle state. For example, the basic safety message may include various information such as the vehicle position, the time corresponding to the vehicle location, and the vehicle speed.

[0044] In some embodiments, the vehicle-mounted sensors may directly obtain the basic safety message of the vehicle. For example, the GPS positioning sensor may obtain the real-time position and corresponding time of the vehicle. Another example is that the speed sensor may obtain the real-time speed of the vehicle, etc.

[0045] In some embodiments, the vehicle-end sensor may transmit a basic safety message to the vehicle-end computing unit, and the vehicle-end computing unit may send the basic safety message outward through a network (such as, near-field communication, etc.) and receive it by the roadside computing unit. For more information about the vehicle-end sensor and the vehicle-end computing unit, reference can be made to Figure 1 and its related description.

[0046] In some embodiments, the roadside computing unit may parse the received basic safety message to obtain the first position information of the vehicle. For example, the roadside computing unit may read and parse the basic safety message through a decoder, and determine the first position information according to the vehicle position and the corresponding time. For example, the GPS positioning sensor of the vehicle may obtain the initial position information of the vehicle in the coordinate system of the positioning sensor, such as, the position information in the World Geodical System-84 (WGS-84). The roadside computing unit may use WGS-84 as the first coordinate system and the initial position information as the first position information. Alternatively, the roadside computing unit may convert the initial position information from WGS-84 to the UTM coordinate system to obtain the first position information.

[0047] In some embodiments of the present specification, the vehicle sends a basic safety message to the roadside device through a network. The roadside computing unit can automatically determine that the vehicle has entered the sensing range of the roadside device by receiving the basic safety message sent by the vehicle, and can accurately obtain the first position information of the vehicle in real time by parsing the basic safety message, so as to determine the conversion relationship between coordinates subsequently, without manual intervention throughout the process, improving the calibration efficiency of the roadside device.

[0048] Step 320, obtain the sensing data of the first range collected by the roadside sensor.

[0049] The sensing data refers to the relevant information of road elements obtained by the roadside sensor. In some embodiments, the sensing data may include the position information, time information, image data, etc. of the road elements. Among them, the road elements refer to the information related to the road. For example, the road elements may include road surface elements (such as, lane lines, turning signs, isolation fences, road surface vehicles, etc.), roadside elements (such as, traffic lights, street lights, signs, etc.). In some embodiments, the sensing data may be collected in real time by the roadside sensor. For example, the lidar sensor may obtain the position information, time information, structural information, etc. of the road elements in real time. Alternatively, the camera sensor may obtain the image data of the road elements in real time.

[0050] In some embodiments, the sensing data may be transmitted in real time by the roadside sensor to the roadside computing unit.

[0051] Step 330: Determine the second position information of the vehicle based on the perception data.

[0052] The second position information refers to the information that can reflect the position of the vehicle in the second coordinate system. In some embodiments, the second position information can be represented by the coordinates in the second coordinate system with timestamps.

[0053] The second coordinate system refers to the coordinate system used to determine the position of the vehicle based on the perception data. For example, the second coordinate system can be a three-dimensional coordinate system or a planar coordinate system with any point as the origin.

[0054] In some embodiments, the second coordinate system can include a roadside coordinate system.

[0055] The roadside coordinate system refers to the coordinate system established based on roadside sensors. In some embodiments, the roadside coordinate system can be a three-dimensional coordinate system and / or a planar coordinate system. For example, the roadside coordinate system can be existing coordinate systems such as a lidar coordinate system, a camera coordinate system, etc., or a coordinate system constructed with the roadside sensor as the origin O and any three mutually perpendicular directions as the X-axis, Y-axis, and Z-axis, or other coordinate systems pre-constructed by humans or systems.

[0056] In some embodiments of this specification, using the world coordinate system as the first coordinate system and the roadside coordinate system as the second coordinate system is beneficial for quickly determining the conversion relationship between the first coordinate system and the second coordinate system subsequently.

[0057] In some embodiments, the roadside computing unit can determine the second position information of the vehicle based on the perception data through various methods. For example, the roadside computing unit can identify the vehicle corresponding to the basic safety message received by the roadside computing unit based on the image data and / or structural information in the perception data through a graph matching algorithm (such as a neural network matching algorithm, etc.), and determine the second position information of the vehicle according to the position information in the perception data.

[0058] Step 340: Determine the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0059] The conversion relationship refers to the corresponding relationship in the transformation process between the first coordinate system and the second coordinate system. For example, the conversion relationship can be represented by a transformation matrix that transforms the coordinates of a certain point in the second coordinate system to the coordinates in the first coordinate system.

[0060] In some embodiments, the roadside computing unit may determine the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information through various methods. For example, the roadside computing unit may randomly select the first position information of the vehicle at any N time points and the second position information at these N time points, and determine the conversion relationship between the first coordinate system and the second coordinate system through a conversion model (such as, the Bursa model, etc.).

[0061] In some embodiments, the roadside computing unit may respectively determine the first trajectory and the second trajectory of the vehicle based on the first position information and the second position information, then construct multiple matching groups by matching the first trajectory points and the second trajectory points, and further determine the conversion relationship according to the first position coordinates of the first trajectory points and the second position coordinates of the second trajectory points in the matching groups. For more content on this part, reference can be made to Figure 4 the relevant description.

[0062] In some embodiments, the roadside computing unit may determine the fourth position information of surrounding objects based on the perception data; and determine the conversion relationship between the first coordinate system and the second coordinate system based on the third position information and the fourth position information. For more content on this part, reference can be made to the relevant description below.

[0063] In some embodiments of this specification, perception data is obtained through roadside sensors, and then the second position information is determined. Based on the first position information and the second position information, the roadside computing unit directly processes and converts various modalities of data received in real time to determine the conversion relationship between the first coordinate system and the second coordinate system. This can reduce the storage record of data, ensure data security while improving the efficiency of information interaction, and can efficiently achieve vehicle-road integrated perception without manual intervention, which is beneficial to the construction and operation of intelligent transportation systems and improves traffic safety and efficiency.

[0064] Figure 4 is an exemplary schematic diagram of determining the conversion relationship shown according to some embodiments of this specification.

[0065] In some embodiments, the roadside computing unit may determine the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information, including: determining, based on the first position information 410, the first trajectory 420 of the vehicle, where the first trajectory 420 includes a plurality of first trajectory points 421, and each first trajectory point 421 corresponds to a first time point 421-1 and a first position coordinate 421-2; determining, based on the second position information 430, the second trajectory 440 of the vehicle, where the second trajectory 440 includes a plurality of second trajectory points 441, and each second trajectory point 441 corresponds to a second time point 441-1 and a second position coordinate 441-2; matching the first trajectory points 421 and the second trajectory points 441 based on the first time point 421-1 and the second time point 441-1 to construct a plurality of matching groups 450; and determining the conversion relationship 460 based on the first position coordinate 421-2 of the first trajectory point 421 and the second position coordinate 441-2 of the second trajectory point 441 in the matching group 450.

[0066] For more information about relevant parameters such as the vehicle, the first position information, the second position information, the first coordinate system, the second coordinate system, and the conversion relationship, reference can be made to Figure 3 the relevant description.

[0067] The first trajectory 420 refers to the trajectory formed by the positions of the vehicle at multiple time points in the first coordinate system. In some embodiments, the first trajectory may include a plurality of first trajectory points. The first trajectory may be represented by a sequence including N first trajectory points. For example, (P1, P2,..., P n ). The value of N can be preset manually or set by default in the system (e.g., N = 100).

[0068] The first trajectory point 421 refers to the position point of the vehicle in the first coordinate system. In some embodiments, each first trajectory point corresponds to a first time point and a first position coordinate. For example, the first time point corresponding to the first trajectory point P n can be represented as T n , and the first position coordinate can be represented as (X n , Y n , Z n ).

[0069] The first time point 421-1 refers to the formation time of the first trajectory point 421, that is, the time point corresponding to when the vehicle is at the first position coordinate 421-2. The first position coordinate 421-2 refers to the coordinate of the vehicle position in the first coordinate system.

[0070] In some embodiments, the roadside computing unit may determine the first trajectory of the vehicle based on the first position information through various methods. For example, when the amount of data of the first position information is small, the roadside computing unit may generate the first trajectory of the vehicle based on the first position information through methods such as Gaussian fitting. For another example, when the amount of data of the first position information is sufficient, the roadside computing unit may randomly select the position coordinates of N time points with the same interval from the first position information to form a sequence, which can be the first trajectory of the vehicle.

[0071] The second trajectory 440 refers to the trajectory formed by the positions of the vehicle at multiple time points in the second coordinate system. In some embodiments, the second trajectory may include multiple second trajectory points. The second trajectory may be represented by a sequence including N second trajectory points. For example, (P * 1, P * 2,..., P* n ). The value of N can be preset manually or set by default in the system (e.g., N = 100).

[0072] The second trajectory point 441 refers to the position point of the vehicle in the second coordinate system. In some embodiments, each second trajectory point corresponds to a second time point and a second position coordinate. For example, the second time corresponding to the second trajectory point P* n can be represented as T* n , and the second position coordinate can be represented as (X* n , Y* n , Z* n ).

[0073] The second time point 441-1 refers to the formation time of the second trajectory point 441, that is, the time point corresponding to when the vehicle is at the second position coordinate 441-2. The second position coordinate 441-2 refers to the coordinate of the vehicle position in the second coordinate system.

[0074] In some embodiments, the roadside computing unit may use the same method for determining the first trajectory of the vehicle to determine the second trajectory of the vehicle based on the second position information.

[0075] In some embodiments, the roadside computing unit may select multiple first time points and second time points at the same time point or with a time interval less than a preset threshold (e.g., 1 ms, etc.), and match the corresponding first trajectory points and second trajectory points to construct multiple matching groups. For example, the first time T n and the second time T* n are the same time point, and the corresponding first trajectory point P n and the second trajectory point P* n are matched to construct a matching group (P n , P* n ).

[0076] In some embodiments, the roadside computing unit may determine the conversion relationship in various ways based on the first position coordinates of the first trajectory point and the second position coordinates of the second trajectory point in the matching group. For example, the roadside computing unit may analyze the corresponding first position coordinates and second position coordinates in the matching group through Singular Value Decomposition (SVD) to determine the conversion relationship between the first coordinate system and the second coordinate system.

[0077] In some embodiments of this specification, two trajectories of the vehicle are determined according to two types of position information, and then by matching the corresponding trajectory points in the two trajectories, multiple matching groups can be constructed. Furthermore, based on the position coordinates in the matching groups, the conversion relationship between the first and second coordinate systems can be determined quickly and accurately.

[0078] In some embodiments, the roadside computing unit may also obtain the third position information of the surrounding objects of the vehicle.

[0079] The surrounding objects refer to other objects around the vehicle. In some embodiments, the surrounding objects of the vehicle may include road elements around the vehicle. For example, other vehicles, traffic lights, pedestrians, road signs, flower beds, buildings, etc. around the vehicle. In some embodiments, the surrounding objects are located within the first range. For more information about road elements and the first range, reference can be made to Figure 3 and its related descriptions.

[0080] The third position information refers to the information that can reflect the position of the surrounding objects in the first coordinate system. In some embodiments, the third position information corresponds to the first coordinate system. For example, the third position information may be represented by coordinates in the first coordinate system with a time stamp. For more information about the first coordinate system, reference can be made to Figure 3 the relevant description.

[0081] In some embodiments, the roadside computing unit may obtain the third position information of the surrounding objects of the vehicle through various methods. For example, the roadside computing unit may analyze the received basic safety messages of other surrounding vehicles to obtain the third position information of other vehicles. For another example, the vehicle may determine the real-time distance and azimuth information of the surrounding objects (such as moving objects like surrounding vehicles and pedestrians) relative to the vehicle itself through vehicle-mounted sensors (such as vehicle-mounted lidar sensors, vehicle-mounted cameras, etc.). After the vehicle obtains the first position information, it can estimate the positions of the surrounding objects based on the corresponding distance and azimuth information to obtain the third position information. For yet another example, the vehicle may obtain the position information of the surrounding objects (such as static objects like roads, flower beds, buildings, traffic lights, road signs, etc.) according to the map, and after converting it to the first coordinate system, obtain the third position information.

[0082] The vehicle can encode the third position information of surrounding objects in the basic safety message and send it to the roadside device. The roadside computing unit parses the basic safety message to obtain the third position information of surrounding objects (such as pedestrians). Among them, the basic safety message is transmitted from the vehicle sensor and / or the vehicle computing unit to the roadside computing unit through the network. For more information about the vehicle sensor and the vehicle computing unit, please refer to Figure 1 the relevant description.

[0083] In some embodiments of the present specification, by obtaining the third position information of the surrounding objects of the vehicle, the roadside device can be calibrated based on more data, improving the calibration efficiency and accuracy.

[0084] In some embodiments, the roadside computing unit determining the conversion relationship between the first coordinate system and the second coordinate system may further include: determining the fourth position information of the surrounding objects based on the perception data; determining the conversion relationship between the first coordinate system and the second coordinate system based on the third position information and the fourth position information.

[0085] For more information about the perception data, please refer to Figure 3 and its related description.

[0086] The fourth position information refers to the information that can reflect the position of the surrounding objects in the second coordinate system. In some embodiments, the fourth position information corresponds to the second coordinate system. For example, the fourth position information can be represented by the coordinates in the second coordinate system with a timestamp. For more information about the second coordinate system, please refer to Figure 3 the relevant description.

[0087] In some embodiments, the roadside computing unit can determine the fourth position information of the surrounding objects of the vehicle based on the perception data through various methods. For example, the roadside computing unit can identify the surrounding objects corresponding to the basic safety message and / or the positioning information received by the roadside computing unit through a graph matching algorithm based on the image data and / or the structure information in the perception data, and determine the fourth position information of the surrounding objects according to the position information in the perception data.

[0088] In some embodiments, the roadside computing unit can determine the conversion relationship between the first coordinate system and the second coordinate system based on the third position information and the fourth position information. For example, the roadside computing unit can randomly select the third position information of the surrounding objects at any N time points and the fourth position information at these N time points, and determine the conversion relationship through a conversion model. The method of determining the conversion relationship between the first coordinate system and the second coordinate system based on the third position information and the fourth position information is similar to the method of determining the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

[0089] In some embodiments of the present specification, fourth position information of surrounding objects is determined based on perception data; based on the third and fourth position information of the objects surrounding the vehicle, the conversion relationship between the first and second coordinate systems can be determined by referring to the surrounding objects from another angle, making the determination method of the conversion relationship more diverse and the determination efficiency more efficient.

[0090] Figure 5 is an exemplary flowchart of an on-road sensor online calibration method shown in some embodiments of the present specification. As Figure 5 shown, process 500 includes the following steps. In some embodiments, process 500 can be executed by a vehicle.

[0091] In some embodiments, the data collected by the on-road sensors can be sent to the vehicle for assisting the vehicle in moving control. The data collected by the vehicle-mounted sensors and the on-road sensors respectively correspond to different coordinate systems (for example, corresponding to the first coordinate system and the second coordinate system respectively). Therefore, it is necessary to unify them into the same coordinate system (for example, the first coordinate system), that is, it is necessary to calibrate the installed on-road equipment. By determining the original conversion relationship between the second coordinate system and the first coordinate system, the on-road equipment can convert the data collected by the on-road sensors from the second coordinate system to the first coordinate system, and then send the converted data to the vehicle. And when the original conversion relationship is unavailable (for example, during the period when the pose of the on-road sensors changes and the original conversion relationship has not been updated yet), the vehicle can convert the data sent by the on-road equipment online according to process 500 and process 600.

[0092] In some embodiments, a vehicle may include vehicle-mounted sensors and a vehicle-mounted computing unit. For more information about the vehicle, vehicle-mounted sensors, and vehicle-mounted computing unit, reference can be made to Figure 1 the relevant description of vehicle 120 therein. In some embodiments, the vehicle-mounted computing unit of the vehicle can execute the on-road sensor online calibration method.

[0093] Step 510, obtain first data within a second range collected by the vehicle-mounted sensors at the current time.

[0094] The current time refers to the current time point.

[0095] The second range refers to the range of detection distance that the vehicle-mounted sensors can sense and detect. In some embodiments, different vehicle-mounted sensors can correspond to different second ranges. For example, for an on-vehicle lidar sensor, the second range can be a range of 130 m from the on-vehicle lidar sensor. For another example, for an on-vehicle camera, the second range can be a range of 100 m from the on-vehicle camera.

[0096] The first data refers to information that can reflect the positions of surrounding objects of the vehicle in the first coordinate system. In some embodiments, the first data corresponds to the first coordinate system. For example, the first data can be represented by coordinates in the first coordinate system with timestamps. For more information about the surrounding objects and the first coordinate system, reference can be made to the relevant descriptions above.

[0097] In some embodiments, the first coordinate system may include a world coordinate system or a vehicle body coordinate system. For more information about the world coordinate system, reference can be made to Figure 3 its relevant description.

[0098] The vehicle body coordinate system refers to a coordinate system established based on the vehicle itself. For example, the vehicle body coordinate system can be a right-handed Cartesian coordinate system constructed with the centroid or center of gravity of the vehicle as the origin O, the X-axis pointing to the right of the vehicle body, the Y-axis pointing to the front of the vehicle body, and the Z-axis perpendicular to the XOY plane and upward, or a coordinate system constructed by other methods.

[0099] In some embodiments, the vehicle can obtain, through vehicle-end sensors, the first data of multiple surrounding objects within a second range collected at the current time. For example, the vehicle can determine, through vehicle-end sensors (such as on-vehicle lidar, on-vehicle cameras, etc.), the real-time distance and azimuth information of surrounding objects (such as moving objects like surrounding vehicles and pedestrians) relative to the host vehicle. After the vehicle obtains its real-time position, it can estimate the real-time positions of the surrounding objects based on the corresponding distance and azimuth information, and convert them to the first coordinate system to obtain the first data. For another example, the vehicle can obtain the position information of surrounding objects (such as static objects like roads, flower beds, buildings, traffic lights, road signs, etc.) according to a map, and after converting them to the first coordinate system, obtain the first data.

[0100] Step 520: Obtain second data within a third range collected by roadside sensors at the current time.

[0101] The third range refers to the range of the detection distance that the roadside sensor can sense and detect. In some embodiments, the third range may refer to the first range. For more information about the first range, reference can be made to Figure 3 its relevant description. In some embodiments, the detection distance of the roadside sensor is greater than that of the vehicle-end sensor, that is, the third range is greater than the second range.

[0102] In some embodiments, there is an overlapping area between the second range and the third range. The overlapping area refers to the area that both the roadside sensor and the vehicle-end sensor can sense and detect.

[0103] The second data refers to information that can reflect the positions of surrounding objects of the vehicle in the second coordinate system. In some embodiments, the second data may correspond to coordinates in the second coordinate system. For example, the second data may be represented by coordinates in the second coordinate system with timestamps. For more information about the second coordinate system, refer to Figure 3 and its related descriptions.

[0104] In some embodiments, the second coordinate system may include a roadside coordinate system. For more information about the roadside coordinate system, refer to Figure 3 the related descriptions.

[0105] In some embodiments, the roadside sensor may collect the second data of each surrounding object within the third range in real time at the current time.

[0106] Step 530, based on the first data, convert the second data into third data.

[0107] The third data refers to the data obtained by corresponding the second data to the first coordinate system.

[0108] In some embodiments, the vehicle may convert the second data into third data in various ways based on the first data. For example, the vehicle may convert the second data into third data based on the first data through coordinate conversion methods (such as, grid method, multi-parameter method, conversion model, etc.).

[0109] In some embodiments, the vehicle may match the first data and the second data to determine the first coincidence data and the second coincidence data; based on the first coincidence data and the second coincidence data, determine the conversion relationship between the first coordinate system and the second coordinate system; based on the conversion relationship, convert the second data to the first coordinate system to determine the third data. For more information about this part, refer to Figure 6 the related descriptions.

[0110] In some embodiments of the present specification, by obtaining the first data within the second range and the second data within the third range collected at the current time, and directly converting the second data into third data by the vehicle based on the first data, the data transfer record can be reduced, and when the original conversion relationship becomes unavailable (for example, during the period when the pose of the roadside sensor changes and the original conversion relationship has not been updated yet), the vehicle can directly determine a new conversion relationship online and automatically to achieve external parameter calibration, enabling efficient vehicle-road fusion perception without manual intervention, which is beneficial to ensuring the safety and efficiency of the intelligent transportation system.

[0111] Figure 6 is an exemplary flowchart for converting the second data into third data according to some embodiments of the present specification. As Figure 6As shown, process 600 includes the following steps. In some embodiments, process 600 may be executed by a vehicle.

[0112] Step 610: Match the first data and the second data to determine the first overlapping data in the overlapping region corresponding to the first data and the second overlapping data in the overlapping region corresponding to the second data.

[0113] For the description of the overlapping region, reference may be made to Figure 5 the corresponding description.

[0114] The first overlapping data refers to the first data located in the overlapping region. The second overlapping data refers to the second data located in the overlapping region.

[0115] In some embodiments, the vehicle may determine the first overlapping data and the second overlapping data based on various methods. For example, the vehicle may match the first data and the second data of the same surrounding object in the overlapping region of the second range and the third range according to the object detection algorithm. The first data and the second data of one surrounding object are constructed as a combination, corresponding to a set of first overlapping data and second overlapping data. Match and combine the first data and the corresponding second data of all surrounding objects in the overlapping region to determine multiple sets of first overlapping data and second overlapping data. Among them, the object detection algorithm may include but is not limited to the YOLO (You Only Look Once) algorithm, etc.

[0116] Step 620: Based on the first overlapping data and the second overlapping data, determine the conversion relationship between the first coordinate system and the second coordinate system.

[0117] In some embodiments, the vehicle may determine the conversion relationship between the first coordinate system and the second coordinate system based on the first overlapping data and the second overlapping data in various ways. For example, the vehicle may perform singular value decomposition on multiple sets of first overlapping data and second overlapping data to determine the pose transformation matrix between the first coordinate system and the second coordinate system, and thus determine the conversion relationship.

[0118] Step 630: Based on the conversion relationship, convert the second data to the first coordinate system to determine the third data.

[0119] In some embodiments, the vehicle may convert the second data to the first coordinate system based on the conversion relationship in various ways to determine the third data. For example, the vehicle may convert the second data to the first coordinate system based on the pose transformation matrix to determine the third data.

[0120] In some embodiments of the present specification, by matching the first data and the second data, the first coincidence data and the second coincidence data can be determined. Based on the first coincidence data and the second coincidence data, the conversion relationship between the first coordinate system and the second coordinate system can be accurately determined. Furthermore, based on the conversion relationship, the second data can be quickly converted into the third data to achieve real-time fusion of multi-source sensor data and quickly complete the online calibration of roadside devices, which is beneficial to realizing the update of external parameters without perception to ensure traffic safety and efficiency.

[0121] Since the roadside sensor collects data frame by frame in real time (for example, when the lidar rotates one circle, one frame of data is collected) and synchronously sends each frame of data to the vehicle online, the process 500 calibrates the frame-by-frame data of the roadside sensor online.

[0122] In some embodiments, if the first coordinate system is the world coordinate system, the conversion relationship between the first coordinate system and the second coordinate system is fixed. Therefore, a relatively stable conversion relationship can be determined based on the previous few frames of data (for example, when the difference between the conversion relationships determined at the current moment and the adjacent past moment is less than a preset threshold, the conversion relationship corresponding to the current moment can be determined as the conversion relationship between the first coordinate system and the second coordinate system). After the current moment, the vehicle can directly apply the determined conversion relationship to convert the data collected by the roadside sensor without having to execute the process 500. In some embodiments, the conversion relationship determined at the current moment and the conversion relationship determined at the adjacent past moment can also be integrated (for example, taking the average, etc.) to determine the final conversion relationship.

[0123] As an example, after the vehicle enters the sensing range of the roadside device, it will successively receive the first frame of data, the second frame of data,..., the Nth frame of data sent by the roadside device. For the first M frames of data, the vehicle can execute the process 500 and the process 600 online to obtain the conversion relationship 1, the conversion relationship 2,..., the conversion relationship M (M < N) respectively. Among them, for the tth (M ≥ t > 1) frame of data, the vehicle can first determine the conversion relationship t' according to step 620, and then determine the average value based on at least one of the conversion relationships 1 to t - 1 and t' to determine the conversion relationship t. For the (M + 1)th frame of data to the Nth frame of data, the vehicle can directly use the conversion relationship M to convert the (M + 1)th frame of data to the Nth frame of data from the second coordinate system to the first coordinate system (world coordinate system) without having to execute the process 500 and the process 600.

[0124] One or more embodiments of this specification provide a roadside device, which includes a roadside computing unit. The roadside computing unit includes at least one memory and at least one processor. The at least one memory is used to store computer instructions, and the at least one processor executes the computer instructions or part of the instructions to implement the above-mentioned roadside sensor calibration method.

[0125] One or more embodiments of this specification provide an online calibration system for roadside devices, which can be implemented on the computing unit of a vehicle. The online calibration system for roadside devices includes: a first acquisition module, configured to acquire first data within a second range collected by a vehicle-end sensor at the current time; a second acquisition module, configured to acquire second data within a third range collected by a roadside sensor at the current time; and a conversion module, configured to convert the second data into third data based on the first data.

[0126] One or more embodiments of this specification provide a vehicle-end device, which includes at least one memory and at least one processor. The at least one memory is used to store computer instructions, and the at least one processor executes the computer instructions or part of the instructions to implement the online calibration method for roadside devices.

[0127] It should be noted that the above descriptions of processes 300, 500, and 600 are only for illustration and explanation, and do not limit the scope of application of this specification. Those skilled in the art can make various corrections and changes to processes 300, 500, and 600 under the guidance of this specification. However, these corrections and changes are still within the scope of this specification.

[0128] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are proposed in this specification, so such modifications, improvements, and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

[0129] At the same time, this specification uses specific terms to describe the embodiments of this specification. Such as "one embodiment", "an embodiment", and / or "some embodiments" mean a certain feature, structure, or characteristic related to at least one embodiment of this specification. Therefore, it should be emphasized and noted that "an embodiment" or "one embodiment" or "an alternative embodiment" mentioned twice or more at different positions in this specification does not necessarily refer to the same embodiment. In addition, certain features, structures, or characteristics in one or more embodiments of this specification can be appropriately combined.

[0130] In addition, unless explicitly stated in the claims, the order of the processing elements and sequences, the use of numerical and alphabetical characters, or the use of other names described in this specification are not used to limit the order of the processes and methods in this specification. Although some currently useful embodiments of the invention are discussed through various examples in the above disclosure, it should be understood that such details are for illustrative purposes only. The appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that conform to the essence and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only through software solutions, such as installing the described system on existing servers or mobile devices.

[0131] Similarly, it should be noted that, in order to simplify the presentation of the disclosure in this specification and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this specification, sometimes multiple features are grouped into one embodiment, drawing, or description thereof. However, this method of disclosure does not mean that the features required by the subject matter of this specification are more than those mentioned in the claims. In fact, the features of the embodiments are fewer than all the features of the individual embodiments disclosed above.

[0132] In some embodiments, numbers are used to describe components and the quantity of attributes. It should be understood that such numbers used to describe the embodiments are, in some examples, modified by the modifiers "about", "approximately", or "substantially". Unless otherwise stated, "about", "approximately", or "substantially" indicate that the stated number allows a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and such approximate values may vary according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used in some embodiments of this specification to confirm the breadth of their scope are approximate values, in specific embodiments, such numerical settings are as precise as possible within the feasible range.

[0133] For each patent, patent application, patent application publication, and other materials cited in this specification, such as articles, books, specifications, publications, documents, etc., their entire contents are hereby incorporated into this specification by reference. Except for the application history documents that are inconsistent with or conflict with the content of this specification, and except for the documents that limit the broadest scope of the claims of this specification (currently or subsequently appended to this specification). It should be noted that if there are any inconsistencies or conflicts between the descriptions, definitions, and / or uses of terms in the supplementary materials of this specification and the content described in this specification, the descriptions, definitions, and / or uses of terms in this specification shall prevail.

[0134] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be regarded as consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly presented and described in this specification.

Claims

1. A calibration method for a road end sensor, characterized in that, The method is executed based on a roadside computing unit and includes: Obtain the first position information of the vehicle, where the vehicle is located in a first range, and the first position information corresponds to a first coordinate system; Obtain the perception data of the first range collected by a roadside sensor; Based on the perception data, determine the second position information of the vehicle, where the second position information corresponds to a second coordinate system; Based on the first position information and the second position information, determine the conversion relationship between the first coordinate system and the second coordinate system.

2. The method according to claim 1, wherein The obtaining of the first position information of the vehicle includes: Receive the basic safety message sent by the vehicle; Parse the basic safety message to obtain the first position information of the vehicle.

3. The method according to claim 1, characterized in that The determining of the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information includes: Based on the first position information, determine the first trajectory of the vehicle, where the first trajectory includes a plurality of first trajectory points, and each first trajectory point corresponds to a first time point and a first position coordinate; Based on the second position information, determine the second trajectory of the vehicle, where the second trajectory includes a plurality of second trajectory points, and each second trajectory point corresponds to a second time point and a second position coordinate; Based on the first time point and the second time point, match the first trajectory points and the second trajectory points to construct a plurality of matching groups; Based on the first position coordinates of the first trajectory points and the second position coordinates of the second trajectory points in the matching groups, determine the conversion relationship.

4. The method according to claim 1, characterized in that The first coordinate system includes a world coordinate system, and the second coordinate system includes a roadside coordinate system.

5. The method according to claim 1, wherein The method further includes: Obtain the third position information of the surrounding objects of the vehicle, where the surrounding objects are located in the first range, and the third position information corresponds to the first coordinate system.

6. The method according to claim 5, wherein The determining of the conversion relationship between the first coordinate system and the second coordinate system includes: Based on the perception data, determine the fourth position information of the surrounding objects, where the fourth position information corresponds to the second coordinate system; Based on the third position information and the fourth position information, determine the conversion relationship between the first coordinate system and the second coordinate system.

7. An on-line calibration method for a terminal sensor, characterized in that, The method is executed based on a vehicle and includes: Obtain the first data in a second range collected by a vehicle-mounted sensor at the current time, where the first data corresponds to a first coordinate system; Obtain the second data in a third range collected by a roadside sensor at the current time, where the second data corresponds to a second coordinate system, and there is an overlapping area between the second range and the third range; Based on the first data, convert the second data into third data, where the third data corresponds to the first coordinate system.

8. The method according to claim 7, wherein The first coordinate system includes a world coordinate system or a vehicle body coordinate system, and the second coordinate system includes a roadside coordinate system.

9. The method according to claim 7, wherein The converting of the second data into third data based on the first data includes: Match the first data and the second data to determine the first overlapping data corresponding to the overlapping area in the first data and the second overlapping data corresponding to the overlapping area in the second data; Determine the conversion relationship between the first coordinate system and the second coordinate system based on the first coincidence data and the second coincidence data; Based on the conversion relationship, convert the second data to the first coordinate system to determine the third data.

10. A roadside device, characterized in that, It includes: An information determination module, configured to obtain the first position information of a vehicle, where the vehicle is located in a first range, and the first position information corresponds to a first coordinate system; A perception module, configured to obtain the perception data of the first range collected by a roadside sensor; A position determination module, configured to determine the second position information of the vehicle based on the perception data, where the second position information corresponds to a second coordinate system; A calibration module, configured to determine the conversion relationship between the first coordinate system and the second coordinate system based on the first position information and the second position information.

Images