A vehicle head positioning method and device, electronic equipment and readable storage medium

Abstract

The application provides a vehicle head positioning method and device, electronic equipment and readable storage medium. The method comprises: acquiring image data collected by a camera and point cloud data collected by a laser radar; determining a first position of a vehicle head of a vehicle to be positioned in a target area according to the image data; and determining a second position of the vehicle head of the vehicle to be positioned in a laser radar coordinate system according to the point cloud data, a pre-stored included angle and the first position. The pre-stored included angle is an included angle between a line connecting a fixed point position in the target area and an optical center of the camera and a first target axis in the laser radar coordinate system. The fixed point position corresponds to the center of a shooting range of the camera. A second target axis of the laser radar coordinate system coincides with a third target axis of a camera coordinate system. The second position of the vehicle head in the laser radar coordinate system is obtained through the first position and the included angle used for converting the first position to the laser radar coordinate system, thereby reducing resource loss of hardware and installation and debugging difficulty of the laser radar and the camera.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of vehicle positioning, and particularly relates to a vehicle head positioning method and device, an electronic device, and a readable storage medium. BACKGROUND

[0002] When there are multiple vehicles in a lane, accurate positioning of the vehicles is required for integrating and matching vehicle information collected by various sensors, and the vehicle information collected by the sensors is matched based on the positioning information of the vehicles to continuously track the vehicles in a target area.

[0003] Although single-line laser radar point cloud data has rich depth information, the point cloud is relatively sparse and does not have semantic information such as color, which leads to errors in target vehicle clustering and extraction. Compared with single-line laser radar, multi-line laser radar has relatively dense point cloud information, and is more likely to cluster and extract target vehicles. However, the scanning angle of multi-line laser radar is limited, and it is difficult to position and track vehicles in the entire target area by using a single multi-line laser radar.

[0004] A camera can extract image information of vehicles in a target area, and mature image processing algorithms can be used for target extraction and classification, but lack the ability to position vehicles. Combining the depth information provided by laser radar point cloud and the rich semantic information of images can accurately cluster and segment vehicles in a target area, and can accurately position vehicles based on the segmentation results.

[0005] Conventional laser radar and image fusion algorithms require complex calibration of image coordinate systems and laser radar coordinate systems, and the laser radar and video images need to be accurately synchronized in time to register the point cloud and the image in the same coordinate system. However, such matching algorithms require high accuracy, making it difficult to install and debug the device and consuming a lot of hardware resources. SUMMARY

[0006] The embodiments of the present application provide a vehicle head positioning method, device, electronic device, and readable storage medium, which can solve the problem of high difficulty in installing and debugging the device and high consumption of hardware resources caused by existing data fusion algorithms.

[0007] In a first aspect, the embodiments of the present application provide a vehicle head positioning method, comprising:

[0008] obtaining image data of a target area collected by a camera and point cloud data of the target area collected by a laser radar;

[0009] determining a first position of a vehicle head of at least one vehicle to be positioned in the target area according to the image data;

[0010] determining a second position of a head of at least one of the vehicles to be positioned in a laser radar coordinate system according to the point cloud data, a pre-stored included angle and the first position, the pre-stored included angle being an included angle between a line connecting a fixed point position in the target area and an optical center of the camera and a first target axis of the laser radar coordinate system, a direction of the first target axis being a vehicle driving direction, the fixed point position corresponding to a center of a shooting range of the camera, a second target axis of the laser radar coordinate system being coincident with a third target axis of the camera coordinate system, the second target axis direction and the third target axis direction being parallel to the target area.

[0011] Further, the determining the first position of the head of at least one of the vehicles to be positioned in the target area according to the image data comprises:

[0012] determining a third position of each of the vehicles to be positioned in the image data in a pixel coordinate system of the image data;

[0013] calculating a first pitch angle of the head of each of the vehicles to be positioned in the target area in a camera coordinate system according to a pixel size of the camera, a distance from the optical center of the camera to an imaging plane and the third position, and taking the first pitch angle as the first position.

[0014] Further, the determining the second position of the head of at least one of the vehicles to be positioned in the laser radar coordinate system according to the point cloud data, the pre-stored included angle and the first position comprises:

[0015] determining a second pitch angle of the head of each of the vehicles to be positioned in the point cloud data in the laser radar coordinate system according to the pre-stored included angle and the first pitch angle;

[0016] determining the second position of the head of each of the vehicles to be positioned in the laser radar coordinate system according to each of the second pitch angles.

[0017] Further, the determining the second pitch angle of the head of each of the vehicles to be positioned in the point cloud data in the laser radar coordinate system according to the pre-stored included angle and the first pitch angle comprises:

[0018] for each of the vehicles to be positioned in the point cloud data, adding the corresponding first pitch angle and the pre-stored included angle to obtain the second pitch angle of the head of the vehicle to be positioned.

[0019] Further, the determining the first position of the head of at least one of the vehicles to be positioned in the target area according to the image data comprises:

[0020] According to the image data, a first pixel coordinate of a vehicle head of each of the to-be-positioned vehicles in a pixel coordinate system is determined, and the first pixel coordinate is taken as a first position.

[0021] Further, the second position of the vehicle head of at least one of the to-be-positioned vehicles in the laser radar coordinate system is determined according to the point cloud data, the pre-stored included angle, and the first position.

[0022] According to the first pixel coordinate, a second pixel coordinate of the vehicle head of each of the to-be-positioned vehicles in the pixel coordinate system in the point cloud data is determined based on a first correspondence relationship;

[0023] According to the second pixel coordinate, a spatial coordinate of the vehicle head of each of the to-be-positioned vehicles in the laser radar coordinate system in the point cloud data is determined based on a second correspondence relationship, and the spatial coordinate of the vehicle head of the to-be-positioned vehicle in the laser radar coordinate system is taken as the second position.

[0024] The first correspondence relationship is a correspondence relationship between the first pixel coordinate and the second pixel coordinate, and the second correspondence relationship is a correspondence relationship between the second pixel coordinate and the spatial coordinate of the vehicle head of the to-be-positioned vehicle in the laser radar coordinate system based on the pre-stored included angle.

[0025] Further, the calibration board is provided with a calibration block.

[0026] Before the image data of the target region collected by the camera and the point cloud data of the target region collected by the laser radar are acquired, the method further includes:

[0027] When the calibration board is in a first calibration position, the installation pose of the laser radar is adjusted based on the calibration block, so that in the laser radar coordinate system, an included angle between a line connecting the position of the calibration block and a coordinate origin of the laser radar coordinate system and a fourth target axis of the laser radar coordinate system is equal to a preset calibration angle, and the fourth target axis is perpendicular to the target region.

[0028] When the calibration board is in a second calibration position, the installation pose of the laser radar is adjusted based on the calibration block, so that a first included angle and a second included angle are equal, and the laser radar coordinate system in which the second target axis coincides with a third target axis of the camera coordinate system is obtained, the first included angle is an included angle between the line connecting the position of the calibration block and the coordinate origin of the laser radar coordinate system and the first target axis, and the second included angle is an included angle between a plane where the position of the calibration board is located and a fifth target axis of the camera coordinate system.

[0029] The first calibration position is a position in which a fifth target axis of a camera coordinate system is parallel to a vehicle driving direction, the calibration board is perpendicular to the fifth target axis, and the calibration block corresponds to a center of a shooting range of the camera.

[0030] The second calibration position is a position in which the calibration board forms an angle with the fifth target axis in the camera coordinate system, and the calibration block corresponds to the center of the shooting range of the camera.

[0031] In a second aspect, an embodiment of the present application provides a vehicle head positioning system, comprising:

[0032] a camera, a laser radar, and an electronic device, a second target axis of a laser radar coordinate system is coincident with a third target axis of a camera coordinate system, and directions of the second target axis and the third target axis are both parallel to the target region;

[0033] The camera is configured to collect image data of the target region.

[0034] The laser radar is configured to collect point cloud data of the target region.

[0035] The electronic device is configured to acquire the image data and the point cloud data.

[0036] determine a first position of a vehicle head of at least one vehicle to be positioned in the target region according to the image data;

[0037] determine a second position of the vehicle head of the at least one vehicle to be positioned in a laser radar coordinate system according to the point cloud data, a pre-stored included angle, and the first position, the pre-stored included angle is an included angle between a line connecting a fixed point position in the target region and an optical center of the camera and a first target axis in the laser radar coordinate system, a direction of the first target axis is a vehicle driving direction, and the fixed point position corresponds to a center of a shooting range of the camera.

[0038] In a third aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the method according to any one of the first aspect when executing the computer program.

[0039] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executable on a processor to implement the method according to any one of the first aspect.

[0040] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on an electronic device, causes the electronic device to perform the method in any one of the first aspect.

[0041] It can be understood that the beneficial effects of the second aspect to the fifth aspect described above can be referred to the related description in the first aspect, which will not be repeated here.

[0042] The beneficial effects of the embodiments of the present application compared with the prior art are:

[0043] The embodiments of the present application obtain image data of a target region collected by a camera and point cloud data of the target region collected by a laser radar; determine a first position of a vehicle head of at least one vehicle to be positioned in the target region according to the image data; and determine a second position of the vehicle head of the at least one vehicle to be positioned in a laser radar coordinate system according to the point cloud data, a pre-stored included angle and the first position, the pre-stored included angle being an included angle between a connecting line between a fixed point position in the target region and an optical center of the camera and a first target axis in the laser radar coordinate system, the first target axis direction being a vehicle driving direction, the fixed point position corresponding to a center of a shooting range of the camera, a second target axis of the laser radar coordinate system coinciding with a third target axis of the camera coordinate system, the second target axis direction and the third target axis direction both being parallel to the target region, so as to obtain the second position of the vehicle head in the laser radar coordinate system through the first position and the included angle for converting the first position to the laser radar coordinate system, thereby reducing the algorithm complexity, and reducing the resource consumption of the hardware and the installation and debugging difficulty of the laser radar and the camera. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0045] Figure 1 is a flowchart of a vehicle head positioning method provided by an embodiment of the present application;

[0046] Figure 2 is a structure diagram of a laser radar provided by an embodiment of the present application;

[0047] Figure 3 is another structure diagram of a laser radar provided by an embodiment of the present application;

[0048] Figure 4 is another structure diagram of a laser radar provided by another embodiment of the present application;

[0049] Figure 5 is another structural diagram of calibrating a laser radar provided by another embodiment of the present application;

[0050] Figure 6 is a structural diagram of a vehicle head positioning system provided by an embodiment of the present application.

[0051] Figure 7 is a structural diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0052] In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular sequences of acts, techniques, etc., in order to provide a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known methods, devices, circuits, and

[0053] It is to be understood that the terminology “includes”, “has”, “holds”, “contains” and / or “comprising”, when used in this specification and in the following claims, indicates the presence of the stated features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0054] It is also to be understood that the terminology “and / or” when used in this specification and in the following claims, refers to at least one of the items, or any combination of the items, and includes all possible combinations when dependent on two or more items.

[0055] As used in this specification and in the claims, the term “if’ can be interpreted as meaning “when”, or “once”, or “in response to a determination”, or “in response to a detection of”, as appropriate, depending on the context. Similarly, the phrase “if determined” or “if detected [the described condition or event]” can be interpreted as meaning “once determined” or “in response to a determination” or “once detected [the described condition or event]” or “in response to a detection of [the described condition or event]”, as appropriate, depending on the context.

[0056] In addition, in the description of the specification and the appended claims, the terms “first”, “second”, “third”, etc. are only used to distinguish descriptions, and cannot be understood as indicating or implying relative importance.

[0057] Reference within the specification of this application to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places within specified

[0058] Figure 1 is a flowchart of a vehicle head positioning method provided by an embodiment of the application. As an example but not limitation, as shown in Figure 1 the method comprises:

[0059] S101: Obtain image data of a target region collected by a camera and point cloud data of the target region collected by a laser radar.

[0060] The second target axis of the laser radar coordinate system coincides with the third target axis of the camera coordinate system, and the directions of the second target axis and the third target axis are both parallel to the target region, and the directions of the second target axis and the third target axis are both perpendicular to the driving direction of the vehicle in the target region.

[0061] For example, the target region can be a certain highway section, a certain road section or other types of road sections, and there is at least one lane on the road section. The camera is installed above the target region to obtain image data of all lanes in the target region, and the image data contains data of at least one vehicle. The laser radar is installed above the target region to obtain point cloud data of all lanes in the target region, and the point cloud data contains data of at least one vehicle.

[0062] It can be understood that the time difference between the time stamp of the image data collected by the camera and the time stamp of the point cloud data collected by the laser radar is less than a preset time difference, so that the vehicle image data in the image data and the vehicle point cloud data in the point cloud data have a corresponding relationship, and the corresponding relationship is that the vehicle image data and the vehicle point cloud data belong to the same vehicle.

[0063] S102: Determine the first position of the head of at least one vehicle to be positioned in the target region according to the image data.

[0064] Specifically, the image data contains data of each vehicle to be positioned, specifically including the position, size, etc. of the vehicle to be positioned, and the first position of each vehicle to be positioned in the image data is calculated.

[0065] S103: Determine a second position of the front of at least one vehicle to be positioned in the laser radar coordinate system based on the point cloud data, the pre-stored angle, and the first position.

[0066] Among them, the pre-stored angle is the angle between the line between the fixed point position in the target area and the optical center of the camera and the first target axis of the lidar coordinate system, the direction of the first target axis is the vehicle's driving direction, and the fixed point position corresponds to the center of the camera's shooting range.

[0067] This embodiment obtains image data of the target area captured by the camera and point cloud data of the target area captured by the laser radar; determines the first position of the front of at least one vehicle to be located in the target area based on the image data; determines the second position of the front of at least one vehicle to be located in the laser radar coordinate system based on the point cloud data, the pre-stored angle and the first position, the pre-stored angle is the angle between the line between the fixed point position in the target area and the optical center of the camera and the first target axis in the laser radar coordinate system, the direction of the first target axis is the vehicle's driving direction, the fixed point position corresponds to the center of the camera's shooting range, the second target axis of the laser radar coordinate system coincides with the third target axis of the camera coordinate system, and the second target axis direction and the third target axis direction are both parallel to the target area, that is, the second position of the vehicle head in the laser radar coordinate system is obtained through the first position and the angle used to convert the first position to the laser radar coordinate system, thereby reducing the algorithm complexity, thereby reducing the hardware resource loss and the difficulty of installation and debugging of the laser radar and camera.

[0068] In another embodiment, determining a first position of a front end of at least one vehicle to be located in a target area based on image data includes:

[0069] First, the third position of each vehicle to be positioned in the image data in the pixel coordinate system of the image data is determined.

[0070] Then, the first pitch angle of the head of each vehicle to be located in the target area in the camera coordinate system is calculated according to the pixel size of the camera, the distance from the optical center of the camera to the imaging plane and the third position.

[0071] The image data is correspondingly provided with a pixel coordinate system.

[0072] For example, for the i-th vehicle to be positioned, the third position (u i ,v i ), and then according to the pixel size d of the camera u ,d v , the distance f from the optical center of the camera to the imaging plane and the third position (u i ,v i ), based on the camera imaging model, through the formula A first pitch angle of the i-th to-be-positioned vehicle in the camera coordinate system is calculated.

[0073] Wherein, before the intrinsic parameters of the camera are obtained, the camera is calibrated.

[0074] Correspondingly, according to the point cloud data, the pre-stored included angle and the first position, a second position of the head of the at least one to-be-positioned vehicle in the laser radar coordinate system is determined, comprising:

[0075] Firstly, according to the pre-stored included angle and the first pitch angle, a second pitch angle of the head of each to-be-positioned vehicle in the laser radar coordinate system is determined.

[0076] Specifically, for each to-be-positioned vehicle in the point cloud data, the corresponding first pitch angle and the pre-stored included angle are added to obtain the second pitch angle of the head of the to-be-positioned vehicle in the laser radar coordinate system.

[0077] For example, for the i-th to-be-positioned vehicle, the second pitch angle of the head of the to-be-positioned vehicle in the laser radar coordinate system is calculated by the formula β i = α i + γ, wherein γ is the pre-stored included angle.

[0078] Then, according to each second pitch angle, a second position of the head of each to-be-positioned vehicle in the laser radar coordinate system is determined.

[0079] In another embodiment, according to the image data, a first position of the head of the at least one to-be-positioned vehicle in the target area is determined, comprising:

[0080] Firstly, in the image data, a target to-be-positioned vehicle in the target area is identified.

[0081] Specifically, the to-be-positioned vehicles in the image data are subjected to clustering analysis and extraction processing, and the vehicle category to which the to-be-positioned vehicles belong is identified, and the target to-be-positioned vehicle is determined according to the vehicle category to which the to-be-positioned vehicles belong.

[0082] For example, the to-be-positioned vehicles belonging to the truck category are determined as the target to-be-positioned vehicles, and the to-be-positioned vehicles belonging to the small car category are determined as non-target to-be-positioned vehicles.

[0083] Then, the first position of the head of the target to-be-positioned vehicle is determined.

[0084] In another embodiment, according to the image data, a first position of the head of the at least one to-be-positioned vehicle in the target area is determined, comprising:

[0085] According to the image data, a first pixel coordinate of the head of each to-be-positioned vehicle in the pixel coordinate system is determined, and the first pixel coordinate is taken as the first position.

[0086] Correspondingly, according to the point cloud data, the pre-stored included angle and the first position, a second position of a vehicle head of at least one vehicle to be positioned in a laser radar coordinate system is determined, and the second position comprises:

[0087] Based on the first correspondence relationship, the second pixel coordinates of the vehicle head of each vehicle to be positioned in the pixel coordinate system are determined according to the first pixel coordinates;

[0088] Based on the second correspondence relationship, the spatial coordinates of the vehicle head of each vehicle to be positioned in the laser radar coordinate system are determined according to the second pixel coordinates, and the spatial coordinates of the vehicle head of the vehicle to be positioned in the laser radar coordinate system are taken as the second position;

[0089] The first correspondence relationship is a correspondence relationship between the first pixel coordinates and the second pixel coordinates, and the second correspondence relationship is a correspondence relationship between the second pixel coordinates and the spatial coordinates of the vehicle head of the vehicle to be positioned in the laser radar coordinate system based on the pre-stored included angle.

[0090] Specifically, the establishment of the second correspondence relationship comprises:

[0091] The spatial coordinates of the vehicle head of each vehicle to be positioned in the laser radar coordinate system are obtained;

[0092] According to the pre-stored included angle, the spatial coordinates of the vehicle head of each vehicle to be positioned in the laser radar coordinate system are converted into the spatial coordinates of the vehicle head of each vehicle to be positioned in the camera coordinate system in the point cloud data;

[0093] Based on the camera imaging model, the second pixel coordinates of the vehicle head of each vehicle to be positioned in the pixel coordinate system are determined according to the intrinsic parameters of the camera and the spatial coordinates of the vehicle head of each vehicle to be positioned in the camera coordinate system in the point cloud data;

[0094] The spatial coordinates of the vehicle head of the vehicle to be positioned in the laser radar coordinate system are corresponded with the second pixel coordinates to obtain the second correspondence relationship.

[0095] For example, the intrinsic parameters of the camera include the pixel size of the camera and the distance from the optical center of the camera to the imaging plane.

[0096] The establishment of the first correspondence relationship comprises:

[0097] The first pixel coordinates of the vehicle head of each vehicle to be positioned in the pixel coordinate system in the image data are obtained;

[0098] The first pixel coordinates are corresponded with the second pixel coordinates to obtain the first correspondence relationship.

[0099] In another embodiment, a laser radar coordinate system and a camera coordinate system are established;

[0100] Coordinate origin o of the laser radar coordinate system r is the center of the laser radar, x r axis direction is the vehicle driving direction, y r axis is perpendicular to the target area, that is, perpendicular to the road surface; z r axis is parallel to the target area, that is, parallel to the road surface, and perpendicular to the vehicle driving direction. Take x r axis as the first target axis, take z r axis as the second target axis, take y r axis as the fourth target axis.

[0101] Coordinate origin o of the camera coordinate system c is the optical center of the camera, x c axis direction is the optical axis direction of the camera, y c axis is perpendicular to the optical axis; z c axis is parallel to the target area, that is, parallel to the road surface, and perpendicular to the optical axis direction, when x c axis is parallel to the vehicle driving direction, z c axis is perpendicular to the vehicle driving direction. Take z c axis as the third target axis, take x c axis as the fifth target axis.

[0102] In this embodiment, a calibration block is arranged on the calibration board for calibration. In order to better calibrate, the calibration block can be arranged at the center of the calibration board.

[0103] Then, before acquiring the image data of the target area collected by the camera and the point cloud data of the target area collected by the laser radar, further comprising:

[0104] Firstly, a first calibration step is performed:

[0105] When the calibration board is in the first calibration position, the installation pose of the laser radar is adjusted based on the calibration block, so that the included angle between the line connecting the position of the calibration block and the coordinate origin of the laser radar coordinate system and the fourth target axis of the laser radar coordinate system in the laser radar coordinate system is equal to a preset calibration angle, and the fourth target axis is perpendicular to the target area.

[0106] The first calibration position is the position when the fifth target axis of the camera coordinate system is parallel to the vehicle driving direction, the calibration board is perpendicular to the fifth target axis, and the calibration block corresponds to the center of the shooting range of the camera. That is, when the fifth target axis is x c axis, the first calibration position is when x c axis is parallel to the vehicle driving direction, the calibration board is perpendicular to the x c axis of the camera coordinate system, the calibration board is directly opposite to the camera, and the calibration block corresponds to the center of the shooting range of the camera.

[0107] It is understandable that when the calibration plate is in the first calibration position, in the laser radar coordinate system, the angle between the line connecting the position of the calibration block and the coordinate origin of the laser radar coordinate system and the fourth target axis of the laser radar coordinate system is not equal to the preset calibration angle, then the first calibration step is performed. The preset calibration angle is set to 90°, but is not limited to this. That is, when the fourth target axis is y r Axis, in the lidar coordinate system, the line between the position of the calibration block and the coordinate origin of the lidar coordinate system is connected to the y axis of the lidar coordinate system r If the angle between the axes is not equal to 90°, the first calibration step is performed.

[0108] For example, Figure 2 It is a structural diagram of a calibration laser radar provided in one embodiment of the present application. Figure 3 This is another structural diagram of a calibration laser radar provided by an embodiment of the present application. Figure 2 、 3 As shown, in Figure 2 、 3 The lidar coordinate system and camera coordinate system in are projections on the xoy plane.

[0109] like Figure 2 As shown, the calibration plate 10 is in the first calibration position, but the line between the position of the calibration block 11 and the coordinate origin of the laser radar coordinate system is aligned with the y coordinate of the laser radar coordinate system. r The angle between the axes is not equal to 90°, then based on the calibration block 11, along the y r Adjust the installation posture of the laser radar in the y-axis direction so that the line between the position of the calibration block 11 and the coordinate origin of the laser radar coordinate system is aligned with the y-axis of the laser radar coordinate system. r The angle between the axes is 90°. Figure 3 Displayed as x r Axis and x c The axes are in the same straight line.

[0110] Second, perform the second calibration step:

[0111] When the calibration plate is in the second calibration position, the installation posture of the laser radar is adjusted based on the calibration block so that the first angle and the second angle are equal, and the laser radar coordinate system in which the second target axis coincides with the third target axis of the camera coordinate system is obtained. The first angle is the angle between the line between the position of the calibration block and the coordinate origin of the laser radar coordinate system and the first target axis, and the second angle is the angle between the plane where the calibration plate is located and the fifth target axis of the camera coordinate system.

[0112] The second calibration position is the position in the camera coordinate system where the calibration plate forms an angle with the fifth target axis and the calibration block corresponds to the center of the camera's shooting range.c the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis c the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis c the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis

[0113] It can be understood that when the calibration plate is in the second calibration position, the first included angle and the second included angle are not equal, and the second calibration step is performed. That is, when the first target axis is the x-axis r the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis c the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis r the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis c the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis

[0114] In an example, Figure 4 is another structural schematic diagram of calibrating a lidar provided in an embodiment of the present application. Figure 5 is another structural schematic diagram of calibrating a lidar provided in an embodiment of the present application. As Figure 4 , 5 indicated, Figure 4 , 5 the lidar coordinate system and the camera coordinate system in are projections on the xoy plane.

[0115] When the calibration plate 10 is in the second calibration position, the calibration plate is parallel to the x-axis of the lidar coordinate system, and in order to facilitate adjustment of the coordinate system, the first included angle is represented as the included angle between the line connecting the position of the calibration block 11 and the coordinate origin of the lidar coordinate system and the plane on which the position of the calibration plate 10 is located. r the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis

[0116] As Figure 4 indicated, the calibration plate 10 is in the second calibration position, but the included angle δ1 between the line connecting the position of the calibration block 11 and the coordinate origin of the lidar coordinate system and the plane on which the position of the calibration plate 10 is located and the second included angle δ2 are not equal, and then based on the calibration block 11, the mounting pose of the lidar is adjusted along the x-axis direction to make the included angle δ1 between the line connecting the position of the calibration block 11 and the coordinate origin of the lidar coordinate system and the plane on which the position of the calibration plate 10 is located and the second included angle δ2 equal. r the x-axis, the second calibration position is that the calibration plate is parallel to the x-axis Figure 3 is shown as δ1 and δ2 being equal.

[0117] The embodiment adjusts the installation pose of the laser radar based on the calibration block when the calibration board is in the first calibration position, so that the included angle between the line connecting the position of the calibration block and the coordinate origin of the laser radar coordinate system and the fourth target axis of the laser radar coordinate system is equal to the preset calibration angle, and the fourth target axis is perpendicular to the target area; when the calibration board is in the second calibration position, the installation pose of the laser radar is adjusted based on the calibration block, so that the first included angle and the second included angle are equal, and the laser radar coordinate system in which the second target axis coincides with the third target axis of the camera coordinate system is obtained, the first included angle is the included angle between the line connecting the position of the calibration block and the coordinate origin of the laser radar coordinate system and the first target axis, and the second included angle is the included angle between the plane where the position of the calibration board is located and the fifth target axis of the camera coordinate system, so as to provide a basis for obtaining the included angle used for converting the first position to the laser radar coordinate system.

[0118] In another embodiment, based on the second target axis of the laser radar coordinate system coinciding with the third target axis of the camera coordinate system, the pre-stored included angle is measured by placing the single-point laser range finder and the gyroscope at the fixed point position, and the single-point laser range finder emits a laser beam to the optical center.

[0119] Specifically, the single-point laser range finder is placed at the fixed point position and emits a laser beam to the optical center, and then the gyroscope is close to the single-point laser range finder to measure the included angle between the line connecting the fixed point position and the optical center of the camera and the first target axis in the laser radar coordinate system, so as to obtain the pre-stored included angle.

[0120] It should be understood that the size of the serial number of each step in the above embodiment does not mean the order of execution, and the execution order of each process should be determined according to its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0121] Corresponding to the method described in the above embodiment, only the part related to the embodiments of the present application is shown for the convenience of description.

[0122] Figure 6 is a structural schematic diagram of a vehicle head positioning system provided by the embodiments of the present application. As an example but not limitation, as shown in Figure 6 , the system comprises a camera 20, a laser radar 21 and an electronic device 22, the second target axis of the laser radar coordinate system coincides with the third target axis of the camera coordinate system, the directions of the second target axis and the third target axis are both parallel to the target area, and the directions of the second target axis and the third target axis are perpendicular to the driving direction of the vehicle.

[0123] The camera 20 is used to collect image data of the target area.

[0124] The camera is mounted above the target area.

[0125] LiDAR 21, used to collect point cloud data of the target area;

[0126] Among them, the laser radar is installed above the target area.

[0127] Electronic device 22, used to acquire image data and point cloud data;

[0128] for determining a first position of a front end of at least one vehicle to be located in a target area based on the image data;

[0129] It is used to determine the second position of the front of at least one vehicle to be positioned in the lidar coordinate system based on point cloud data, a pre-stored angle and a first position. The pre-stored angle is the angle between the line between the fixed point position in the target area and the optical center of the camera and the first target axis in the lidar coordinate system. The direction of the first target axis is the vehicle's driving direction. The fixed point position corresponds to the center of the shooting range of the camera 20.

[0130] In another embodiment, the system further includes a mobile device, and the laser radar is installed on the mobile device.

[0131] For example, the moving device may be a two-dimensional slide.

[0132] The electronic device is also used to adjust the installation position of the laser radar by controlling the movement of the mobile device.

[0133] For example, the electronic device controls the mobile device to move along the vehicle's driving direction and in a direction perpendicular to the target area to adjust the installation posture of the laser radar.

[0134] Figure 7 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present application. Figure 7 As shown, the electronic device 3 of this embodiment includes: at least one processor 30 ( Figure 7 Only one is shown), a memory 31 and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 implements the steps of any of the above-mentioned method embodiments when executing the computer program 32.

[0135] The electronic device 3 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The electronic device 3 may include, but is not limited to, a processor 30 and a memory 31. Those skilled in the art will understand that Figure 7 This is merely an example of the electronic device 3 and does not constitute a limitation on the electronic device 3 . The electronic device 3 may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device 3 may also include input and output devices, network access devices, etc.

[0136] The processor 30 can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0137] The memory 31 can be an internal storage unit of the electronic device 3 in some embodiments, for example, a hard disk or a memory of the electronic device 3. The memory 31 can also be an external storage device of the electronic device 3 in other embodiments, for example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Further, the memory 31 can include both the internal storage unit and the external storage device of the electronic device 3. The memory 31 is used to store an operating system, an application program, a boot loader, data and other programs, for example, program codes of the computer program, etc. The memory 31 can also be used to temporarily store data that has been output or is to be output.

[0138] It should be noted that the information interaction, execution process, etc. between the above apparatuses / units, since based on the same concept as the method embodiments of the present application, the specific functions and the brought technical effects can be referred to the method embodiments part, and will not be described here in detail.

[0139] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or software functional unit. In addition, the specific name of each functional unit and module is only for easy distinction, and does not limit the protection scope of the application. The specific working process of the unit and module in the above system can refer to the corresponding process in the foregoing method embodiment, which will not be described here.

[0140] The embodiment of the application further provides a computer readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in each of the above method embodiments can be implemented.

[0141] The embodiment of the application provides a computer program product. When the computer program product is run on an electronic device, the steps in each of the above method embodiments can be implemented.

[0142] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above embodiment methods can be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps in each of the above method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer readable medium at least includes any entity or device capable of carrying the computer program code to the photographing device / terminal device, recording medium, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium. For example, U disk, mobile hard disk, magnetic disk or optical disk, etc. In some jurisdictions, according to legislation and patent practice, the computer readable medium cannot be an electrical carrier signal and a telecommunication signal.

[0143] In the above embodiments, the description of each embodiment focuses on different aspects, and the parts not described in detail or recorded in a certain embodiment can be referred to the relevant description of other embodiments.

[0144] Those skilled in the art can understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. The skilled person can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0145] In the embodiments provided in the present application, it should be understood that the disclosed apparatus / network device and method can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely schematic, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0146] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiments.

[0147] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A vehicle head positioning method, characterized by, The method comprises: acquiring image data of a target region captured by a camera and point cloud data of the target region captured by a laser radar; determining a first position of a vehicle head of at least one vehicle to be positioned in the target region according to the image data; determining a second position of the vehicle head of the at least one vehicle to be positioned in a laser radar coordinate system according to the point cloud data, a pre-stored included angle and the first position, wherein the pre-stored included angle is an included angle between a line connecting a fixed point position in the target region and an optical center of the camera and a first target axis of the laser radar coordinate system, the first target axis is in a vehicle driving direction, the fixed point position corresponds to a center of a shooting range of the camera, a second target axis of the laser radar coordinate system is coincident with a third target axis of the camera coordinate system, and the second target axis and the third target axis are both parallel to the target region; the determining of the first position of the vehicle head of the at least one vehicle to be positioned in the target region according to the image data comprises: determining a third position of each vehicle to be positioned in the image data in a pixel coordinate system of the image data; calculating a first pitch angle of the vehicle head of each vehicle to be positioned in the target region in a camera coordinate system according to a pixel size of the camera, a distance from the optical center of the camera to an imaging plane and the third position, and taking the first pitch angle as the first position; the determining of the second position of the vehicle head of the at least one vehicle to be positioned in the laser radar coordinate system according to the point cloud data, the pre-stored included angle and the first position comprises: determining a second pitch angle of the vehicle head of each vehicle to be positioned in the point cloud data in the laser radar coordinate system according to the pre-stored included angle and the first pitch angle; determining the second position of the vehicle head of each vehicle to be positioned in the laser radar coordinate system according to each second pitch angle.

2. The method of claim 1, wherein, the determining of the second pitch angle of the vehicle head of each vehicle to be positioned in the point cloud data according to the pre-stored included angle and the first pitch angle comprises: for each vehicle to be positioned in the point cloud data, adding the corresponding first pitch angle and the pre-stored included angle to obtain the second pitch angle of the vehicle head of the vehicle to be positioned.

3. The method of claim 1, wherein, the determining of the first position of the vehicle head of the at least one vehicle to be positioned in the target region according to the image data comprises: determining a first pixel coordinate of the vehicle head of each vehicle to be positioned in the pixel coordinate system according to the image data, and taking the first pixel coordinate as the first position.

4. The method of claim 3, wherein, the determining of the second position of the vehicle head of the at least one vehicle to be positioned in the laser radar coordinate system according to the point cloud data, the pre-stored included angle and the first position comprises: determining a second pixel coordinate of the vehicle head of each vehicle to be positioned in the pixel coordinate system in the point cloud data according to the first pixel coordinate based on a first correspondence relationship; determine, based on the second pixel coordinates, a spatial coordinate of a head of each of the to-be-positioned vehicles in the point cloud data in the laser radar coordinate system according to a second correspondence relationship, and take the spatial coordinate of the head of the to-be-positioned vehicle in the laser radar coordinate system as the second position; The first correspondence relationship is a correspondence relationship between the first pixel coordinates and the second pixel coordinates, and the second correspondence relationship is a correspondence relationship between the second pixel coordinates and the spatial coordinate of the head of the to-be-positioned vehicle in the laser radar coordinate system based on the pre-stored included angle.

5. The method of claim 1, wherein, The calibration board is provided with a calibration block. Before the image data of the target region collected by the camera and the point cloud data of the target region collected by the laser radar are acquired, the method further includes: When the calibration board is in a first calibration position, the installation pose of the laser radar is adjusted based on the calibration block, so that, in the laser radar coordinate system, an included angle between a line connecting the position of the calibration block and an origin of the laser radar coordinate system and a fourth target axis of the laser radar coordinate system is equal to a preset calibration angle, and the fourth target axis is perpendicular to the target region. When the calibration board is in a second calibration position, the installation pose of the laser radar is adjusted based on the calibration block, so that a first included angle and a second included angle are equal, and the laser radar coordinate system in which the second target axis coincides with a third target axis of the camera coordinate system is obtained, the first included angle is an included angle between the line connecting the position of the calibration block and the origin of the laser radar coordinate system and the first target axis, and the second included angle is an included angle between a plane in which the calibration board is located and a fifth target axis of the camera coordinate system. The first calibration position is a position in which the fifth target axis of the camera coordinate system is parallel to a vehicle driving direction, the calibration board is perpendicular to the fifth target axis, and the calibration block corresponds to the center of the shooting range of the camera. The second calibration position is a position in which, in the camera coordinate system, the calibration board forms an angle with the fifth target axis, and the calibration block corresponds to the center of the shooting range of the camera.

6. A vehicle head positioning system, characterized by, The method includes: A camera, a laser radar, and an electronic device, a second target axis of a laser radar coordinate system coincides with a third target axis of a camera coordinate system, and directions of the second target axis and the third target axis are both parallel to a target region; The camera is configured to collect image data of the target region. The laser radar is configured to collect point cloud data of the target region. The electronic device is configured to acquire the image data and the point cloud data. The electronic device is configured to determine a first position of a head of at least one to-be-positioned vehicle in the target region according to the image data. The electronic device is configured to determine a second position of the head of the at least one to-be-positioned vehicle in the laser radar coordinate system according to the point cloud data, a pre-stored included angle, and the first position, the pre-stored included angle is an included angle between a line connecting a fixed point position in the target region and an optical center of the camera and a first target axis in the laser radar coordinate system, and a direction of the first target axis is a vehicle driving direction, the fixed point position corresponds to a center of a shooting range of the camera. The method further comprises: determining a third position of each of the to-be-positioned vehicles in the image data in a pixel coordinate system of the image data; calculating a first pitch angle of a head of each of the to-be-positioned vehicles in the target area in a camera coordinate system according to a pixel size of the camera, a distance from an optical center of the camera to an imaging plane, and the third position, and taking the first pitch angle as the first position; The method further comprises: determining a second pitch angle of a head of each of the to-be-positioned vehicles in the point cloud data in a laser radar coordinate system according to the pre-stored included angle and the first pitch angle; determining a second position of a head of each of the to-be-positioned vehicles in the laser radar coordinate system according to each of the second pitch angles.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, The processor implements the method of any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 7. The computer program, when executed by the processor, implements the method of any one of claims 1 to 5. The computer program, when executed by the processor, implements the method of any one of claims 1 to 5.

Images