Geographic coordinate positioning method and device of roadside image and electronic equipment

Abstract

The embodiment of the invention provides a geographical coordinate positioning method and device of a roadside image and electronic equipment, and relates to the technical field of image processing, in particular to the technical field of automatic driving, vehicle-road cooperation and intelligent traffic. According to the specific implementation scheme, the method comprises the steps of obtaining road point cloud and a road image collected for a target road; homonymy point pairs formed by the point cloud points and pixel points in the road image are obtained; determining external parameters of the roadside camera based on the homonymy point pairs and the internal parameters of the roadside camera; and projecting the road point cloud to the road image based on the internal reference of the roadside camera and the external reference of the roadside camera, and determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the geographic coordinates of the point cloud points. Based on the scheme, the mapping relation between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, and a basis is provided for environment perception based on the road image acquired by the roadside camera.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to the field of autonomous driving, vehicle-road collaboration and intelligent transportation technology. Specifically, the present application relates to a method, device and electronic device for geographic coordinate positioning of roadside images. Background Art

[0002] In the vehicle-road cooperative system, roadside equipment has a larger observation range and better environmental perception. Therefore, it is very important to make good use of the roadside environmental perception to assist the vehicle's environmental perception.

[0003] At present, the camera-based perception method of roadside equipment generally performs 2D detection on roadside images to obtain pixel-level detection results. This pixel-level detection result needs to be mapped to the geographic coordinate system before it can be effectively used by the vehicle.

[0004] Therefore, how to map the pixel coordinates of pixel points in roadside images to geographic coordinates has become an important technical issue. Summary of the invention

[0005] The present application provides a method, device and electronic device for geographical coordinate positioning of roadside images, aiming to solve at least one of the above technical defects. The technical solution adopted by the present application is as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for geographical coordinate positioning of a roadside image, the method comprising:

[0007] Obtain the road point cloud and road image collected for the target road. Each point cloud point in the road point cloud corresponds to the geographical coordinates of the point cloud point. The road image is collected by a roadside camera.

[0008] Obtain the same-name point pairs consisting of point cloud points and pixel points in the road image;

[0009] Determine the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera;

[0010] Based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and based on the geographic coordinates of the point cloud points, the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is determined.

[0011] In a second aspect, an embodiment of the present application provides a device for geographical coordinate positioning of a roadside image, the device comprising:

[0012] A data acquisition module is used to obtain a road point cloud and a road image collected for a target road. Each point in the road point cloud corresponds to a geographical coordinate of the point cloud point. The road image is collected by a roadside camera.

[0013] A same-name point pair acquisition module is used to acquire same-name point pairs consisting of point cloud points and pixel points in the road image;

[0014] An external parameter determination module, used to determine the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera;

[0015] The positioning module is used to project the road point cloud onto the road image based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, and to determine the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the geographic coordinates of the point cloud points.

[0016] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-mentioned method for geographic coordinate positioning of roadside images.

[0017] In a fourth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising:

[0018] one or more processors; and

[0019] A memory associated with the one or more processors is used to store program instructions, and when the program instructions are read and executed by the one or more processors, the steps of the method for geographic coordinate positioning of roadside images are executed.

[0020] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for geographic coordinate positioning of roadside images.

[0021] The beneficial effects of the technical solution provided by the embodiment of the present application are:

[0022] In this scheme, by projecting the road point cloud onto the road image and based on the geographic coordinates of the road point cloud, a mapping relationship between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, providing a basis for environmental perception based on the road images collected by roadside cameras.

[0023] Compared with the traditional method of using RTK equipment to mark points, the use of drone-mounted laser scanning equipment to collect road point clouds has higher accuracy and higher collection efficiency, and can also avoid safety hazards during operation. In addition, the use of drone-mounted laser scanning equipment to collect road point clouds has altitude information, which can sense the uneven areas on the road and effectively reflect the actual situation of the target road. Incorporating altitude information into the calculation of the mapping relationship between pixel coordinates and geographic coordinates in the roadside image in this application can effectively improve the accuracy of the calculation results.

[0024] In addition, this solution uses drone-mounted laser scanning equipment to quickly collect road point clouds, and automatically calculates and determines the mapping relationship between the pixel coordinates of pixel points in the roadside image and the geographic coordinates, which helps to achieve fast and convenient deployment of roadside equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in describing the embodiments of the present application are briefly introduced below.

[0026] Figure 1 A system architecture diagram applicable to the embodiments of the present application;

[0027] Figure 2 A schematic diagram of a flow chart of a method for geographical coordinate positioning of a roadside image provided in an embodiment of the present application;

[0028] Figure 3 Schematic diagram of the road point cloud collected by the laser scanning equipment on the drone;

[0029] Figure 4 A schematic diagram of an operation interface for performing the same-name point calibration operation on a road point cloud and a road image;

[0030] Figure 5 It is a schematic diagram of an image obtained after the road point cloud is projected onto the road image;

[0031] Figure 6 A schematic diagram of the structure of a geographic coordinate positioning device for roadside images provided in an embodiment of the present application;

[0032] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of this application.

[0034] The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

[0035] It should be understood that the term "and / or" used in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0036] The word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting", depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to determining" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)", depending on the context.

[0037] The vehicle's perception equipment, such as cameras, is usually installed at a low position, so that obstacles or pedestrians may be blocked by surrounding vehicles, which will affect the vehicle's environmental perception and driving safety. In a vehicle-road cooperative system with roadside equipment, the roadside equipment will be installed at a higher position, giving the roadside equipment a larger observation range, making it less likely to be blocked by vehicles, and having better environmental perception. Therefore, it is very important to make good use of the roadside's environmental perception to assist the vehicle's environmental perception.

[0038] At present, the camera-based perception method of roadside equipment generally performs 2D detection in roadside images, and the retrieval results are all expressed in pixel coordinates. This pixel-level detection result needs to be mapped to the geographic coordinate system before it can be effectively used by the vehicle side, or used to project the pixel detection results into the physical world in real time to form a holographic intersection.

[0039] Therefore, how to map the pixel coordinates of pixel points in roadside images to geographic coordinates has become an important technical issue.

[0040] In addition, determining the mapping relationship between the pixel coordinates of the pixel points in the roadside image and the geographic coordinates can be regarded as part of the roadside equipment deployment work. Therefore, if the above mapping relationship can be determined quickly and conveniently, the deployment efficiency of the roadside equipment can be effectively improved.

[0041] In the related art, the mapping relationship between the coordinates of the pixel points in the roadside image and the geographic coordinates is generally obtained based on the real-time kinematic (RTK) technology. The surveying and mapping personnel use handheld RTK equipment to collect fixed points on the zebra crossing to obtain the geographic coordinates of these points. This method has the problems of poor accuracy and low efficiency, and the operation process of handheld RTK equipment to collect points on the zebra crossing also has certain safety hazards. In addition, this method needs to assume that the ground is flat first, and then obtain the homography matrix of the mapping relationship between the coordinates of the pixel points in the roadside image and the geographic coordinates, so as to calculate the mapping relationship between the coordinates of the pixel points in the roadside image and the geographic coordinates. In reality, there are generally some uneven places on the road, and the lack of height information will affect the accuracy of the calculation.

[0042] In view of this, the present application provides a new idea. In order to facilitate the understanding of the present application, the system architecture on which the present application is based is first described. Figure 1 An exemplary system architecture to which the embodiments of the present application can be applied is shown. Figure 1 As shown in , the system architecture may include: a server, a network, a roadside camera, and a point cloud acquisition device.

[0043] The network is used as a medium to provide a communication link between the server and the roadside camera, and between the server and the point cloud acquisition device. The network can include various connection types, such as wired, wireless communication links, or fiber optic cables.

[0044] It is understandable that the device for determining the geographic coordinates of the roadside image provided in the embodiment of the present application can be deployed on the server side.

[0045] The roadside camera can exchange information with the geographic coordinate positioning device of the roadside image on the server side through the network. For example, the roadside camera sends the road image collected for the target road to the geographic coordinate positioning device of the roadside image on the server side. The point cloud acquisition device can exchange information with the geographic coordinate positioning device of the roadside image on the server side through the network. For example, the point cloud acquisition device sends the road point cloud collected for the target road to the geographic coordinate positioning device of the roadside image on the server side.

[0046] The geographic coordinate positioning device of the roadside image on the server side can determine the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the road point cloud and the road image.

[0047] like Figure 1The server side shown can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, and big data and artificial intelligence platforms. The user terminal can be a smart terminal such as a smartphone, tablet computer, laptop computer, desktop computer, smart TV, etc.

[0048] It should be noted that, in addition to performing geographic coordinate positioning of roadside images online, the above-mentioned geographic coordinate positioning device of roadside images can also perform geographic coordinate positioning of roadside images offline.

[0049] It should be understood that Figure 1 The number of servers, roadside image geographic coordinate positioning devices, roadside cameras, and point cloud acquisition devices in the figure is only illustrative. Any number of servers, roadside image geographic coordinate positioning devices, roadside cameras, and point cloud acquisition devices may be provided according to implementation requirements.

[0050] Figure 2 A schematic diagram of a method for locating the geographic coordinates of a roadside image provided by an embodiment of the present application is shown. The method can be performed by Figure 1 The server side execution in the system shown. Figure 2 As shown, the method may mainly include:

[0051] Step S210: Acquire a road point cloud and a road image collected for the target road, each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera;

[0052] Step S220: obtaining point pairs with the same name consisting of point cloud points and pixel points in the road image;

[0053] Step S230: Determine the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera;

[0054] Step S240: Based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and based on the geographic coordinates of the point cloud points, the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is determined.

[0055] The roadside camera is used to collect images of the target road to obtain a roadside image. As an example, the roadside camera can be installed on a mounting pole above the target road, maintaining a certain angle with the lane.

[0056] The road point cloud can be obtained by collecting the target road by the point cloud collection device. The point cloud collection device also needs to provide the geographical coordinates of the point cloud points corresponding to each point cloud point in the road point cloud.

[0057] To project the road point cloud onto the road image, the extrinsic parameters of the roadside camera are required. The extrinsic parameters of the roadside camera are determined by the position of the roadside camera in the world coordinate system, and the extrinsic parameters of the roadside camera need to be calculated.

[0058] A set of same-name point pairs is composed of a point cloud point and a pixel point. The same-name point pairs can be obtained by manual marking or by matching based on a matching algorithm. The intrinsic parameters of the roadside camera are known, so the external parameters of the roadside camera can be determined based on the same-name point pairs and the internal parameters of the roadside camera.

[0059] After determining the external parameters of the roadside camera, the road point cloud can be projected onto the road image based on the internal parameters and external parameters of the roadside camera, that is, each point cloud point in the road point cloud is projected onto the image coordinate system of the road avatar, so that the pixel points in the road image can be matched with the point cloud points, and then the geographic coordinates of the matching pixel points are determined based on the geographic coordinates of the point cloud points, that is, a mapping relationship between pixel coordinates and geographic coordinates is established.

[0060] The method provided in the embodiment of the present application obtains a road point cloud and a road image collected for a target road, each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; obtains a pair of points of the same name consisting of point cloud points and pixel points in the road image; determines the external parameters of the roadside camera based on the point pairs of the same name and the internal parameters of the roadside camera; projects the road point cloud onto the road image based on the internal parameters of the roadside camera and the external parameters of the roadside camera, and determines the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the point cloud point geographic coordinates. In this solution, by projecting the road point cloud onto the road image and based on the geographic coordinates of the road point cloud, a mapping relationship between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, which provides a basis for environmental perception based on the road image collected by the roadside camera.

[0061] The point cloud acquisition device may be various types of laser scanning devices. As an example, the point cloud acquisition device may be a laser scanning device carried by an unmanned aerial vehicle.

[0062] Figure 3 It is a schematic diagram of the road point cloud collected by the drone-mounted laser scanning equipment.

[0063] In this example, the acquisition results of the drone-mounted laser scanning device will be output in the form of an oblique photography file. The oblique photography file is a special type of file similar to an image. Its content contains the red, green, and blue (RGB) information of the point cloud points, the three-dimensional coordinates of the point cloud points, and the transformation information of each point cloud point to the geographic longitude and latitude coordinates.

[0064] After obtaining the road point cloud and the road image, the operation of calibrating the same-name point pairs can be performed. In this example, an operation interface for the same-name point calibration operation can be provided to the annotator, and the annotator performs manual calibration based on the calibration operation interface.

[0065] As an example, Figure 4 A schematic diagram of an operation interface for performing the same-name point calibration operation on a road point cloud and a road image.

[0066] The camera image is the road image captured by the roadside camera, and the drone TIF is the road point cloud collected by the drone-mounted laser scanning device.

[0067] By displaying the road image and road point cloud on both sides of the operation interface, annotators can easily perform the calibration operation of the same-name points.

[0068] In this example, a high-precision color point cloud can be used for calibration to improve the accuracy of the same-name point pairs.

[0069] Since the road images collected by the roadside cameras may be distorted, the road images can be dedistorted in advance to prevent the distorted images from affecting the accuracy of the same-name point calibration operation.

[0070] Specifically, as an optional method, obtaining the same-name point pairs consisting of point cloud points and pixel points in the road image includes:

[0071] De-distorting the roadside image based on the internal parameters of the roadside camera to obtain a de-distorted roadside image;

[0072] A calibration operation is performed based on the pixel points in the dedistorted roadside image and the road point cloud to determine the same-name point pairs consisting of the point cloud points and the pixel points.

[0073] The internal parameters of the roadside camera include distortion coefficients, and the roadside image can be dedistorted based on the camera internal parameters and a preset dedistortion algorithm to obtain a dedistorted roadside image. The dedistortion algorithm is a known algorithm and is not limited in this application.

[0074] After the roadside image is dedistorted to obtain the dedistorted roadside image, a calibration operation is performed based on the dedistorted roadside image and the road point cloud, which can make the calibrated same-name point pairs have higher accuracy, thereby making the roadside camera extrinsic parameters subsequently calculated based on the same-name point pairs have higher accuracy.

[0075] After obtaining the same-name point pairs, the external parameters of the roadside camera can be determined based on the same-name point pairs and the internal parameters of the roadside camera.

[0076] Specifically, as an optional method, based on the same-name point pairs and the intrinsic parameters of the roadside camera, the extrinsic parameters of the roadside camera are determined, including:

[0077] Obtain the relative coordinates of the road point cloud in the road relative coordinate system. The road relative coordinate system takes the point in the target road as the origin;

[0078] The external parameters of the roadside camera are used as variables, and based on the internal parameters of the roadside camera and the relative coordinates of the point cloud, the pixel coordinates of the road point cloud in the image coordinate system are calculated;

[0079] An error formula is constructed based on the point cloud pixel coordinates of the point cloud points in the same-name point pair and the pixel coordinates of the pixel points in the same-name point pair;

[0080] Minimize the error formula to determine the external parameters of the roadside camera.

[0081] The road relative coordinate system is a spatial coordinate system established with a point in the target road as the origin. As an example, when the target road includes an intersection, the origin of the road relative coordinate system is the center point of the intersection.

[0082] Refer to the above Figure 3 In the example, when collecting road point clouds through drone-mounted laser scanning equipment, the three-dimensional coordinates of the point cloud and the transformation information of each point cloud point to geographic longitude and latitude coordinates can be output simultaneously.

[0083] The process of converting the road point cloud to the road relative coordinate system is as follows:

[0084] Firstly, based on the above transformation information, the horizontal coordinates and vertical coordinates of the point cloud points in the road point cloud are transformed into the geographic latitude and longitude coordinate system.

[0085] The horizontal and vertical coordinates of the point cloud points are transformed by the following formula to obtain the geographic longitude coordinates of the point cloud points.

[0086] longitude=geo_information[0]+pix_x×geo_information[1]+pix_y×geo_information[2]; (Formula 1)

[0087] Among them, longitude represents the geographic longitude coordinate of the point cloud point. pix_x represents the horizontal coordinate of the point cloud point, and pix_y represents the vertical coordinate of the point cloud point. geo_information[0], geo_information[1], and geo_information[2] are all transformation coefficients.

[0088] The horizontal and vertical coordinates of the point cloud points are transformed using the following formula 2 to obtain the geographic dimension coordinates of the point cloud points.

[0089] latitude=geo_information[3]+pix_y×geo_information[4]+pix_y×geo_information[5]; (Formula 2)

[0090] Where latitude represents the geographic latitude coordinate of the point cloud point. pix_x represents the horizontal coordinate of the point cloud point, and pix_y represents the vertical coordinate of the point cloud point. geo_information[3], geo_information[4], and geo_information[5] are all transformation coefficients.

[0091] Secondly, after determining the geographic longitude and latitude coordinates of the point cloud points in the road point cloud, the road point cloud can be converted to a road relative coordinate system.

[0092] As an example, the geographic longitude and latitude coordinates of the point cloud points in the road point cloud can be converted to the Xi'an 80 coordinate system through the coordinate conversion library Proj.4, as shown in Formula 3.

[0093] x,y=transform_wgs84_2_xian80(longitude,latitude); (Formula 3)

[0094] Among them, longitude represents the geographic longitude coordinate of the point cloud point, and latitude represents the geographic latitude coordinate of the point cloud point. x,y represent the horizontal coordinate and vertical coordinate of the point cloud point in the road point cloud in the Xi'an 80 coordinate system. transform_wgs84_2_xian80 is the conversion function provided by the coordinate conversion library Proj.4.

[0095] After converting the road point cloud into the geographic coordinates in the Xi'an 80 coordinate system, the road point cloud can be converted into the road relative coordinate system.

[0096] Specifically, the origin of the Xi'an 80 coordinate system can be recorded as x0, y0. The horizontal coordinate x of the point cloud point in the road point cloud in the Xi'an 80 coordinate system is subtracted from x0 to obtain x w, subtract y0 from the vertical coordinate y of the point cloud point in the Xi'an 80 coordinate system to get y w .

[0097] Finally, the height coordinate z in the three-dimensional coordinates of the point cloud point in the road point cloud is assigned to the height coordinate z of the point cloud point in the road relative coordinate system w , thus obtaining the geographic coordinates (x w ,y w , z w ).

[0098] After the road point cloud is converted to the road relative coordinate system to obtain the geographic coordinates (x1, y1, z1) of the point cloud points in the road relative coordinate system, the external parameters of the roadside camera can be used as variables, and based on the internal parameters of the roadside camera and the point cloud relative coordinates, the point cloud pixel coordinates of the road point cloud in the image coordinate system can be calculated.

[0099] Specifically, the camera extrinsic parameters can be expressed as a rotation matrix R and a translation matrix t, where R and t are both unknown variables.

[0100] The geographical coordinates (x w ,y w , z w ) is converted to the camera coordinate system, as shown in Formula 5.

[0101]

[0102] Among them, x c ,y c 、z c Represents the coordinates of the point cloud points in the road point cloud in the camera coordinate system. R represents the rotation matrix of the roadside camera, t represents the translation matrix of the roadside camera, x w ,y w , z w Represents the coordinates of the point cloud points in the road point cloud in the road relative coordinate system.

[0103] The coordinates of the point cloud points in the road point cloud in the camera coordinate system (x c ,y c 、z c ) is projected into the image coordinate system, as shown in Formula 6.

[0104]

[0105] Among them, u and v are the pixel coordinates of the point cloud points in the road point cloud after being projected into the image coordinate system. c ,y c 、z cIndicates the coordinates of the point cloud point in the road point cloud in the camera coordinate system. x 、f y is the focal length of the roadside camera, c x 、c y are the principal point coordinates of the road image.

[0106] Finally, an error formula can be constructed based on the point cloud pixel coordinates of the point cloud points in the same-name point pair and the pixel coordinates of the pixel points in the same-name point pair. The error formula is shown in Formula 4.

[0107]

[0108] Among them, ξ represents the external parameter, R represents the rotation matrix of the roadside camera, and t represents the translation matrix of the roadside camera. is a function that is used to find the combination of variables that minimizes the function value. The variable here is ξ, which is equivalent to R and t. i is the geographic coordinate of the point cloud point in the road relative coordinate system, which is equivalent to the aforementioned (x w ,y w , z w ). K represents the camera internal parameter. i Represents the pixel coordinates of the pixel point in the same-name point pair that corresponds to the point cloud point.

[0109] By minimizing the above function, the error between the pixel coordinates of the point cloud points in the same-name point pair after projecting them into the image coordinate system and the pixel coordinates of the corresponding pixel points in the same-name point pair is minimized, thereby solving the extrinsic parameters of the roadside camera.

[0110] In an optional method of the present application, based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and based on the geographic coordinates of the point cloud points, the mapping relationship between the pixel coordinates and the geographic coordinates of each pixel point in the road image is determined, including:

[0111] Based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and the correspondence between the projected point cloud points in the road point cloud and the projected pixel points in the road image is determined;

[0112] Based on the geographic coordinates of the projected point cloud points, determining a mapping relationship between the pixel coordinates of the corresponding projected pixel points and the geographic coordinates;

[0113] Based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is determined.

[0114] In the embodiment of the present application, after the external parameters of the roadside camera are determined, the road point cloud can be projected onto the road image based on the internal parameters of the roadside camera and the external parameters of the roadside camera.

[0115] Specifically, the geographic coordinates of the road point cloud in the road relative coordinate system can be converted to pixel coordinates in the image coordinate system based on the conversion formula between pixel coordinates and geographic coordinates, that is, camera depth × pixel coordinates = camera intrinsic parameters × camera extrinsic parameters × geographic boundary coordinates. The above conversion formula can be shown as Formula 8.

[0116]

[0117] where x w ,y w , z w Represents the coordinates of the point cloud points in the road point cloud in the road relative coordinate system. R represents the rotation matrix of the roadside camera, and t represents the translation matrix of the roadside camera. x 、f y is the focal length of the roadside camera, u0 represents the horizontal distance from the pixel coordinate of the center of the roadside image to the origin of the pixel coordinate system, and v0 represents the vertical distance from the pixel coordinate of the center of the roadside image to the origin of the pixel coordinate system. f is the image distance, and dx and dy represent the physical length of a pixel in the X and Y directions on the camera photosensitive plate, respectively. c Represents the depth value in the camera coordinate system.

[0118] After projecting the road point cloud onto the road image and obtaining the pixel coordinates of the point cloud points in the image coordinate system, they can be compared with the pixel coordinates of the pixel points in the road image to determine the pixel points that match the point cloud points, that is, the correspondence between the projected point cloud points and the projected pixel points in the road image is obtained.

[0119] As an example, for any point cloud point, its pixel coordinates in the image coordinate system are compared with the pixel coordinates of the pixel points in the road image. If there is a pixel point whose deviation value between the pixel coordinates and the pixel coordinates of the point cloud point in the image coordinate system is less than a preset value, it can be considered that the point cloud point matches the pixel point, that is, a set of corresponding projected point cloud points and projected pixel points are determined.

[0120] Through the above-mentioned projection process, the correspondence between the projected pixel points and the projected point cloud points can be determined, and the pixel coordinates of the corresponding projected pixel points can be determined based on the geographic coordinates of the projected point cloud points, thereby establishing a mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates.

[0121] Continuing from above Figure 3 In the example, Figure 5 It is a schematic diagram of the image obtained after the road point cloud is projected onto the road image.

[0122] In an optional manner of the embodiment of the present application, there are unprojected pixels in the road image, and the unprojected pixels are pixels other than the projected pixels in the road image. Based on the mapping relationship between the pixel coordinates of the projected pixels and the geographic coordinates, determining the mapping relationship between the pixel coordinates of each pixel in the road image and the geographic coordinates includes:

[0123] Based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates is determined, thereby determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates.

[0124] The density of the point cloud points in the road point cloud may be insufficient, resulting in a portion of unprojected pixels in the road image, which can be recorded as unprojected pixels. In order to ensure that the mapping relationship between the pixel coordinates and the geographic coordinates of each pixel in the road image is determined, the mapping relationship between the pixel coordinates and the geographic coordinates of the unprojected pixels needs to be supplemented.

[0125] Specifically, the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates may be used to supplement the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates.

[0126] In an optional manner of the embodiment of the present application, based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, determining the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates includes:

[0127] Determine N nearest neighboring pixel points of the unprojected pixel point from the projected pixel points, where N is a positive integer greater than 1;

[0128] Based on the mapping relationship between the pixel coordinates of the nearest neighbor pixel and the geographic coordinates, the mapping relationship between the pixel coordinates of the unprojected pixel and the geographic coordinates is determined.

[0129] In an embodiment of the present application, the geographic coordinates of adjacent pixels will have a certain continuity. Therefore, the N nearest neighboring pixels of the unprojected pixel can be determined from the projected pixel points, and the mapping relationship between the pixel coordinates of the nearest neighboring pixel points and the geographic coordinates can be used to supplement the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates.

[0130] As an example, N may be 4.

[0131] In an optional manner of the embodiment of the present application, based on the mapping relationship between the pixel coordinates of the nearest neighbor pixel and the geographic coordinates, determining the mapping relationship between the pixel coordinates of the non-projected pixel and the geographic coordinates includes:

[0132] The geographical coordinates corresponding to the pixel coordinates of each nearest neighbor pixel are averaged as the geographical coordinates corresponding to the pixel coordinates of the unprojected pixel, so as to determine the mapping relationship between the pixel coordinates of the unprojected pixel and the geographical coordinates.

[0133] In an embodiment of the present application, the average of the geographic coordinates corresponding to the pixel coordinates of each nearest neighbor pixel point can be used as the geographic coordinates corresponding to the pixel coordinates of the unprojected pixel point, thereby determining the mapping relationship between the pixel coordinates of the unprojected pixel point and the geographic coordinates, effectively ensuring that the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates can be determined.

[0134] As an example, N can be 4, and the pixel coordinates of the four nearest neighboring pixels of the unprojected pixel a are (u1, v1), (u2, v2), (u3, v3) and (u4, v4). The geographic coordinates corresponding to the pixel coordinate (u1, v1) are (X1, Y1, Z1), the geographic coordinates corresponding to the pixel coordinate (u2, v2) are (X2, Y2, Z2), the geographic coordinates corresponding to the pixel coordinate (u3, v3) are (X3, Y3, Z3), and the geographic coordinates corresponding to the pixel coordinate (u4, v4) are (X4, Y4, Z4). The geographic coordinates (Xa, Ya, Za) of the unprojected pixel a are obtained by taking the average of the geographic coordinates corresponding to the above four nearest neighboring pixels, where Xa = (X1 + X2 + X3 + X4) / 4; Ya = (Y1 + Y2 + Y3 + Y4) / 4; Za = (Z1 + Z2 + Z3 + Z4) / 4.

[0135] In an optional manner of the embodiment of the present application, determining N nearest neighboring pixel points of a non-projected pixel point from the projected pixel points includes:

[0136] A K-dimensional tree (kd-tree) is constructed based on the projected pixels, and a nearest neighbor search is performed on the kd-tree to determine the N nearest neighbor pixels of the unprojected pixels.

[0137] In the embodiment of the present application, when searching for the nearest neighbor pixel point, the projected pixel point kd-tree can be traversed through the kd-tree to quickly search for the N nearest neighbor pixel points.

[0138] In an optional manner of the embodiment of the present application, the road point cloud is collected by an unmanned aerial vehicle laser scanning device. After collecting the road point cloud, the unmanned aerial vehicle laser scanning device outputs the point cloud coordinates of the road point cloud. The road point cloud is pre-configured with a longitude and latitude coordinate conversion relationship. The above method also includes:

[0139] Based on the longitude and latitude coordinate conversion relationship, the geographic coordinates corresponding to the pixel coordinates of each pixel point in the road image are converted into longitude and latitude coordinates.

[0140] In the embodiment of the present application, when the unmanned aerial vehicle laser scanning device outputs the road point cloud, it also outputs the transformation information of each point cloud point to the geographic longitude and latitude coordinates. The transformation information includes the longitude and latitude coordinate conversion relationship. Based on the longitude and latitude coordinate conversion relationship, the geographic coordinates corresponding to each pixel can be converted to longitude and latitude coordinates. At the same time, the depth coordinates in the geographic coordinates corresponding to each pixel are assigned to the elevation coordinates, so that the longitude, latitude, and elevation information of the pixel point can be obtained.

[0141] In the embodiment of the present application, the road point cloud is collected by using an unmanned aerial vehicle laser scanning device, which has higher accuracy and higher collection efficiency than the traditional method of marking points with RTK equipment, and can also avoid safety hazards during operation. The road point cloud collected by using an unmanned aerial vehicle laser scanning device has altitude information, which can sense the uneven areas on the road and effectively reflect the actual situation of the target road. Incorporating altitude information into the calculation of the mapping relationship between pixel coordinates and geographic coordinates in the roadside image in this application can effectively improve the accuracy of the calculation results.

[0142] In addition, based on the solution provided in this application, it is possible to use drone-mounted laser scanning equipment to quickly collect road point clouds, and automatically calculate and determine the mapping relationship between the pixel coordinates of pixel points in the roadside image and the geographic coordinates, which helps to achieve rapid and convenient deployment of roadside equipment.

[0143] In an optional manner of the embodiment of the present application, after determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, the method further includes at least one of the following:

[0144] The mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is sent to the vehicles traveling on the target road, so that the vehicles can perceive the environment based on the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates;

[0145] Based on the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, a three-dimensional holographic road model of the target road is constructed.

[0146] In an embodiment of the present application, after determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, the mapping relationship can be sent to vehicles passing through the target road, so that the vehicles can perform environmental perception based on the mapping relationship, thereby improving the environmental perception effect of the vehicles.

[0147] Specifically, the mapping relationship between pixel coordinates and geographic coordinates can be sent to the vehicle together with the real-time road image captured by the roadside camera, so that the vehicle can perform target detection based on the real-time road image captured by the roadside camera, obtain the target detection result, and convert the target detection result to the real physical space based on the mapping relationship between pixel coordinates and geographic coordinates to effectively utilize the target detection result. The mapping relationship between pixel coordinates and geographic coordinates provided in this solution can use roadside perception to supplement vehicle perception, improve the vehicle's environmental perception effect, and improve driving safety.

[0148] In an embodiment of the present application, after determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, the mapping relationship can be used to construct a three-dimensional model of the holographic road to achieve effective perception of the environment and vehicle traffic conditions within the target road.

[0149] As an optional solution, laser radar can also be used in this application, based on the simultaneous localization and mapping (SLAM) technology, combined with camera image acquisition to generate a color road point cloud to replace the road point cloud collected by the drone-mounted laser scanning equipment.

[0150] Based on Figure 1 The same principle as shown in the method, Figure 6 A schematic diagram of the structure of a geographic coordinate positioning device for a roadside image provided by an embodiment of the present application is shown. Figure 6 As shown, the geographic coordinate positioning device 60 of the roadside image may include:

[0151] The data acquisition module 610 is used to acquire a road point cloud and a road image collected for a target road. Each point in the road point cloud corresponds to a geographical coordinate of the point cloud point. The road image is collected by a roadside camera.

[0152] The same-name point pair acquisition module 620 is used to acquire the same-name point pairs formed by the point cloud points and the pixel points in the road image;

[0153] An external parameter determination module 630, configured to determine the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera;

[0154] The positioning module 640 is used to project the road point cloud onto the road image based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, and determine the mapping relationship between the pixel coordinates and the geographic coordinates of each pixel point in the road image based on the geographic coordinates of the point cloud points.

[0155] The device provided by the embodiment of the present application obtains a road point cloud and a road image collected for a target road, each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; obtains a pair of points of the same name consisting of point cloud points and pixel points in the road image; determines the external parameters of the roadside camera based on the point pairs of the same name and the internal parameters of the roadside camera; projects the road point cloud onto the road image based on the internal parameters of the roadside camera and the external parameters of the roadside camera, and determines the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the point cloud point geographic coordinates. In this solution, by projecting the road point cloud onto the road image, and based on the geographic coordinates of the road point cloud, a mapping relationship between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, which provides a basis for environmental perception based on the road image collected by the roadside camera.

[0156] Optionally, the positioning module is specifically used for:

[0157] Based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and the correspondence between the projected point cloud points in the road point cloud and the projected pixel points in the road image is determined;

[0158] Based on the geographic coordinates of the projected point cloud points, determining a mapping relationship between the pixel coordinates of the corresponding projected pixel points and the geographic coordinates;

[0159] Based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is determined.

[0160] Optionally, there are unprojected pixels in the road image, and the unprojected pixels are pixels other than the projected pixels in the road image. When the positioning module determines the mapping relationship between the pixel coordinates of each pixel in the road image and the geographic coordinates based on the mapping relationship between the pixel coordinates of the projected pixels and the geographic coordinates, it is specifically used to:

[0161] Based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates is determined, thereby determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates.

[0162] Optionally, when the positioning module determines the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, it is specifically used to:

[0163] Determine N nearest neighboring pixel points of the unprojected pixel point from the projected pixel points, where N is a positive integer greater than 1;

[0164] Based on the mapping relationship between the pixel coordinates of the nearest neighbor pixel and the geographic coordinates, the mapping relationship between the pixel coordinates of the unprojected pixel and the geographic coordinates is determined.

[0165] Optionally, when the positioning module determines the mapping relationship between the pixel coordinates of the non-projected pixel points and the geographic coordinates based on the mapping relationship between the pixel coordinates of the nearest neighboring pixel points and the geographic coordinates, it is specifically used to:

[0166] The geographical coordinates corresponding to the pixel coordinates of each nearest neighbor pixel are averaged as the geographical coordinates corresponding to the pixel coordinates of the unprojected pixel, so as to determine the mapping relationship between the pixel coordinates of the unprojected pixel and the geographical coordinates.

[0167] Optionally, when the positioning module determines N nearest neighboring pixels of the unprojected pixel from the projected pixel, it is specifically used to:

[0168] A kd-tree is constructed based on the projected pixels, and a nearest neighbor search is performed on the kd-tree to determine the N nearest neighbor pixels of the unprojected pixels.

[0169] Optionally, the same-name point pair acquisition module is specifically used for:

[0170] De-distorting the roadside image based on the internal parameters of the roadside camera to obtain a de-distorted roadside image;

[0171] Based on the dedistorted roadside image and the road point cloud, a calibration operation is performed to determine the same-name point pairs consisting of point cloud points and pixel points.

[0172] Optionally, the external parameter determination module is specifically used for:

[0173] Obtain the relative coordinates of the road point cloud in the road relative coordinate system. The road relative coordinate system takes the point in the target road as the origin;

[0174] The external parameters of the roadside camera are used as variables, and based on the internal parameters of the roadside camera and the relative coordinates of the point cloud, the pixel coordinates of the road point cloud in the image coordinate system are calculated;

[0175] An error formula is constructed based on the point cloud pixel coordinates of the point cloud points in the same-name point pair and the pixel coordinates of the pixel points in the same-name point pair;

[0176] Minimize the error formula to determine the external parameters of the roadside camera.

[0177] Optionally, the road point cloud is collected by a laser scanning device carried by an unmanned aerial vehicle. After collecting the road point cloud, the laser scanning device carried by the unmanned aerial vehicle outputs point cloud coordinates of the road point cloud. The road point cloud is pre-configured with a longitude and latitude coordinate conversion relationship. The method further includes a conversion module, which is used to:

[0178] Based on the longitude and latitude coordinate conversion relationship, the horizontal coordinate and the vertical coordinate in the geographic coordinate corresponding to the pixel coordinate of each pixel point in the road image are converted into the longitude and latitude coordinate;

[0179] The depth coordinates in the geographic coordinates corresponding to the pixel coordinates of each pixel point in the road image are used as elevation coordinates.

[0180] Optionally, the above device further includes a mapping relationship processing module, which is used for at least one of the following:

[0181] After determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is sent to vehicles traveling on the target road, so that the vehicles can perform environmental perception based on the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates;

[0182] After determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, a three-dimensional model of the holographic road of the target road is constructed based on the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates.

[0183] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment. The device embodiment described above is merely schematic, in which the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative work.

[0184] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0185] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any one of the method embodiments described above.

[0186] Compared with the prior art, the computer-readable storage medium provided in the embodiment of the present application obtains a road point cloud and a road image collected for a target road, each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; obtains a pair of points of the same name consisting of point cloud points and pixel points in the road image; determines the external parameters of the roadside camera based on the point pairs of the same name and the internal parameters of the roadside camera; projects the road point cloud onto the road image based on the internal parameters of the roadside camera and the external parameters of the roadside camera, and determines the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the point cloud point geographic coordinates. In this solution, by projecting the road point cloud onto the road image, and based on the geographic coordinates of the road point cloud, a mapping relationship between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, which provides a basis for environmental perception based on the road image collected by the roadside camera.

[0187] The present application also provides an electronic device, including:

[0188] one or more processors; and

[0189] A memory associated with the one or more processors, the memory being used to store program instructions, which, when read and executed by the one or more processors, executes the steps of any one of the method embodiments described above.

[0190] Compared with the prior art, the electronic device provided by the embodiment of the present application obtains a road point cloud and a road image collected for a target road, each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; obtains a pair of points of the same name consisting of point cloud points and pixel points in the road image; determines the external parameters of the roadside camera based on the point cloud and the internal parameters of the roadside camera; projects the road point cloud onto the road image based on the internal parameters of the roadside camera and the external parameters of the roadside camera, and determines the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the point cloud point geographic coordinates. In this solution, by projecting the road point cloud onto the road image, and based on the geographic coordinates of the road point cloud, a mapping relationship between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, which provides a basis for environmental perception based on the road image collected by the roadside camera.

[0191] As an example, Figure 7 A schematic diagram of the structure of an electronic device applicable to the embodiment of the present application is shown. Figure 7As shown, the electronic device 70 includes: a processor 710 and a memory 730. The processor 710 and the memory 730 are connected, such as through a bus 720. Optionally, the electronic device 70 may also include a transceiver 740. It should be noted that in actual applications, the transceiver 740 is not limited to one, and the structure of the electronic device 70 does not constitute a limitation on the embodiments of the present application.

[0192] The processor 710 is used in the embodiment of the present application to implement the method shown in the above method embodiment. The transceiver 740 may include a receiver and a transmitter. The transceiver 740 is used in the embodiment of the present application to implement the function of the electronic device of the embodiment of the present application communicating with other devices when executed.

[0193] The processor 710 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of this application. The processor 710 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0194] The bus 720 may include a path to transmit information between the above components. The bus 720 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 720 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0195] The memory 730 can be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, optical disk storage (including compressed optical disk, laser disk, optical disk, digital versatile disk, Blu-ray disk, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0196] Optionally, the memory 730 is used to store application code for executing the solution of the present application, and the execution is controlled by the processor 710. The processor 710 is used to execute the application code stored in the memory 730 to implement the steps of any one of the methods in the aforementioned method embodiments.

[0197] The present application also provides a computer program product, including a computer program, which implements the steps of any one of the method embodiments described above when executed by a processor.

[0198] Compared with the prior art, the computer program product provided in the embodiment of the present application obtains a road point cloud and a road image collected for a target road, each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; obtains a pair of points of the same name consisting of point cloud points and pixel points in the road image; determines the external parameters of the roadside camera based on the point cloud and the internal parameters of the roadside camera; projects the road point cloud onto the road image based on the internal parameters of the roadside camera and the external parameters of the roadside camera, and determines the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates based on the point cloud point geographic coordinates. In this solution, by projecting the road point cloud onto the road image, and based on the geographic coordinates of the road point cloud, a mapping relationship between the pixel coordinates of the pixel points in the road image and the geographic coordinates can be established, which provides a basis for environmental perception based on the road image collected by the roadside camera.

[0199] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0200] The above descriptions are only some embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for geographical coordinate positioning of roadside images, characterized in that: include: Acquire a road point cloud and a road image collected for a target road, wherein each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; Acquire a point pair with the same name formed by the point cloud point and the pixel point in the road image; Determining the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera; Based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and based on the geographic coordinates of the point cloud points, a mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is determined.

2. The method according to claim 1, characterized in that The method of projecting the road point cloud onto the road image based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, and determining the mapping relationship between the pixel coordinates and the geographic coordinates of each pixel point in the road image based on the geographic coordinates of the point cloud points, includes: Based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, the road point cloud is projected onto the road image, and a correspondence between projected point cloud points in the road point cloud and projected pixel points in the road image is determined; Based on the geographic coordinates of the projected point cloud points, determining a mapping relationship between the pixel coordinates of the corresponding projected pixel points and the geographic coordinates; Based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates is determined.

3. The method according to claim 2, characterized in that There are unprojected pixels in the road image, and the unprojected pixels are pixels in the road image other than the projected pixels. The determining the mapping relationship between the pixel coordinates of each pixel in the road image and the geographic coordinates based on the mapping relationship between the pixel coordinates of the projected pixels and the geographic coordinates includes: Based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates, the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates is determined, thereby determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates.

4. The method according to claim 3, characterized in that The determining the mapping relationship between the pixel coordinates of the unprojected pixel points and the geographic coordinates based on the mapping relationship between the pixel coordinates of the projected pixel points and the geographic coordinates includes: Determine N nearest neighboring pixel points of the non-projected pixel point from the projected pixel points, where N is a positive integer greater than 1; Based on the mapping relationship between the pixel coordinates of the nearest neighbor pixel and the geographic coordinates, the mapping relationship between the pixel coordinates of the non-projected pixel and the geographic coordinates is determined.

5. The method according to claim 4, characterized in that The determining the mapping relationship between the pixel coordinates of the non-projected pixel and the geographic coordinates based on the mapping relationship between the pixel coordinates of the nearest neighbor pixel and the geographic coordinates includes: The geographical coordinates corresponding to the pixel coordinates of each of the nearest neighboring pixels are averaged as the geographical coordinates corresponding to the pixel coordinates of the non-projected pixel, so as to determine a mapping relationship between the pixel coordinates of the non-projected pixel and the geographical coordinates.

6. The method according to claim 4, characterized in that The determining of N nearest neighboring pixel points of the non-projected pixel point from the projected pixel points comprises: A K-dimensional tree kd-tree is constructed based on the projected pixel points, and a nearest neighbor search is performed on the kd-tree to determine N nearest neighboring pixel points of the unprojected pixel point.

7. The method according to claim 1, characterized in that The obtaining of the same-name point pairs formed by the point cloud points and the pixel points in the road image comprises: Performing dedistortion processing on the roadside image based on the internal parameters of the roadside camera to obtain the dedistorted roadside image; A calibration operation is performed based on the dedistorted roadside image and the road point cloud to determine the same-name point pairs formed by the point cloud points and the pixel points.

8. The method according to claim 1, characterized in that The determining the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera includes: Obtaining relative coordinates of the road point cloud in a road relative coordinate system, wherein the road relative coordinate system takes the point in the target road as an origin; Taking the external parameters of the roadside camera as variables, and based on the internal parameters of the roadside camera and the relative coordinates of the point cloud, calculating the point cloud pixel coordinates of the road point cloud in the image coordinate system; Constructing an error formula based on the point cloud pixel coordinates of the point cloud points in the same-name point pair and the pixel coordinates of the pixel points in the same-name point pair; The error formula is minimized to determine the external parameters of the roadside camera.

9. The method according to claim 1, characterized in that: The road point cloud is collected by an unmanned aerial vehicle laser scanning device, and after collecting the road point cloud, the unmanned aerial vehicle laser scanning device outputs the point cloud coordinates of the road point cloud, and the road point cloud is pre-configured with a longitude and latitude coordinate conversion relationship. The method also includes: Based on the longitude and latitude coordinate conversion relationship, the horizontal coordinate and the vertical coordinate in the geographic coordinate corresponding to the pixel coordinate of each pixel point in the road image are converted into longitude and latitude coordinates; The depth coordinates in the geographic coordinates corresponding to the pixel coordinates of each pixel point in the road image are used as elevation coordinates.

10. The method according to any one of claims 1 to 9, characterized in that After determining the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, the method further includes at least one of the following: Sending the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates to vehicles traveling on the target road, so that the vehicles can perform environmental perception based on the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates; Based on the mapping relationship between the pixel coordinates of each pixel point in the road image and the geographic coordinates, a three-dimensional model of the holographic road of the target road is constructed.

11. A geographic coordinate positioning device for roadside images, characterized in that: include: A data acquisition module, used to acquire a road point cloud and a road image collected for a target road, wherein each point cloud point in the road point cloud corresponds to a point cloud point geographic coordinate, and the road image is collected by a roadside camera; A same-name point pair acquisition module, used to acquire same-name point pairs consisting of the point cloud points and pixel points in the road image; An external parameter determination module, used to determine the external parameters of the roadside camera based on the same-name point pairs and the internal parameters of the roadside camera; A positioning module is used to project the road point cloud onto the road image based on the intrinsic parameters of the roadside camera and the extrinsic parameters of the roadside camera, and to determine the mapping relationship between the pixel coordinates and the geographic coordinates of each pixel point in the road image based on the geographic coordinates of the point cloud points.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

13. An electronic device, characterized in that: include: one or more processors; as well as A memory associated with the one or more processors, the memory being used to store program instructions, wherein the program instructions, when read and executed by the one or more processors, execute the steps of the method described in any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images