Image generation method and device

The image generation method for digital orthophoto maps addresses the high costs and complexity of existing methods by using camera-based coordinate conversion to generate precise maps without lidar, enhancing efficiency and robustness.

CN113963122BActive Publication Date: 2024-12-03BEIJING SANKUAI ONLINE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111124195.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-09-24
Publication Date
2024-12-03
Estimated Expiration
2041-09-24

AI Technical Summary

Technical Problem

Existing methods for constructing digital orthophoto maps of roads are costly due to the need for synchronization of lidar and camera data, and the visual SLAM algorithm is complex and lacks robustness, especially in areas with missing texture.

Method used

An image generation method that involves acquiring road images with a target camera, performing coordinate conversion based on the correspondence between the image and geographic coordinate systems, and generating high-precision digital orthophoto maps without relying on lidar point cloud data, using a camera calibration process to establish this correspondence.

Benefits of technology

Reduces construction costs and simplifies the process by eliminating the need for lidar synchronization, while ensuring robustness and fast road mapping.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113963122B_ABST
    Figure CN113963122B_ABST
Patent Text Reader

Abstract

The embodiments of the present disclosure provide an image generation method and device. The method includes: acquiring a road image collected by a target camera; performing coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, and determining the pixel geographic coordinates corresponding to each image pixel; based on the pixel geographic coordinates and pixel information, generating a high-precision digital road orthophoto map of the road area corresponding to the road image. The embodiments of the present disclosure can reduce the cost of constructing a digital orthophoto map, and reduce the operation of synchronizing with a laser radar, and the algorithm has low complexity, strong robustness, and a fast road construction speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of map drawing based on unmanned driving, and more particularly to an image generation method and device. Background Art

[0002] At present, the mapping of high-precision maps is mainly based on two data sources. One is the laser reflectivity base map, which is generated by laser point cloud data and is mainly used to produce a complete version of the high-precision map. The other is the digital orthophoto map, which can be generated by combining laser point cloud data and RGB (Red / Green / Blue) image data, and is mainly used for updating high-precision maps. At present, some high-precision map manufacturers or autonomous driving companies are trying to unify the data for high-precision map mapping and updates into digital orthophoto maps. On the one hand, because the digital orthophoto map contains richer road scene information, it can reduce the difficulty of manual mapping; on the other hand, the digital orthophoto map fuses laser and RGB images, which is more conducive to the development of algorithms.

[0003] There are currently two mainstream methods for producing digital orthophotos. One is based on laser point cloud data to obtain the geographic coordinates of each pixel on the ground in the corresponding RGB image, and then fuse different images according to their geographic coordinates to generate a digital orthophoto of the road area; the other is to build digital orthophotos based on a visual approach.

[0004] The existing technology for constructing orthophoto digital image maps of roads has the following two problems: 1. The method based on laser point cloud and RGB image needs to combine lidar and camera data, which firstly leads to high cost and secondly requires synchronization of sensors such as lidar and camera, which makes the process complicated; 2. The algorithm of the method based on visual SLAM is difficult to implement and has low robustness. It has high requirements on image features and cannot guarantee the effect on sections with missing texture. Summary of the invention

[0005] The embodiments of the present disclosure provide an image generation method and device to reduce the cost of constructing an orthophoto digital image of a road, reduce synchronization operations with a laser radar, and increase the speed of road construction.

[0006] According to a first aspect of an embodiment of the present disclosure, there is provided an image generating method, comprising:

[0007] Acquire a road image collected by a target camera;

[0008] According to the correspondence between the image coordinate system and the geographic coordinate system, coordinate conversion processing is performed on the pixel coordinates of each image pixel in the road image to determine the pixel geographic coordinates corresponding to each image pixel;

[0009] Based on the pixel geographic coordinates and pixel information, a high-precision road digital orthophoto map of the road area corresponding to the road image is generated.

[0010] Optionally, before acquiring the road image acquired by the target camera, the method further includes:

[0011] Performing calibration processing on the target camera based on the calibration grid;

[0012] Acquire a regional image obtained by photographing the calibration grid by the target camera; the calibration grid has the same size as the regional image;

[0013] Based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid, a corresponding relationship between the image coordinate system and the geographic coordinate system is established.

[0014] Optionally, establishing a correspondence between an image coordinate system and a geographic coordinate system based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid includes:

[0015] Obtaining first corner point coordinates of four corner points of the calibration grid and second corner point coordinates of four corner points of the regional image;

[0016] Obtaining the lateral distance and the longitudinal distance between the target camera and the calibration grid;

[0017] Based on the first corner point coordinates, the second corner point coordinates, the horizontal distance, and the vertical distance, a corresponding relationship between the image coordinate system and the geographic coordinate system is determined.

[0018] Optionally, generating a high-precision digital orthophoto map of a road area corresponding to the road image based on the pixel geographic coordinates and the pixel information includes:

[0019] Determining a map area corresponding to the road image on the electronic map according to the pixel geographic coordinates;

[0020] Based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, the pixel point information on the road image is filled into the map area to generate the high-precision road digital orthophoto map.

[0021] Optionally, the step of filling the map area with pixel information on the road image based on the timestamp information corresponding to each pixel on the road image and the distance information between each pixel and the target camera to generate the high-precision road digital orthophoto map includes:

[0022] Determine a pixel weight corresponding to each pixel point based on the timestamp information and the distance information;

[0023] Based on the pixel weight, the RGB value of each pixel is adjusted to obtain an adjusted RGB value;

[0024] According to the pixel geographic coordinates corresponding to each pixel point, the adjusted RGB value is filled in the map area to obtain the high-precision road digital orthophoto map.

[0025] According to a second aspect of an embodiment of the present disclosure, there is provided an image generating device, including:

[0026] A road image acquisition module is used to acquire road images collected by a target camera;

[0027] A geographic coordinate determination module, used to perform coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the corresponding relationship between the image coordinate system and the geographic coordinate system, and determine the pixel geographic coordinates corresponding to each image pixel;

[0028] The orthophoto generation module is used to generate a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates and pixel information.

[0029] Optionally, the device further comprises:

[0030] A camera calibration module, used for calibrating the target camera based on a calibration grid;

[0031] A regional image acquisition module, used to acquire a regional image obtained by photographing the calibration grid by the target camera; the calibration grid has the same size as the regional image;

[0032] The corresponding relationship establishing module is used to establish the corresponding relationship between the image coordinate system and the geographic coordinate system based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid.

[0033] Optionally, the correspondence establishing module includes:

[0034] A corner point coordinate acquisition unit, used to acquire first corner point coordinates of four corner points of the calibration grid and second corner point coordinates of four corner points of the regional image;

[0035] A horizontal and vertical distance acquisition unit, used to acquire the horizontal distance and vertical distance between the target camera and the calibration grid;

[0036] A correspondence determination unit is used to determine the correspondence between the image coordinate system and the geographic coordinate system based on the first corner point coordinates, the second corner point coordinates, the horizontal distance and the vertical distance.

[0037] Optionally, the orthophoto generation module includes:

[0038] A map area determination unit, used to determine the map area corresponding to the road image on the electronic map according to the pixel geographic coordinates;

[0039] The orthophoto generating unit is used to fill the pixel point information on the road image into the map area based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, so as to generate the high-precision road digital orthophoto map.

[0040] Optionally, the orthophoto generating unit comprises:

[0041] A pixel weight determination subunit, used to determine a pixel weight corresponding to each pixel point based on the timestamp information and the distance information;

[0042] The RGB value acquisition subunit is used to adjust the RGB value of each pixel based on the pixel weight to obtain an adjusted RGB value;

[0043] The orthophoto acquisition subunit is used to fill the adjusted RGB value into the map area according to the pixel geographic coordinates corresponding to each pixel point to obtain the high-precision road digital orthophoto map.

[0044] According to a third aspect of an embodiment of the present disclosure, there is provided an electronic device, including:

[0045] A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, any of the above-mentioned image generation methods is implemented.

[0046] According to a fourth aspect of an embodiment of the present disclosure, a readable storage medium is provided. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute any of the above-mentioned image generating methods.

[0047] The embodiments of the present disclosure provide an image generation method and device, which obtains a road image collected by a target camera, performs coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, determines the pixel geographic coordinates corresponding to each image pixel, and generates a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates. The embodiments of the present disclosure complete the construction of the digital orthophoto map by pre-calibrating the correspondence between the image coordinate system and the geographic coordinate system, and fusing and splicing the image area with geographic coordinates, without combining laser radar point cloud data, which can reduce the cost of constructing the digital orthophoto map, and reduce the operation of synchronizing with the laser radar, and the algorithm has low complexity, strong robustness, and fast road construction speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the drawings required for use in the description of the embodiments of the present disclosure will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0049] Figure 1 A flowchart of a method for generating an image according to an embodiment of the present disclosure;

[0050] Figure 2 A flowchart of another image generation method provided by an embodiment of the present disclosure;

[0051] Figure 3 A schematic diagram of establishing a corresponding relationship provided in an embodiment of the present disclosure;

[0052] Figure 4 A schematic diagram of constructing a digital orthophoto according to an embodiment of the present disclosure;

[0053] Figure 5 A schematic diagram of the structure of an image generating device provided in an embodiment of the present disclosure;

[0054] Figure 6 A schematic diagram of the structure of another image generating device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0055] The following will be combined with the drawings in the embodiments of the present disclosure to clearly and completely describe the technical solutions in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, rather than all of the embodiments. Based on the embodiments in the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the embodiments of the present disclosure.

[0056] Embodiment 1

[0057] Reference Figure 1 , which shows a flowchart of a method for generating an image according to an embodiment of the present disclosure, such as Figure 1 As shown, the image generation method may specifically include the following steps:

[0058] Step 101: Acquire a road image captured by a target camera.

[0059] The embodiments of the present disclosure may be applied to the scene of constructing a digital orthophoto of a road.

[0060] This embodiment can be applied to an autonomous driving system, that is, to realize the construction of a road image through an autonomous driving device.

[0061] The target camera refers to a camera used to collect road images. In this example, the target camera may be a camera installed on an autonomous driving device, such as a camera installed on an unmanned vehicle.

[0062] In this example, the target camera may be one camera or multiple cameras, etc. Specifically, it may be determined according to business requirements, and this embodiment does not impose any limitation on this.

[0063] Road images refer to images collected by a target camera on the road to be used to generate a digital orthophoto of the road.

[0064] When a digital orthophoto of a road needs to be constructed, the road image can be obtained by running an autonomous driving device on the road and taking images of the road through a target camera set on the autonomous driving device.

[0065] After the road image collected by the target camera is acquired, step 102 is executed.

[0066] Step 102: According to the correspondence between the image coordinate system and the geographic coordinate system, coordinate conversion processing is performed on the pixel coordinates of each image pixel in the road image to determine the pixel geographic coordinates corresponding to each image pixel.

[0067] A correspondence between the image coordinate system and the geographic coordinate system is pre-established in the autonomous driving device system. The process of establishing the correspondence will be described in detail in the following embodiment 2, and will not be elaborated in this embodiment.

[0068] Pixel coordinates refer to the coordinates of each image pixel on the road image in the image coordinate system.

[0069] Pixel geographic coordinates refer to the coordinates of each image pixel on the road image in the geographic coordinate system.

[0070] After acquiring the road image collected by the target camera, the pixel coordinates of each image pixel in the road image can be converted into coordinates according to the corresponding relationship between the image coordinate system and the geographic coordinate system to obtain the pixel geographic coordinates corresponding to each image pixel. In a specific implementation, the corresponding relationship between the image coordinate system and the geographic coordinate system can be represented by a coordinate conversion matrix. After acquiring the pixel coordinates of each image pixel in the road image, the pixel coordinates and the coordinate conversion matrix can be combined to obtain the geographic coordinates corresponding to each image pixel.

[0071] Of course, in a specific implementation, other methods may be used to achieve the conversion between pixel coordinates and geographic coordinates. Specifically, it may be determined according to business requirements, and this embodiment does not limit this.

[0072] After coordinate conversion processing is performed on the pixel coordinates of each image pixel in the road image to obtain the pixel geographic coordinates corresponding to each image pixel, step 103 is executed.

[0073] Step 103: Based on the pixel geographic coordinates and pixel information, a high-precision digital road orthophoto map of the road area corresponding to the road image is generated.

[0074] After obtaining the pixel geographic coordinates of each image pixel on the road image, a high-precision digital road orthophoto map of the road area corresponding to the road image can be generated based on the pixel geographic coordinates and the pixel information of each image pixel. Specifically, the pixel information of each image pixel can be filled in the corresponding position on the electronic map in combination with the pixel geographic coordinates of each image pixel to obtain a high-precision digital road orthophoto map.

[0075] The embodiments of the present disclosure complete the construction of a digital orthophoto map by pre-calibrating the correspondence between the image coordinate system and the geographic coordinate system, and fusing and splicing the image areas with geographic coordinates. This does not require the use of lidar point cloud data, and can reduce the cost of constructing a digital orthophoto map.

[0076] The image generation method provided by the embodiment of the present disclosure obtains a road image collected by a target camera, and performs coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, determines the pixel geographic coordinates corresponding to each image pixel, and generates a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates. The embodiment of the present disclosure completes the construction of the digital orthophoto map by pre-calibrating the correspondence between the image coordinate system and the geographic coordinate system, and fusing and splicing the image area with geographic coordinates, without combining laser radar point cloud data, which can reduce the cost of constructing the digital orthophoto map, and reduce the operation of synchronization with the laser radar, and the algorithm has low complexity, strong robustness, and fast road construction speed.

[0077] Embodiment 2

[0078] Reference Figure 2 , shows a flowchart of another image generation method provided by an embodiment of the present disclosure, such as Figure 2 As shown, the image generation method may specifically include the following steps:

[0079] Step 201: calibrate the target camera based on the calibration grid.

[0080] The embodiments of the present disclosure may be applied to the scene of constructing a digital orthophoto of a road.

[0081] This embodiment can be applied to an autonomous driving system, that is, to realize the construction of a road image through an autonomous driving device.

[0082] The calibration grid refers to a grid used to calibrate the target camera. In this example, the calibration grid may be a grid in the form of a chessboard or the like. Specifically, the specific form of the calibration grid may be determined according to business requirements, and this embodiment does not impose any limitation on this.

[0083] When it is necessary to establish a correspondence between the image coordinate system and the geographic coordinate system, the target camera can be calibrated based on the calibration grid.

[0084] After the target camera is calibrated based on the calibration grid, step 202 is executed.

[0085] Step 202: Acquire a regional image obtained by photographing the calibration grid with the target camera; the calibration grid has the same size as the regional image.

[0086] After the target camera is calibrated based on the calibration grid, the target camera can shoot the calibration grid to obtain a regional image, and the calibration grid and the regional image have the same size.

[0087] After obtaining the regional image obtained by photographing the calibration grid by the target camera, step 203 is executed.

[0088] Step 203: Based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid, a corresponding relationship between the image coordinate system and the geographic coordinate system is established.

[0089] After obtaining the regional image by photographing the calibration grid with the target camera, the correspondence between the image coordinate system and the geographic coordinate system can be established based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid. Specifically, it can be described in detail in combination with the following specific implementation method.

[0090] In a specific implementation of the present disclosure, the above step 203 may include:

[0091] Sub-step S1: Obtain the first corner point coordinates of the four corner points of the calibration grid and the second corner point coordinates of the four corner points of the regional image.

[0092] The process of establishing the correspondence between the image coordinate system and the geographic coordinate system can be combined Figure 2 This is described in detail as follows.

[0093] The first corner point coordinates are the coordinates of the four corner points on the index grid.

[0094] The second corner point coordinates refer to the coordinates of the four corner points on the region image.

[0095] After the target camera is used to capture the calibration grid to obtain the regional image, the first corner point coordinates of the four corner points and the second corner point coordinates of the four corner points of the regional image can be obtained.

[0096] After obtaining the coordinates of the first corner point and the coordinates of the second corner point, sub-step S2 is performed.

[0097] Sub-step S2: Obtain the lateral distance and longitudinal distance between the target camera and the calibration grid.

[0098] After obtaining the coordinates of the first corner point and the second corner point, the horizontal distance and the vertical distance between the target camera and the calibration grid can be obtained, and then sub-step S3 is executed.

[0099] Sub-step S3: Determine the correspondence between the image coordinate system and the geographic coordinate system based on the first corner point coordinates, the second corner point coordinates, the horizontal distance and the vertical distance.

[0100] After obtaining the first corner point coordinates, the second corner point coordinates, the horizontal distance and the vertical distance, the corresponding relationship between the image coordinate system and the geographic coordinate system can be determined based on the first corner point coordinates, the second corner point coordinates, the horizontal distance and the vertical distance. Figure 2 As shown, after the regional image is captured, corner point detection is performed first, and then corner point transformation is performed based on the corner point detection result to obtain a corner point transformation matrix. Then, the tires of the autonomous driving equipment are fixed on the calibration table. Then, the lateral distance and longitudinal distance between the camera and the calibration grid can be obtained. Then, the correspondence between the image coordinate system and the geographic coordinate system is established in combination with the corner point transformation matrix and the longitudinal distance and lateral distance.

[0101] After the corresponding relationship between the image coordinate system and the geographic coordinate system is established, step 204 is executed.

[0102] Step 204: Acquire a road image captured by a target camera.

[0103] The target camera refers to a camera used to collect road images. In this example, the target camera may be a camera installed on an autonomous driving device, such as a camera installed on an unmanned vehicle.

[0104] In this example, the target camera may be one camera or multiple cameras, etc. Specifically, it may be determined according to business requirements, and this embodiment does not impose any limitation on this.

[0105] Road images refer to images collected by a target camera on the road to be used to generate a digital orthophoto of the road.

[0106] When a digital orthophoto of a road needs to be constructed, the road image can be obtained by running an autonomous driving device on the road and taking images of the road through a target camera set on the autonomous driving device.

[0107] After the road image collected by the target camera is acquired, step 205 is executed.

[0108] Step 205: According to the correspondence between the image coordinate system and the geographic coordinate system, coordinate conversion processing is performed on the pixel coordinates of each image pixel in the road image to determine the pixel geographic coordinates corresponding to each image pixel.

[0109] Pixel coordinates refer to the coordinates of each image pixel on the road image in the image coordinate system.

[0110] Pixel geographic coordinates refer to the coordinates of each image pixel on the road image in the geographic coordinate system.

[0111] After acquiring the road image collected by the target camera, the pixel coordinates of each image pixel in the road image can be converted into coordinates according to the corresponding relationship between the image coordinate system and the geographic coordinate system to obtain the pixel geographic coordinates corresponding to each image pixel. In a specific implementation, the corresponding relationship between the image coordinate system and the geographic coordinate system can be represented by a coordinate conversion matrix. After acquiring the pixel coordinates of each image pixel in the road image, the pixel coordinates and the coordinate conversion matrix can be combined to obtain the geographic coordinates corresponding to each image pixel.

[0112] Of course, in a specific implementation, other methods may be used to achieve the conversion between pixel coordinates and geographic coordinates. Specifically, it may be determined according to business requirements, and this embodiment does not limit this.

[0113] After coordinate conversion processing is performed on the pixel coordinates of each image pixel in the road image to obtain the pixel geographic coordinates corresponding to each image pixel, step 206 is executed.

[0114] Step 206: Determine the map area corresponding to the road image on the electronic map according to the pixel geographic coordinates.

[0115] After the pixel geographic coordinates of each image pixel on the road image are obtained, the map area corresponding to the road image on the electronic map can be determined according to the pixel geographic coordinates.

[0116] After the map area corresponding to the road image on the electronic map is determined according to the pixel geographic coordinates, step 207 is executed.

[0117] Step 207: Based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, the pixel point information on the road image is filled into the map area to generate the high-precision road digital orthophoto map.

[0118] After determining the map area corresponding to the road image on the electronic map according to the pixel geographic coordinates, the pixel information on the road image can be filled into the map area based on the timestamp information corresponding to each pixel on the road image and the distance information between each pixel and the target camera to generate a high-precision digital orthophoto of the road. Specifically, it can be described in detail in combination with the following specific implementation methods.

[0119] In another specific implementation of the present disclosure, the above step 207 may include:

[0120] Sub-step M1: Determine the pixel weight corresponding to each pixel point based on the timestamp information and the distance information.

[0121] In this embodiment, the pixel weight refers to a weight used to adjust the RGB value of a pixel.

[0122] After the map area corresponding to the road image on the electronic map is determined according to the pixel geographic coordinates, the pixel weight corresponding to each pixel point may be determined based on the timestamp information and the distance information.

[0123] After the pixel weight corresponding to each pixel point on the road image is obtained, sub-step M2 is executed.

[0124] Sub-step M2: Based on the pixel weight, the RGB value of each pixel is adjusted to obtain an adjusted RGB value.

[0125] After obtaining the pixel weight corresponding to each pixel point on the road image, the RGB value of each pixel point can be adjusted according to the pixel weight corresponding to each pixel point to obtain the adjusted RGB value corresponding to each pixel point.

[0126] After adjusting the RGB value of each pixel point based on the pixel weight to obtain the adjusted RGB value, sub-step M3 is executed.

[0127] Sub-step M3: filling the map area with the adjusted RGB value according to the pixel geographic coordinates corresponding to each pixel point to obtain the high-precision road digital orthophoto map.

[0128] After obtaining the adjusted RGB value corresponding to each pixel point, the adjusted RGB value can be filled in the map area according to the pixel geographic coordinates corresponding to each pixel point to obtain a high-precision road digital orthophoto map. Figure 3 As shown, after adjusting the RGB value of each pixel, the adjusted RGB value and geographic coordinate information can be combined to perform image stitching and fusion, that is, the adjusted RGB value is filled in the corresponding geographic coordinates in the map area of ​​the electronic map, thereby achieving stitching and fusion to obtain an orthophoto map.

[0129] Low-cost high-precision maps are the production goals of many companies engaged in autonomous driving and mapping. Low-cost acquisition of base map data required for mapping is an important part of reducing the cost of high-precision maps. Road digital orthophoto data can have both the characteristics of laser reflectivity and the quality of the base map. Figure 1 It can not only accurately locate the road elements, but also provide the color information of the road elements, and can serve the tasks of high-precision map making and updating at the same time.

[0130] The image generation method provided by the embodiment of the present disclosure obtains a road image collected by a target camera, and performs coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, determines the pixel geographic coordinates corresponding to each image pixel, and generates a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates. The embodiment of the present disclosure completes the construction of the digital orthophoto map by pre-calibrating the correspondence between the image coordinate system and the geographic coordinate system, and fusing and splicing the image area with geographic coordinates, without combining laser radar point cloud data, which can reduce the cost of constructing the digital orthophoto map, and reduce the operation of synchronization with the laser radar, and the algorithm has low complexity, strong robustness, and fast road construction speed.

[0131] Embodiment 3

[0132] Reference Figure 5 , which shows a schematic diagram of the structure of an image generating device provided by an embodiment of the present disclosure, such as Figure 5 As shown, the image generating device 300 may specifically include the following modules:

[0133] A road image acquisition module 310 is used to acquire a road image based on the image collected by the target camera;

[0134] A geographic coordinate determination module 320, configured to perform coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, and determine the pixel geographic coordinates corresponding to each image pixel;

[0135] The orthophoto generation module 330 is used to generate a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates and pixel information.

[0136] The image generation device provided by the embodiment of the present disclosure obtains a road image collected by a target camera, and performs coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, determines the pixel geographic coordinates corresponding to each image pixel, and generates a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates. The embodiment of the present disclosure completes the construction of the digital orthophoto map by pre-calibrating the correspondence between the image coordinate system and the geographic coordinate system, and fusing and splicing the image area with geographic coordinates, without combining laser radar point cloud data, which can reduce the cost of constructing the digital orthophoto map, and reduce the operation of synchronizing with the laser radar, and the algorithm has low complexity, strong robustness, and fast road construction speed.

[0137] Embodiment 4

[0138] Reference Figure 6 , which shows a schematic diagram of the structure of another image generating device provided by an embodiment of the present disclosure, such as Figure 6 As shown, the image generating device 400 may specifically include the following modules:

[0139] A camera calibration module 410, configured to perform calibration processing on the target camera based on a calibration grid;

[0140] The regional image acquisition module 420 is used to acquire a regional image obtained by photographing the calibration grid by the target camera; the calibration grid has the same size as the regional image;

[0141] A correspondence establishing module 430, for establishing a correspondence between an image coordinate system and a geographic coordinate system based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid;

[0142] A road image acquisition module 440 is used to acquire a road image based on the image collected by the target camera;

[0143] A geographic coordinate determination module 450 is used to perform coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, so as to determine the pixel geographic coordinates corresponding to each image pixel;

[0144] The orthophoto generation module 460 is used to generate a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates and pixel information.

[0145] Optionally, the correspondence establishing module 430 includes:

[0146] A corner point coordinate acquisition unit 431 is used to acquire first corner point coordinates of four corner points of the calibration grid and second corner point coordinates of four corner points of the regional image;

[0147] A horizontal and vertical distance acquisition unit 432 is used to acquire the horizontal distance and the vertical distance between the target camera and the calibration grid;

[0148] The correspondence determination unit 433 is configured to determine the correspondence between the image coordinate system and the geographic coordinate system based on the first corner point coordinates, the second corner point coordinates, the horizontal distance, and the vertical distance.

[0149] Optionally, the orthophoto generation module 460 includes:

[0150] A map area determination unit 461, configured to determine the map area corresponding to the road image on the electronic map according to the pixel geographic coordinates;

[0151] The orthophoto generating unit 462 is used to fill the pixel point information on the road image into the map area based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, so as to generate the high-precision road digital orthophoto map.

[0152] Optionally, the orthophoto generating unit 462 includes:

[0153] A pixel weight determination subunit, used to determine a pixel weight corresponding to each pixel point based on the timestamp information and the distance information;

[0154] The RGB value acquisition subunit is used to adjust the RGB value of each pixel based on the pixel weight to obtain an adjusted RGB value;

[0155] The orthophoto acquisition subunit is used to fill the adjusted RGB value into the map area according to the pixel geographic coordinates corresponding to each pixel point to obtain the high-precision road digital orthophoto map.

[0156] The image generation device provided by the embodiment of the present disclosure obtains a road image collected by a target camera, and performs coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, determines the pixel geographic coordinates corresponding to each image pixel, and generates a high-precision digital orthophoto map of the road area corresponding to the road image based on the pixel geographic coordinates. The embodiment of the present disclosure completes the construction of the digital orthophoto map by pre-calibrating the correspondence between the image coordinate system and the geographic coordinate system, and fusing and splicing the image area with geographic coordinates, without combining laser radar point cloud data, which can reduce the cost of constructing the digital orthophoto map, and reduce the operation of synchronizing with the laser radar, and the algorithm has low complexity, strong robustness, and fast road construction speed.

[0157] An embodiment of the present disclosure further provides an electronic device, comprising: a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the image generating method of the aforementioned embodiment is implemented when the processor executes the program.

[0158] An embodiment of the present disclosure further provides a readable storage medium. When instructions in the storage medium are executed by a processor of an electronic device, the electronic device can execute the image generating method of the aforementioned embodiment.

[0159] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0160] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system or other device. Various general purpose systems may also be used together with the teachings based thereon. According to the above description, it is apparent that the structure required for constructing such systems is. In addition, the embodiments of the present disclosure are not directed to any particular programming language either. It should be understood that the contents of the embodiments of the present disclosure described herein may be realized using various programming languages, and the description of the specific languages ​​above is intended to disclose the best mode of implementation of the embodiments of the present disclosure.

[0161] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present disclosure can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0162] Similarly, it should be understood that in order to streamline the present disclosure and aid in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present disclosure, the various features of the embodiments of the present disclosure are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be interpreted as reflecting the following intention: the claimed embodiments of the present disclosure require more features than the features explicitly recited in each claim. More specifically, as reflected in the claims below, the inventive aspects lie in less than all the features of the single embodiment disclosed above. Therefore, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, with each claim itself serving as a separate embodiment of the embodiments of the present disclosure.

[0163] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and in addition they may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive, all features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature providing the same, equivalent or similar purpose.

[0164] The various component embodiments of the embodiments of the present disclosure may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some or all of the components of the dynamic picture generation device according to the embodiments of the present disclosure. The embodiments of the present disclosure may also be implemented as a device or apparatus program for executing part or all of the methods described herein. Such a program implementing an embodiment of the present disclosure may be stored on a computer-readable medium, or may be in the form of one or more signals. Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0165] It should be noted that the above embodiments illustrate rather than limit the embodiments of the present disclosure, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The embodiments of the present disclosure may be implemented by means of hardware including several different elements and by means of appropriately programmed computers. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0166] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0167] The above description is only a preferred embodiment of the embodiments of the present disclosure and is not intended to limit the embodiments of the present disclosure. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the embodiments of the present disclosure should be included in the protection scope of the embodiments of the present disclosure.

[0168] The above is only a specific implementation of the embodiment of the present disclosure, but the protection scope of the embodiment of the present disclosure is not limited thereto. Any technician familiar with the technical field can easily think of changes or replacements within the technical scope disclosed by the embodiment of the present disclosure, which should be included in the protection scope of the embodiment of the present disclosure. Therefore, the protection scope of the embodiment of the present disclosure should be based on the protection scope of the claims.

Claims

1. An image generation method, characterized in that: include: Acquire a road image collected by a target camera; According to the correspondence between the image coordinate system and the geographic coordinate system, coordinate conversion processing is performed on the pixel coordinates of each image pixel in the road image to determine the pixel geographic coordinates corresponding to each image pixel; according to the pixel geographic coordinates, a map area corresponding to the road image on the electronic map is determined; Based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, the pixel point information on the road image is filled in the map area to generate a high-precision digital orthophoto of the road, wherein the pixel point information on the road image is filled in the map area to generate the high-precision digital orthophoto of the road based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, including: Determine a pixel weight corresponding to each pixel point based on the timestamp information and the distance information; Based on the pixel weight, the RGB value of each pixel is adjusted to obtain an adjusted RGB value; According to the pixel geographic coordinates corresponding to each pixel point, the adjusted RGB value is filled into the map area to obtain the high-precision road digital orthophoto map.

2. The method according to claim 1, characterized in that Before acquiring the road image collected by the target camera, the method further includes: calibrating the target camera based on a calibration grid; Acquire a regional image obtained by photographing the calibration grid by the target camera; the calibration grid has the same size as the regional image; Based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid, a corresponding relationship between the image coordinate system and the geographic coordinate system is established.

3. The method according to claim 2, characterized in that The establishing of a correspondence between an image coordinate system and a geographic coordinate system based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid comprises: Obtaining the first corner point coordinates of the four corner points of the calibration grid and the second corner point coordinates of the four corner points of the regional image; obtaining the horizontal distance and the vertical distance between the target camera and the calibration grid; Based on the first corner point coordinates, the second corner point coordinates, the horizontal distance, and the vertical distance, a corresponding relationship between the image coordinate system and the geographic coordinate system is determined.

4. An image generating device, characterized in that: include: A road image acquisition module is used to acquire road images collected by a target camera; A geographic coordinate determination module, used to perform coordinate conversion processing on the pixel coordinates of each image pixel in the road image according to the correspondence between the image coordinate system and the geographic coordinate system, and determine the pixel geographic coordinates corresponding to each image pixel; An orthophoto generation module, used to determine the map area corresponding to the road image on the electronic map according to the pixel geographic coordinates; Based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, the pixel point information on the road image is filled in the map area to generate a high-precision digital orthophoto of the road, wherein the pixel point information on the road image is filled in the map area to generate the high-precision digital orthophoto of the road based on the timestamp information corresponding to each pixel point on the road image and the distance information between each pixel point and the target camera, including: Determine a pixel weight corresponding to each pixel point based on the timestamp information and the distance information; Based on the pixel weight, the RGB value of each pixel is adjusted to obtain an adjusted RGB value; According to the pixel geographic coordinates corresponding to each pixel point, the adjusted RGB value is filled into the map area to obtain the high-precision road digital orthophoto map.

5. The device according to claim 4, characterized in that The device also includes: A camera calibration module, used for calibrating the target camera based on a calibration grid; A regional image acquisition module, used to acquire a regional image obtained by photographing the calibration grid by the target camera; the calibration grid has the same size as the regional image; The corresponding relationship establishing module is used to establish the corresponding relationship between the image coordinate system and the geographic coordinate system based on the geographic coordinates of each pixel point on the regional image and the coordinates of each point of the calibration grid.

6. The device according to claim 5, characterized in that The corresponding relationship establishing module includes: A corner point coordinate acquisition unit, used to acquire first corner point coordinates of four corner points of the calibration grid and second corner point coordinates of four corner points of the regional image; A horizontal and vertical distance acquisition unit is used to acquire the horizontal distance and vertical distance between the target camera and the calibration grid; a corresponding relationship determination unit is used to determine the corresponding relationship between the image coordinate system and the geographic coordinate system based on the first corner point coordinates, the second corner point coordinates, the horizontal distance and the vertical distance.

7. An electronic device, characterized in that: include: A processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the image generating method according to any one of claims 1 to 3 is implemented when the processor executes the program.

8. A readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the image generating method according to any one of method claims 1 to 3.