Point cloud reconstruction model generation method, point cloud reconstruction method and electronic device

Abstract

The present application provides a point cloud reconstruction model generation method, a point cloud reconstruction method and an electronic device, and relates to the field of point cloud technology. The method includes: determining at least one target reconstructed ground in the ground to be reconstructed; performing image acquisition on each target reconstructed ground from different perspectives to obtain image pairs corresponding to each target reconstructed ground; determining the local optimization target corresponding to each target reconstructed ground based on at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground; optimizing the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground, and obtaining the target point cloud reconstruction model of the ground to be reconstructed. The present application achieves the effect of fast and accurate point cloud reconstruction of a large range of map elements.

Description

Technical Field

[0001] The present application relates to the field of point cloud technology, and in particular to a point cloud reconstruction model generation method, a point cloud reconstruction method and an electronic device. Background Art

[0002] In the process of electronic map production, electronic maps not only need to present detailed geographic information, but also need to accurately label various map elements. Therefore, map element labeling is a key link. The accurate labeling of map elements depends on the point cloud that can accurately and completely express the scene. In practical applications, map elements can be labeled based on the point cloud obtained by LiDAR, but the acquisition equipment with LiDAR cannot be put into use on a large scale due to its high cost, and the point cloud reflectivity pseudo-color image of the point cloud obtained by LiDAR is not as accurate and complete as the real image in expressing the scene, which also has a great impact on the labeling efficiency.

[0003] To address the above issues, related technologies achieve large-scale mapping through pure vision-based point cloud reconstruction. Among them, pure vision-based point cloud reconstruction uses inexpensive cameras to collect data and applies multi-view stereo (MVS) matching to generate dense depth maps to reconstruct the point cloud of map elements. However, this method requires a lot of computing time to complete the point cloud reconstruction of map elements. Summary of the invention

[0004] The present application provides a point cloud reconstruction model generation method, a point cloud reconstruction method and an electronic device, so as to achieve the effect of fast and accurate point cloud reconstruction of a large range of map elements.

[0005] In a first aspect, the present application provides a method for generating a point cloud reconstruction model, comprising:

[0006] Determine at least one target reconstruction ground surface among the ground surfaces to be reconstructed;

[0007] Capturing images of each target reconstructed ground surface from different viewing angles to obtain image pairs corresponding to each target reconstructed ground surface;

[0008] Determine a local optimization target corresponding to each target reconstructed ground surface based on at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground surface;

[0009] According to the local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed.

[0010] In a possible implementation, items to be optimized in the basic point cloud reconstruction model are set based on associated feature items, and the items to be optimized include at least one of point feature optimization items, line feature optimization items, and plane feature optimization items; according to local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain a target point cloud reconstruction model of the ground to be reconstructed, including: according to local optimization targets corresponding to each target reconstructed ground, the items to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed are optimized to obtain a target point cloud reconstruction model of the ground to be reconstructed.

[0011] In a possible implementation, image acquisition is performed on each target reconstructed ground surface through different perspectives to obtain image pairs corresponding to each target reconstructed ground surface, including: performing the following processing on each target reconstructed ground surface: acquiring a first image acquired from the acquisition perspective of the target reconstructed ground surface, acquiring a second image acquired from the query perspective of the target reconstructed ground surface, and using the first image and the second image as image pairs corresponding to the target reconstructed ground surface; determining a local optimization target corresponding to each target reconstructed ground surface based on at least one group of associated feature items in the image pairs corresponding to each target reconstructed ground surface, including: acquiring at least one group of associated feature items based on the first image and the second image, each group of associated feature items including at least two feature items that match in the first image and the second image; determining a local optimization target corresponding to the target reconstructed ground surface based on a geometric distribution relationship between at least two feature items in each group of associated feature items.

[0012] In a possible implementation, a local optimization target corresponding to the target reconstructed ground is determined based on the geometric distribution relationship of at least two feature items in each group of associated feature items, including: determining a feature position difference value corresponding to each group of associated feature items, the feature position difference value representing the position deviation of at least two feature items in the associated feature items of the corresponding group; and determining a local optimization target corresponding to the target reconstructed ground based on the distribution characteristics of each determined feature position difference value.

[0013] In one possible implementation, based on the distribution characteristics of the determined feature position difference values, a local optimization target corresponding to the target reconstructed ground is determined, including: determining the standard variance of each feature position difference value as the local optimization target corresponding to the target reconstructed ground, and the standard variance of each feature position difference value reflects the distribution characteristics of the corresponding feature position difference value.

[0014] In a possible implementation, when the item to be optimized includes a point feature optimization item, the associated feature item is set for the point feature optimization item; at least one group of associated feature items is obtained based on the first image and the second image, including: processing the image pair in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pair: determining a first point in the first image; determining the intersection of the characteristic ray of the first point and the basic point cloud reconstruction model as the second point, and determining the three-dimensional coordinates of the second point; projecting the second point into the second image to obtain a projection point, and determining a point in the second image that is close to the coordinates of the projection point as a third point, wherein the first point, the second point, the projection point, and the third point are at least two feature items in the associated feature items; determining the feature position difference value corresponding to each group of associated feature items, including: determining the difference in distance between the projection point and the third point as the feature position difference value corresponding to the associated feature item.

[0015] In a possible implementation, according to the local optimization targets corresponding to each target reconstructed ground surface, the basic point cloud reconstruction model corresponding to the ground surface to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground surface to be reconstructed, including: according to the following formula, based on the local optimization targets corresponding to each target reconstructed ground surface, determining the global optimization target of the ground surface to be reconstructed; optimizing the point feature optimization items in the basic point cloud reconstruction model in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0016]

[0017] in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground for the i-th target; Reconstruct the local optimization target corresponding to the ground for the i-th target; N Represents the total amount of target reconstructed ground; Represents the second point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; Represents the third point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the i-th target; Represents a projection function that projects the second point into the second image.

[0018] In a possible implementation, when the item to be optimized includes a line feature optimization item, the associated feature item is set for the line feature optimization item; at least one group of associated feature items is obtained based on the first image and the second image, including: processing the image pair in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pair: determining a first line in the first image; determining the intersection of the first line and the basic point cloud reconstruction model as the second line, and determining the three-dimensional coordinates of the second line; projecting the second line into the second image to obtain a projection line, and determining a line in the second image that is close to the projection line coordinates as a third line; wherein the first line, the second line, the projection line, and the third line are at least two feature items in the associated feature items; determining feature position difference values ​​corresponding to each group of associated feature items, including: determining the difference in distance between the projection line and the third line as the feature position difference value corresponding to the associated feature item.

[0019] In a possible implementation, according to the local optimization targets corresponding to each target reconstructed ground surface, the basic point cloud reconstruction model corresponding to the ground surface to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground surface to be reconstructed, including: according to the following formula, based on the local optimization targets corresponding to each target reconstructed ground surface, determining the global optimization target of the ground surface to be reconstructed; in the optimization direction of taking the minimum value of the global optimization target, optimizing the line feature optimization items of the basic point cloud reconstruction model to obtain the target point cloud reconstruction model;

[0020]

[0021] in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground on behalf of the nth target; Reconstruct the local optimization target corresponding to the ground for the nth target; N Represents the total amount of target reconstructed ground; The second line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; The third line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the nth target; represents a projection function for projecting the second line into the second image.

[0022] In a possible implementation, when the item to be optimized includes a plane feature optimization item, an associated feature item is set for the plane feature optimization item; obtaining at least one group of associated feature items based on the first image and the second image includes: processing the image pair in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pair: determining the first plane vertices of two adjacent target reconstructed grounds from the first image included in the image pair for reconstructing the ground from different targets, and determining the second plane vertices of two adjacent target reconstructed grounds from the second image included in the image pair for reconstructing the ground from different targets, wherein the first plane vertex and the second plane vertex are at least two feature items in the associated feature items; determining the feature position difference value corresponding to each group of associated feature items, including: determining the difference between the depth value of the first plane vertex on the target reconstructed ground and the depth value of the second plane vertex on the target reconstructed ground as the feature position difference value corresponding to the associated feature item.

[0023] In a possible implementation, according to the local optimization targets corresponding to each target reconstructed ground surface, the basic point cloud reconstruction model corresponding to the ground surface to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground surface to be reconstructed, including: according to the following formula, based on the local optimization targets corresponding to each target reconstructed ground surface, determining the global optimization target of the ground surface to be reconstructed; optimizing the plane feature optimization items of the basic point cloud reconstruction model in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0024]

[0025] in, represents the global optimization objective of the ground to be reconstructed; Represents obtaining the depth value of the first plane vertex on the target reconstructed ground based on the horizontal and vertical coordinates of the first plane vertex in the first image; represents the depth value of the second plane vertex on the target reconstructed ground obtained based on the horizontal and vertical coordinates of the second plane vertex in the second image; M represents the i-th 1 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; N 1 Represents the total number of reconstructed ground surfaces between two adjacent targets; Represents the i 1 The local optimization objective of reconstructing the ground correspondence between two adjacent objects.

[0026] In a possible implementation, the method further includes: acquiring a third image acquired for the ground to be reconstructed; determining, based on the third image and based on the principle of multi-view geometry, initial values ​​of extrinsic parameters of an acquisition device; determining, based on the initial values ​​of extrinsic parameters, an extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed; and constructing, based on the extrinsic parameter calibration relationship, a basic point cloud reconstruction model corresponding to the ground to be reconstructed.

[0027] In a possible implementation, according to the third image, based on the principle of multi-view geometry, the initial values ​​of the extrinsic parameters of the acquisition device are determined, including: determining the posture of the acquisition device according to the third image; and determining the initial values ​​of the extrinsic parameters of the acquisition device based on the principle of multi-view geometry and the posture of the acquisition device.

[0028] In a possible implementation, the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed is determined based on the initial value of the extrinsic parameter, including: determining the intersection of the characteristic ray of the fourth point in the third image and the ground to be reconstructed as the fifth point, and determining the three-dimensional coordinates of the fifth point; based on the initial value of the extrinsic parameter, projecting the fifth point to the fourth image acquired for the ground to be reconstructed to obtain a sixth point, and determining the point in the second image that is close to the coordinates of the sixth point as the seventh point, wherein the viewing angle for acquiring the fourth image is different from the viewing angle for acquiring the third image; determining an optimization function based on the sixth point and the seventh point; and optimizing the optimization function based on the optimization algorithm to determine the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed.

[0029] In a second aspect, the present application provides a point cloud reconstruction model generation device, comprising:

[0030] A first determination module, used to determine at least one target reconstructed ground surface among the ground surfaces to be reconstructed;

[0031] An acquisition module is used to collect images of each target reconstructed ground surface from different viewing angles to obtain image pairs corresponding to each target reconstructed ground surface;

[0032] A second determination module is used to determine a local optimization target corresponding to each target reconstructed ground surface based on at least one set of associated feature items in the image pair corresponding to each target reconstructed ground surface;

[0033] The processing module is used to optimize the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground, so as to obtain the target point cloud reconstruction model of the ground to be reconstructed.

[0034] In a possible implementation, the processing module is specifically used to: optimize the items to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization targets corresponding to each target reconstructed ground, so as to obtain the target point cloud reconstruction model of the ground to be reconstructed;

[0035] In a possible implementation, the acquisition module is specifically used to: acquire a first image acquired through an acquisition perspective for target reconstruction of the ground, acquire a second image acquired through a query perspective for target reconstruction of the ground, and use the first image and the second image as an image pair corresponding to the target reconstruction of the ground;

[0036] The second determination module is specifically used to: obtain at least one group of associated feature items based on the first image and the second image, each group of associated feature items includes at least two feature items that match the first image and the second image; determine the local optimization target corresponding to the target reconstructed ground according to the geometric distribution relationship of at least two feature items in each group of associated feature items.

[0037] In a possible implementation, the second determination module is also used to: determine the feature position difference value corresponding to each group of associated feature items, the feature position difference value representing the position deviation of at least two feature items in the associated feature items of the corresponding group; and determine the local optimization target corresponding to the target reconstructed ground based on the distribution characteristics of the determined feature position difference values.

[0038] In a possible implementation, the second determination module is used to: determine the standard deviation of each feature position difference value as a local optimization target corresponding to the target reconstructed ground, and the standard deviation of each feature position difference value reflects the distribution characteristics of the corresponding feature position difference value;

[0039] In a possible implementation, the acquisition module is specifically used to: determine a first point in a first image; determine the intersection of the characteristic ray of the first point and the basic point cloud reconstruction model as a second point, and determine the three-dimensional coordinates of the second point; project the second point into the second image to obtain a projection point, and determine a point in the second image that is close to the coordinates of the projection point as a third point, wherein the first point, the second point, the projection point, and the third point are at least two feature items in the associated feature items; the second determination module is used to: determine the difference in the distance between the projection point and the third point as the feature position difference value corresponding to the associated feature item.

[0040] In a possible implementation manner, the processing module is specifically used for:

[0041] According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground, the global optimization target of the ground to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the point feature optimization items in the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model;

[0042]

[0043] in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground for the i-th target; Reconstruct the local optimization target corresponding to the ground for the i-th target; N Represents the total amount of target reconstructed ground; Represents the second point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; Represents the third point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the i-th target; Represents a projection function that projects the second point into the second image.

[0044] In a possible implementation, the acquisition module is specifically used to: determine a first line in a first image; determine the intersection line of the first line and the basic point cloud reconstruction model as a second line, and determine the three-dimensional coordinates of the second line; project the second line into the second image to obtain a projection line, and determine a line in the second image that is close to the projection line coordinates as a third line; wherein the first line, the second line, the projection line, and the third line are at least two feature items in the associated feature items; the second determination module is specifically used to: determine the difference in distance between the projection line and the third line as a feature position difference value corresponding to the associated feature item.

[0045] In a possible implementation, the processing module is specifically used to: determine the global optimization target of the ground to be reconstructed based on the local optimization targets corresponding to each target reconstructed ground according to the following formula; optimize the line feature optimization items of the basic point cloud reconstruction model in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0046]

[0047] in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground on behalf of the nth target; Reconstruct the local optimization target corresponding to the ground for the nth target; N Represents the total amount of target reconstructed ground; The second line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; The third line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the nth target; represents a projection function for projecting the second line into the second image.

[0048] In a possible implementation, the acquisition module is specifically used to: determine the first plane vertices of two adjacent target reconstructed ground from a first image included in an image pair for reconstructing ground from different targets, and determine the second plane vertices of two adjacent target reconstructed ground from a second image included in an image pair for reconstructing ground from different targets, wherein the first plane vertex and the second plane vertex are at least two feature items in the associated feature items; the second determination module is specifically used to: determine the difference between the depth value of the first plane vertex on the target reconstructed ground and the depth value of the second plane vertex on the target reconstructed ground as the feature position difference value corresponding to the associated feature item.

[0049] In a possible implementation, the processing module is used to:

[0050] According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground, the global optimization target of the ground to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the plane feature optimization items of the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model;

[0051]

[0052] in, represents the global optimization objective of the ground to be reconstructed; Represents obtaining the depth value of the first plane vertex on the target reconstructed ground based on the horizontal and vertical coordinates of the first plane vertex in the first image; represents the depth value of the second plane vertex on the target reconstructed ground obtained based on the horizontal and vertical coordinates of the second plane vertex in the second image; M represents the i-th 1 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; N 1 Represents the total number of reconstructed ground surfaces between two adjacent targets; Represents the i 1 The local optimization objective of reconstructing the ground correspondence between two adjacent objects.

[0053] In a possible implementation, the processing module is also used to: obtain a third image captured for the ground to be reconstructed; determine the initial values ​​of the extrinsic parameters of the acquisition device based on the third image and the principle of multi-view geometry; determine the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed based on the initial values ​​of the extrinsic parameters; and construct a basic point cloud reconstruction model corresponding to the ground to be reconstructed based on the extrinsic parameter calibration relationship.

[0054] In a possible implementation, the processing module is further used to: determine the posture of the acquisition device according to the third image; and determine the initial value of the external parameter of the acquisition device according to the posture of the acquisition device based on the principle of multi-view geometry.

[0055] In one possible implementation, the processing module is used to: determine the intersection of the characteristic ray of the fourth point in the third image and the ground to be reconstructed as the fifth point, and determine the three-dimensional coordinates of the fifth point; based on the initial value of the extrinsic parameter, project the fifth point into the fourth image collected for the ground to be reconstructed to obtain a sixth point, and determine the point in the second image that is close to the coordinates of the sixth point as the seventh point, wherein the viewing angle for collecting the fourth image is different from the viewing angle for collecting the third image; determine an optimization function based on the sixth point and the seventh point; and optimize the optimization function based on the optimization algorithm to determine the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed.

[0056] In a third aspect, the present application provides a point cloud reconstruction method, comprising:

[0057] Acquire a target image collected for the ground to be reconstructed;

[0058] The target image is processed by a target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed. The target point cloud reconstruction model is obtained by the above-mentioned point cloud reconstruction model generation method.

[0059] In a possible implementation, before acquiring the target image collected for the ground to be reconstructed, it also includes: dividing the ground to be reconstructed into a number of local grounds; processing the target image through a target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed, including: processing the ground point cloud corresponding to each local ground through the target point cloud reconstruction model to generate a ground point cloud corresponding to the ground to be reconstructed.

[0060] In a possible implementation, the target image is processed through a target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed, including: taking the point in the upper left corner of the target image as the starting point, traversing the points in the target image based on a preset direction and a preset step size, and determining the depth value of at least one traversed point according to the target point cloud reconstruction model of the ground to be reconstructed; based on the depth values ​​of each point in the target image, generating a ground point cloud corresponding to the ground to be reconstructed.

[0061] In a possible implementation, a colorless ground point cloud corresponding to the ground to be reconstructed is determined according to the depth value of each point in the target image; and the colorless ground point cloud is rendered to obtain a colored ground point cloud corresponding to the ground to be reconstructed.

[0062] In one possible implementation, for at least one colorless ground point cloud, a K-dimensional tree including at least one acquisition perspective of the colorless ground point cloud is constructed; based on each colorless ground point cloud, a first acquisition perspective corresponding to the colorless ground point cloud is determined from the K-dimensional tree, the first acquisition perspective being the acquisition perspective having the minimum distance from the colorless ground point cloud; when the colorless ground point cloud is projected to the first acquisition perspective, the color of the colorless ground point cloud is determined to be the color of the first acquisition perspective.

[0063] In a fourth aspect, the present application provides a point cloud reconstruction device, comprising:

[0064] An acquisition module, used for acquiring a target image collected for the ground to be reconstructed;

[0065] The generation module is used to process the target image through the target point cloud reconstruction model of the ground to be reconstructed, and generate a ground point cloud corresponding to the ground to be reconstructed. The target point cloud reconstruction model is obtained through the above-mentioned point cloud reconstruction model generation method.

[0066] In a possible implementation, the generation module is specifically used to: process the ground point clouds corresponding to each local ground surface through the target point cloud reconstruction model to generate a ground point cloud corresponding to the ground surface to be reconstructed.

[0067] In a possible implementation, the generation module is also used to: take the point in the upper left corner of the target image as the starting point, traverse the points in the target image based on a preset direction and with a preset step size, and reconstruct the model of the target point cloud of the ground to be reconstructed, determine the depth value of at least one traversed point; based on the depth values ​​of each point in the target image, generate a ground point cloud corresponding to the ground to be reconstructed.

[0068] In a possible implementation, the generation module is used to: determine the colorless ground point cloud corresponding to the ground to be reconstructed according to the depth value of each point in the target image; perform point cloud rendering on the colorless ground point cloud to obtain a colored ground point cloud corresponding to the ground to be reconstructed.

[0069] In one possible implementation, the generation module is specifically used to: for at least one colorless ground point cloud, construct a K-dimensional tree including at least one acquisition perspective of the colorless ground point cloud; based on each colorless ground point cloud, determine from the K-dimensional tree a first acquisition perspective corresponding to the colorless ground point cloud, the first acquisition perspective being the acquisition perspective having the minimum distance from the colorless ground point cloud; when the colorless ground point cloud is projected to the first acquisition perspective, determine the color of the colorless ground point cloud as the color of the first acquisition perspective.

[0070] In a fifth aspect, the present application provides an electronic map updating method, comprising:

[0071] Through the third aspect and / or various possible implementations of the third aspect, a ground point cloud corresponding to the ground to be reconstructed is obtained; based on the ground point cloud corresponding to the ground to be reconstructed, an area corresponding to the ground to be reconstructed in the electronic map is updated.

[0072] In a sixth aspect, the present application provides an electronic map updating device, comprising:

[0073] An acquisition module, used to acquire a ground point cloud corresponding to the ground to be reconstructed through the third aspect and / or various possible implementations of the third aspect;

[0074] The updating module is used to update the area corresponding to the ground to be reconstructed in the electronic map based on the ground point cloud corresponding to the ground to be reconstructed.

[0075] In a seventh aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0076] Memory stores computer-executable instructions;

[0077] The processor executes the computer-executable instructions to implement various possible implementations of the first aspect and / or the first aspect and / or the third aspect and / or the third aspect and / or the fifth aspect and / or the fifth aspect as described above.

[0078] In an eighth aspect, the present application provides a computer-readable storage medium, in which computer execution instructions are stored. When the computer execution instructions are executed, they are used to implement various possible implementations of the first aspect and / or the first aspect and / or the third aspect and / or the third aspect and / or the fifth aspect and / or the fifth aspect as above.

[0079] In a ninth aspect, the present application provides a computer program product, comprising a computer program, which, when executed, implements various possible implementations of the first aspect and / or the first aspect and / or the third aspect and / or the third aspect and / or the fifth aspect and / or the fifth aspect as above.

[0080] The point cloud reconstruction model generation method, point cloud reconstruction method and electronic device provided by the present application are to determine at least one target reconstructed ground in the ground to be reconstructed; to collect images of each target reconstructed ground through different perspectives to obtain image pairs corresponding to each target reconstructed ground; to determine the local optimization target corresponding to each target reconstructed ground based on at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground; to optimize the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground, and obtain the target point cloud reconstruction model of the ground to be reconstructed. The present application collects images of each target reconstructed ground through different perspectives to obtain different sides and features of each target reconstructed ground, providing more details and information for the target reconstructed ground; based on at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground, the local optimization target corresponding to each target reconstructed ground can be accurately identified, thereby reducing the amount of optimization calculation and improving the optimization speed; through the local optimization target corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized, which can effectively reduce the random noise in the target reconstructed ground, improve the accuracy of the target point cloud reconstruction model of the ground to be reconstructed, and achieve the effect of fast and accurate point cloud reconstruction of a large range of map elements based on the target point cloud reconstruction model. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0082] Figure 1 A schematic diagram of a process for generating a point cloud reconstruction model provided in an embodiment of the present application;

[0083] Figure 2 A schematic diagram of a principle for determining a target point cloud reconstruction model of a ground to be reconstructed provided in an embodiment of the present application;

[0084] Figure 3 A schematic diagram of a principle for determining a target point cloud reconstruction model of a ground to be reconstructed provided in an embodiment of the present application;

[0085] Figure 4 A schematic diagram of a principle for determining a target point cloud reconstruction model of a ground to be reconstructed provided in an embodiment of the present application;

[0086] Figure 5 A schematic diagram of determining the external parameter calibration relationship between the acquisition device and the ground to be reconstructed in an embodiment of the present application;

[0087] Figure 6 A schematic diagram of a process of a point cloud reconstruction method provided in an embodiment of the present application;

[0088] Figure 7A schematic diagram of dividing a ground surface to be reconstructed into multiple local ground surfaces provided in an embodiment of the present application;

[0089] Figure 8 A flowchart of an electronic map updating method provided in an embodiment of the present application;

[0090] Fig. 9 The overall flow chart of point cloud reconstruction provided by the embodiment of the present application;

[0091] Fig.10 A schematic diagram of the structure of the point cloud reconstruction model generation device provided in this application;

[0092] Fig.11 A schematic diagram of the structure of a point cloud reconstruction device provided in an embodiment of the present application;

[0093] Fig.12 A schematic diagram of the structure of an electronic map updating device provided in an embodiment of the present application;

[0094] Fig.13 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application.

[0095] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0096] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0097] It should be noted that the collection, use, processing, transmission, provision, disclosure and application of various data and related information involved in this application shall comply with relevant laws, regulations and standards.

[0098] In the process of electronic map production, office staff often need to use the point cloud of the lidar and the pseudo-color image of the corresponding reflectivity to mark the map elements. However, high-precision data collection vehicles with lasers are often not put into use on a large scale due to their high production costs, which makes it difficult to generate high-precision maps on a large scale and over a wide range.

[0099] Furthermore, the pseudo-color image of the laser radar's point cloud reflectivity is not as clear as the real image, resulting in the laser radar's point cloud reflectivity pseudo-color image not being able to express the scene more accurately and completely than the real image, which also has a great impact on the annotation efficiency of office personnel.

[0100] Furthermore, large-scale mapping based on pure vision has better application prospects in large-scale mapping because it only requires low-cost cameras to achieve functions similar to lidar. However, existing vision-based mapping solutions usually rely on MVS to generate dense depth maps, or use deep learning-based methods such as MVSNet and Neural Radiance Fields (NeRF) for dense reconstruction. These methods usually require complex algorithms and a lot of computing time to achieve high-quality dense reconstruction.

[0101] In view of the high cost of the above-mentioned radar and the large requirements of existing vision-based mapping technology in terms of computing time and computing power, this application provides a self-developed global ground reconstruction method, which realizes a fast, robust and accurate dense reconstruction solution of the ground by optimizing the ground equation. By optimizing the ground equation, the influence of noise generated in some deep blurred areas on the reconstruction effect can be effectively avoided, making the reconstruction effect smoother.

[0102] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0103] Figure 1 A schematic diagram of a process for generating a point cloud reconstruction model provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method comprises:

[0104] S101: Determine at least one target reconstruction ground in the ground to be reconstructed.

[0105] In this step, it can be understood that determining at least one target reconstruction ground in the ground to be reconstructed means that before performing the reconstruction task of the ground to be reconstructed, at least one target reconstruction ground in the ground to be reconstructed can be identified and selected.

[0106] S102 . Capture images of each target reconstructed ground surface from different viewing angles to obtain image pairs corresponding to each target reconstructed ground surface.

[0107] In this step, different viewing angles may refer to observing the same object from different angles or positions. By changing the position or angle of the observer, different views of the same object may be obtained, thereby obtaining more information about the same object.

[0108] In the process of reconstructing the ground to be reconstructed, it is necessary to collect images of each target reconstructed ground from different perspectives to obtain image pairs corresponding to each target reconstructed ground. This means that when images of the same target reconstructed ground are collected from different perspectives, the images obtained are image pairs corresponding to the target reconstructed ground.

[0109] Furthermore, the images of the ground to be reconstructed collected from different perspectives can improve the detailed information of the ground to be reconstructed, so as to facilitate the subsequent point cloud reconstruction. The different perspectives include but are not limited to the following: frontal perspective, side perspective, and oblique perspective.

[0110] S103: Determine a local optimization target corresponding to each target reconstructed ground surface based on at least one set of associated feature items in the image pair corresponding to each target reconstructed ground surface.

[0111] In this step, it can be understood that the local optimization target corresponding to each target reconstructed ground is determined by at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground obtained in S102. Among them, the associated feature items refer to specific features with relevant associated characteristics in different images of the image pairs corresponding to the target reconstructed ground. These specific features include but are not limited to any one or more of the following: points, lines or other significant geometric shapes in different images contained in the same image pair. The local optimization target corresponding to the target reconstructed ground is determined by at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground. In the embodiment of the present application, the local optimization target is not overly limited. Those skilled in the art can set it based on actual business needs. For example, it can be but not limited to setting the local optimization target corresponding to the target reconstructed ground to the error between the target reconstructed ground and the real reconstructed ground being less than a certain value.

[0112] S104 , optimizing the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground, to obtain the target point cloud reconstruction model of the ground to be reconstructed.

[0113] Based on the local optimization target corresponding to the target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain an optimized basic point cloud reconstruction model, and the optimized basic point cloud reconstruction model is determined as the target point cloud reconstruction model of the ground to be reconstructed.

[0114] Furthermore, in an embodiment of the present application, the ground to be reconstructed can be divided into several local grounds in step S101, and then N local grounds are selected from all the local grounds obtained by division as target reconstructed grounds (one local ground is a target reconstructed ground), where the value of N can be the total number of all the local grounds obtained by division, or a value less than the total number of all the local grounds obtained by division, and the above N is an integer greater than 0.

[0115] In one implementation, the ground to be reconstructed is divided into 4-meter-sized pieces to obtain a number of local grounds of 4 meters×4 meters in size, and all the local grounds obtained by segmentation are used as target reconstructed grounds.

[0116] In another implementation, the ground to be reconstructed is divided into a number of n×n local grounds, each local ground covering an area within 3 meters from the center of the local ground, and all the local grounds obtained by segmentation are used as target reconstructed grounds.

[0117] In another implementation, the ground to be reconstructed is divided into 4-meter sizes to obtain several local grounds of 4 meters × 4 meters in size, and 4 local grounds are selected from all the local grounds obtained by segmentation as target reconstructed grounds, wherein the selection basis can be set according to demand, for example, the 4 local grounds located at the center of the ground to be reconstructed are determined as the target reconstructed grounds.

[0118] Correspondingly, in step S104, the sum of the local optimization targets of each target reconstructed ground among the N target reconstructed grounds can be used as the global optimization target of the ground to be reconstructed, and then based on the global optimization target, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed; such as in the optimization direction of taking the minimum value of the global optimization target, the line feature optimization items of the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model.

[0119] The embodiments of the present application collect images of each target reconstructed ground from different perspectives to obtain different sides and features of each target reconstructed ground, providing more details and information for the target reconstructed ground; based on at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground, the local optimization target corresponding to each target reconstructed ground can be accurately identified, thereby reducing the amount of optimization calculations and improving the optimization speed; through the local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized, which can effectively reduce random noise in the target reconstructed ground, improve the accuracy of the target point cloud reconstruction model of the ground to be reconstructed, and achieve fast and accurate point cloud reconstruction of a large range of map elements based on the target point cloud reconstruction model.

[0120] On the basis of the above embodiments, image acquisition is performed on each target reconstructed ground through different perspectives to obtain image pairs corresponding to each target reconstructed ground, including: performing the following processing on each target reconstructed ground: acquiring a first image acquired from the acquisition perspective of the target reconstructed ground, acquiring a second image acquired from the query perspective of the target reconstructed ground, and using the first image and the second image as image pairs corresponding to the target reconstructed ground; determining a local optimization target corresponding to each target reconstructed ground based on at least one group of associated feature items in the image pairs corresponding to each target reconstructed ground, including: acquiring at least one group of associated feature items based on the first image and the second image, each group of associated feature items including at least two feature items that match in the first image and the second image; determining a local optimization target corresponding to the target reconstructed ground according to a geometric distribution relationship between at least two feature items in each group of associated feature items.

[0121] In this embodiment, it can be understood that one of the different perspectives is used as a collection perspective, and the other perspectives are used as query perspectives, wherein the specific settings of the collection perspective and the query perspective can be made according to actual conditions.

[0122] After determining the acquisition perspective and the query perspective, the following processing is performed on each target reconstructed ground: obtain a first image acquired through the acquisition perspective of the target reconstructed ground, obtain a second image acquired through the query perspective of the target reconstructed ground, and use the first image and the second image as an image pair corresponding to the target reconstructed ground.

[0123] After determining the image pair corresponding to the target reconstructed ground, at least one group of associated feature items in the image pair is obtained based on the image pair, wherein each group of associated feature items includes at least two feature items matched in the image pair. According to the geometric distribution relationship of at least two feature items in each group of associated feature items, the local optimization target corresponding to the target reconstructed ground is determined. Furthermore, the geometric distribution relationship of the feature items may be the arrangement and / or relative position relationship of each feature item in space, and the geometric distribution relationship of the feature items may include one or more of the relationships of geometric parameters such as the position and distance of the feature items in space.

[0124] In summary, firstly, the image pairs corresponding to each target reconstructed ground and at least one group of associated feature items in the image pairs are determined, and then the local optimization target corresponding to each target reconstructed ground is determined according to the geometric distribution relationship of at least two feature items in each group of associated feature items in the image pairs.

[0125] The embodiment of the present application determines the local optimization target corresponding to each target reconstructed ground according to the geometric distribution relationship of at least two feature items in each group of associated feature items in the image pair, which can reduce the error of each target reconstructed ground relative to the real reconstructed ground and help obtain a more accurate target point cloud reconstruction model.

[0126] Furthermore, based on the geometric distribution relationship between at least two feature items in each group of associated feature items, a local optimization target corresponding to the target reconstructed ground is determined, including: determining a feature position difference value corresponding to each group of associated feature items, the feature position difference value representing the position deviation of at least two feature items in the associated feature items of the corresponding group; based on the distribution characteristics of the determined feature position difference values, determining the local optimization target corresponding to the target reconstructed ground.

[0127] In this embodiment, it can be understood that if one wants to determine the local optimization target corresponding to the target reconstructed ground, it is necessary to determine the feature position difference value corresponding to each group of associated feature items, and then determine the local optimization target corresponding to the target reconstructed ground based on the distribution characteristics of the determined feature position difference values; wherein the feature position difference value corresponding to each group of associated feature items is determined by the position deviation of at least two feature items in the associated features of the corresponding group, wherein the position deviation of the feature item includes but is not limited to any one or combination of the following: distance of the feature item, depth value, etc.

[0128] In one implementation, a local optimization target corresponding to the target reconstructed ground is determined based on distribution characteristics of each feature position difference value determined, and each feature position difference value is determined by a distance deviation of at least two feature items in the associated feature items of the corresponding group.

[0129] In another implementation, a local optimization target corresponding to the target reconstructed ground is determined based on distribution characteristics of each feature position difference value determined, and each feature position difference value is a depth value deviation of at least two feature items in the associated feature items of the corresponding group.

[0130] The embodiment of the present application determines the local optimization target corresponding to the target reconstructed ground according to the geometric distribution relationship of at least two feature items in each group of associated feature items, which can reduce the error of each target reconstructed ground relative to the real reconstructed ground and help obtain a more accurate target point cloud reconstruction model.

[0131] Furthermore, the distribution characteristics of the difference values ​​of each characteristic position may be set according to actual conditions. For example, the distribution characteristics of the difference values ​​of each characteristic position include any one or more of mean, standard deviation, skewness, etc.

[0132] In one implementation, when the distribution characteristics of each feature position difference value include a standard deviation, determining the local optimization target corresponding to the target reconstructed ground based on the determined distribution characteristics of each feature position difference value includes: determining the standard deviation of each feature position difference value as the local optimization target corresponding to the target reconstructed ground; wherein the standard deviation of each feature position difference value reflects the distribution characteristics of the corresponding feature position difference value; further, the local optimization target corresponding to the target reconstructed ground may be determined based on, but not limited to, the following formula (1a);

[0133] Formula (1a)

[0134] In formula (1a), i 2 Reconstruct the ground identity for the target; For the i 2 The target reconstructs the ground (i.e., 2 The local optimization goal corresponding to the goal of reconstructing the ground) M Represents the i 2 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; Represents the i 2 A feature item of the qth associated feature item in the image pair corresponding to the target reconstructed ground; Represents the i 2 The qth associated feature item in the image pair corresponding to the target reconstructed ground Another feature item that is different.

[0135] Furthermore, based on the following formula (1b) and the processing logic of formula (1a), the global optimization target of the ground to be reconstructed can be determined, and then the basic point cloud reconstruction model corresponding to the ground to be reconstructed can be optimized based on the global optimization target to obtain the target point cloud reconstruction model;

[0136] Formula (1b)

[0137] In formula (1b), is the global optimization target of the ground to be reconstructed; i 2 Reconstruct the ground identity for the target; N Represents the total amount of target reconstructed ground; M Represents the i 2 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; Represents the i 2 A feature item of the qth associated feature item in the image pair corresponding to the target reconstructed ground; Represents the i 2 The qth associated feature item in the image pair corresponding to the target reconstructed ground Another feature item that is different.

[0138] In this embodiment, it can be understood that when the distribution characteristics of the difference values ​​of each feature position include standard deviation, the standard deviation of the difference values ​​of each feature position is determined as the local optimization target corresponding to the target reconstructed ground. Among them, the standard deviation is a variable used to measure the degree of dispersion of the difference values ​​of each feature position. By determining the standard deviation of the difference values ​​of each feature position as the local optimization target corresponding to the target reconstructed ground, reducing the standard deviation can make the target point cloud reconstruction model more robust in the face of noise and outliers.

[0139] On the basis of the above embodiments, a point cloud reconstruction model generation method is provided. In the point cloud reconstruction model generation method provided in the aforementioned steps S101 to S104, one or more items to be optimized in the basic point cloud reconstruction model can also be set based on associated feature items, and the items to be optimized include any one or a combination of point feature optimization items, line feature optimization items, and plane feature optimization items; and then in step S104, the items to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed can be optimized according to the local optimization targets corresponding to each target reconstructed ground, so as to obtain the target point cloud reconstruction model of the ground to be reconstructed.

[0140] In one embodiment, any one of a point feature optimization item, a line feature optimization item, and a plane feature optimization item can be set as an item to be optimized for the basic point cloud reconstruction model. After the item to be optimized is determined, the global optimization target of the ground to be reconstructed can be determined by, but not limited to, the following formula (1c). Then, based on the global optimization target, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain a target point cloud reconstruction model.

[0141] Formula (1c)

[0142] In formula (1c), s is the identifier of the optimization item set for the basic point cloud reconstruction model; It is the global optimization target of the ground to be reconstructed corresponding to the optimization item set as s for the basic point cloud reconstruction model; i 2 Reconstruct the ground identity for the target; N Represents the total amount of target reconstructed ground; M Represents the i 2 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; Represents the i 2 A feature item of the qth associated feature item in the image pair corresponding to the target reconstructed ground; Represents the i 2 The qth associated feature item in the image pair corresponding to the target reconstructed ground Another feature item that is different.

[0143] For example, when the item to be optimized includes any one of the point feature optimization item, the line feature optimization item, and the plane feature optimization item, when iteratively optimizing the item to be optimized in the basic point cloud reconstruction model corresponding to the reconstructed ground, the item to be optimized needs to be optimized in each iterative optimization until the termination condition of the iterative optimization is met. The termination condition of the iterative optimization can be set according to actual needs. For example, the termination condition of the iterative optimization includes but is not limited to any one or combination of the following: the number of iterations, the error between the target reconstructed ground and the actual reconstructed ground is less than a certain value.

[0144] In another embodiment, any two or three of the point feature optimization items, line feature optimization items, and plane feature optimization items may be set as items to be optimized for the basic point cloud reconstruction model, that is, the items to be optimized may include any one of “point feature optimization items and line feature optimization items”, “point feature optimization items and plane feature optimization items”, “line feature optimization items and plane feature optimization items”, and “point feature optimization items, line feature optimization items, and plane feature optimization items”;

[0145] For example, when the items to be optimized include at least two of the point feature optimization items, line feature optimization items, and plane feature optimization items, the items to be optimized in the basic point cloud reconstruction model corresponding to the reconstructed ground are iteratively optimized, wherein the termination condition of the iterative optimization can be set according to actual needs, for example, the termination condition of the iterative optimization includes but is not limited to any one or combination of the following: the number of iterations, the error between the target reconstructed ground and the actual reconstructed ground is less than a certain value. In each iterative optimization process, at least two items to be optimized need to be optimized until the termination condition of the iterative optimization is met. In each iterative optimization process, at least two items to be optimized can be optimized in the optimization order, or at least two items to be optimized can be optimized at the same time.

[0146] Furthermore, the optimization order can be set according to requirements. For example, when the items to be optimized include three items among point feature optimization items, line feature optimization items, and plane feature optimization items, the point feature optimization items in the basic point cloud reconstruction model corresponding to the reconstructed ground are optimized first, and then the line feature optimization items in the basic point cloud reconstruction model corresponding to the reconstructed ground are optimized, and finally the plane feature optimization items in the basic point cloud reconstruction model corresponding to the reconstructed ground are optimized.

[0147] Furthermore, in each iterative optimization process, at least two items to be optimized are optimized at the same time, which means that in each iterative optimization process, at least two items to be optimized need to be optimized at the same time. For example, when the items to be optimized include point feature optimization items and line feature optimization items, the point feature optimization items and the line feature optimization items need to be optimized at the same time in each iteration until the termination condition of the iterative optimization is met.

[0148] The embodiment of the present application obtains a target point cloud reconstruction model of the ground to be reconstructed by optimizing the items to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed, which can effectively reduce the random noise in the target reconstructed ground and improve the accuracy of the target point cloud reconstruction model of the ground to be reconstructed.

[0149] In one example, when the item to be optimized includes a point feature optimization item, the associated feature item is set for the point feature optimization item; at least one group of associated feature items is obtained based on the first image and the second image, including: processing the image pair in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pair: determining a first point in the first image; determining the intersection of the characteristic ray of the first point and the basic point cloud reconstruction model as the second point, and determining the three-dimensional coordinates of the second point; projecting the second point into the second image to obtain a projection point, and determining a point in the second image that is close to the coordinates of the projection point as a third point, wherein the first point, the second point, the projection point, and the third point are at least two feature items in the associated feature items; determining the feature position difference values ​​corresponding to each group of associated feature items, including: determining the difference in distance between the projection point and the third point as the feature position difference value corresponding to the associated feature item.

[0150] In this embodiment, it can be understood that when the items to be optimized in the basic point cloud reconstruction model include point feature optimization items, it is necessary to obtain at least two feature items in at least one group of associated feature items based on the first image and the second image. To obtain at least two feature items in at least one group of associated feature items based on the first image and the second image, it is necessary to first determine the first point in the first image, determine the intersection of the characteristic ray of the first point and the basic point cloud reconstruction model as the second point, determine the three-dimensional coordinates of the second point, project the second point into the second image to obtain the projection point, and determine the point in the second image that is close to the projection point coordinates as the third point, wherein the first point, the second point, the projection point, and the third point are at least two feature items in each group of associated feature items.

[0151] Based on at least two feature items in each group of determined associated feature items, feature position difference values ​​corresponding to each group of associated feature items are determined, and the feature position difference values ​​corresponding to each group of associated feature items are determined by the difference in distance between the projection point and the third point.

[0152] The standard deviation of the feature position difference values ​​corresponding to each group of associated feature items is determined as the local optimization target corresponding to the target reconstructed ground. According to the determined local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed. Among them, according to the local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed, including: according to the following formula (2), the global optimization target of the ground to be reconstructed is determined through the local optimization targets corresponding to each target reconstructed ground; the point feature optimization items in the basic point cloud reconstruction model are optimized in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0153] Formula (2)

[0154] In formula (2), represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground for the i-th target; Reconstruct the local optimization target corresponding to the ground for the i-th target; N Represents the total amount of target reconstructed ground; Represents the second point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; Represents the third point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the i-th target; Represents a projection function that projects the second point into the second image.

[0155] Furthermore, the point features to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed are optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed. The specific principle can be referred to Figure 2 , Figure 2 A schematic diagram of a principle for determining a target point cloud reconstruction model of a ground to be reconstructed provided in an embodiment of the present application, Figure 2 The a in it represents a three-dimensional point, the parallelogram represents the local ground, and different boxes represent images from different perspectives.

[0156] The embodiment of the present application optimizes the point features to be optimized in the basic point cloud reconstruction model corresponding to the reconstructed ground, so that a good reconstruction effect can be maintained even in the face of adverse conditions such as noise, occlusion or data missing.

[0157] Since the features corresponding to the midpoint of the ground to be reconstructed are often scarce, the embodiment of the present application introduces more feature constraints to ensure the robustness of the ground to be reconstructed. The embodiment of the present application adds the optimization of line features. The optimization method of line features is similar to the optimization method of point features. The first line in the first image intersects with the plane to be reconstructed to obtain a three-dimensional line, which is represented by a plucker line. The three-dimensional plucker line is projected into the second image to ensure that the distance between the projection and the two-dimensional line in the second image is short to constrain the plane.

[0158] In another example, when the item to be optimized includes a line feature optimization item, the associated feature item is set for the line feature optimization item; at least one group of associated feature items is obtained based on the first image and the second image, including: processing the image pair in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pair: determining a first line in the first image; determining the intersection line of the first line and the basic point cloud reconstruction model as the second line, and determining the three-dimensional coordinates of the second line; projecting the second line into the second image to obtain a projection line, and determining a line in the second image that is close to the projection line coordinates as a third line, wherein the first line, the second line, the projection line, and the third line are at least two feature items in the associated feature items; determining the feature position difference values ​​corresponding to each group of associated feature items, including: determining the difference in distance between the projection line and the third line as the feature position difference value corresponding to the associated feature item.

[0159] In this embodiment, it can be understood that when the items to be optimized in the basic point cloud reconstruction model include line feature optimization items, it is necessary to obtain at least two feature items in at least one group of associated feature items based on the first image and the second image. To obtain at least two feature items in at least one group of associated feature items based on the first image and the second image, it is necessary to first determine the first line in the first image, determine the intersection of the first ray and the basic point cloud reconstruction model as the second line, determine the three-dimensional coordinates of the second line, project the second line into the second image to obtain the projection line, and determine the line in the second image that is close to the projection line coordinates as the third line, wherein the first line, the second line, the projection line, and the third line are at least two feature items in each group of associated feature items.

[0160] Based on at least two feature items in each group of determined associated feature items, feature position difference values ​​corresponding to each group of associated feature items are determined, and the feature position difference values ​​corresponding to each group of associated feature items are determined by the difference in distance between the projection line and the third line.

[0161] The standard deviation of the feature position difference values ​​corresponding to each group of associated feature items is determined as the local optimization target corresponding to the target reconstructed ground. According to the determined local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed. Among them, according to the local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed, including: according to the following formula (3), the global optimization target of the ground to be reconstructed is determined through the local optimization targets corresponding to each target reconstructed ground; the line feature optimization items of the basic point cloud reconstruction model are optimized in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0162] Formula (3)

[0163] In formula (3), represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground on behalf of the nth target; Reconstruct the local optimization target corresponding to the ground for the nth target; N Represents the total amount of target reconstructed ground; The second line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; The third line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the nth target; represents a projection function for projecting the second line into the second image.

[0164] Furthermore, the line feature to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed. The specific principle can be referred to Figure 3 , Figure 3 A schematic diagram of a principle for determining a target point cloud reconstruction model of a ground to be reconstructed provided in an embodiment of the present application, Figure 3 Here, b represents a two-dimensional line in the first image, c represents a two-dimensional line in the second image, and d represents a three-dimensional line.

[0165] The embodiment of the present application can improve the robustness of point cloud reconstruction by optimizing the line features to be optimized in the basic point cloud reconstruction model corresponding to the reconstructed ground, thereby avoiding the influence of noise generated in some deep blurred areas on the point cloud reconstruction.

[0166] In another example, when the item to be optimized includes a plane feature optimization item, the associated feature item is set for the plane feature optimization item; at least one group of associated feature items is obtained based on the first image and the second image, including: processing the image pair in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pair; determining the first plane vertices of two adjacent target reconstructed grounds from the first image included in the image pair of different target reconstructed grounds, and determining the second plane vertices of two adjacent target reconstructed grounds from the second image included in the image pair of different target reconstructed grounds, wherein the first plane vertex and the second plane vertex are at least two feature items in the associated feature items; determining the feature position difference value corresponding to each group of associated feature items, including: determining the difference between the depth value of the first plane vertex on the target reconstructed ground and the depth value of the second plane vertex on the target reconstructed ground as the feature position difference value corresponding to the associated feature item.

[0167] In this embodiment, it can be understood that when the items to be optimized in the basic point cloud reconstruction model include plane feature optimization items, it is necessary to obtain at least two feature items in at least one group of associated feature items based on the first image and the second image. To obtain at least two feature items in at least one group of associated feature items based on the first image and the second image, it is necessary to first determine the first plane vertices of the ground reconstructed by two adjacent targets from the first image included in the image pair of the ground reconstructed by different targets, and determine the second plane vertices of the ground reconstructed by two adjacent targets from the second image included in the image pair of the ground reconstructed by different targets, wherein the first plane vertex and the second plane vertex are at least two feature items in the associated feature items.

[0168] Based on at least two feature items in each determined group of associated feature items, a feature position difference value corresponding to each group of associated feature items is determined. The feature position difference value corresponding to each group of associated feature items is determined by the difference between the depth value of the first plane vertex on the target reconstructed ground and the depth value of the second plane vertex on the target reconstructed ground.

[0169] The standard deviation of the feature position difference values ​​corresponding to each group of associated feature items is determined as the local optimization target corresponding to the target reconstructed ground. According to the determined local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed. Among them, according to the local optimization targets corresponding to each target reconstructed ground, the basic point cloud reconstruction model corresponding to the ground to be reconstructed is optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed, including: according to the following formula (4), the global optimization target of the ground to be reconstructed is determined through the local optimization targets corresponding to each target reconstructed ground; the plane feature optimization items of the basic point cloud reconstruction model are optimized in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0170] Formula (4)

[0171] In formula (4), represents the global optimization objective of the ground to be reconstructed; Represents obtaining the depth value of the first plane vertex on the target reconstructed ground based on the horizontal and vertical coordinates of the first plane vertex in the first image; Represents obtaining the depth value of the second plane vertex on the target reconstructed ground based on the horizontal and vertical coordinates of the second plane vertex in the second image; Represents the i 1 The local optimization target corresponding to the ground reconstruction of two adjacent targets; M represents the i-th 1 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; N 1 Represents the total number of reconstructed ground planes between two adjacent objects.

[0172] The specific principle of optimizing the plane feature to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed and obtaining the target point cloud reconstruction model of the ground to be reconstructed can be referred to Figure 4 , Figure 4 A schematic diagram of the principle of determining a target point cloud reconstruction model of the ground to be reconstructed provided in an embodiment of the present application.

[0173] The embodiment of the present application optimizes the planar features to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed, which can ensure the smoothness of the reconstructed ground between two adjacent targets and provide a data basis for subsequent smooth point cloud reconstruction.

[0174] On the basis of the above-mentioned embodiment, a method for generating a point cloud reconstruction model also includes: acquiring a third image collected for the ground to be reconstructed; determining the initial values ​​of the extrinsic parameters of the acquisition device based on the third image and the principle of multi-view geometry; determining the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed based on the initial values ​​of the extrinsic parameters; and constructing a basic point cloud reconstruction model corresponding to the ground to be reconstructed based on the extrinsic parameter calibration relationship.

[0175] In this embodiment, it can be understood that if you want to construct a basic point cloud reconstruction model corresponding to the ground to be reconstructed, you first need to obtain a third image collected for the ground to be reconstructed. After obtaining the third image, determine the initial value of the external parameter of the acquisition device based on the third image and the principle of multi-view geometry; then determine the external parameter calibration relationship between the acquisition device and the ground to be reconstructed based on the initial value of the external parameter of the acquisition device; and then construct a basic point cloud reconstruction model corresponding to the ground to be reconstructed based on the external parameter calibration relationship. Among them, the acquisition device can be selected according to the actual situation, and the acquisition device includes but is not limited to the following: stereo camera, depth camera, three-dimensional scanning device.

[0176] The embodiment of the present application constructs a basic point cloud reconstruction model corresponding to the ground to be reconstructed, thereby providing more accurate data for subsequent point cloud reconstruction, thereby improving the accuracy of point cloud reconstruction.

[0177] Furthermore, according to the third image, based on the principle of multi-view geometry, the initial values ​​of the extrinsic parameters of the acquisition device are determined, including: according to the third image, the posture of the acquisition device is determined; based on the principle of multi-view geometry, the initial values ​​of the extrinsic parameters of the acquisition device are determined.

[0178] In this embodiment, it can be understood that, based on the information in the third image, the pose of the acquisition device is first determined, that is, the specific position and direction of the acquisition device in space are determined, and the pose of the acquisition device is achieved by analyzing the points in the third image and the relative positions of the corresponding points. After determining the pose of the acquisition device, the initial value of the external parameter of the acquisition device is calculated based on the principle of multi-view geometry. The principle of multi-view geometry infers the position and shape of objects in three-dimensional space by using images from multiple perspectives. In this process, the external parameters include the position and direction of the acquisition device in the world coordinate system, which are key parameters that describe the state of the acquisition device in space.

[0179] The embodiment of the present application determines the initial value of the extrinsic parameter of the acquisition device according to the third image based on the principle of multi-view geometry, thereby ensuring that the position and direction of the acquisition device in space are accurately described, thereby providing a reliable basis for subsequent point cloud reconstruction.

[0180] Furthermore, based on the initial values ​​of the extrinsic parameters, the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed is determined, including: determining the intersection of the characteristic ray of the fourth point in the third image and the ground to be reconstructed as the fifth point, and determining the three-dimensional coordinates of the fifth point; based on the initial values ​​of the extrinsic parameters, projecting the fifth point onto the fourth image acquired for the ground to be reconstructed to obtain the sixth point, and determining the point in the second image that is close to the coordinates of the sixth point as the seventh point, wherein the viewing angle for acquiring the fourth image is different from the viewing angle for acquiring the third image, and the fourth image and the third image constitute an image pair of the ground to be reconstructed; determining an optimization function based on the sixth point and the seventh point, and the sixth point and the seventh point constitute an associated feature item; optimizing the optimization function based on the optimization algorithm to determine the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed.

[0181] The above process can be expressed by the following formula (5):

[0182] Formula (5)

[0183] In formula (5), Represents the jth image pair corresponding to the ground to be reconstructed 3 The fifth point in the associated feature items; Represents the jth image pair corresponding to the ground to be reconstructed 3 The seventh point in the associated feature items; M represents the total number of associated feature items in the image pair corresponding to the ground to be reconstructed; A projection function representing the projection of the fifth point into a fourth image acquired for the ground to be reconstructed; Represents the optimization goal of the external parameter calibration relationship between the acquisition equipment and the ground to be reconstructed.

[0184] Furthermore, according to the initial value of the external parameter, the specific principle of determining the external parameter calibration relationship between the acquisition equipment and the ground to be reconstructed can be referred to Figure 5, Figure 5 Schematic diagram of determining the external parameter calibration relationship between the acquisition device and the ground to be reconstructed in the embodiment of the present application. Figure 5 The e in represents a 3D point, the parallelogram represents the ground to be reconstructed, and different boxes represent images from different perspectives. Figure 5 It can be seen that based on the principle of multi-view geometry, the characteristic ray of the fourth point in the third image will intersect with the ground to be reconstructed, and the intersection will obtain a 3D point. Based on the initial value of the extrinsic parameter, the 3D point is projected into the fourth image. The coordinates of the point corresponding to the projection of the 3D point to the fourth image are close to those of the 3D point. Based on the above, the optimization function is determined, and then the optimization function is optimized based on the optimization algorithm to determine the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed.

[0185] Furthermore, the optimization algorithm will optimize and iterate the above optimization function until a preset condition is met, wherein the preset condition can be set according to actual conditions. In one example, the preset condition is that the optimization function converges to a preset value.

[0186] Optionally, the optimization algorithm may be set according to actual conditions, and the embodiments of the present application do not impose any restrictions on this.

[0187] In one implementation, a gradient descent algorithm is selected as the optimization algorithm. The gradient descent algorithm is an iterative optimization algorithm that gradually updates parameters by calculating the gradient of the optimization function so that the optimization function gradually decreases. This method may fall into a local minimum, making the calibration relationship between the determined image data acquisition device and the external parameters of the ground to be reconstructed inaccurate, but this method is relatively simple and easy to implement.

[0188] In another implementation, the Levenberg-Marquardt algorithm is selected as the optimization algorithm. The Levenberg-Marquardt algorithm is an optimization algorithm for nonlinear minimization, which is usually used to fit model parameters to minimize the residual between observed data and the model. The Levenberg-Marquardt algorithm combines the advantages of the gradient descent algorithm and the Gauss-Newton algorithm, and converges quickly, but the algorithm needs to calculate the Jacobian matrix and is sensitive to the initial value.

[0189] In summary, different optimization algorithms have different advantages and disadvantages, and choosing a suitable optimization algorithm depends on actual needs. By choosing a suitable optimization algorithm, the external parameter calibration relationship between the image data acquisition device and the ground to be reconstructed can be determined more accurately.

[0190] Based on the above embodiments, Figure 6 A schematic diagram of a point cloud reconstruction method provided in an embodiment of the present application is shown in FIG. Figure 6As shown, the method includes: acquiring a target image collected for the ground to be reconstructed; processing the target image through the target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed, and the target point cloud reconstruction model is obtained through the above-mentioned point cloud reconstruction model generation method.

[0191] In this embodiment, it can be understood that if one wants to generate a ground point cloud corresponding to the ground to be reconstructed, it is necessary to obtain a target image collected for the ground to be reconstructed, and process the target image through the target point cloud reconstruction model of the ground to be reconstructed. Furthermore, before obtaining the target image collected for the ground to be reconstructed, it also includes: dividing the ground to be reconstructed into several local grounds; processing the target image through the target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed, including: processing the ground point cloud corresponding to each local ground through the target point cloud reconstruction model to generate a ground point cloud corresponding to the ground to be reconstructed. This means that the ground point cloud corresponding to the ground to be reconstructed is related to the ground point cloud corresponding to each local ground, and each local ground is obtained by dividing the ground to be reconstructed by a set size.

[0192] In one implementation, the ground to be reconstructed is divided into 4-meter-sized blocks to obtain multiple 4-meter × 4-meter-sized local ground blocks. The specific principle can be found in Figure 7 , Figure 7 A schematic diagram of dividing the ground to be reconstructed into multiple local grounds provided in an embodiment of the present application.

[0193] In another implementation, the ground to be reconstructed is divided into n×n local grounds, and each local ground covers an area within 3 meters of the center of the local ground.

[0194] The embodiment of the present application obtains the point cloud of the ground to be reconstructed based on the ground point clouds corresponding to each local ground, so that the generated ground point cloud can be closer to the real point cloud.

[0195] Furthermore, the target image is processed through the target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed, including: taking the point in the upper left corner of the target image as the starting point, traversing the points in the target image based on a preset direction and a preset step size, and determining the depth value of at least one traversed point according to the target point cloud reconstruction model of the ground to be reconstructed; based on the depth values ​​of each point in the target image, generating a ground point cloud corresponding to the ground to be reconstructed.

[0196] For example, after obtaining the target point cloud reconstruction model of the ground to be reconstructed, the point in the upper left corner of the target image is used as the starting point, and the points in the target image are traversed in the horizontal and vertical directions with a step size of 2, and the traversed points are brought into the target point cloud reconstruction model to determine the depth value corresponding to the point. Then, based on the depth value corresponding to the determined point, the ground point cloud corresponding to the ground to be reconstructed is obtained.

[0197] The embodiment of the present application processes the target image through a target point cloud reconstruction model of the ground to be reconstructed, and generates a ground point cloud corresponding to the ground to be reconstructed, so that the generated ground point cloud can be closer to the real point cloud.

[0198] On the basis of the above-mentioned embodiment, based on the depth coordinates of each point in the target image, a ground point cloud corresponding to the ground to be reconstructed is generated, including: determining the colorless ground point cloud corresponding to the ground to be reconstructed according to the depth value of each point in the target image; performing point cloud rendering on the colorless ground point cloud to obtain a colored ground point cloud corresponding to the ground to be reconstructed. This means that the ground point cloud obtained by processing the target image through the target point cloud reconstruction model of the ground to be reconstructed is a colorless ground point cloud. If a colored ground point cloud is to be obtained, the colorless ground point cloud needs to be rendered. Further, the colorless ground point cloud is rendered to obtain a colored ground point cloud corresponding to the ground to be reconstructed, including: for at least one colorless ground point cloud, constructing a K-dimensional tree containing at least one acquisition perspective of the colorless ground point cloud; based on each colorless ground point cloud, determining the first acquisition perspective corresponding to the colorless ground point cloud from the K-dimensional tree, the first acquisition perspective being the acquisition perspective with the minimum distance from the colorless ground point cloud; when the colorless ground point cloud is projected to the first acquisition perspective, the color of the colorless ground point cloud is determined as the color of the first acquisition perspective.

[0199] The embodiment of the present application performs point cloud rendering on a colorless ground point cloud to obtain a colored ground point cloud corresponding to the ground to be reconstructed, so that the obtained colored ground point cloud can be more consistent with the real point cloud.

[0200] Based on the above embodiments, Figure 8 A flowchart of an electronic map updating method provided in an embodiment of the present application is shown as follows: Figure 8 As shown, the electronic map updating method includes: obtaining a ground point cloud corresponding to the ground to be reconstructed through the above-mentioned point cloud reconstruction method; and updating the area corresponding to the ground to be reconstructed in the electronic map based on the ground point cloud corresponding to the ground to be reconstructed.

[0201] In one example, a ground point cloud corresponding to the ground to be reconstructed is obtained through the point cloud reconstruction method provided by the above-mentioned method embodiment; based on the ground point cloud corresponding to the ground to be reconstructed, the map annotation information of the area corresponding to the ground to be reconstructed in the electronic map is updated to provide the latest map annotation information.

[0202] In another example, the point cloud reconstruction method provided by the above-mentioned method embodiment is used to obtain a ground point cloud corresponding to the ground to be reconstructed; based on the ground point cloud corresponding to the ground to be reconstructed, the traffic signs, traffic lights and other information in the area corresponding to the ground to be reconstructed in the electronic map are updated to ensure that the map content is consistent with the actual road conditions, thereby assisting users to drive safely and accurately.

[0203] The embodiment of the present application can improve the accuracy of the electronic map by updating the area corresponding to the ground to be reconstructed in the electronic map according to the reconstructed ground.

[0204] Next, taking the point cloud reconstruction of a city street as an example, it is explained how to use the point cloud reconstruction method provided in the embodiment of the present application to reconstruct the point cloud of a city street. The overall flow chart of the method is as follows: Fig. 9 As shown, Fig. 9 The overall flow chart of point cloud reconstruction provided by the embodiment of the present application. Fig. 9 As shown, the overall process of this method is as follows:

[0205] 1. Obtain image data of the ground to be reconstructed collected at multiple viewing angles, and construct a plane equation corresponding to the ground to be reconstructed based on the image data;

[0206] 1.1 Obtain image data of the ground to be reconstructed collected from multiple perspectives;

[0207] 1.2 For image data from at least one of the multiple perspectives, determine the camera posture according to the image data; based on the principle of multi-perspective geometry, determine the three-dimensional coordinates of the feature points according to the camera posture; determine the initial value of the external parameter of the acquisition device when acquiring image data at the corresponding perspective according to the three-dimensional coordinates of the feature points; based on the image data from multiple perspectives, construct the association relationship of the feature points of the image data from multiple perspectives;

[0208] 1.3 According to the correlation relationship between feature points and the initial value of external parameters, determine the external parameter calibration relationship between the image data acquisition device and the ground to be reconstructed: determine the three-dimensional coordinates of the feature points according to the characteristic rays of the feature points in the reference view among multiple view angles and the ground to be reconstructed; project the feature points in the reference view angle to the query view angle based on the initial value of the external parameters; determine the optimization function according to the feature points in the reference view angle and the corresponding feature points projected to the query view angle; optimize the optimization function based on the optimization algorithm to determine the external parameter calibration relationship between the image data acquisition device and the ground to be reconstructed;

[0209] 1.4 According to the external parameter calibration relationship, construct the plane equation corresponding to the ground to be reconstructed;

[0210] 2. Optimize the items to be optimized corresponding to the plane equation to obtain the optimized plane equation, where the items to be optimized include numerical optimization items and / or regular optimization items: take the feature point in the upper left corner of the optimized plane equation as the starting point, traverse the feature points in the optimized plane equation based on a preset direction and a preset step size, and determine the depth coordinate of at least one traversed feature point; based on the depth coordinates of each feature point in the optimized plane equation, generate a ground point cloud of the ground to be reconstructed;

[0211] 3. Generate a ground point cloud of the ground to be reconstructed according to the optimized plane equation: Generate a colorless ground point cloud corresponding to the ground to be reconstructed according to the optimized plane equation; for at least one colorless ground point cloud, construct a K-dimensional tree including at least one acquisition perspective of the colorless ground point cloud; based on each colorless ground point cloud, determine a first acquisition perspective corresponding to the colorless ground point cloud from the K-dimensional tree, the first acquisition perspective being the acquisition perspective with the minimum distance from the colorless ground point cloud; when the colorless ground point cloud is projected to the first acquisition perspective, determine the color of the colorless ground point cloud as the color of the first acquisition perspective.

[0212] In summary, a method for point cloud reconstruction of a global ground provided in an embodiment of the present application can be divided into three steps: the first step is to estimate the extrinsic calibration relationship between the image data acquisition device and the ground to be reconstructed. This step assumes that there is a plane around the image data acquisition device, and estimates the extrinsic calibration relationship between the image data acquisition device and the ground to be reconstructed based on this plane. The calibration relationship can be used as the initial value of the ground to be reconstructed, so as to ensure the robustness of the reconstructed ground; the second step is to initialize the ground to be reconstructed according to the calibration relationship of the first step as the initial value. When optimizing the ground to be reconstructed, point features and line features are used as numerical optimization items of the ground to be reconstructed, and the plane equations that should be kept the same for adjacent local grounds are used as regularization items for overall optimization, and finally the optimized ground equation to be reconstructed is obtained; the third step is to generate a colorless ground point cloud according to the optimized ground equation to be reconstructed, and based on point cloud rendering, the colorless ground point cloud is rendered as a colored dense ground point cloud.

[0213] Furthermore, an embodiment of the present application provides a method for global ground point cloud reconstruction, which realizes a fast, robust and accurate dense reconstruction scheme of the ground by optimizing the ground equation. By optimizing the ground equation, it can effectively avoid the influence of noise generated in some deep blurred areas on the reconstruction effect, making the reconstruction effect smoother.

[0214] Furthermore, a point cloud reconstruction method provided in an embodiment of the present application only requires one camera and a central processing unit (CPU) to achieve the effect that requires radar or multiple graphics processing units (GPU).

[0215] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0216] Fig.10 A schematic diagram of the structure of the point cloud reconstruction model generation device provided in this application, such as Fig.10 As shown, the point cloud reconstruction model generation device 1000 provided in this embodiment includes:

[0217] A first determining module 1001 is used to determine at least one target reconstructed ground surface among the ground surfaces to be reconstructed;

[0218] An acquisition module 1002 is used to collect images of each target reconstructed ground surface from different viewing angles to obtain image pairs corresponding to each target reconstructed ground surface;

[0219] The second determination module 1003 is used to determine the local optimization target corresponding to each target reconstructed ground surface based on at least one set of associated feature items in the image pair corresponding to each target reconstructed ground surface;

[0220] The processing module 1004 is used to optimize the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground, so as to obtain the target point cloud reconstruction model of the ground to be reconstructed.

[0221] In a possible implementation, the processing module 1003 is specifically used to: optimize the items to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization targets corresponding to each target reconstructed ground, so as to obtain the target point cloud reconstruction model of the ground to be reconstructed;

[0222] In a possible implementation, the acquisition module 1002 is specifically used to: acquire a first image acquired through an acquisition perspective for reconstructing the target ground, acquire a second image acquired through a query perspective for reconstructing the target ground, and use the first image and the second image as an image pair corresponding to the target reconstructing the ground;

[0223] The second determination module 1003 is specifically used to: obtain at least one group of associated feature items based on the first image and the second image, each group of associated feature items includes at least two feature items that match in the first image and the second image; determine the local optimization target corresponding to the target reconstructed ground according to the geometric distribution relationship between at least two feature items in each group of associated feature items.

[0224] In a possible implementation, the second determination module 1003 is also used to: determine the feature position difference value corresponding to each group of associated feature items, the feature position difference value representing the position deviation of at least two feature items in the associated feature items of the corresponding group; and determine the local optimization target corresponding to the target reconstructed ground based on the distribution characteristics of the determined feature position difference values.

[0225] In a possible implementation, the second determination module 1003 is used to: determine the standard deviation of each feature position difference value as a local optimization target corresponding to the target reconstructed ground, and the standard deviation of each feature position difference value reflects the distribution characteristics of the corresponding feature position difference value;

[0226] In a possible implementation, the acquisition module 1002 is specifically used to: determine a first point in the first image; determine the intersection of the characteristic ray of the first point and the basic point cloud reconstruction model as the second point, and determine the three-dimensional coordinates of the second point; project the second point into the second image to obtain a projection point, and determine a point in the second image that is close to the coordinates of the projection point as a third point, wherein the first point, the second point, the projection point, and the third point are at least two feature items in the associated feature items; the second determination module 1003 is used to: determine the difference in distance between the projection point and the third point as the feature position difference value corresponding to the associated feature item.

[0227] In a possible implementation, the processing module 1004 is specifically configured to:

[0228] According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground, the global optimization target of the ground to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the point feature optimization items in the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model;

[0229]

[0230] in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground for the i-th target; Reconstruct the local optimization target corresponding to the ground for the i-th target; N Represents the total amount of target reconstructed ground; Represents the second point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; Represents the third point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the i-th target; Represents a projection function that projects the second point into the second image.

[0231] In a possible implementation, the acquisition module 1002 is specifically used to: determine a first line in a first image; determine the intersection of the first line and the basic point cloud reconstruction model as a second line, and determine the three-dimensional coordinates of the second line; project the second line into the second image to obtain a projection line, and determine a line in the second image that is close to the projection line coordinates as a third line; wherein the first line, the second line, the projection line, and the third line are at least two feature items in the associated feature items; the second determination module 1003 is specifically used to: determine the difference in distance between the projection line and the third line as a feature position difference value corresponding to the associated feature item.

[0232] In a possible implementation, the processing module 1004 is specifically used to: determine the global optimization target of the ground to be reconstructed based on the local optimization target corresponding to each target reconstructed ground according to the following formula; optimize the line feature optimization item of the basic point cloud reconstruction model in the optimization direction of taking the minimum value of the global optimization target to obtain the target point cloud reconstruction model;

[0233]

[0234] in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground on behalf of the nth target; Reconstruct the local optimization target corresponding to the ground for the nth target; N Represents the total amount of target reconstructed ground; The second line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; The third line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; M Represents the total number of associated feature items in the image pair corresponding to the reconstructed ground of the nth target; represents a projection function for projecting the second line into the second image.

[0235] In a possible implementation, the acquisition module 1002 is specifically used to: determine the first plane vertices of two adjacent target reconstructed ground from a first image included in an image pair of different target reconstructed grounds, and determine the second plane vertices of two adjacent target reconstructed grounds from a second image included in an image pair of different target reconstructed grounds, wherein the first plane vertex and the second plane vertex are at least two feature items in the associated feature items; the second determination module 1003 is specifically used to: determine the difference between the depth value of the first plane vertex on the target reconstructed ground and the depth value of the second plane vertex on the target reconstructed ground as the feature position difference value corresponding to the associated feature item.

[0236] In a possible implementation, the processing module 1004 is used to:

[0237] According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground, the global optimization target of the ground to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the plane feature optimization items of the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model;

[0238]

[0239] in, represents the global optimization objective of the ground to be reconstructed; Represents obtaining the depth value of the first plane vertex on the target reconstructed ground based on the horizontal and vertical coordinates of the first plane vertex in the first image; represents the depth value of the second plane vertex on the target reconstructed ground obtained based on the horizontal and vertical coordinates of the second plane vertex in the second image; M represents the i-th 1 The total number of associated feature items in the image pairs corresponding to the target reconstructed ground; N 1 Represents the total number of reconstructed ground surfaces between two adjacent targets; Represents the i 1 The local optimization objective of reconstructing the ground correspondence between two adjacent objects.

[0240] In a possible implementation, the processing module 1004 is also used to: obtain a third image captured for the ground to be reconstructed; determine the initial values ​​of the extrinsic parameters of the acquisition device based on the third image and the principle of multi-view geometry; determine the extrinsic parameter calibration relationship between the acquisition device and the ground to be reconstructed based on the initial values ​​of the extrinsic parameters; and construct a basic point cloud reconstruction model corresponding to the ground to be reconstructed based on the extrinsic parameter calibration relationship.

[0241] In a possible implementation, the processing module 1004 is further used to: determine the posture of the acquisition device according to the third image; and determine the initial value of the extrinsic parameter of the acquisition device according to the posture of the acquisition device based on the principle of multi-view geometry.

[0242] In one possible implementation, the processing module 1004 is used to: determine the intersection of the characteristic ray of the fourth point in the third image and the ground to be reconstructed as the fifth point, and determine the three-dimensional coordinates of the fifth point; based on the initial value of the extrinsic parameter, project the fifth point into the fourth image collected for the ground to be reconstructed to obtain a sixth point, and determine the point in the second image that is close to the coordinates of the sixth point as the seventh point, wherein the viewing angle for collecting the fourth image is different from the viewing angle for collecting the third image; determine an optimization function based on the sixth point and the seventh point; and optimize the optimization function based on the optimization algorithm to determine the extrinsic parameter calibration relationship between the collection device and the ground to be reconstructed.

[0243] Fig.11 A schematic diagram of the structure of a point cloud reconstruction device provided in an embodiment of the present application is shown in FIG. Fig.11 As shown, the point cloud reconstruction device 1100 provided in this embodiment includes:

[0244] An acquisition module 1101 is used to acquire a target image collected for the ground to be reconstructed;

[0245] The generation module 1102 is used to process the target image through the target point cloud reconstruction model of the ground to be reconstructed, and generate a ground point cloud corresponding to the ground to be reconstructed. The target point cloud reconstruction model is obtained through the above-mentioned point cloud reconstruction model generation method.

[0246] In a possible implementation, the generation module 1102 is specifically configured to: process the ground point clouds corresponding to each local ground surface through the target point cloud reconstruction model to generate a ground point cloud corresponding to the ground surface to be reconstructed.

[0247] In a possible implementation, the generation module 1102 is also used to: take the point in the upper left corner of the target image as the starting point, traverse the points in the target image based on a preset direction and with a preset step size, and reconstruct the model of the target point cloud of the ground to be reconstructed, determine the depth value of at least one traversed point; and generate a ground point cloud corresponding to the ground to be reconstructed based on the depth values ​​of each point in the target image.

[0248] In a possible implementation, the generation module 1102 is used to: determine the colorless ground point cloud corresponding to the ground to be reconstructed according to the depth value of each point in the target image; perform point cloud rendering on the colorless ground point cloud to obtain a colored ground point cloud corresponding to the ground to be reconstructed.

[0249] In one possible implementation, the generation module 1102 is specifically used to construct a K-dimensional tree including at least one acquisition perspective of the colorless ground point cloud for at least one colorless ground point cloud; based on each colorless ground point cloud, determine from the K-dimensional tree a first acquisition perspective corresponding to the colorless ground point cloud, the first acquisition perspective being the acquisition perspective having the minimum distance from the colorless ground point cloud; when the colorless ground point cloud is projected to the first acquisition perspective, determine the color of the colorless ground point cloud as the color of the first acquisition perspective.

[0250] Fig.12 A schematic diagram of the structure of the electronic map updating device provided in the embodiment of the present application is shown in FIG. Fig.12 As shown, the electronic map updating device 1200 provided in the embodiment of the present application includes:

[0251] An acquisition module 1201 is used to acquire a ground point cloud corresponding to the ground to be reconstructed through various possible implementations of the above point cloud reconstruction method;

[0252] The updating module 1202 is used to update the area corresponding to the ground to be reconstructed in the electronic map based on the ground point cloud corresponding to the ground to be reconstructed.

[0253] The point cloud reconstruction model generation device, point cloud reconstruction device, and electronic map updating device provided in this embodiment can execute the method provided in the above method embodiment. The implementation principles and technical effects are similar, and are not described in detail in this embodiment.

[0254] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the processing module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each module above can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.

[0255] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a module is implemented in the form of a processing element to allocate program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processor that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0256] Fig.13 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Fig.13 As shown, the electronic device 1300 provided in the embodiment of the present application may include: a processor 1301, and a memory 1302 communicatively connected to the processor, wherein:

[0257] Memory stores computer-executable instructions;

[0258] The processor executes the computer-executable instructions stored in the memory to implement the method described in the foregoing method embodiment.

[0259] It should be understood that the processor 1301 can be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor can be a microprocessor or the processor can be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as being executed by a hardware processor, or can be executed by a combination of hardware and software modules in the processor. The memory 1302 may include a high-speed random access memory (RAM), and may also include non-volatile storage NVM (non-volatile memory), such as at least one disk storage, and can also be a USB flash drive, a mobile hard disk, a read-only memory, a disk or an optical disk, etc.

[0260] Optionally, the electronic device 1300 may further include a communication interface 1303. In a specific implementation, if the communication interface 1303, the memory 1302 and the processor 1301 are implemented independently, the communication interface 1303, the memory 1302 and the processor 1301 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0261] Optionally, in a specific implementation, if the communication interface 1303, the memory 1302 and the processor 1301 are integrated on a chip, the communication interface 1303, the memory 1302 and the processor 1301 can communicate through an internal interface.

[0262] An embodiment of the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed, they are used to implement the method described in any of the aforementioned embodiments.

[0263] It is understood that the computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special-purpose computer.

[0264] An exemplary computer-readable storage medium is coupled to a processor so that the processor can read information from the computer-readable storage medium and write information to the computer-readable storage medium. Of course, the computer-readable storage medium can also be an integral part of the processor. The processor and the computer-readable storage medium can be located in an ASIC. Of course, the processor and the computer-readable storage medium can also exist in an electronic device as discrete components.

[0265] The above-mentioned integrated module implemented in the form of a software function module can be stored in a computer-readable storage medium. The above-mentioned software function module is stored in a computer-readable storage medium, including a number of instructions for enabling an electronic device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform some steps of the method described in each embodiment of the present application.

[0266] An embodiment of the present application also provides a computer program product, including a computer program, which implements the method described in any of the above embodiments when executed.

[0267] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0268] It should be further noted that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flow chart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0269] In the above embodiments, the description of each embodiment has its own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0270] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0271] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A method for generating a point cloud reconstruction model, characterized in that: include: Determine at least one target reconstruction ground surface among the ground surfaces to be reconstructed; Capturing images of each target reconstructed ground surface from different viewing angles to obtain image pairs corresponding to each target reconstructed ground surface; Determine a local optimization target corresponding to each target reconstructed ground surface based on at least one set of associated feature items in the image pairs corresponding to each target reconstructed ground surface; According to the following formula, according to the local optimization target corresponding to each target reconstructed ground, the items to be optimized in the basic point cloud reconstruction model corresponding to the ground to be reconstructed are optimized to obtain the target point cloud reconstruction model of the ground to be reconstructed, wherein the items to be optimized include at least one of a point feature optimization item, a line feature optimization item, and a plane feature optimization item; Among them, s is the identifier of the optimization item set for the basic point cloud reconstruction model; It is the global optimization target of the ground to be reconstructed corresponding to the optimization item s set for the basic point cloud reconstruction model; i2 is the identifier of the target reconstructed ground; N represents the total number of target reconstructed ground; M represents the total number of associated feature items in the image pair corresponding to the i2th target reconstructed ground; A feature item representing the qth associated feature item in the image pair corresponding to the reconstructed ground of the i2th target; Represents the qth associated feature item in the image pair corresponding to the i2th target reconstructed ground Another feature item that is different.

2. The point cloud reconstruction model generation method according to claim 1, characterized in that: The method of collecting images of each target reconstructed ground surface from different viewing angles to obtain image pairs corresponding to each target reconstructed ground surface includes: The following processing is performed on the ground reconstruction of each target: Acquire a first image acquired through a capture perspective to reconstruct the target ground, acquire a second image acquired through a query perspective to reconstruct the target ground, and use the first image and the second image as an image pair corresponding to the target reconstructed ground; The determining of the local optimization target corresponding to each target reconstructed ground surface based on at least one set of associated feature items in the image pair corresponding to each target reconstructed ground surface comprises: Acquire at least one group of associated feature items based on the first image and the second image, each group of associated feature items including at least two feature items that match in the first image and the second image; According to the geometric distribution relationship between at least two feature items in each group of associated feature items, a local optimization target corresponding to the target reconstructed ground is determined.

3. The method for generating a point cloud reconstruction model according to claim 2, characterized in that: The determining of the local optimization target corresponding to the target reconstructed ground according to the geometric distribution relationship of at least two feature items in each group of associated feature items includes: Determine a feature position difference value corresponding to each group of associated feature items, wherein the feature position difference value represents a position deviation of at least two feature items in the associated feature items of the corresponding group; Based on the determined distribution characteristics of the difference values ​​of each feature position, a local optimization target corresponding to the target reconstructed ground is determined.

4. The method for generating a point cloud reconstruction model according to claim 3, characterized in that: The determining of the local optimization target corresponding to the target reconstructed ground based on the determined distribution characteristics of the difference values ​​of each feature position includes: The standard deviation of each feature position difference value is determined as the local optimization target corresponding to the target reconstructed ground, and the standard deviation of each feature position difference value reflects the distribution characteristics of the corresponding feature position difference value.

5. The method for generating a point cloud reconstruction model according to claim 3, characterized in that: When the item to be optimized includes a point feature optimization item, the associated feature item is set for the point feature optimization item; The acquiring at least one set of associated feature items based on the first image and the second image comprises: The image pairs are processed in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pairs: determining a first point in the first image; Determine the intersection of the characteristic ray of the first point and the basic point cloud reconstruction model as the second point, and determine the three-dimensional coordinates of the second point; Projecting the second point into the second image to obtain a projection point, and determining a point in the second image that is close to the coordinates of the projection point as a third point, wherein the first point, the second point, the projection point, and the third point are at least two feature items in the associated feature items; The determining of the feature position difference values ​​corresponding to each group of associated feature items includes: The difference between the distances between the projection point and the third point is determined as the feature position difference value corresponding to the associated feature item.

6. The method for generating a point cloud reconstruction model according to claim 5, characterized in that: The method of optimizing the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground surface to obtain the target point cloud reconstruction model of the ground to be reconstructed includes: According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground, the global optimization target of the ground to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the point feature optimization items in the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model; in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground for the i-th target; The local optimization target corresponding to the ground reconstruction of the i-th target; N represents the total number of ground reconstruction targets; Represents the second point of the jth associated feature item in the image pair corresponding to the reconstructed ground of the i-th target; represents the third point of the jth associated feature item in the image pair corresponding to the i-th target reconstructed ground; M represents the total number of associated feature items in the image pair corresponding to the i-th target reconstructed ground; Represents a projection function that projects the second point into the second image.

7. The method for generating a point cloud reconstruction model according to claim 3, characterized in that: When the item to be optimized includes a line feature optimization item, the associated feature item is set for the line feature optimization item; The acquiring at least one set of associated feature items based on the first image and the second image comprises: The image pairs are processed in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pairs: determining a first line in a first image; Determine an intersection line of the first line and the basic point cloud reconstruction model as a second line, and determine the three-dimensional coordinates of the second line; Projecting the second line into the second image to obtain a projection line, and determining a line in the second image that is close to the projection line coordinates as a third line; wherein the first line, the second line, the projection line, and the third line are at least two feature items in the associated feature items; The determining of the feature position difference values ​​corresponding to each group of associated feature items includes: The difference between the distances between the projection line and the third line is determined as the feature position difference value corresponding to the associated feature item.

8. The method for generating a point cloud reconstruction model according to claim 7, characterized in that: The method of optimizing the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground surface to obtain the target point cloud reconstruction model of the ground to be reconstructed includes: According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground surface, the global optimization target of the ground surface to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the line feature optimization items of the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model; in, represents the global optimization objective of the ground to be reconstructed; Reconstruct the ground on behalf of the nth target; The local optimization target corresponding to the ground reconstruction of the nth target; N represents the total number of ground reconstruction targets; The second line representing the mth associated feature item in the image pair corresponding to the reconstructed ground of the nth target; represents the third line of the mth associated feature item in the image pair corresponding to the nth target reconstructed ground; M represents the total number of associated feature items in the image pair corresponding to the nth target reconstructed ground; represents a projection function for projecting the second line into the second image.

9. The method for generating a point cloud reconstruction model according to claim 3, characterized in that: When the item to be optimized includes a plane feature optimization item, the associated feature item is set for the plane feature optimization item; The acquiring at least one set of associated feature items based on the first image and the second image comprises: The image pairs are processed in the following manner to obtain at least two feature items in each group of associated feature items in the corresponding image pairs: Determine, from a first image included in an image pair for reconstructing the ground from different targets, first plane vertices for reconstructing the ground from two adjacent targets, and determine, from a second image included in an image pair for reconstructing the ground from different targets, second plane vertices for reconstructing the ground from two adjacent targets, wherein the first plane vertex and the second plane vertex are at least two feature items in the associated feature items; The determining of the feature position difference values ​​corresponding to each group of associated feature items includes: The difference between the depth value of the first plane vertex on the target reconstructed ground and the depth value of the second plane vertex on the target reconstructed ground is determined as the feature position difference value corresponding to the associated feature item.

10. The method for generating a point cloud reconstruction model according to claim 9, characterized in that: The method of optimizing the basic point cloud reconstruction model corresponding to the ground to be reconstructed according to the local optimization target corresponding to each target reconstructed ground surface to obtain the target point cloud reconstruction model of the ground to be reconstructed includes: According to the following formula, based on the local optimization targets corresponding to each target reconstructed ground surface, the global optimization target of the ground surface to be reconstructed is determined; in the optimization direction of taking the minimum value of the global optimization target, the plane feature optimization items of the basic point cloud reconstruction model are optimized to obtain the target point cloud reconstruction model; in, represents the global optimization objective of the ground to be reconstructed; Represents obtaining the depth value of the first plane vertex on the target reconstructed ground based on the horizontal and vertical coordinates of the first plane vertex in the first image; represents the depth value of the second plane vertex on the target reconstructed ground obtained based on the horizontal and vertical coordinates of the second plane vertex in the second image; M represents the total number of associated feature items in the image pair corresponding to the i1th target reconstructed ground; N1 represents the total number of two adjacent target reconstructed grounds; Represents the local optimization objective of reconstructing the ground correspondence between the i1th two adjacent targets.

11. The method for generating a point cloud reconstruction model according to any one of claims 1 to 10, characterized in that: Also includes: Acquire a third image collected for the ground to be reconstructed; According to the third image, based on the principle of multi-view geometry, determining the initial value of the external parameter of the acquisition device; Determining the external parameter calibration relationship between the acquisition device and the ground to be reconstructed according to the initial value of the external parameter; According to the external parameter calibration relationship, a basic point cloud reconstruction model corresponding to the ground to be reconstructed is constructed.

12. The method for generating a point cloud reconstruction model according to claim 11, characterized in that: Determining the external parameter calibration relationship between the acquisition device and the ground to be reconstructed according to the initial value of the external parameter includes: Determine the intersection of the characteristic ray of the fourth point in the third image and the ground to be reconstructed as the fifth point, and determine the three-dimensional coordinates of the fifth point; Based on the initial value of the external parameter, the fifth point is projected onto a fourth image collected for the ground to be reconstructed to obtain a sixth point, and a point in the second image close to the coordinates of the sixth point is determined as a seventh point, wherein a viewing angle for collecting the fourth image is different from a viewing angle for collecting the third image; According to the sixth point and the seventh point, determining an optimization function; The optimization function is optimized based on an optimization algorithm to determine an external parameter calibration relationship between the acquisition device and the ground to be reconstructed.

13. A point cloud reconstruction method, characterized in that: include: Acquire a target image collected for the ground to be reconstructed; The target image is processed by using the target point cloud reconstruction model of the ground to be reconstructed to generate a ground point cloud corresponding to the ground to be reconstructed, and the target point cloud reconstruction model is obtained by the point cloud reconstruction model generation method described in any one of claims 1 to 12.

14. The point cloud reconstruction method according to claim 13, characterized in that: Before acquiring the target image collected for the ground to be reconstructed, the method further includes: Dividing the ground to be reconstructed into a plurality of local grounds; The step of reconstructing the target point cloud model of the ground to be reconstructed, processing the target image, and generating a ground point cloud corresponding to the ground to be reconstructed includes: The ground point clouds corresponding to each local ground are processed by the target point cloud reconstruction model to generate the ground point clouds corresponding to the ground to be reconstructed.

15. The point cloud reconstruction method according to claim 13 or 14, characterized in that: The step of reconstructing the target point cloud model of the ground to be reconstructed, processing the target image, and generating a ground point cloud corresponding to the ground to be reconstructed includes: Taking the point at the upper left corner of the target image as the starting point, traversing the points in the target image based on a preset direction and with a preset step length, and reconstructing the model of the target point cloud of the ground to be reconstructed, determining the depth value of at least one traversed point; Based on the depth value of each point in the target image, a ground point cloud corresponding to the ground to be reconstructed is generated.

16. The point cloud reconstruction method according to claim 15, characterized in that: The step of generating a ground point cloud corresponding to the ground to be reconstructed based on the depth value of each point in the target image includes: Determine, according to the depth value of each point in the target image, a colorless ground point cloud corresponding to the ground to be reconstructed; Point cloud rendering is performed on the colorless ground point cloud to obtain a colored ground point cloud corresponding to the ground to be reconstructed.

17. An electronic map updating method, characterized in that: include: Obtaining a ground point cloud corresponding to the ground to be reconstructed by the point cloud reconstruction method described in any one of claims 13 to 16; Based on the ground point cloud corresponding to the ground to be reconstructed, the area corresponding to the ground to be reconstructed in the electronic map is updated.

18. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory is used to store computer-executable instructions; The processor is used to execute the computer-executable instructions to implement the method according to any one of claims 1 to 12, or 13 to 16, or 17.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 12, or 13 to 16, or 17 when executed.

20. A computer program product, characterized in that The invention comprises a computer program which, when executed, implements the method according to any one of claims 1 to 12, or 13 to 16, or 17.

Images