A method and apparatus for map construction

Abstract

The present specification discloses a method and device for map construction, and specifically discloses constructing an image sequence according to the acquisition time sequence of image data collected by a specified device during driving, then taking each frame of image data in the image sequence as a candidate composition image, determining image data whose acquisition interval between frames of image data of each candidate composition image is within a set interval range as a reference image, and then determining the pose change parameter of the specified device when collecting the reference image and the candidate composition image according to the reference image and the candidate composition image, and adding the candidate composition image to a composition image set if the pose change parameter is determined to fall within the determined pose change parameter range. In this way, the accuracy of the constructed map is improved.

Description

Technical Field

[0001] This specification relates to the field of autonomous driving technology, and in particular to a method and apparatus for map construction. Background Technology

[0002] When autonomous driving devices achieve autonomous driving, they need to rely on pre-set autonomous driving maps to realize technologies such as perception, localization, planning, and decision-making. Therefore, the construction and maintenance of autonomous driving maps is one of the important development directions in the field of autonomous driving technology.

[0003] In existing technologies, autonomous driving maps can be constructed based on image data collected by dashcams. When constructing an autonomous driving map, the image data collected by the dashcam needs to be input into a trained 3D reconstruction model to obtain the relative pose of the acquisition device when acquiring two adjacent frames of image data. Simultaneously, the image data collected by the dashcam is input into a depth prediction network to predict the depth value corresponding to each pixel in each frame of image data. Then, based on the relative pose of the acquisition device when acquiring two adjacent frames of image data and the predicted depth value corresponding to each pixel in each frame of image data, a 3D dense model can be performed to obtain 3D point cloud data of the road covered by the image data collected by the dashcam. The autonomous driving map can then be constructed based on the obtained 3D point cloud data.

[0004] In real-world scenarios, when the data acquisition device is traveling at a slow speed, the pose change of the device is not significant when two adjacent frames of image data are acquired by the dashcam. Directly performing 3D reconstruction based on such image data will make it difficult to accurately fit the relative pose of the acquisition device during the acquisition of those two adjacent frames, resulting in a low-accuracy map. If image data is extracted from the dashcam at set time intervals to construct the map, at faster speeds, the contiguous area between the extracted adjacent frames may be small. This will prevent the extraction of sufficient feature points to determine the relative pose of the acquisition device during the acquisition of those two frames, again leading to a low-accuracy map.

[0005] Therefore, how to select image data from the collected image data to construct the map in order to ensure the accuracy of the actual constructed map is an urgent problem to be solved. Summary of the Invention

[0006] This specification provides a method and apparatus for map construction, which partially solves the aforementioned problems existing in the prior art.

[0007] The following technical solution is adopted in this specification:

[0008] This manual provides a method for map construction, including:

[0009] Acquire image data collected by a specified device during operation, and construct an image sequence based on the acquisition time sequence corresponding to each image data.

[0010] For each frame of image data in the image sequence, the frame of image data is used as a candidate composition image, and image data whose acquisition interval is within a set interval range is determined as a reference image;

[0011] Based on the reference image and the candidate composition image, determine the pose change parameters of the designated device when acquiring the reference image and the candidate composition image. The pose change parameters include at least one of the following: the distance between the position of the designated device when acquiring the reference image and the position when acquiring the candidate composition image, and the change value of the turning angle of the designated device when acquiring the reference image and the candidate composition image.

[0012] If it is determined that the pose change parameters fall within the determined pose change parameter range, the candidate composition image is added to the composition image set;

[0013] Maps are constructed based on the image data contained in the image set.

[0014] Optionally, for each frame of image data in the image sequence, that frame of image data is used as a candidate composition image, specifically including:

[0015] For each frame of image data in the image sequence, determine the image quality parameters corresponding to that frame of image data. The image quality parameters include at least one of the following: the shutter speed when the specified device acquires the frame of image data, the proportion of the image corresponding to the dynamic object contained in the frame of image data in the frame of image data, and the illumination information corresponding to the frame of image data.

[0016] If the image quality parameters are determined to meet the preset quality parameter conditions, the image data of that frame is used as the candidate composition image.

[0017] Optionally, for each frame of image data in the image sequence, that frame of image data is used as a candidate composition image, and image data whose acquisition interval is within a set interval range is determined as a reference image, specifically including:

[0018] Other images whose acquisition timing is before the current frame image data and whose acquisition interval between them is within a set interval range are determined. The other images include: image data other than the current frame image data in the image sequence, and at least one of the image data outside the image sequence.

[0019] If it is determined that the image overlap between the frame image data and the other images is within a set overlap range, the frame image data is used as the candidate composition image, and the other images are used as the reference images.

[0020] Optionally, the pose change parameter range includes: a distance parameter range and a steering angle parameter range;

[0021] Determining that the pose change parameters fall within the determined pose change parameter range specifically includes:

[0022] If it is determined that the distance between the position of the designated device when acquiring the reference image and the position when acquiring the frame image data falls within the range of the distance parameter, and / or the change value of the steering angle of the designated device when acquiring the reference image and the frame image data falls within the range of the steering angle parameter, then it is determined that the pose change parameter falls within the determined pose change parameter range.

[0023] Optionally, the range of pose change parameters is determined, specifically including:

[0024] The model input parameters are determined, including at least one of the following: device information of the acquisition device on which the specified device acquires images, and image data acquired by the specified device;

[0025] The model input parameters are input into a pre-trained parameter prediction model to determine the range of pose change parameters.

[0026] Optionally, training a parameter prediction model specifically includes:

[0027] Obtain training samples, which include sample input parameters;

[0028] The sample input parameters are input into the parameter prediction model to obtain the predicted pose change parameter range;

[0029] Based on the range of the predicted pose change parameters, images for map construction are selected from each sample image and used as sample target images.

[0030] A map is constructed based on the sample target images to obtain a predicted map;

[0031] The deviation between the estimated map and the actual map is determined as the map deviation, and the parameter deviation corresponding to the estimated pose change parameter is determined based on the map deviation.

[0032] The parameter prediction model is trained with the goal of minimizing the parameter deviation.

[0033] Optionally, map construction is performed based on the image data contained in the image set, specifically including:

[0034] Based on the image data contained in the composition image set, a map is constructed to obtain a reference map;

[0035] The reference map is compared with the map already in use to obtain the comparison results;

[0036] If, based on the comparison results, it is determined that the road information within the road segment corresponding to the reference map has changed, the used map is reconstructed based on the reference map.

[0037] This specification provides a map building apparatus, comprising:

[0038] The data acquisition module is used to acquire image data collected by the specified device during operation, and to construct an image sequence based on the acquisition time sequence corresponding to each image data.

[0039] The reference image determination module is used to, for each frame of image data in the image sequence, take that frame of image data as a candidate composition image and determine the image data whose acquisition interval is within a set interval range from that frame of image data as a reference image;

[0040] The pose change parameter determination module is used to determine the pose change parameters of the designated device when acquiring the reference image and the candidate composition image based on the reference image and the candidate composition image. The pose change parameters include at least one of the following: the distance between the position of the designated device when acquiring the reference image and the position when acquiring the candidate composition image, and the change value of the steering angle of the designated device when acquiring the reference image and the candidate composition image.

[0041] The composition image selection module is used to add the candidate composition image to the composition image set if it is determined that the pose change parameter falls within the determined pose change parameter range.

[0042] The map building module is used to build a map based on the image data contained in the image set.

[0043] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described map construction method.

[0044] This specification provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described map construction method.

[0045] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0046] In the map construction method provided in this specification, image data collected by a specified device during operation is acquired, and an image sequence is constructed according to the acquisition time sequence corresponding to each image data. Then, for each frame of image data in the image sequence, that frame of image data is used as a candidate composition image, and image data whose acquisition interval between it and the frame of image data is within a set interval range is determined as a reference image. Next, based on the reference image and the candidate composition image, the pose change parameters of the specified device when acquiring the reference image and the candidate composition image are determined. Then, if the pose change parameters are determined to fall within the determined pose change parameter range, the candidate composition image is added to the composition image set. Finally, map construction is performed based on the image data contained in the composition image set. The pose change parameters include at least one of the following: the distance between the position of the specified device when acquiring the reference image and the position when acquiring the candidate composition image, and the steering angle change value of the specified device when acquiring the reference image and the candidate composition image.

[0047] As can be seen from the above method, this method filters the acquired image data based on the determined pose change parameters, identifies images whose pose change parameters meet the conditions, and then constructs a map based on the filtered image data. Since the pose change parameters reflect the degree of pose change when the acquisition device acquires different frames of images, by reasonably setting the conditions that the pose change parameters must meet, images whose pose change parameters meet the preset conditions can be extracted from the acquired image data, and a map can be constructed based on the extracted images, thereby improving the accuracy of the constructed map. Attached Figure Description

[0048] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and are used to explain this specification, but do not constitute an undue limitation thereof. In the drawings:

[0049] Figure 1 This is a flowchart illustrating a map construction method described in this specification.

[0050] Figure 2 This is a schematic diagram illustrating the selection of composition images for this instruction manual;

[0051] Figure 3 This is a schematic diagram illustrating the training process of the model for determining the range of pose change parameters in this specification.

[0052] Figure 4 This is a flowchart illustrating the process of selecting composition images for this instruction manual;

[0053] Figure 5 A schematic diagram of a map-building apparatus provided in this specification;

[0054] Figure 6 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0056] Currently, dashcams equipped with Global Navigation Satellite System (GNSS) data are widely deployed in vehicles. These dashcams not only collect image data but also acquire the vehicle's location information. Thus, the image data collected by the dashcam can be used to build high-precision maps. Therefore, when building autonomous driving maps based on dashcam image data, the hardware costs of map construction can be significantly reduced because dedicated sensors are not needed to collect user-generated map data.

[0057] When constructing a map based on image data, it is necessary to predict the relative pose of the camera when each two adjacent frames of images are acquired, as well as the depth value of each pixel in each frame of the acquired image. Based on the predicted relative pose of the camera when each two adjacent frames of images are acquired, and the depth value of each pixel in each frame of the acquired image, a 3D model is performed to obtain a digital orthophoto map (DOM) of the area covered by the image data.

[0058] When predicting the relative pose of the camera when capturing two adjacent image frames, feature points need to be extracted from each of the two frames, and feature point matching is performed to obtain the matching target feature points in the two adjacent frames. Then, based on the fact that the spatial location points corresponding to the target feature points are the same spatial location points, the relative pose of the acquisition device from the location of the acquisition device when capturing the previous frame to the location of the acquisition device when capturing the next frame is predicted. In this way, when the relative pose change of the camera when actually capturing two adjacent image frames is small, the image coordinate change of each feature point in the acquired image data is small, making it difficult to accurately determine the relative pose of the camera when actually capturing two adjacent image frames, resulting in a relatively low accuracy of the actually constructed map.

[0059] To address this issue, this specification provides a method for identifying target feature points. In its implementation, after acquiring image data for map construction, the method filters the image data based on determined pose change parameters, identifying images whose pose change parameters meet certain conditions. Map construction is then performed using these filtered images. The pose change parameters reflect the degree of pose change of the acquisition device when acquiring different frames of images. By appropriately setting the conditions that the pose change parameters must meet, images whose pose change parameters satisfy the preset conditions can be extracted from the acquired image data, and a map can be constructed based on these extracted images, thereby improving the accuracy of the constructed map.

[0060] The map construction scheme provided in this specification will be described in detail below with reference to the embodiments.

[0061] Figure 1 This is a flowchart illustrating a map construction method described in this specification, which specifically includes the following steps:

[0062] Step S100: Obtain image data collected by the specified device during driving, and construct an image sequence according to the acquisition time sequence corresponding to each image data.

[0063] The map construction method provided in this specification is used to construct high-precision maps for the aforementioned assisted autonomous driving devices to achieve autonomous driving. These high-precision maps contain accurate geographic location information and rich semantic information about road elements, which can help autonomous vehicles anticipate complex road conditions such as slope, curvature, and heading, thus better avoiding potential risks. This is one of the key technologies for achieving autonomous driving.

[0064] When building a map, the terminal device first acquires image data for map construction. Specifically, the terminal device acquires image data collected by the specified device during operation and constructs an image sequence based on the acquisition time sequence corresponding to each image data. Then, the terminal device builds the map based on the constructed image sequence.

[0065] The aforementioned designated device can refer to any device equipped with a GNSS-enabled dashcam, such as transport vehicles (freight trucks, passenger vehicles), cars, driverless vehicles, dedicated data collection equipment, and other devices capable of operating on roads. There may be one or more designated devices; when there is only one designated device...

[0066] The image data collected by the aforementioned designated device during driving can be the same as the image data collected by the aforementioned dashcam during driving. While collecting image data, the dashcam will also obtain the location information of the designated device based on GNSS functionality.

[0067] It should be noted that the execution entity of the map construction method provided in this specification can be either the terminal device mentioned above or a server providing business support for autonomous driving of unmanned vehicles. For ease of description, the following example will only use a terminal device as the execution entity.

[0068] It should also be noted that the map constructed in this specification can be used to enable autonomous driving of unmanned vehicles. Unmanned vehicles can refer to unmanned vehicles, robots, automated delivery equipment, and other devices capable of autonomous driving. Based on this, unmanned vehicles equipped with the features provided in this specification can be used to perform delivery tasks in the delivery field, such as in business scenarios involving the delivery of express packages, logistics, and food delivery.

[0069] Step S102: For each frame of the image sequence, the image data of that frame is taken as a candidate composition image, and the image data whose acquisition interval is within a set interval range is determined as reference image data.

[0070] In practice, when determining candidate mapping images, the terminal device will first perform quality detection on each frame of image data in the constructed image sequence, and then only the image data that passes the quality detection will be used as candidate mapping images that can be used to construct the map.

[0071] Specifically, the terminal device will determine the image quality parameters corresponding to each frame of the image sequence, and then determine whether the image quality parameters meet the preset quality parameter conditions. If they do, the image frame will be used as a candidate image for map construction. If they do not meet the conditions, the image data will be discarded.

[0072] The image quality parameters mentioned above may include at least one of the following: the shutter speed when the specified device acquires the image data of the frame, the proportion of the image corresponding to the dynamic object in the image data of the frame, and the illumination information corresponding to the image data of the frame.

[0073] Specifically, when a designated device acquires image data, if the camera's shutter speed is too short, less light enters the lens, resulting in a generally dark image. Conversely, if the shutter speed is too long, less light enters the lens, resulting in a generally bright image. When image data is too bright or too dark, the image contrast drops significantly, leading to blurry images. Predicting relative pose based on such image data may result in inaccurate predictions. Therefore, this specification considers the shutter speed of the designated device when acquiring the image frame as a factor influencing image quality. Only when the shutter speed falls within a set range is the image data selected as a candidate image for map construction.

[0074] Furthermore, the map constructed in this specification is primarily used to record the semantic information of road elements on each road. When the acquired image data contains a large number of moving objects, these moving objects will obscure the road elements, thus affecting the recognition and extraction of road elements. Therefore, this specification will also detect moving objects in the acquired image data, selecting image data containing fewer or no moving objects to construct the map. Accordingly, this specification uses the shutter speed of the specified device when acquiring the image data frame as a factor influencing the image quality. When the proportion of images containing moving objects in the image data frame reaches a set threshold, the image data will be discarded.

[0075] Furthermore, when ambient light is strong, strong reflected light may enter the lens. In this case, the image data acquired by the acquisition device may contain overexposed light spots. Since there is no corresponding valid image information at the light spots, the image information in that frame is incomplete, affecting the recognition and extraction of road elements. Therefore, this specification considers the illumination information corresponding to the image data as a factor influencing the image quality. When the proportion of light spots in the image data reaches a set threshold, the image data will be discarded.

[0076] In practical applications, relative pose prediction is based on image data within a shared viewing area in two adjacent frames. Therefore, the proportion of the image corresponding to a dynamic object in that frame of image data can refer to the proportion of the image corresponding to the dynamic object within the shared viewing area. Similarly, the proportion of the light spot in that frame of image data can refer to the proportion of the light spot located within the shared viewing area within the shared viewing area.

[0077] In this specification, after selecting image data whose image quality parameters meet the preset quality parameter conditions, each selected frame of image data can be used as a candidate composition image, and a reference image corresponding to the candidate composition image can be determined. Based on the pose change parameters of the specified device when acquiring the reference image and the candidate composition image, it is determined whether to add the candidate composition image to the composition image set to construct a map.

[0078] In specific implementation, for each frame in the constructed image sequence, the terminal device identifies other images whose acquisition time is before that frame and whose acquisition interval is within a set interval range. Then, it determines whether the image overlap between that frame and other images is within a set overlap range. If so, that frame is used as a candidate composition image, and the other images are used as reference images. If not, that frame is discarded.

[0079] In the constructed image sequence, if the image data used for map construction is searched forward, and the current image frame is located at the beginning of the image sequence (i.e., an image acquired earlier in the image sequence), then the other images determined based on this image data can include image data other than that frame in the image sequence, as well as image data preceding it. Conversely, if the image data used for map construction is searched backward, and the current image frame is located at the end of the image sequence (i.e., an image acquired earlier in the image sequence), then the other images determined based on this image data can include image data other than that frame in the image sequence, as well as image data following it.

[0080] In specific implementation, see Figure 2When the terminal device filters image data to form a composition image set according to the acquisition time sequence, if the image data acquired at time t has already been selected as the composition image, then the next frame of image data following the image data acquired at time t (such as the image data acquired at time t+1) can be determined from the image sequence according to the acquisition time sequence. It is then determined whether the image quality parameters of this image data meet the image quality requirements. If they do, the location of the specified device when acquiring the image data at time t+1, the distance between this location and the location when acquiring the image data at time t, and the change in the turning angle between acquiring the image data at time t+1 and the image data at time t are determined, and a judgment is made. If the distance or the change in steering angle falls within the set distance parameter range, or if the change in steering angle falls within the set steering angle parameter range, then the image data at time t+1 is used as the image data for map construction and stored in the mapping image set. Otherwise, the image data at time t+1 is discarded, and image data collected at time t+2 is acquired. The system records whether the image data collected at time t+2 meets the above requirements. If it does, the image data at time t+2 is used as the image data for map construction and stored in the mapping image set. Otherwise, the image data at time t+2 is discarded, and image data collected at the next acquisition time is acquired, until image data that meets the above requirements is determined. Then, based on the determined image data that meets the above requirements, the system continues to check whether each frame of image data following that image data meets the above requirements. This process is repeated until no new image can be extracted from the image data.

[0081] After obtaining each candidate composition image and its corresponding reference image, the terminal device will determine whether to add each candidate composition image to the composition image set based on the candidate composition image and its corresponding reference image, so as to construct a map based on the image data contained in the composition image set.

[0082] Step S104: Based on the reference image data and the candidate composition image, determine the pose change parameters of the designated device when acquiring the reference image and the candidate composition image. The pose change parameters include at least one of the following: the distance between the position of the designated device when acquiring the reference image and the position when acquiring the candidate composition image, and the change value of the turning angle of the designated device when acquiring the reference image and the candidate composition image.

[0083] Step S106: If it is determined that the pose change parameter falls within the determined pose change parameter range, the candidate composition image is added to the composition image set.

[0084] The aforementioned pose transformation parameters are used to measure the degree of pose change of a specified device when acquiring reference images and candidate composition images. At least two parameters are used to measure the degree of pose change of the specified device. For example, the distance between the position of the specified device when acquiring the reference image and its position when acquiring the candidate composition image; the greater the distance, the greater the degree of pose change of the specified device. Another example is the change in the steering angle of the specified device when acquiring reference images and candidate composition images; the greater the change in the steering angle, the greater the degree of pose change of the specified device.

[0085] Accordingly, for each pose transformation parameter, there will be a corresponding pose transformation parameter range. When it is determined that there is at least one pose transformation parameter that falls within the corresponding pose transformation parameter range, the candidate composition image is added to the composition image set.

[0086] Therefore, when the pose transformation parameter is the distance between the position of the specified device when acquiring the reference image and the position when acquiring the candidate composition image, the range of pose change parameters corresponding to this pose transformation parameter is the range of distance parameters. When the pose transformation parameter is the change value of the steering angle of the specified device when acquiring the reference image and the candidate composition image, the range of pose change parameters corresponding to this pose transformation parameter is the range of steering angle parameters.

[0087] Thus, the pose change parameter is determined to fall within the defined pose change parameter range when the following conditions are met: For example, if the distance between the location of the specified device when acquiring the reference image and its location when acquiring the candidate composition image falls within the defined distance parameter range, then the pose change parameter is determined to fall within the defined pose change parameter range. Another example is if the change in the steering angle of the specified device when acquiring both the reference image and the candidate composition image falls within the defined steering angle parameter range, then the pose change parameter is determined to fall within the defined pose change parameter range. Yet another example is if the distance between the location of the specified device when acquiring the reference image and its location when acquiring the candidate composition image falls within the defined distance parameter range, and simultaneously, the change in the steering angle of the specified device when acquiring both the reference image and the candidate composition image falls within the defined steering angle parameter range, then the pose change parameter is determined to fall within the defined pose change parameter range.

[0088] In practical applications, the size of the shared field of view (COP) between adjacent image data can affect the accuracy of the constructed map. The COP between image data acquired by a camera is influenced by lens intrinsic parameters; for example, the field of view of a wide-angle camera is larger than that of a fisheye camera. Therefore, to ensure a sufficiently large COP between the image data extracted based on pose change parameters, it is necessary to establish a corresponding pose change parameter range based on the camera type. Furthermore, the camera's posture during image acquisition also affects the size of the COP between the acquired image data. Therefore, when setting the pose change parameter range, the camera's posture during image acquisition may also need to be considered.

[0089] Based on this, the terminal device in this specification will determine the range of pose change parameters to be used according to the situation of the acquisition device on the specified device when acquiring image data.

[0090] Specifically, the terminal device determines the model input parameters and inputs these parameters into a pre-trained parameter prediction model to determine the range of pose change parameters applicable to the current image data. The model input parameters may include at least one of the following: device information of the acquisition device on which the specified device acquires the image, and the image data acquired by the specified device.

[0091] The device information of the acquisition device on which the specified device acquires images refers to the type of acquisition device (i.e., the type of camera used) and the posture of the acquisition device when acquiring images (i.e., the posture information of the camera installed on the specified device).

[0092] Step S108: Construct a map based on the image data contained in the image set.

[0093] In practice, the terminal device constructs a map based on the image data contained in the determined image set to obtain a reference map. Then, it compares the reference map with the map already in use to obtain a comparison result. If the comparison result indicates that the road information in the road segment corresponding to the reference map has changed, the map already in use is reconstructed based on the reference map.

[0094] In determining the reference map, the terminal device inputs the image data contained in the image set into a trained 3D reconstruction model to obtain the matching feature points in two adjacent frames of the image set, as well as the relative poses of the device when acquiring those two adjacent frames. Simultaneously, the terminal device also inputs the image data contained in the image set into a trained depth prediction model to obtain the depth value corresponding to each pixel in each frame. Then, based on the matching feature points in two adjacent frames of the image set, the relative poses of the device when acquiring those two adjacent frames, and the depth values ​​of each pixel in each frame, the terminal device performs dense 3D modeling to obtain a 3D model of the road area covered by the image data contained in the image set. Finally, the terminal device projects this 3D model of the road area onto a horizontal plane to obtain a reference map of the road area covered by the image data contained in the image set.

[0095] Furthermore, the terminal device compares the reference map with the already used map. Based on the comparison results, when it is determined that the road information in the road segment corresponding to the reference map has changed, it identifies each road element from both the reference map and the already used map, and then compares each road element one by one. If it is determined that there are road elements in the reference map that are inconsistent with the road elements in the already used map, it is determined that the road information in the road segment corresponding to the reference map has changed, and the already used map needs to be reconstructed based on the reference map.

[0096] Through the above steps, the acquired image data is filtered based on the determined pose change parameters to identify images whose pose change parameters meet the specified conditions. Maps are then constructed based on the filtered image data. Since these pose change parameters reflect the degree of pose change when the acquisition device captures different frames of images, by reasonably setting the conditions that the pose change parameters must meet, images whose pose change parameters meet the preset conditions can be extracted from the acquired image data. Maps can then be constructed based on these extracted images, thereby improving the accuracy of the constructed map.

[0097] In addition, this manual also provides the training method for the above parameter prediction model, see [link / reference]. Figure 3 .

[0098] Step S300: Obtain training samples, wherein the training samples include sample input parameters.

[0099] In this step, the sample input parameters in the training samples acquired by the terminal device are consistent with those used in actual applications. These sample input parameters may include: sample images, and device information of the acquisition device on which the specified device acquired the sample images.

[0100] Step S302: Input the sample input parameters into the parameter prediction model to obtain the predicted pose change parameter range.

[0101] Step S304: Based on the estimated pose change parameter range, select images from each sample image to construct a map, and use them as sample target images.

[0102] In this step, the process by which the terminal device determines the sample target image based on the range of the estimated pose change parameters is consistent with the actual time taken, so it will not be described in detail here.

[0103] Step S306: Construct a map based on the sample target image to obtain a predicted map.

[0104] In this step, the time taken for the terminal device to construct the map based on the sample target image is the same as the actual time taken, so it will not be described in detail.

[0105] Step S308: Determine the deviation between the estimated map and the actual map as the map deviation, and determine the parameter deviation corresponding to the estimated pose change parameter based on the map deviation.

[0106] Step S310: Train the parameter prediction model with the goal of minimizing the parameter deviation.

[0107] In steps S308 and S310, when the terminal device determines the deviation between the estimated map and the actual map, it identifies each road element contained in the estimated map and determines the region center of the image area of ​​each road element. Then, it obtains the image area of ​​each road element from the actual map and determines the region center. Next, it determines the distance between the region centers of each road element in the estimated map and the region centers of each road element in the actual map, and uses this distance as the map deviation. Based on this map deviation, it determines the parameter deviation corresponding to the estimated pose change parameters, and trains the parameter prediction model with the optimization objective of minimizing this parameter deviation.

[0108] In this way, through repeated debugging, the parameter prediction model will autonomously learn the range of predicted pose change parameters required to construct a map with good quality under the current sample input parameters.

[0109] The following diagram, with reference to illustrations, details the process of extracting image data for map construction from the initial collected image data. See attached diagram for further details. Figure 4 .

[0110] In step S400, the terminal device extracts a new frame of image data from the original image dataset according to the order of acquisition time, and obtains the frame of composition image data whose acquisition time is closest to that frame of image data as the reference image data.

[0111] In step S402, the terminal device determines the image quality parameters corresponding to the frame image data.

[0112] In step S404, the terminal device determines whether each image quality parameter meets the preset quality parameter conditions. If it does, step S406 is executed; otherwise, step S418 is executed.

[0113] In step S406, the terminal device uses the frame image data as a candidate composition image that can be used to construct a map.

[0114] Step S408: The terminal device determines the distance between the positions of the specified device when acquiring the reference image and the candidate image.

[0115] In step S410, the terminal device determines the change value of the steering angle when the designated device acquires the reference image and the candidate composition image.

[0116] In step S412, the terminal device determines whether the determined distance falls within the range of the set distance parameter. If it does not fall within the range of the set distance parameter, step S414 is executed; otherwise, step S416 is executed.

[0117] In step S414, the terminal device determines whether the determined change in steering angle falls within the range of the set steering angle parameter. If it falls within the range of the set distance parameter, step S416 is executed; otherwise, step S418 is executed.

[0118] In step S416, the terminal device stores the frame image data into the composition image set and returns to step S300 until every frame image data in the original image dataset has been filtered.

[0119] In step S418, the terminal device discards the image data of that frame.

[0120] There is no necessary connection between the execution order of steps S402, S406, and S408; they can be performed simultaneously.

[0121] The above describes a map construction method provided by one or more embodiments of this specification. Based on the same idea, this specification also provides a corresponding map construction apparatus, such as... Figure 5 As shown.

[0122] Figure 5 This specification provides a schematic diagram of a map-building apparatus, specifically including:

[0123] The data acquisition module 500 is used to acquire image data collected by the specified device during driving, and to construct an image sequence according to the acquisition time sequence corresponding to each image data.

[0124] The reference image determination module 501 is used to, for each frame of image data in the image sequence, take the frame of image data as a candidate composition image and determine the image data whose acquisition interval is within a set interval range from the frame of image data as a reference image;

[0125] The pose change parameter determination module 502 is used to determine the pose change parameters of the designated device when acquiring the reference image and the candidate composition image based on the reference image and the candidate composition image. The pose change parameters include at least one of the following: the distance between the position of the designated device when acquiring the reference image and the position when acquiring the candidate composition image, and the change value of the steering angle of the designated device when acquiring the reference image and the candidate composition image.

[0126] The composition image selection module 503 is used to add the candidate composition image to the composition image set if it is determined that the pose change parameter falls within the determined pose change parameter range.

[0127] The map building module 504 is used to build a map based on the image data contained in the map image set.

[0128] Optionally, the reference image determination module 501 is specifically used to determine the image quality parameters corresponding to each frame of image data in the image sequence. The image quality parameters include at least one of the following: the shutter speed when the designated device acquires the frame of image data, the proportion of the image corresponding to the dynamic object contained in the frame of image data in the frame of image data, and the illumination information corresponding to the frame of image data; if the image quality parameters are determined to meet the preset quality parameter conditions, the frame of image data is used as the candidate composition image.

[0129] Optionally, the reference image determination module 501 is specifically used to determine other images whose acquisition time is before the current frame image data and whose acquisition interval with the current frame image data is within a set interval range. The other images include at least one of the following: image data located in the image sequence other than the current frame image data, and image data located outside the image sequence. If it is determined that the image overlap between the current frame image data and the other images is within a set overlap range, the current frame image data is used as the candidate composition image, and the other images are used as the reference image.

[0130] Optionally, the pose change parameter range includes: a distance parameter range and a steering angle parameter range;

[0131] The image selection module 503 is specifically used to determine that the pose change parameter falls within the determined pose change parameter range if the distance between the position of the designated device when acquiring the reference image and the position when acquiring the frame image data falls within the range of the distance parameter, and / or the change value of the steering angle of the designated device when acquiring the reference image and the frame image data falls within the range of the steering angle parameter.

[0132] Optionally, the image selection module 503 is specifically used to determine the model input parameters, which include at least one of the following: device information of the acquisition device on which the specified device acquires the image, and image data acquired by the specified device; and inputting the model input parameters into a pre-trained parameter prediction model to determine the pose change parameter range.

[0133] Optionally, the device further includes:

[0134] The training module 505 is used to acquire training samples, which include sample input parameters; input the sample input parameters into the parameter prediction model to obtain the range of predicted pose change parameters; select images for map construction from each sample image according to the range of predicted pose change parameters, as sample target images; construct a map based on the sample target images to obtain a predicted map; determine the deviation between the predicted map and the actual map as the map deviation, and determine the parameter deviation corresponding to the predicted pose change parameters based on the map deviation; and train the parameter prediction model with minimizing the parameter deviation as the optimization objective.

[0135] Optionally, the map construction module 504 is specifically used to construct a map based on the image data contained in the image set to obtain a reference map; compare the reference map with the used map to obtain a comparison result; if, based on the comparison result, it is determined that the road information in the road segment corresponding to the reference map has changed, the used map is reconstructed based on the reference map.

[0136] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The provided method for detecting faulty areas.

[0137] This instruction manual also provides Figure 6 The diagram shows a schematic structural representation of the electronic device. Figure 4At the hardware level, the electronic device includes a processor, an internal bus, a network interface, memory, and non-volatile storage, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile storage into memory and then runs it to achieve the above-mentioned functions. Figure 1 The method for detecting the fault area is described above. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0138] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0139] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0140] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0141] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware.

[0142] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0143] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0144] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0145] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0148] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0149] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0150] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0152] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0153] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A method for map construction, characterized in that, include: Acquire image data collected by a specified device during operation, and construct an image sequence based on the acquisition time sequence corresponding to each image data. For each frame of image data in the image sequence, the frame of image data is used as a candidate composition image, and image data whose acquisition interval is within a set interval range is determined as a reference image; Based on the reference image and the candidate composition image, determine the pose change parameters of the designated device when acquiring the reference image and the candidate composition image. The pose change parameters include at least one of the following: the distance between the position of the designated device when acquiring the reference image and the position when acquiring the candidate composition image, and the change value of the turning angle of the designated device when acquiring the reference image and the candidate composition image. If it is determined that the pose change parameters fall within the determined pose change parameter range, the candidate composition image is added to the composition image set; Maps are constructed based on the image data contained in the image set. Determine the range of pose change parameters, specifically including: The model input parameters are determined, including at least one of the following: device information of the acquisition device on which the specified device acquires images, and image data acquired by the specified device; The model input parameters are input into a pre-trained parameter prediction model to determine the range of pose change parameters; training the parameter prediction model specifically includes: Obtain training samples, which include sample input parameters; The sample input parameters are input into the parameter prediction model to obtain the predicted pose change parameter range; Based on the range of the predicted pose change parameters, images for map construction are selected from each sample image and used as sample target images. A map is constructed based on the sample target images to obtain a predicted map; The deviation between the estimated map and the actual map is determined as the map deviation, and the parameter deviation corresponding to the estimated pose change parameter is determined based on the map deviation. The parameter prediction model is trained with the goal of minimizing the parameter deviation.

2. The method as described in claim 1, characterized in that, For each frame of image data in the image sequence, that frame of image data is used as a candidate composition image, specifically including: For each frame of image data in the image sequence, determine the image quality parameters corresponding to that frame of image data. The image quality parameters include at least one of the following: the shutter speed when the specified device acquires the frame of image data, the proportion of the image corresponding to the dynamic object contained in the frame of image data in the frame of image data, and the illumination information corresponding to the frame of image data. If the image quality parameters are determined to meet the preset quality parameter conditions, the image data of that frame is used as the candidate composition image.

3. The method as described in claim 1, characterized in that, For each frame of image data in the image sequence, that frame of image data is used as a candidate composition image, and image data whose acquisition interval is within a set interval range is determined as a reference image, specifically including: Other images whose acquisition timing is before the current frame image data and whose acquisition interval between them is within a set interval range are determined. The other images include: image data other than the current frame image data in the image sequence, and at least one of the image data outside the image sequence. If it is determined that the image overlap between the frame image data and the other images is within a set overlap range, the frame image data is used as the candidate composition image, and the other images are used as the reference images.

4. The method as described in claim 1, characterized in that, The pose change parameter range includes: the distance parameter range and the steering angle parameter range; Determining that the pose change parameters fall within the determined pose change parameter range specifically includes: If it is determined that the distance between the position of the designated device when acquiring the reference image and the position when acquiring the frame image data falls within the range of the distance parameter, and / or the change value of the steering angle of the designated device when acquiring the reference image and the frame image data falls within the range of the steering angle parameter, then it is determined that the pose change parameter falls within the determined pose change parameter range.

5. The method as described in claim 1, characterized in that, Map construction is performed based on the image data contained in the image set, specifically including: Based on the image data contained in the composition image set, a map is constructed to obtain a reference map; The reference map is compared with the map already in use to obtain the comparison results; If, based on the comparison results, it is determined that the road information within the road segment corresponding to the reference map has changed, the used map is reconstructed based on the reference map.

6. A fault area detection device, characterized in that, include: The data acquisition module is used to acquire image data collected by the specified device during operation, and to construct an image sequence based on the acquisition time sequence corresponding to each image data. The reference image determination module is used to, for each frame of image data in the image sequence, take that frame of image data as a candidate composition image and determine the image data whose acquisition interval is within a set interval range from that frame of image data as a reference image; The pose change parameter determination module is used to determine the pose change parameters of the designated device when acquiring the reference image and the candidate composition image based on the reference image and the candidate composition image. The pose change parameters include at least one of the following: the distance between the position of the designated device when acquiring the reference image and the position when acquiring the candidate composition image, and the change value of the steering angle of the designated device when acquiring the reference image and the candidate composition image. The composition image selection module is used to add the candidate composition image to the composition image set if it is determined that the pose change parameter falls within the determined pose change parameter range. The map building module is used to build a map based on the image data contained in the image set. Determine the range of pose change parameters, specifically including: The model input parameters are determined, including at least one of the following: device information of the acquisition device on which the specified device acquires images, and image data acquired by the specified device; The model input parameters are input into a pre-trained parameter prediction model to determine the range of pose change parameters; Training parameter prediction models, specifically including: Obtain training samples, which include sample input parameters; The sample input parameters are input into the parameter prediction model to obtain the predicted pose change parameter range; Based on the range of the predicted pose change parameters, images for map construction are selected from each sample image and used as sample target images. A map is constructed based on the sample target images to obtain a predicted map; The deviation between the estimated map and the actual map is determined as the map deviation, and the parameter deviation corresponding to the estimated pose change parameter is determined based on the map deviation. The parameter prediction model is trained with the goal of minimizing the parameter deviation.

7. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 5.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 5.

Images