A high-precision map construction method, device, storage medium and electronic device

Abstract

This specification discloses a method, device, storage medium and electronic device for constructing a high-precision map. In the embodiments of this specification, the standard depth interval corresponding to each pixel in the target image is determined based on the reference depth map corresponding to the target image in the binocular image and the recognition result of the object category in the target image. Among them, the object categories include static objects and dynamic objects. According to the standard depth interval corresponding to each pixel, the depth value obtained by mapping each point cloud point to the target image is filtered to obtain the depth map corresponding to the static object in the target image, and a high-precision map is constructed based on the depth map. In this method, according to the standard depth interval of each pixel, the depth value of the dynamic object point cloud point mapping on each pixel is filtered to obtain the depth value of the static object point cloud point mapping, which can avoid the problem of overlapping of the dynamic object point cloud points and the static object point cloud points mapped to the same pixel, thereby improving the accuracy of the depth value in the depth map.

Description

Technical Field

[0001] This specification relates to the field of unmanned driving, and in particular to a high-precision map construction method, device, storage medium and electronic device. Background Art

[0002] In the process of building a high-precision map, it is very important to build a depth map based on the distance between each target and the visual sensor. The higher the accuracy of the distance between the target and the visual sensor in the depth map, the higher the accuracy of the constructed high-precision map.

[0003] In the prior art, when constructing a depth map, each frame of point cloud can be acquired through a laser radar, and at the same time, a target image can be acquired through a monocular vision sensor. Then, the acquired multi-frame point cloud is fused, and the fused multi-frame point cloud is mapped to the target image through an artificially calibrated external parameter matrix to obtain a depth map corresponding to the target image. Among them, the depth map can represent the distance between the target object in the target image and the vision sensor.

[0004] When mapping a multi-frame point cloud to a target image, the point cloud of static objects and the point cloud of dynamic objects in the multi-frame point cloud are mapped to the target image. The point cloud of the static object and the point cloud of the dynamic object may overlap at the same pixel point on the target image, resulting in inaccurate depth values ​​corresponding to each pixel point in the depth map obtained by mapping the multi-frame point cloud. Summary of the invention

[0005] The embodiments of this specification provide a high-precision map construction method, device, storage medium and electronic device to partially solve the problems existing in the above-mentioned prior art.

[0006] The embodiments of this specification adopt the following technical solutions:

[0007] This specification provides a method for constructing a high-precision map, including:

[0008] Acquire a point cloud and a binocular image corresponding to the point cloud, and use any one of the binocular images as a target image;

[0009] Determine a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and a recognition result of object category recognition for each object involved in the target image, wherein the object category includes: static objects and dynamic objects;

[0010] According to the determined standard depth interval corresponding to each pixel in the target image, the depth value of each pixel point obtained by depth mapping each point cloud point in the point cloud to the target image is filtered to obtain a depth map corresponding to the static object involved in the target image;

[0011] A high-precision map is constructed based on the depth map.

[0012] Optionally, obtaining a binocular image corresponding to the point cloud specifically includes:

[0013] Acquire binocular images through a binocular vision sensor, and use one of the binocular images as an original target image, and use the other of the binocular images as an original non-target image;

[0014] The original target image and the original non-target image are subjected to stereo correction to obtain a target image and a non-target image.

[0015] Optionally, determining a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and a recognition result of object category recognition for each object involved in the target image specifically includes:

[0016] Matching the target image with a non-target image to obtain a reference depth map corresponding to the target image, wherein the non-target image and the target image are binocular images acquired at the same time based on a binocular vision sensor;

[0017] Identify the images of each object contained in the target image, and identify the pixel points corresponding to the images of different object categories contained in the target image;

[0018] According to the object category corresponding to each pixel in the target image and the depth value corresponding to each pixel in the reference depth map, a standard depth interval corresponding to each pixel in the target image is determined.

[0019] Optionally, matching the target image with a non-target image to obtain a reference depth map corresponding to the target image specifically includes:

[0020] The target image and the non-target image are input into a pre-trained first depth prediction model, so as to obtain a reference depth map corresponding to the target image based on a matching relationship between pixels of the target image and the non-target image through the first depth prediction model.

[0021] Optionally, training the first depth prediction model specifically includes:

[0022] Obtaining binocular image samples through a binocular vision sensor, taking one of the binocular image samples as a target image sample and the other as a non-target image sample, and obtaining a point cloud matching the target image sample as a target point cloud;

[0023] Inputting the target image sample and the non-target image sample into the first depth prediction model to be trained, so as to obtain a reference depth map to be optimized corresponding to the target image sample based on a matching relationship between pixels of the target image sample and the non-target image sample through the first depth prediction model;

[0024] Determine the depth interval to be optimized corresponding to each pixel in the target image sample according to the depth value corresponding to each pixel in the reference depth image to be optimized and the depth error range corresponding to each pixel in the target image sample;

[0025] According to the depth interval to be optimized corresponding to each pixel in the target image sample, the depth value of each pixel obtained by depth mapping each point cloud point in the target point cloud to the target image sample is filtered to obtain a true depth map to be optimized corresponding to the target image sample, and the true depth map to be optimized is used as the label of the first depth prediction model; the first depth prediction model is iteratively trained with the difference between the reference depth map to be optimized and the true depth map to be optimized minimized, and the depth error range corresponding to each pixel in the target image sample gradually reduced as training objectives.

[0026] Optionally, determining a standard depth interval corresponding to each pixel in the target image according to an object category corresponding to each pixel in the target image and a depth value corresponding to each pixel in the reference depth map specifically includes:

[0027] For each pixel in the target image, determine a depth value of a pixel corresponding to the pixel from the reference depth map as a reference depth value corresponding to the pixel;

[0028] The standard depth interval corresponding to the pixel is determined according to the object category corresponding to the pixel, the reference depth value corresponding to the pixel, and a preset depth error range.

[0029] Optionally, obtaining a target image specifically includes:

[0030] Acquire a binocular image through a binocular vision sensor, and use one of the binocular images as a target image;

[0031] According to the depth map, a high-precision map is constructed, which specifically includes:

[0032] Acquire multiple frames of image samples through a monocular vision sensor, and determine an image matching the target image from the multiple frames of image samples as a matching image, and determine a plurality of frames of continuous image samples containing the matching image from the multiple frames of image samples;

[0033] Inputting the plurality of frames of continuous image samples into a second depth prediction model to be trained, so as to output a predicted depth map to be optimized corresponding to the matching image through the second depth prediction model to be trained;

[0034] The second depth prediction model is trained with the minimization of the difference between the predicted depth map to be optimized and the depth map as the training goal, and a high-precision map is constructed through the trained second depth prediction model and the multiple frames of images collected by the monocular vision sensor.

[0035] This specification provides a high-precision map construction device, including:

[0036] An acquisition module, used to acquire a point cloud and a binocular image corresponding to the point cloud, and use any one of the binocular images as a target image;

[0037] A determination module, configured to determine a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and a recognition result of object category recognition for each object involved in the target image, wherein the object category includes: static objects and dynamic objects;

[0038] A filtering module, configured to filter the depth value of each pixel point obtained by depth mapping each point cloud point in the point cloud to the target image according to the determined standard depth interval corresponding to each pixel point in the target image, so as to obtain a depth map corresponding to the static object involved in the target image;

[0039] A mapping module is used to construct a high-precision map based on the depth map.

[0040] This specification provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned high-precision map construction method is implemented.

[0041] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned high-precision map construction method when executing the program.

[0042] At least one of the above technical solutions adopted in the embodiments of this specification can achieve the following beneficial effects:

[0043] In the embodiments of this specification, the standard depth interval corresponding to each pixel in the target image is determined based on the reference depth map corresponding to the target image in the binocular image and the recognition result of the object category in the target image. Among them, the object categories include static objects and dynamic objects. Then, according to the standard depth interval corresponding to each pixel, the depth value obtained by mapping each point cloud point to the target image is filtered to obtain the depth map corresponding to the static object in the target image, and a high-precision map is constructed based on the depth map. In this method, according to the standard depth interval of the pixel points corresponding to the static object and the dynamic object, the depth value of the dynamic object point cloud point mapping on each pixel point is filtered to obtain the depth value of the static object point cloud point mapping. In this way, the problem of overlapping of the dynamic object point cloud points and the static object point cloud points mapped to the same pixel point can be avoided, thereby improving the accuracy of the depth value in the depth map and improving the accuracy of the high-precision map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of this specification and constitute a part of this specification. The illustrative embodiments and descriptions of this specification are used to explain this specification and do not constitute an improper limitation on this specification. In the drawings:

[0045] Figure 1 A flowchart of a high-precision map construction method provided in an embodiment of this specification;

[0046] Figure 2 A schematic diagram of a process for obtaining a depth map of a static object provided in an embodiment of this specification;

[0047] Figure 3 A schematic diagram of a process of iteratively training a first depth prediction model provided in an embodiment of this specification;

[0048] Figure 4 A schematic diagram of the structure of a high-precision map construction device provided in an embodiment of this specification;

[0049] Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0050] The high-precision map construction method provided in this specification is intended to determine the standard depth interval corresponding to each pixel in the target image based on the reference depth map corresponding to the target image in the binocular image and the recognition result of the object category in the target image. Then, according to the standard depth interval corresponding to each pixel, the depth values ​​corresponding to different point cloud point mappings on the same pixel are filtered to obtain the depth value of the point cloud point mapping of the static object. Finally, according to the depth value of the point cloud point mapping of the static object corresponding to each pixel, a depth map is constructed. In the depth map constructed in this way, only the depth value of the point cloud point mapping of the static object is retained.

[0051] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0052] The technical solutions provided by the embodiments of this specification are described in detail below in conjunction with the accompanying drawings.

[0053] Figure 1 The flowchart of the high-precision map construction method provided in the embodiment of this specification includes:

[0054] S100: Acquire a point cloud and a binocular image corresponding to the point cloud, and use any one of the binocular images as a target image.

[0055] In the embodiments of this specification, the environmental information within the preset range of the unmanned equipment can be obtained by using the laser radar and image sensor installed on the unmanned equipment, and then the collected environmental information is processed to construct a depth map, and then a high-precision map is constructed based on the depth map, that is, Figure 1 The high-precision map construction method shown can be applied to unmanned equipment and can also be applied to servers that control unmanned equipment. Among them, the unmanned equipment in this specification may include unmanned vehicles and drones, and the unmanned equipment can be used in the field of logistics and distribution, including both instant delivery fields such as takeout and delivery, and other non-instant delivery fields.

[0056] In the embodiments of the present specification, multiple frames of point clouds are acquired by a laser radar, and multiple frames of images are acquired by an image sensor. The image sensor may be a monocular vision sensor, a binocular vision sensor, etc. When the image sensor is a binocular vision sensor, binocular images acquired at the same time may be acquired by the binocular vision sensor, or binocular images acquired at different times may be acquired by the binocular vision sensor, wherein the similarity between the binocular images is greater than a preset threshold.

[0057] In the embodiment of the present specification, an image can be selected from multiple frames of images as the target image. When the image sensor is a binocular vision sensor, that is, an image is selected from the binocular images as the target image. Then, at least one point cloud frame matching the acquisition time of the target image is selected from the multi-frame point cloud. In order to make the point cloud denser, multiple point cloud frames matching the acquisition moment of the target image can be selected from the multi-frame point cloud. The point cloud matching the acquisition moment of the target image can be a point cloud collected within a time period consisting of moments continuous with the acquisition moment of the target image, centered at the acquisition moment of the target image.

[0058] For example, the image acquired at the 5th second is selected from the multiple frames as the target image. Then, multiple point cloud frames acquired between the 4th second and the 6th second are selected from the multiple point cloud frames.

[0059] When constructing a depth map, the point cloud of static objects is usually mapped to the target image to obtain the depth values ​​corresponding to the pixels in the target image containing static objects, thereby constructing the depth map corresponding to the target image. The point cloud obtained by the LiDAR contains point clouds of dynamic objects and static objects, so the point cloud of dynamic objects needs to be filtered.

[0060] S102: Determine a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and an object category recognition result of each object involved in the target image, wherein the object category includes: static objects and dynamic objects.

[0061] In the embodiment of the present specification, after acquiring the target image and the multi-frame point cloud matching the target image acquisition time, in order to filter out the point cloud points of dynamic objects in the multi-frame point cloud, the standard depth interval corresponding to each pixel point in the target image can be determined. For each pixel point, when each point cloud point is mapped to the pixel point, each point cloud point can be filtered according to the standard depth interval corresponding to the pixel point.

[0062] Among them, the method for determining the standard depth interval corresponding to each pixel in the target image may include: determining the standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image, and the recognition result of object category recognition for each object involved in the target image, wherein the object category includes: static objects and dynamic objects. The binocular image may be acquired by a binocular vision sensor. The other image in the binocular image may be another image in the binocular image other than the target image, and the image may be an image collected at the same time as the target image, or an image collected at a different time than the target image.

[0063] Specifically, when the image sensor is a binocular vision sensor, the target image and the non-target image can be acquired by the binocular vision sensor, wherein the non-target image and the target image are binocular images acquired at the same time based on the binocular vision sensor.

[0064] Then, the acquired target image is matched with the non-target image to obtain a reference depth map corresponding to the target image, wherein the depth value of each pixel in the reference depth map is used as the reference depth value. At the same time, the images containing each object in the target image are identified, and the pixels corresponding to the images containing different object categories in the target image are identified.

[0065] Finally, according to the object category corresponding to each pixel in the target image and the reference depth value corresponding to each pixel in the reference depth map, the standard depth interval corresponding to each pixel in the target image is determined. That is, the standard depth interval corresponding to the dynamic object pixel and the standard depth interval corresponding to the static object pixel in the target image are determined. Among them, the method for identifying the image containing each object in the target image can be to perform semantic segmentation on the target image.

[0066] When determining the standard depth interval corresponding to each pixel in the target image and when the image sensor is a monocular vision sensor, the acquired multi-frame images other than the target image can be matched with the target image for feature points to determine the matching pixels in the multi-frame images and the target image. For each matching pixel in the target image, the depth value corresponding to the pixel when it is in the target image and the multi-frame images is determined based on the posture of the image sensor when acquiring the multi-frame images and the posture of the image sensor when acquiring the target image. The standard depth interval corresponding to the pixel in the target image is determined based on the depth value corresponding to the pixel in the multi-frame images matching the target image acquisition moment and the depth value corresponding to the pixel in the target image.

[0067] S104: According to the determined standard depth interval corresponding to each pixel in the target image, the depth value of each pixel point obtained by depth mapping each point cloud point in the point cloud to the target image is filtered to obtain a depth map corresponding to the static objects involved in the target image.

[0068] In an embodiment of the present specification, after determining the standard depth interval corresponding to each pixel in the target image, the depth value mapped to the pixel by each point cloud point in the multi-frame point cloud can be filtered, that is, the depth value obtained by mapping the dynamic object point cloud point is filtered out, and the depth value obtained by mapping the static object point cloud point is retained.

[0069] Specifically, in the process of mapping multi-frame point clouds, based on the standard depth range determined for each pixel in the target image, the depth value obtained by depth mapping each point cloud point in the multi-frame point cloud to the target image is filtered to obtain a depth map corresponding to static objects in the target image.

[0070] Furthermore, for each point cloud point in the multi-frame point cloud, the point cloud point is mapped to the target image, and the pixel point corresponding to the point cloud point is determined from the target image as the designated pixel point. Then, based on the depth value corresponding to the mapping of the point cloud point to the designated pixel point and the standard depth interval corresponding to the designated pixel point, it is determined whether the depth value obtained by mapping the point cloud point is in the standard depth interval corresponding to the designated pixel point. If so, retain the depth value obtained by mapping the point cloud point to the designated pixel point; if not, filter out the depth value obtained by mapping the point cloud point to the designated pixel point. Among them, each pixel point on the target image can correspond to multiple point cloud points.

[0071] Then, for each pixel in the target image, multiple depth values ​​mapped by multiple point cloud points after filtering corresponding to the pixel are determined, and the minimum depth value is selected from the multiple depth values ​​as the depth value corresponding to the pixel. According to the depth value corresponding to each pixel, a depth map corresponding to the target image is constructed.

[0072] When filtering the depth values ​​mapped to the target image from the point cloud, for the standard depth interval corresponding to each pixel in the target image, if the object category of the pixel is a static object, the standard depth interval corresponding to the pixel is a closed interval. When the point cloud is mapped to the pixel, the depth values ​​obtained by the point cloud mapping within the standard depth interval of the pixel are retained, and the depth values ​​obtained by the point cloud mapping outside the standard depth interval of the pixel are filtered out.

[0073] If the object category of the pixel is a dynamic object, the standard depth interval corresponding to the pixel is an open interval. When the point cloud is mapped to the pixel, the depth values ​​obtained by the point cloud mapping that are greater than the minimum depth value in the standard depth interval of the pixel are retained, and the depth values ​​obtained by the point cloud mapping that are less than the minimum depth value in the standard depth interval of the pixel are filtered out.

[0074] For example, if the standard depth interval of the static object pixel 1 in the target image is [4, 6], in the process of mapping the point cloud to the target image after initial filtering, if the dynamic object point cloud point a and the static object point cloud point b in the point cloud after initial filtering are both mapped to the pixel 1 in the target image, the depth value obtained by mapping the dynamic object point cloud point a is 3, and the depth value obtained by mapping the static object point cloud point b is 5, then according to the standard depth interval of pixel 1, the depth value in the standard depth interval is retained. That is, the depth value obtained by mapping the static object point cloud point b is retained, and the depth value obtained by filtering the dynamic object point cloud point b is filtered.

[0075] S106: Construct a high-precision map based on the depth map.

[0076] In the embodiments of this specification, after determining the depth map corresponding to the static object involved in the target image, the three-dimensional information of the static object involved in the target image in the world coordinate system can be determined according to the depth value of each pixel point in the depth map corresponding to the target image and the posture of the image sensor when collecting the target image. A high-precision map is constructed based on the three-dimensional information of the static object involved in the target image.

[0077] Through the above Figure 1 It can be seen from the method shown that this specification determines the standard depth interval corresponding to each pixel in the target image based on the reference depth map corresponding to the target image in the binocular image and the recognition result of the object category in the target image. Among them, the object categories include static objects and dynamic objects. Then, according to the standard depth interval corresponding to each pixel, the depth value obtained by mapping each point cloud point to the target image is filtered to obtain the depth map corresponding to the static object in the target image, and based on the depth map, a high-precision map is constructed. In this method, according to the standard depth interval of the pixel points corresponding to the static object and the dynamic object, the depth value of the dynamic object point cloud point mapping on each pixel point is filtered to obtain the depth value of the static object point cloud point mapping. In this way, the problem of overlapping of the dynamic object point cloud points and the static object point cloud points mapped to the same pixel point can be avoided, thereby improving the accuracy of the depth value in the depth map and improving the accuracy of the high-precision map construction.

[0078] Furthermore, in Figure 1 In step S100 shown, after obtaining the target image and the multi-frame point cloud matching the target image acquisition time, the multi-frame point cloud may be initially filtered for the point cloud points of dynamic objects.

[0079] Specifically, multiple frames of point clouds can be fused to obtain a point cloud set. The point clouds of the same object in the point cloud set are clustered, and the object categories corresponding to the point clouds of each clustered object are detected. According to the point cloud corresponding to each object category, the point cloud points of the dynamic object are initially filtered out to obtain the point cloud after initial filtering. Among them, the object categories may include: dynamic objects and static objects.

[0080] In addition to the above-mentioned method of filtering out point cloud points of dynamic objects, for each frame of point cloud, the frame point cloud can be input into a pre-trained dynamic object filtering model, and the point cloud points of the dynamic objects in the frame point cloud can be initially filtered out through the dynamic object filtering model to obtain the point cloud after initial filtering.

[0081] Among them, the dynamic object filtering model is supervisedly trained: multi-frame point cloud samples are input into the dynamic object filtering model to be trained, and the dynamic object point cloud points in the point cloud frame are filtered out by the dynamic object filtering model to obtain the point cloud after optimization and filtering, so as to minimize the difference between the point cloud after optimization and the labeled real filtered point cloud, and train the dynamic object filtering model.

[0082] After the initial filtering of dynamic object point cloud points in the multi-frame point cloud, the dynamic object point cloud points in the multi-frame point cloud may not be completely filtered out. Therefore, in the process of mapping the point cloud after the initial filtering to the target image, the depth value of each pixel point obtained by depth mapping each point cloud point in the point cloud after the initial filtering to the target image can be filtered according to the standard depth interval corresponding to each pixel point in the target image to obtain the depth map corresponding to the static objects involved in the target image, that is, the dynamic object point cloud points are filtered again to ensure that the dynamic object point cloud points in the point cloud after the initial filtering are completely filtered out. In this way, the dynamic object point cloud points and the static object point cloud points can be avoided from being mapped to the same pixel point on the target image.

[0083] In such Figure 1 In step S102 shown, in order to improve the matching accuracy between the target image and the non-target image, a binocular image can be obtained by a binocular vision sensor, one image is used as the original target image, and the other image is used as the original non-target image. Then, the original target image and the original non-target image are stereo corrected to obtain the target image and the non-target image. Among them, the stereo correction is to correct the target image and the non-target image to be aligned in the same plane.

[0084] The process of stereo correction is as follows: distortion elimination, binocular correction, and image cropping. That is, according to the external parameters corresponding to the original target image and the original non-target image, the original target image and the original non-target image are rotated so that the original target image and the original non-target image are aligned in the same plane. Then, the invalid image area caused by the point rotation is cropped to obtain the target image and the non-target image.

[0085] When matching the target image and the non-target image, template matching, feature matching, stereo matching algorithms, etc. can be used.

[0086] Taking the stereo matching algorithm as an example, the target image and the non-target image can be input into a pre-trained first depth prediction model, so as to obtain a reference depth map corresponding to the target image based on the matching relationship of pixels between the target image and the non-target image through the first depth prediction model. The first depth prediction model can be a stereo matching model.

[0087] When the acquisition stereo matching algorithm (model) matches the target image with the non-target image, for each pixel in the target image, the cost value between each pixel in the non-target image and the pixel is calculated by the matching cost algorithm according to all the estimated disparity values, and the disparity space map is generated according to the cost value corresponding to the pixel. The matching cost algorithm can be a mutual information algorithm, a rank transform, a BT algorithm, etc. Then, the cumulative cost corresponding to the pixel at the specified estimated disparity value is obtained by the cost aggregation algorithm. Through the "winner takes all" strategy, the point with the best cumulative cost is selected within the disparity search range as the matching pixel of the pixel, and the estimated disparity value between the matching pixel and the pixel is used as the disparity value corresponding to the pixel. Finally, according to the disparity value corresponding to the pixel, the depth value corresponding to the pixel is determined. According to the depth value corresponding to each pixel in the target image, the depth map corresponding to the target image is determined as the reference depth map.

[0088] After obtaining the reference depth map corresponding to the target image, the depth value of the pixel corresponding to each pixel in the target image can be determined from the reference depth map as the reference depth value corresponding to the pixel. According to the object category corresponding to the pixel, the reference depth value corresponding to the pixel, and the preset depth error range, the standard depth interval corresponding to the pixel is determined. Among them, the depth error range can set different depth error ranges for pixels belonging to different object categories. The depth error range can be set artificially based on experience.

[0089] Specifically, for each pixel in the target image, if the object category to which the pixel belongs is a static object, the reference depth value corresponding to the pixel can be adjusted according to the reference depth value corresponding to the pixel and the first depth error range set based on the static object to obtain the minimum depth value and the maximum depth value, and the standard depth range corresponding to the pixel can be determined based on the minimum depth value and the maximum depth value.

[0090] For example, if the reference depth value corresponding to the pixel is 5 and the first depth error range is 1, then the standard depth interval corresponding to the pixel is [4, 6].

[0091] If the object category to which the pixel belongs is a dynamic object, the reference depth value corresponding to the pixel can be adjusted according to the reference depth value corresponding to the pixel and the second depth error range set based on the dynamic object to obtain a boundary depth value between the depth value of the dynamic object point cloud mapping and the depth value of the static object point cloud mapping. The second depth error range can be the depth error range determined by the first depth prediction model after iterative training.

[0092] In the case of the depth value of the point cloud mapping of the dynamic object, the demarcation depth value corresponding to the pixel is the maximum depth value of the dynamic object point cloud mapped to the pixel; in the case of the depth value of the point cloud mapping of the static object, the demarcation depth value corresponding to the pixel is the minimum depth value of the static object point cloud mapped to the pixel. That is, if the depth value mapped to the pixel is less than the demarcation depth value, the point cloud mapped to the pixel is determined to be a dynamic object point cloud; if the depth value mapped to the pixel is not less than the demarcation depth value, the point cloud mapped to the pixel is determined to be a static object point cloud. Because the mapping depth value of the static object point cloud needs to be retained in this manual, the standard depth interval corresponding to the pixel is a depth range that is not less than the demarcation depth value.

[0093] For example, if the reference depth value corresponding to the pixel is 5 and the second depth error range is 2, then the standard depth interval corresponding to the pixel is [7, +∞).

[0094] According to the standard depth interval corresponding to each pixel in the target image, the depth value obtained in the process of mapping the multi-frame point cloud to each pixel in the target image is filtered to obtain the depth value of the static object point cloud mapped to the target image. According to the depth value of the static object point cloud mapped to the target image, a depth map that retains the depth value corresponding to the static object is constructed, that is, the depth map corresponding to the target image.

[0095] Among them, when the target image and the non-target image are obtained after the original target image and the original non-target image are stereo corrected, the multi-frame point cloud can be mapped to the target image according to the rotation matrix (i.e., external parameters and internal parameters) determined after the stereo correction of the original target image and the original non-target image.

[0096] Based on the above description of filtering multiple frame point clouds using a reference depth image predicted by a binocular vision algorithm to obtain a depth map retaining a static object, a schematic diagram of constructing a depth map of a static object is provided in an embodiment of this specification, such as Figure 2 shown.

[0097] In addition, when training the first depth prediction model, a binocular image sample is obtained through a binocular vision sensor, one of the image samples in the binocular image sample is used as the original target image sample, and the other is used as the original non-target image sample, and a point cloud matching the original target image sample is obtained as the target point cloud. Then, the original target image sample and the original non-target image sample are stereo corrected to obtain the target image sample and the non-target image sample as well as the corrected internal and external parameters. Then, the target image sample and the non-target image sample are input into the first depth prediction model to be trained, so as to obtain the reference depth map to be optimized corresponding to the target image sample based on the matching relationship between the pixel points between the target image sample and the non-target image sample through the first depth prediction model. According to the depth value corresponding to each pixel point in the reference depth image to be optimized and the depth error range corresponding to each pixel point in the target image sample, the depth interval to be optimized corresponding to each pixel point in the target image sample is determined. According to the depth interval to be optimized corresponding to each pixel in the target image sample, the depth value of each pixel obtained by depth mapping each point cloud point in the target point cloud to the target image sample according to the corrected internal and external parameters is filtered to obtain the true value depth map to be optimized corresponding to the target image sample, and the true value depth map to be optimized is used as the label of the first depth prediction model. Finally, the first depth prediction model is iteratively trained with the training objectives of minimizing the difference between the reference depth map to be optimized and the true value depth map to be optimized, and gradually reducing the depth error range corresponding to each pixel in the target image sample. Among them, the depth error range determined during the training process of the first depth prediction model does not distinguish between dynamic objects and static objects, that is, the depth error range applies to the depth values ​​of all target point clouds mapped to the target image samples.

[0098] Based on the iterative training of the first depth prediction model, the accuracy of the true depth map to be optimized obtained by mapping the reference depth map to be optimized output by the first depth prediction model and the target point cloud is made higher and higher.

[0099] In addition, based on the above description of the iterative training of the first depth prediction model, the embodiment of this specification provides a schematic diagram of the iterative training process of the first depth prediction model, such as Figure 3 shown.

[0100] After constructing the depth map corresponding to the target image, a high-precision map can be directly constructed based on the depth map corresponding to the target image, or the depth map corresponding to the target image can be used as a label in a monocular vision algorithm that predicts the depth map, and then a high-precision map can be constructed based on the depth map predicted by the monocular vision algorithm.

[0101] Training the monocular vision algorithm: Obtain multiple frames of image samples through a monocular vision sensor, and determine an image that matches the target image from the multiple frames of image samples as a matching image, and determine a number of consecutive image samples containing the matching image from the multiple frames of image samples. The matching image has the same image content as the target image.

[0102] Input several frames of continuous image samples into the second depth prediction model to be trained, so as to output the predicted depth map to be optimized corresponding to the matching image through the second depth prediction model to be trained. The second depth prediction model is trained with the minimum difference between the predicted depth map to be optimized and the depth map corresponding to the target image as the training goal, and a high-precision map is constructed through the trained second depth prediction model and multiple frames of images collected by the monocular vision sensor. The second depth prediction model can be a CNN convolutional neural network.

[0103] The above is a high-precision map construction method provided in the embodiments of this specification. Based on the same idea, this specification also provides corresponding devices, storage media and electronic devices.

[0104] Figure 4 A schematic diagram of a high-precision map construction device provided in an embodiment of this specification, the device comprising:

[0105] An acquisition module 401 is used to acquire a point cloud and a binocular image corresponding to the point cloud, and use any one of the binocular images as a target image;

[0106] A determination module 402 is used to determine a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and a recognition result of object category recognition for each object involved in the target image, wherein the object category includes: static objects and dynamic objects;

[0107] A filtering module 403 is used to filter the depth value of each pixel point obtained by performing depth mapping of each point cloud point in the point cloud to the target image according to the determined standard depth interval corresponding to each pixel point in the target image, so as to obtain a depth map corresponding to the static object involved in the target image;

[0108] The mapping module 404 is used to construct a high-precision map based on the depth map.

[0109] Optionally, the acquisition module 401 is specifically used to acquire a binocular image through a binocular vision sensor, and use one image in the binocular image as an original target image, and use the other image in the binocular image as an original non-target image; perform stereo correction on the original target image and the original non-target image to obtain a target image and a non-target image.

[0110] Optionally, the determination module 402 is specifically used to match the acquired target image with the non-target image to obtain a reference depth map corresponding to the target image, wherein the non-target image and the target image are binocular images acquired at the same time based on a binocular vision sensor; identify images containing various objects in the target image, and identify pixel points corresponding to images of different object categories in the target image; determine a standard depth interval corresponding to each pixel point in the target image according to the object category corresponding to each pixel point in the target image and the depth value corresponding to each pixel point in the reference depth map.

[0111] Optionally, the determination module 402 is specifically used to input the target image and the non-target image into a pre-trained first depth prediction model, so as to obtain a reference depth map corresponding to the target image based on a matching relationship between pixels between the target image and the non-target image through the first depth prediction model.

[0112] Optionally, the determination module 402 is specifically used to obtain a binocular image sample through a binocular vision sensor, use one image sample in the binocular image sample as a target image sample and the other as a non-target image sample, and obtain a point cloud matching the target image sample as a target point cloud; input the target image sample and the non-target image sample into the first depth prediction model to be trained, so as to obtain a reference depth map to be optimized corresponding to the target image sample based on the matching relationship between the pixel points of the target image sample and the non-target image sample through the first depth prediction model; and according to the depth value corresponding to each pixel point in the reference depth image to be optimized and the depth value of each pixel point in the target image sample The depth error range corresponding to the pixel point is used to determine the depth interval to be optimized corresponding to each pixel point in the target image sample; according to the depth interval to be optimized corresponding to each pixel point in the target image sample, the depth value of each pixel point obtained by depth mapping each point cloud point in the target point cloud to the target image sample is filtered to obtain the true depth map to be optimized corresponding to the target image sample, and the true depth map to be optimized is used as the label of the first depth prediction model; the first depth prediction model is iteratively trained with the difference between the reference depth map to be optimized and the true depth map to be optimized minimized, and the depth error range corresponding to each pixel point in the target image sample gradually reduced as the training objectives.

[0113] Optionally, the determination module 402 is specifically used to determine, for each pixel in the target image, a depth value of a pixel corresponding to the pixel from the reference depth map as a reference depth value corresponding to the pixel; and determine a standard depth interval corresponding to the pixel based on the object category corresponding to the pixel, the reference depth value corresponding to the pixel, and a preset depth error range.

[0114] Optionally, the mapping module 404 is specifically used to obtain multiple frames of image samples through a monocular vision sensor, and determine an image that matches the target image from the multiple frames of image samples as a matching image, and determine several frames of continuous image samples containing the matching image from the multiple frames of image samples; input the several frames of continuous image samples into a second depth prediction model to be trained, so as to output a predicted depth map to be optimized corresponding to the matching image through the second depth prediction model to be trained; train the second depth prediction model with minimizing the difference between the predicted depth map to be optimized and the depth map as the training goal, and construct a high-precision map through the trained second depth prediction model and the multiple frames of images collected by the monocular vision sensor.

[0115] This specification also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, it can be used to perform the above Figure 1 The high-precision map construction method provided.

[0116] based on Figure 1 The high-precision map construction method shown in the present specification also provides Figure 5 The structural diagram of the electronic device shown in FIG. Figure 5 At the hardware level, the electronic device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 The high-precision map construction method described.

[0117] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0118] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0119] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0120] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0121] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0122] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0123] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0124] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0126] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0127] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0128] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0129] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0130] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0131] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0132] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0133] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for constructing a high-precision map, characterized in that: include: Acquire a point cloud and a binocular image corresponding to the point cloud, and use any one of the binocular images as a target image; Determine a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and a recognition result of object category recognition for each object involved in the target image, wherein the object category includes: static objects and dynamic objects; According to the determined standard depth interval corresponding to each pixel in the target image, the depth value of each pixel point obtained by depth mapping each point cloud point in the point cloud to the target image is filtered to obtain a depth map corresponding to the static object involved in the target image; Constructing a high-precision map based on the depth map; The obtaining of the binocular image corresponding to the point cloud specifically includes: Acquire binocular images through a binocular vision sensor, and use one of the binocular images as an original target image, and use the other of the binocular images as an original non-target image; The original target image and the original non-target image are subjected to stereo correction to obtain a target image and a non-target image; the standard depth interval corresponding to each pixel in the target image is determined according to a reference depth map corresponding to the target image determined based on other images in the binocular image and an identification result of object category identification of each object involved in the target image, specifically including: Matching the target image with a non-target image to obtain a reference depth map corresponding to the target image, wherein the non-target image and the target image are binocular images acquired at the same time based on a binocular vision sensor; Identify the images containing each object in the target image, and identify the pixel points corresponding to the images containing different object categories in the target image; Determine a standard depth interval corresponding to each pixel in the target image according to the object category corresponding to each pixel in the target image and the depth value corresponding to each pixel in the reference depth map; The method further comprises filtering the depth value of each pixel point obtained by performing depth mapping of each point cloud point in the point cloud to the target image according to the determined standard depth interval corresponding to each pixel point in the target image, so as to obtain a depth map corresponding to the static object involved in the target image, including: For each point cloud point in the multi-frame point cloud, the point cloud point is mapped to the target image, and the pixel point corresponding to the point cloud point is determined from the target image as the designated pixel point; according to the depth value corresponding to the point cloud point mapped to the designated pixel point and the standard depth interval corresponding to the designated pixel point, it is determined whether the depth value obtained by mapping the point cloud point is in the standard depth interval corresponding to the designated pixel point. If so, the depth value obtained by mapping the point cloud point to the designated pixel point is retained; if not, the depth value obtained by mapping the point cloud point to the designated pixel point is filtered out; for each pixel point in the target image, multiple depth values ​​mapped by multiple point cloud points corresponding to the pixel point after filtering are determined, and the smallest depth value is selected from the multiple depth values ​​as the depth value corresponding to the pixel point. According to the depth value corresponding to each pixel point Value, construct the depth map corresponding to the target image, when filtering the depth value mapped to the target image by the point cloud, for the standard depth interval corresponding to each pixel in the target image, if the object category of the pixel is a static object, the standard depth interval corresponding to the pixel is a closed interval, when the point cloud is mapped to the pixel, the depth value obtained by the point cloud mapping within the standard depth interval of the pixel is retained, and the depth value obtained by the point cloud mapping outside the standard depth interval of the pixel is filtered out, if the object category of the pixel is a dynamic object, the standard depth interval corresponding to the pixel is an open interval, when the point cloud is mapped to the pixel, the depth value obtained by the point cloud mapping greater than the minimum depth value in the standard depth interval of the pixel is retained, and the depth value obtained by the point cloud mapping less than the minimum depth value of the standard depth interval of the pixel is filtered out.

2. The method according to claim 1, characterized in that Matching the target image with the non-target image to obtain a reference depth map corresponding to the target image specifically includes: The target image and the non-target image are input into a pre-trained first depth prediction model, so as to obtain a reference depth map corresponding to the target image based on a matching relationship between pixels of the target image and the non-target image through the first depth prediction model.

3. The method according to claim 2, characterized in that Training the first depth prediction model specifically includes: Obtaining binocular image samples through a binocular vision sensor, taking one of the binocular image samples as a target image sample and the other as a non-target image sample, and obtaining a point cloud matching the target image sample as a target point cloud; Inputting the target image sample and the non-target image sample into the first depth prediction model to be trained, so as to obtain a reference depth map to be optimized corresponding to the target image sample based on a matching relationship between pixels of the target image sample and the non-target image sample through the first depth prediction model; Determine the depth interval to be optimized corresponding to each pixel in the target image sample according to the depth value corresponding to each pixel in the reference depth image to be optimized and the depth error range corresponding to each pixel in the target image sample; According to the depth interval to be optimized corresponding to each pixel in the target image sample, the depth value of each pixel obtained by depth mapping each point cloud point in the target point cloud to the target image sample is filtered to obtain a true depth map to be optimized corresponding to the target image sample, and the true depth map to be optimized is used as the label of the first depth prediction model; the first depth prediction model is iteratively trained with the difference between the reference depth map to be optimized and the true depth map to be optimized minimized, and the depth error range corresponding to each pixel in the target image sample gradually reduced as training objectives.

4. The method according to claim 1, characterized in that Determining a standard depth interval corresponding to each pixel in the target image according to the object category corresponding to each pixel in the target image and the depth value corresponding to each pixel in the reference depth map specifically includes: For each pixel in the target image, determine a depth value of a pixel corresponding to the pixel from the reference depth map as a reference depth value corresponding to the pixel; The standard depth interval corresponding to the pixel is determined according to the object category corresponding to the pixel, the reference depth value corresponding to the pixel, and a preset depth error range.

5. The method according to claim 1, characterized in that Constructing a high-precision map according to the depth map specifically includes: acquiring multiple frames of image samples through a monocular vision sensor, and determining an image matching the target image from the multiple frames of image samples as a matching image, and determining a plurality of frames of continuous image samples containing the matching image from the multiple frames of image samples; Inputting the plurality of frames of continuous image samples into a second depth prediction model to be trained, so as to output a predicted depth map to be optimized corresponding to the matching image through the second depth prediction model to be trained; The second depth prediction model is trained with the minimization of the difference between the predicted depth map to be optimized and the depth map as the training goal, and a high-precision map is constructed through the trained second depth prediction model and the multiple frames of images collected by the monocular vision sensor.

6. A high-precision map construction device, characterized in that: include: An acquisition module, used to acquire a point cloud and a binocular image corresponding to the point cloud, and use any one of the binocular images as a target image; A determination module, configured to determine a standard depth interval corresponding to each pixel in the target image according to a reference depth map corresponding to the target image determined based on other images in the binocular image and a recognition result of object category recognition for each object involved in the target image, wherein the object category includes: static objects and dynamic objects; A filtering module, configured to filter the depth value of each pixel point obtained by depth mapping each point cloud point in the point cloud to the target image according to the determined standard depth interval corresponding to each pixel point in the target image, so as to obtain a depth map corresponding to the static object involved in the target image; A mapping module is used to construct a high-precision map according to the depth map; the obtaining of the binocular image corresponding to the point cloud specifically includes: Acquire binocular images through a binocular vision sensor, and use one of the binocular images as an original target image, and use the other of the binocular images as an original non-target image; The original target image and the original non-target image are subjected to stereo correction to obtain a target image and a non-target image; the standard depth interval corresponding to each pixel in the target image is determined according to a reference depth map corresponding to the target image determined based on other images in the binocular image and an identification result of object category identification of each object involved in the target image, specifically including: Matching the target image with a non-target image to obtain a reference depth map corresponding to the target image, wherein the non-target image and the target image are binocular images acquired at the same time based on a binocular vision sensor; Identify the images containing each object in the target image, and identify the pixel points corresponding to the images containing different object categories in the target image; Determine a standard depth interval corresponding to each pixel in the target image according to the object category corresponding to each pixel in the target image and the depth value corresponding to each pixel in the reference depth map; The method further comprises filtering the depth value of each pixel point obtained by performing depth mapping of each point cloud point in the point cloud to the target image according to the determined standard depth interval corresponding to each pixel point in the target image, so as to obtain a depth map corresponding to the static object involved in the target image, including: For each point cloud point in the multi-frame point cloud, the point cloud point is mapped to the target image, and the pixel point corresponding to the point cloud point is determined from the target image as the designated pixel point; according to the depth value corresponding to the point cloud point mapped to the designated pixel point and the standard depth interval corresponding to the designated pixel point, it is determined whether the depth value obtained by mapping the point cloud point is in the standard depth interval corresponding to the designated pixel point. If so, the depth value obtained by mapping the point cloud point to the designated pixel point is retained; if not, the depth value obtained by mapping the point cloud point to the designated pixel point is filtered out; for each pixel point in the target image, multiple depth values ​​mapped by multiple point cloud points corresponding to the pixel point after filtering are determined, and the smallest depth value is selected from the multiple depth values ​​as the depth value corresponding to the pixel point. According to the depth value corresponding to each pixel point Value, construct the depth map corresponding to the target image, when filtering the depth value mapped to the target image by the point cloud, for the standard depth interval corresponding to each pixel in the target image, if the object category of the pixel is a static object, the standard depth interval corresponding to the pixel is a closed interval, when the point cloud is mapped to the pixel, the depth value obtained by the point cloud mapping within the standard depth interval of the pixel is retained, and the depth value obtained by the point cloud mapping outside the standard depth interval of the pixel is filtered out, if the object category of the pixel is a dynamic object, the standard depth interval corresponding to the pixel is an open interval, when the point cloud is mapped to the pixel, the depth value obtained by the point cloud mapping greater than the minimum depth value in the standard depth interval of the pixel is retained, and the depth value obtained by the point cloud mapping less than the minimum depth value of the standard depth interval of the pixel is filtered out.

7. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method described in any one of claims 1 to 5 is implemented.

Images