Method and apparatus for generating depth map, electronic device and storage medium

Abstract

Embodiments of the present application relate to the technical field of computer stereo vision, and particularly relate to a depth map generation method and device, electronic equipment and storage medium. The depth map generation method comprises: acquiring a left-eye color image and a right-eye color image of a target object; performing image segmentation on the left-eye color image and the right-eye color image to generate a left-eye region map and a right-eye region map; generating a first depth region map according to the left-eye region map and the right-eye region map; extracting a target depth region in the first depth region map and a preset monocular depth map of the target object that satisfies a preset condition to form an initial depth map; and generating a depth map of the target object according to the initial depth map and the left-eye region map. The present application can improve the robustness and accuracy of depth image generation in a complex scene.

Description

Technical Field

[0001] This application relates to the field of computer stereo vision technology, and in particular to a method, apparatus, electronic device, and storage medium for generating depth maps. Background Technology

[0002] Current common methods for acquiring depth images include stereo cameras, RGB-D cameras, LiDAR scanners, or using deep learning to reconstruct depth from monocular images. However, stereo depth reconstruction has low efficiency and accuracy. RGB-D cameras generally use Time-of-Flight (TOF) depth sensors to acquire depth data, but the maximum acquisition depth of TOF depth sensors is often no more than 5 meters. Therefore, in complex scenes, the depth maps reconstructed by RGB-D cameras have low confidence. LiDAR scanners are too expensive and generally produce sparse depth maps. Monocular depth reconstruction lacks true scale information, so it is not robust enough in complex scenes. Summary of the Invention

[0003] The main objective of this application is to provide a method, apparatus, electronic device, and storage medium for generating depth maps. The aim is to improve the robustness and accuracy of depth image generation in complex scenes.

[0004] To achieve the above objectives, embodiments of this application provide a method for generating a depth map, comprising: acquiring a left-eye color image and a right-eye color image of a target object; performing image segmentation on the left-eye color image and the right-eye color image to generate a left-eye region image and a right-eye region image; generating a first depth region image based on the left-eye region image and the right-eye region image; extracting target depth regions that meet preset conditions from the first depth region image and a preset monocular depth image of the target object to form an initial depth map; and generating a depth map of the target object based on the initial depth map and the left-eye region image.

[0005] To achieve the above objectives, embodiments of this application also provide a depth map generation apparatus, comprising: an acquisition module for acquiring a left-eye color image and a right-eye color image of a target object; a segmentation module for performing image segmentation on the left-eye color image and the right-eye color image to generate a left-eye region image and a right-eye region image; a first generation module for generating a first depth region image based on the left-eye region image and the right-eye region image; a second generation module for extracting target depth regions that meet preset conditions from the first depth region image and a preset monocular depth image of the target object to form an initial depth map; and a third generation module for generating a depth map of the target object based on the initial depth map and the left-eye region image.

[0006] To achieve the above objectives, embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described depth map generation method.

[0007] To achieve the above objectives, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the above-described method for generating depth maps.

[0008] The depth map generation method proposed in this application involves acquiring left-eye and right-eye color images of a target object during the depth map generation process; performing image segmentation on the left-eye and right-eye color images to generate left-eye region images and right-eye region images; generating a first depth region image based on the left-eye and right-eye region images; extracting target depth regions that meet preset conditions from the first depth region image and the preset monocular depth image of the target object to form an initial depth map; generating a depth map of the target object based on the initial depth map and the left-eye region image; by selecting target regions that meet the conditions from the first depth region image generated based on binocular images and the preset monocular depth image to form the initial depth map, and then combining the initial depth map with the left-eye image in the binocular image to generate a depth map, it is possible to select regions with high confidence from the monocular depth map and the binocular depth map to generate a new depth map. This enables the application to obtain high-precision depth information in complex scenes, improving the robustness and accuracy of depth image generation in complex scenes; and solving the technical problem of poor robustness of various depth recovery methods in complex scenes in the prior art. Attached Figure Description

[0009] Figure 1 This is a flowchart of the depth map generation method provided in the embodiments of this application;

[0010] Figure 2 This is a flowchart of step 103 in the depth map generation method provided in the embodiments of this application;

[0011] Figure 3 This is a flowchart of the depth map generation method provided in the embodiments of this application;

[0012] Figure 4 This is a schematic diagram of the structure of the image acquisition device in the depth map generation method provided in the embodiments of this application;

[0013] Figure 5 This is a terminal schematic diagram in the depth map generation method provided in the embodiments of this application;

[0014] Figure 6 This is a flowchart of the depth map generation method provided in the embodiments of this application;

[0015] Figure 7 This is a schematic diagram of the structure of the depth map generation apparatus provided in the embodiments of this application;

[0016] Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this application to help readers better understand this application. However, the technical solutions claimed in this application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0018] One embodiment of this application relates to a method for generating a depth map, such as... Figure 1 As shown, it includes:

[0019] Step 101: Obtain the left and right color images of the target object.

[0020] In one example implementation, a target object is photographed using a binocular camera or binocular sensor to generate a left-eye color image and a right-eye color image of the target object; the left-eye color image and the right-eye color image generated by the binocular camera or binocular sensor can be a sequence of left-eye color images and a sequence of right-eye color images.

[0021] Step 102: Perform image segmentation on the left and right eye color images to generate left and right eye region images.

[0022] In one example implementation, the left-eye color image and the right-eye color image are input into a preset semantic analysis neural network for image segmentation. Based on the information of each pixel in the left-eye color image and the right-eye color image, the left-eye color image and the right-eye color image are segmented to generate a left-eye region image and a right-eye region image.

[0023] Step 103: Generate a first depth region map based on the left and right eye region maps.

[0024] In one example implementation, the first depth region map is generated based on the principle of triangulation ranging. Essentially, it obtains the depth value of each pixel in the left-eye region map based on the disparity information of the left and right-eye region maps and the camera parameters of the binocular camera or binocular sensor that forms the left and right color images, thus constructing the first depth region map. When the left and right cameras in the binocular camera or sensor are not on the same horizontal line, the rotation factor of the left and right region maps also needs to be considered. Step 103 can be generated as follows: Figure 2 The implementation of each sub-step shown includes:

[0025] Sub-step 1031: Perform feature extraction processing on the left and right eye region maps respectively to obtain the left eye feature points of the left eye region map and the right eye feature points of the right eye region map.

[0026] In one example implementation, a scale-invariant feature transform (SIFT) is performed on the left and right eye region maps to extract left eye feature points and right eye feature points.

[0027] Sub-step 1032: Perform feature matching on each left-eye feature point and each right-eye feature point to obtain the correspondence between each left-eye feature point and each right-eye feature point.

[0028] In one example implementation, feature matching is performed on each left-eye feature point on the left-eye region map and each right-eye feature point on the right-eye region map to obtain the correspondence between each left-eye feature point and the right-eye feature point. The correspondence is actually the right-eye feature point whose position on the right-eye region map is the left-eye feature point after the viewpoint transformation.

[0029] Sub-step 1033: Obtain the disparity information between each left-eye feature point and the corresponding right-eye feature point according to the correspondence.

[0030] In one example implementation, the correspondence is actually a pair of feature points. A pair of feature points contains a left-eye feature point and a right-eye feature point. Based on each pair of feature points in the correspondence, the disparity information of each left-eye feature point and its corresponding right-eye feature point is calculated.

[0031] Sub-step 1034: Generate depth information for each left-eye feature point based on camera parameters and disparity information of each left-eye feature point.

[0032] In one example implementation, after obtaining the parallax information, the depth information of each left-eye feature point can be calculated by combining the camera parameters of the binocular camera or binocular sensor when capturing the left-eye color image and the right-eye color image using the triangulation formula.

[0033] Sub-step 1035: Generate a first depth region map based on the depth information and position information of each left eye feature point.

[0034] In one example implementation, the position of each left eye feature point in each left eye region of the left eye region map is determined, and the depth value of the pixel block of a preset size (such as 9*9, 3*3, etc.) around the feature point is set as the depth information of the left eye feature point. By traversing all left eye feature points, the first depth region map can be generated.

[0035] Step 104: Extract the target depth regions that meet the preset conditions from the first depth region map and the preset monocular depth map of the target object to form an initial depth map.

[0036] In one example implementation, the monocular depth map of the target object is generated by photographing the target object using a TOF signal transmitter / receiver; each left-eye color image in the left-eye color image sequence and each right-eye color image in the right-eye color image sequence can be associated with the monocular depth map; the shooting environment, shooting distance, and shooting angle of the binocular camera or binocular sensor and the TOF signal transmitter / receiver should be consistent to ensure that the generated left-eye and right-eye color images can be associated with the monocular depth map.

[0037] In one example implementation, the monocular depth map contains regions corresponding to the first depth region map. Based on preset conditions, a depth region can be selected as the target depth region from the first depth region and its corresponding depth regions in the monocular depth map. All first depth regions and their corresponding depth regions in the monocular depth map are traversed to filter out the target depth regions. These target depth regions are then used to construct the initial depth map. The conditions can include the average depth value of the second depth region, the distribution of depth values, etc.

[0038] In one example implementation, the algorithm flow of performing semantic segmentation on the left and right color images before performing depth estimation can improve the robustness of the depth estimation algorithm in complex scenes.

[0039] Step 105: Generate a depth map of the target object based on the initial depth map and the left eye region map.

[0040] In one example implementation, the initial depth map and the left eye region map are input into a preset optimization network for optimization processing. The optimization network can be a DeepCompletion network, which includes semantic edge depth consistency optimization and depth map hole filling, etc. The boundaries of each left eye region in the left eye region map are used as boundary constraints when optimizing the initial depth map, thereby optimizing the initial depth map and finally generating the depth map of the target object.

[0041] In this embodiment, during the generation of the depth map, the left-eye color image and the right-eye color image of the target object are acquired; image segmentation is performed on the left-eye color image and the right-eye color image to generate a left-eye region image and a right-eye region image; a first depth region image is generated based on the left-eye region image and the right-eye region image; target depth regions that meet preset conditions are extracted from the first depth region image and the preset monocular depth image of the target object to form an initial depth map; a depth map of the target object is generated based on the initial depth map and the left-eye region image; by selecting target regions that meet the conditions from the first depth region image generated based on the binocular image and the preset monocular depth image to form the initial depth map, and then combining the initial depth map and the left-eye image in the binocular image to generate a depth map, it is possible to select regions with high confidence from the monocular depth map and the binocular depth map to generate a new depth map, enabling this application to obtain high-precision depth information in complex scenes, improving the robustness and accuracy of depth image generation in complex scenes; and solving the technical problem of poor robustness of various depth recovery methods in complex scenes in the prior art.

[0042] One embodiment of this application relates to a method for generating a depth map, such as... Figure 3 As shown, it includes:

[0043] Step 201: Acquire a left-eye image sequence and a right-eye image sequence using a binocular camera. The left-eye image sequence contains each left-eye image and its timestamp, and the left-eye image sequence contains each right-eye image and its timestamp.

[0044] In one example implementation, a smart device containing binocular cameras (such as smart glasses) can acquire left and right eye color images. A schematic diagram of the smart glasses structure is shown below. Figure 4 As shown, the smart glasses also include a time-of-flight transmitter and receiver, which is used to obtain the initial depth map of the target object. Under normal circumstances, the smart device is connected to the terminal, and after obtaining the left and right color images, it sends the left and right color images to the terminal, which then transmits them to the cloud for processing.

[0045] In one example implementation, the smart device acquires a monocular depth map of the target object via a time-of-flight transmitter-receiver and acquires a left-eye image sequence and a right-eye image sequence of the target object via a binocular camera. The left-eye image sequence contains each left-eye image and its timestamp, and the left-eye image sequence contains each right-eye image and its timestamp. Each left-eye image in the left-eye image sequence and each right-eye image in the right-eye image sequence can correspond to the monocular depth map.

[0046] Step 202: Extract the key points of each left eye image and each right eye image.

[0047] In one example implementation, keypoint extraction is performed on each left image in the left image sequence and each right image in the right image sequence to extract keypoints that are rotation-invariant and photometric-invariant.

[0048] Step 203: Based on the number of key points in each left-eye image and the timestamp of each left-eye image, select the left-eye keyframe image from each left-eye image as the left-eye color image.

[0049] In one example implementation, for each left-eye image in each left-eye image, the number of key points and the timestamp of the left-eye image are obtained. When the number of key points is greater than 75 and the timestamp interval between the left-eye image and the previous left-eye image is greater than 0.5s, or the number of key points is greater than 35 and the timestamp interval between the left-eye image and the previous left-eye image is greater than 1s, or the number of key points is greater than 35 and the timestamp interval between the left-eye image and the previous left-eye image is zero, it is indicated that the current left-eye image is a keyframe image, and the left-eye image is used as the left-eye color image.

[0050] Step 204: Select right-eye keyframe images from each right-eye image based on the number of key points and the timestamp of each right-eye image, and use them as right-eye color images.

[0051] In one example implementation, for each right-eye image in each left-eye image, the number of key points and the timestamp of the right-eye image are obtained. When the number of key points is greater than 75 and the timestamp interval between the right-eye image and the previous right-eye image is greater than 0.5s, or the number of key points is greater than 35 and the timestamp interval between the right-eye image and the previous right-eye image is greater than 1s, or the number of key points is greater than 35 and the timestamp interval between the right-eye image and the previous right-eye image is zero, it is indicated that the current right-eye image is a keyframe image, and the left-eye image is used as the right-eye color image.

[0052] In one example implementation, the selection of both the left and right eye color images is handled by the processor of the smart device. After acquiring all the left and right eye color images, the left eye image sequence, right eye image sequence, left eye color image, right eye color image, and monocular depth are then processed. Figure 1 The images are sent to the terminal, which then inputs the left-eye color image, right-eye color image, and monocular depth image into the cloud or processes them locally. The left-eye image sequence and right-eye image sequence are displayed on the terminal's image preview interface. The terminal can control the frame rate of the binocular camera video recording, up to 30Hz, and the resolution, up to 1920×1080.

[0053] In one example implementation, such as Figure 5As shown, the terminal's interface also includes a start button. When the user triggers this button, the smart device begins to collect relevant data and displays the left and right eye image sequences in the image preview interface.

[0054] Step 205: Perform image segmentation on the left and right eye color images to generate left and right eye region images.

[0055] In one example implementation, this step is largely the same as step 102 in the embodiments of this application, and will not be described in detail here.

[0056] Step 206: Generate a first depth region map based on the left and right eye region maps.

[0057] In one example implementation, this step is largely the same as step 103 in the embodiments of this application, and will not be described in detail here.

[0058] Step 207: Extract the target depth regions that meet the preset conditions from the first depth region map and the preset monocular depth map of the target object to form an initial depth map.

[0059] In one example implementation, this step is largely the same as step 104 in the embodiments of this application, and will not be described in detail here.

[0060] Step 208: Generate a depth map of the target object based on the initial depth map and the left eye region map.

[0061] In one example implementation, this step is largely the same as step 105 in the embodiments of this application, and will not be described in detail here.

[0062] The implementation method of this application, based on other embodiments, can also acquire relevant images through smart devices. The method of acquiring image data is very simple. The person acquiring the image only needs to wear a smart device, set the relevant parameters, click the "Start Recording" button on the terminal, and then walk around in the environment to be scanned. Furthermore, since the scene seen by the person acquiring the image is almost identical to the actual captured image, the person acquiring the image does not need to hold up a mobile phone or camera and constantly monitor their captured image as in traditional methods.

[0063] One embodiment of this application relates to a method for generating a depth map, such as... Figure 6 As shown, it includes:

[0064] Step 301: Obtain the left and right color images of the target object.

[0065] In one example implementation, this step is largely the same as step 101 in the embodiments of this application, and will not be described in detail here.

[0066] Step 302: Perform image segmentation on the left and right color images to generate left and right region images, and segment the monocular depth image based on the left region image to generate the first depth region image.

[0067] In one example implementation, this step is largely the same as step 102 in the embodiments of this application, and will not be described in detail here.

[0068] Step 303: Generate a second depth region map based on the left and right eye region maps.

[0069] In one example implementation, this step is largely the same as step 103 in the embodiments of this application, and will not be described in detail here.

[0070] Step 304: For each first depth region in the first depth region map, obtain the average depth value of the first depth region.

[0071] In one example implementation, for each first depth region, the average depth value of the first depth region is calculated based on the depth information of each left eye feature point or each pixel in the first depth region.

[0072] Step 305: When the average depth value meets the condition, the depth region corresponding to the first depth region in the monocular depth map is taken as the target depth region.

[0073] In one example implementation, when the average depth value of the first depth region is less than a preset threshold, the first depth region is considered to be a shallow depth-of-field region. In this case, the data collected by the TOF signal transmitter and receiver is more relied upon, and the depth region corresponding to the first depth region in the monocular depth map is taken as the target depth region; wherein, the condition is considered to be met if the average depth value is less than the preset threshold.

[0074] In one example implementation, before extracting the target depth region, the monocular depth map can be segmented based on the position of each left-eye region in the left-eye region map to generate a second depth region map; that is, the position of each left-eye region in the left-eye region map is mapped to the monocular depth map; after segmentation, each left-eye region contains a corresponding first depth region; such that each first depth region in the first depth region map has a corresponding second depth region map in the monocular depth map; when the average depth value of the first depth region is less than a preset threshold, the second depth region in the second depth region map corresponding to the first depth region is taken as the target depth region.

[0075] Step 306: When the average depth value does not meet the conditions, input the first depth region and the left eye region corresponding to the first depth region in the left eye region map into the preset depth recovery network to generate the first depth recovery region, and use it as the target depth region.

[0076] In one example implementation, when the average depth value of the first depth region is greater than or equal to a preset threshold, the first depth region is considered a large depth-of-field region. In this case, the data acquired by the binocular camera is relied upon more heavily. The first depth region and the corresponding left-eye region in the left-eye region image are then input into a preset depth restoration network to generate the first depth restoration region, which is then used as the target depth region. The preset depth restoration network can be a fully convolutional depth restoration neural network based on residual learning, consisting of convolutional layers, pooling layers, and fully connected layers, such as MonoDepth, FCRN-DepthPrediction, and DenseDepth.

[0077] Step 307: Traverse each first depth region in the first depth region map to obtain each target depth region and form an initial depth map.

[0078] In one example implementation, by traversing each first depth region in the first depth region map, each target depth region used to construct the initial depth map can be obtained.

[0079] Step 308: Generate a depth map of the target object based on the initial depth map and the left eye region map.

[0080] In one example implementation, this step is largely the same as step 105 in the embodiments of this application, and will not be described in detail here.

[0081] In addition to other embodiments, the implementation of this application can also use the average depth value of feature points in each first depth region in the first depth region map as the interpretation criterion to select a certain target depth region from the first depth region map or the monocular depth map to form an initial depth map; so that the initial depth map generated by this application can fuse the regions with higher confidence in the first depth region map and the monocular depth map, thereby improving the accuracy of the generated initial depth map.

[0082] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0083] Another embodiment of this application relates to a depth map generation apparatus. The details of the depth map generation apparatus of this embodiment are described below. The following implementation details are provided for ease of understanding and are not essential for implementing this example. Figure 7This is a schematic diagram of the depth map generation device in this embodiment, including: an acquisition module 401, a segmentation module 402, a first generation module 403, a second generation module 404, and a third generation module 405.

[0084] The acquisition module 401 is used to acquire the left and right color images of the target object.

[0085] The segmentation module 402 is used to segment the left-eye color image and the right-eye color image to generate a left-eye region image and a right-eye region image.

[0086] The first generation module 403 is used to generate a second depth region map based on the left-eye region map and the right-eye region map.

[0087] The second generation module 404 is used to extract the target depth region that meets the preset conditions from the first depth region map and the monocular depth map of the target object to form an initial depth map.

[0088] The third generation module 405 is used to generate a depth map of the target object based on the initial depth map and the left eye region map.

[0089] It is not difficult to see that this embodiment is a system embodiment corresponding to the above method embodiments, and this embodiment can be implemented in conjunction with the above method embodiments. The relevant technical details and technical effects mentioned in the above embodiments are still valid in this embodiment, and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiments.

[0090] It should be noted that this device embodiment mainly describes the depth map generation method provided in the method embodiment at the software implementation level. Its implementation also requires hardware support. For example, the functions of relevant modules can be deployed on the processor so that the processor can run and implement the corresponding functions. In particular, the relevant data generated during the operation can be stored in the memory for subsequent inspection and use.

[0091] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problems proposed in this application; however, this does not mean that other units are absent in this embodiment.

[0092] Another embodiment of this application relates to an electronic device, such as... Figure 8As shown, it includes: at least one processor 501; and a memory 502 communicatively connected to at least one processor 501; wherein the memory 502 stores instructions executable by at least one processor 501, the instructions being executed by at least one processor 501 to enable at least one processor 501 to perform the depth map generation method in the above embodiments.

[0093] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0094] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0095] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the method embodiments described above.

[0096] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0097] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for generating a depth map, characterized in that, include: Acquire the left and right color images of the target object; Image segmentation is performed on the left-eye color image and the right-eye color image to generate a left-eye region image and a right-eye region image; A first depth region map is generated based on the left eye region map and the right eye region map; Extract the target depth regions that meet the preset conditions from the first depth region map and the preset monocular depth map of the target object to form an initial depth map; A depth map of the target object is generated based on the initial depth map and the left eye region map; The step of extracting target depth regions that meet preset conditions from the first depth region map and the preset monocular depth map of the target object to form an initial depth map includes: For each first depth region in the first depth region map, obtain the average depth value of the first depth region; When the average depth value meets the condition, the depth region corresponding to the first depth region in the monocular depth map is taken as the target depth region. When the average depth value does not meet the condition, the first depth region and the left eye region corresponding to the first depth region in the left eye region map are input into a preset depth recovery network to generate a first depth recovery region, which is then used as the target depth region. Traverse each of the first depth regions in the first depth region map to obtain each of the target depth regions, thereby forming the initial depth map.

2. The method for generating a depth map according to claim 1, characterized in that, The method further includes: obtaining camera parameters of the binocular camera that generates the left-eye color image and the right-eye color image; The step of generating a first depth region map based on the left-eye region map and the right-eye region map includes: Feature extraction processing is performed on the left eye region map and the right eye region map respectively to obtain each left eye feature point of the left eye region map and each right eye feature point of the right eye region map; Feature matching is performed on each of the left-eye feature points and each of the right-eye feature points to obtain the correspondence between each of the left-eye feature points and each of the right-eye feature points; Based on the correspondence, obtain the disparity information between each left-eye feature point and the corresponding right-eye feature point; Based on the camera parameters and the disparity information of each left-eye feature point, depth information of each left-eye feature point is generated; The first depth region map is generated based on the depth information and position information of each left eye feature point.

3. The method for generating a depth map according to claim 1, characterized in that, The step of extracting the target depth region that meets the preset conditions from the first depth region map and the preset monocular depth map of the target object includes: Based on the position of each left-eye region in the left-eye region map, the monocular depth map is segmented to generate a second depth region map; The step of extracting the target depth region that meets the preset conditions in the first depth region map and the preset monocular depth map of the target object includes: extracting the target depth region in the first depth region map and the second depth region map.

4. The method for generating a depth map according to any one of claims 1 to 3, characterized in that, The acquisition of the left and right color images of the target object includes: A left-eye image sequence and a right-eye image sequence are acquired using a binocular camera. The left-eye image sequence includes each left-eye image and a timestamp of each left-eye image, and the left-eye image sequence includes each right-eye image and a timestamp of each right-eye image. Extract the key points of each left eye image and each right eye image; Based on the number of key points in each left-eye image and the timestamp of each left-eye image, a left-eye keyframe image is selected from each left-eye image as the left-eye color image; Based on the number of key points in each right eye image and the timestamp of each right eye image, a right eye keyframe image is selected from each right eye image and used as the right eye color image.

5. The method for generating a depth map according to any one of claims 1 to 3, characterized in that, The step of generating a depth map of the target object based on the initial depth map and the left eye region map includes: The initial depth map and the left eye region map are input into a preset optimization network for optimization processing to generate the depth map.

6. The method for generating a depth map according to claim 5, characterized in that, The step of inputting the initial depth map and the left eye region map into a preset optimization network for optimization processing to generate the depth map includes: In the optimization network, the boundaries of each left-eye region in the left-eye region map are used as boundary constraints to optimize the initial depth map and generate the depth map of the target object.

7. A depth map generation apparatus, characterized in that, include: The acquisition module is used to acquire the left and right color images of the target object; The segmentation module is used to segment the left-eye color image and the right-eye color image to generate a left-eye region image and a right-eye region image. The first generation module is used to generate a first depth region map based on the left eye region map and the right eye region map; The second generation module is used to extract target depth regions that meet preset conditions from the first depth region map and the preset monocular depth map of the target object to form an initial depth map; The third generation module is used to generate a depth map of the target object based on the initial depth map and the left eye region map; The step of extracting target depth regions that meet preset conditions from the first depth region map and the preset monocular depth map of the target object to form an initial depth map includes: For each first depth region in the first depth region map, obtain the average depth value of the first depth region; When the average depth value meets the condition, the depth region corresponding to the first depth region in the monocular depth map is taken as the target depth region. When the average depth value does not meet the condition, the first depth region and the left eye region corresponding to the first depth region in the left eye region map are input into a preset depth recovery network to generate a first depth recovery region, which is then used as the target depth region. Traverse each of the first depth regions in the first depth region map to obtain each of the target depth regions, thereby forming the initial depth map.

8. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method for generating a depth map as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for generating a depth map according to any one of claims 1 to 6.

Images