Depth image acquisition method, electronic device, and storage medium

Abstract

The application provides a depth image acquisition method, an electronic device and a storage medium. The method comprises the following steps: acquiring a first image and a second image; processing the first image and the second image based on a deep learning network model to obtain a first predicted depth image and a second predicted depth image; converting the first predicted depth image and the second predicted depth image into a first point cloud image and a second point cloud image; converting the first point cloud image and the second point cloud image into a third point cloud image and a fourth point cloud image according to a camera conversion matrix; performing point cloud matching on the first point cloud image and the fourth point cloud image, and calculating a first error value; performing point cloud matching on the second point cloud image and the third point cloud image, and calculating a second error value; adjusting the deep learning network model according to the first error value and the second error value to obtain a target deep learning network model; inputting a to-be-detected image into the target deep learning network model to obtain a depth image corresponding to the to-be-detected image. The application can obtain a more accurate depth image.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of machine vision, and in particular to a depth image acquisition method, an electronic device and a storage medium. BACKGROUND

[0002] During automatic driving, a vehicle needs to monitor pedestrians, vehicles or obstacles around the vehicle by using a depth image. At present, the depth image is acquired by using a binocular vision method, a structured light method and a time-of-flight method, etc. However, the depth image acquired by using the above methods is not accurate enough, thereby affecting the safety of automatic driving. SUMMARY

[0003] In view of the above, it is necessary to provide a depth image acquisition method, an electronic device and a storage medium, which can solve the technical problem that an inaccurate depth image affects the safety of automatic driving of a vehicle.

[0004] The embodiment of the present application provides a depth image acquisition method, which comprises the following steps: acquiring a first image and a second image photographed by a camera at adjacent times; processing the first image based on a depth learning network model to obtain a first predicted depth map; processing the second image to obtain a second predicted depth map; processing the first image and the second image to obtain a camera conversion matrix between the first image and the second image; converting the first predicted depth map into a first point cloud map and converting the second predicted depth map into a second point cloud map; converting the first point cloud map into a third point cloud map and converting the second point cloud map into a fourth point cloud map according to the camera conversion matrix; performing point cloud matching on the first point cloud map and the fourth point cloud map, and calculating a first error value of the point cloud matching of the first point cloud map and the fourth point cloud map; performing point cloud matching on the second point cloud map and the third point cloud map, and calculating a second error value of the point cloud matching of the second point cloud map and the third point cloud map; adjusting the depth learning network model according to the first error value and the second error value to obtain a target depth learning network model; inputting a to-be-detected image into the target depth learning network model to obtain a depth image corresponding to the to-be-detected image. By using the method, an accurate depth image can be acquired to avoid affecting the automatic driving of a vehicle.

[0005] In an optional implementation, the converting the first predicted depth map into a first point cloud map and the converting the second predicted depth map into a second point cloud map comprises: obtaining first pixel point coordinates from the first predicted depth map, and obtaining the first point cloud map according to the first pixel point coordinates and an internal and external parameter matrix transformation formula of the camera; and obtaining second pixel point coordinates from the second predicted depth map, and obtaining the second point cloud map according to the second pixel point coordinates and the internal and external parameter matrix transformation formula of the camera.

[0006] In an optional implementation, the camera conversion matrix comprises a rotation matrix and a translation matrix, and the converting the first point cloud map into a third point cloud map and the converting the second point cloud map into a fourth point cloud map according to the camera conversion matrix comprises: obtaining coordinates of points in the first point cloud map and combining the coordinates into a first coordinate set; obtaining coordinates of points in the second point cloud map and combining the coordinates into a second coordinate set; multiplying the rotation matrix, the translation matrix and the first coordinate set to obtain the third point cloud map; and multiplying the rotation matrix, the translation matrix and the second coordinate set to obtain the fourth point cloud map.

[0007] In an optional implementation, the deep learning network model comprises a depth estimation convolutional neural network and a pose estimation convolutional neural network, and the processing the first image based on the deep learning network model to obtain a first predicted depth map and processing the second image based on the deep learning network model to obtain a second predicted depth map comprises: performing convolution operation and deconvolution operation on the first image based on the depth estimation convolutional neural network to obtain the first predicted depth map; and performing convolution operation and deconvolution operation on the second image based on the depth estimation convolutional neural network to obtain the second predicted depth map.

[0008] In an optional implementation, the processing the first image and the second image to obtain a camera conversion matrix between the first image and the second image comprises: performing convolution operation on the first image and the second image based on the pose estimation convolutional neural network to obtain a camera conversion matrix.

[0009] In an optional implementation, the point cloud matching the first point cloud map with the fourth point cloud map comprises: performing point cloud matching on the first point cloud map and the fourth point cloud map according to the camera conversion matrix and an ICP algorithm formula; wherein the ICP algorithm formula is:

[0010]

[0011] wherein E(X, Y) is a point cloud matching result, R is the rotation matrix, x i is the i-th point in the first point cloud map, t is the translation matrix, and yi is the i-th point in the fourth point cloud map.

[0012] In an optional implementation, the calculating the first error value of the point cloud matching between the first point cloud map and the fourth point cloud map comprises: determining a plurality of corresponding point pairs of the first point cloud map and the fourth point cloud map; and calculating the first error value by using a loss function of point cloud matching according to the transformation matrix of the camera and the plurality of corresponding point pairs, the loss function being:

[0013]

[0014] wherein, LOSS is the first error value, n is the number of corresponding point pairs of the first point cloud map and the fourth point cloud map, A j is the j-th point in the first point cloud map, B j is the j-th point in the fourth point cloud map, A j and B j are a pair of corresponding point pairs.

[0015] In an optional implementation, the adjusting the deep learning network model according to the first error value and the second error value to obtain a target deep learning network model comprises: calculating a sum of the first error value and the second error value to obtain a training total error value of the deep learning network model; and adjusting configuration parameters of the deep learning network model until the training total error value no longer decreases, to obtain the target deep learning network model.

[0016] Embodiments of the present application also provide an electronic device, which comprises a processor and a memory, and the processor is configured to execute a computer program stored in the memory to implement the depth image acquisition method.

[0017] Embodiments of the present application also provide a computer readable storage medium, which stores at least one instruction, and the at least one instruction is executed by a processor to implement the depth image acquisition method.

[0018] In the technical solution of the present application, the accuracy and fineness of the depth image can be improved, so that the autonomous driving is safer. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 is an application environment diagram of a depth image acquisition method provided by an embodiment of the present application.

[0020] Figure 2 is a flowchart of a depth image acquisition method provided by an embodiment of the present application.

[0021] Figure 3A structural schematic diagram of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to more clearly understand the above objectives, features and advantages of the present application, the present application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the specific embodiments described herein are merely used to explain the present application and should not be used to limit the present application.

[0023] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. The described embodiments are merely part of the embodiments of the present application, but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of the present application.

[0024] Hereinafter, the terms "first" and "second" are used only for the purpose of description, and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more of the features. In the description of some embodiments of the present application, the words "exemplary" or "for example" are used to mean serving as an example, instance, or illustration, rather than preferred or advantageous over other embodiments or design solutions. In fact, the use of "exemplary" or "for example" is intended to present the relevant concept in a particular manner.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application.

[0026] As shown in Figure 1 , Figure 1 An application environment diagram of a depth image acquisition method is provided for an embodiment of the present application. The depth image acquisition method can be applied to one or more electronic devices 1, which are in communication with a monocular camera 2.

[0027] The depth image acquisition method is applied to one or more electronic devices. The electronic device is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored computer-readable instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0028] The electronic device can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.

[0029] The electronic devices may include network devices and / or user devices. The network devices include, but are not limited to, single network electronic devices, groups of multiple network electronic devices, or cloud computing-based systems consisting of a large number of hosts or network electronic devices.

[0030] The network in which the electronic device is located includes, but is not limited to: the Internet, wide area network, metropolitan area network, local area network, virtual private network (VPN), etc.

[0031] like Figure 2 As shown, Figure 2 This is a flowchart illustrating a depth image acquisition method provided in an embodiment of this application. The order of steps in the flowchart can be adjusted according to different needs, and some steps may be omitted. The method is executed by an electronic device, such as... Figure 1 Electronic device 1 shown.

[0032] Step 201: Acquire the first image and the second image captured by the camera at adjacent time intervals.

[0033] In at least one embodiment of this application, the camera may be a monocular camera. The first image and the second image may be images captured by the monocular camera while the vehicle is in motion.

[0034] In at least one embodiment of this application, the method for acquiring the first image and the second image further includes:

[0035] record a video of the front view content of the vehicle during driving by using a monocular camera, and obtain images of adjacent frames in the video as the first image and the second image by using OpenCV technology.

[0036] In step 202, a first predicted depth map of the first image and a second predicted depth map of the second image are obtained based on a deep learning network model.

[0037] In at least one embodiment of the present application, the deep learning network model comprises a depth estimation convolutional neural network and a pose estimation convolutional neural network.

[0038] In at least one embodiment of the present application, obtaining the first predicted depth map of the first image and the second predicted depth map of the second image based on the deep learning network model comprises:

[0039] The first predicted depth map is obtained by processing the first image by using the depth estimation convolutional neural network, and the second predicted depth map is obtained by processing the second image by using the depth estimation convolutional neural network.

[0040] In this embodiment, the depth estimation convolutional neural network comprises an input layer, a convolutional layer, and a deconvolutional layer.

[0041] In this embodiment, the first image is input into the depth estimation convolutional neural network through the input layer of the depth estimation convolutional neural network, convolutional operation is performed on the first image by using the convolutional layer of the depth estimation convolutional neural network to obtain a feature map of the first image, the feature map is input into the deconvolutional layer to perform deconvolutional operation to obtain a first high-dimension feature map, and finally each pixel point in the first high-dimension feature map is mapped into a corresponding depth to output the first predicted depth map.

[0042] In this embodiment, the second image is input into the depth estimation convolutional neural network through the input layer, convolutional operation is performed on the second image by using the convolutional layer to obtain a feature map of the second image, the feature map is input into the deconvolutional layer to perform deconvolutional operation to obtain a second high-dimension feature map, and finally each pixel point in the second high-dimension feature map is mapped into a corresponding depth to output the second predicted depth map.

[0043] In step 203, the first image and the second image are processed to obtain a camera transformation matrix between the first image and the second image.

[0044] In at least one embodiment of the present application, the processing of the first image and the second image to obtain the camera transformation matrix between the first image and the second image comprises:

[0045] The pose estimation convolutional neural network is used to process the first image and the second image to obtain a camera transformation matrix between the first image and the second image.

[0046] In this embodiment, the pose estimation convolutional neural network comprises an input layer and a convolutional layer.

[0047] In this embodiment, the first image and the second image are input into the pose estimation convolutional neural network through the input layer of the pose estimation convolutional neural network, and convolutional operation is performed on the first image and the second image through the convolutional layer of the pose estimation convolutional neural network to obtain feature maps of the first image and the second image. A camera pose estimation algorithm is used to obtain the camera pose, i.e. the camera transformation matrix, between the first image and the second image.

[0048] In step 204, the first predicted depth map is converted into a first point cloud map, and the second predicted depth map is converted into a second point cloud map.

[0049] In at least one embodiment of the present application, the first point cloud map and the second point cloud map are both three-dimensional (3D) point cloud maps.

[0050] In at least one embodiment of the present application, converting the first predicted depth map into a first point cloud map comprises:

[0051] A first pixel point coordinate is obtained from the first predicted depth map, and a first point cloud map is obtained according to the first pixel point coordinate and an internal and external parameter matrix transformation formula of the camera. In this embodiment, the essence of converting a depth map into a point cloud is coordinate system transformation, i.e. converting an image coordinate system into a world coordinate system. Based on the internal and external parameter matrix transformation formula of the camera, a depth map can be converted into a 3D point cloud map, thereby solving the problem of how to convert a depth map into a point cloud. In this embodiment, the first pixel point coordinate can be any pixel point coordinate in the first predicted depth map.

[0052] In this embodiment, the internal and external parameter matrix transformation formula of the camera is:

[0053]

[0054] where (x, y, z) is the world coordinate, i.e. the point cloud corresponding to the first pixel point coordinate, f is the focal length of the camera, D is the depth value, and (x1, y1) is the first pixel point coordinate. The pixel point coordinates in the first predicted depth map are converted into world coordinates one by one using the above formula (1) to obtain the first point cloud map.

[0055] In at least one embodiment of the present application, the converting the second predicted depth map into a second point cloud map is similar to the converting the first predicted depth map into a first point cloud map, which will not be repeated herein.

[0056] In step 205, the first point cloud map is converted into a third point cloud map and the second point cloud map is converted into a fourth point cloud map according to the camera conversion matrix.

[0057] In at least one embodiment of the present application, the camera conversion matrix comprises a rotation matrix and a translation matrix.

[0058] In at least one embodiment of the present application, the converting the first point cloud map into a third point cloud map and the second point cloud map into a fourth point cloud map according to the camera conversion matrix comprises:

[0059] obtaining coordinates of points in the first point cloud map and combining the coordinates into a first coordinate set;

[0060] obtaining coordinates of points in the second point cloud map and combining the coordinates into a second coordinate set;

[0061] multiplying the rotation matrix, the translation matrix and the first coordinate set to obtain the third point cloud map; and

[0062] multiplying the rotation matrix, the translation matrix and the second coordinate set to obtain the fourth point cloud map.

[0063] In step 206, the first point cloud map and the fourth point cloud map are subjected to point cloud matching, and a first error value of the point cloud matching of the first point cloud map and the fourth point cloud map is calculated.

[0064] In at least one embodiment of the present application, the first point cloud map and the fourth point cloud map are subjected to point cloud matching according to the camera conversion matrix and an ICP algorithm formula. In this embodiment, the ICP algorithm formula is:

[0065]

[0066] wherein E(X, Y) is a point cloud matching result, R is the rotation matrix, x i is the i-th point in the first point cloud map, t is the translation matrix, y i is the i-th point in the fourth point cloud map. The first point cloud map and the fourth point cloud map are subjected to point cloud matching by using the above formula.

[0067] In at least one embodiment of the present application, calculating the first error value of the point cloud matching of the first point cloud map and the fourth point cloud map comprises:

[0068] determining a plurality of corresponding point pairs of the first point cloud map and the fourth point cloud map;

[0069] calculating the first error value by using a loss function of point cloud matching according to the transformation matrix of the camera and the plurality of corresponding point pairs, the loss function being:

[0070]

[0071] wherein LOSS is the first error value, n is the number of corresponding point pairs of the first point cloud map and the fourth point cloud map, A j is the jth point in the first point cloud map, B j is the jth point in the fourth point cloud map, A j and B j are a pair of corresponding point pairs.

[0072] In the embodiment, determining the plurality of corresponding point pairs of the first point cloud map and the fourth point cloud map comprises:

[0073] obtaining any point in the first point cloud map and any point in the fourth point cloud map, performing point cloud matching according to the camera transformation matrix and formula (2), and selecting points for point cloud matching according to the first point cloud and the fourth point cloud, and using formula (4);

[0074]

[0075] calculating the distance between two points, and determining that the selected points of the first point cloud and the fourth point cloud are a pair of corresponding point pairs if the distance is less than a preset threshold.

[0076] In formula (4), wherein d is the distance of the selected points of the first point cloud and the fourth point cloud, m is the number of the selected points of the first point cloud map and the fourth point cloud, A k is the kth point in the first point cloud map, B k is the kth point in the fourth point cloud map.

[0077] Step 207, performing point cloud matching on the second point cloud map and the third point cloud map, and calculating a second error value of the point cloud matching of the second point cloud map and the third point cloud map.

[0078] In at least one embodiment of the present application, the method of performing point cloud matching on the second point cloud map and the third point cloud map and calculating a second error value of the point cloud matching of the second point cloud map and the third point cloud map is similar to the method of performing point cloud matching on the first point cloud map and the fourth point cloud map and calculating a first error value of the point cloud matching of the first point cloud map and the fourth point cloud map, which will not be repeated here.

[0079] Step 208: Adjust the deep learning network model according to the first error value and the second error value to obtain the target deep learning network model.

[0080] In at least one embodiment of this application, adjusting the deep learning network model based on the first error value and the second error value to obtain the target deep learning network model includes:

[0081] The sum of the first error value and the second error value is calculated to obtain the total training error value of the deep learning network model;

[0082] Adjust the configuration parameters of the deep learning network model until the total training error value no longer decreases, and obtain the target deep learning network model.

[0083] The configuration parameters refer to the initial configuration parameters in the depth estimation convolutional neural network and the pose estimation convolutional neural network. For example, the configuration parameters include the number of convolutional layers and deconvolutional layers in the depth estimation convolutional neural network and the size of the convolutional kernels in the convolutional layers and deconvolutional layers. The configuration parameters also include the number of convolutional layers in the pose estimation convolutional neural network and the size of the convolutional kernels in the convolutional layers.

[0084] Step 209: Input the image to be detected into the target deep learning network model to obtain the depth image corresponding to the image to be detected.

[0085] In at least one embodiment of this application, the image to be detected includes an image corresponding to the field of vision of the autonomous vehicle during driving.

[0086] In at least one embodiment of this application, the depth image corresponding to the image to be detected can be obtained by inputting the image to be detected into the target deep learning network model, thereby achieving the technical effect of acquiring a depth image based on a monocular image.

[0087] It should be noted that the above description is only a specific implementation of this application, but the protection scope of this application is not limited thereto. For those skilled in the art, improvements can be made without departing from the inventive concept of this application, but these all fall within the protection scope of this application.

[0088] like Figure 3 As shown, Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. The electronic device 1 includes a memory 301, at least one processor 302, a computer program 303 stored in the memory 301 and executable on the at least one processor 302, and at least one communication bus 304.

[0089] Those skilled in the art can understand that Figure 3 The schematic diagram shown is merely an example of the electronic device 1 and does not constitute a limitation on the electronic device 1, and can include more or fewer components than shown, or combine certain components, or different components, for example, the electronic device 1 can also include an input / output device, a network access device, etc.

[0090] The at least one processor 302 can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic, discrete hardware components, etc. The at least one processor 302 can be a microprocessor or the at least one processor 302 can also be any conventional processor, etc. The at least one processor 302 is a control center of the electronic device 1, and connects various parts of the entire electronic device 1 through various interfaces and lines.

[0091] The memory 301 can be used to store the computer program 303, and the at least one processor 302 realizes various functions of the electronic device 1 by running or executing the computer program 303 stored in the memory 301, and calling data stored in the memory 301. The memory 301 can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required by a function (such as a sound playing function, an image playing function, etc.), etc.; and the data storage area can store data (such as audio data) created according to the use of the electronic device 1, etc. In addition, the memory 301 can include a non-volatile memory, for example, a hard disk, a memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device.

[0092] The modules / units integrated in the electronic device 1, if implemented in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the computer program can be stored in a computer readable storage medium. When the computer program is executed by a processor, the steps of the above-mentioned various method embodiments can be implemented. The computer program includes computer program code, which can be in the form of source code, object code, executable files or some intermediate forms. The computer readable medium can include any entity or device capable of carrying the computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory and read-only memory (ROM).

[0093] It is obvious for those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. Therefore, the embodiments should be regarded as exemplary and non-limiting, and the scope of the present application is defined by the appended claims rather than the above description, and all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be included in the present application. Any additional reference signs in the claims should not be considered as limiting the claims to which they relate.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting. Although the present application has been described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A method of depth image acquisition, characterized by, The depth image acquisition method comprises: acquiring a first image and a second image taken by a camera at adjacent times; and processing the first image based on a deep learning network model to obtain a first predicted depth image; and processing the second image to obtain a second predicted depth image; processing the first image and the second image to obtain a camera conversion matrix between the first image and the second image; converting the first predicted depth image into a first point cloud image and converting the second predicted depth image into a second point cloud image; converting the first point cloud image into a third point cloud image and converting the second point cloud image into a fourth point cloud image according to the camera conversion matrix; performing point cloud matching on the first point cloud image and the fourth point cloud image and calculating a first error value of the point cloud matching; and performing point cloud matching on the second point cloud image and the third point cloud image and calculating a second error value of the point cloud matching; adjusting the deep learning network model according to the first error value and the second error value to obtain a target deep learning network model; inputting a to-be-detected image into the target deep learning network model to obtain a depth image corresponding to the to-be-detected image.

2. The depth image acquisition method of claim 1, wherein, The conversion of the first predicted depth image into a first point cloud image and the conversion of the second predicted depth image into a second point cloud image comprise: acquiring a plurality of first pixel point coordinates from the first predicted depth image and obtaining the first point cloud image according to the plurality of first pixel point coordinates and an internal and external parameter matrix conversion formula of the camera; and acquiring a plurality of second pixel point coordinates from the second predicted depth image and obtaining the second point cloud image according to the plurality of second pixel point coordinates and the internal and external parameter matrix conversion formula of the camera.

3. The depth image acquisition method of claim 1, wherein, The camera conversion matrix comprises a rotation matrix and a translation matrix, and the conversion of the first point cloud image into a third point cloud image and the conversion of the second point cloud image into a fourth point cloud image according to the camera conversion matrix comprise: acquiring coordinates of points in the first point cloud image and combining the coordinates into a first coordinate set; acquiring coordinates of points in the second point cloud image and combining the coordinates into a second coordinate set; multiplying the rotation matrix, the translation matrix and the first coordinate set to obtain the third point cloud image; and multiplying the rotation matrix, the translation matrix and the second coordinate set to obtain the fourth point cloud image.

4. The depth image acquisition method of claim 1, wherein, The deep learning network model comprises a depth estimation convolutional neural network and a pose estimation convolutional neural network, and the processing of the first image based on the deep learning network model to obtain a first predicted depth image and the processing of the second image to obtain a second predicted depth image comprise: performing convolution operation and deconvolution operation on the first image based on the depth estimation convolutional neural network to obtain the first predicted depth image; and performing convolution operation and deconvolution operation on the second image based on the depth estimation convolutional neural network to obtain the second predicted depth image.

5. The depth image acquisition method of claim 4, wherein, The processing of the first image and the second image to obtain a camera conversion matrix between the first image and the second image comprises: The pose estimation convolutional neural network is used to perform convolution operation on the first image and the second image to obtain a camera conversion matrix.

6. The depth image acquisition method of claim 3, wherein, The point cloud matching of the first point cloud map and the fourth point cloud map comprises: The point cloud matching of the first point cloud map and the fourth point cloud map is performed according to the camera conversion matrix and an ICP algorithm formula; The ICP algorithm formula is: Wherein, E(X, Y) is the point cloud matching result, R is the rotation matrix, x i is the i th point in the first point cloud map, t is the translation matrix, y i is the i th point in the fourth point cloud map.

7. The depth image acquisition method of claim 1, wherein, The calculation of the first error value of the point cloud matching of the first point cloud map and the fourth point cloud map comprises: A plurality of corresponding point pairs of the first point cloud map and the fourth point cloud map are determined; The first error value is calculated by using a loss function of point cloud matching according to the conversion matrix of the camera and the plurality of corresponding point pairs, and the loss function is: wherein LOSS is the first error value, n is the number of pairs of corresponding points of the first point cloud map and the fourth point cloud map, A j is the jth point in the first point cloud map, B j is the jth point in the fourth point cloud map, A j and B j are a pair of corresponding points.

8. The depth image acquisition method of claim 1, wherein, The adjustment of the deep learning network model according to the first error value and the second error value to obtain a target deep learning network model comprises: The sum of the first error value and the second error value is calculated to obtain a training total error value of the deep learning network model; The configuration parameters of the deep learning network model are adjusted until the training total error value no longer decreases, and the target deep learning network model is obtained.

9. An electronic device, comprising: The electronic device comprises a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the depth image acquisition method in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores at least one instruction, and the at least one instruction is executed by the processor to implement the depth image acquisition method in any one of claims 1 to 8.

Images