Image processing method and device, electronic equipment and storage medium

[0084] As an optional implementation, the method also includes:

[0085] Obtain the preset image processing speed;

[0086] determining a second number of operations for performing the upsampling operation according to the image processing speed;

[0087] The step S14 performs an upsampling operation on the feature image, and obtaining a depth prediction image includes:

[0088] According to the second number of operations, an upsampling operation is performed on the feature image to obtain a depth prediction image.

[0089] In this optional implementation manner, an image processing speed can be preset, and the image processing speed can be used to measure the processing speed of the preprocessed image after the preprocessed image is sent to the convolutional neural network model. The image processing speed is often related to the number of up-sampling operations, so the first number of operations for performing the up-sampling operation may be determined according to the preset image processing speed. According to many experiments, when the number of upsampling operations is 4, the image processing speed is the best.

[0090] Please also see Figure 4 , Figure 4 It is a structural schematic diagram of an upsampling operation disclosed in the present invention. like Figure 4 As shown, the upsampling operation may include a convolution operation. in, Figure 4 Convolution f, convolution g, convolution h, convolution i, and convolution j in Convolution represent the convolution operations of different convolution kernels.

[0091] Figure 4 In , mainly through the convolution operation, the extracted feature image is matched with the final depth prediction image that needs to be generated. At the same time, each pixel in the feature image is mapped, that is, the corresponding depth prediction value is given to each pixel. .

[0092] also, Figure 4 In the process of generating the depth prediction image, the intermediate feature information extracted in the downsampling operation can also be added, which can make up for the image feature information lost in the downsampling operation to a certain extent, so as to improve the accuracy of depth prediction. Wherein, the intermediate feature information is also a kind of feature image. However, compared with the feature image, the feature extracted by the intermediate feature information is not complete enough, and contains some redundant feature image information, because the intermediate feature information is an unrefined feature image obtained through a small number of down-sampling operations. The downsampling operation can be understood as refining the preprocessed image, so that the feature image obtained after multiple downsampling operations is the purest information. However, at the same time, the final feature image will lose some useful information more or less, so the intermediate feature information can be used to supplement the lost part.

[0093] As an optional implementation, the method also includes:

[0094] If the image to be processed is a frame image obtained by processing a video stream, each depth prediction image is integrated to obtain a final depth prediction image.

[0095] In this optional implementation, the convolutional neural network model can only process a single image. If a single camera obtains a video stream, it needs to perform frame extraction processing on the video stream to obtain multiple frame images, and then send In this way, multiple depth prediction images can be obtained, and further, the multiple depth prediction images need to be integrated to obtain the final depth prediction image.

[0096] exist figure 1 In the described method flow, the image to be processed can be acquired through a single camera; further, the image to be processed is preprocessed to obtain a preprocessed image; further, based on a pre-trained convolutional neural network model, the A downsampling operation is performed on the preprocessed image to obtain a feature image; an upsampling operation is performed on the feature image to obtain a depth prediction image. It can be seen that in the present invention, a single camera is used to obtain the image to be processed, the cost is low, and the computing power is not high, and only the image to be processed needs to be sent to the pre-trained convolutional neural network model. The environment depth prediction is carried out, and the depth prediction image is finally generated. In addition, the accuracy and rate of the environment depth prediction are improved.