Zoom method, device and storage medium based on deep learning

Abstract

The present application discloses a zoom method, device and storage medium based on deep learning, which includes: inputting a target image acquired by an image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model; comparing a target focus area in the target image with a predicted focus area in the focus prediction image to obtain a focus comparison result; and zooming the image acquisition device according to the focus comparison result. The above scheme can improve the zoom accuracy.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a zoom method, device and storage medium based on deep learning. Background Art

[0002] With the rapid development of related technologies in the fields of image processing and computer vision, image acquisition and transmission have become popular in all aspects of life.

[0003] Especially in some special scenarios, such as railway road inspection, power transmission line inspection or pipeline inspection, instrument reading inspection and many other industrial scenarios, it is necessary to be able to focus quickly and clearly on the target object in the area of ​​interest of the captured image at a long distance and high magnification.

[0004] However, due to the limitations of the optical imaging principle of the lens, especially the telephoto lens, the target object and the background in the captured image cannot be kept in focus at the same time, which is usually manifested as the presence of double peaks in the focus evaluation function. During user use, it may cause the user to originally expect to focus on the foreground target object in the captured image, but the actual lens focuses on the distant view of the image, reducing the zoom accuracy. Summary of the invention

[0005] The present application at least provides a zoom method, apparatus, device and computer-readable storage medium based on deep learning.

[0006] In a first aspect, the present application provides a zoom method based on deep learning, comprising: inputting a target image captured by an image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model; comparing a target focus area in the target image with a predicted focus area in the focus prediction image to obtain a focus comparison result; and performing zoom processing on the image acquisition device according to the focus comparison result.

[0007] In one embodiment, before the step of inputting the target image captured by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model, the method further includes: performing focus evaluation processing on the first image and the second image captured by the image acquisition device respectively to obtain a first focus evaluation value corresponding to the first image and a second focus evaluation value corresponding to the second image; wherein the image acquisition focal lengths of the first image and the second image are different; judging whether there is a focus evaluation peak between the first focus evaluation value and the second focus evaluation value according to the first focus evaluation value, the second focus evaluation value and a preset focus evaluation threshold; and determining the first image or the second image as the target image in response to the presence of the focus evaluation peak between the first focus evaluation value and the second focus evaluation value.

[0008] In one embodiment, before the step of comparing the target focus area in the target image with the predicted focus area in the focus prediction image, the method further includes: determining the target focus area in the target image according to the clarity of the target image, and determining the predicted focus area in the focus prediction image according to the clarity of the focus prediction image; the step of comparing the target focus area in the target image with the predicted focus area in the focus prediction image to obtain a focus comparison result includes: comparing position information of the target focus area with position information of the predicted focus area to obtain a position deviation between the target focus area and the predicted focus area, and determining the focus comparison result according to the position deviation.

[0009] In one embodiment, the steps of determining the target focus area in the target image according to the clarity of the target image, and determining the predicted focus area in the focus prediction image according to the clarity of the focus prediction image, include: performing image segmentation processing on the target image and the focus prediction image respectively to obtain multiple target sub-images in the target image and multiple focus prediction sub-images in the focus prediction image; determining the target sub-image with the highest clarity among the target sub-images as the target focus area, and determining the predicted focus area with the focus prediction sub-image with the highest clarity among the focus prediction sub-images.

[0010] In one embodiment, the step of determining the focus comparison result based on the position deviation includes: in response to the position deviation being greater than a preset deviation threshold, determining that the image acquisition device has failed to focus, and obtaining the focus comparison result; in response to the position deviation being less than or equal to the preset deviation threshold, determining that the image acquisition device has successfully focused, and obtaining the focus comparison result.

[0011] In one embodiment, the step of performing zoom processing on the image acquisition device according to the focus comparison result includes: in response to the focus comparison result indicating that the image acquisition device fails to focus, performing zoom processing on the image acquisition device according to a preset zoom step, and obtaining a third image and a fourth image obtained by the image acquisition device after the zoom processing; wherein the image acquisition focal lengths of the third image and the fourth image are different; performing focus evaluation processing on the third image and the fourth image respectively to obtain a third focus evaluation value corresponding to the third image and a fourth focus evaluation value corresponding to the fourth image; in response to the presence of a peak between the third focus evaluation value and the fourth focus evaluation value, reducing the preset zoom step to obtain a reduced zoom step; performing zoom processing on the image acquisition device according to the reduced zoom step until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is less than a preset focus error threshold.

[0012] In one embodiment, the step of performing zoom processing on the image acquisition device according to the reduced zoom step length until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is less than a preset focus error threshold comprises: adjusting the image acquisition focal length of the image acquisition device according to the reduced zoom step length to obtain an adjusted image acquisition focal length; controlling the image acquisition device to perform image acquisition according to the adjusted image acquisition focal length to obtain the fifth image and the sixth image; wherein the image acquisition focal lengths of the fifth image and the sixth image are different.

[0013] In one embodiment, after the step of controlling the image acquisition device to acquire images according to the adjusted image acquisition focal length to obtain the fifth image and the sixth image, the method further includes: performing focus evaluation processing on the fifth image and the sixth image respectively to obtain a fifth focus evaluation value corresponding to the fifth image and a sixth focus evaluation value corresponding to the sixth image; determining the focus evaluation difference according to the fifth focus evaluation value and the sixth focus evaluation value; and pausing or stopping zoom processing of the image acquisition device in response to the focus evaluation difference being less than or equal to the focus error threshold.

[0014] In one embodiment, the step of performing focus evaluation processing on the first image and the second image captured by the image acquisition device respectively to obtain a first focus evaluation value corresponding to the first image and a second focus evaluation value corresponding to the second image includes: obtaining the grayscale value of the first image; and calculating according to a preset focus evaluation function and the grayscale value of the first image to obtain the first focus evaluation value.

[0015] In one embodiment, before the step of inputting the target image acquired by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model, the method further includes: inputting the acquired sample image into an initial model to be trained so that the initial model performs forward diffusion processing on the sample image to obtain a noise image; wherein the sample image is focused on a target area; performing reverse diffusion processing on the noise image according to the initial model to obtain a denoised image and a model loss value corresponding to the denoised image; and performing model training on the initial model according to the loss value to obtain the diffusion model.

[0016] The second aspect of the present application provides a zoom device based on deep learning, including: a focus prediction module, which is used to input the target image captured by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model; a focus comparison module, which is used to compare the target focus area in the target image with the predicted focus area in the focus prediction image to obtain a focus comparison result; and a zoom processing module, which is used to perform zoom processing on the image acquisition device according to the focus comparison result.

[0017] A third aspect of the present application provides an electronic device, comprising a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the above-mentioned deep learning-based zoom method.

[0018] A fourth aspect of the present application provides a computer-readable storage medium having program instructions stored thereon, which implement the above-mentioned deep learning-based zoom method when executed by a processor.

[0019] The above scheme inputs the target image captured by the image acquisition device into a pre-trained diffusion model, and the diffusion model performs diffusion processing on the received target image to obtain a focus prediction image output by the diffusion model; the target focus area in the target image is compared with the predicted focus area in the focus prediction image to obtain a focus comparison result, thereby judging whether the target area of ​​the target image captured by the image acquisition device based on the current zoom parameters matches the predicted focus area in the focus prediction image predicted by the diffusion model according to the focus comparison result; the image acquisition device is zoomed according to the focus comparison result, thereby realizing zooming of the image acquisition device based on deep learning technology and improving the zoom accuracy of the zoom process.

[0020] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present application and are used together with the specification to illustrate the technical solution of the present application.

[0022] Figure 1 is a flowchart of an exemplary embodiment of a zoom method based on deep learning of the present application;

[0023] Figure 2 Schematic diagram of double peaks of the focus evaluation function in the zoom method based on deep learning of the present application;

[0024] Figure 3 is an exemplary image segmentation schematic diagram in the zoom method based on deep learning of the present application;

[0025] Figure 4 is a schematic diagram of forward diffusion in the zoom method based on deep learning of the present application;

[0026] Figure 5 is a schematic diagram of constructing training samples in the zoom method based on deep learning of the present application;

[0027] Figure 6 is a schematic diagram of reverse diffusion in the zoom method based on deep learning of the present application;

[0028] Figure 7 is a schematic diagram of a zoom device based on deep learning shown in an exemplary embodiment of the present application;

[0029] Figure 8 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0030] Fig. 9 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0031] The scheme of the embodiment of the present application is described in detail below in conjunction with the drawings of the specification.

[0032] In the following description, for the purpose of explanation rather than limitation, specific details such as specific system structures, interfaces, and technologies are provided to facilitate a thorough understanding of the present application.

[0033] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. In addition, "many" in this article means two or more than two. In addition, the term "at least one" in this article means any combination of at least two of any one or more of a plurality of, for example, including at least one of A, B, and C, can mean including any one or more elements selected from the set consisting of A, B, and C.

[0034] For ease of understanding, the applicable scenarios of this application are now explained exemplarily. With the rapid development of related technologies such as image processing and computer vision, image acquisition and transmission have been popularized in all aspects of life. Especially in some special scenarios, such as railway road detection, power transmission line or pipeline detection, instrument reading detection and many other industrial scenarios, image acquisition equipment is required to be able to quickly and clearly focus on the target in the area of ​​interest at a long distance and a large magnification. However, due to the limitations of telephoto lenses, during the image acquisition process, the target and background areas often cannot be kept in focus at the same time, which also causes the focus evaluation function to have double peaks.

[0035] Although the traditional hill climbing algorithm can obtain a clear focus point, it cannot guarantee that the focus point is the focus position of the area of ​​interest. Therefore, during the image acquisition process, there may be a problem that the lens actually focuses on the distant view while the foreground target is to be seen clearly, and the zoom accuracy of the image acquisition process cannot be ensured.

[0036] See also Figure 1 , Figure 1 : is a flowchart of an exemplary embodiment of the zoom method based on deep learning of the present application. Specifically, it may include the following steps:

[0037] Step S110: input the target image acquired by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model.

[0038] The target image may refer to an image acquired by the image acquisition device after a certain degree of zoom processing (including manual zoom and / or automatic zoom) during the image acquisition process, as opposed to an initial image acquired by the image acquisition device before zoom processing. It should be noted that in the present application, in the process of obtaining the target image by zoom processing based on the initial image, relevant image acquisition parameters (such as zoom direction, zoom step, focal length, pixel grayscale value, etc.) may be recorded and analyzed.

[0039] Diffusion Model, also known as a generative model. In the training process of a common diffusion model, it is usually necessary to define a Markov chain of diffusion steps to slowly add random noise to the data, and then learn the reverse diffusion process to construct the required data samples from the noise. In simple terms, the diffusion model can be used to generate data similar to the training data. Its working principle is mainly to destroy the training data by continuously adding Gaussian noise, and then learn to restore the training data by reversing (reverse diffusion) the noise process. In the zoom method of the present application, it is equivalent to predicting the focus area (region of interest) in the image through the reverse diffusion process of the diffusion model, that is, obtaining the focus prediction image output by the diffusion model.

[0040] Step S120 , comparing the target focus area in the target image with the predicted focus area in the focus prediction image to obtain a focus comparison result.

[0041] The target focus area refers to a clearly focused area in the target image, and the predicted focus area refers to a clearly focused area in the focus prediction image.

[0042] For example, in the present application, the sharp focus area can be determined by the sharpness of each pixel in the image. For example, the target focus area of ​​the target image is determined by the sharpness of each pixel in the target image, and the predicted focus area of ​​the predicted focus image is determined by the sharpness of each pixel in the predicted focus image.

[0043] It should be noted that the position of the target focus area in the target image and the position of the predicted focus area in the focus prediction image may be the same or different, and this application does not limit this. In this application, it is equivalent to using the position of the predicted focus area generated by the diffusion model as the reference data for focusing accuracy; if the position of the target focus area is the same as the position of the predicted focus area, it indicates that the image acquisition device is currently focusing accurately; if the position of the target focus area is different from the position of the predicted focus area, it indicates that the image acquisition device is currently not focusing accurately, and the image acquisition device needs to continue to perform zoom processing. Among them, the resolution information of the target image and the focus prediction image of this application can be kept consistent so that the target focus area and the predicted focus area can be accurately compared.

[0044] It should also be noted that in the present application, the standard for determining whether the image acquisition device is accurately focused can be that the position of the target focus area is required to be exactly the same as the position of the predicted focus area, or it can be that a position deviation threshold is set to provide an error interval for the processing process of the present application. For example, the position deviation between the position of the target focus area and the position of the predicted focus area is obtained; if the position deviation is less than or equal to the position deviation threshold, it indicates that the target focus area matches the predicted focus area, and it can be determined that the image acquisition device is accurately focused; if the position deviation is greater than the position deviation threshold, it indicates that the target focus area does not match the predicted focus area, and it can be determined that the image acquisition device is not accurately focused, that is, it indicates that the target focus area currently focused by the image acquisition device is not the area of ​​interest.

[0045] Step S130: zooming the image acquisition device according to the focus comparison result.

[0046] Combined with the above steps, if the focus comparison result indicates that the image acquisition device is accurately focused, the zoom processing process can be paused or stopped; if the focus comparison result indicates that the image acquisition device is not accurately focused, the image acquisition device can continue to be zoomed until the actual focus area of ​​the image acquisition device matches the focus area predicted by the diffusion model, indicating that the image acquisition device is accurately focused.

[0047] Exemplarily, the method of performing zoom processing on the image acquisition device in this step may include, but is not limited to, recording the zoom direction from the initial image to the target image as the first direction, and recording the opposite direction (the zoom direction from the target image to the initial image) as the second direction. Controlling the image acquisition device to perform zoom processing in the first direction and / or the second direction.

[0048] It can be seen that the present application inputs the target image captured by the image acquisition device into a pre-trained diffusion model, and the diffusion model performs diffusion processing on the received target image to obtain a focus prediction image output by the diffusion model; the target focus area in the target image is compared with the predicted focus area in the focus prediction image to obtain a focus comparison result, thereby determining whether the target area of ​​the target image captured by the image acquisition device based on the current zoom parameters matches the predicted focus area in the focus prediction image predicted by the diffusion model based on the focus comparison result; the image acquisition device is zoomed according to the focus comparison result, thereby achieving zooming of the image acquisition device based on deep learning technology and improving the zoom accuracy of the zoom process.

[0049] Based on the above embodiments, the present embodiment of the application describes the steps before inputting the target image acquired by the image acquisition device into the pre-trained diffusion model to obtain the focus prediction image output by the diffusion model. Specifically, the method of this embodiment includes the following steps:

[0050] Focus evaluation processing is performed on a first image and a second image captured by an image capture device respectively to obtain a first focus evaluation value corresponding to the first image and a second focus evaluation value corresponding to the second image; wherein the image capture focal lengths of the first image and the second image are different; based on the first focus evaluation value, the second focus evaluation value and a preset focus evaluation threshold, it is determined whether there is a focus evaluation peak between the first focus evaluation value and the second focus evaluation value; in response to the presence of a focus evaluation peak between the first focus evaluation value and the second focus evaluation value, the first image or the second image is determined as a target image.

[0051] Focus evaluation processing refers to evaluating the focus degree of an image, which can be based on all pixels in the image or based on some pixels in the image, and is not limited here. Focus evaluation processing of an image can obtain a focus evaluation value of the image, and the focus degree of the image can be determined by the focus evaluation value.

[0052] For example, in the present application, the image (which may include the first image and the second image) may be evaluated and processed according to a preset focus evaluation function. The mathematical expression of the focus evaluation function is:

[0053]

[0054] Where f(i,j) refers to the grayscale value of the digital signal of the image at point (i,j), FV(u,v) refers to the value after discrete cosine transform (DCT) in the frequency domain of the image (focus evaluation value), u and v refer to the uv coordinates of the image (which can represent the coordinates of the image pixels in the horizontal and vertical directions respectively), N refers to the number of pixels in the image participating in the focus evaluation process (which can be the total number of pixels in the image), c(u) and c(v) refer to the preset compensation coefficients, so that the DCT transformation matrix is ​​an orthogonal matrix, and c(u)= if and only if u=0. , the value in other scenarios can be 1.

[0055] Specifically, in this embodiment, the image acquisition device can perform image acquisition according to at least two different image acquisition focal lengths to obtain at least two images, such as a first image and a second image. It should be noted that the relationship between the first image and the second image can refer to the relationship between the initial image and the target image in the aforementioned embodiment. The image acquisition device can acquire the second image after acquiring the first image through zoom processing, or the image acquisition device can acquire the first image after acquiring the second image through zoom processing, which is not limited here. For ease of explanation, in the examples of this application, the acquisition timing of the first image can be taken as an example to illustrate earlier than the acquisition timing of the second image.

[0056] For example, the first image collected is , the position of the lens of the image acquisition device when the first image is acquired can be recorded as the initial position (It may also be a recording of the corresponding initial focal length, etc., which is not limited here). After the first image is subjected to focus evaluation processing, a first focus evaluation value of the first image may be obtained. . Perform zoom processing on the image acquisition device to obtain the current position of the image acquisition device lens (or the corresponding current focal length) and perform image acquisition, the second image can be obtained After performing focus evaluation processing on the second image, a second focus evaluation value of the second image can be obtained. .

[0057] It should be noted that in the present application, the zoom direction when capturing the initial image can be used as a reference direction of the image capture device in the subsequent zoom process. For example, the direction corresponding to the automatic zoom command and / or manual zoom command received by the image capture device when capturing the initial image is used as the initial movement direction of the focusing lens motor and is recorded as Dira (equivalent to the first direction of the aforementioned embodiment), and the corresponding reverse direction is recorded as Dirb (equivalent to the second direction of the aforementioned embodiment).

[0058] According to the first focus evaluation value , Second focus evaluation value The focus evaluation threshold FV' is preset to determine whether there is a focus evaluation peak between the first focus evaluation value and the second focus evaluation value. A hill climbing algorithm may be used to determine whether there is a focus evaluation peak between the first focus evaluation value and the second focus evaluation value.

[0059] Exemplarily, referring to the above description, the position of the lens when capturing the first image is , the initial moving direction of the lens is recorded as Dira, that is, Dir=Dira. The lens can be controlled to move △L in the direction of Dir according to the preset zoom step (or moving step) to reach the current position And perform image acquisition to obtain the second image . Calculate the focus evaluation values ​​of the first image and the second image respectively. If FV0<FV1<FV', the FV change trend is a positive trend at this time, indicating that the focus accuracy of the image collected by the image acquisition device before and after the zoom processing is gradually improved, and then continue to repeat the above-mentioned zoom processing steps. If FV'>FV0>FV1, the FV change trend is a negative trend at this time, indicating that the focus accuracy of the image collected by the image acquisition device before and after the zoom processing is gradually reduced, then it is necessary to adjust the zoom parameters (including but not limited to adjusting the zoom reverse direction and / or zoom step). For example, the movement direction of the lens becomes Dirb, that is, Dir=Dirb, the zoom step △L can be shortened to half of the original, and then repeat the above zoom process. After n consecutive zoom processing based on the hill climbing algorithm, when FVn-1 and FVn are both greater than the focus evaluation threshold FV' and the FV change trend changes from positive to negative, the hill climbing algorithm processing can be stopped. At this time, the positions of the lens corresponding to FVn-1 and FVn are respectively and , and can determine and There is a peak of the focusing evaluation function (ie, the focusing evaluation peak) between them.

[0060] In summary, we can Corresponding image or Corresponding image Determined as the target image; or and Select a lens position Corresponding image As the target image. It is understandable that if it is determined that there is a focus evaluation peak between the first image and the second image, the first image or the second image can be directly determined as the target image, or zoom processing based on the hill climbing algorithm can be performed in accordance with the above method to determine a target image with more accurate focus.

[0061] On the basis of the above embodiments, the embodiment of the present application describes the steps before comparing the target focus area in the target image with the predicted focus area in the focus prediction image and the steps of comparing the target focus area in the target image with the predicted focus area in the focus prediction image to obtain the focus comparison result. Specifically, the method of this embodiment includes the following steps:

[0062] A target focus area in the target image is determined according to the clarity of the target image, and a predicted focus area in the predicted focus image is determined according to the clarity of the predicted focus image; position information of the target focus area is compared with position information of the predicted focus area to obtain a position deviation between the target focus area and the predicted focus area, and a focus comparison result is determined according to the position deviation.

[0063] Among them, the clarity can be determined according to the focus evaluation value of each pixel point in the aforementioned embodiment, and reference is also made to existing image clarity evaluation methods, such as Brenner gradient method, Tenegrad gradient method, laplace gradient method, variance method, energy gradient method, etc., which are not limited in this application.

[0064] In combination with the above-mentioned embodiment, since the lens depth of field scene may have a double peak of the focus evaluation function, it is necessary to determine whether the current lens position is near the focus position corresponding to the area of ​​interest of the scene. Input the trained Diffusion Model to perform reverse diffusion processing to obtain the output of the diffusion model, which is the focus prediction image. '.

[0065] Exemplarily, a partial image area with higher clarity in the target image is determined as the target focus area; and a partial image area with higher clarity in the focus prediction image is determined as the predicted focus area. By comparing the position information of the target focus area with the position information of the predicted focus area, the position deviation between the target focus area and the predicted focus area can be obtained. If the position deviation is less than or equal to the preset position deviation threshold, a focus comparison result is obtained, indicating that the target focus area matches the predicted focus area, and it can be determined that the image acquisition device is accurately focused; if the position deviation is greater than the position deviation threshold, a focus comparison result is obtained, indicating that the target focus area does not match the predicted focus area, and it can be determined that the image acquisition device is not accurately focused, and the double peak situation described in the above example may occur. For specific examples, please refer to Figure 2 , Figure 2 is a schematic diagram of double peaks of the focus evaluation function in the zoom method based on deep learning of the present application, wherein: and Although there is a peak 1 between The target focus area and ' is greater than the position deviation threshold, it means that the image area corresponding to peak 1 is a clearly focused area, but not an area of ​​interest. Therefore, it is necessary to control the image acquisition device to perform zoom processing until it enters the valley value of the focus evaluation function curve, and then continue to perform the hill climbing algorithm to find the second peak (peak 2), and determine that the lens position range (focal length range) near peak 2 is the focus range when accurately focusing on the area of ​​interest.

[0066] Based on the above embodiments, the present embodiment of the present application describes the steps of determining the target focus area in the target image according to the clarity of the target image, and determining the predicted focus area in the predicted focus image according to the clarity of the predicted focus image. Specifically, the method of this embodiment includes the following steps:

[0067] The target image and the focus prediction image are respectively subjected to image segmentation processing to obtain a plurality of target sub-images in the target image and a plurality of focus prediction sub-images in the focus prediction image; the target sub-image with the highest clarity among the target sub-images is determined as the target focus area, and the focus prediction sub-image with the highest clarity among the focus prediction sub-images is determined as the predicted focus area.

[0068] In combination with the above-mentioned embodiment, the method for determining the target focus area and the predicted focus area in this embodiment can be determined according to the image block processing method, thereby improving the processing efficiency.

[0069] Exemplarily, the target image and the focus prediction image are respectively subjected to image segmentation processing to obtain a plurality of target sub-images (or target image blocks) in the target image and a plurality of focus prediction sub-images (or prediction image blocks) in the focus prediction image. Figure 3 As shown, Figure 3 FIG. 1 is an exemplary image segmentation diagram in the zoom method based on deep learning of the present application. and focus prediction image 'Divide into N*M image square matrices according to the resolution (each square is equivalent to a sub-image). Each square can determine the filling value of the square according to the clarity or focus evaluation value of each pixel in the square. Specifically, the average value calculation method, weighted average calculation method, summation calculation method, etc. can be used, which are not limited here. For example, each square is filled with a value of 0-10 according to the focus situation. The higher the focus evaluation value or the higher the clarity of the square, the higher the filling value. In this way, two filled N*M image square matrices can be obtained. The argmax function can be used to quickly determine the specific positions of the maximum value squares in the two matrices, that is, determine the target sub-image with the highest clarity in the target image, obtain the target focus area, and determine the focus prediction sub-image with the highest clarity in the focus prediction sub-image, and obtain the predicted focus area.

[0070] Based on the above embodiment, the embodiment of the present application describes the steps of determining the focus comparison result according to the position deviation. Specifically, the method of this embodiment includes the following steps:

[0071] In response to the position deviation being greater than a preset deviation threshold, it is determined that the image acquisition device has failed to focus, and a focus comparison result is obtained; in response to the position deviation being less than or equal to the preset deviation threshold, it is determined that the image acquisition device has successfully focused, and a focus comparison result is obtained.

[0072] The above embodiments are described in conjunction with the above embodiments. Figure 3 If the position deviation between the target focus area and the predicted focus area is greater than the preset deviation threshold, it is determined that the image acquisition device has failed to focus (or is not focused accurately), and a focus comparison result is obtained. If the position deviation is less than or equal to the preset deviation threshold, it is determined that the image acquisition device has successfully focused (or is focused accurately), and a focus comparison result is obtained.

[0073] It should be noted that if [ , ] to select the target image The target focus area and the focus prediction image ' is less than or equal to the preset deviation threshold, it can be determined that [ , ] is the focus range corresponding to the accurate focus on the area of ​​interest. Otherwise, it is necessary to continue to perform zoom processing on the image acquisition device.

[0074] Based on the above embodiment, the embodiment of the present application describes the steps of performing zoom processing on the image acquisition device according to the focus contrast result. Specifically, the method of this embodiment includes the following steps:

[0075] In response to the focus comparison result indicating that the image acquisition device has failed to focus, the image acquisition device is zoomed according to a preset zoom step, and a third image and a fourth image are acquired by the image acquisition device after the zoom processing; wherein the third image and the fourth image have different image acquisition focal lengths; the third image and the fourth image are respectively subjected to focus evaluation processing to obtain a third focus evaluation value corresponding to the third image and a fourth focus evaluation value corresponding to the fourth image; in response to the presence of a peak between the third focus evaluation value and the fourth focus evaluation value, the preset zoom step is reduced to obtain a reduced zoom step; the image acquisition device is zoomed according to the reduced zoom step until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is less than a preset focus error threshold.

[0076] In conjunction with the above embodiment, in response to the focus comparison result indicating that the image acquisition device fails to focus (or is not focused accurately), it is necessary to continue to perform zoom processing on the image acquisition device. and After determining that the image acquisition device is not focused accurately, the image acquisition device is iteratively zoomed until the image acquisition device is focused accurately. and It is determined that the image acquisition device is accurately focused, and the zoom adjustment of the image acquisition device can be paused or stopped.

[0077] For example, in response to the focus comparison result indicating that the image acquisition device fails to focus, the image acquisition device can be zoomed according to the preset zoom step length △L. The relationship between the third image and the fourth image can refer to the relationship between the first image and the second image in the aforementioned embodiment, which will not be described in detail here. In this embodiment, the acquisition timing of the third image is earlier than the acquisition timing of the fourth image. This is equivalent to adjusting the lens to +△L, image acquisition is performed to obtain the third image; when the lens is adjusted to +△L, image acquisition is performed to obtain the fourth image.

[0078] With reference to the description of the above-mentioned embodiment, the third image and the fourth image are similarly subjected to focus evaluation processing to obtain a third focus evaluation value corresponding to the third image and a fourth focus evaluation value corresponding to the fourth image; in response to the presence of a peak between the third focus evaluation value and the fourth focus evaluation value, the preset zoom step is reduced to obtain a reduced zoom step; the image acquisition device is subjected to zoom processing according to the reduced zoom step, and the image acquisition device is controlled to perform image acquisition according to the image acquisition parameters after the zoom processing, until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is less than the preset focus error threshold. Wherein, after the correct focus range is determined, the preset zoom step is reduced mainly to reduce the focus adjustment amplitude during the zoom processing (the reduced zoom step can be recorded as △l), and then the zoom processing is continued based on the hill climbing algorithm, so that the image acquisition device can accurately approach the focus evaluation peak of the focus evaluation function until the difference between the FVs of the two images is less than the tolerable focus error threshold.

[0079] It is understandable that the embodiments of the present application are mainly explained by taking the double peak situation as an example. Therefore, if the image area corresponding to the peak 1 determined between the first image and the second image is not the area of ​​interest, then after the zoom processing based on the hill climbing algorithm, if the peak 2 is determined between the third image and the fourth image, the image area corresponding to the peak 2 can be directly determined as the area of ​​interest, and there is no need to input the collected image into the diffusion model for prediction again and then compare the focus area deviation of the two images, thereby improving the execution efficiency. Therefore, the position of the lens corresponding to the peak 2 (also equivalent to the focal length) can be determined by the method of reducing the zoom step size for zoom processing, and the focus on the area of ​​interest can be completed. Of course, the present application does not limit the above steps. In order to improve the accuracy of the method, it is also possible to select the collected image to be input into the diffusion model for prediction again and then compare the focus area deviation of the two images.

[0080] On the basis of the above embodiment, the embodiment of the present application describes the steps of performing zoom processing on the image acquisition device according to the reduced zoom step length until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is less than a preset focus error threshold. Specifically, the method of this embodiment includes the following steps:

[0081] The image acquisition focal length of the image acquisition device is adjusted according to the reduced zoom step length to obtain the adjusted image acquisition focal length; the image acquisition device is controlled to perform image acquisition according to the adjusted image acquisition focal length to obtain a fifth image and a sixth image; wherein the image acquisition focal lengths of the fifth image and the sixth image are different.

[0082] In conjunction with the above-mentioned embodiment, after determining that there is a peak 2 in the third image and the fourth image, the zoom processing can be continued based on the reduced zoom step △l and the climbing zoom processing method in the above-mentioned embodiment to obtain an adjusted image acquisition focal length. The image acquisition device is controlled to perform image acquisition according to the adjusted image acquisition focal length to obtain the fifth image and the sixth image. Among them, the image acquisition focal lengths of the fifth image and the sixth image are different, and the relationship between the fifth image and the sixth image can be similarly referred to the description of the relationship between the first image and the second image in the above-mentioned embodiment, which will not be repeated here. It should be noted that the fifth image and the sixth image can be images captured after zooming once or multiple times on the basis of the third image and the fourth image, which is not limited here.

[0083] Based on the above embodiment, the embodiment of the present application describes the steps after controlling the image acquisition device to acquire images according to the adjusted image acquisition focal length to obtain the fifth image and the sixth image. Specifically, the method of this embodiment includes the following steps:

[0084] Perform focus evaluation processing on the fifth image and the sixth image respectively to obtain a fifth focus evaluation value corresponding to the fifth image and a sixth focus evaluation value corresponding to the sixth image; determine a focus evaluation difference according to the fifth focus evaluation value and the sixth focus evaluation value; in response to the focus evaluation difference being less than or equal to a focus error threshold, pause or stop zoom processing of the image acquisition device.

[0085] In conjunction with the above-mentioned embodiment, it can be known that there is a focal length corresponding to the focus evaluation peak value between the focal length of the fifth image and the focal length of the sixth image after the processing of the above-mentioned embodiment. Therefore, after the processing of the above-mentioned embodiment, the fifth focus evaluation value of the fifth image and the sixth focus evaluation value of the sixth image can be obtained to determine whether the focus evaluation difference between the fifth focus evaluation value and the sixth focus evaluation value is less than or equal to the focus error threshold; if it is greater than the focus error threshold, it indicates that the gap between the fifth image and the sixth image is large, and the focus evaluation peak value range cannot be accurately located, so it is necessary to continue the hill-climbing zoom processing; if it is less than or equal to the focus error threshold, it indicates that the gap between the fifth image and the sixth image is small, and the focus evaluation peak value range can be accurately reflected, so the lens position (or focal length) in the interval range from the lens position (or focal length) corresponding to the fifth image to the lens position (or focal length) corresponding to the sixth image can be selected as the lens position (or focal length) that accurately focuses on the area of ​​interest. For example, the lens motor can be stopped directly at the lens position (or focal length) of the sixth image to complete the focusing on the area of ​​interest.

[0086] On the basis of the above embodiments, the embodiment of the present application describes the steps of performing focus evaluation processing on the first image and the second image captured by the image acquisition device, respectively, to obtain a first focus evaluation value corresponding to the first image and a second focus evaluation value corresponding to the second image. Specifically, the method of this embodiment includes the following steps:

[0087] The grayscale value of the first image is obtained; and a first focusing evaluation value is obtained by performing calculation according to a preset focusing evaluation function and the grayscale value of the first image.

[0088] In conjunction with the above-mentioned embodiments, this embodiment mainly explains the focus evaluation process of the first image. It is understandable that the focus evaluation process of other images in this application can be referred to in the same way, so it will not be described in detail.

[0089] Specifically, the grayscale value of each pixel of the first image and the number of pixels of the first image can be obtained, and the grayscale value and the number of pixels that need to be evaluated for focus are input into a preset focus evaluation function to obtain the focus evaluation value output by the focus evaluation function.

[0090] Based on the above embodiments, the present embodiment of the application describes the steps before inputting the target image acquired by the image acquisition device into the pre-trained diffusion model to obtain the focus prediction image output by the diffusion model. Specifically, the method of this embodiment includes the following steps:

[0091] The acquired sample image is input into the initial model to be trained, so that the initial model performs forward diffusion processing on the sample image to obtain a noise image; wherein the sample image is focused on the target area; the noise image is subjected to reverse diffusion processing according to the initial model to obtain a denoised image and a model loss value corresponding to the denoised image; the initial model is trained according to the loss value to obtain a diffusion model.

[0092] In conjunction with the above-mentioned embodiment, before inputting the target image into the diffusion model, it is necessary to use a real image clearly focused on the area of ​​interest as a sample for model training, thereby enabling the trained diffusion model to generate an accurately focused prediction image based on the input target image.

[0093] For example, first, we need to construct training samples (sample images). We need to obtain a sufficient number of real pictures with clear focus on the area of ​​interest as training samples. Each forward diffusion process of the Diffusion Model is to add Gaussian noise to the sample picture x0 through T accumulations to obtain x1, x2, ..., xT. Figure 4 As shown, Figure 4is a schematic diagram of forward diffusion in the zoom method based on deep learning of the present application, and the mathematical expression of its noise superposition method is shown in the following formula:

[0094]

[0095] in, The Gaussian distribution variance of the series is a hyperparameter that needs to be customized. It determines the step size of the training. The specific principle of the formula can be referred to the description of the forward diffusion noise superposition method in the prior art, which will not be repeated here.

[0096] Please refer to Figure 5 As shown, Figure 5 It is a schematic diagram of constructing training samples in the zoom method based on deep learning of this application. Among them, as t increases, xt gets closer and closer to pure noise. When T tends to infinity, xT becomes complete Gaussian noise. Due to the need to balance the benefits between training effect and training duration, T can select a suitable value according to the actual application scenario, which is not limited here. Perform the above-mentioned forward diffusion operation on S real pictures that are clearly focused on the area of ​​interest, and a training sample set can be obtained, which can be used as input for subsequent reverse diffusion training.

[0097] Furthermore, the reverse diffusion process in the Diffusion Model training process is the noise removal inference process. Figure 6 , Figure 6 This is a schematic diagram of the reverse diffusion in the zoom method based on deep learning of this application. Traverse the Markov chain of forward diffusion backward, sample from q(xt-1|xt), convert the noise into samples of the source target distribution, and generate new image data x0'. The mathematical expression of the training conditional probability distribution can be as follows:

[0098]

[0099] By Bayesian formula , we can know that:

[0100]

[0101] Among them, the present application can choose to minimize the negative log-likelihood as the training loss Loss=Eq[-logpθ(x0)]. When the loss is minimized and converged, the model training is completed and the diffusion model of the present application is obtained.

[0102] After the operation that can trigger the refocusing of the lens of the image acquisition device is completed, for example, the pan / tilt of the ball camera changes from rotating to stationary, the tracking and zooming process ends, etc., the image acquisition device can be controlled to acquire the initial image according to the method of the above embodiment. and the target image , by inputting the two images into the diffusion model, the focus evaluation values ​​of the two images output by the diffusion model can be obtained.

[0103] It should be further explained that the execution subject of the zoom method based on deep learning may be a zoom device based on deep learning, for example, the zoom method based on deep learning may be executed by a terminal device or a server or other processing device, wherein the terminal device may be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the zoom method based on deep learning may be implemented by a processor calling computer-readable instructions stored in a memory.

[0104] Figure 7 FIG. 1 is a schematic diagram of a zoom device based on deep learning, shown in an exemplary embodiment of the present application. Figure 7 As shown, the exemplary zoom device 700 based on deep learning includes: a focus prediction module 710, a focus comparison module 720 and a zoom processing module 730. Specifically:

[0105] The focus prediction module 710 is used to input the target image acquired by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model.

[0106] The focus comparison module 720 is used to compare the target focus area in the target image with the predicted focus area in the focus prediction image to obtain a focus comparison result.

[0107] The zoom processing module 730 is used to perform zoom processing on the image acquisition device according to the focus comparison result.

[0108] In this exemplary zoom device based on deep learning, the target image captured by the image acquisition device is input into a pre-trained diffusion model, and the diffusion model performs diffusion processing on the received target image to obtain a focus prediction image output by the diffusion model; the target focus area in the target image is compared with the predicted focus area in the focus prediction image to obtain a focus comparison result, thereby determining whether the target area of ​​the target image captured by the image acquisition device based on the current zoom parameters matches the predicted focus area in the focus prediction image predicted by the diffusion model based on the focus comparison result; the image acquisition device is zoomed according to the focus comparison result, thereby achieving zooming of the image acquisition device based on deep learning technology and improving the zoom accuracy of the zoom process.

[0109] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept, wherein the specific manner in which each module and unit performs the operation has been described in detail in the method embodiment, and will not be repeated here. In practical applications, the device provided in the above embodiment can distribute the above functions to different functional modules as needed, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above, and this is not limited here.

[0110] Among them, the functions of each module can be found in the embodiment of the zoom method based on deep learning, which will not be repeated here.

[0111] See also Figure 8 , Figure 8 1 is a schematic diagram of the structure of an embodiment of an electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102, and the processor 102 is used to execute program instructions stored in the memory 101 to implement the steps in any of the above-mentioned zoom method embodiments based on deep learning. In a specific implementation scenario, the electronic device 100 may include but is not limited to: a microcomputer, a server, and in addition, the electronic device 100 may also include a mobile device such as a laptop computer and a tablet computer, which is not limited here.

[0112] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above-mentioned zoom method embodiments based on deep learning. The processor 102 can also be called a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 102 can be implemented by an integrated circuit chip.

[0113] In this exemplary electronic device, the target image captured by the image acquisition device is input into a pre-trained diffusion model, and the diffusion model performs diffusion processing on the received target image to obtain a focus prediction image output by the diffusion model; the target focus area in the target image is compared with the predicted focus area in the focus prediction image to obtain a focus comparison result, thereby determining whether the target area of ​​the target image captured by the image acquisition device based on the current zoom parameters matches the predicted focus area in the focus prediction image predicted by the diffusion model based on the focus comparison result; the image acquisition device is zoomed according to the focus comparison result, thereby enabling zooming of the image acquisition device based on deep learning technology to improve the zoom accuracy of the zoom process.

[0114] See also Fig. 9 , Fig. 9 The computer-readable storage medium 110 stores program instructions 111 that can be executed by a processor, and the program instructions 111 are used to implement the steps in any of the above-mentioned zoom method embodiments based on deep learning.

[0115] In this exemplary storage medium, the target image captured by the image acquisition device is input into a pre-trained diffusion model by running the program instructions in the storage medium, and the diffusion model performs diffusion processing on the received target image to obtain a focus prediction image output by the diffusion model; the target focus area in the target image is compared with the predicted focus area in the focus prediction image to obtain a focus comparison result, thereby determining whether the target area of ​​the target image captured by the image acquisition device based on the current zoom parameters matches the predicted focus area in the focus prediction image predicted by the diffusion model based on the focus comparison result; the image acquisition device is zoomed according to the focus comparison result, thereby enabling zooming of the image acquisition device based on deep learning technology to improve the zoom accuracy of the zoom process.

[0116] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0117] The above description of various embodiments tends to emphasize the differences between the various embodiments. The same or similar aspects can be referenced to each other, and for the sake of brevity, they will not be repeated herein.

[0118] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0119] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or part of the contribution to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

Claims

1. A zoom method based on deep learning, characterized in that: The method comprises: Inputting a target image acquired by an image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model; the target image is any one of a first image and a second image acquired by the image acquisition device, and there is a focus evaluation peak between the focus evaluation values ​​of the first image and the second image; Comparing a target focus area in the target image with a predicted focus area in the focus prediction image to obtain a focus comparison result; The step of comparing the target focus area in the target image with the predicted focus area in the predicted focus image to obtain a focus comparison result comprises: comparing position information of the target focus area with position information of the predicted focus area to obtain a position deviation between the target focus area and the predicted focus area, and determining the focus comparison result according to the position deviation; In response to the position deviation being greater than a preset deviation threshold, determining that the image acquisition device fails to focus, and obtaining the focus comparison result; Performing zoom processing on the image acquisition device according to the focus comparison result; The method comprises: performing zoom processing on the image acquisition device according to a preset zoom step length until there is another focus evaluation peak between the focus evaluation values ​​of two adjacent frames of images acquired by the image acquisition device during the zoom processing, and the focus evaluation difference between the focus evaluation values ​​of the two adjacent frames of images is less than a preset focus error threshold.

2. The method according to claim 1, characterized in that Before the step of inputting the target image acquired by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model, the method further includes: Performing focus evaluation processing on a first image and a second image captured by an image acquisition device respectively to obtain a first focus evaluation value corresponding to the first image and a second focus evaluation value corresponding to the second image; wherein the image acquisition focal lengths of the first image and the second image are different; Determining whether there is a focus evaluation peak between the first focus evaluation value and the second focus evaluation value according to the first focus evaluation value, the second focus evaluation value and a preset focus evaluation threshold; In response to the focus evaluation peak existing between the first focus evaluation value and the second focus evaluation value, the first image or the second image is determined as the target image.

3. The method according to claim 1, characterized in that Before the step of comparing the target focus area in the target image with the predicted focus area in the focus prediction image, the method further includes: A target focus area in the target image is determined according to the definition of the target image, and a predicted focus area in the predicted focus image is determined according to the definition of the predicted focus image.

4. The method according to claim 3, characterized in that The step of determining a target focus area in the target image according to the clarity of the target image, and determining a predicted focus area in the predicted focus image according to the clarity of the predicted focus image, comprises: Performing image segmentation processing on the target image and the focus prediction image respectively to obtain a plurality of target sub-images in the target image and a plurality of focus prediction sub-images in the focus prediction image; The target sub-image with the highest definition among the target sub-images is determined as the target focus area, and the predicted focus area is determined as the predicted focus area from the predicted focus sub-image with the highest definition among the predicted focus sub-images.

5. The method according to claim 1, characterized in that The step of determining the focus comparison result according to the position deviation comprises: In response to the position deviation being less than or equal to the preset deviation threshold, it is determined that the image acquisition device is focused successfully, and the focus comparison result is obtained.

6. The method according to claim 5, characterized in that The step of performing zoom processing on the image acquisition device according to the focus comparison result includes: In response to the focus comparison result indicating that the image acquisition device fails to focus, zoom processing is performed on the image acquisition device according to a preset zoom step length, and a third image and a fourth image are acquired by the image acquisition device after the zoom processing; wherein the image acquisition focal lengths of the third image and the fourth image are different; performing focus evaluation processing on the third image and the fourth image respectively to obtain a third focus evaluation value corresponding to the third image and a fourth focus evaluation value corresponding to the fourth image; In response to the presence of a peak value between the third focus evaluation value and the fourth focus evaluation value, reducing the preset zoom step to obtain a reduced zoom step; The image acquisition device is zoomed according to the reduced zoom step size until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is smaller than a preset focus error threshold.

7. The method according to claim 6, characterized in that The step of performing zoom processing on the image acquisition device according to the reduced zoom step length until the focus evaluation difference between the fifth image and the sixth image acquired by the image acquisition device is less than a preset focus error threshold comprises: Adjusting the image acquisition focal length of the image acquisition device according to the reduced zoom step length to obtain an adjusted image acquisition focal length; The image acquisition device is controlled to perform image acquisition according to the adjusted image acquisition focal length to obtain the fifth image and the sixth image; wherein the image acquisition focal lengths of the fifth image and the sixth image are different.

8. The method according to claim 7, characterized in that After the step of controlling the image acquisition device to acquire images according to the adjusted image acquisition focal length to obtain the fifth image and the sixth image, the method further includes: performing focus evaluation processing on the fifth image and the sixth image respectively to obtain a fifth focus evaluation value corresponding to the fifth image and a sixth focus evaluation value corresponding to the sixth image; determining the focus evaluation difference value according to the fifth focus evaluation value and the sixth focus evaluation value; In response to the focus evaluation difference being less than or equal to the focus error threshold, the zoom processing of the image acquisition device is paused or stopped.

9. The method according to claim 2, characterized in that: The step of performing focus evaluation processing on the first image and the second image respectively acquired by the image acquisition device to obtain a first focus evaluation value corresponding to the first image and a second focus evaluation value corresponding to the second image comprises: Acquire the grayscale value of the first image; The first focus evaluation value is obtained by performing calculation according to a preset focus evaluation function and the gray value of the first image.

10. The method according to claim 1, characterized in that Before the step of inputting the target image acquired by the image acquisition device into a pre-trained diffusion model to obtain a focus prediction image output by the diffusion model, the method further includes: Inputting the acquired sample image into the initial model to be trained, so that the initial model performs forward diffusion processing on the sample image to obtain a noise image; wherein the sample image is focused on the target area; Performing a back diffusion process on the noise image according to the initial model to obtain a denoised image and a model loss value corresponding to the denoised image; The initial model is trained according to the loss value to obtain the diffusion model.

11. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 10.

12. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images