Clarity detection method, device and computer storage medium

Abstract

The embodiment of the present application discloses a clarity detection method, device and computer storage medium, which belongs to the field of computer technology. The method includes: determining a second clarity evaluation value of a target image according to a determined first clarity evaluation value of the target image and the scene complexity. The embodiment of the present application converts the first clarity evaluation value of the target image into a second clarity evaluation value, so that the clarity evaluation values ​​in different scenes can be converted into clarity evaluation values ​​of the same standard system. In this way, it is possible to effectively avoid the phenomenon that the image acquisition device performs incorrect focusing processing on the captured clear image, thereby not causing shooting delays, and improving the efficiency of the user's use of the image acquisition device to capture images. In addition, when the target image is a frame image in a real-time video stream, if the scene changes, it is possible to avoid misfocusing or temporary out-of-focus phenomena in the real-time video stream, thereby not affecting attention or disturbing vision.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a clarity detection method, device and computer storage medium. Background Art

[0002] Clarity is the standard for judging whether an image is clear or blurry, and the clarity rating determines whether the image is clear or not. In an unchanged shooting scene, the captured image changes from blurry to clear, and then from clear to blurry. At this time, the clarity rating of the image will also change accordingly. If the clarity rating of the image reaches a certain position, the image may appear unclear, which will result in poor viewing experience for the user. Therefore, it is very important to perform clarity detection on the image captured by the image acquisition device.

[0003] In the related art, when an image acquisition device captures a scene, after the captured scene is stabilized, the clarity of the image of the current captured scene captured by the image acquisition device is detected, and focusing processing is automatically performed to make the captured image approach the clearest point.

[0004] In the above technology, the clarity evaluation value is not based on the same standard system for different shooting scenes. Every time the shooting scene of the image acquisition device changes, it needs to refocus after the shooting scene stabilizes. It does not consider whether the image captured by the current scene is clear. It only recognizes that this is a different shooting scene, and then the image acquisition device will focus. As a result, when the shooting scene is changed, the image acquisition device performs incorrect focusing on the clear image captured. This will not only cause delays in shooting, but also make users feel uncomfortable. Summary of the invention

[0005] The embodiments of the present application provide a method, device and computer storage medium for detecting clarity, which can avoid the phenomenon that the image acquisition device performs incorrect focusing processing on the captured clear image. The technical solution is as follows:

[0006] In one aspect, a method for detecting clarity is provided, the method comprising:

[0007] Determining a first clarity evaluation value of the target image to be detected based on a clarity evaluation operator;

[0008] Determining the scene complexity of the target image based on edge information in the target image, wherein the scene complexity indicates the complexity of the captured scene in the target image;

[0009] If the scene complexity indicates that the shooting scene in the target image is a complex scene, the first clarity evaluation value is adjusted to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value; or, if the scene complexity indicates that the shooting scene in the target image is a simple scene, the first clarity evaluation value is adjusted to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value.

[0010] Optionally, the target image is an image currently captured by an image capture device;

[0011] The method further comprises:

[0012] If the image is determined to be a clear image based on the clarity indicated by the second clarity evaluation value, the focus of the image acquisition device is not adjusted; if the target image is determined to be a blurred image based on the clarity indicated by the second clarity evaluation value, the focus of the image acquisition device is adjusted.

[0013] Optionally, the image acquisition device is an endoscope system, the endoscope system includes an endoscope, and the endoscope is used to be inserted into human tissue;

[0014] The target image is an image captured by the image acquisition device when the endoscope is inserted into human tissue.

[0015] Optionally, determining the scene complexity of the target image based on edge information in the target image includes:

[0016] Acquire an edge image corresponding to the target image, where the edge image indicates edge information of a shooting scene in the target image;

[0017] Performing a defocus blur process on the edge in the edge image based on a reference radius to obtain a critical blurred image;

[0018] The scene complexity of the target image is determined based on the image two-dimensional entropy of the critical blurred image, wherein the image two-dimensional entropy indicates a distribution feature of pixel values ​​in the critical blurred image.

[0019] Optionally, determining the scene complexity of the target image based on the two-dimensional image entropy of the critical blurred image includes:

[0020] Performing a scale transformation on the critical fuzzy image to obtain a plurality of scale images corresponding to the critical fuzzy image, wherein the scales of the plurality of scale images are different from the scale of the critical fuzzy image;

[0021] Determining the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images;

[0022] The image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images are weighted and fused to obtain the scene complexity of the target image.

[0023] Optionally, the edges in the edge image include first-type edges and second-type edges, and an average edge width of the first-type edges is less than or equal to an average edge width of the second-type edges;

[0024] The performing defocus blur processing on the edge in the edge image based on the reference radius comprises:

[0025] For a first edge in the edge image, a defocus blur process is performed using a first reference radius, where the first edge refers to an edge that belongs to the first type of edge but does not belong to the second type of edge;

[0026] For a second edge in the edge image, a defocus blur process is performed using a second reference radius, where the second edge refers to an edge that belongs to the second type of edge but does not belong to the first type of edge;

[0027] For a third edge in the edge image, a third reference radius is used to perform defocus blur processing, wherein the third edge refers to an edge belonging to both the first type of edge and the second type of edge;

[0028] For other parts of the edge image except the first type of edges and the second type of edges, a fourth reference radius is used to perform defocus blur processing;

[0029] Among them, the first reference radius is greater than or equal to the third reference radius, the third reference radius is greater than or equal to the second reference radius, and the second reference radius is greater than or equal to the fourth reference radius.

[0030] Optionally, before acquiring the edge image corresponding to the target image, the method further includes:

[0031] Eliminating invalid information in the target image, wherein the invalid information is pixels in a non-viewing range and / or pixels in an overexposed area in the target image;

[0032] The step of obtaining an edge image corresponding to the target image includes:

[0033] An edge image in the target image after the invalid information is removed is obtained.

[0034] Optionally, before acquiring the edge image corresponding to the target image, the method further includes:

[0035] Using a low-pass filter to filter the target image to obtain low-frequency information in the target image;

[0036] removing the low-frequency information from the target image;

[0037] The step of obtaining an edge image corresponding to the target image includes:

[0038] An edge image in the target image is obtained after the low-frequency information is removed.

[0039] Optionally, a difference between the second clarity evaluation value and the first clarity evaluation value is related to a complexity level indicated by the scene complexity indicator.

[0040] Optionally, before determining the scene complexity of the target image based on edge information in the target image, the method further includes:

[0041] If the target image is determined to be a non-deeply out-of-focus image based on the clarity level indicated by the first clarity evaluation value, determining the scene complexity of the target image based on edge information in the target image is performed.

[0042] On the other hand, a clarity detection device is provided, the device comprising:

[0043] A determination module, used to determine a first clarity evaluation value of the target image to be detected based on a clarity evaluation operator;

[0044] The determination module is used to determine the scene complexity of the target image based on the edge information in the target image, wherein the scene complexity indicates the complexity of the shooting scene in the target image;

[0045] A judgment module, configured to adjust the first clarity evaluation value to obtain a second clarity evaluation value if the scene complexity indicates that the shooting scene in the target image is a complex scene, and the clarity level indicated by the second clarity evaluation value is lower than the clarity level indicated by the first clarity evaluation value; or, if the scene complexity indicates that the shooting scene in the target image is a simple scene, adjust the first clarity evaluation value to obtain a second clarity evaluation value, and the clarity level indicated by the second clarity evaluation value is higher than the clarity level indicated by the first clarity evaluation value.

[0046] Optionally, the target image is an image currently captured by an image capture device;

[0047] The device also includes:

[0048] An adjustment module is configured to: not adjust the focal length of the image acquisition device if the target image is determined to be a clear image based on the clarity indicated by the second clarity evaluation value, and to adjust the focal length of the image acquisition device if the target image is determined to be a blurred image based on the clarity indicated by the second clarity evaluation value.

[0049] Optionally, the image acquisition device is an endoscope system, the endoscope system includes an endoscope, and the endoscope is used to be inserted into human tissue;

[0050] The target image is an image captured by the image acquisition device when the endoscope is inserted into human tissue.

[0051] Optionally, the determining module includes:

[0052] A first acquisition unit, configured to acquire an edge image corresponding to the target image, wherein the edge image indicates edge information of a shooting scene in the target image;

[0053] A first determining unit, configured to perform a defocus blur process on the edge in the edge image based on a reference radius to obtain a critical blurred image;

[0054] The second determining unit is used to determine the scene complexity of the target image based on the image two-dimensional entropy of the critical blurred image, and the image two-dimensional entropy indicates a distribution feature of pixel values ​​in the critical blurred image.

[0055] Optionally, the second determining unit includes:

[0056] A determination unit, configured to scale the critical fuzzy image to obtain a plurality of scale images corresponding to the critical fuzzy image, wherein the scales of the plurality of scale images are different from the scale of the critical fuzzy image;

[0057] The determining unit is used to determine the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images;

[0058] The determination unit is used to weightedly fuse the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images to obtain the scene complexity of the target image.

[0059] Optionally, the edges in the edge image include first-type edges and second-type edges, and an average edge width of the first-type edges is less than or equal to an average edge width of the second-type edges;

[0060] The first determining unit includes:

[0061] a processing unit, configured to perform defocus blur processing on a first edge in the edge image by using a first reference radius, wherein the first edge refers to an edge that belongs to the first type of edge but does not belong to the second type of edge;

[0062] The processing unit is configured to perform defocus blur processing on a second edge in the edge image by using a second reference radius, where the second edge refers to an edge that belongs to the second type of edge but does not belong to the first type of edge;

[0063] The processing unit is configured to perform defocus blur processing on a third edge in the edge image by using a third reference radius, wherein the third edge refers to an edge belonging to both the first type of edge and the second type of edge;

[0064] The processing unit is configured to perform defocus blur processing on other parts of the edge image except the first-type edge and the second-type edge by using a fourth reference radius;

[0065] Among them, the first reference radius is greater than or equal to the third reference radius, the third reference radius is greater than or equal to the second reference radius, and the second reference radius is greater than or equal to the fourth reference radius.

[0066] Optionally, the determining module further includes:

[0067] A rejection unit, used to reject invalid information in the target image, wherein the invalid information is pixels in a non-viewing range and / or pixels in an overexposed area in the target image;

[0068] The first acquisition unit is used for:

[0069] An edge image in the target image after the invalid information is removed is obtained.

[0070] Optionally, the determining module further includes:

[0071] A filtering unit, configured to filter the target image using a low-pass filter to obtain low-frequency information in the target image;

[0072] removing the low-frequency information from the target image;

[0073] The first acquisition unit is used to acquire an edge image in the target image after the low-frequency information is removed.

[0074] Optionally, a difference between the second clarity evaluation value and the first clarity evaluation value is related to a complexity level indicated by the scene complexity indicator.

[0075] Optionally, the determining module is further used for:

[0076] If the target image is determined to be a non-deeply out-of-focus image based on the clarity level indicated by the first clarity evaluation value, determining the scene complexity of the target image based on edge information in the target image is performed.

[0077] In a third aspect, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a processor, a clarity detection method described in the first aspect is implemented.

[0078] According to a fourth aspect, a computer device is provided, the device comprising:

[0079] processor;

[0080] a memory for storing processor-executable instructions;

[0081] Wherein, the processor is configured to execute a clarity detection method described in the above aspect.

[0082] In a fifth aspect, a computer program product comprising instructions is provided, which, when executed on a computer, enables the computer to execute the clarity detection method described in the first aspect.

[0083] The beneficial effects brought by the technical solution provided by the embodiment of the present application include at least:

[0084] The second clarity evaluation value of the target image is determined by the first clarity evaluation value of the target image and the scene complexity. Since the clarity evaluation value can judge the clarity and blurriness of the image, the second clarity evaluation value is used to further determine whether the image is clear or not. If the result of the determination is that the image is not clear, in order to make the image clear, the method of adjusting the focal length of the image acquisition device can be used to make the image clear. Since the second clarity evaluation value is related to the scene complexity, and when the scene complexity indicates that the shooting scene in the target image is a complex scene, the clarity indicated by the adjusted second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value, and when the scene complexity indicates that the shooting scene in the target image is a simple scene, the clarity indicated by the adjusted second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value, so that the clarity evaluation values ​​under scenes of different complexities can be converted into clarity evaluation values ​​of the same standard system. In this way, each time the image acquisition device changes the shooting scene, it only needs to judge whether the image acquisition device needs to adjust the focus based on the second clarity evaluation value, so that the situation where the shooting scene switches but the focus does not need to be adjusted can be correctly handled, instead of having to readjust the focus every time the shooting scene is changed. Therefore, the phenomenon that the image acquisition device performs incorrect focusing processing on the clear image can be effectively avoided, so that the shooting delay will not be caused, and the efficiency of the user using the image acquisition device to capture images is improved. In addition, when the target image is a frame image in a real-time video stream, if the scene changes, the method of the embodiment of the present application can prevent refocusing every time, so that the real-time video stream can be avoided from being misfocused or temporarily out of focus. In this way, when the real-time video stream is a video stream shot by an endoscope, the method of the embodiment of the present application can prevent the doctor from being distracted or disturbed by the video misfocusing or temporary out of focus when watching the real-time video. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0086] Figure 1 It is a schematic diagram of the architecture of a clarity detection system provided in an embodiment of the present application.

[0087] Figure 2 This is a flow chart of a clarity detection method provided in an embodiment of the present application.

[0088] Figure 3This is a flow chart for determining scene stability provided by an embodiment of the present application.

[0089] Figure 4 This is a flowchart for determining scene complexity provided by an embodiment of the present application.

[0090] Figure 5 It is a specific flow chart of a clarity detection method provided in an embodiment of the present application.

[0091] Figure 6 It is a structural schematic diagram of a clarity detection device provided in an embodiment of the present application.

[0092] Figure 7 It is a structural block diagram of a terminal provided in an embodiment of the present application.

[0093] Figure 8 It is a schematic diagram of a server structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0094] In order to make the objectives, technical solutions and advantages of the embodiments of the present application clearer, the implementation methods of the present application will be further described in detail below in conjunction with the accompanying drawings.

[0095] For the convenience of subsequent explanation, the application scenarios of the embodiments of the present application are first introduced and explained here.

[0096] Whether the image is clear affects the user's perception of the image. The size of the clarity rating value directly reflects the clarity of the image and determines whether the image is clear. The clarity rating value and the clarity of the image indicated by the clarity rating value are divided into the following two situations: if the clarity rating value and the clarity of the image indicated by the clarity rating value are positively correlated, it means that the larger the clarity rating value, the clearer the image. If the clarity rating value and the clarity of the image indicated by the clarity rating value are negatively correlated, it means that the smaller the clarity rating value, the clearer the image.

[0097] It is very important to detect the clarity of the image in different shooting scenes. For two images in different shooting scenes, when the user feels that the clarity of the two images is the same, the clarity evaluation values ​​of the two images may be different. For example, there are two shooting scenes, one is a complex shooting scene (such as a colorful painting) and the other is a simple shooting scene (such as a white ceiling). When the images captured by the two shooting scenes are very clear, if the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value show a positive correlation, the clarity evaluation value of the image captured by the complex shooting scene is greater than the clarity evaluation value of the image captured by the other simple shooting scene. Or, when the images captured by the two shooting scenes are very clear, if the clarity evaluation value and the indicated clarity show a negative correlation, the clarity evaluation value of the image captured by the complex shooting scene is less than the clarity evaluation value of the image captured by the other simple shooting scene. In other words, regardless of whether the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value show a positive correlation or a negative correlation, the clarity evaluation values ​​of complex scenes and simple scenes are not in the same standard system.

[0098] Based on the above principle, when the current simple shooting scene of the image acquisition device changes to other complex shooting scenes, and the image captured by the current simple shooting scene is clear, since the image acquisition device will determine the clarity evaluation value of the image captured by the simple shooting scene as the clarity evaluation value of the image captured by the other complex shooting scene, at this time, even if the user feels that the image captured by the other complex shooting scene is clear, the image acquisition device will judge the image captured by the other complex shooting scene as unclear. In the case where it is judged that the image captured by the other complex shooting scene is not clear, the other complex shooting scene will be refocused until the image captured by the other complex shooting scene is clear. That is, each time the image acquisition device changes the shooting scene, since the clarity evaluation values ​​under different shooting scenes are not of the same standard system, the image acquisition device needs to refocus each time after changing the shooting scene until a clear image under the shooting scene is obtained. In addition, when the target image is a frame image in a real-time video stream, since the picture in the video is constantly changing, that is, each frame image obtained may be a different scene. Therefore, each frame of the video stream may be refocused, and the real-time video stream may be misfocused or temporarily out of focus, which may affect attention or disturb vision, leading to some serious accidents. For example, in the medical field, when a doctor is performing surgery on a patient, he or she views the situation inside the human body through a display screen. At this time, if the scene switches and the wrong focus is performed, if it happens to be the key point of the surgery, it may affect the doctor's attention and lead to misjudgment.

[0099] The method provided in the embodiment of the present application is applied to a shooting scene in which the clarity of an image is detected when the shooting scene of the image acquisition device changes. Optionally, the clarity detection method provided in the embodiment of the present application can also be applied to other shooting scenes that require clarity detection. Examples will not be given one by one here.

[0100] In order to implement a clarity detection method, the present application embodiment provides a clarity detection system. For the convenience of subsequent description, the clarity detection system is first explained in detail.

[0101] Figure 1 Schematic diagram of the structure of a clarity detection system provided in an embodiment of the present application. Figure 1 As shown, the clarity detection system 100 includes a scene stability detection module 101 , a clarity evaluation value calculation module 102 , a scene complexity calculation module 103 , a clarity evaluation value correction module 104 , and a focus control module 105 .

[0102] The scene stability detection module is used to detect whether the current shooting scene is stable, and send the image of the stabilized shooting scene to the clarity evaluation value calculation module.

[0103] The clarity evaluation value calculation module is used to receive the stabilized image of the shooting scene sent by the scene stability detection module, determine the first clarity evaluation value of the received image, and then send the determined first clarity evaluation value of the image to the scene complexity calculation module.

[0104] The scene complexity calculation module is used to receive the first clarity evaluation value of the image sent by the clarity evaluation value calculation module, determine the shooting scene complexity of the image according to the first clarity evaluation value, and send the determined shooting scene complexity to the clarity evaluation value correction module.

[0105] The clarity evaluation value correction module is used to receive the shooting scene complexity determined by the scene complexity calculation module, and correct the first clarity evaluation value based on the shooting scene complexity to further determine the second clarity evaluation value. If the second clarity evaluation value is lower than the first clarity threshold, a focus adjustment instruction is sent to the focus control module.

[0106] The focus control module is used to receive the focus instruction sent by the clarity evaluation value correction module, and the focus control module responds to the instruction and adjusts the focal length of the image acquisition device.

[0107] The functions of the above modules will be described in detail in the subsequent method embodiments and will not be repeated here.

[0108] In the embodiment of the present application, the clarity detection system performs clarity detection based on the image or video captured by the image acquisition device. The image acquisition device includes a camera and other hardware parts. The camera can be a medical endoscope, a surveillance camera, a mobile phone camera, etc. The clarity detection system detects the clarity of the image captured by the camera, and determines whether the image acquisition device needs to focus according to the clarity evaluation value, and then presents a clear image to the user.

[0109] It should be noted that Figure 1 The clarity detection system shown can be centrally deployed in a terminal or in a server. Optionally, each module in the clarity detection system can also be distributed and deployed on different devices, which is not limited in the embodiments of the present application.

[0110] also, Figure 1 Each module of the medium definition detection system is a software module, and the naming of each module is based on the function of the software module. When applying the embodiment of the present application, different names can be given based on needs. For example, the scene stability detection module can be named the first module, the definition evaluation value calculation module can be named the second module, the scene complexity calculation module can be named the third module, the definition evaluation value correction module can be named the fourth module, and the focus control module can be named the fifth module. The embodiment of the present application does not limit the naming of the above modules.

[0111] based on Figure 1 The clarity detection system shown in the figure is further described below with respect to the method provided in the embodiment of the present application. Figure 1 It can be seen from the clarity detection system shown that the execution subject of the method is not limited. For the convenience of subsequent description, the following embodiment is described by taking the clarity detection system centrally deployed on the terminal as an example.

[0112] Figure 2 : is a flow chart of a clarity detection method provided in an embodiment of the present application. The clarity detection method may include the following steps:

[0113] Step 201: The terminal determines a first clarity evaluation value of a target image to be detected based on a clarity evaluation operator.

[0114] In order to determine whether the target image to be detected is clear, and then determine whether to adjust the focus of the image acquisition device, it is necessary to use Figure 1 The medium definition evaluation value calculation module detects the definition of the target image to be detected, and specifically determines a first definition evaluation value based on the definition evaluation operator. The target image may be an image currently captured by the image capture device.

[0115] It should be noted that, in the embodiment of the present application, the image acquisition device may be an endoscope system, the endoscope system includes an endoscope, the endoscope is used to be inserted into human tissue, and the target image is an image acquired by the image acquisition device when the endoscope is inserted into human tissue. Therefore, the method provided in the embodiment of the present application is mainly used in the scenario of examining a patient using an endoscope system. The methods described below are all used in the scenario of examining a patient using an endoscope system, and will not be explained again. In addition, in the embodiment of the present application, the name of the endoscope is not limited. When examining the abdomen of a patient, the endoscope is called a laparoscope, and when examining the stomach of a patient, the endoscope is called a gastroscope.

[0116] The clarity evaluation operator calculates the relevant information of the image, and the size of the first clarity evaluation value obtained by the calculation directly corresponds to whether the image is clear or not, thereby providing a basis for judging whether to adjust the focal length of the image acquisition device.

[0117] The above-mentioned method for determining the first clarity evaluation value based on the clarity evaluation operator is as follows: the clarity evaluation operator may be a Brenner (a gradient evaluation function) function, and the difference between the pixel values ​​of two adjacent pixels in the target image is squared according to the Brenner function, and then all the squared values ​​are added to obtain the first clarity evaluation value. The pixel value of the pixel point may be represented by a grayscale value, which is not limited here and will not be described in detail below.

[0118] It should be noted that the clarity evaluation operator can also be a grayscale gradient operator, a Tenengrad (an image clarity evaluation function) function, a Laplacian (Laplace) operator, a Variance (variance) function, a Kirsch (a new edge detection algorithm) operator, etc., and the clarity evaluation operator is not limited here. Determining the first clarity evaluation value of the target image according to the clarity evaluation operators such as the grayscale gradient operator, the Tenengrad function, the Laplacian operator, the Variance function, the Kirsch operator, etc. is already well known to those skilled in the art, and will not be elaborated in this embodiment.

[0119] In addition, since the user determines whether the image is clear based on the stability of the shooting scene, the first clarity evaluation value of the image needs to be determined under the condition that the shooting scene is stable, so that the first clarity evaluation value determined is meaningful. Specifically, the terminal determines the stability of the current shooting scene, and determines the first clarity evaluation value of the target image based on the stable shooting scene.

[0120] The above terminal determines the stability of the shooting scene by using Figure 1 The scene stability detection module in the image acquisition device detects whether the shooting scene of the target image currently being shot is stable.

[0121] Among them, the scene stability detection module may include an automatic white balance stability detection module, an automatic exposure stability detection module, an image system parameter stability detection module, a status information summary module, and a historical information judgment module. Specifically, the terminal detects whether the automatic white balance of the current shooting scene has reached a stable state according to the automatic white balance stability detection module, detects whether the automatic exposure of the current shooting scene has reached a stable state according to the automatic exposure stability detection module, and detects whether the image system parameters of the current shooting scene have reached a stable state according to the image system parameter stability detection module. After the status information summary module determines that the automatic white balance, automatic exposure and image system parameters of the current shooting scene have all reached a stable state, the historical information judgment module determines the stable state of the current shooting scene using the historical stability information of the current shooting scene.

[0122] When the camera of the image acquisition device moves significantly and causes the shooting scene to change, the color temperature of the image captured by the image acquisition device will change greatly. Therefore, the above-mentioned terminal detects whether the automatic white balance of the current shooting scene has reached a stable state according to the automatic white balance stability detection module as follows: the automatic white balance stability detection module determines whether the automatic white balance has reached a stable state according to the current color temperature of the current shooting scene. If the absolute value of the difference between the current color temperature and the target color temperature is less than the color temperature threshold, it means that the automatic white balance has reached a stable state. If the absolute value of the difference between the current color temperature and the target color temperature is greater than the color temperature threshold, it means that the automatic white balance has not reached a stable state. Among them, the target color temperature is the color temperature set in advance. For example, the white balance set by the user is daylight, and the color temperature of daylight is 5000-5500k, then 5000-5500k is the target color temperature. If the color temperature threshold is 100K, when the image acquisition device is automatically adjusted, if the absolute value of the difference between the color temperature of the current shooting scene and the target color temperature is less than 100K, that is, the color temperature of the current shooting scene is between 4900-5600K, it means that the automatic white balance has reached a stable state.

[0123] When the automatic exposure is adjusting the brightness significantly, the exposure time of the image captured by the image acquisition device will change greatly. Therefore, the above-mentioned automatic exposure stability detection module detects whether the automatic exposure of the current shooting scene has reached a stable state in the following way: the automatic exposure stability detection module determines whether the automatic exposure has reached a stable state according to the current brightness of the current shooting scene. If the absolute value of the difference between the current brightness and the target brightness is less than the brightness threshold, the automatic exposure has reached a stable state. If the absolute value of the difference between the current brightness and the target brightness is greater than the brightness threshold, the automatic exposure has not reached a stable state. Among them, the target brightness is the brightness expected by the user, and the target brightness is related to the aperture, sensitivity, gain and other parameters of the image acquisition device.

[0124] When the image system parameters are adjusted, the various system parameters of the image system will change greatly. Therefore, the above-mentioned implementation method of detecting whether the image system parameters of the current shooting scene have reached a stable state according to the image system parameter stability detection module is as follows: the image system parameter stability detection module determines whether the image system parameters have reached a stable state according to the current image system parameters of the current shooting scene. If the current image system parameters are the same as the target image system parameters, the image system parameters have reached a stable state. If the current image system parameters are different from the target image system parameters, the image system parameters have not reached a stable state. Among them, the image system parameters include noise reduction level, sharpening level, contrast, saturation and advanced algorithms such as image defogging, dark area improvement, fluorescence imaging, etc., and the target image system parameters are the set image system parameters.

[0125] The above-mentioned historical information judgment module uses the historical stability information of the current shooting scene to determine the stable state of the current shooting scene in the following manner: the historical information judgment module will store the relevant information of each frame of the historically collected image, and these relevant information include white balance parameters, automatic exposure time, and image system parameters, etc. If these relevant information of the continuous n frames of images traced back from the current frame are all within a range, it is determined that the shooting scene corresponding to the n frames of images is in a stable state. Therefore, the image frame number threshold of the continuous stable state can be pre-set to T, and only when the aforementioned n>T, the shooting scene of the current frame will be determined to be in a stable state. For example, T is 60 frames, and only when the continuous 60 frames of images before the current frame are in a stable state can it be determined that the current shooting scene is in a stable state.

[0126] like Figure 3 As shown, Figure 3This is a flow chart for determining scene stability provided by an embodiment of the present application. After determining the target image, the image acquisition device starts the scene stability detection model, and detects whether the automatic white balance has reached a stable state according to the automatic white balance stability detection module in the scene stability detection model. If it has reached a stable state, the automatic white balance stability state information is transmitted to the state information summary module, otherwise, the automatic white balance continues to be detected. The automatic exposure stability detection module detects whether the automatic exposure has reached a stable state. If it has reached a stable state, the automatic exposure stability state information is transmitted to the state information summary module, otherwise, the automatic exposure continues to be detected. The image system parameter detection module detects whether the image system parameter has reached a stable state. If it has reached a stable state, the image system parameter stability state information is transmitted to the state information summary module, otherwise, the image system parameter continues to be detected. When the state information summary module determines whether the automatic white balance, automatic exposure and image system parameters have all reached a stable state, if so, the relevant information of the image of the current frame after all reaching a stable state is transmitted to the historical information judgment module, if not, the state information summary module continues to determine whether the automatic white balance, automatic exposure and image system parameters have all reached a stable state. When the historical information judgment module receives the information that the automatic white balance, automatic exposure and image system parameters have reached a stable state, it determines whether the relevant information of the images of the continuous T frames traced back from the current frame is within a range. If they are all within a range, the shooting scene stable state information is output; if they are not within a range, the historical information judgment module continues to use the relevant information of the images of the continuous T frames traced back from the current frame to determine whether they are all within a range.

[0127] Step 202: The terminal determines the scene complexity of the target image based on edge information in the target image.

[0128] The edge information in the image can reflect the texture features of the image, and the scene complexity of the image can be determined according to the complexity of the texture features of the image. Therefore, in the embodiment of the present application, the terminal first obtains the edge information in the target image, and then determines the scene complexity of the target image based on the edge information in the target image.

[0129] In a possible implementation, the terminal determines the scene complexity of the target image based on the edge information in the target image in the following manner: the terminal obtains an edge image corresponding to the target image, wherein the edge image indicates edge information of the shooting scene in the target image. The terminal performs defocus blur processing on the edge in the edge image based on a reference radius to obtain a critical blurred image. The terminal determines the scene complexity of the target image based on the image two-dimensional entropy of the critical blurred image, wherein the image two-dimensional entropy indicates the pixel value distribution characteristics in the critical blurred image.

[0130] The implementation method of the terminal obtaining the edge image corresponding to the target image is as follows: the edge image of the target image includes the first type of edge and the second type of edge of the target image. The terminal obtains the first type of edge and the second type of edge in the target image, and determines the edge image corresponding to the target image based on the first type of edge and the second type of edge in the target image. The average edge width of the first type of edge is less than or equal to the average edge width of the second type of edge. That is, the first type of edge represents a relatively thin edge in the target image, and the second type of edge represents a relatively thick edge in the target image.

[0131] The terminal obtains the first type of edge and the second type of edge in the target image in the following manner: the terminal determines the first type of edge in the target image according to the Canny (an edge detection algorithm) algorithm. The terminal determines the second type of edge in the target image according to the Sobel (Sobel algorithm) algorithm.

[0132] The terminal determines the first type of edge in the target image according to the Canny algorithm specifically as follows: Since there is noise in the target image, the noise is a pixel point where the grayscale value of the image changes greatly, and the noise is easily identified as a pseudo edge, so the terminal performs noise reduction processing on the target image to remove the noise and obtain the target image after the noise is removed. The gradient of the image represents the place where the grayscale changes significantly. The terminal determines the gradient of the target image after the noise is removed, determines the gradient of the target image, and obtains the possible edge. The terminal performs non-maximum suppression on the target image, retains the pixel points with the largest grayscale change, and does not retain the others, so that a large part of the pixel points can be eliminated. After non-maximum suppression, there are still many possible edge points that have not been detected, and a double threshold is further set, namely a low threshold and a high threshold. Pixels with a grayscale change greater than the high threshold are set as strong edge pixels, that is, obvious edges. Pixels with a grayscale change less than the low threshold are less obvious edges and are eliminated. Those between the low threshold and the high threshold are set as weak edges. Further judgment, if there are strong edge pixels in the area of ​​the target image, they are retained, if not, they are eliminated. The edges retained in this way are the first-class edges of the target image.

[0133] The above terminal determines the second type of edge in the target image according to the Sobel algorithm as follows: the terminal determines the gradient of each pixel in the target image according to the Sobel template, and if the gradient of any pixel is greater than the edge threshold, the pixel is considered to be an edge pixel. All the determined edge pixels are connected, so that the second type of edge of the target image is determined.

[0134] It should be noted that the use of the Canny algorithm to determine the first type of edge in the target image is only an optional method provided by the embodiment of the present application. Other methods may be used to obtain the first type of edge in the target image, such as the Laplacian operator, etc. The embodiment of the present application does not limit how to obtain the first type of edge in the target image. Similarly, the use of the Sobel algorithm to determine the second type of edge in the target image is only an optional method provided by the embodiment of the present application. Other methods may be used to obtain the second type of edge in the target image, such as the Prewitt (edge ​​detection of a first-order differential operator) operator, etc. The embodiment of the present application does not limit how to obtain the second type of edge in the target image.

[0135] The above edge image can be divided into four parts, namely, the first edge, the second edge, the third edge and other parts of the edge image except the first type edge and the second type edge. Among them, the first edge refers to an edge that belongs to the first type edge but not to the second type edge. The second edge refers to an edge that belongs to the second type edge but not to the first type edge. The third edge refers to an edge that belongs to both the first type edge and the second type edge.

[0136] Therefore, in step 202, the terminal performs defocus blur processing on the edge in the edge image based on the reference radius, and the implementation method of obtaining the critical blurred image can be: the terminal performs defocus blur processing on different parts in the edge image based on different reference radii, so that the blur degree of all parts in the edge image reaches the same level, that is, the target image is adjusted to a blurred image. The blurred image is a critical blurred image where the clear image reaches the blurred image.

[0137] Specifically, for the first edge in the edge image, the first reference radius is used for defocus blur processing, and the first reference radius is denoted as r1. For the second edge in the edge image, the second reference radius is used for defocus blur processing, and the second reference radius is denoted as r2. For the third edge in the edge image, the third reference radius is used for defocus blur processing, and the third reference radius is denoted as r3. For other parts of the edge image except the first and second edges, the fourth reference radius is used for defocus blur processing, and the fourth reference radius is denoted as r4. Among them, the first reference radius is greater than or equal to the third reference radius, the third reference radius is greater than or equal to the second reference radius, and the second reference radius is greater than or equal to the fourth reference radius. In this way, the four parts that have been defocused and blurred according to different reference radii constitute the critical blurred image.

[0138] It should be noted that the larger the reference radius, the larger the blur span of the defocus blur processing. Since all parts in the edge image are blurred to the same level, the smaller the edge, the higher the clarity, so the larger the reference radius is selected, and the larger the edge, the lower the clarity, so the smaller the reference radius is selected. Therefore, the above r1>r3>r2>r4.

[0139] The above-mentioned terminal implements defocus blur processing on different parts of the edge image based on different reference radii as follows: for each pixel point in different parts of the edge image, any pixel point is taken as the center pixel point, and the reference radius corresponding to the corresponding part where the pixel point is located is taken as the radius, and all the neighboring pixel points within the radius range of the center pixel point are weighted superimposed to obtain the pixel value of the pixel point. Among them, the weights of all neighboring pixel points within the radius range are determined according to the distance from the center pixel point. The closer to the center pixel point, the greater the weight. Specifically, the pixel value of any pixel point after defocus blur processing can be determined according to the formula for defocus blur processing. As shown in Formula 1.

[0140] Formula 1:

[0141]

[0142] In formula 1, r ranges from 1 to 4, α r is the image smoothing adjustment parameter, f r is the filter mask of the corresponding radius. I(i, j) is the pixel value of the pixel at the i-th row and j-th column.

[0143] Since the above critical fuzzy image can only reflect the structural information of the target image, but the basis for determining the scene complexity of the image is the information of the grayscale distribution spatial characteristics of the pixels. The two-dimensional entropy of the image can reflect the amount of information of the grayscale distribution spatial characteristics. Therefore, the two-dimensional entropy of the above critical fuzzy image needs to be determined. Among them, the structural information of the target image indicates the fuzzy area where the object appears in the target image. For example, if there is a tree in the target image, the structural information is the approximate outline of the tree.

[0144] In step 202, the terminal determines the two-dimensional image entropy of the critical blurred image in the following manner: the terminal determines the two-dimensional histogram of the critical blurred image according to Formula 2, as shown in Formula 2. Then, the two-dimensional image entropy of the critical blurred image is determined according to the two-dimensional histogram of the critical blurred image.

[0145] Formula 2:

[0146] P i,j =S i,j / N

[0147] Where i represents the gray value of the image, and j represents the average gray value of the neighborhood within the reference radius. i,j Indicates the ratio of the number of times a pixel appears in all valid pixels, S i,j Indicates the number of times a pixel appears among all valid pixels, the total number of valid pixels in N images.

[0148] The above-mentioned method of determining the image two-dimensional entropy of the critical blurred image according to the two-dimensional histogram of the critical blurred image is as follows: As shown in Formula 3, in Formula 3, for P i,j Taking the logarithm, H represents the image two-dimensional entropy of the critical blurred image.

[0149] Formula 3:

[0150]

[0151] The two-dimensional image entropy of the critical blurred image can reflect the comprehensive characteristics of the grayscale information of the pixel position in the critical blurred image and the grayscale distribution in the pixel neighborhood.

[0152] In a possible implementation, the terminal determines the scene complexity of the target image based on the two-dimensional image entropy of the critical blurred image in the following manner: the terminal determines H as the scene complexity of the target image. That is, the two-dimensional image entropy of the critical blurred image is determined as the scene complexity of the target image.

[0153] In another possible implementation, the terminal determines the scene complexity of the target image based on the edge information in the target image in the following manner: in order to make the scene complexity of the target image more accurate, the critical fuzzy image is scaled, and different scales reflect different image structures. The smaller the scale, the more it can express large and general structures, and the larger the scale, the more it can express small and precise structures. The scale refers to the size of the image. For example, 480p (Progressive, line by line), 720p, 1080p.

[0154] Specifically, after the terminal determines the image two-dimensional entropy of the critical fuzzy image, the critical fuzzy image is scaled to obtain multiple scale images corresponding to the critical fuzzy image, wherein the scales of the multiple scale images are different from the scale of the critical fuzzy image. The terminal determines the image two-dimensional entropy of the critical fuzzy image and the image two-dimensional entropy of each scale image in the multiple scale images. The terminal weightedly fuses the image two-dimensional entropy of the critical fuzzy image and the image two-dimensional entropy of each scale image in the multiple scale images to obtain the scene complexity of the target image.

[0155] The above-mentioned method of obtaining multiple scale images corresponding to the critical fuzzy image can be implemented by: reducing the critical fuzzy image to multiple different scales by downsampling. Specifically, one pixel is extracted every M pixels in all pixels of the critical fuzzy image, and the extracted pixel is removed, and finally an image with fewer pixels than the critical fuzzy image is obtained. At this time, the image with the pixel removed is an image of another scale. The terminal obtains multiple scale images using this method.

[0156] In addition, other methods may be used to obtain multiple scale images corresponding to the critical blurred image, for example, through "bilinear downsampling", "Gaussian convolution downsampling", "frequency domain downsampling", etc., which are not limited in the embodiments of the present application.

[0157] After the terminal obtains multiple scale images corresponding to the critical fuzzy image, the image two-dimensional entropy of the different scale images is determined. The method is the same as the method of determining the image two-dimensional entropy of the critical fuzzy image by the above terminal, which will not be repeated here.

[0158] The above terminal weights and fuses the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images to obtain the scene complexity of the target image in the following manner: the image two-dimensional entropy at each scale is recorded as H k If the number of multiple scale images is M, then as shown in Formula 4, β k is the weight coefficient at the kth scale. The larger the average pixel value of all pixels at any scale, the greater the weight of the scale.

[0159] Formula 4:

[0160]

[0161] In addition, since the edge image needs to be defocused and blurred in step 202, in order to reduce the complexity of the processing. In a possible implementation, before the terminal obtains the edge image corresponding to the target image, the terminal may first use a low-pass filter to filter the target image to obtain low-frequency information in the target image. The terminal removes the low-frequency information from the target image. The terminal obtains the edge image in the target image after the low-frequency information is removed.

[0162] Optionally, since there are some pixels in the target image that do not affect the complexity of the scene, these pixels are referred to as invalid information, and the invalid information is the pixels in the non-field of view and / or the pixels in the overexposed area in the target image. For example, in a medical endoscope scene, the effective field of view of the image is not the entire image, but a circular range with the center of the circle at the center of the image limited by the aperture of the endoscope, and other pixels outside the circle are regarded as non-field of view pixels. Overexposed areas are usually caused by reflections, and all details are lost, which are invalid pixels. Therefore, the terminal can first eliminate the invalid information in the target image. Then the terminal obtains the edge image in the target image after the invalid information in the target image is eliminated.

[0163] Optionally, before the terminal obtains the edge image corresponding to the target image, the terminal may first filter the target image using a low-pass filter to obtain low-frequency information in the target image. The terminal removes the low-frequency information from the target image to obtain the target image from which the low-frequency information has been removed. After the terminal obtains the target image from which the low-frequency information has been removed, the terminal removes invalid information from the target image from which the low-frequency information has been removed, and then obtains an edge image in the target image from which the invalid information in the target image has been removed.

[0164] In addition, in step 202, if the target image is originally blurred, then there is no need to subsequently adjust the first clarity evaluation value to the second clarity evaluation value. Therefore, the terminal can determine whether the target image is a non-deep defocused image or a deep defocused image based on the first clarity evaluation value of the target image before determining the scene complexity of the target image based on the edge information in the target image. A non-deep defocused image represents a clear image or a slightly blurred image, and a deep defocused image represents a blurred image. If the clarity indicated by the first clarity evaluation value determines that the target image is a non-deep defocused image, the scene complexity of the target image is determined based on the edge information in the target image. If it is a deep defocused image, the focal length of the image acquisition device can be directly adjusted.

[0165] Specifically, in order to facilitate the determination of whether the target image is a deep defocused image or a non-defocused image, a second clarity threshold is set. When the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value show a positive correlation, if the first clarity evaluation value exceeds the second clarity threshold, the scene complexity of the target image is determined based on the edge information in the target image. That is, when the first clarity evaluation value exceeds the second clarity threshold, the target image is a non-defocused image. If the first clarity evaluation value is less than the second clarity threshold, it means that the target image is a deep defocused image, and the focal length of the camera can be directly adjusted. Among them, the second clarity threshold is used to divide deep defocused images and non-defocused images.

[0166] The specific implementation process of step 202 is as follows Figure 4 As shown, Figure 4 It is a flow chart for determining scene complexity provided by an embodiment of the present application. After determining the first clarity evaluation value, the clarity evaluation value calculation module determines whether the target image is a deep defocused image or a non-defocused defocused image according to the first clarity evaluation value. If it is a non-defocused defocused image, the scene complexity calculation module is turned on to perform filtering preprocessing on the target image, that is, the terminal uses a low-pass filter to filter the target image to obtain low-frequency information in the target image. The terminal removes the low-frequency information from the target image to obtain the target image with the low-frequency information removed. After the terminal obtains the target image with the low-frequency information removed. The terminal removes the invalid information in the target image with the low-frequency information removed, and then obtains the edge image in the target image after the invalid information in the target image is removed. The edge image is defocused and blurred using the image structure consistency processing module, and then the image complexity statistics module is used to determine the image two-dimensional entropy of the target image after the defocused and blurred processing, and then the complexity of the target image is determined, and the complexity statistics information is output. At this time, the complexity statistics information is also called the complexity of the shooting scene.

[0167] Step 203: If the scene complexity indicates that the captured scene in the target image is a complex scene, the first clarity evaluation value is adjusted to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value; or, if the scene complexity indicates that the captured scene in the target image is a simple scene, the first clarity evaluation value is adjusted to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value.

[0168] In a possible implementation, when the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value present a positive correlation, if the scene complexity indicates that the shooting scene in the target image is a complex scene, the first clarity evaluation value is reduced to obtain a second clarity evaluation value, and at this time, the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value. If the scene complexity indicates that the shooting scene in the target image is a simple scene, the first clarity evaluation value is increased to obtain a second clarity evaluation value, and at this time, the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value.

[0169] In another possible implementation, when the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value present a negative correlation, if the scene complexity indicates that the shooting scene in the target image is a complex scene, the first clarity evaluation value is increased to obtain a second clarity evaluation value, and at this time, the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value. If the scene complexity indicates that the shooting scene in the target image is a simple scene, the first clarity evaluation value is reduced to obtain a second clarity evaluation value, and at this time, the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value.

[0170] The difference between the second definition evaluation value and the first definition evaluation value is related to the complexity of the scene complexity indication.

[0171] Since the above two possible implementation methods are both based on the relationship between the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value, it is only necessary to adjust the first clarity evaluation value according to the relationship between the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value. Therefore, the following explanation is given under the condition that the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value show a positive correlation.

[0172] In order to facilitate the adjustment of the first clarity evaluation value, a complexity threshold is set. It is assumed that there is a positive correlation between the value of the scene complexity itself and the indicated scene complexity. If the scene complexity exceeds the complexity threshold, it means that the scene complexity indicates that the shooting scene in the target image is a complex scene, then the first clarity evaluation value is reduced to obtain a second clarity evaluation value. Alternatively, if the scene complexity is lower than the complexity threshold, it means that the scene complexity indicates that the shooting scene in the target image is a simple scene, then the first clarity evaluation value is increased to obtain a second clarity evaluation value.

[0173] At this time, the difference between the second clarity evaluation value and the first clarity evaluation value is positively correlated with the difference between the scene complexity and the complexity threshold. The complexity threshold is specified by the user. That is, the higher the complexity value, the greater the reduction of the clarity evaluation value. The lower the complexity value, the greater the increase of the clarity evaluation value.

[0174] Specifically, if the scene complexity exceeds the complexity threshold, it means that the current shooting scene is relatively complex, so the peak value corresponding to the first clarity evaluation value is relatively large, and the first clarity evaluation value is likely to not reach the clarity evaluation value of the same standard system. Therefore, it is necessary to reduce the first clarity evaluation value to obtain the second clarity evaluation value, so that the correlation between the second clarity evaluation value and the scene complexity is weakened. If the scene complexity is lower than the complexity threshold, it means that the current shooting scene is relatively simple, so the peak value corresponding to the first clarity evaluation value is relatively small, that is, the first clarity evaluation value is likely to not reach the clarity evaluation value of the same standard system. Therefore, it is necessary to increase the first clarity evaluation value to obtain the second clarity evaluation value. Through these two adjustment methods, the clarity evaluation values ​​of images under shooting scenes of different complexities can be mapped to the same standard system, so that the clarity evaluation values ​​can be used to detect the clarity of images under the same standard system, which can effectively avoid the phenomenon that the image acquisition device performs incorrect focusing processing on the clear image, thereby not causing shooting delays, and improving the efficiency of users using the image acquisition device to acquire images. In addition, when the target image is a frame of an image in a real-time video stream, if the scene changes, mapping the clarity evaluation values ​​of images in shooting scenes of different complexity to the same standard system can effectively prevent refocusing every time, thereby avoiding the phenomenon of misfocusing or temporary out-of-focus in the real-time video stream. In this way, when the real-time video stream is a video stream shot by an endoscope, mapping the clarity evaluation values ​​of images in shooting scenes of different complexity to the same standard system can prevent doctors from losing their attention or disturbing their vision when watching real-time videos due to misfocusing or temporary out-of-focus, thus avoiding the occurrence of some medical accidents.

[0175] The purpose of determining the second clarity evaluation value is to adjust the clarity evaluation values ​​of complex scenes and simple scenes to the same level, and then determine the clarity and blurriness of the target image based on the second clarity evaluation value, and then determine whether the focus of the image acquisition device needs to be adjusted. This avoids the phenomenon that the image acquisition device performs incorrect focusing processing on the captured clear image when the shooting scene changes.

[0176] The above-mentioned method of determining the clarity and blurriness of the target image based on the second clarity evaluation value and then determining whether it is necessary to adjust the focal length of the image acquisition device is implemented as follows: if the target image is determined to be a clear image based on the clarity indicated by the second clarity evaluation value, the focal length of the image acquisition device is not adjusted; if the target image is determined to be a blurred image based on the clarity indicated by the second clarity evaluation value, the focal length of the image acquisition device is adjusted.

[0177] In order to determine whether the target image is a clear image or a blurred image based on the clarity indicated by the second clarity evaluation value, a first clarity threshold is set. When the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value show a positive correlation, if the second clarity evaluation value exceeds the first clarity threshold, it indicates that the target image is a clear image, and the focus of the image acquisition device is not adjusted at this time. If the second clarity evaluation value is lower than the first clarity threshold, it indicates that the target image is a blurred image, and the focus of the image acquisition device is adjusted at this time. The first clarity threshold is a critical value for determining whether the image acquisition device adjusts the focus. That is, the automatic focus control module determines the current clarity and blur according to the second clarity evaluation value, and then gives the focus length when the image acquisition device is focusing. That is, when it is clear, the focus control module is not started, and the focus length is 0. When it is blurred, the focus control module is started. The higher the blur, the longer the focus length. The focus length is the length when the image acquisition device adjusts the focus, when the current focus reaches the target focus.

[0178] In summary, in the embodiment of the present application, the second clarity evaluation value of the target image is determined by the first clarity evaluation value of the target image and the scene complexity. Since the clarity evaluation value can judge the clarity and blurriness of the image, the second clarity evaluation value is used to further determine whether the image is clear or not. If the result of the judgment is that the image is not clear, in order to make the image clear, the method of adjusting the focal length of the image acquisition device can be used to make the image clear. If it is not clear, the focal length of the image acquisition device can be adjusted. Since the second clarity evaluation value is related to the scene complexity, and when the scene complexity indicates that the shooting scene in the target image is a complex scene, the clarity indicated by the adjusted second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value, and when the scene complexity indicates that the shooting scene in the target image is a simple scene, the clarity indicated by the adjusted second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value, so that the clarity evaluation values ​​under different complexity scenes can be converted into clarity evaluation values ​​of the same standard system. In this way, each time the image acquisition device changes the shooting scene, it only needs to judge whether the image acquisition device needs to adjust the focus based on the second clarity evaluation value, so that the situation where the shooting scene switches but the focus does not need to be adjusted can be correctly handled, instead of having to readjust the focus every time the shooting scene is changed. Therefore, the phenomenon that the image acquisition device performs incorrect focusing processing on the clear image can be effectively avoided, so that the shooting delay will not be caused, and the efficiency of the user using the image acquisition device to capture images is improved. In addition, when the target image is a frame image in a real-time video stream, if the scene changes, the method of the embodiment of the present application can prevent refocusing every time, so that the real-time video stream can be avoided from being misfocused or temporarily out of focus. In this way, when the real-time video stream is a video stream shot by an endoscope, the method of the embodiment of the present application can prevent the doctor from being distracted or disturbed by the video misfocusing or temporary out of focus when watching the real-time video.

[0179] All the above optional technical solutions can be combined in any way to form optional embodiments of the present application, and the embodiments of the present application will not be described in detail one by one.

[0180] Below Figure 5 The method provided in the embodiment of the present application is further explained by taking the example. Figure 5 is a specific flow chart of a clarity detection method provided in an embodiment of the present application. It should be noted that: Figure 5 The embodiment shown is only the aforementioned Figure 2 Some optional technical solutions in the illustrated embodiments do not constitute a limitation on the clarity detection method provided in the embodiments of the present application.

[0181] 1. After determining the target image, the scene stability detection module detects whether the scene of shooting the target image is in a stable state. If it is stable, the scene stability detection module sends the stable state of the scene and the image of the scene to the clarity evaluation value calculation module. If it is unstable, the scene stability detection module continues to determine whether the shooting scene is stable.

[0182] 2. The clarity evaluation value calculation module receives the image of the stabilized scene sent by the scene stability detection module, determines the first clarity evaluation value of the received image, and then sends the determined first clarity evaluation value of the image to the scene complexity calculation module. The scene complexity calculation module receives the first clarity evaluation value of the image sent by the clarity evaluation value calculation module, and determines whether the target image is a deep defocused image or a non-defocused defocused image according to the first clarity evaluation value. Determine the scene complexity of the non-defocused defocused image, and send the determined scene complexity to the clarity evaluation value correction module.

[0183] 3. The clarity evaluation value correction module receives the scene complexity determined by the scene complexity calculation module, corrects the first clarity evaluation value based on the scene complexity, and then determines a second clarity evaluation value. If the second clarity evaluation value is lower than the first clarity threshold, a focus adjustment instruction is sent to the focus control module.

[0184] 4. The focus control module controls the image acquisition device to adjust the focus. If the focus is adjusted to a state where the target image is clear, the focusing action is completed. After the focus is completed, the image acquisition device continues to collect images for scene stability detection and subsequent steps when the shooting scene changes next time.

[0185] In summary, in the embodiment of the present application, the second clarity evaluation value of the target image is determined by the first clarity evaluation value of the target image and the scene complexity. Since the clarity evaluation value can judge the clarity and blurriness of the image, the second clarity evaluation value is used to further determine whether the image is clear or not. If the result of the judgment is that the image is not clear, in order to make the image clear, the method of adjusting the focal length of the image acquisition device can be used to make the image clear. If it is not clear, the focal length of the image acquisition device can be adjusted. Since the second clarity evaluation value is related to the scene complexity, and when the scene complexity indicates that the shooting scene in the target image is a complex scene, the clarity indicated by the adjusted second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value, and when the scene complexity indicates that the shooting scene in the target image is a simple scene, the clarity indicated by the adjusted second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value, so that the clarity evaluation values ​​under different complexity scenes can be converted into clarity evaluation values ​​of the same standard system. In this way, each time the image acquisition device changes the shooting scene, it only needs to judge whether the image acquisition device needs to adjust the focus based on the second clarity evaluation value, so that the situation where the shooting scene switches but the focus does not need to be adjusted can be correctly handled, instead of having to readjust the focus every time the shooting scene is changed. Therefore, the phenomenon that the image acquisition device performs incorrect focusing processing on the clear image can be effectively avoided, so that the shooting delay will not be caused, and the efficiency of the user using the image acquisition device to capture images is improved. In addition, when the target image is a frame image in a real-time video stream, if the scene changes, the method of the embodiment of the present application can prevent refocusing every time, so that the real-time video stream can be avoided from being misfocused or temporarily out of focus. In this way, when the real-time video stream is a video stream shot by an endoscope, the method of the embodiment of the present application can prevent the doctor from being distracted or disturbed by the video misfocusing or temporary out of focus when watching the real-time video.

[0186] Figure 6 6 is a schematic diagram of a structure of a definition detection device provided in an embodiment of the present application. The definition detection device can be implemented by software, hardware or a combination of both. The definition detection device 600 includes: a determination module 601 and a judgment module 602.

[0187] A determination module, used to determine a first clarity evaluation value of the target image to be detected based on a clarity evaluation operator;

[0188] The determination module is further used to determine the scene complexity of the target image based on the edge information in the target image, where the scene complexity indicates the complexity of the shooting scene in the target image;

[0189] A judgment module is used to adjust the first clarity evaluation value to obtain a second clarity evaluation value if the scene complexity indicates that the shooting scene in the target image is a complex scene, and the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value; or, if the scene complexity indicates that the shooting scene in the target image is a simple scene, adjust the first clarity evaluation value to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value.

[0190] Optionally, the target image is an image currently captured by the image capture device;

[0191] The device also includes:

[0192] The adjustment module is used for not adjusting the focal length of the image acquisition device if the target image is determined to be a clear image based on the clarity indicated by the second clarity evaluation value, and for adjusting the focal length of the image acquisition device if the target image is determined to be a blurred image based on the clarity indicated by the second clarity evaluation value.

[0193] Optionally, the image acquisition device is an endoscope system, the endoscope system includes an endoscope, and the endoscope is used to be inserted into human tissue;

[0194] The target image is an image captured by the image acquisition device when the endoscope is inserted into human tissue.

[0195] Optionally, the determination module includes:

[0196] A first acquisition unit, used to acquire an edge image corresponding to the target image, where the edge image indicates edge information of a shooting scene in the target image;

[0197] A first determining unit, configured to perform a defocus blur process on an edge in the edge image based on a reference radius to obtain a critical blurred image;

[0198] The second determination unit is used to determine the scene complexity of the target image based on the image two-dimensional entropy of the critical blurred image, and the image two-dimensional entropy indicates the pixel value distribution characteristics in the critical blurred image.

[0199] Optionally, the second determining unit includes:

[0200] A determination unit, configured to scale the critical fuzzy image to obtain a plurality of scale images corresponding to the critical fuzzy image, wherein the scales of the plurality of scale images are different from the scale of the critical fuzzy image;

[0201] A determination unit, used to determine the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the plurality of scale images;

[0202] The determination unit is used to weightedly fuse the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images to obtain the scene complexity of the target image.

[0203] Optionally, the edges in the edge image include first-category edges and second-category edges, and an average edge width of the first-category edges is less than or equal to an average edge width of the second-category edges;

[0204] The first determining unit includes:

[0205] a processing unit, configured to perform defocus blur processing on a first edge in the edge image by using a first reference radius, wherein the first edge refers to an edge that belongs to the first type of edge but does not belong to the second type of edge;

[0206] a processing unit, configured to perform defocus blur processing on a second edge in the edge image by using a second reference radius, wherein the second edge refers to an edge that belongs to the second type of edge but does not belong to the first type of edge;

[0207] a processing unit, configured to perform defocus blur processing on a third edge in the edge image by using a third reference radius, wherein the third edge refers to an edge that belongs to both the first and second types of edges;

[0208] A processing unit, configured to perform defocus blur processing on other parts of the edge image except the first-type edges and the second-type edges by using a fourth reference radius;

[0209] Among them, the first reference radius is greater than or equal to the third reference radius, the third reference radius is greater than or equal to the second reference radius, and the second reference radius is greater than or equal to the fourth reference radius.

[0210] Optionally, the determining module further includes:

[0211] A rejection unit, used to reject invalid information in the target image, where the invalid information is pixels in a non-viewing range and / or pixels in an overexposed area in the target image;

[0212] The first acquisition unit is used for:

[0213] Get the edge image in the target image after removing invalid information.

[0214] Optionally, the determining module further includes:

[0215] A filtering unit, used for filtering the target image using a low-pass filter to obtain low-frequency information in the target image;

[0216] Remove low-frequency information from the target image;

[0217] The first acquisition unit is used to acquire an edge image in the target image after low-frequency information is removed.

[0218] Optionally, the difference between the second clarity evaluation value and the first clarity evaluation value is related to the complexity level indicated by the scene complexity indicator.

[0219] Optionally, the determining module is further used for:

[0220] If the target image is determined to be a non-deeply out-of-focus image based on the clarity level indicated by the first clarity evaluation value, the scene complexity of the target image is determined based on edge information in the target image.

[0221] In summary, in the embodiment of the present application, the second clarity evaluation value of the target image is determined by the first clarity evaluation value of the target image and the scene complexity. Since the clarity evaluation value can judge the clarity and blurriness of the image, the second clarity evaluation value is used to further determine whether the image is clear or not. If the result of the judgment is that the image is not clear, in order to make the image clear, the method of adjusting the focal length of the image acquisition device can be used to make the image clear. If it is not clear, the focal length of the image acquisition device can be adjusted. Since the second clarity evaluation value is related to the scene complexity, and when the scene complexity indicates that the shooting scene in the target image is a complex scene, the clarity indicated by the adjusted second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value, and when the scene complexity indicates that the shooting scene in the target image is a simple scene, the clarity indicated by the adjusted second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value, so that the clarity evaluation values ​​under different complexity scenes can be converted into clarity evaluation values ​​of the same standard system. In this way, each time the image acquisition device changes the shooting scene, it only needs to judge whether the image acquisition device needs to adjust the focus based on the second clarity evaluation value, so that the situation where the shooting scene switches but the focus does not need to be adjusted can be correctly handled, instead of having to readjust the focus every time the shooting scene is changed. Therefore, the phenomenon that the image acquisition device performs incorrect focusing processing on the clear image can be effectively avoided, so that the shooting delay will not be caused, and the efficiency of the user using the image acquisition device to capture images is improved. In addition, when the target image is a frame image in a real-time video stream, if the scene changes, the method of the embodiment of the present application can prevent refocusing every time, so that the real-time video stream can be avoided from being misfocused or temporarily out of focus. In this way, when the real-time video stream is a video stream shot by an endoscope, the method of the embodiment of the present application can prevent the doctor from being distracted or disturbed by the video misfocusing or temporary out of focus when watching the real-time video.

[0222] It should be noted that: the clarity detection device provided in the above embodiment only uses the division of the above functional modules as an example to illustrate when performing clarity detection. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the clarity detection device and the clarity detection method embodiment provided in the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0223] Figure 7 1 is a block diagram of a terminal 700 provided in an embodiment of the present application. The terminal 700 may be: a smart phone, a tablet computer, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 player (Moving Picture Experts Group Audio Layer IV), a laptop computer or a desktop computer. The terminal 700 may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0224] Typically, the terminal 700 includes a processor 701 and a memory 702 .

[0225] The processor 701 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 701 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 701 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 701 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 701 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0226] The memory 702 may include one or more computer-readable storage media, which may be non-transitory. The memory 702 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 702 is used to store at least one instruction, which is used to be executed by the processor 701 to implement the clarity detection method provided in the method embodiment of the present application.

[0227] In some embodiments, the terminal 700 may further optionally include: a peripheral device interface 703 and at least one peripheral device. The processor 701, the memory 702 and the peripheral device interface 703 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 703 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 704, a display screen 705, a camera assembly 706, an audio circuit 707, a positioning assembly 708 and a power supply 709.

[0228] The peripheral device interface 703 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 701 and the memory 702. In some embodiments, the processor 701, the memory 702, and the peripheral device interface 703 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 701, the memory 702, and the peripheral device interface 703 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0229] The radio frequency circuit 704 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 704 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 704 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 704 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 704 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 704 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0230] The display screen 705 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 705 is a touch display screen, the display screen 705 also has the ability to collect touch signals on the surface or above the surface of the display screen 705. The touch signal can be input to the processor 701 as a control signal for processing. At this time, the display screen 705 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 705 can be one, and the front panel of the terminal 700 is set; in other embodiments, the display screen 705 can be at least two, which are respectively set on different surfaces of the terminal 700 or are folded; in other embodiments, the display screen 705 can be a flexible display screen, which is set on the curved surface or folded surface of the terminal 700. Even, the display screen 705 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 705 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0231] The camera assembly 706 is used to capture images or videos. Optionally, the camera assembly 706 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 706 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0232] The audio circuit 707 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 701 for processing, or input them into the radio frequency circuit 704 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal 700. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 701 or the radio frequency circuit 704 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 707 may also include a headphone jack.

[0233] Positioning component 708 is used to locate the current geographical location of terminal 700 to implement navigation or LBS (Location Based Service). Positioning component 708 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0234] The power supply 709 is used to power various components in the terminal 700. The power supply 709 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 709 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0235] In some embodiments, the terminal 700 further includes one or more sensors 710 , including but not limited to: an acceleration sensor 711 , a gyroscope sensor 712 , a pressure sensor 713 , a fingerprint sensor 714 , an optical sensor 715 , and a proximity sensor 716 .

[0236] The acceleration sensor 711 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal 700. For example, the acceleration sensor 711 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 701 can control the display screen 705 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 711. The acceleration sensor 711 can also be used to collect motion data of games or users.

[0237] The gyro sensor 712 can detect the body direction and rotation angle of the terminal 700, and the gyro sensor 712 can cooperate with the acceleration sensor 711 to collect the user's 3D actions on the terminal 700. The processor 701 can implement the following functions based on the data collected by the gyro sensor 712: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0238] The pressure sensor 713 can be set on the side frame of the terminal 700 and / or the lower layer of the display screen 705. When the pressure sensor 713 is set on the side frame of the terminal 700, it can detect the user's holding signal of the terminal 700, and the processor 701 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 713. When the pressure sensor 713 is set on the lower layer of the display screen 705, the processor 701 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 705. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0239] The fingerprint sensor 714 is used to collect the user's fingerprint, and the processor 701 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 714, or the fingerprint sensor 714 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 701 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 714 can be set on the front, back, or side of the terminal 700. When a physical button or a manufacturer logo is set on the terminal 700, the fingerprint sensor 714 can be integrated with the physical button or the manufacturer logo.

[0240] The optical sensor 715 is used to collect the ambient light intensity. In one embodiment, the processor 701 can control the display brightness of the display screen 705 according to the ambient light intensity collected by the optical sensor 715. Specifically, when the ambient light intensity is high, the display brightness of the display screen 705 is increased; when the ambient light intensity is low, the display brightness of the display screen 705 is reduced. In another embodiment, the processor 701 can also dynamically adjust the shooting parameters of the camera assembly 706 according to the ambient light intensity collected by the optical sensor 715.

[0241] The proximity sensor 716, also called a distance sensor, is usually arranged on the front panel of the terminal 700. The proximity sensor 716 is used to collect the distance between the user and the front of the terminal 700. In one embodiment, when the proximity sensor 716 detects that the distance between the user and the front of the terminal 700 is gradually decreasing, the processor 701 controls the display screen 705 to switch from the screen-on state to the screen-off state; when the proximity sensor 716 detects that the distance between the user and the front of the terminal 700 is gradually increasing, the processor 701 controls the display screen 705 to switch from the screen-off state to the screen-on state.

[0242] Those skilled in the art will understand that Figure 7 The structure shown in the figure does not constitute a limitation on the terminal 700, and the terminal 700 may include more or less components than those shown in the figure, or combine some components, or adopt a different component arrangement.

[0243] The embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the terminal, the terminal is enabled to execute the clarity detection method provided in the above embodiment.

[0244] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a terminal, enables the terminal to execute the clarity detection method provided in the above embodiment.

[0245] Figure 8 Schematic diagram of a server structure provided in an embodiment of the present application. The server may be a server in a backend server cluster. Specifically:

[0246] The server 800 includes a central processing unit (CPU) 801, a system memory 804 including a random access memory (RAM) 802 and a read-only memory (ROM) 803, and a system bus 805 connecting the system memory 804 and the central processing unit 801. The server 800 also includes a basic input / output system (I / O system) 806 that helps transfer information between various components in the computer, and a large-capacity storage device 807 for storing an operating system 813, application programs 814, and other program modules 815.

[0247] The basic input / output system 806 includes a display 808 for displaying information and an input device 809 such as a mouse and a keyboard for user inputting information. The display 808 and the input device 809 are connected to the central processing unit 801 through an input / output controller 810 connected to the system bus 805. The basic input / output system 806 may also include an input / output controller 810 for receiving and processing inputs from a plurality of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 810 also provides output to a display screen, a printer, or other types of output devices.

[0248] The mass storage device 807 is connected to the central processing unit 801 through a mass storage controller (not shown) connected to the system bus 805. The mass storage device 807 and its associated computer-readable media provide non-volatile storage for the server 800. That is, the mass storage device 807 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM drive.

[0249] Without loss of generality, computer readable media may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, cassettes, magnetic tapes, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that computer storage media are not limited to the above. The above-mentioned system memory 804 and mass storage device 807 can be collectively referred to as memory.

[0250] According to various embodiments of the present application, the server 800 can also be connected to a remote computer on the network through a network such as the Internet. That is, the server 800 can be connected to the network 812 through the network interface unit 811 connected to the system bus 805, or the network interface unit 811 can be used to connect to other types of networks or remote computer systems (not shown).

[0251] The memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU. The one or more programs include instructions for performing the clarity detection method provided in the embodiment of the present application.

[0252] The embodiment of the present application also provides a non-temporary computer-readable storage medium. When the instructions in the storage medium are executed by the processor of the server, the server can execute the clarity detection method provided in the above embodiment.

[0253] The embodiment of the present application also provides a computer program product including instructions, which, when executed on a server, enables the server to execute the clarity detection method provided in the above embodiment.

[0254] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0255] The above description is only a preferred embodiment of the embodiments of the present application and is not intended to limit the embodiments of the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present application should be included in the scope of protection of the present application.

Claims

1. A clarity detection method, characterized in that: The method comprises: Determining a first clarity evaluation value of the target image to be detected based on a clarity evaluation operator; If the target image is determined to be a non-deeply out-of-focus image based on the clarity indicated by the first clarity evaluation value, then determining the scene complexity of the target image based on edge information in the target image, where the scene complexity indicates the complexity of the shooting scene in the target image; In a case where the clarity evaluation value and the clarity of the image indicated by the clarity evaluation value show a positive correlation, if the scene complexity indicates that the shooting scene in the target image is a complex scene, the first clarity evaluation value is adjusted to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value; or, if the scene complexity indicates that the shooting scene in the target image is a simple scene, the first clarity evaluation value is adjusted to obtain a second clarity evaluation value, and the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value.

2. The method according to claim 1, characterized in that The target image is the image currently captured by the image acquisition device; The method further comprises: If the target image is determined to be a clear image based on the clarity indicated by the second clarity evaluation value, the focus of the image acquisition device is not adjusted; if the target image is determined to be a blurred image based on the clarity indicated by the second clarity evaluation value, the focus of the image acquisition device is adjusted.

3. The method according to claim 1, characterized in that The image acquisition device is an endoscope system, and the endoscope system includes an endoscope, and the endoscope is used to be inserted into human tissue; The target image is an image captured by the image acquisition device when the endoscope is inserted into human tissue.

4. The method according to claim 1, characterized in that The determining the scene complexity of the target image based on edge information in the target image includes: Acquire an edge image corresponding to the target image, where the edge image indicates edge information of a shooting scene in the target image; Performing a defocus blur process on the edge in the edge image based on a reference radius to obtain a critical blurred image; The scene complexity of the target image is determined based on the image two-dimensional entropy of the critical blurred image, wherein the image two-dimensional entropy indicates a distribution feature of pixel values ​​in the critical blurred image.

5. The method according to claim 4, characterized in that The determining the scene complexity of the target image based on the image two-dimensional entropy of the critical blurred image includes: Performing a scale transformation on the critical fuzzy image to obtain a plurality of scale images corresponding to the critical fuzzy image, wherein the scales of the plurality of scale images are different from the scale of the critical fuzzy image; Determining the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images; The image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images are weighted and fused to obtain the scene complexity of the target image.

6. The method according to claim 4, characterized in that The edges in the edge image include first-type edges and second-type edges, and the average edge width of the first-type edges is less than or equal to the average edge width of the second-type edges; The performing defocus blur processing on the edge in the edge image based on the reference radius comprises: For a first edge in the edge image, a defocus blur process is performed using a first reference radius, where the first edge refers to an edge that belongs to the first type of edge but does not belong to the second type of edge; For a second edge in the edge image, a defocus blur process is performed using a second reference radius, where the second edge refers to an edge that belongs to the second type of edge but does not belong to the first type of edge; For a third edge in the edge image, a third reference radius is used to perform defocus blur processing, wherein the third edge refers to an edge belonging to both the first type of edge and the second type of edge; For other parts of the edge image except the first type of edges and the second type of edges, a fourth reference radius is used to perform defocus blur processing; Among them, the first reference radius is greater than or equal to the third reference radius, the third reference radius is greater than or equal to the second reference radius, and the second reference radius is greater than or equal to the fourth reference radius.

7. The method according to claim 4, characterized in that Before acquiring the edge image corresponding to the target image, the method further includes: Eliminating invalid information in the target image, wherein the invalid information is pixels in a non-viewing range and / or pixels in an overexposed area in the target image; The obtaining of the edge image corresponding to the target image comprises: An edge image in the target image after the invalid information is removed is obtained.

8. The method according to claim 4, characterized in that Before acquiring the edge image corresponding to the target image, the method further includes: Using a low-pass filter to filter the target image to obtain low-frequency information in the target image; removing the low-frequency information from the target image; The obtaining of the edge image corresponding to the target image comprises: An edge image in the target image is obtained after the low-frequency information is removed.

9. The method according to claim 1, characterized in that The difference between the second definition evaluation value and the first definition evaluation value is related to the complexity level of the scene complexity indication.

10. A clarity detection device, characterized in that: The device comprises: A determination module, used for determining a first clarity evaluation value of a target image to be detected based on a clarity evaluation operator; The determination module is further configured to determine the scene complexity of the target image based on edge information in the target image if the target image is determined to be a non-deeply out-of-focus image based on the clarity indicated by the first clarity evaluation value, wherein the scene complexity indicates the complexity of the shooting scene in the target image; A judgment module, for, when a clarity evaluation value and the clarity of an image indicated by the clarity evaluation value present a positive correlation, if the scene complexity indicates that the shooting scene in the target image is a complex scene, adjusting the first clarity evaluation value to obtain a second clarity evaluation value, wherein the clarity indicated by the second clarity evaluation value is lower than the clarity indicated by the first clarity evaluation value, or, if the scene complexity indicates that the shooting scene in the target image is a simple scene, adjusting the first clarity evaluation value to obtain a second clarity evaluation value, wherein the clarity indicated by the second clarity evaluation value is higher than the clarity indicated by the first clarity evaluation value.

11. The device according to claim 10, characterized in that The target image is the image currently captured by the image acquisition device; The device also includes: an adjustment module, configured to: if the target image is determined to be a clear image based on the clarity indicated by the second clarity evaluation value, not adjust the focus of the image acquisition device; and if the target image is determined to be a blurred image based on the clarity indicated by the second clarity evaluation value, adjust the focus of the image acquisition device; Wherein, the image acquisition device is an endoscope system, the endoscope system includes an endoscope, and the endoscope is used to be inserted into human tissue; The target image is an image acquired by the image acquisition device when the endoscope is inserted into human tissue; Wherein, the determination module includes: A first acquisition unit, configured to acquire an edge image corresponding to the target image, wherein the edge image indicates edge information of a shooting scene in the target image; A first determining unit, configured to perform a defocus blur process on the edge in the edge image based on a reference radius to obtain a critical blurred image; A second determining unit, configured to determine the scene complexity of the target image based on an image two-dimensional entropy of the critical blurred image, wherein the image two-dimensional entropy indicates a distribution feature of pixel values ​​in the critical blurred image; Wherein, the second determining unit includes: A determination unit, configured to scale the critical fuzzy image to obtain a plurality of scale images corresponding to the critical fuzzy image, wherein the scales of the plurality of scale images are different from the scale of the critical fuzzy image; The determining unit is used to determine the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images; The determining unit is used to weightedly fuse the image two-dimensional entropy of the critical blurred image and the image two-dimensional entropy of each scale image in the multiple scale images to obtain the scene complexity of the target image; The edges in the edge image include first-type edges and second-type edges, and the average edge width of the first-type edges is less than or equal to the average edge width of the second-type edges; The first determining unit includes: a processing unit, configured to perform defocus blur processing on a first edge in the edge image by using a first reference radius, wherein the first edge refers to an edge that belongs to the first type of edge but does not belong to the second type of edge; The processing unit is configured to perform defocus blur processing on a second edge in the edge image by using a second reference radius, where the second edge refers to an edge that belongs to the second type of edge but does not belong to the first type of edge; The processing unit is configured to perform defocus blur processing on a third edge in the edge image by using a third reference radius, wherein the third edge refers to an edge belonging to both the first type of edge and the second type of edge; The processing unit is configured to perform defocus blur processing on other parts of the edge image except the first type of edges and the second type of edges by using a fourth reference radius; Wherein, the first reference radius is greater than or equal to the third reference radius, the third reference radius is greater than or equal to the second reference radius, and the second reference radius is greater than or equal to the fourth reference radius; Wherein, the determination module further includes: A rejection unit, used to reject invalid information in the target image, wherein the invalid information is pixels in a non-viewing range and / or pixels in an overexposed area in the target image; The first acquisition unit is used for: Acquire an edge image in the target image after removing the invalid information; Wherein, the determination module further includes: A filtering unit, configured to filter the target image using a low-pass filter to obtain low-frequency information in the target image; and remove the low-frequency information from the target image; The first acquisition unit is used for: Acquire an edge image in the target image after removing the low-frequency information; The difference between the second clarity evaluation value and the first clarity evaluation value is related to the complexity level indicated by the scene complexity indicator.

12. A computer device, characterized in that: The device comprises: processor; a memory for storing processor-executable instructions; The processor is configured to execute the steps of the method described in any one of claims 1 to 9.

Images