Endoscope image processing method and device, electronic equipment and storage medium

Abstract

The invention provides an endoscope image processing method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring multiple frames of endoscope images acquired by an endoscope; calculating the similarity between the first endoscope image and the second endoscope image in the multiple frames of endoscope images; the first endoscope image is an endoscope image acquired before the second endoscope image; when the similarity is greater than or equal to a similarity threshold value, determining a first processing result corresponding to the first endoscope image as a second processing result corresponding to the second endoscope image; the first processing result and the second processing result are results corresponding to the target processing; when the similarity is smaller than a similarity threshold value, performing target processing on the second endoscope image to determine a second processing result corresponding to the second endoscope image; wherein the similarity is also used for basic image processing different from target processing. According to the scheme, the image processing speed and the image processing efficiency can be improved.

Description

Technical Field

[0001] This application belongs to the technical field of endoscopes, and specifically relates to an endoscope image processing method, an endoscope image processing device, an electronic device, and a storage medium. Background Art

[0002] An endoscope is a medical electronic optical instrument that can be inserted into the body cavities and internal organs of the human body for direct observation, diagnosis, and treatment. Electronic endoscopes can include gastroscopes, colonoscopes, etc. The endoscope can collect endoscope images of a specific area and transmit them to a display so that a doctor can judge the lesions based on the endoscope images.

[0003] In the related art, after obtaining the endoscope image, image processing calculations need to be performed on the endoscope image. The image processing calculations generally include basic image processing for improving the image display effect, image quality, or operation convenience, and auxiliary processing for assisting the user in image recognition and analysis. However, the image processing calculations of a large number of endoscope images, especially the auxiliary processing, consume a large amount of hardware resources of the system, which easily affects the system stability. Moreover, the current system hardware resources are already insufficient, making it difficult to meet the image processing requirements of a large number of endoscope images, resulting in an overly slow image processing speed and affecting the image processing efficiency. Summary of the Invention

[0004] This application is proposed in view of the above problems. This application provides an endoscope image processing method, an endoscope image processing device, an electronic device, and a storage medium.

[0005] According to one aspect of this application, there is provided an endoscope image processing method, including: acquiring multiple frames of endoscope images collected by an endoscope; calculating the similarity between a first endoscope image and a second endoscope image among the multiple frames of endoscope images; the first endoscope image being an endoscope image collected before the second endoscope image; when the similarity is greater than or equal to a similarity threshold, determining the first processing result corresponding to the first endoscope image as the second processing result corresponding to the second endoscope image; the first processing result and the second processing result being results corresponding to a target processing; when the similarity is less than the similarity threshold, performing the target processing on the second endoscope image to determine the second processing result corresponding to the second endoscope image; wherein, the similarity is also used for basic image processing different from the target processing.

[0006] Exemplarily, the first endoscope image and the second endoscope image are adjacent endoscope images among the multiple frames of endoscope images.

[0007] Exemplarily, calculating the similarity between a first endoscopic image and a second endoscopic image in multiple frames of endoscopic images includes: for each endoscopic image in the first endoscopic image and the second endoscopic image, performing multi-scale feature extraction on the endoscopic image to obtain a multi-layer feature image corresponding to the endoscopic image; determining the similarity between the first endoscopic image and the second endoscopic image based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image.

[0008] Exemplarily, determining the similarity between the first endoscopic image and the second endoscopic image based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image includes: for each endoscopic image in the first endoscopic image and the second endoscopic image, calculating the feature values of each layer of the multi-layer feature image corresponding to the endoscopic image; calculating the similarity between the same-layer feature images of the first endoscopic image and the second endoscopic image based on the feature values of the same-layer feature images corresponding to the first endoscopic image and the second endoscopic image; averaging the similarities corresponding to all layers of feature images between the first endoscopic image and the second endoscopic image to obtain a target similarity; the target similarity is the similarity between the first endoscopic image and the second endoscopic image.

[0009] Exemplarily, calculating the feature values of each layer of the multi-layer feature image corresponding to the endoscopic image includes: for each layer of the multi-layer feature image, calculating the mean value of the pixel values of each pixel point of the layer of the feature image to obtain the feature value of the layer of the feature image.

[0010] Exemplarily, before calculating the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images, the method further includes: for each endoscopic image in the first endoscopic image and the second endoscopic image, determining the clarity of the endoscopic image; determining whether the clarity is greater than a clarity threshold; calculating the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images when the respective clarities of the first endoscopic image and the second endoscopic image are both greater than the clarity threshold.

[0011] Exemplarily, determining the clarity of the endoscopic image includes: performing multi-scale feature extraction on the endoscopic image to obtain a multi-layer feature image corresponding to the endoscopic image; determining the gradient of each layer of the multi-layer feature image corresponding to the endoscopic image; averaging the gradients of the respective multi-layer feature images corresponding to the endoscopic image to obtain the clarity of the endoscopic image.

[0012] Exemplarily, the method further includes: when receiving an image freezing instruction, using the sharpness of the determined first endoscopic image and second endoscopic image to select a frozen image, so as to select a frame of endoscopic image with the highest sharpness from multiple frames of endoscopic images as the frozen image.

[0013] According to another aspect of the present application, there is provided an endoscopic image processing apparatus, including: an acquisition module for acquiring multiple frames of endoscopic images collected by an endoscope; a calculation module for calculating the similarity between a first endoscopic image and a second endoscopic image in the multiple frames of endoscopic images; the first endoscopic image is an endoscopic image collected before the second endoscopic image; a first determination module for, when the similarity is greater than or equal to a similarity threshold, determining the first processing result corresponding to the first endoscopic image as the second processing result corresponding to the second endoscopic image; the first processing result and the second processing result are results corresponding to a target process; a processing module for, when the similarity is less than the similarity threshold, performing a target process on the second endoscopic image to determine the second processing result corresponding to the second endoscopic image; wherein, the similarity is further used for basic image processing different from the target process.

[0014] According to yet another aspect of the present application, there is provided an electronic device, including a processor and a memory, and computer program instructions are stored in the memory, and when the computer program instructions are run by the processor, they are used to execute the endoscopic image processing method described in any of the above embodiments.

[0015] According to still another aspect of the present application, there is provided a storage medium, and program instructions are stored on the storage medium, and when the program instructions are run, they are used to execute the endoscopic image processing method described in any of the above embodiments.

[0016] According to the above technical solutions of the embodiments of the present application, when the similarity between the first endoscopic image and the second endoscopic image is greater than or equal to the similarity threshold, directly reuse the first processing result corresponding to the first endoscopic image as the second processing result; when the similarity between the first endoscopic image and the second endoscopic image is less than the similarity threshold, perform a target process on the second endoscopic image to determine the second processing result. This solution can reasonably reduce the number of endoscopic images to be subjected to the target process without affecting the accuracy of the processing result by using the similarity between the first endoscopic image and the second endoscopic image. And the similarity calculation result can be used to perform at least one basic image process. In this way, while avoiding introducing unnecessary computational amount, the computational amount of auxiliary calculation (i.e., target process) can be reduced. In short, this solution helps to improve the image processing speed and image processing efficiency, reduce system resource consumption, optimize the overall performance, and improve the stability of the system. At the same time, based on the system resources released by this solution, more functions required by users can be realized, which helps to better assist users in completing endoscopic examinations.

[0017] The above description is only an overview of the technical solution of the present application. In order to understand the technical means of the present application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of the present application more obvious and understandable, the following specific embodiments of the present application are given. Description of the Drawings

[0018] The following drawings of the present application are hereby incorporated as part of the present application for understanding the present application. The embodiments of the present application and their descriptions are shown in the drawings to explain the principles of the present application. In the drawings,

[0019] Figure 1 A schematic flowchart showing an endoscopic image processing method according to an embodiment of the present application;

[0020] Figure 2 A schematic flowchart showing an endoscopic image processing method according to a specific embodiment of the present application;

[0021] Figure 3 A schematic block diagram showing an endoscopic image processing apparatus according to an embodiment of the present application; and

[0022] Figure 4 A schematic block diagram showing an electronic device according to an embodiment of the present application. Detailed Description of the Embodiments

[0023] In order to make the purpose, technical solution and advantages of the present application more obvious, the exemplary embodiments according to the present application will be described in detail below with reference to the drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. It should be understood that the present application is not limited by the exemplary embodiments described herein. Based on the embodiments described in the present application, all other embodiments obtained by those skilled in the art without creative efforts shall fall within the protection scope of the present application.

[0024] Currently, after obtaining an endoscopic image, it is usually necessary to perform image processing calculations on the endoscopic image. Image processing calculations generally include basic image processing for improving image display effects, image quality, or operation convenience, and auxiliary processing for assisting users in image recognition and analysis. The auxiliary processing can use the image after basic image processing as input, perform image analysis on the image to obtain corresponding calculation results. The auxiliary processing can be, for example, identifying and marking specific parts in the image, measuring specific measurement items (such as the area, diameter, etc. of specific parts) for the identified specific parts. As users' requirements for the quality of endoscopic images are getting higher and higher, most of the hardware resources of the endoscopic image processing system are consumed by endoscopic image processing calculations, especially auxiliary processing. As a result, the following problems occur: On the one hand, due to insufficient system hardware resources, the speed of endoscopic image processing calculations is affected. In some application scenarios that require real-time display of endoscopic images, slow image processing calculations or high display delays of endoscopic images will seriously affect the user experience. Taking the clinical examination scenario as an example, clinical examinations have relatively high requirements for the real-time display of endoscopic images. However, due to slow image processing calculations, the display delay of endoscopic images is high, seriously affecting the user experience and the effect of clinical examinations. On the other hand, due to the excessive consumption of system (endoscopic image processing system) hardware resources by endoscopic image processing calculations, the system hardware resource consumption is close to saturation. Hardware resources such as the central processing unit (CPU), graphics processing unit (GPU), and memory are all heavily occupied, affecting system stability and the real-time feedback of auxiliary processing results. In view of this, the present application provides an endoscopic image processing method, an endoscopic image processing device, an electronic device, and a storage medium to at least solve the above at least one technical problem.

[0025] According to one aspect of the present application, an endoscopic image processing method is provided. Figure 1 The schematic flowchart of the endoscopic image processing method according to an embodiment of the present application is shown. As Figure 1 shown, the method 100 may include but is not limited to the following steps S110, step S120, step S130, and step S140.

[0026] In step S110, obtain multiple frames of endoscopic images collected by the endoscope.

[0027] Optionally, the number of endoscopic images in the multiple frames of endoscopic images can be fixed. For example, the number of multiple frames of endoscopic images can be 20 frames. Alternatively, the number of endoscopic images in the multiple frames of endoscopic images can also be unfixed. For example, each frame of image collected by the endoscope can be obtained in real time to obtain multiple frames of endoscopic images. In this embodiment, the number of multiple frames of endoscopic images increases with the increase of the endoscopic collection time.

[0028] Optionally, the multi-frame endoscopic images can be continuously acquired by the endoscope. Alternatively, the multi-frame endoscopic images can be partial images among a series of images acquired by the endoscope. For example, a series of images acquired by the endoscope can be sampled at a fixed number of frames interval to obtain the multi-frame endoscopic images.

[0029] In step S120, the similarity between the first endoscopic image and the second endoscopic image in the multi-frame endoscopic images is calculated, where the first endoscopic image is the endoscopic image acquired before the second endoscopic image.

[0030] Optionally, before calculating the similarity between the first endoscopic image and the second endoscopic image in the multi-frame endoscopic images in step S120, method 100 may further include: preprocessing the multi-frame endoscopic images. The preprocessing methods may include operations such as filtering, contrast stretching, dynamic range compression, etc. In this embodiment, by preprocessing the multi-frame endoscopic images, irrelevant information in the images can be eliminated, etc., which helps to reduce the consumption of computing resources in subsequent steps (such as step S120).

[0031] Optionally, the first endoscopic image and the second endoscopic image can be any two endoscopic images in the multi-frame endoscopic images. Optionally, the first endoscopic image and the second endoscopic image can be any two adjacent endoscopic images in the multi-frame endoscopic images. Optionally, the first endoscopic image and the second endoscopic image can be any two endoscopic images in the multi-frame endoscopic images that meet the target requirements. The target requirements can be set as needed. For example, the target requirement can be that the brightness of the endoscopic image is greater than the brightness threshold. Another example is that the target requirement can be that the clarity of the endoscopic image is greater than the clarity threshold.

[0032] As shown above, the first endoscopic image is the endoscopic image acquired before the second endoscopic image. In some embodiments, the first endoscopic image can be the first frame of endoscopic image acquired during the current acquisition process, and the second endoscopic image can be any subsequent acquired frame of endoscopic image. In other embodiments, the first endoscopic image can also be the previous frame of endoscopic image of the second endoscopic image.

[0033] Optionally, the similarity between the first endoscopic image and the second endoscopic image can be calculated by any one of algorithms such as cosine similarity, hash algorithm, mean square error (MSE) algorithm, structural similarity algorithm, feature matching algorithm, etc. For example, the cosine similarity between the first endoscopic image and the second endoscopic image can be calculated to determine the similarity between the first endoscopic image and the second endoscopic image.

[0034] In step S130, when the similarity is greater than or equal to the similarity threshold, the first processing result corresponding to the first endoscopic image is determined as the second processing result corresponding to the second endoscopic image; the first processing result and the second processing result are results corresponding to the target processing.

[0035] Optionally, the similarity threshold can be set as needed. It can be understood that the higher the similarity threshold, the higher the requirement for the second processing result corresponding to the second endoscopic image. Therefore, the similarity threshold can be determined according to the expected accuracy of the user for the processing result.

[0036] Optionally, the first processing result corresponding to the first endoscopic image can be the processing result actually obtained by performing the target processing on the first endoscopic image. Alternatively, the first processing result of the first endoscopic image can be determined by using the processing results corresponding to the endoscopic images acquired before the first endoscopic image. For example, when the similarity between the endoscopic image (which can be referred to as the third endoscopic image) acquired before the first endoscopic image and the first endoscopic image is greater than or equal to the similarity threshold, the third processing result corresponding to the third endoscopic image (that is, the processing result obtained by performing the target processing on the third endoscopic image) is determined as the first processing result.

[0037] The target processing can be the above-mentioned auxiliary processing. The specific manner of performing the target processing on the endoscopic image can be determined according to the acquisition scenario of the endoscopic image. For example, when applying the endoscopic image for clinical examination, the target processing can be to identify the target examination area in the endoscopic image. The target examination area can be the area where the target observation object (such as a lesion, a suspected tumor, etc.) of the human body is located. The processing result can be the position of the target examination area in the endoscopic image. Optionally, the manner of identifying the target examination area in the endoscopic image can adopt any existing or future-developed identification method. For example, the target examination area in the endoscopic image can be automatically identified by a trained neural network model. Those skilled in the art can understand the specific manner of identifying the target examination area in the endoscopic image, which will not be elaborated here. Another example is that the target processing can be to measure any measurement item of the endoscopic image.

[0038] When the similarity is greater than or equal to the similarity threshold, it indicates that the image information of the first endoscopic image and the second endoscopic image is relatively close. At this time, the first processing result corresponding to the first endoscopic image can be directly determined as the second processing result corresponding to the second endoscopic image, without the need to perform the target processing calculation on the second endoscopic image again, which can effectively reduce the calculation amount.

[0039] In step S140, when the similarity is less than the similarity threshold, target processing is performed on the second endoscopic image to determine a second processing result corresponding to the second endoscopic image; wherein, the similarity is also used for basic image processing different from the target processing.

[0040] In an embodiment of the present invention, the basic image processing is processing for visually optimizing or controlling the images collected during the endoscopic imaging process. The basic image processing may be processing for optimizing the image display effect and quality, such as noise reduction, white balance, color correction, and image processing for adapting to different lighting modes, or may be control processing such as image acquisition, image storage, image freezing, and video stream processing for improving operation convenience and processing real-time performance. Exemplarily, as will be introduced later, the calculated similarity may also be used for basic image processing different from the target processing, such as controlling the image acquisition frequency.

[0041] It can be understood that when the similarity between the first endoscopic image and the second endoscopic image is low, it indicates that there are certain differences in the image information of the first endoscopic image and the second endoscopic image, and the processing result corresponding to one frame of the endoscopic image is not sufficient to determine the processing result corresponding to the other frame of the endoscopic image. Therefore, when the similarity is less than the similarity threshold, the first processing result cannot be directly reused as the second processing result. In this case, target processing may be performed on the second endoscopic image to obtain a more accurate second processing result.

[0042] In the above technical solution, when the similarity between the first endoscopic image and the second endoscopic image is greater than or equal to the similarity threshold, the first processing result corresponding to the first endoscopic image is directly reused as the second processing result; when the similarity between the first endoscopic image and the second endoscopic image is less than the similarity threshold, target processing is performed on the second endoscopic image to determine the second processing result. This solution can reasonably reduce the number of endoscopic images to be subjected to target processing without affecting the accuracy of the processing result by using the similarity between the first endoscopic image and the second endoscopic image. And the similarity calculation result can be used to perform at least one basic image processing. In this way, while avoiding introducing the computational amount of unnecessary basic image processing, the computational amount of auxiliary calculation (i.e., target processing) can be reduced. In general, this solution helps to improve the image processing speed and efficiency, reduce system resource consumption, optimize the overall performance, and improve the stability of the system. At the same time, based on the system resources released by this solution, more functions required by users can be realized, which helps to better assist users in completing endoscopic examinations.

[0043] Note that Figure 1The flowchart is only an example and does not limit the execution order of the steps in the endoscopic image processing method 100. In one example, steps S110, S120, S130, and S140 can all be executed in real time. Each time a frame of endoscopic image collected by the endoscope is obtained in step S110, it is added to the multi-frame endoscopic images, and this endoscopic image is used as the second endoscopic image. The similarity between this endoscopic image and another frame of endoscopic image (i.e., the first endoscopic image) is determined through step S120. According to the comparison result between the similarity and the similarity threshold, after executing step S130 or step S140, the next frame of endoscopic image collected by the endoscope is continuously obtained, and the above process is repeated. In another example, steps S110, S120, S130, and S140 can be executed sequentially. For example, after all the endoscopic images in the multi-frame endoscopic images are obtained in step S110, step S120 starts to be executed, and after the similarity between the first endoscopic image and the second endoscopic image in each pair of endoscopic images in the multi-frame endoscopic images is determined in step S120, step S130 or step S140 is executed according to the comparison result between the similarity between the first endoscopic image and the second endoscopic image and the similarity threshold. Of course, there can be other suitable execution orders and execution manners for the steps in the endoscopic image processing method 100, which will not be elaborated here.

[0044] Exemplarily, the first endoscopic image and the second endoscopic image are adjacent endoscopic images in the multi-frame endoscopic images.

[0045] In this solution, the first endoscopic image and the second endoscopic image are adjacent endoscopic images in the multi-frame endoscopic images. In other words, this solution only considers the similarity between two adjacent frames of endoscopic images. In some embodiments, when collecting endoscopic images, the image collection site can be changed by moving the endoscope. It can be understood that for the endoscopic images collected during the movement of the endoscope, there may be more identical image information in two adjacent frames of endoscopic images. In this solution, only the similarity between two adjacent frames of endoscopic images needs to be calculated. Thus, this solution helps to further reduce the calculation amount, improve the calculation efficiency, and thereby helps to further improve the image processing speed.

[0046] Exemplarily, step S120, calculating the similarity between the first endoscopic image and the second endoscopic image in the multi-frame endoscopic images, can specifically include the following steps: For each frame of endoscopic image in the first endoscopic image and the second endoscopic image, multi-scale feature extraction is performed on this frame of endoscopic image to obtain a multi-layer feature image corresponding to this frame of endoscopic image. Based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image, the similarity between the first endoscopic image and the second endoscopic image is determined.

[0047] Optionally, the method for multi-scale feature extraction of endoscopic images can be implemented using any existing or future-developed image pyramid algorithm. For example, the algorithm can be a Gaussian pyramid, a Laplacian pyramid, etc. In a specific embodiment, for each frame of endoscopic image in the first endoscopic image and the second endoscopic image, the Gaussian pyramid can be used to downsample and expand the frame of endoscopic image into a multi-layer feature image with different scales.

[0048] In some embodiments, the similarity between the first endoscopic image and the second endoscopic image can be determined by a preset number of layers of feature images in the multi-layer feature images corresponding to the first endoscopic image and the second endoscopic image respectively. For example, the similarity between the first endoscopic image and the second endoscopic image can be determined by the first three layers of feature images with the largest scales in the multi-layer feature images corresponding to the first endoscopic image and the second endoscopic image respectively. This embodiment determines the similarity between the first endoscopic image and the second endoscopic image using only a few feature images, which helps to reduce the computational amount. In other embodiments, the similarity between the first endoscopic image and the second endoscopic image can be determined by all the feature images in the multi-layer feature images corresponding to the first endoscopic image and the second endoscopic image respectively. This embodiment uses all the layers of feature images corresponding to the first endoscopic image and the second endoscopic image respectively to determine the similarity between the first endoscopic image and the second endoscopic image, which helps to improve the computational accuracy and thus helps to more accurately determine the processing result corresponding to the endoscopic image.

[0049] In the above technical solution, the similarity between the first endoscopic image and the second endoscopic image is determined using the multi-layer feature images corresponding to the first endoscopic image and the second endoscopic image respectively. This solution helps to filter redundant image information by performing multi-scale feature extraction on the endoscopic images, and thus helps to more accurately determine the similarity between the first endoscopic image and the second endoscopic image.

[0050] Exemplarily, based on the multi-layer feature images corresponding to the first endoscopic image and the second endoscopic image respectively, determining the similarity between the first endoscopic image and the second endoscopic image can specifically include the following steps: For each frame of endoscopic image in the first endoscopic image and the second endoscopic image, calculate the feature values of each layer of feature image in the multi-layer feature image corresponding to the frame of endoscopic image. Based on the feature values of the same layer of feature images corresponding to the first endoscopic image and the second endoscopic image, calculate the similarity of the same layer of feature images between the first endoscopic image and the second endoscopic image. Calculate the average value of the similarities corresponding to all the layers of feature images between the first endoscopic image and the second endoscopic image to obtain the target similarity. The target similarity is the similarity between the first endoscopic image and the second endoscopic image.

[0051] Optionally, the step of calculating the similarity of the same-layer feature images between the first endoscopic image and the second endoscopic image based on the feature values of the same-layer feature images corresponding to the first endoscopic image and the second endoscopic image may specifically include the following steps: calculating the Hamming distance of the same-layer feature images between the first endoscopic image and the second endoscopic image; determining the similarity based on the Hamming distance. In this embodiment, the mean value of the Hamming distances corresponding to all the layer feature images between the first endoscopic image and the second endoscopic image may be calculated (this mean value may be referred to as the target Hamming distance), and the target similarity is the reciprocal of the target Hamming distance. It can be understood that the Hamming distance is inversely proportional to the degree of similarity. Therefore, in this embodiment, the greater the target Hamming distance, the smaller the target similarity, and the greater the difference between the first endoscopic image and the second endoscopic image; otherwise, vice versa.

[0052] Optionally, the step of calculating the similarity of the same-layer feature images between the first endoscopic image and the second endoscopic image based on the feature values of the same-layer feature images corresponding to the first endoscopic image and the second endoscopic image may specifically include the following steps: calculating the Euclidean distance of the same-layer feature images between the first endoscopic image and the second endoscopic image; determining the similarity based on the Euclidean distance. In this embodiment, the mean value of the Euclidean distances corresponding to all the layer feature images between the first endoscopic image and the second endoscopic image may be calculated (this mean value may be referred to as the target Euclidean distance), and the target similarity is the reciprocal of the target Euclidean distance. Similar to the Hamming distance, the Euclidean distance is inversely proportional to the degree of similarity. Therefore, in this embodiment, the greater the target Euclidean distance, the smaller the target similarity, and the greater the difference between the first endoscopic image and the second endoscopic image; otherwise, vice versa.

[0053] Optionally, any existing or future-developed mean algorithm may be used to determine the target similarity. For example, the target similarity may be determined by any one of the arithmetic mean, weighted mean, quadratic mean, etc. In some embodiments, the arithmetic mean of the similarities corresponding to all the layer feature images between the first endoscopic image and the second endoscopic image may be directly calculated to determine the target similarity. In other embodiments, the weight values of the similarities corresponding to different layer feature images may be preset, and the weighted mean of the similarities corresponding to all the layer feature images between the first endoscopic image and the second endoscopic image may be calculated. This solution helps to more accurately determine the target similarity by presetting the weights of the similarities corresponding to each layer feature image.

[0054] In the above technical solution, the similarity of the same-layer feature images between the first endoscopic image and the second endoscopic image is calculated based on the eigenvalues of the same-layer feature images corresponding to the first endoscopic image and the second endoscopic image, and the average value of the similarities corresponding to all the layer feature images between the first endoscopic image and the second endoscopic image is used to obtain the target similarity. This solution helps to accurately determine the similarity between the first endoscopic image and the second endoscopic image, thereby providing a relatively accurate basis for determining the processing result of the endoscopic image in the subsequent steps.

[0055] Exemplarily, calculating the eigenvalues of each layer of feature images in the multi-layer feature images corresponding to this frame of endoscopic image may specifically include the following steps: For each layer of feature images in the multi-layer feature images, calculate the average value of the pixel values of each pixel point of this layer of feature image to obtain the eigenvalue of this layer of feature image.

[0056] In this solution, using the average value of the pixel values of each pixel point of this layer of feature image can more accurately determine the eigenvalue of this layer of feature image, thereby facilitating providing a relatively reliable basis for calculating the similarity between two layers of endoscopic images in the subsequent steps.

[0057] Exemplarily, before step S120 of calculating the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images, method 100 may further include the following steps: For each frame of endoscopic image in the first endoscopic image and the second endoscopic image, determine the clarity of this frame of endoscopic image; determine whether the clarity is greater than the clarity threshold. Calculating the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images is performed when the respective clarities of the first endoscopic image and the second endoscopic image are both greater than the clarity threshold.

[0058] Optionally, method 100 may further include the following steps: When the clarity is less than or equal to the clarity threshold, no target processing is performed on this frame of endoscopic image or this frame of endoscopic image is deleted. When the clarity is less than or equal to the clarity threshold, it indicates that the image quality of this frame of endoscopic image is poor, and the credibility of the processing result obtained based on this frame of endoscopic image is low. Therefore, no target processing may be performed on this frame of endoscopic image and the endoscopic image without the processing result may be directly displayed, or this frame of endoscopic image may be directly deleted.

[0059] Optionally, any existing or future-developed image sharpness evaluation algorithm can be used to determine the sharpness of the endoscopic image. For example: Tenengrad gradient method, Laplacian gradient method, variance method, etc. When calculating the sharpness of the endoscopic image using the above algorithms, the sharpness of the endoscopic image can be determined according to the relationship between the evaluation index of the corresponding algorithm and the image sharpness. For example, when the evaluation index is directly proportional to the image sharpness, the evaluation index can be directly used to represent the image sharpness. When the evaluation index is inversely proportional to the image sharpness, the reciprocal of the evaluation index can be used to represent the image sharpness. Taking the Tenengrad gradient method as an example, in the embodiment, the evaluation index is the gradient calculated using the Tenengrad gradient method. It can be understood that the gradient is directly proportional to the image sharpness. Therefore, in this embodiment, the sharpness of the endoscopic image is the gradient calculated using the Tenengrad gradient method.

[0060] Optionally, the sharpness threshold can be set as needed. It can be understood that the higher the sharpness threshold, the higher the sharpness of the first endoscopic image and the second endoscopic image used to calculate the similarity, and the more reliable the processing results determined based on the first endoscopic image and the second endoscopic image respectively. However, at the same time, the higher the sharpness threshold, the fewer the number of endoscopic images (i.e., multiple frames of endoscopic images) whose sharpness meets the requirements, and the fewer the number of endoscopic images used to determine the processing results. When the target process is a processing method such as target inspection area recognition, fewer endoscopic images may affect the final target inspection area recognition result. Therefore, a suitable sharpness threshold can be determined based on the credibility of the processing results and the requirement for the number of multiple frames of endoscopic images.

[0061] Optionally, the first endoscopic image and the second endoscopic image are adjacent endoscopic images among multiple frames of endoscopic images. Calculating the similarity between the first endoscopic image and the second endoscopic image among multiple frames of endoscopic images is performed when the sharpness of each of the first endoscopic image and the second endoscopic image is greater than the sharpness threshold. Method 100 may further include the following steps: when only one frame of the endoscopic image among the first endoscopic image and the second endoscopic image has a sharpness greater than the sharpness threshold, perform target processing on the endoscopic image with a sharpness greater than the sharpness threshold to determine the processing result of this frame of endoscopic image.

[0062] In this embodiment, any two adjacent endoscopic images can be first selected from multiple endoscopic images as the first endoscopic image and the second endoscopic image. Then, the sharpness corresponding to each of the first endoscopic image and the second endoscopic image is determined respectively. If the sharpness of both the first endoscopic image and the second endoscopic image is greater than the sharpness threshold, the similarity between the first endoscopic image and the second endoscopic image can be calculated, and step S130 or step S140 is executed according to the calculated similarity. If only one of the first endoscopic image and the second endoscopic image has a sharpness greater than the sharpness threshold, only the endoscopic image with a sharpness greater than the sharpness threshold is subjected to target processing, and the endoscopic image with a sharpness less than or equal to the sharpness threshold is not processed. If the sharpness corresponding to each of the first endoscopic image and the second endoscopic image is less than or equal to the sharpness threshold, the current first endoscopic image and the second endoscopic image are not processed, and the first endoscopic image and the second endoscopic image are reselected from the multiple endoscopic images.

[0063] Figure 2 FIG. shows a schematic flowchart of an endoscopic image processing method according to a specific embodiment of the present application. In this embodiment, the current frame endoscopic image is the second endoscopic image, and the previous frame endoscopic image of the current frame endoscopic image is the first endoscopic image. The total number of endoscopic images is at least two frames. As Figure 2As shown, first, step S210 is executed to obtain an endoscopic image. In this step, each frame of the endoscopic image collected by the endoscope can be obtained in real time, and after each collection is completed, the following steps S220 to S290 are executed. Then, step S220 is executed to calculate the sharpness of the current frame endoscopic image. The sharpness of the current frame endoscopic image can be referred to as the first sharpness. Next, step S230 is executed to determine whether the first sharpness is greater than the sharpness threshold. When the sharpness of the current frame endoscopic image is less than or equal to the sharpness threshold, return to execute step S210 to re-obtain the endoscopic image. At this time, the current frame endoscopic image is the re-obtained endoscopic image. When the sharpness of the current frame endoscopic image is greater than the sharpness threshold and there is a previous frame endoscopic image of the current frame endoscopic image, step S240 is executed to obtain the sharpness of the previous frame endoscopic image. The sharpness of the previous frame endoscopic image can be referred to as the second sharpness. Next, step S250 is executed to determine whether the second sharpness is greater than the sharpness threshold. When the sharpness of the previous frame endoscopic image is greater than the sharpness threshold, step S260 is executed, otherwise, step S290 (perform target processing on the current frame endoscopic image) is executed. In step S260, the similarity between the current frame endoscopic image and the previous frame endoscopic image is calculated. In step S270, it is determined whether the similarity between the current frame endoscopic image and the previous frame endoscopic image is less than the similarity threshold. If the similarity between the current frame endoscopic image and the previous frame endoscopic image is less than the similarity threshold, step S290 is executed. If the similarity between the current frame endoscopic image and the previous frame endoscopic image is greater than or equal to the similarity threshold, step S280 is executed to output the processing result corresponding to the previous frame endoscopic image as the processing result corresponding to the current frame endoscopic image.

[0064] It can be understood that for any frame of endoscopic image, when the sharpness is greater than the sharpness threshold, it indicates that the image quality of this frame of endoscopic image is relatively high, and the accuracy of the target processing result determined based on this frame of endoscopic image is higher. When the sharpness is less than or equal to the sharpness threshold, it indicates that the image quality of this frame of endoscopic image is relatively poor, and the accuracy of the target processing result determined based on this frame of endoscopic image is lower. In this solution, only the endoscopic images with sharpness greater than the sharpness threshold are determined as one of the multiple frames of endoscopic images. On the one hand, if the sharpness of the previous frame of endoscopic image (i.e., the first endoscopic image) meets the requirements, it indicates that the accuracy of its processing result is high; if the sharpness of the current frame of endoscope (i.e., the second endoscopic image) also meets the requirements, then the reliability of the similarity judgment result between the two frames of images is high. Furthermore, the reliability of reusing the target processing result of the previous frame of image based on this similarity judgment result is also high. Thus, the accuracy of the target processing result of the current frame of image can be ensured. On the other hand, it helps to reduce the number of endoscopic images used to determine the target processing result, thereby helping to further improve the image processing speed and image processing efficiency.

[0065] Exemplarily, the steps of determining the clarity of the endoscopic image of this frame may specifically include the following steps: performing multi-scale feature extraction on the endoscopic image of this frame to obtain multi-layer feature images corresponding to the endoscopic image of this frame. Determining the gradient of each layer of feature image in the multi-layer feature images corresponding to the endoscopic image of this frame. Calculating the mean value of the gradients of the multi-layer feature images corresponding to the endoscopic image of this frame respectively to obtain the clarity of the endoscopic image of this frame. The specific manner of multi-scale feature extraction has been described in detail above and will not be elaborated.

[0066] Optionally, any existing or future-developed mean algorithm can be used to determine the clarity of the endoscopic image. For example, the clarity can be determined by any mean algorithm such as arithmetic mean, weighted mean, quadratic mean, etc. In some embodiments, the arithmetic mean value of the gradients of the multi-layer feature images corresponding to the endoscopic image of this frame can be directly calculated to determine the clarity. In other embodiments, the weight values corresponding to the gradients of different layers of feature images can be preset, and the weighted mean value of the gradients of the multi-layer feature images corresponding to the endoscopic image of this frame can be calculated. This solution helps to more accurately determine the clarity of the endoscopic image by presetting the weights of the similarities corresponding to each layer of feature images.

[0067] The above technical solution can more accurately determine the clarity of the endoscopic image of this frame by using the mean value of the gradients of each layer of feature image, which is beneficial to accurately determining whether the endoscopic image of this frame belongs to multi-frame endoscopic images.

[0068] Exemplarily, method 100 may further include the following steps: when an image freezing instruction is received, using the determined clarity of the first endoscopic image and the second endoscopic image for the selection of the frozen image, so as to select the endoscopic image with the highest clarity from the multi-frame endoscopic images as the frozen image.

[0069] In some embodiments, in order to improve the clarity of the frozen image, the endoscopic image with the highest clarity can be selected from the multi-frame images collected before and after the image freezing instruction is received as the frozen image. Therefore, in this embodiment, when the image freezing instruction is received, the determined clarity of the first endoscopic image and the second endoscopic image can be directly used for the selection of the frozen image, thereby improving the utilization rate of the clarity calculation result.

[0070] In the above technical solution, by calculating the clarity of the endoscopic image, on the one hand, a relatively reliable basis is provided for the similarity calculation in subsequent steps. On the other hand, the clarity calculation result can be reused in other basic image processing processes (such as image freezing). Thus, while reducing the computational amount of the target processing, unnecessary computational amount is not introduced additionally. Therefore, this solution helps to reduce the calculation steps, improve the calculation efficiency, and further helps to reduce the system resource consumption and improve the system stability.

[0071] Exemplarily, the basic image processing includes: switching the sampling frequency of the endoscopic image based on the similarity between the first endoscopic image and the second endoscopic image.

[0072] Optionally, switching the sampling frequency of the endoscope based on the similarity between the first endoscopic image and the second endoscopic image may include the following steps: when the similarity between the first endoscopic image and the second endoscopic image is greater than or equal to the similarity threshold, reducing the current sampling frequency; and / or when the similarity between the first endoscopic image and the second endoscopic image is less than the similarity threshold, increasing the current sampling frequency. It can be understood that when the similarity between the first endoscopic image and the second endoscopic image is greater than the similarity threshold, it can be considered that the image information contained in the two frames of endoscopic images is nearly the same. Therefore, when the similarity between the two frames of endoscopic images is high (i.e., greater than the similarity threshold), a lower sampling frequency can be used to sample the image to avoid repeated sampling. Conversely, a higher sampling frequency can be used to sample the image to avoid missing important image information.

[0073] In the above technical solution, by using the similarity between the first endoscopic image and the second endoscopic image calculated for basic image processing to determine whether the second image can reuse the target processing result of the first endoscopic image, the computational amount of the target processing can be reduced without adding new computational amount, which helps to improve the image processing speed and efficiency, reduce the system resource consumption, optimize the overall performance, and improve the stability of the system.

[0074] According to another aspect of the present application, an endoscopic image processing device is provided. Figure 3 A schematic block diagram showing an endoscopic image processing device according to an embodiment of the present application is as follows Figure 3 As shown, the endoscopic image processing device 300 may include an acquisition module 310, a calculation module 320, a first determination module 330, and a processing module 340.

[0075] The acquisition module 310 is configured to acquire multiple frames of endoscopic images collected by the endoscope.

[0076] A calculation module 320 is configured to calculate the similarity between a first endoscopic image and a second endoscopic image in multiple frames of endoscopic images; the first endoscopic image is an endoscopic image acquired before the second endoscopic image.

[0077] A first determination module 330 is configured to, when the similarity is greater than or equal to a similarity threshold, determine the first processing result corresponding to the first endoscopic image as the second processing result corresponding to the second endoscopic image; the first processing result and the second processing result are results corresponding to a target processing.

[0078] A processing module 340 is configured to, when the similarity is less than the similarity threshold, perform a target processing on the second endoscopic image to determine the second processing result corresponding to the second endoscopic image; wherein, the similarity is also used for basic image processing different from the target processing.

[0079] Exemplarily, the calculation module 320 includes: a first feature extraction sub-module, configured to perform multi-scale feature extraction on each frame of endoscopic image in the first endoscopic image and the second endoscopic image to obtain a multi-layer feature image corresponding to the frame of endoscopic image; a first determination sub-module, configured to determine the similarity between the first endoscopic image and the second endoscopic image based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image.

[0080] Exemplarily, the first determination sub-module includes: a first calculation unit, configured to calculate the feature value of each layer of feature image in the multi-layer feature image corresponding to each frame of endoscopic image in the first endoscopic image and the second endoscopic image; a second calculation unit, configured to calculate the similarity between the same layer feature images of the first endoscopic image and the second endoscopic image based on the feature values of the same layer feature images corresponding to the first endoscopic image and the second endoscopic image; an averaging unit, configured to average the similarities corresponding to all layer feature images between the first endoscopic image and the second endoscopic image to obtain a target similarity; the target similarity is the similarity between the first endoscopic image and the second endoscopic image.

[0081] Exemplarily, the averaging unit includes: a calculation sub-unit, configured to calculate the mean value of the pixel values of each pixel point of each layer of feature image in the multi-layer feature image to obtain the feature value of the layer of feature image.

[0082] Exemplarily, the endoscopic image processing device 300 further includes: a second determination module, configured to determine the sharpness of each frame of endoscopic image in the first endoscopic image and the second endoscopic image before the calculation module 320 calculates the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images; a judgment module, configured to judge whether the sharpness is greater than a sharpness threshold for each frame of endoscopic image in the first endoscopic image and the second endoscopic image; the calculation module 320 performs corresponding operations when the sharpness corresponding to each of the first endoscopic image and the second endoscopic image is greater than the sharpness threshold.

[0083] Exemplarily, the second determination module includes: a second feature extraction sub-module, configured to perform multi-scale feature extraction on the frame of endoscopic image to obtain a multi-layer feature image corresponding to the frame of endoscopic image; a second determination sub-module, configured to determine the gradient of each layer of feature image in the multi-layer feature image corresponding to the frame of endoscopic image; a calculation sub-module, configured to calculate the average value of the gradients of each of the multi-layer feature images corresponding to the frame of endoscopic image to obtain the sharpness of the frame of endoscopic image.

[0084] Exemplarily, the endoscopic image processing device 300 further includes: a selection module, configured to, when receiving an image freezing instruction, use the sharpness of the determined first endoscopic image and second endoscopic image for selecting a frozen image, so as to select a frame of endoscopic image with the highest sharpness from multiple frames of endoscopic images as the frozen image.

[0085] According to another aspect of the present application, an electronic device is further provided. Figure 4 The schematic block diagram of an electronic device 4 according to an embodiment of the present application is shown. As Figure 4 shown, the electronic device 4 includes a processor 410 and a memory 420. Among them, computer program instructions are stored in the memory 420, and when the computer program instructions are run by the processor 410, they are used to execute the endoscopic image processing method described above.

[0086] Optionally, when the computer program instructions are run by the processor 410, they are used to perform the following operations: acquiring multiple frames of endoscopic images collected by the endoscope; calculating the similarity between the first endoscopic image and the second endoscopic image in the multiple frames of endoscopic images; the first endoscopic image is an endoscopic image collected before the second endoscopic image; when the similarity is greater than or equal to a similarity threshold, determining the first processing result corresponding to the first endoscopic image as the second processing result corresponding to the second endoscopic image; the first processing result and the second processing result are results corresponding to a target process; when the similarity is less than the similarity threshold, performing a target process on the second endoscopic image to determine the second processing result corresponding to the second endoscopic image; wherein, the similarity is further used for basic image processing different from the target process.

[0087] Exemplarily, the steps for calculating the similarity between a first endoscopic image and a second endoscopic image in multiple frames of endoscopic images when computer program instructions are run by the processor 410 may include the following steps: For each endoscopic image in the first endoscopic image and the second endoscopic image, perform multi-scale feature extraction on the endoscopic image to obtain a multi-layer feature image corresponding to the endoscopic image; Based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image, determine the similarity between the first endoscopic image and the second endoscopic image.

[0088] Exemplarily, the steps for determining the similarity between the first endoscopic image and the second endoscopic image based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image when computer program instructions are run by the processor 410 may include the following steps: For each endoscopic image in the first endoscopic image and the second endoscopic image, calculate the feature values of each layer of the feature image in the multi-layer feature image corresponding to the endoscopic image; Based on the feature values of the same-layer feature images corresponding to the first endoscopic image and the second endoscopic image, calculate the similarity of the same-layer feature images between the first endoscopic image and the second endoscopic image; Take the average of the similarities corresponding to all layers of feature images between the first endoscopic image and the second endoscopic image to obtain the target similarity; The target similarity is the similarity between the first endoscopic image and the second endoscopic image.

[0089] Exemplarily, the steps for calculating the feature values of each layer of the feature image in the multi-layer feature image corresponding to the endoscopic image when computer program instructions are run by the processor 410 may include the following steps: For each layer of the multi-layer feature image, calculate the average of the pixel values of each pixel point of the layer of the feature image to obtain the feature value of the layer of the feature image.

[0090] Exemplarily, before the steps for calculating the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images when computer program instructions are run by the processor 410, the computer program instructions are also used by the processor 410 to perform the following operations: For each endoscopic image in the first endoscopic image and the second endoscopic image, determine the clarity of the endoscopic image; Determine whether the clarity is greater than the clarity threshold; The steps for calculating the similarity between the first endoscopic image and the second endoscopic image in multiple frames of endoscopic images when computer program instructions are run by the processor 410 are executed when the clarity corresponding to each of the first endoscopic image and the second endoscopic image is greater than the clarity threshold.

[0091] Exemplarily, the steps for determining the clarity of the endoscopic image when the computer program instructions are run by the processor 410 may include the following steps: performing multi-scale feature extraction on the endoscopic image of the frame to obtain a multi-layer feature image corresponding to the endoscopic image of the frame; determining the gradient of each layer of the feature image in the multi-layer feature image corresponding to the endoscopic image of the frame; and averaging the gradients of the multi-layer feature images corresponding to the endoscopic image of the frame respectively to obtain the clarity of the endoscopic image of the frame.

[0092] Exemplarily, when the computer program instructions are run by the processor 410, they are further used to perform the following operations: when an image freeze instruction is received, using the determined clarity of the first endoscopic image and the second endoscopic image for the selection of the frozen image, so as to select a frame of endoscopic image with the highest clarity from multiple frames of endoscopic images as the frozen image.

[0093] According to another aspect of the present application, a storage medium is further provided. Program instructions are stored on the storage medium, and the program instructions are used to execute the endoscopic image processing method described above when running.

[0094] Optionally, the program instructions are used to perform the following operations when running: obtaining multiple frames of endoscopic images collected by the endoscope; calculating the similarity between the first endoscopic image and the second endoscopic image in the multiple frames of endoscopic images; the first endoscopic image is an endoscopic image collected before the second endoscopic image; when the similarity is greater than or equal to the similarity threshold, determining the first processing result corresponding to the first endoscopic image as the second processing result corresponding to the second endoscopic image; the first processing result and the second processing result are results corresponding to the target processing; when the similarity is less than the similarity threshold, performing target processing on the second endoscopic image to determine the second processing result corresponding to the second endoscopic image; wherein, the similarity is also used for basic image processing different from the target processing.

[0095] The storage medium may, for example, include a storage component of a medical device such as an endoscope, a hard disk of a personal computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disc read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer-readable storage medium may be any combination of one or more computer-readable storage media.

[0096] Exemplarily, the steps for calculating the similarity between a first endoscopic image and a second endoscopic image in multiple endoscopic images that the program instructions are used to execute during runtime may include the following steps: For each endoscopic image in the first endoscopic image and the second endoscopic image, perform multi-scale feature extraction on the endoscopic image to obtain a multi-layer feature image corresponding to the endoscopic image; Based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image, determine the similarity between the first endoscopic image and the second endoscopic image.

[0097] Exemplarily, the steps for determining the similarity between the first endoscopic image and the second endoscopic image based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image that the program instructions are used to execute during runtime may include the following steps: For each endoscopic image in the first endoscopic image and the second endoscopic image, calculate the feature values of each layer of the feature image corresponding to the endoscopic image; Based on the feature values of the same layer feature images corresponding to the first endoscopic image and the second endoscopic image, calculate the similarity of the same layer feature images between the first endoscopic image and the second endoscopic image; Take the average of the similarities corresponding to all layer feature images between the first endoscopic image and the second endoscopic image to obtain the target similarity; The target similarity is the similarity between the first endoscopic image and the second endoscopic image.

[0098] Exemplarily, the steps for calculating the feature values of each layer of the feature image corresponding to the endoscopic image that the program instructions are used to execute during runtime may include the following steps: For each layer of the multi-layer feature image, calculate the average of the pixel values of each pixel point of the layer of the feature image to obtain the feature value of the layer of the feature image.

[0099] Exemplarily, before the program instructions execute the steps for calculating the similarity between the first endoscopic image and the second endoscopic image in multiple endoscopic images during runtime, the program instructions are also used to perform the following operations during runtime: For each endoscopic image in the first endoscopic image and the second endoscopic image, determine the clarity of the endoscopic image; Determine whether the clarity is greater than the clarity threshold; The steps for calculating the similarity between the first endoscopic image and the second endoscopic image in multiple endoscopic images that the program instructions are used to execute during runtime are executed when the clarity corresponding to each of the first endoscopic image and the second endoscopic image is greater than the clarity threshold.

[0100] Exemplarily, the steps for determining the clarity of the endoscopic image that the program instructions are used to execute during runtime may include the following steps: performing multi-scale feature extraction on the endoscopic image of the frame to obtain multi-layer feature images corresponding to the endoscopic image of the frame; determining the gradient of each layer of the multi-layer feature images corresponding to the endoscopic image of the frame; and averaging the gradients of the multi-layer feature images corresponding to the endoscopic image of the frame respectively to obtain the clarity of the endoscopic image of the frame.

[0101] Exemplarily, the program instructions are further used to execute the following operation during runtime: when an image freeze instruction is received, using the determined clarity of the first endoscopic image and the second endoscopic image for the selection of the frozen image, so as to select one endoscopic image with the highest clarity from multiple frames of endoscopic images as the frozen image.

[0102] Those of ordinary skill in the art can understand the specific implementation solutions of the above endoscopic image processing device, electronic device, and storage medium by reading the above relevant descriptions of the endoscopic image processing method. For the sake of brevity, they will not be elaborated here.

[0103] Although example embodiments have been described herein with reference to the accompanying drawings, it should be understood that the above example embodiments are merely exemplary and are not intended to limit the scope of the present application. Those of ordinary skill in the art can make various changes and modifications therein without departing from the scope and spirit of the present application. All such changes and modifications are intended to be included within the scope of the present application as claimed by the appended claims.

[0104] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional person can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0105] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.

[0106] In the specification provided herein, numerous specific details are set forth. However, it will be understood that embodiments of the present application may be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0107] Similarly, it should be understood that, in order to streamline the present application and assist in understanding one or more of the various aspects thereof, in the description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together in a single embodiment, figure, or description thereof. However, the methods of the present application should not be construed as reflecting an intention that the claimed present application requires more features than are expressly recited in each claim. Rather, as reflected by the corresponding claims, the point of the application is that the corresponding technical problems can be solved with features less than all the features of a single disclosed embodiment. Accordingly, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present application.

[0108] Those skilled in the art will appreciate that, except where features are mutually exclusive, any combination may be employed of all the features disclosed in this specification (including the accompanying claims, abstract and drawings), as well as of all the processes or units of any method or apparatus so disclosed. Each feature disclosed in this specification (including the accompanying claims, abstract and drawings) may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

[0109] In addition, those skilled in the art will be able to understand that, although some embodiments herein include certain features included in other embodiments but not others, combinations of the features of different embodiments are meant to be within the scope of the present application and form different embodiments. For example, in the claims, any one of the claimed embodiments may be used in any combination.

[0110] Each component embodiment of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules in the endoscopic image processing device and the electronic device according to the embodiments of the present application. The present application can also be implemented as a device program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or in any other form.

[0111] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several devices, several of these devices can be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words can be interpreted as names.

[0112] The above is only the specific implementation manner of the present application or the description of the specific implementation manner. The protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present application, and all of them should be covered by the protection scope of the present application. The protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. An endoscopic image processing method, characterized in that, Including: Obtaining multiple endoscopic images collected by an endoscope; Calculating the similarity between a first endoscopic image and a second endoscopic image among the multiple endoscopic images; The first endoscopic image is an endoscopic image collected before the second endoscopic image; When the similarity is greater than or equal to a similarity threshold, determining the first processing result corresponding to the first endoscopic image as the second processing result corresponding to the second endoscopic image; The first processing result and the second processing result are results corresponding to a target processing; When the similarity is less than the similarity threshold, performing the target processing on the second endoscopic image to determine the second processing result corresponding to the second endoscopic image; Wherein, the similarity is also used for basic image processing different from the target processing.

2. The endoscopic image processing method according to claim 1, characterized in that, The first endoscopic image and the second endoscopic image are adjacent endoscopic images among the multiple endoscopic images.

3. The endoscopic image processing method according to claim 1, wherein, The calculating the similarity between the first endoscopic image and the second endoscopic image among the multiple endoscopic images includes: For each endoscopic image in the first endoscopic image and the second endoscopic image, performing multi-scale feature extraction on the endoscopic image to obtain a multi-layer feature image corresponding to the endoscopic image; Based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image, determining the similarity between the first endoscopic image and the second endoscopic image.

4. The endoscopic image processing method according to claim 3, characterized in that, The based on the multi-layer feature images respectively corresponding to the first endoscopic image and the second endoscopic image, determining the similarity between the first endoscopic image and the second endoscopic image includes: For each endoscopic image in the first endoscopic image and the second endoscopic image, calculating the feature value of each layer of the feature image in the multi-layer feature image corresponding to the endoscopic image; Based on the feature values of the same-layer feature images corresponding to the first endoscopic image and the second endoscopic image, calculating the similarity of the same-layer feature images between the first endoscopic image and the second endoscopic image; Taking the average of the similarities corresponding to all layers of feature images between the first endoscopic image and the second endoscopic image to obtain a target similarity; the target similarity is the similarity between the first endoscopic image and the second endoscopic image.

5. The endoscopic image processing method according to claim 4, characterized in that, The calculating the feature value of each layer of the feature image in the multi-layer feature image corresponding to the endoscopic image includes: For each layer of the feature image in the multi-layer feature image, calculating the average of the pixel values of each pixel point of the layer of the feature image to obtain the feature value of the layer of the feature image.

6. The endoscopic image processing method according to any one of claims 1-5, characterized in that, Before calculating the similarity between the first endoscopic image and the second endoscopic image among the multiple endoscopic images, the method further includes: For each endoscopic image in the first endoscopic image and the second endoscopic image, Determining the clarity of the endoscopic image; Judging whether the clarity is greater than a clarity threshold; Calculating the similarity between the first endoscopic image and the second endoscopic image in the multi-frame endoscopic images is performed when the sharpness corresponding to each of the first endoscopic image and the second endoscopic image is greater than the sharpness threshold.

7. The endoscopic image processing method according to claim 6, wherein Determining the sharpness of the frame of endoscopic image includes: Performing multi-scale feature extraction on the frame of endoscopic image to obtain a multi-layer feature image corresponding to the frame of endoscopic image; Determining the gradient of each layer of the multi-layer feature image corresponding to the frame of endoscopic image; Calculating the mean value of the gradients of the multi-layer feature images corresponding to the frame of endoscopic image to obtain the sharpness of the frame of endoscopic image.

8. The endoscopic image processing method according to claim 6, wherein The method further includes: When an image freezing instruction is received, using the determined sharpness of the first endoscopic image and the second endoscopic image for selecting a frozen image, so as to select a frame of endoscopic image with the highest sharpness from the multi-frame endoscopic images as the frozen image.

9. An endoscopic image processing apparatus, characterized in that, Includes: An acquisition module, configured to acquire multi-frame endoscopic images collected by an endoscope; A calculation module, configured to calculate the similarity between the first endoscopic image and the second endoscopic image in the multi-frame endoscopic images; The first endoscopic image is an endoscopic image collected before the second endoscopic image; A first determination module, configured to, when the similarity is greater than or equal to the similarity threshold, determine the first processing result corresponding to the first endoscopic image as the second processing result corresponding to the second endoscopic image; The first processing result and the second processing result are results corresponding to a target processing; A processing module, configured to, when the similarity is less than the similarity threshold, perform the target processing on the second endoscopic image to determine the second processing result corresponding to the second endoscopic image; Wherein, the similarity is further used for basic image processing different from the target processing.

10. An electronic device, characterized in that, Includes a processor and a memory, and computer program instructions are stored in the memory, and when the computer program instructions are run by the processor, they are used to execute the endoscopic image processing method according to any one of claims 1-8.

11. A storage medium, characterized in that, Program instructions are stored on the storage medium, and when the program instructions are run, they are used to execute the endoscopic image processing method according to any one of claims 1-8.

Images