Endoscope image recognition method

Abstract

The invention relates to an endoscope image recognition method which comprises the following steps: acquiring an endoscope detection item, and determining a recognition area, a recognition target and recognition precision of the endoscope detection item; performing image acquisition on the recognition area through the endoscope; the collected endoscope image is preprocessed; according to the endoscope detection item, the preprocessed image is recognized through a preset image recognition model, and a recognized recognition target is output. The method has the effect of improving the endoscope image recognition efficiency and recognition precision.

Description

Technical Field

[0001] The present application relates to the field of image recognition, and in particular to an endoscope image recognition method. Background Art

[0002] An endoscope is a medical device or industrial inspection tool used to observe the internal structure of an object. It mainly transmits internal images to an external display device through a slender, flexible optical or electronic tube.

[0003] In the medical field, endoscopes are an important diagnostic tool that can penetrate deep into the human body and provide doctors with intuitive image information. However, traditional endoscopic image recognition mainly relies on the doctor's experience and subjective judgment, which has certain limitations. In addition, with the continuous development of medical technology, the number of endoscopic images is also increasing. Doctors need to spend a lot of time and energy to analyze these images, which undoubtedly increases the workload of doctors.

[0004] With respect to the above-mentioned related technologies, the inventors believe that the existing endoscopic image recognition has low efficiency and poor recognition accuracy. Summary of the invention

[0005] In order to solve the above problems, the present application provides an endoscopic image recognition method.

[0006] In a first aspect, the present application provides an endoscopic image recognition method, which adopts the following technical solution:

[0007] An endoscope image recognition method comprises the following steps:

[0008] Obtaining endoscope detection items, and determining the identification area, identification target, and identification accuracy of the endoscope detection items;

[0009] Capturing images of the identified area through an endoscope;

[0010] Preprocessing the collected endoscopic images;

[0011] According to the endoscopic detection project, the preprocessed image is recognized by a preset image recognition model, and the recognized recognition target is output; the image recognition model is a convolutional neural network model obtained by deep learning through endoscopic image data of various endoscopic detection projects as samples.

[0012] Preferably, the obtaining of the endoscope detection items and determining the identification area, identification target and identification accuracy of the endoscope detection items specifically include the following steps:

[0013] Collect the identity information of medical personnel and verify the identity of the medical personnel performing the operation;

[0014] After the identity verification is passed, based on the identity authority of the medical staff, a list of endoscopic inspection items corresponding to their authority is provided;

[0015] After interaction with medical personnel, the endoscopic inspection items to be performed are selected, and the recognition area, recognition target and recognition accuracy of the endoscopic inspection items are determined.

[0016] Preferably, the preprocessing of the collected endoscopic images specifically comprises the following steps:

[0017] Filter the collected endoscopic images to remove texture background and noise points in the endoscopic images;

[0018] Perform motion blur processing on the filtered endoscopic image to enhance the clarity of the endoscopic image;

[0019] The histogram of the endoscopic image after motion blur processing is equalized to make the histogram of the endoscopic image as flat as possible, so as to obtain a high-quality endoscopic image in which all pixel values ​​can be used on average;

[0020] Gamma correction is performed on the endoscopic image after histogram equalization;

[0021] The endoscopic image that has completed Gamma correction is normalized to complete image preprocessing.

[0022] Preferably, the step of performing motion blur processing on the filtered endoscopic image to enhance the clarity of the endoscopic image specifically comprises the following steps:

[0023] Determine the motion estimation window specifications based on the recognition accuracy of the endoscope detection items, and calculate the motion vector field of the endoscope image based on the optical flow algorithm;

[0024] Determine the size and direction of the blur kernel according to the motion vector field of the endoscopic image;

[0025] According to the endoscopic detection project, the power spectrum of the original clear image and the power spectrum of the noise that are preset corresponding to the endoscopic detection project are obtained, and the endoscopic image is deblurred by the Wiener filtering method to enhance the clarity of the endoscopic image.

[0026] Preferably, the identifying of the preprocessed image by a preset image recognition model according to the endoscopic detection project also includes: the image recognition model calls a standard blank model of the corresponding recognition area according to the endoscopic detection project, and receives the preprocessed image in real time for cropping, splicing and pasting on the standard blank model to form an endoscopic three-dimensional model.

[0027] Preferably, the image recognition model outputting the recognized recognition target also includes: the image recognition model marking the recognized recognition target on the endoscopic three-dimensional model.

[0028] Preferably, the image recognition model positions the endoscope based on the image information collected by the endoscope, marks it on the endoscopic three-dimensional model in real time, and highlights the areas on the endoscopic three-dimensional model that are not covered with the image, instructing medical personnel to collect images of missed areas.

[0029] Preferably, the identifying of the preprocessed image by a preset image recognition model according to the endoscopic detection project further includes: the image recognition model summarizes the identified targets that are below the recognition accuracy standard and marks them on the endoscopic three-dimensional model.

[0030] In a second aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution:

[0031] A computer-readable storage medium, characterized in that it stores a computer program that can be loaded by a processor and execute any of the above methods.

[0032] In summary, the present application includes at least one of the following beneficial technical effects:

[0033] 1. According to the endoscope detection project, the identification area, identification target and identification accuracy are clearly defined, so that the entire identification process has a clear direction and standard. After high-quality image acquisition and effective image preprocessing, the quality of the endoscope image is improved. Finally, the image recognition model is used to perform targeted and efficient identification according to the detection project, so as to effectively improve the image recognition efficiency and recognition accuracy;

[0034] 2. Image Gamma correction can further optimize the image presentation effect on the display device, improve the target acquisition efficiency and acquisition accuracy, especially improve the recognition effect of small lesions, and effectively improve the efficiency and accuracy of endoscopic image recognition;

[0035] 3. By performing motion blur processing on endoscopic images, the influence of motion factors on the image during image acquisition (organ motion interference and motion effects during operation) can be effectively reduced, making the recognition target in the image more obvious and stable, which helps to improve the performance and generalization ability of the image recognition model. Images that have been processed by motion blur can achieve better results when performing other image processing operations. In addition, during the real-time monitoring and diagnosis of the endoscope, motion blur processing can quickly pre-process the real-time acquired images, reduce the impact of motion blur on image quality, and enable doctors to observe the internal organs in a timely and accurate manner and make correct diagnosis and treatment decisions. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1is a method flow chart of an endoscope image recognition method in an embodiment of the present application;

[0037] Figure 2 is a flow chart of a method for obtaining endoscope detection items in an embodiment of the present application;

[0038] Figure 3 is a flow chart of a method for preprocessing an endoscopic image in an embodiment of the present application;

[0039] Figure 4 It is a flow chart of a method for performing motion blur processing on an endoscopic image in an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following is combined with Figure 1-4 This application is described in further detail.

[0041] The present application embodiment discloses an endoscope image recognition method. Figure 1 , an endoscope image recognition method, comprising the following steps:

[0042] S1. Determine the project: obtain the endoscope detection project, determine the recognition area, recognition target and recognition accuracy of the endoscope detection project;

[0043] S2, image acquisition: image acquisition of the identification area through the endoscope;

[0044] S3, image preprocessing: preprocessing the collected endoscopic images;

[0045] S4, image recognition: according to the endoscope detection project, the pre-processed image is recognized by the pre-set image recognition model, and the recognized recognition target is output; the image recognition model is a convolutional neural network model obtained by deep learning through the endoscope image data of various endoscope detection projects as samples. It should be noted that the specific training steps of the convolutional neural network model are prior art and will not be repeated here. Through the setting of the image recognition model, the convolutional neural network model used is deeply learned through a large amount of endoscope image data. It has strong feature extraction and pattern recognition capabilities, and it also has strong universality. It can realize various medical endoscope detection projects, reducing the cost and time of developing different recognition algorithms for different projects. According to the endoscope detection project, the recognition area, recognition target and recognition accuracy are clearly identified, so that the entire recognition process has a clear direction and standard. After high-quality image acquisition and effective image preprocessing, the quality of the endoscope image is improved. Finally, the image recognition model is used to perform targeted and efficient recognition according to the detection project, so as to effectively improve the efficiency and accuracy of endoscope image recognition.

[0046] Reference Figure 2The step of obtaining the endoscope detection items and determining the identification area, identification target and identification accuracy of the endoscope detection items specifically includes the following steps:

[0047] A1. Identity verification: Collect the identity information of medical personnel and verify the identity of the medical personnel performing the operation;

[0048] A2, simplified option: After identity authentication is passed, based on the medical staff's identity authority, a list of endoscopic inspection items corresponding to their authority is provided;

[0049] A3. Selected items: After interaction with medical staff, select the endoscopic inspection items to be performed, and determine the recognition area, recognition target, and recognition accuracy of the endoscopic inspection items. Since the convolutional neural network model is used for deep learning through a large amount of endoscopic image data, it is able to recognize endoscopic images of various endoscopic inspection items, which greatly enhances the universality of endoscopic image recognition. This also leads to a large number of endoscopic inspection items supported by the image recognition model. When determining the endoscopic inspection items, the identity of the operating medical staff is authenticated to determine the department to which they belong and the endoscopic inspection items that they can perform with their authority, and the list of endoscopic inspection items is streamlined. On the basis of improving the security of the system, it helps medical staff to easily and efficiently select the endoscopic inspection items that need to be performed, thereby improving the convenience of operation.

[0050] Reference Figure 3 The preprocessing of the collected endoscopic images specifically includes the following steps:

[0051] B1. Noise removal: Filter the collected endoscopic images to remove texture background and noise points in the endoscopic images;

[0052] B2. Motion blur processing: Motion blur processing is performed on the filtered endoscopic image to enhance the clarity of the endoscopic image;

[0053] B3. Contrast enhancement: The histogram of the endoscopic image after motion blur processing is equalized to make the histogram of the endoscopic image as flat as possible, so as to obtain a high-quality endoscopic image that can use all pixel values ​​on average;

[0054] B4. Image Gamma Correction: Perform Gamma correction on the endoscopic image after histogram equalization;

[0055] B5. Image normalization: Normalize the endoscopic images that have completed Gamma correction to complete image preprocessing. Through the above steps, the collected endoscopic images, on the basis of using standard preprocessing methods such as noise removal, contrast enhancement and image normalization enhancement to improve image clarity, further use motion blur processing and image Gamma correction. Motion blur processing can effectively reduce the impact of the endoscope's own movement and organ peristalsis on the images obtained by the endoscope during operation, improve the stability and recognizability of the image, and help improve the performance and generalization ability of the image recognition model; and image Gamma correction can further optimize the image presentation effect on the display device, improve the efficiency and accuracy of target acquisition, especially improve the recognition effect of small lesions, and effectively improve the efficiency and accuracy of endoscopic image recognition.

[0056] For example, in endoscopic image recognition of early gastric cancer, gamma correction can make tiny lesion areas on the gastric mucosa (such as tiny ridges or color changes of early precancerous lesions) more prominent, helping to improve the model's sensitivity to tiny gastric cancer lesions.

[0057] Reference Figure 4 The motion blur processing of the filtered endoscopic image to enhance the clarity of the endoscopic image specifically includes the following steps:

[0058] C1. Motion estimation: Determine the motion estimation window specifications based on the recognition accuracy of the endoscope detection items, and calculate the motion vector field of the endoscope image based on the optical flow algorithm;

[0059] C2, blur kernel estimation: determine the size and direction of the blur kernel based on the motion vector field of the endoscopic image;

[0060] C3, Wiener filter deblurring: According to the endoscopic detection project, the power spectrum and noise power spectrum of the original clear image corresponding to the endoscopic detection project are obtained, and the endoscopic image is deblurred by the Wiener filter method to enhance the clarity of the endoscopic image. By performing motion blur processing on the endoscopic image, the influence of motion factors on the image during image acquisition (organ motion interference and motion influence during operation) can be effectively reduced, making the recognition target in the image more obvious and stable, which helps to improve the performance and generalization ability of the image recognition model. The image after motion blur processing can achieve better results when performing other image processing operations. In addition, in the real-time monitoring and diagnosis process of the endoscope, motion blur processing can quickly pre-process the real-time acquired image, reduce the influence of motion blur on image quality, and enable doctors to observe the condition of internal organs in a timely and accurate manner and make correct diagnosis and treatment decisions.

[0061] For example, if the endoscopy project is an intestinal inspection project (looking for intestinal polyps), then based on the project, the motion estimation window size is determined to be 5×5 pixels. For the intestinal image collected by the endoscope, a window of size 5×5 pixels is selected, and the motion vector field of the endoscope image is calculated based on the optical flow algorithm. Specifically, the gradient I of the pixels in the window in the x and y directions is calculated using the differential operator. x ,I y , and the temporal gradient I between adjacent frames t (obtained by subtracting the pixel values ​​at the same position of the two frames of images). Based on the constant brightness assumption and Taylor series expansion, equation I is constructed for each pixel in the window. xu +I yv +I t =0 (where u and V are motion vectors in the x and y directions, respectively). Since there are 25 pixels in the window, an overdetermined set of equations can be obtained. The least squares method is used to solve the set of equations to obtain the motion vector (u, v) of the central pixel. This process is repeated for each pixel or region of interest (suspected polyp area) in the image to obtain the motion vector field of the entire image. The shape of the blur kernel is then determined based on the direction of the motion vector. If the motion vector is mainly in the horizontal direction, the blur kernel is in the shape of a horizontal strip; if it is in the vertical direction, it is in the shape of a vertical strip; if it is a complex oblique motion, it is in the corresponding inclined shape. The blur kernel size is estimated based on the motion vector amplitude. For example, if the motion vector indicates that the pixel has moved a distance of n pixels, the blur kernel length can be initially set to a value related to n. Other factors such as the continuity of the motion can also be considered. Finally, according to the endoscopic detection project, the power spectrum of the original clear image and the power spectrum of the noise that are preset for the endoscopic detection project are obtained. The size and shape of the blur kernel are known, and its Fourier transform H(k, l) is calculated, where k and l are frequency domain variables. Then, the Wiener filter transfer function Hw(k, l) is calculated according to the Wiener filter transfer function formula. The blurred intestinal image is Fourier transformed to obtain its frequency domain representation Y(k, l), and then the result of the restored image in the frequency domain is calculated by X(k, l) = Hw(k, l) * Y(k, l). Finally, an inverse Fourier transform is performed to obtain the restored spatial domain image, which makes the image of the suspected polyp area clearer and reduces the influence of motion blur on the observation of polyp features.

[0062] The above formula for calculating the Wiener filter transfer function is:

[0063]

[0064] In addition, when the preprocessed image is recognized by the pre-set image recognition model according to the endoscopic detection project, the image recognition model calls the standard blank model of the corresponding recognition area according to the endoscopic detection project, and receives the preprocessed image in real time for cropping, splicing and pasting on the standard blank model to form an endoscopic three-dimensional model. By cropping, splicing and pasting the preprocessed image on the standard blank model to form an endoscopic three-dimensional model, the doctor can observe the detection area more intuitively from a three-dimensional perspective. The image recognition model then marks the identified recognition target on the endoscopic three-dimensional model. The image recognition model marks the identified recognition target on the endoscopic three-dimensional model. It can help doctors better understand the position, size and shape of lesions in three-dimensional space, which is crucial for accurately judging the scope and severity of lesions, and accurately locate lesions (recognize targets), effectively improving diagnostic efficiency and accuracy.

[0065] At the same time, the image recognition model locates the endoscope based on the image information collected by the endoscope, annotates it in real time on the endoscopic 3D model, and highlights the areas on the endoscopic 3D model that are not covered with images, instructing medical staff to collect images of missed areas. By positioning the endoscope through the image recognition model and annotating it in real time on the 3D model, medical staff can clearly know the specific position and observation direction of the endoscope in the body, which helps to accurately detect the target area and ensure that important lesions are not missed. It can also flexibly adjust the inspection strategy according to the actual situation of the patient, which can greatly improve the comprehensiveness of image acquisition in complex identification areas and achieve the effect of improving the accuracy of endoscopic detection.

[0066] In addition, in order to improve the positioning accuracy of the endoscope, auxiliary positioning can be performed according to the motion vector in the motion blur processing, which can compensate for motion errors and adapt to different motion states.

[0067] In addition, when the image recognition model recognizes the pre-processed image according to the endoscopic detection items, the image recognition model will summarize the recognition targets that are lower than the recognition accuracy standard and mark them on the endoscopic three-dimensional model. It can fully present suspicious areas and assist doctors to comprehensively evaluate the relationship between these suspicious areas and surrounding tissues from the perspective of the overall structure, so as to more accurately determine whether further observation or examination is needed, and can efficiently discover some early and tiny lesions, which improves the scientific nature of endoscopic image recognition.

[0068] The embodiment of the present application also discloses a computer-readable storage medium, which stores a computer program that can be loaded by a processor and executed as in the above method. The computer-readable storage medium includes, for example: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0069] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the scope of protection of the invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on these embodiments, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in this field can still combine, add, delete or make other adjustments to the features in the various embodiments of the present invention according to the circumstances without conflict, without making creative work, so as to obtain different other technical solutions that do not deviate from the concept of the present invention in essence, and these technical solutions also belong to the scope of protection of the present invention.

Claims

1. An endoscope image recognition method, characterized in that: The following steps are involved: Obtaining endoscope detection items, and determining the identification area, identification target, and identification accuracy of the endoscope detection items; Capturing images of the identified area through an endoscope; Preprocessing the collected endoscopic images; According to the endoscopic detection project, the preprocessed image is recognized by a preset image recognition model, and the recognized recognition target is output; the image recognition model is a convolutional neural network model obtained by deep learning through endoscopic image data of various endoscopic detection projects as samples.

2. The endoscopic image recognition method according to claim 1, characterized in that: The obtaining of endoscope detection items and determining the identification area, identification target and identification accuracy of the endoscope detection items specifically include the following steps: Collect the identity information of medical personnel and verify the identity of the medical personnel performing the operation; After the identity verification is passed, based on the identity authority of the medical staff, a list of endoscopic inspection items corresponding to their authority is provided; After interaction with medical personnel, the endoscopic inspection items to be performed are selected, and the recognition area, recognition target and recognition accuracy of the endoscopic inspection items are determined.

3. The endoscopic image recognition method according to claim 1, characterized in that: The preprocessing of the collected endoscopic images specifically includes the following steps: Filter the collected endoscopic images to remove texture background and noise points in the endoscopic images; Perform motion blur processing on the filtered endoscopic image to enhance the clarity of the endoscopic image; The histogram of the endoscopic image after motion blur processing is equalized to make the histogram of the endoscopic image as flat as possible, so as to obtain a high-quality endoscopic image in which all pixel values ​​can be used on average; Gamma correction is performed on the endoscopic image after histogram equalization; The endoscopic image that has completed Gamma correction is normalized to complete image preprocessing.

4. The endoscopic image recognition method according to claim 3, characterized in that: The motion blur processing of the filtered endoscopic image to enhance the clarity of the endoscopic image specifically comprises the following steps: The specification of the motion estimation window is determined based on the recognition accuracy of the endoscope detection items, and the motion vector field of the endoscope image is calculated based on the optical flow algorithm; Determine the size and direction of the blur kernel according to the motion vector field of the endoscopic image; According to the endoscopic detection project, the power spectrum of the original clear image and the power spectrum of the noise that are preset corresponding to the endoscopic detection project are obtained, and the endoscopic image is deblurred by the Wiener filtering method to enhance the clarity of the endoscopic image.

5. The method for endoscope image recognition according to claim 1, characterized in that: The identifying of the preprocessed image by a preset image recognition model according to the endoscopic detection project also includes: the image recognition model calls a standard blank model of the corresponding recognition area according to the endoscopic detection project, and receives the preprocessed image in real time to crop, splice and overlay it on the standard blank model to form an endoscopic three-dimensional model.

6. The method for endoscope image recognition according to claim 5, characterized in that: The image recognition model outputting the recognized recognition target also includes: the image recognition model marking the recognized recognition target on the endoscopy three-dimensional model.

7. The endoscopic image recognition method according to claim 5, characterized in that: The image recognition model positions the endoscope based on the image information collected by the endoscope, marks it on the endoscopic three-dimensional model in real time, and highlights the areas on the endoscopic three-dimensional model that are not covered with images, instructing medical personnel to collect images of missed areas.

8. The method for endoscope image recognition according to claim 5, characterized in that: The identifying of the preprocessed image by a preset image recognition model according to the endoscope detection project also includes: the image recognition model summarizes the identified targets that are below the recognition accuracy standard and marks them on the endoscope three-dimensional model.

9. A computer-readable storage medium, characterized in that: A computer program is stored which can be loaded by a processor and execute the method according to any one of claims 1 to 8.

Images