Image processing method and system of endoscope

Abstract

The invention discloses an endoscope image processing method and system, and relates to the technical field of image processing, and the method comprises the steps: collecting an RGB image, calculating the mass center of an edge image, taking the mass center as an initial distortion center, adding the displacement and the initial distortion center, calculating an adjusted distortion center, calculating a distortion radial distance by using a radial distortion model, and obtaining an adjusted distortion center. Calculating corrected coordinates by using polar coordinate transformation; and calculating a frequency domain real reflection image by using a Wiener filtering formula, performing spatial domain conversion on the frequency domain real reflection image by using two-dimensional inverse fast Fourier transform to obtain a restored spatial domain image, and calculating a pixel value of an enhanced image by using point-by-point addition to generate an enhanced image. The distortion radial distance is calculated through a radial distortion model, corrected coordinates are calculated in combination with polar coordinate transformation, the geometric accuracy and the visual reality sense of the image are improved, the pixel value of the enhanced image is calculated through point-by-point addition in combination with a gradient field and an enhancement factor, and the quality and the precision of the endoscope image are enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to an image processing method and system for an endoscope. Background Art

[0002] With the continuous development of endoscope technology, image processing has become its core component. Endoscope technology is widely used in medical diagnosis, industrial inspection, and other fields. Its image quality directly affects the accuracy of diagnosis results and the efficiency of detection. Especially in the medical field, high-definition endoscope images can help doctors better observe lesions and improve the accuracy of treatment.

[0003] Existing endoscope image processing technologies still have some deficiencies. When traditional methods process image distortion, they fail to fully consider factors such as the camera angle, lens distortion, and dynamic changes between consecutive frames during the actual acquisition process, often resulting in unsatisfactory correction effects and affecting the accuracy of subsequent image analysis and processing. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides an image processing method and system for an endoscope, which solves the problem that traditional methods, when processing image distortion, fail to fully consider factors such as the camera angle, lens distortion, and dynamic changes between consecutive frames during the actual acquisition process, often resulting in unsatisfactory correction effects and affecting the accuracy of subsequent image analysis and processing.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides an image processing method for an endoscope, which includes,

[0008] Collect RGB images, use the Harris corner response function method to calculate the corner response value, screen the screening threshold greater than the corner response value as feature points, calculate the centroid of the edge image as the initial distortion center, add the displacement to the initial distortion center to calculate the adjusted distortion center, use the radial distortion model to calculate the distortion radial distance, use polar coordinate transformation to calculate the corrected coordinates, and use the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image;

[0009] Use the Wiener filtering formula to calculate the frequency-domain true reflection image, perform spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, use point-by-point addition to calculate the pixel values of the enhanced image to generate the enhanced image, and use color space conversion to map the mapped image to the RGB space to obtain the enhanced RGB image;

[0010] Extract the feature vectors of the enhanced RGB image, construct a convolutional neural network (CNN) model to calculate the class probabilities, and perform status detection;

[0011] Construct a visualization to display the status detection results, and store the collected and analyzed RGB images.

[0012] As a preferred solution of the image processing method of the endoscope described in the present invention, wherein: for the collection of the RGB image, the corrected coordinates are calculated using polar coordinate transformation to generate the final corrected image, including:

[0013] Collect continuous RGB images by using the continuous frame acquisition method through the CCD sensor of the endoscope device,

[0014] Convert the RGB image to grayscale using the weighted average method, calculate the grayscale value to generate a grayscale image, calculate the gradients of the grayscale image in the horizontal and vertical directions respectively using the Sobel operator, calculate the product of the horizontal and vertical gradients, construct a Harris response matrix, set the empirical constant using the empirical parameter tuning method, and calculate the corner response value using the Harris corner response function method;

[0015] Set the screening threshold using the percentage threshold method, and screen the values greater than the screening threshold of the corner response value as feature points;

[0016] Extract the positions of the feature points in the corresponding grayscale image, set the neighborhood window using the sliding window method, encode the neighborhood window using the local binary pattern, calculate the LBP value, calculate the Hamming distance between adjacent LBP values using the Hamming distance, and select the matching pair with the minimum Hamming distance using the K-nearest neighbor matching to generate the inter-frame matching relationship;

[0017] Extract the gradients of the first frame of the RGB image in the horizontal and vertical directions, calculate the edge intensity, and calculate the centroid of the edge image based on the edge intensity as the initial distortion center;

[0018] Calculate the displacement of each pair of adjacent frame feature points based on the inter-frame matching relationship, and calculate the sum of the absolute values of all displacements;

[0019] Add the displacement to the initial distortion center, calculate the adjusted distortion center, and calculate the global distortion center using the arithmetic mean method;

[0020] Calculate the distance from the spatial coordinates of the frame grayscale image to the global distortion center using the Euclidean distance formula, denoted as the radial distance, and calculate the mean value of the radial distance, and calculate the angle of the spatial coordinates of the frame grayscale image relative to the center using the arctangent function;

[0021] Use the linear approximation method to calculate the radial distortion coefficient, and calculate the distorted radial distance based on the radial distance using the radial distortion model;

[0022] Subtract the distorted radial distance from the radial distance to calculate the distortion offset, and use polar coordinate transformation to calculate the corrected coordinates;

[0023] Extract the positions of the corrected coordinates in the corresponding RGB image, use nearest neighbor search to select four nearest neighbor pixels, and use bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image.

[0024] As a preferred solution of the image processing method of the endoscope described in the present invention, wherein: calculating the frequency-domain true reflection image using the Wiener filtering formula to obtain the enhanced RGB image includes:

[0025] Use weighted average method to perform gray conversion on the final corrected image, calculate the gray value to generate the enhanced image, use geometric average method to calculate the geometric average of the four edge pixels of the enhanced image, and use linear interpolation method to generate the illumination distribution;

[0026] Use point-by-point division to calculate the illumination image;

[0027] Use moment estimation method to set the standard deviation of the Gaussian function, and use two-dimensional Gaussian function formula to calculate the point spread function;

[0028] Use two-dimensional fast Fourier transform to represent the illumination image and the point spread function in the frequency domain respectively, use conjugate complex operation to calculate the frequency-domain conjugate complex of the point spread function, use modulus square operation to calculate the frequency-domain modulus square of the illumination image, use fixed threshold method to set the noise power estimation constant, and use Wiener filtering formula to calculate the frequency-domain true reflection image;

[0029] Use two-dimensional inverse fast Fourier transform to convert the frequency-domain true reflection image to the spatial domain to obtain the restored spatial domain image;

[0030] Use Prewitt convolution kernel to convolve the normalized edge tensor field to obtain the gradients in the horizontal and vertical directions, and merge the gradients in the horizontal and vertical directions to obtain the gradient field;

[0031] Based on the spatial domain image, use the maximum value function to set the enhancement factor, and use point-by-point addition to calculate the pixel values of the enhanced image to generate the enhanced image;

[0032] Use threshold clipping method to limit the value range of the enhanced image, define the pixel values of the enhanced image greater than 255 as 255, and define the pixel values of the enhanced image less than 0 as 0 to obtain the clipped enhanced image;

[0033] Use the Sigmoid function to perform non-linear gray mapping on the clipped enhanced image, and use color space conversion to map the mapped image to the RGB space to obtain the enhanced RGB image.

[0034] As a preferred solution of the image processing method of the endoscope described in the present invention, wherein: the extraction of the feature vector of the enhanced RGB image includes:

[0035] Separate the enhanced RGB image using color channel separation to generate three color channels, and calculate the mean and variance of the three color channels using statistical analysis methods;

[0036] Use the feature flat stitching method to combine the mean and variance of the three color channels to generate a feature vector.

[0037] As a preferred solution of the image processing method of the endoscope described in the present invention, wherein: the construction of the convolutional neural network CNN model to calculate the class probability and perform state detection includes:

[0038] Collect historical RGB images with labels and perform feature extraction, calculate feature vectors, and generate a training set;

[0039] The construction of the convolutional neural network CNN model includes an input layer, a convolutional layer, an activation function, a pooling layer, a fully connected layer, and an output layer:

[0040] Use the training set to train the convolutional neural network CNN model, and use the cross-entropy loss function and the Adam optimizer to perform parameter iterative optimization;

[0041] Input the feature vector into the trained convolutional neural network CNN model to output the class probability, set a classification threshold using the empirical rule, when the class probability is greater than the classification threshold, it is judged as an abnormal state, and when the class probability is less than or equal to the classification threshold, it is judged as a normal state.

[0042] As a preferred solution of the image processing method of the endoscope described in the present invention, wherein: the construction of visualizing and displaying the state detection results includes:

[0043] Use the front-end framework React.js to build a visualization interface and display the state detection results;

[0044] Allow users who have passed real-name verification to view.

[0045] As a preferred solution of the image processing method of the endoscope described in the present invention, wherein: the storage of the RGB images generated by collection and analysis includes:

[0046] Store the collected RGB images and the detection results generated by analysis in the central database, and set security access measures. The central database will back up the stored data to the cloud, and regularly perform integrity detection on the stored data and the backup data. After the detection is completed, generate an integrity detection record and synchronously store it in the central database.

[0047] In a second aspect, the present invention provides an image processing system for an endoscope, including:

[0048] A collection and correction module, configured to collect RGB images, calculate corner response values using the Harris corner response function method, screen for feature points greater than the screening threshold of the corner response value, calculate the centroid of the edge image as the initial distortion center, add the displacement to the initial distortion center to calculate the adjusted distortion center, calculate the distortion radial distance using the radial distortion model, calculate the corrected coordinates using polar coordinate transformation, and interpolate the corrected coordinates using the bilinear interpolation formula to generate the final corrected image;

[0049] An image enhancement module, configured to calculate the frequency-domain true reflection image using the Wiener filtering formula, perform spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, calculate the pixel values of the enhanced image using point-by-point addition to generate the enhanced image, and map the mapped image to the RGB space using color space conversion to obtain the enhanced RGB image;

[0050] A feature detection module, configured to extract the feature vectors of the enhanced RGB image, construct a convolutional neural network (CNN) model to calculate the class probabilities, and perform status detection;

[0051] A visualization and storage module, configured to construct a visualization to display the status detection results and store the RGB images generated by collection and analysis.

[0052] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the image processing method for an endoscope as described in the first aspect of the present invention is implemented.

[0053] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the image processing method for an endoscope as described in the first aspect of the present invention is implemented.

[0054] The beneficial effects of the present invention are as follows: The present invention collects RGB images, uses the Harris corner response function method to calculate the corner response value, screens the screening threshold greater than the corner response value as the feature points, calculates the centroid of the edge image as the initial distortion center, adds the displacement to the initial distortion center to calculate the adjusted distortion center, uses the radial distortion model to calculate the distortion radial distance, uses polar coordinate transformation to calculate the corrected coordinates, and uses the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image; uses the Wiener filtering formula to calculate the frequency-domain true reflection image, uses the two-dimensional inverse fast Fourier transform to convert the frequency-domain true reflection image to the spatial domain to obtain the restored spatial domain image, uses point-by-point addition to calculate the pixel values of the enhanced image to generate the enhanced image, and uses color space conversion to map the mapped image to the RGB space to obtain the enhanced RGB image; improving the geometric accuracy and visual realism of the image and enhancing the quality and accuracy of the endoscope image. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0056] Figure 1 It is a flowchart of the image processing method for the endoscope in Embodiment 1.

[0057] Figure 2 It is a schematic diagram of the image processing system for the endoscope in Embodiment 1.

[0058] Figure 3 It is a schematic diagram of generating the final corrected image in Embodiment 1.

[0059] Figure 4 It is a flowchart of the enhanced RGB image in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will describe the specific embodiments of the present invention in detail with reference to the drawings of the specification.

[0061] Many specific details are set forth in the following description to facilitate a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0062] Second, the "one embodiment" or "embodiment" mentioned herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that excludes other embodiments.

[0063] Embodiment 1, referring to Figures 1 to 4 , is the first embodiment of the present invention. This embodiment provides an image processing method for an endoscope, including the following steps:

[0064] S1. Collect RGB images, use the Harris corner response function method to calculate the corner response value, filter out the feature points greater than the screening threshold of the corner response value, calculate the centroid of the edge image as the initial distortion center, add the displacement to the initial distortion center to calculate the adjusted distortion center, use the radial distortion model to calculate the distortion radial distance, use polar coordinate transformation to calculate the corrected coordinates, and use the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image;

[0065] Specifically, collecting RGB images and using polar coordinate transformation to calculate the corrected coordinates to generate the final corrected image includes:

[0066] Collect continuous RGB images through the CCD sensor of the endoscope device using the continuous frame acquisition method, and preprocess the continuous RGB images, including denoising using blur filtering and normalizing the RGB images;

[0067] Convert the RGB image to grayscale using the weighted average method, calculate the grayscale value to generate a grayscale image, use the Sobel operator to calculate the gradients in the horizontal and vertical directions of the grayscale image respectively, calculate the product of the gradients in the horizontal and vertical directions, construct the Harris response matrix, use the empirical parameter tuning method to set the empirical constant, and use the Harris corner response function method to calculate the corner response value. The formula is:

[0068] R(x,y) = det(M) - k·trace(M) 2 ,

[0069] where R(x,y) is the corner response value at the spatial coordinates (x,y) of the grayscale image, det(M) is the determinant of the Harris response matrix M, G x is the gradient in the horizontal direction, representing the intensity change of the image in the horizontal direction, G y is the gradient in the vertical direction, representing the intensity change of the image in the vertical direction, and trace(M) is the trace of the Harris response matrix M. k is an empirical constant used to balance the influence of the determinant and the trace;

[0070] Use the percentage threshold method to set the screening threshold, and screen the screening threshold greater than the corner response value as the feature points;

[0071] Extract the positions of the feature points in the corresponding grayscale image, set the neighborhood window using the sliding window method, encode the neighborhood window using the local binary pattern, calculate the LBP value, calculate the Hamming distance between adjacent LBP values using the Hamming distance, and use the K-nearest neighbor matching to select the matching pair with the smallest Hamming distance to generate the inter-frame matching relationship;

[0072] Extract the gradients of the first frame of the RGB image in the horizontal and vertical directions, and calculate the edge intensity. The formula is:

[0073]

[0074] where E 1 (x, y) is the edge intensity of the first-frame grayscale image at the spatial coordinates (x, y), is the gradient in the horizontal direction of the first frame, is the gradient in the vertical direction of the first frame;

[0075] Calculate the centroid of the edge image based on the edge intensity as the initial distortion center. The formula is:

[0076]

[0077] where and are the initial distortion center coordinates of the first-frame grayscale image, x and y are the spatial coordinates of the grayscale image;

[0078] Calculate the displacement of each pair of adjacent-frame feature points based on the inter-frame matching relationship, and calculate the sum of the absolute values of all displacements. The formula is:

[0079]

[0080] where Δx t and Δy t are the horizontal and vertical displacements of the matching feature points between the t-th frame and the (t + 1)-th frame, and are the coordinates of the feature points in the t-th frame, i represents the feature point index, t represents the frame index, and are the coordinates of the matching feature points in the (t + 1)-th frame, j represents the feature point index, t + 1 represents the frame index;

[0081] Add the displacement to the initial distortion center to calculate the adjusted distortion center, and use the arithmetic mean method to calculate the global distortion center;

[0082] Calculate the distance from the spatial coordinates of the frame grayscale image to the global distortion center using the Euclidean distance formula, denoted as the radial distance, and calculate the mean value of the radial distance. Use the arctangent function to calculate the angle of the spatial coordinates of the frame grayscale image relative to the center;

[0083] Use the linear approximation method to calculate the radial distortion coefficient. The formula is:

[0084]

[0085] where k1 is the radial distortion coefficient, S is the sum of absolute values, N is the total number of matching feature points, and r avg is the mean value of the radial distance;

[0086] Based on the radial distance, use the radial distortion model to calculate the distorted radial distance. The formula is:

[0087]

[0088] where is the distorted radial distance of the t-th frame, and r t is the radial distance of the t-th frame;

[0089] Subtract the distorted radial distance from the radial distance to calculate the distortion offset;

[0090] Use polar coordinate transformation to calculate the corrected coordinates. The formula is:

[0091] x ′t = x c +(r t + Δr t )cos(θ t ), y ′t = y c +(r t + Δr t )sin(θ t ),

[0092] where x′ t and y′ t are the corrected coordinates, x c and y c are the global distortion centers, Δr t is the distortion offset of the t-th frame, and θ t is the angle of the t-th frame;

[0093] Extract the positions of the corrected coordinates in the corresponding RGB image. Use the nearest neighbor search to select four nearest neighbor pixels. Use the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image.

[0094] Through the CCD sensor of the endoscope device, combined with the continuous frame acquisition method, continuously acquire real-time RGB images. This method can improve the efficiency and accuracy of image acquisition. By using the weighted average method for grayscale conversion, important structural information in the image can be retained while reducing noise in the image. The Harris corner response function method can effectively identify stable feature points in the image, greatly improving the accuracy and robustness of image matching. By calculating the displacement of feature points for each pair of adjacent frames, the present invention can efficiently estimate the dynamic changes of the image and further precisely adjust the distortion center of the image. Use the arithmetic mean method to calculate the global distortion center, and further optimize the distortion correction by calculating the radial distance and angle. Use the radial distortion model to accurately correct the image according to the calculated distortion radial distance, thereby eliminating image distortion caused by lens distortion. Interpolate the corrected coordinates by the bilinear interpolation method, which can not only effectively reconstruct the image but also avoid the jagged effect that may occur in traditional interpolation methods, thus providing a smoother and higher-quality image.

[0095] S2. Calculate the frequency-domain true reflection image using the Wiener filter formula, perform spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, calculate the pixel values of the enhanced image using point-by-point addition to generate the enhanced image, and map the mapped image to the RGB space using color space conversion to obtain the enhanced RGB image;

[0096] Specifically, calculating the frequency-domain true reflection image using the Wiener filter formula to obtain the enhanced RGB image includes:

[0097] Perform grayscale conversion on the final corrected image using the weighted average method, calculate the grayscale values to generate the enhanced image, calculate the geometric mean of the four-edge pixels of the enhanced image using the geometric mean method, and generate the illumination distribution using the linear interpolation method;

[0098] Calculate the illumination image using point-by-point division, and the formula is:

[0099]

[0100] where L corr (u, v) is the illumination image at the spatial coordinates (u, v), I(u, v) is the enhanced image at the spatial coordinates (u, v), u and v are the spatial coordinates of the enhanced image, and ε is a minimum value to avoid division by zero and suppress noise amplification;

[0101] Set the standard deviation of the Gaussian function using the moment estimation method, and calculate the point spread function using the two-dimensional Gaussian function formula. The formula is:

[0102]

[0103] where h λ (u, v) is the point spread function at the spatial coordinates (u, v), λ is the wavelength of light, and σ is the standard deviation of the Gaussian function;

[0104] Use two-dimensional fast Fourier transform to represent the illumination image and the point spread function in the frequency domain respectively. Use conjugate complex operation to calculate the frequency domain conjugate complex of the point spread function. Use modulus square operation to calculate the frequency domain modulus square of the illumination image. Use the fixed threshold method to set the noise power estimation constant. Use the Wiener filtering formula to calculate the frequency domain true reflection image. The formula is:

[0105]

[0106] where O freq (q, p) is the true reflection image at the frequency domain coordinates (q, p), ρ[L corr (u, v)] is the frequency domain modulus square of the illumination image, ρ[h λ * is the frequency domain conjugate complex of the point spread function, |ρ[h λ | 2 is the modulus square of the frequency domain of the point spread function, and K is the noise power estimation constant;

[0107] Use two-dimensional inverse fast Fourier transform to convert the frequency domain true reflection image to the spatial domain, and obtain the restored spatial domain image;

[0108] Use the Prewitt convolution kernel to convolve the normalized edge tensor field to obtain the gradients in the horizontal and vertical directions. Combine the gradients in the horizontal and vertical directions to obtain the gradient field;

[0109] Based on the spatial domain image, use the maximum value function to set the enhancement factor, use point-by-point addition to calculate the pixel values of the enhanced image, and generate the enhanced image. The formula is:

[0110]

[0111] where O enh (u, v) is the pixel value of the enhanced image at the spatial coordinates (u, v), O(u, v) is the restored spatial domain image at the spatial coordinates (u, v), α is the enhancement factor, is the gradient field at the spatial coordinates (u, v);

[0112] Use the threshold clipping method to limit the value range of the enhanced image. Define the pixel value of the enhanced image greater than 255 as 255, and define the pixel value of the enhanced image less than 0 as 0 to obtain the clipped enhanced image;

[0113] ​The non - linear gray - level mapping is performed on the cropped and enhanced image using the Sigmoid function, and the mapped image is mapped to the RGB space using color - space conversion to obtain the enhanced RGB image.

[0114] By performing Fourier transforms on the illumination image and the point - spread function, the influence of noise in the image can be minimized, thereby improving the clarity and quality of the image. The illumination distribution generated by the linear interpolation method further improves the illumination uniformity in the image, making the finally generated enhanced image present a smoother and more natural illumination transition, enhancing the structural details in the image, which is crucial for improving the edge clarity and detail capture in the image. Using the maximum - value function to set the enhancement factor ensures that the enhancement effect of the image is proportional to the overall structure and brightness of the image. Using the Sigmoid function for non - linear gray - level mapping can adjust the contrast and brightness of the image as needed, making the image more balanced visually. Through non - linear gray - level mapping and color - space conversion, the finally generated RGB image is more balanced and natural visually and has a wide range of application prospects.

[0115] S3. Extract the feature vectors of the enhanced RGB image, construct a convolutional neural network CNN model to calculate the class probabilities, and perform state detection;

[0116] Specifically, extracting the feature vectors of the enhanced RGB image includes:

[0117] Separate the enhanced RGB image using color - channel separation to generate three color channels, and calculate the mean and variance of the three color channels using statistical analysis methods;

[0118] Use the feature - plane splicing method to combine the mean and variance of the three color channels to generate feature vectors.

[0119] By calculating the mean and variance for each channel separately, the basic statistical features of the color channel can be obtained, which represents the central tendency of the color distribution in that channel, while the variance measures the amplitude of color variation. The combination of the two provides rich information for image features. Through the feature splicing method, the mean and variance values of the three color channels are combined into a unified feature vector, which can preserve the unique information of each channel while not losing the color distribution characteristics of the overall image. Compared with directly using the original image or large-scale pixel data, this feature vector not only improves the processing efficiency but also reduces the computational complexity. Especially in applications such as image recognition, object detection, and image classification, this feature vector can efficiently provide the core information of the image, helping the algorithm make quick and accurate decisions. Fusing the information of different color channels into a compact feature vector helps improve the performance of machine learning models in tasks such as image classification and recognition. During the training process of machine learning models, using these extracted feature vectors as data inputs can not only improve the training speed but also enhance the model's sensitivity to color patterns in images.

[0120] Furthermore, a Convolutional Neural Network (CNN) model is constructed to calculate the class probability and perform status detection, including:

[0121] Collect historical RGB images with labels and perform feature extraction, calculate the feature vector, and generate a training set;

[0122] The constructed Convolutional Neural Network (CNN) model includes an input layer, a convolutional layer, an activation function, a pooling layer, a fully connected layer, and an output layer:

[0123] Use the training set to train the Convolutional Neural Network (CNN) model, and use the cross-entropy loss function and the Adam optimizer to perform parameter iterative optimization;

[0124] Input the feature vector into the trained Convolutional Neural Network (CNN) model to output the class probability. Set the classification threshold using the empirical rule. When the class probability is greater than the classification threshold, it is judged as an abnormal state; when the class probability is less than or equal to the classification threshold, it is judged as a normal state.

[0125] By extracting representative features from images, the complexity of the problem can be effectively reduced, and the classification accuracy can be improved. A convolutional neural network (CNN) model is constructed, including an input layer, a convolutional layer, an activation function, a pooling layer, a fully connected layer, and an output layer. The convolutional layer is responsible for extracting local features in the image. The pooling layer retains the most important information by reducing the dimension of the feature map. The activation function introduces non-linearity to increase the expressive power of the model. The fully connected layer maps the extracted features to the final class output. By using the cross-entropy loss function and the Adam optimizer, the convolutional neural network can automatically adjust the parameters to achieve more accurate classification ability. The update process can continuously iterate and optimize the weights of the network, enabling the model to learn the correct patterns from the sample data in the training set and improving the classification ability and generalization ability of the model. Through multi-level feature extraction and deep learning, the convolutional neural network can automatically identify complex patterns in the image, improving the accuracy and robustness of classification.

[0126] S4. Construct a visual display of the status detection results and store the RGB images generated by collection and analysis.

[0127] Specifically, constructing a visual display of the status detection results includes:

[0128] Use the front-end framework React.js to build a visual interface and display the status detection results.

[0129] Allow users who have passed real-name verification to view.

[0130] The virtual DOM of React.js can improve the page rendering efficiency, reduce unnecessary DOM operations, and thus enhance the user experience. Especially in the scenario of real-time data updates, it can ensure smooth page response. Through React.js, the front-end interface can quickly respond to changes in back-end data and achieve real-time display and update of the status detection results. By setting strict identity verification and access control, it is ensured that each user has a clear identity authentication when performing the viewing operation, reducing the risk of data leakage. Through real-time and intuitive visual display, users can identify abnormal states in the system in the first time and make timely decisions.

[0131] Furthermore, storing the RGB images generated by collection and analysis includes:

[0132] Store the collected RGB images and the detection results generated by analysis in the central database, and set security access measures. The central database backs up the stored data to the cloud and regularly performs integrity detection on the stored data and the backup data. After the detection is completed, an integrity detection record is generated and synchronously stored in the central database.

[0133] By collecting RGB images and analyzing to generate detection results, the system can save relevant data in the central database. Security access measures include means such as identity authentication, access control, and encrypted communication, which are crucial in the medical industry, financial industry, and government data processing. It not only improves the security of data storage but also ensures compliance with relevant laws and industry standards. Cloud backup can provide disaster recovery capabilities in case of local hardware damage or natural disasters, ensuring data is not lost and providing high availability. Integrity detection technology can compare the original state and stored state of data by calculating hash values, checksums, etc., to ensure data accuracy. By generating integrity detection records and synchronously storing them in the central database, detailed log information can be provided for subsequent auditing, fault analysis, and optimization.

[0134] This embodiment also provides an image processing system for an endoscope, including:

[0135] A collection and calibration module, configured to collect RGB images, use the Harris corner response function method to calculate the corner response value, filter the values greater than the screening threshold of the corner response value as feature points, calculate the centroid of the edge image as the initial distortion center, add the displacement to the initial distortion center to calculate the adjusted distortion center, use the radial distortion model to calculate the distortion radial distance, use polar coordinate transformation to calculate the corrected coordinates, and use the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image;

[0136] An image enhancement module, configured to calculate the frequency-domain true reflection image using the Wiener filtering formula, perform spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, calculate the pixel values of the enhanced image using point-by-point addition to generate the enhanced image, and map the mapped image to the RGB space using color space conversion to obtain the enhanced RGB image;

[0137] A feature detection module, configured to extract the feature vectors of the enhanced RGB image, construct a convolutional neural network CNN model to calculate the class probability, and perform state detection;

[0138] A visualization and storage module, configured to construct a visualization to display the state detection results and store the RGB images generated by collection and analysis.

[0139] This embodiment also provides a computer device applicable to the image processing method of an endoscope, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the image processing method of the endoscope as proposed in the above embodiment.

[0140] The computer device may be a terminal, which includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covering the display screen, or a button, a trackball, or a touchpad provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0141] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for processing endoscopic images proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (abbreviated as SRAM), electrically erasable programmable read-only memory (abbreviated as EEPROM), erasable programmable read-only memory (abbreviated as EPROM), programmable read-only memory (abbreviated as PROM), read-only memory (abbreviated as ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0142] In summary, the present invention collects RGB images, uses the Harris corner response function method to calculate the corner response value, filters out the screening threshold greater than the corner response value as the feature points, calculates the centroid of the edge image as the initial distortion center, adds the displacement to the initial distortion center to calculate the adjusted distortion center, uses the radial distortion model to calculate the distortion radial distance, uses polar coordinate transformation to calculate the corrected coordinates, and uses the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image; uses the Wiener filtering formula to calculate the frequency-domain true reflection image, performs a spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, uses point-by-point addition to calculate the pixel values of the enhanced image to generate the enhanced image, and uses color space conversion to map the mapped image to the RGB space to obtain the enhanced RGB image; improving the geometric accuracy and visual realism of the image and enhancing the quality and accuracy of the endoscopic image.

[0143] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered within the scope of the claims of the present invention.

Claims

1. An image processing method for an endoscope, characterized in that: including, collecting RGB images, using the Harris corner response function method to calculate the corner response values, screening for the screening threshold greater than the corner response values as feature points, calculating the centroid of the edge image as the initial distortion center, adding the displacement to the initial distortion center to calculate the adjusted distortion center, using the radial distortion model to calculate the distorted radial distance, using polar coordinate transformation to calculate the corrected coordinates, and using the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image; using the Wiener filtering formula to calculate the frequency-domain true reflection image, performing spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, using point-by-point addition to calculate the pixel values of the enhanced image to generate the enhanced image, and using color space conversion to map the mapped image to the RGB space to obtain the enhanced RGB image; extracting the feature vectors of the enhanced RGB image, constructing a convolutional neural network CNN model to calculate the class probabilities, and performing state detection; constructing a visual display of the state detection results and storing the RGB images generated by the collection and analysis.

2. The image processing method of the endoscope according to claim 1, characterized in that: The collecting of RGB images, using polar coordinate transformation to calculate the corrected coordinates and generating the final corrected image includes: collecting consecutive RGB images by using the continuous frame acquisition method through the CCD sensor of the endoscope device; performing gray-scale conversion on the RGB images using the weighted average method, calculating the gray-scale values to generate a gray-scale image, using the Sobel operator to calculate the gradients in the horizontal and vertical directions of the gray-scale image respectively, calculating the product of the gradients in the horizontal and vertical directions, constructing the Harris response matrix, setting the empirical constant using the empirical parameter tuning method, and using the Harris corner response function method to calculate the corner response values; setting the screening threshold using the percentage threshold method and screening for the screening threshold greater than the corner response values as feature points; extracting the positions of the feature points in the corresponding gray-scale image, setting the neighborhood window using the sliding window method, encoding the neighborhood window using the local binary pattern to calculate the LBP values, calculating the Hamming distance between adjacent LBP values using the Hamming distance, and using the K-nearest neighbor matching to select the matching pair with the minimum Hamming distance to generate the inter-frame matching relationship; extracting the gradients in the horizontal and vertical directions of the first frame of the RGB image, calculating the edge intensity, and calculating the centroid of the edge image based on the edge intensity as the initial distortion center; calculating the displacement of each pair of adjacent frame feature points based on the inter-frame matching relationship and calculating the absolute value sum of all displacements; adding the displacement to the initial distortion center to calculate the adjusted distortion center and calculating the global distortion center using the arithmetic mean method; calculating the distance from the spatial coordinates of the frame gray-scale image to the global distortion center using the Euclidean distance formula, denoted as the radial distance, and calculating the mean of the radial distances, and using the arctangent function to calculate the angle of the spatial coordinates of the frame gray-scale image relative to the center; using the linear approximation method to calculate the radial distortion coefficient, and based on the radial distance, using the radial distortion model to calculate the distorted radial distance; subtracting the distorted radial distance from the radial distance to calculate the distortion offset, and using polar coordinate transformation to calculate the corrected coordinates; Extract the positions of the corrected coordinates in the corresponding RGB image, select four nearest neighbor pixels using nearest neighbor search, and interpolate the corrected coordinates using the bilinear interpolation formula to generate the final corrected image.

3. The image processing method of the endoscope according to claim 2, characterized in that: The calculation of the frequency domain true reflection image using the Wiener filtering formula to obtain the enhanced RGB image includes: Convert the final corrected image to grayscale using the weighted average method, calculate the grayscale values to generate the enhanced image, calculate the geometric mean of the four edge pixels of the enhanced image using the geometric mean method, and generate the illumination distribution using the linear interpolation method; Calculate the illumination image using the point-by-point division method; Set the standard deviation of the Gaussian function using the moment estimation method and calculate the point spread function using the two-dimensional Gaussian function formula; Perform frequency domain representation of the illumination image and the point spread function respectively using the two-dimensional fast Fourier transform, calculate the frequency domain conjugate complex number of the point spread function using conjugate complex number operation, calculate the frequency domain modulus square of the illumination image using the modulus square operation, set the noise power estimation constant using the fixed threshold method, and calculate the frequency domain true reflection image using the Wiener filtering formula; Perform spatial domain conversion on the frequency domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial domain image; Convolve the normalized edge tensor field using the Prewitt convolution kernel to obtain the gradients in the horizontal and vertical directions, and combine the gradients in the horizontal and vertical directions to obtain the gradient field; Based on the spatial domain image, set the enhancement factor using the maximum value function, calculate the pixel values of the enhanced image using the point-by-point addition method to generate the enhanced image; Limit the value range of the enhanced image using the threshold clipping method, define the pixel values of the enhanced image greater than 255 as 255, and define the pixel values of the enhanced image less than 0 as 0 to obtain the clipped enhanced image; Perform non-linear grayscale mapping on the clipped enhanced image using the Sigmoid function and map the mapped image to the RGB space using color space conversion to obtain the enhanced RGB image.

4. The image processing method of the endoscope according to claim 3, characterized in that: The extraction of the feature vectors of the enhanced RGB image includes: Separate the enhanced RGB image using color channel separation to generate three color channels, and calculate the mean and variance of the three color channels using the statistical analysis method; Combine the mean and variance of the three color channels using the feature plane splicing method to generate the feature vectors.

5. The image processing method of the endoscope according to claim 4, characterized in that: The construction of the convolutional neural network CNN model to calculate the class probability and perform status detection includes: Collect historical RGB images with labels and perform feature extraction, calculate the feature vectors to generate the training set; The construction of the convolutional neural network CNN model includes an input layer, a convolutional layer, an activation function, a pooling layer, a fully connected layer, and an output layer: Use the training set to train the convolutional neural network CNN model, and use the cross-entropy loss function and the Adam optimizer to perform parameter iterative optimization; Input the feature vectors into the trained convolutional neural network CNN model to output the class probability, set the classification threshold using the empirical rule, when the class probability is greater than the classification threshold, judge it as an abnormal state, and when the class probability is less than or equal to the classification threshold, judge it as a normal state.

6. The image processing method of the endoscope according to claim 5, characterized in that: The construction of the visualization to display the status detection results includes: Build a visualization interface using the front-end framework React.js to display the status detection results; Allow users who have passed real-name verification to view.

7. The image processing method of the endoscope according to claim 6, characterized in that: The storage of the RGB images generated by collection and analysis includes: Store the collected RGB images and the detection results generated by analysis in the central database, and set security access measures. The central database backs up the stored data to the cloud, and regularly performs integrity detection on the stored data and the backup data. After the detection is completed, an integrity detection record is generated and synchronously stored in the central database.

8. An image processing system for an endoscope, based on the image processing method for an endoscope according to any one of claims 1 to 7, characterized in that: Including, A collection and correction module for collecting RGB images, using the Harris corner response function method to calculate the corner response value, filtering the screening threshold greater than the corner response value as feature points, calculating the centroid of the edge image as the initial distortion center, adding the displacement to the initial distortion center to calculate the adjusted distortion center, using the radial distortion model to calculate the distortion radial distance, using polar coordinate transformation to calculate the corrected coordinates, and using the bilinear interpolation formula to interpolate the corrected coordinates to generate the final corrected image; An image enhancement module for calculating the frequency-domain true reflection image using the Wiener filtering formula, performing spatial-domain conversion on the frequency-domain true reflection image using the two-dimensional inverse fast Fourier transform to obtain the restored spatial-domain image, calculating the pixel values of the enhanced image using point-by-point addition to generate the enhanced image, and mapping the mapped image to the RGB space using color space conversion to obtain the enhanced RGB image; A feature detection module for extracting the feature vectors of the enhanced RGB image, constructing a convolutional neural network CNN model to calculate the class probability, and performing status detection; A visualization storage module for constructing a visualization to display the status detection results and storing the RGB images generated by collection and analysis.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the image processing method of the endoscope according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the image processing method of the endoscope according to any one of claims 1 to 7.

Images