Neurosurgery intraoperative lesion recognition system based on ultrasonic image enhancement technology

Abstract

The invention discloses a neurosurgery intraoperative lesion recognition system based on an ultrasonic image enhancement technology, and the system comprises an image collection module which is represented as a two-dimensional function I; the self-adaptive median filtering module is used for converting into an output image J; the gray level normalization module is used for generating an output image K; the multi-scale Retinex algorithm enhancement module is used for obtaining an enhanced output image E; the histogram equalization module is used for carrying out histogram equalization on the output image E; the feature extraction module is used for carrying out feature extraction based on a convolutional neural network CNN; and the classification module is used for obtaining an ultrasonic image at the focus. Therefore, noise can be suppressed through adaptive median filtering, contrast and definition can be enhanced through gray level normalization and histogram equalization, lesion features are highlighted through a multi-scale Retinex algorithm, and lesion features are accurately extracted and classified through combination of a convolutional neural network and a support vector machine.

Description

Technical Field

[0001] The present invention relates to the field of medical information technology, and particularly to a neurosurgical intraoperative lesion recognition system based on ultrasonic image enhancement technology. Background Art

[0002] In the field of neurosurgery, ultrasonic imaging technology has been widely used during surgery due to its advantages of good real-time performance, convenient operation, and no radiation, and can provide doctors with instant image assistance for judgment.

[0003] However, there are certain limitations in this technology at present. Ultrasonic images are easily interfered by noise. These noise sources are diverse, such as the equipment itself, human tissue scattering, and surgical environment factors. The noise will blur the image details and make it difficult to distinguish the lesion boundaries. For example, small lesions may be difficult to accurately identify due to noise interference. At the same time, the contrast and clarity of ultrasonic images are poor. The differences in the reflection and absorption characteristics of different tissues to ultrasound are small, resulting in an insignificant gray-scale difference between the lesion tissue and the normal tissue in the image, making it difficult for doctors to judge the scope and nature of the lesion and affecting the determination of the surgical resection boundary.

[0004] Existing image enhancement technologies and lesion recognition systems have deficiencies in dealing with neurosurgical ultrasonic images. Traditional enhancement algorithms, such as simple gray-scale transformation and histogram equalization, have limited enhancement effects, may over-enhance noise and generate artifacts. Advanced deep learning methods face problems of difficult model training and insufficient generalization ability due to the small amount and uneven quality of ultrasonic image data. The recognition method based on manual features has high requirements for image quality, poor accuracy and robustness, and deep learning recognition methods are prone to overfitting, and their reliability in actual surgery needs to be improved. Summary of the Invention

[0005] The present invention aims to solve at least to some extent the technical problems in the above-mentioned technologies.

[0006] To this end, the present invention discloses a neurosurgical intraoperative lesion recognition system based on ultrasonic image enhancement technology, including:

[0007] An image acquisition module, configured to acquire lesion images and represent them as a two-dimensional function I(x, y) of M×N, where x∈[1, M], y∈[1, N];

[0008] An adaptive median filtering module, configured to set a filtering window W centered on the coordinate (x, y) and calculate the median z med of the pixels within the window, the minimum value z min and the maximum value z max , and convert the two-dimensional function I(x, y) into an output image J(x, y);

[0009] The grayscale normalization module is used to set the grayscale range [J min , J max of the output image J(x, y), and normalize the output image J(x, y) to generate the output image K(x, y);

[0010] The multi-scale Retinex algorithm enhancement module is used to, based on the image imaging reflection model K(x, y) = R(x, y)L(x, y), convolve the image K(x, y) with Gaussian functions G(x, y, σ i ) of different scales to calculate the illumination component L i (x, y), and calculate the reflection component R i (x, y) = logK(x, y) - logL i (x, y), and finally perform weighted fusion on the reflection components at multiple scales to obtain the enhanced output image

[0011] The histogram equalization module is used to perform histogram equalization on the output image E(x, y) to obtain the output image X(x, y);

[0012] The feature extraction module is used to input the output image X(x, y) and perform feature extraction based on the convolutional neural network CNN. Among them, the convolutional kernel of the convolutional layer is W ij , and the bias is b i , and the output Y i = f(∑ j W ij ×X + b i );

[0013] The classification module is used to input the extracted feature vector into the support vector machine SVM for classification to obtain the ultrasonic image at the lesion site.

[0014] According to the intraoperative lesion recognition system for neurosurgery based on ultrasonic image enhancement technology disclosed by the present invention, it can suppress noise through adaptive median filtering, enhance contrast and clarity through grayscale normalization and histogram equalization, highlight lesion features through the multi-scale Retinex algorithm, and accurately extract and classify lesion features by combining the convolutional neural network with the support vector machine, providing a reliable basis for neurosurgery and improving the success rate and safety of the surgery.

[0015] In addition, the intraoperative lesion recognition system for neurosurgery based on ultrasonic image enhancement technology disclosed by the present invention may also have the following additional technical features:

[0016] In an embodiment of the present invention, in the image acquisition module, the matrix element I mn corresponds to the coordinate (xm , x n ), the grayscale value of the image or the ultrasonic echo intensity value at

[0017] In an embodiment of the present invention, in the adaptive median filtering module, the initial window size of the filtering window W is set to (2k + 1)×(2k + 1), (k = 1), and the pixel set S within the window xy = I(u, v): (x - k ≤ u ≤ x + k, y - k ≤ v ≤ y + k)}.

[0018] In an embodiment of the present invention, the value of the output image J(x, y) is determined according to the following rules. Specifically, If the current window does not satisfy z min < z med < z max , then increase the value of k, recalculate the statistics within the window until the condition is satisfied or k reaches the preset maximum value.

[0019] In an embodiment of the present invention, in the grayscale normalization module, the value of the output image K(x, y) is determined according to the following rules. Specifically, Grayscale normalization maps the grayscale value of the image to the interval [0, 1].

[0020] In an embodiment of the present invention, in the multi-scale Retinex algorithm enhancement module, the illumination component L i (x, y) is determined according to the following rules. Specifically, Among them, the Gaussian function

[0021] In an embodiment of the present invention, in the output image Among them, ω i is the weight, and

[0022] In an embodiment of the present invention, in the histogram equalization module, the number of pixels with the gray level k in the output image E(x, y) is n k , the total number of pixels in the image is N, and the probability density function of the gray level k Cumulative distribution function The gray level s of the image after histogram equalization k = (L - 1)c(k).

[0023] In an embodiment of the present invention, in the feature extraction module, the output Y of the convolutional layer i = f(∑ j W ij × X + bi )。

[0024] In one embodiment of the present invention, the pooling window size is (2p + 1) × (2p + 1).

[0025] Additional content and advantages of the present invention will be given in the following description, or understood through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The technical solutions and beneficial effects of the present invention will become obvious and easy to understand from the following content in conjunction with the accompanying drawings, where:

[0027] Figure 1 is a system block diagram of a neurosurgical intraoperative lesion recognition system based on ultrasound image enhancement technology of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments.

[0029] The following will describe a neurosurgical intraoperative lesion recognition system based on ultrasound image enhancement technology disclosed in the present invention with reference to the accompanying drawings.

[0030] As Figure 1 shown, for the image acquisition module, the ultrasound probe collects images in real time during neurosurgery. The collected image data is mathematically represented as a two-dimensional function I(x, y), and is actually stored as a discrete M × N matrix, where x ∈ [1, M], y ∈ [1, N], and the matrix element I mn corresponds to the image gray value or ultrasound echo intensity value at the coordinates (x m , x n );

[0031] Specifically, for superficial brain lesions, a high-frequency linear array probe can provide higher resolution and clearly present the fine structure of the lesions. For lesions in deeper positions of the brain, a low-frequency convex array probe needs to be selected to ensure that the ultrasound signal can penetrate deep enough and obtain clear images;

[0032] In addition, during the image acquisition process, in order to obtain more comprehensive and accurate lesion information, it may be necessary to scan the lesion from multiple angles.

[0033] For the adaptive median filtering module, since the original collected image may be subject to noise interference, adaptive median filtering is first performed. For the image I(x, y), a filtering window W is set with the coordinates (x, y) as the center, and the initial window size is set to (2k + 1) × (2k + 1), (k = 1). The pixel set S within the window xy= I(u, v): (x - k ≤ u ≤ x + k, y - k ≤ v ≤ y + k)}, calculate the median z of the pixels within the window med , the minimum value z min and the maximum value z max , the value of the output image J(x, y) is determined according to the following rules. Specifically, If the current window does not satisfy z min < z med < z max , then increase the value of k and recalculate the statistics within the window until the condition is satisfied or k reaches the preset maximum value. After adaptive median filtering, the noise in the image is initially suppressed;

[0034] Specifically, the setting of the preset maximum value needs to comprehensively consider the resolution and noise situation of the image. If the preset maximum value is too small, the noise may not be fully removed. If the preset maximum value is too large, the computational amount will increase, and the image may be over-smoothed, losing some details. Therefore, relevant staff can determine a more appropriate preset maximum value through multiple experiments, combined with the processing effects of different types of noise images. For example, for images with a high noise level and high resolution, the preset maximum value can be appropriately increased. For images with a low noise level and high requirements for details, the preset maximum value should be relatively decreased.

[0035] For the gray-scale normalization module, perform gray-scale normalization on the filtered image J(x, y). Assume its gray-scale range is [J min , J max . The normalized image Gray-scale normalization maps the gray-scale values of the image to the [0, 1] interval, enhancing the contrast of the image;

[0036] Specifically, to improve the computational efficiency, the method of block calculation can be adopted. Divide the image into multiple small blocks, and calculate [J min , J max for each small block respectively, and then comprehensively obtain [J min , J max of the entire image.

[0037] For the multi-scale Retinex algorithm enhancement module, then use the multi-scale Retinex algorithm to further enhance the image. Based on the image imaging reflection model I(x, y) = R(x, y)L(x, y), use Gaussian functions G(x, y, σ i ) with different scales to convolve with the image K(x, y) to estimate the illumination component L i (x, y). Specifically, Among them, the Gaussian function Then calculate the reflection component Ri (x, y) = logK(x, y) - logL i (x, y), and finally, the reflection components at multiple scales are weighted and fused to obtain the enhanced image where ω i is the weight, and The multi-scale Retinex algorithm can effectively enhance the local details and overall contrast of the image, making the features of the lesion site more prominent;

[0038] Specifically, relevant staff can use machine learning methods, such as the least squares method and the gradient descent method, through training samples, to optimize the weight according to the contribution degree of the reflection components at different scales to the enhancement of lesion features, so that the finally enhanced image can highlight the lesion features to the greatest extent while maintaining the natural appearance of the image

[0039] For the histogram equalization module, the image E(x, y) is further subjected to histogram equalization. Let the range of the image gray level be [0, L - 1], the number of pixels with gray level k in the image be n k , the total number of pixels in the image be N, and the probability density function of the gray level k cumulative distribution function The gray level s of the image after histogram equalization k =(L - 1)c(k), mapping the original image gray level k to s k to obtain the histogram equalized image, further stretching the gray dynamic range of the image and improving the image clarity;

[0040] Specifically, if the gray distribution of the original image is concentrated in a certain small interval, after histogram equalization, the gray values will be stretched to a wider range, improving the contrast and clarity of the image. For example, for an image that is originally concentrated in the low gray area, after histogram equalization, the details in the low gray area will be more clearly displayed, and at the same time, the visual effect of the entire image will be improved, which is more conducive to doctors to observe and identify lesions.

[0041] For the feature extraction module, the enhanced image is input into the convolutional neural network CNN for feature extraction. Let the input image of the convolutional neural network be X (i.e., the image after histogram equalization), the convolutional kernel of the convolutional layer be W ij , and the bias be b i , and the output Y after passing through the convolutional layer i =f(∑ j W ij ×X + b i), where f is an activation function, such as the ReLU function f(x) = max(0, x). Assume that after passing through m convolutional layers and l pooling layers, the feature map F is obtained. In the pooling layer (such as max pooling), assume the pooling window size is (2p + 1)×(2p + 1), centered at the coordinates (x', y'), and the output after pooling is P(x', y') = max{F(u, v): (x' - p ≤ u ≤ x' + p, y' - p ≤ v ≤ y' + p)}. After being processed by the convolutional neural network CNN, the lesion features of the image are extracted to form a feature vector;

[0042] Specifically, relevant staff can set parameters such as the size, number, and stride of the convolutional kernel. According to details such as the edges and textures of the lesions, smaller convolutional kernels can capture the local detailed features of the image, while larger convolutional kernels are more suitable for the overall structural features of the image.

[0043] For the classification module, the extracted feature vector is input into the support vector machine SVM for classification. For a binary classification problem, the goal of the support vector machine SVM is to find the optimal hyperplane ω T x + b = 0, which maximizes the distances from the two types of samples to the hyperplane. Assume the training sample set is (x 1 , y 1 ), (x 2 , y 2 ), …, (x n , y n ), where x i is the feature vector and y i ∈ {-1, 1} is the class label. The optimal ω and b are obtained by solving the following optimization problem. Specifically, s.t. y i (ω T x i + b) ≥ 1 - ξ i , ξ i ≥ 0, where C is the penalty parameter and ξ i is the slack variable.

[0044] Specifically, in practical applications, the appropriate C value can be selected through the cross - validation method. For example, divide the training samples into k parts, each time select one part as the test set, and the remaining k - 1 parts as the training set. Train the model and calculate the accuracy on the test set. After repeating k times, take the average accuracy. By trying different C values, select the C that maximizes the average accuracy as the final parameter.

[0045] In summary, according to the neurosurgical intraoperative lesion recognition system based on ultrasonic image enhancement technology disclosed by the present invention, it can suppress noise through adaptive median filtering, enhance contrast and clarity through gray normalization and histogram equalization, highlight lesion features by the multi-scale Retinex algorithm, and accurately extract and classify lesion features by combining convolutional neural network with support vector machine, providing a reliable basis for neurosurgical operations and improving the success rate and safety of operations.

[0046] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by terms such as "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation to the present invention.

[0047] In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, "a plurality of" means two or more, unless otherwise specifically defined.

[0048] In the present invention, unless otherwise clearly defined and limited, terms such as "installed", "connected", "connected with", "fixed" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the internal connection of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0049] In the present invention, unless otherwise clearly defined and limited, the first feature being "on" or "under" the second feature may be that the first and second features are in direct contact, or the first and second features are indirectly in contact through an intermediate medium. Moreover, the first feature being "above", "over" and "on the top of" the second feature may be that the first feature is directly above or obliquely above the second feature, or merely indicates that the horizontal height of the first feature is higher than that of the second feature. The first feature being "under", "beneath" and "under the bottom of" the second feature may be that the first feature is directly below or obliquely below the second feature, or merely indicates that the horizontal height of the first feature is lower than that of the second feature.

[0050] In the description of this specification, the descriptions referring to terms such as "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc. mean that the specific features, structures, materials, or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0051] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A neurosurgery intraoperative lesion recognition system based on ultrasonic image enhancement technology, characterized in that: include: An image acquisition module is used to acquire lesion images and express them as a two-dimensional function I(x, y) of M×N, where x∈[1, M], y∈[1, N]; The adaptive median filter module is used to set the filter window W with the coordinate (x, y) as the center and calculate the median z of the pixels in the window med , minimum value z min and the maximum value z max , converting the two-dimensional function I(x, y) into an output image J(x, y); Grayscale normalization module, used to set the grayscale range of the output image J(x, y) [J min , J max ], and normalize the output image J(x, y) to generate an output image K(x, y); Multi-scale Retinex algorithm enhancement module is used to use Gaussian functions G(x, y, σ) of different scales based on the image imaging reflection model K(x, y) = R(x, y)L(x, y) i ) is convolved with the image K(x, y) to calculate the illumination component L i (x, y), and calculate the reflection component R i (x, y) = logK(x, y) - logL i (x, y), and finally the reflection components at multiple scales are weighted fused to obtain the enhanced output image A histogram equalization module, used for performing histogram equalization on the output image E(x, y) to obtain an output image X(x, y); The feature extraction module is used to input the output image X(x, y) and perform feature extraction based on the convolutional neural network CNN, wherein the convolution kernel of the convolution layer is W ij , with a bias of b i , the output Y after the convolution layer i =f(∑ j W ij ×X+b i ); The classification module is used to input the extracted feature vector into the support vector machine (SVM) for classification to obtain an ultrasound image of the lesion.

2. The neurosurgery intraoperative lesion recognition system based on ultrasonic image enhancement technology according to claim 1, characterized in that: In the image acquisition module, the matrix element I mn The corresponding coordinates (x m , x n ) is the image grayscale value or ultrasonic echo intensity value at .

3. The neurosurgery intraoperative lesion identification system based on ultrasonic image enhancement technology according to claim 1, characterized in that: In the adaptive median filter module, the initial window size of the filter window W is set to (2k+1)×(2k+1), (k=1), and the pixel set S in the window xy ={I(u, v): (xk≤u≤x+k, yk≤v≤y+k)}.

4. The neurosurgery intraoperative lesion recognition system based on ultrasonic image enhancement technology as claimed in claim 3, characterized in that: The value of the output image J(x, y) is determined according to the following rules, specifically, If the current window does not satisfy z min <z med <z max , then increase the k value and recalculate the statistics in the window until the condition is met or k reaches the preset maximum value.

5. The neurosurgery intraoperative lesion recognition system based on ultrasonic image enhancement technology according to claim 1, characterized in that: In the grayscale normalization module, the value of the output image K(x, y) is determined according to the following rules, specifically, Grayscale normalization maps the image grayscale value to the [0, 1] interval.

6. The neurosurgery intraoperative lesion identification system based on ultrasonic image enhancement technology according to claim 1, characterized in that: In the multi-scale Retinex algorithm enhancement module, the illumination component L i The value of (x, y) is determined according to the following rules, specifically, Among them, the Gaussian function 7. The neurosurgery intraoperative lesion identification system based on ultrasonic image enhancement technology according to claim 6, characterized in that: In the output image Among them, ω i is the weight, and 8. The neurosurgery intraoperative lesion identification system based on ultrasonic image enhancement technology according to claim 1, characterized in that: In the histogram equalization module, the number of pixels with gray level k in the output image E(x, y) is n. k , the total number of image pixels is N, the probability density function of gray level k Cumulative distribution function Image gray level s after histogram equalization k =(L-1)c(k).

9. The neurosurgery intraoperative lesion recognition system based on ultrasonic image enhancement technology according to claim 1, characterized in that: In the feature extraction module, the output Y of the convolutional layer i =f(∑ j W ij ×X+b i ).

10. The neurosurgery intraoperative lesion identification system based on ultrasonic image enhancement technology according to claim 9, characterized in that: The pooling window size is (2p+1)×(2p+1).

Images