Medical image focus detection and classification method based on machine vision

Abstract

The invention relates to the technical field of medical treatment, in particular to a medical image focus detection and classification method based on machine vision, and the method comprises the following steps: S1, image preprocessing; s2, focus detection and evaluation; s3, model training and verification; according to the method, deep features of input data are automatically learned through a multilayer structure, local features such as colors and geometric shapes can be learned through a shallow network, more abstract features such as attributes and outlines of objects can be learned through a deep network, convolution operation is only carried out in a local area, the calculation amount of a model is greatly reduced, and the calculation efficiency is improved. The method is suitable for a large-scale data set, detection and classification of the focus can still be correctly carried out through autonomous learning when new and unseen data is faced, and a doctor can conveniently provide a treatment scheme better.

Description

Technical Field

[0001] The present invention relates to the field of medical technology, and in particular to a medical image lesion detection and classification method based on machine vision. Background Art

[0002] With the continuous development of machine vision technology, medical images provide doctors with a wealth of information about the patient's body, thereby assisting doctors in improving the accuracy of disease diagnosis, so that they can quickly complete the determination and treatment of the disease. In modern medicine, the accuracy and reliability of lesion detection are directly related to the patient's health and treatment effect.

[0003] With the rapid development of computer technology, the use of computers to analyze and process medical images can greatly improve the accuracy and efficiency of lesion detection, thereby providing more accurate diagnosis and treatment plans. However, existing detection and classification methods mainly rely on past data and directly compare images with previous data to detect and classify lesions. They do not have the ability to learn independently and are unable to correctly detect and classify new, unseen data.

[0004] Therefore, in order to address the above-mentioned problem of lack of autonomous learning ability, a medical image lesion detection and classification method based on machine vision can be designed. Summary of the invention

[0005] In order to overcome the problem of lack of independent learning ability.

[0006] The technical solution of the present invention is: a medical image lesion detection and classification method based on machine vision, and the steps are as follows:

[0007] S1: Image preprocessing

[0008] S11: grayscale processing;

[0009] S12: Noise removal, the methods used include smoothing filtering, wavelet transform, non-local mean filtering and partial differential equation method;

[0010] S13: Image enhancement, using methods including histogram equalization, spatial filtering, edge enhancement, and multi-scale transformation;

[0011] S14: Registration and Alignment

[0012] S141: Select an appropriate transformation model;

[0013] S142: Selecting a suitable feature space, based on grayscale or based on features;

[0014] S143: Determine the range of possible parameter changes according to the parameter configuration of the transformation model and the selected features, and select the optimal search strategy;

[0015] S144: applying the similarity measure to search in the search space according to the optimization criterion to find the maximum correlation point, thereby solving the unknown parameters in the transformation model;

[0016] S145: Corresponding the image to be registered to the reference image according to the transformation model to achieve matching between images;

[0017] S15: Standardization processing, including determining the standardization target, extracting image features, converting or adjusting, and applying standardized methods to perform standardization processing;

[0018] S16: size normalization;

[0019] S17: Segmentation

[0020] S171: Using the FCM algorithm to calculate the probability value of each pixel belonging to each category;

[0021] S172: Determine the target boundary using the image gradient, traverse the image segmented by the FCM algorithm, and mark the pixel points belonging to the edge;

[0022] S173: Find the interference area and delete it by region growing method;

[0023] S18: edge detection, the edge detection algorithm adopts any one of the Sobel operator, Prewitt operator and Canny operator;

[0024] S2: Lesion detection and assessment

[0025] S21: Extract the features of the lesion image through the convolution layer. The convolution layer uses multiple convolution kernels to perform a sliding window operation on the input image. Each convolution kernel generates a feature map. These feature maps contain the local feature information of the image. The convolution operation is mathematically expressed as: C(i, j) = (I*H)(i, j) = ∑ x ∑ y I(i+m,j+n) H(x,y), where C represents the convolution operation, I represents the input image, H represents the convolution kernel, (i,j) represents the position of the feature map, and x and y represent the dimensions of the convolution kernel;

[0026] S22: The pooling layer is used to reduce the dimension of the feature map, reduce the amount of calculation and improve the generalization ability of the model. Common pooling methods include maximum pooling and average pooling. Maximum pooling selects the maximum value of each area, while average pooling calculates the average value of each area.

[0027] S23: After passing through multiple convolutional layers and pooling layers, the feature map is flattened and input into the fully connected layer, which classifies the extracted features and outputs the final classification result;

[0028] S3: Model training and validation

[0029] S31: Initialize the network weights;

[0030] S32: The input data is forward propagated through the convolution layer, pooling layer, and fully connected layer to obtain the output value;

[0031] S33: Calculate the error between the output value of the network and the target value;

[0032] S34: When the error is greater than our expected value, the error is passed back to the network, and the errors of the fully connected layer, pooling layer, and convolutional layer are obtained in turn. When the error is equal to or less than the expected value, the training ends;

[0033] S35: Update the weights according to the obtained error, and then proceed to S32.

[0034] Preferably, the grayscale processing method includes:

[0035] (1) Quantization: The brightness of the three components in the color image is used as the grayscale values ​​of the three grayscale images. One grayscale image can be selected according to the application requirements:

[0036] Gray 1 (i, j) = R(i, j)

[0037] Gray 2 (i, j) = G(i, j)

[0038] Gray 3 (i, j) = B(i, j)

[0039] (2) Maximum value method: the maximum value of the three-component brightness in the color image is used as the grayscale 1 value of the grayscale image:

[0040] Gray(i,j)=max{R(i,j),G(i,j),B(i,j)}

[0041] (3) Average value method: average the three brightness components in the color image to obtain a gray value:

[0042] Gray(i,j)=(R(i,j)+G(i,j)+B(i,j)) / 3

[0043] (4) Weighted average method: according to the importance and other indicators, the three components are weighted averaged with different weights. The weighted average of the three RGB components can be used to obtain a more reasonable grayscale image.

[0044] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j).

[0045] Preferably, histogram equalization redistributes the grayscale levels of image pixels to make the histogram of the image evenly distributed, and is often used to enhance X-ray, CT scan and MRI images; spatial filtering removes noise and unnecessary details in the image by smoothing the image pixels, thereby improving the quality and clarity of the image. Spatial filtering is applied to denoising and smoothing in medical images; edge enhancement separates the edge area from the background area through gradient calculation or multi-threshold processing, making the edge clearer and improving the recognition and analysis of tumors and vascular structures; multi-scale transformation separates the edge area from the background area through gradient calculation or multi-threshold processing, making the edge clearer and improving the recognition and analysis of tumors and vascular structures.

[0046] Preferably, the objectives of standardization include but are not limited to the range, mean and variance of pixel values; the extracted image features include histogram and grayscale; the conversion or adjustment adopts linear or nonlinear conversion; the standardization method is local or global standardization.

[0047] As a preferred embodiment, the specific steps of the FCM algorithm are:

[0048] (1) Determine the number of clusters k and the iteration termination condition. The user needs to determine the number of categories k to be divided and set the iteration termination condition, which is generally the maximum number of iterations allowed or the minimum change in the cluster center point.

[0049] (2) Initialize the membership matrix U. The size of the membership matrix U is (n, k), where n is the number of samples and k is the number of categories. Each element in the membership matrix U represents the probability that the sample belongs to some category. Initially, the value can be randomly assigned or assigned according to certain rules.

[0050] (3) Calculate the coordinates of the center point of each category. According to the membership matrix U, the coordinates of the center point of each category can be calculated. Usually, the “weighted average method” is used for calculation.

[0051] (4) Update the membership matrix U. By calculating the distance between the sample and the center point of the category, the membership matrix U can be updated so that the membership degree of each sample belonging to each category meets the requirements;

[0052] (5) Determine whether the iteration is terminated, compare the change between the current membership matrix U and the previous membership matrix U, if it is less than the set termination condition, stop the iteration; otherwise, return to step (3) and continue the iteration;

[0053] (6) Output the clustering results. The final membership matrix U can be used to determine which category each sample belongs to and divide the samples into corresponding categories.

[0054] Preferably, the Sobel operator performs convolution operations on each pixel of the image and its neighborhood, and calculates the gradients in the horizontal direction and the vertical direction respectively. By combining the gradients in these two directions, the strength and direction of the edge in the image can be obtained.

[0055] Preferably, the Prewitt operator consists of the following two convolution kernels:

[0056]

[0057] As a preference, the steps of the Canny operator are as follows:

[0058] (1) Use Gaussian filter to smooth the image and reduce noise interference;

[0059] (2) Calculate the gradient magnitude and direction of each pixel in the image;

[0060] (3) Apply non-maximum suppression to refine the edge and reduce the edge width to a single pixel;

[0061] (4) Apply a dual threshold algorithm to mark edge pixels as weak edges or strong edges;

[0062] (5) Track and connect weak edge pixels to form a complete edge.

[0063] The beneficial effects of the present invention are as follows: the deep features of the input data can be automatically learned through a multi-layer structure. The shallow network can learn local features, such as color, geometric shape, etc., while the deep network can learn more abstract features, such as object attributes, contours, etc. The convolution operation is only performed in the local area, which greatly reduces the computational complexity of the model and is suitable for large-scale data sets. Through autonomous learning, when faced with new and unseen data, lesions can still be correctly detected and classified, which facilitates doctors to better provide treatment plans. DETAILED DESCRIPTION

[0064] The present invention will be further described below in conjunction with the embodiments.

[0065] The present invention provides an embodiment: a method for detecting and classifying medical image lesions based on machine vision, the steps of which are as follows:

[0066] S1: Image preprocessing

[0067] S11: grayscale processing;

[0068] S12: Noise removal, the methods used include smoothing filtering, wavelet transform, non-local mean filtering and partial differential equation method;

[0069] S13: Image enhancement, using methods including histogram equalization, spatial filtering, edge enhancement, and multi-scale transformation;

[0070] S14: Registration and Alignment

[0071] S141: Select an appropriate transformation model;

[0072] S142: Selecting a suitable feature space, based on grayscale or based on features;

[0073] S143: Determine the range of possible parameter changes according to the parameter configuration of the transformation model and the selected features, and select the optimal search strategy;

[0074] S144: applying the similarity measure to search in the search space according to the optimization criterion to find the maximum correlation point, thereby solving the unknown parameters in the transformation model;

[0075] S145: Corresponding the image to be registered to the reference image according to the transformation model to achieve matching between images;

[0076] S15: Standardization processing, including determining the standardization target, extracting image features, converting or adjusting, and applying standardized methods to perform standardization processing;

[0077] S16: size normalization;

[0078] S17: Segmentation

[0079] S171: Using the FCM algorithm to calculate the probability value of each pixel belonging to each category;

[0080] S172: Determine the target boundary using the image gradient, traverse the image segmented by the FCM algorithm, and mark the pixel points belonging to the edge;

[0081] S173: Find the interference area and delete it by region growing method;

[0082] S18: edge detection, the edge detection algorithm adopts any one of the Sobel operator, Prewitt operator and Canny operator;

[0083] S2: Lesion detection and assessment

[0084] S21: Extract the features of the lesion image through the convolution layer. The convolution layer uses multiple convolution kernels to perform a sliding window operation on the input image. Each convolution kernel generates a feature map. These feature maps contain the local feature information of the image. The convolution operation is mathematically expressed as: C(i, j) = (I*H)(i, j) = ∑ x∑yI(i+m,j+n)·H(x,y), where C represents the convolution operation, I represents the input image, H represents the convolution kernel, (i,j) represents the position of the feature map, and x and y represent the dimensions of the convolution kernel;

[0085] S22: The pooling layer is used to reduce the dimension of the feature map, reduce the amount of calculation and improve the generalization ability of the model. Common pooling methods include maximum pooling and average pooling. Maximum pooling selects the maximum value of each area, while average pooling calculates the average value of each area.

[0086] S23: After passing through multiple convolutional layers and pooling layers, the feature map is flattened and input into the fully connected layer, which classifies the extracted features and outputs the final classification result;

[0087] S3: Model training and validation

[0088] S31: Initialize the network weights;

[0089] S32: The input data is forward propagated through the convolution layer, pooling layer, and fully connected layer to obtain the output value;

[0090] S33: Calculate the error between the output value of the network and the target value;

[0091] S34: When the error is greater than our expected value, the error is passed back to the network, and the errors of the fully connected layer, pooling layer, and convolutional layer are obtained in turn. When the error is equal to or less than the expected value, the training ends;

[0092] S35: Update the weights according to the obtained error, and then proceed to S32.

[0093] Preferably, the grayscale processing method includes:

[0094] (1) Quantization: The brightness of the three components in the color image is used as the grayscale values ​​of the three grayscale images. One grayscale image can be selected according to the application requirements:

[0095] Gray 1 (i, j) = R(i, j)

[0096] Gray 2 (i, j) = G(i, j)

[0097] Gray 3 (i, j) = B(i, j)

[0098] (2) Maximum value method: the maximum value of the three-component brightness in the color image is used as the grayscale 1 value of the grayscale image:

[0099] Gray(i,j)=max{R(i,j),G(i,j),B(i,j)}

[0100] (3) Average value method: average the three brightness components in the color image to obtain a gray value:

[0101] Gray(i,j)=(R(i,j)+G(i,j)+B(i,j)) / 3

[0102] (4) Weighted average method: according to the importance and other indicators, the three components are weighted averaged with different weights. The weighted average of the three RGB components can be used to obtain a more reasonable grayscale image.

[0103] Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j).

[0104] Preferably, histogram equalization redistributes the grayscale levels of image pixels to make the histogram of the image evenly distributed, and is often used to enhance X-ray, CT scan and MRI images; spatial filtering removes noise and unnecessary details in the image by smoothing the image pixels, thereby improving the quality and clarity of the image. Spatial filtering is applied to denoising and smoothing in medical images; edge enhancement separates the edge area from the background area through gradient calculation or multi-threshold processing, making the edge clearer and improving the recognition and analysis of tumors and vascular structures; multi-scale transformation separates the edge area from the background area through gradient calculation or multi-threshold processing, making the edge clearer and improving the recognition and analysis of tumors and vascular structures.

[0105] Preferably, the objectives of standardization include but are not limited to the range, mean and variance of pixel values; the extracted image features include histogram and grayscale; the conversion or adjustment adopts linear or nonlinear conversion; the standardization method is local or global standardization.

[0106] As a preferred embodiment, the specific steps of the FCM algorithm are:

[0107] (1) Determine the number of clusters k and the iteration termination condition. The user needs to determine the number of categories k to be divided and set the iteration termination condition, which is generally the maximum number of iterations allowed or the minimum change in the cluster center point.

[0108] (2) Initialize the membership matrix U. The size of the membership matrix U is (n, k), where n is the number of samples and k is the number of categories. Each element in the membership matrix U represents the probability that the sample belongs to some category. Initially, the value can be randomly assigned or assigned according to certain rules.

[0109] (3) Calculate the coordinates of the center point of each category. According to the membership matrix U, the coordinates of the center point of each category can be calculated. Usually, the “weighted average method” is used for calculation.

[0110] (4) Update the membership matrix U. By calculating the distance between the sample and the center point of the category, the membership matrix U can be updated so that the membership degree of each sample belonging to each category meets the requirements;

[0111] (5) Determine whether the iteration is terminated, compare the change between the current membership matrix U and the previous membership matrix U, if it is less than the set termination condition, stop the iteration; otherwise, return to step (3) and continue the iteration;

[0112] (6) Output the clustering results. The final membership matrix U can be used to determine which category each sample belongs to and divide the samples into corresponding categories.

[0113] Preferably, the Sobel operator performs convolution operations on each pixel of the image and its neighborhood, and calculates the gradients in the horizontal direction and the vertical direction respectively. By combining the gradients in these two directions, the strength and direction of the edge in the image can be obtained.

[0114] Preferably, the Prewitt operator consists of the following two convolution kernels:

[0115]

[0116] As a preference, the steps of the Canny operator are as follows:

[0117] (1) Use Gaussian filter to smooth the image and reduce noise interference;

[0118] (2) Calculate the gradient magnitude and direction of each pixel in the image;

[0119] (3) Apply non-maximum suppression to refine the edge and reduce the edge width to a single pixel;

[0120] (4) Apply a dual threshold algorithm to mark edge pixels as weak edges or strong edges;

[0121] (5) Track and connect weak edge pixels to form a complete edge.

[0122] Through the above steps, the shallow network can learn local features, such as color, geometric shape, etc., while the deep network can learn more abstract features, such as object attributes, contours, etc. This greatly reduces the computational complexity of the model and is suitable for large-scale data sets. Through autonomous learning, when faced with new and unseen data, it can still correctly detect and classify lesions, making it easier for doctors to provide better treatment plans to solve the problem of lack of autonomous learning ability.

[0123] The embodiments of the present invention are described in detail above, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of those skilled in the art without departing from the spirit of the present invention.

Claims

1. A medical image lesion detection and classification method based on machine vision, characterized by: The steps are as follows: S1: Image preprocessing S11: grayscale processing; S12: Noise removal, the methods used include smoothing filtering, wavelet transform, non-local mean filtering and partial differential equation method; S13: Image enhancement, using methods including histogram equalization, spatial filtering, edge enhancement, and multi-scale transformation; S14: Registration and Alignment S141: Select an appropriate transformation model; S142: Selecting a suitable feature space, based on grayscale or based on features; S143: Determine the range of possible parameter changes according to the parameter configuration of the transformation model and the selected features, and select the optimal search strategy; S144: applying the similarity measure to search in the search space according to the optimization criterion to find the maximum correlation point, thereby solving the unknown parameters in the transformation model; S145: Corresponding the image to be registered to the reference image according to the transformation model to achieve matching between images; S15: Standardization processing, including determining the standardization target, extracting image features, converting or adjusting, and applying standardized methods to perform standardization processing; S16: size normalization; S17: Segmentation S171: Using the FCM algorithm to calculate the probability value of each pixel belonging to each category; S172: Determine the target boundary using the image gradient, traverse the image segmented by the FCM algorithm, and mark the pixel points belonging to the edge; S173: Find the interference area and delete it by region growing method; S18: edge detection, the edge detection algorithm adopts any one of the Sobel operator, Prewitt operator and Canny operator; S2: Lesion detection and assessment S21: The features of the lesion image are extracted through the convolution layer. The convolution layer uses multiple convolution kernels to perform a sliding window operation on the input image. Each convolution kernel generates a feature map. These feature maps contain the local feature information of the image. The convolution operation is mathematically expressed as: C(i,j)=(I*H)(i,j)=∑ x ∑yI(i+m,j+n)·H(x,y), where C represents the convolution operation, I represents the input image, H represents the convolution kernel, (i,j) represents the position of the feature map, and x and y represent the dimensions of the convolution kernel; S22: The pooling layer is used to reduce the dimension of the feature map, reduce the amount of calculation and improve the generalization ability of the model. Common pooling methods include maximum pooling and average pooling. Maximum pooling selects the maximum value of each area, while average pooling calculates the average value of each area. S23: After passing through multiple convolutional layers and pooling layers, the feature map is flattened and input into the fully connected layer, which classifies the extracted features and outputs the final classification result; S3: Model training and validation S31: Initialize the network weights; S32: The input data is forward propagated through the convolution layer, pooling layer, and fully connected layer to obtain the output value; S33: Calculate the error between the output value of the network and the target value; S34: When the error is greater than our expected value, the error is passed back to the network, and the errors of the fully connected layer, pooling layer, and convolutional layer are obtained in turn. When the error is equal to or less than the expected value, the training ends; S35: Update the weights according to the obtained error, and then proceed to S32.

2. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: Grayscale processing methods include: (1) Quantization: The brightness of the three components in the color image is used as the grayscale values ​​of the three grayscale images. One grayscale image can be selected according to the application requirements: Gray1(i,j)=R(i,j) Gray2(i,j)=G(i,j) Gray3(i,j)=B(i,j) (2) Maximum value method: the maximum value of the three-component brightness in the color image is used as the grayscale value of the grayscale image: Gray(i,j)=max{R(i,j),G(i,j),B(i,j)} (3) Average value method: average the three brightness components in the color image to obtain a gray value: Gray(i,j)=(R(i,j)+G(i,j)+B(i,j)) / 3 (4) Weighted average method: according to the importance and other indicators, the three components are weighted averaged with different weights. The weighted average of the three RGB components can be used to obtain a more reasonable grayscale image. Gray(i,j)=0.299*R(i,j)+0.578*G(i,j)+0.114*B(i,j).

3. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: Histogram equalization redistributes the grayscale levels of image pixels to make the image histogram evenly distributed, and is often used to enhance X-ray, CT scan and MRI images; spatial filtering removes noise and unnecessary details in the image by smoothing the image pixels, thereby improving the quality and clarity of the image. Spatial filtering is used for denoising and smoothing in medical images; edge enhancement separates the edge area from the background area through gradient calculation or multi-threshold processing, making the edge clearer and improving the recognition and analysis of tumors and vascular structures; multi-scale transformation separates the edge area from the background area through gradient calculation or multi-threshold processing, making the edge clearer and improving the recognition and analysis of tumors and vascular structures.

4. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: The objectives of standardization include but are not limited to the range, mean and variance of pixel values; the extracted image features include histogram and grayscale; the conversion or adjustment adopts linear or nonlinear conversion; the method of standardization is local or global standardization.

5. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: The specific steps of the FCM algorithm are: (1) Determine the number of clusters k and the iteration termination condition. The user needs to determine the number of categories k to be divided and set the iteration termination condition, which is generally the maximum number of iterations allowed or the minimum change in the cluster center point. (2) Initialize the membership matrix U. The size of the membership matrix U is (n, k), where n is the number of samples and k is the number of categories. Each element in the membership matrix U represents the probability that the sample belongs to some category. Initially, the value can be randomly assigned or assigned according to certain rules. (3) Calculate the coordinates of the center point of each category. According to the membership matrix U, the coordinates of the center point of each category can be calculated. Usually, the "weighted average method" is used for calculation. (4) Update the membership matrix U. By calculating the distance between the sample and the center point of the category, the membership matrix U can be updated so that the membership degree of each sample belonging to each category meets the requirements; (5) Determine whether the iteration is terminated, compare the change between the current membership matrix U and the previous membership matrix U, if it is less than the set termination condition, stop the iteration; otherwise, return to step (3) and continue the iteration; (6) Output the clustering results. The final membership matrix U can be used to determine which category each sample belongs to and divide the samples into corresponding categories.

6. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: The Sobel operator performs convolution operations on each pixel of the image and its neighborhood, calculating the gradients in the horizontal and vertical directions respectively. By combining the gradients in these two directions, the strength and direction of the edge in the image can be obtained.

7. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: The Prewitt operator consists of the following two convolution kernels:

8. The method for detecting and classifying medical image lesions based on machine vision according to claim 1, characterized in that: The steps of the Canny operator are as follows: (1) Use Gaussian filter to smooth the image and reduce noise interference; (2) Calculate the gradient magnitude and direction of each pixel in the image; (3) Apply non-maximum suppression to refine the edge and reduce the edge width to a single pixel; (4) Apply a dual threshold algorithm to mark edge pixels as weak edges or strong edges; (5) Track and connect weak edge pixels to form a complete edge.

Images