Ultrasonic diagnosis auxiliary method and system based on machine learning

Abstract

The invention provides an ultrasonic diagnosis assisting method and system based on machine learning, and belongs to the field of diagnosis assisting. The problem of low ultrasonic diagnosis efficiency is solved; the method specifically comprises the following steps: S1, obtaining a picture set to be extracted; s2, extracting texture features and shape features of the picture set to be extracted to obtain ultrasonic picture features of the patient; s3, acquiring a reference picture set; constructing an ultrasonic image diagnosis model by taking the reference image set as a training set of computer model training; s4, substituting the ultrasonic image features of the patient into the ultrasonic diagnosis model to obtain a disease name of the patient; by acquiring, extracting and analyzing the ultrasonic image of the patient, the disease of the patient is judged, and the diagnosis efficiency of a doctor is improved.

Description

Technical Field

[0001] The present invention discloses an ultrasonic diagnosis assistance method and system based on machine learning, and relates to the field of diagnosis assistance. Background Art

[0002] The existing ultrasonic diagnostic methods and systems have the following deficiencies:

[0003] Strong data dependence: Existing machine learning systems are highly dependent on the quality and quantity of training data sets. In terms of medical imaging data, it is difficult to obtain sufficient high-quality data sets. In addition, the data aggregated by different institutions and systems are biased, and preprocessing is also difficult.

[0004] Complex operation: The machine learning system is a completely mathematical, symbolic and computer-simulated process. Especially during the system's model training process, professionals are required to adjust the model training strategy based on the system's real-time feedback, which is not friendly to medical workers.

[0005] High maintenance cost: The application of existing ultrasonic diagnostic methods or systems in the medical field requires the participation and evaluation of professionals; at the same time, the maintenance and upgrade of the system also requires the support of professional technicians. This increases the cost of using the system and the difficulty of maintenance to a certain extent. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide an ultrasonic diagnosis auxiliary method and system based on machine learning, aiming to solve the problem of low efficiency of ultrasonic diagnosis.

[0007] In order to achieve the above object, the present invention is implemented by the following technical solution: an ultrasonic diagnosis auxiliary method based on machine learning includes:

[0008] Step S1: Obtaining ultrasound images of the patient; performing preprocessing of denoising, standardization and tilt correction on the ultrasound images to obtain a set of images to be extracted;

[0009] Step S2: construct a gray-level co-occurrence matrix according to the image set to be extracted; calculate the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance corresponding to the images in the image set to be extracted based on the gray-level co-occurrence matrix to obtain the texture features of the image set to be extracted; use the OpenCV library to extract the shape features of the image set to be extracted to obtain the patient's ultrasound image features;

[0010] Step S3: Obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set;

[0011] Preprocess and extract features of the original picture set to obtain a reference picture set;

[0012] Using the reference image set as a training set for computer model training, an ultrasound image diagnosis model is constructed;

[0013] Step S4: Substitute the patient's ultrasonic image features into the ultrasonic diagnostic model to obtain the patient's disease name; and prescribe a prescription for the patient based on the disease name.

[0014] Furthermore, the specific steps of step S1 are as follows:

[0015] Step S11: Acquire an ultrasound image of the patient as an initial image;

[0016] Count the number of initial images, denoted as pn;

[0017] Step S12: the first initial picture is recorded as picture A;

[0018] Perform denoising, enhancement and normalization preprocessing on the image A to obtain the image 1 to be extracted;

[0019] Step S13: Repeat the same steps of preprocessing the first initial picture, and preprocess the second to pnth initial pictures to obtain the to-be-extracted pictures 2 to pn;

[0020] Take the to-be-extracted pictures 1 to to-be-extracted pictures pn as the to-be-extracted picture set and proceed to step S2.

[0021] Furthermore, the specific steps of step S12 are as follows:

[0022] Step S121: performing denoising processing on the image A to obtain a filtered image of the image A;

[0023] Get the pixels of image A, recorded as w×l; w represents the pixel length and l represents the pixel width;

[0024] Use the Pillow library function to get the RGB value of each pixel in image A and get the RGB matrix:

[0025] Among them, X 11 , represents the RGB code corresponding to the pixel in the first row and first column of image A; X 12 , represents the RGB code corresponding to the pixel in the first row and second column of image A; and so on, X wl , represents the RGB code corresponding to the pixel in the wth row and the lth column in the image A;

[0026] Step S122: Calculate X 11 ~X wl The average value is recorded as aX; calculate X 11 ~X wlThe standard deviation of is denoted as μX;

[0027] The order of the convolution kernel is recorded as J; compare the size of w and l to determine the value of J;

[0028] If w ≥ l, let the square root of w be gw, and continue to take the square root of w until the value of gw is between (0,10), and use gw as the value of J;

[0029] If w < l, let the square root of l be gl, and continue to take the square root of l until the value of lw is between (0,10), and gl is used as the value of J;

[0030] Step S123: performing Gaussian filtering on the image A to obtain a filtered image of the image A;

[0031] Step S124: normalize and tilt-correct the filtered image of picture A to obtain picture 1 to be extracted.

[0032] 4. The method for assisting ultrasonic diagnosis based on machine learning according to claim 3, wherein the specific steps of step S124 are as follows:

[0033] Step S1241: The filtered image of picture A is recorded as filtered image A;

[0034] Read the maximum and minimum RGB values ​​of each pixel in the filtered image A, denoted as X MAX and X MIN ; Calculate the average RGB value of each pixel in the filtered image A, denoted as X AVE ;

[0035] Step S1242: Read the RGB value of each pixel in the filtered image A, denoted as CO (1,1) , CO (1,2) ~CO (w,l) ;

[0036] Among them, CO (1,1) represents the RGB value corresponding to the pixel at (1,1) in the filtered image A; CO (1,2) represents the RGB value corresponding to the pixel at (1,2) in the filtered image A; and so on, CO (w,w) Represents the RGB value corresponding to the pixel at (w, l) in the filtered image A;

[0037] Step S1243: Normalize the pixel at (1,1) in the filtered image A and calculate CO (1,1) The standardized value of (1,1) ;

[0038] Determine CO (1,1) Is it in X MAX ~XAVE between;

[0039] Step S1244: Repeat the calculation of CO (1,1) The same procedure as the normalized values ​​was used to calculate CO (1,2) ~CO (w,l) The standardized value of is used to complete the standardization of image A and obtain the standardized image A;

[0040] Step S1245: Use an edge detection algorithm to detect the image edge of the standardized image A; detect the image edge through Hough transform, calculate the rotation angle of the image, and use the rotation transformation instruction to correct the tilt of the image so that the image direction of the standardized image A is aligned with the horizontal direction to obtain the image 1 to be extracted.

[0041] 5. The method for assisting ultrasonic diagnosis based on machine learning according to claim 2, wherein the specific steps of step S2 are as follows:

[0042] Step S21: constructing a gray-level co-occurrence matrix according to the image set to be extracted;

[0043] Based on the gray-level co-occurrence matrix, the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance of each image are calculated to obtain the texture features of the image set to be extracted;

[0044] Step S22: using the OpenCV library to extract shape features of the images in the image set to be extracted;

[0045] Step S221: Use the imread() function to read each picture in the picture set to be extracted;

[0046] Use the Canny edge detection algorithm to find the edges of each image in the image set to be extracted;

[0047] Step S222: Locate the picture according to the edge of each picture, and use the cv2.findContours function to find the contour of each picture in the picture set to be extracted;

[0048] For each contour, use the arcLength() function to calculate the length of the contour;

[0049] Use the contourArea() function to calculate the area of ​​the contour;

[0050] Use the boundingRect() function to calculate the bounding box of the contour;

[0051] Use the approxPolyDP() function to calculate the approximate polygon of the contour;

[0052] Step S223: Summarize the contour length, contour area, contour bounding box and contour approximate polygon of each picture in the picture set to be extracted as the shape feature of the picture set to be extracted;

[0053] Step S23: Summarize the texture features and shape features of the image set to be extracted as the patient's ultrasound image features, and proceed to step S3.

[0054] 6. The method for assisting ultrasonic diagnosis based on machine learning according to claim 5, wherein the specific steps of step S21 are as follows:

[0055] Step S211: extracting texture features of the image set to be extracted;

[0056] Define a 256×256 empty matrix as the initial gray-level co-occurrence matrix;

[0057] With 1 as the distance between pixel pairs, 0° as the direction, and 256 levels as grayscale, the greycomatrix function is used to scan the image set to be extracted, fill in the initial grayscale co-occurrence matrix, and obtain the grayscale co-occurrence matrix of the image set to be extracted, which is recorded as co-occurrence matrix A;

[0058] Step S212: Count the number of pixel pairs in the co-occurrence matrix A, denoted as pip;

[0059] Count the number of occurrences of each pixel pair in the co-occurrence matrix A, recorded as occ1, occ2~occ pip ;

[0060] Among them, occ1 represents the number of occurrences of the first pixel pair in the co-occurrence matrix A; occ2 represents the number of occurrences of the second pixel pair in the co-occurrence matrix A; and so on. pip represents the number of occurrences of the pip-th pixel pair in the co-occurrence matrix A;

[0061] Step S213: Calculate occ1 to occ pip The sum of is denoted as aocc;

[0062] Calculate the contrast, dissimilarity, homogeneity coefficient, angular second moment and correlation coefficient of the first pixel pair;

[0063] Step S2131: Assume the first pixel pair is (i, j); wherein i and j both represent grayscale values, and the value range of i and j is 0 to 255;

[0064] Step S2132: Calculate the homogeneity coefficient of the first pixel pair, denoted as hom(1);

[0065] Step S214: Repeat the same steps of calculating the contrast, dissimilarity, homogeneity coefficient, angular second moment and correlation coefficient of the first pixel pair, calculate the contrast of the second to pip-th pixel pairs, and obtain con(2) to con(pip).

[0066] 7. The method for assisting ultrasonic diagnosis based on machine learning according to claim 6, characterized in that the subsequent steps of step S215 are as follows:

[0067] Step S216: Calculate the angular mean square difference of the first pixel pair;

[0068] Step S2161: Count the number of occurrences of the first pixel pair in each picture in the picture set to be extracted, recorded as oc1, oc2~oc pn ; Wherein, pn represents the number of pictures in the picture set to be extracted;

[0069] oc1 represents the number of times the first pixel pair appears in the first picture to be extracted in the picture set to be extracted;

[0070] oc2 represents the number of times the first pixel pair appears in the second image to be extracted in the image set to be extracted;

[0071] By analogy, oc pn Indicates the number of times the first pixel pair appears in the pnth picture to be extracted in the picture set to be extracted;

[0072] oc1~oc pn The sum is occ1;

[0073] Step S2162: Calculate the total grayscale value of all pixels in each image in the image set to be extracted, denoted as hn1, hn2~hn pn ;

[0074] Among them, hn1 represents the total grayscale value of all pixels in the first picture in the picture set to be extracted;

[0075] hn2 represents the total grayscale value of all pixels in the second picture in the picture set to be extracted;

[0076] By analogy, hn pn Represents the total grayscale value of all pixels in the pnth image in the image set to be extracted;

[0077] Step S2163: Calculate the angular mean square difference of the first pixel pair, denoted asm(1);

[0078] hn i Represents the total grayscale value of all pixels in the i-th image in the image set to be extracted;

[0079] Step S217: Repeat the same steps of calculating the diagonal average square difference of the first pixel, and calculate the diagonal average square differences of the second to pip-th pixels to obtain asm(2) to asm(pip);

[0080] Step S218: Summarize the contrast, dissimilarity, homogeneity coefficient, angular second-order moment and correlation coefficient of the picture set to be extracted and the diagonal mean square difference corresponding to the 1st to pipth pixels as the texture features of the picture set to be extracted.

[0081] An ultrasound diagnosis auxiliary system based on machine learning includes:

[0082] Image acquisition module: used to acquire ultrasound images of patients; pre-process the ultrasound images by denoising, standardizing and tilt correction to obtain a set of images to be extracted;

[0083] Feature extraction module: used to construct a gray-level co-occurrence matrix based on the image set to be extracted; based on the gray-level co-occurrence matrix, the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance corresponding to the images in the image set to be extracted are calculated to obtain the texture features of the image set to be extracted; the shape features of the image set to be extracted are extracted using the OpenCV library to obtain the patient's ultrasound image features;

[0084] Model training module: used to obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set;

[0085] Preprocess and extract features of the original picture set to obtain a reference picture set;

[0086] Using the reference image set as a training set for computer model training, an ultrasound image diagnosis model is constructed;

[0087] Medical pre-diagnosis module: used to substitute the patient's ultrasound image features into the ultrasound diagnosis model to obtain the patient's disease name; prescribe medicine for the patient based on the disease name.

[0088] Compared with the prior art, the present invention has the following beneficial effects:

[0089] Improve diagnostic accuracy: The present invention can automatically analyze and diagnose the patient's ultrasound images; the present invention uses a machine learning system to quickly process a large amount of pathological image data, automatically identify and classify symptoms, reduce human errors, and improve diagnostic accuracy.

[0090] Standardized diagnostic process: The present invention can set a unified operating process and inspection standard for ultrasound diagnosis to ensure the consistency and comparability of diagnostic results. Through data analysis and machine learning, the present invention enables the system to predict and identify the patient's existing symptoms based on the patient's ultrasound images, providing a reference for doctors.

[0091] Reducing the burden on doctors: The present invention can replace doctors to complete a large amount of repetitive and mechanical ultrasonic image analysis work, which not only reduces the workload of doctors, but also improves work efficiency, allowing more patients to receive timely diagnosis. BRIEF DESCRIPTION OF THE DRAWINGS

[0092] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0093] Figure 1 It is a schematic diagram of the method of the present invention;

[0094] Figure 2 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0095] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0096] Embodiment 1

[0097] See also Figure 1 , an ultrasound diagnosis auxiliary method based on machine learning includes:

[0098] Step S1: Obtaining ultrasound images of the patient; performing preprocessing of denoising, standardization and tilt correction on the ultrasound images to obtain a set of images to be extracted;

[0099] The specific steps of step S1 are as follows:

[0100] Step S11: Acquire an ultrasound image of the patient as an initial image;

[0101] Count the number of initial images, denoted as pn;

[0102] Step S12: the first initial picture is recorded as picture A;

[0103] Perform denoising, enhancement and normalization preprocessing on the image A to obtain the image 1 to be extracted;

[0104] Step S121: performing denoising processing on the image A to obtain a filtered image of the image A;

[0105] Get the pixels of image A, recorded as w×l; w represents the pixel length and l represents the pixel width;

[0106] Use the Pillow library function to get the RGB value of each pixel in image A and get the RGB matrix:

[0107] Among them, X 11, represents the RGB code corresponding to the pixel in the first row and first column of image A; X 12 , represents the RGB code corresponding to the pixel in the first row and second column of image A; and so on, X wl , represents the RGB code corresponding to the pixel in the wth row and the lth column in the image A;

[0108] Step S122: Calculate X 11 ~X wl The average value is recorded as aX; calculate X 11 ~X wl The standard deviation of is denoted as μX;

[0109] The order of the convolution kernel is recorded as J; compare the size of w and l to determine the value of J;

[0110] If w ≥ l, let the square root of w be gw, and continue to take the square root of w until the value of gw is between (0,10), and use gw as the value of J;

[0111] If w < l, let the square root of l be gl, and continue to take the square root of l until the value of lw is between (0,10), and gl is used as the value of J;

[0112] Step S123: performing Gaussian filtering on image A;

[0113] Step S1231: Calculate the absolute value of the RGB value of each pixel and the subtraction of (aX+μX) and (aX-μX) in turn; obtain (aXμ 11 , aμX 11 )、(aXμ 12 , aμX 12 )~(aXμ wl , aμX wl );

[0114] Among them, aXμ 11 Represents the absolute value of the RGB value of the pixel at position (1,1) in the RGB matrix minus (aX+μX), aμX 11 Represents the absolute value of the RGB value of the pixel at position (1,1) in the RGB matrix minus (aX-μX);

[0115] aX 12 Represents the absolute value of the RGB value of the pixel at position (1,2) in the RGB matrix minus (aX+μX), aμX 12 Represents the absolute value of the RGB value of the pixel at position (1,2) in the RGB matrix minus (aX-μX);

[0116] By analogy, aXμ wlRepresents the absolute value of the RGB value of the pixel at position (w, l) in the RGB matrix minus (aX+μX), aμX wl Represents the absolute value of the RGB value of the pixel at position (w, l) in the RGB matrix minus (aX-μX);

[0117] Step S1232: taking (0.01, 0.01) to (0.001, 0.001) as the error range;

[0118] Statistics (aXμ 11 , aμX 11 )、~(aXμ wl , aμX wl ) The number of pixels that do not fall within the error range among the corresponding pixels is recorded as c; the number of pixels that fall within the error range is recorded as d;

[0119] Step S1233: the RGB correction value is recorded as GC; the RGB correction value is recorded as GD;

[0120] The calculation formula of GC is as follows:

[0121]

[0122] The calculation formula of GD is as follows:

[0123]

[0124] Step S1234: Taking the pixel at position (1,1) in the RGB matrix as the starting point, taking out a J×J Gaussian unit matrix in the RGB matrix, recorded as Gaussian matrix (1,1);

[0125] In the RGB matrix, the image of the J×J region with (1,1) as the starting point is recorded as image A(1,1);

[0126] Create a J×J blank matrix, recorded as the initial convolution kernel; all elements in the initial convolution kernel are filled with 0;

[0127] In the initial convolution kernel (aXμ 11 , aμX 11 )~(aXμ JJ , aμX JJ ) In the position corresponding to the error, fill in GC at the pixel position that does not belong to the error range; fill in GD at the pixel position that belongs to the error range; and obtain the convolution kernel with parameters;

[0128] Use matrix multiplication to calculate the result of multiplying the "Gaussian matrix (1, 1)" by the "parametric convolution kernel" to obtain the filtered image of image A(1, 1);

[0129] Step S1235: taking the pixel points at positions (1,1+J) to (l-J,w-J) in the RGB matrix as starting points, taking out the J×J Gaussian unit matrix in the RGB matrix, and obtaining images A(1,1+J) to A(l-J,w-J); repeating the same step of determining the filtered image of image A(1,1) (i.e., step S1234), processing the pixel values ​​of the corresponding areas of images A(1,1+J) to A(l-J,w-J), and obtaining filtered images of images A(1,1+J) to A(l-J,w-J);

[0130] Step S1236: Summarize the filtered images of image A(1,1) to image A(l-J,w-J) as the filtered image of picture A;

[0131] It should be noted that the "error range (0.01, 0.01) to (0.001, 0.001)" in step S1232 is only a reference range; users or relevant technical personnel can adjust the "error range" according to actual needs.

[0132] Step S124: performing standardization and tilt correction processing on the filtered image of picture A to obtain picture 1 to be extracted;

[0133] Step S1241: The filtered image of picture A is recorded as filtered image A;

[0134] Read the maximum and minimum RGB values ​​of each pixel in the filtered image A, denoted as X MAX and X MIN ; Calculate the average RGB value of each pixel in the filtered image A, denoted as X AVE ;

[0135] Step S1242: Read the RGB value of each pixel in the filtered image A, denoted as CO (1,1) , CO (1,2) ~CO (w,l) ;

[0136] Among them, CO (1,1) represents the RGB value corresponding to the pixel at (1,1) in the filtered image A; CO (1,2) represents the RGB value corresponding to the pixel at (1,2) in the filtered image A; and so on, CO (w,w) Represents the RGB value corresponding to the pixel at (w, l) in the filtered image A;

[0137] Step S1243: Normalize the pixel at (1,1) in the filtered image A and calculate CO (1,1) The standardized value of (1,1) ;

[0138] Determine CO (1,1)Is it in X MAX ~X AVE between;

[0139] If yes, then CB (1,1) The calculation formula is as follows:

[0140] CB (1,1) =[1+(CO (1,1) -X AVE ) / (X MAX -X AVE )]×128-1;

[0141] If not, then CB (1,1) The calculation formula is as follows:

[0142] CB (1,1) =[1-(X AVE -CO (1,1) ) / (X AVE -X MIN )]×128-1;

[0143] Step S1244: Repeat the calculation of CO (1,1) The same steps as the normalized values ​​(i.e., step S1243) are used to calculate CO (1,2) ~CO (w,l) The standardized value of is used to complete the standardization of image A and obtain the standardized image A;

[0144] Step S1245: using an edge detection algorithm to detect the image edge of the standardized image A; detecting the image edge through Hough transform, calculating the image rotation angle, and correcting the image tilt with a rotation transform instruction, so that the image direction of the standardized image A is aligned with the horizontal direction, and obtaining the image 1 to be extracted;

[0145] Step S13: repeat the same steps of preprocessing the first initial picture (i.e., steps S121 to S124), and preprocess the second to pnth initial pictures to obtain the to-be-extracted pictures 2 to pn;

[0146] Take the to-be-extracted pictures 1 to to-be-extracted pictures pn as the to-be-extracted picture set and proceed to step S2.

[0147] The specific steps of step S2 are as follows:

[0148] Step S21: constructing a gray-level co-occurrence matrix according to the image set to be extracted;

[0149] Based on the gray-level co-occurrence matrix, the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance of each image are calculated to obtain the texture features of the image set to be extracted;

[0150] Step S211: extracting texture features of the image set to be extracted;

[0151] Define a 256×256 empty matrix as the initial gray-level co-occurrence matrix;

[0152] With 1 as the distance between pixel pairs, 0° as the direction, and 256 levels as grayscale, the greycomatrix function is used to scan the image set to be extracted, fill in the initial grayscale co-occurrence matrix, and obtain the grayscale co-occurrence matrix of the image set to be extracted, which is recorded as co-occurrence matrix A;

[0153] Step S212: Count the number of pixel pairs in the co-occurrence matrix A, denoted as pip;

[0154] Count the number of occurrences of each pixel pair in the co-occurrence matrix A, recorded as occ1, occ2~occ pip ;

[0155] Among them, occ1 represents the number of occurrences of the first pixel pair in the co-occurrence matrix A; occ2 represents the number of occurrences of the second pixel pair in the co-occurrence matrix A; and so on. pip represents the number of occurrences of the pip-th pixel pair in the co-occurrence matrix A;

[0156] Step S213: Calculate occ1 to occ pip The sum of is denoted as aocc;

[0157] Calculate the contrast, dissimilarity, homogeneity coefficient, angular second moment and correlation coefficient of the first pixel pair;

[0158] Step S2131: Assume the first pixel pair is (i, j); wherein i and j both represent grayscale values, and the value range of i and j is 0 to 255;

[0159] Calculate the contrast of the first pixel pair, denoted as con(1); the calculation formula of con(1) is as follows:

[0160] con(1)=(i-j) 2 ×(occ1 / aocc);

[0161] Calculate the dissimilarity of the first pixel pair, denoted as dim(1); the calculation formula of dim(1) is as follows:

[0162] dim(1)=|i-j|×(occ1 / aocc);

[0163] Step S2132: Calculate the homogeneity coefficient of the first pixel pair, denoted as hom(1); the calculation formula of hom(1) is as follows:

[0164] hom(1)=(occ1 / aocc) / (1+|i-j|)

[0165] Calculate the angular second-order moment of the first pixel pair, denoted as ene(1); the calculation formula of ene(1) is as follows:

[0166] ene(1)=(occ1 / aocc) 2 ;

[0167] Calculate the average value of i and j, denoted as μa; calculate the standard deviation of i and j, denoted as σa;

[0168] Calculate the correlation coefficient of the first pixel pair, denoted as cor(1); the calculation formula of cor(1) is as follows:

[0169] cor(1)=[(i-μa)×(j-μa)×(occ1 / aocc)] / (σa) 2 ;

[0170] Step S214: Repeat the same steps of calculating the contrast, dissimilarity, homogeneity coefficient, angular second moment and correlation coefficient of the first pixel pair, and calculate the contrast of the second to pip-th pixel pairs to obtain con(2) to con(pip);

[0171] Calculate the dissimilarity of the 2nd to pipth pixel pairs to obtain dim(2)~dim(pip);

[0172] Calculate the homogeneity coefficients of the 2nd to pipth pixel pairs and obtain hom(2)~hom(pip);

[0173] Calculate the angular second-order moment of the 2nd to pipth pixel pairs to obtain ene(2)~ene(pip);

[0174] Calculate the correlation coefficients of the 2nd to pipth pixel pairs to obtain cor(2)~cor(pip);

[0175] Step S215: Calculate the average value of con(1) to con(pip) as the contrast feature value of the image set to be extracted;

[0176] Calculate the average value of dim(2)~dim(pip) as the dissimilarity feature value of the image set to be extracted;

[0177] Calculate the average value of hom(2)~hom(pip) as the homogeneity coefficient eigenvalue of the image set to be extracted;

[0178] Calculate the average value of ene(2)~ene(pip) as the angular second-order moment eigenvalue of the image set to be extracted;

[0179] Calculate the average value of cor(2)~cor(pip) as the correlation coefficient feature value of the image set to be extracted;

[0180] Step S216: Calculate the angular mean square difference of the first pixel pair;

[0181] Step S2161: Count the number of occurrences of the first pixel pair in each picture in the picture set to be extracted, recorded as oc1, oc2~oc pn ; Wherein, pn represents the number of pictures in the picture set to be extracted;

[0182] oc1 represents the number of times the first pixel pair appears in the first picture to be extracted in the picture set to be extracted;

[0183] oc2 represents the number of times the first pixel pair appears in the second image to be extracted in the image set to be extracted;

[0184] By analogy, oc pn Indicates the number of times the first pixel pair appears in the pnth picture to be extracted in the picture set to be extracted;

[0185] oc1~oc pn The sum is occ1;

[0186] Step S2162: Calculate the total grayscale value of all pixels in each image in the image set to be extracted, denoted as hn1, hn2~hn pn ;

[0187] Among them, hn1 represents the total grayscale value of all pixels in the first picture in the picture set to be extracted;

[0188] hn2 represents the total grayscale value of all pixels in the second picture in the picture set to be extracted;

[0189] By analogy, hn pn Represents the total grayscale value of all pixels in the pnth image in the image set to be extracted;

[0190] Step S2163: Calculate the angular mean square difference of the first pixel pair, denoted asm(1); the calculation formula of asm(1) is as follows:

[0191] Among them, the value range of i is: 1~pn;

[0192] oc i Indicates the number of times the first pixel pair appears in the i-th picture to be extracted in the picture set to be extracted;

[0193] hn i Represents the total grayscale value of all pixels in the i-th image in the image set to be extracted;

[0194] Step S217: Repeat the same step of calculating the diagonal average square difference of the first pixel (i.e., step S216), and calculate the diagonal average square differences of the second to pip-th pixels to obtain asm(2) to asm(pip);

[0195] Step S218: Summarize the contrast, dissimilarity, homogeneity coefficient, angular second-order moment and correlation coefficient of the picture set to be extracted and the diagonal mean square difference corresponding to the 1st to pipth pixels as the texture features of the picture set to be extracted;

[0196] Step S22: using the OpenCV library to extract shape features of the images in the image set to be extracted;

[0197] Step S221: Use the imread() function to read each picture in the picture set to be extracted;

[0198] Use the Canny edge detection algorithm to find the edges of each image in the image set to be extracted;

[0199] Step S222: Locate the picture according to the edge of each picture, and use the cv2.findContours function to find the contour of each picture in the picture set to be extracted;

[0200] For each contour, use the arcLength() function to calculate the length of the contour;

[0201] Use the contourArea() function to calculate the area of ​​the contour;

[0202] Use the boundingRect() function to calculate the bounding box of the contour;

[0203] Use the approxPolyDP() function to calculate the approximate polygon of the contour;

[0204] Step S223: Summarize the contour length, contour area, contour bounding box and contour approximate polygon of each picture in the picture set to be extracted as the shape feature of the picture set to be extracted;

[0205] Step S23: Summarize the texture features and shape features of the image set to be extracted as the patient's ultrasound image features, and proceed to step S3.

[0206] Step S3: Obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set;

[0207] Preprocess and extract features of the original picture set to obtain a reference picture set;

[0208] Using the reference image set as a training set for computer model training, an ultrasound image diagnosis model is constructed;

[0209] It should be noted that the "target hospital" in the present invention refers to a municipal hospital that uses the present invention (an ultrasound diagnosis assistance method and system based on machine learning) for diagnosis assistance.

[0210] The specific steps of step S3 are as follows:

[0211] Step S31: Obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set;

[0212] Preprocessing the original picture set (i.e., taking the original picture set as the processing object, executing step S1) and feature extraction (i.e., taking the original picture set as the processing object, executing step S2) are performed on the original picture set to obtain a reference picture set;

[0213] Step S32: constructing an ultrasound image diagnosis model according to the reference image set;

[0214] Step S321: Using the SqueezeNet model as the basic model of the ultrasound image diagnosis model;

[0215] Use softmax function as activation function;

[0216] Step S322: creating two pooling layers, Pool 1 and Pool 2, for the ultrasound image diagnosis model;

[0217] Create 5 convolutional layers, starting with the convolutional layer Con1 and the subsequent convolutional layers Conv2 to Conv5;

[0218] Create a connection layer Concat; create two fully connected layers Fc6~Fc8;

[0219] Step S323: upload the image of the reference picture set as a primary input to the Con1 layer and the Pool1 layer, perform bottom-level feature extraction, and obtain extracted feature 1;

[0220] Take the extracted feature 1 as the secondary input and upload it to the Conv2 layer and the Pool2 layer to obtain the extracted feature 2;

[0221] Backup extracted feature 2 as the underlying feature;

[0222] Step S324: Upload the extracted feature 2 to the Conv3 layer for processing to obtain the extracted feature 3; upload the extracted feature 3 to the Conv4 layer for processing to obtain the extracted feature 4; upload the extracted feature 4 to the Conv5 layer for processing to obtain the extracted feature 5 as the high-level feature;

[0223] Step S325: perform BN operation on the bottom-level features and high-level features respectively to obtain fused features; use the fused features as the input of the fully connected layer fc6;

[0224] Among them, the BN operation represents: the Batch Normalization operation that removes linear changes;

[0225] Step S326: The model input of the ultrasound image diagnosis model is taken as x (reference image set), the model output is taken as y, and the loss function J is defined as:

[0226]

[0227] Where N represents the number of iterations (for ultrasound image diagnosis model training), i.e., batch-size;

[0228] x i and i , respectively represent the model input and model output when the ultrasound image diagnosis model is iterated i times; w represents the weight, and f(w, xi) represents the weight of the model input xi when the ultrasound image diagnosis model is iterated i times;

[0229] R(w) represents the regularization term, and λ represents the weight of the control regularization term;

[0230] L represents the cross entropy loss function;

[0231] Step S327: defining the output of the ultrasound image diagnosis model;

[0232] If the image to be analyzed is associated with the reference picture set (i.e., there is an intersection), then the reference picture associated with the image to be analyzed is output as the judgment picture;

[0233] According to the judgment picture, the name of the disease suffered by the patient is searched in the OBIA database;

[0234] The name of the disease the patient suffers from is used as the final output of the ultrasound image diagnosis model;

[0235] (OBIA database is a domestic database used to archive biomedical imaging and related clinical data)

[0236] If the image to be analyzed is not associated with the reference image set (ie, there is no intersection), it will not be output.

[0237] Step S4: Substitute the patient's ultrasonic image features into the ultrasonic diagnostic model to obtain the patient's disease name; and prescribe a prescription for the patient based on the disease name.

[0238] Embodiment 2

[0239] See also Figure 2An ultrasonic diagnostic auxiliary system based on machine learning includes: an image acquisition module, a feature extraction module, a model training module, a medical prefabricated diagnosis module, a database and a server; wherein the image acquisition module, the feature extraction module, the model training module and the medical prefabricated diagnosis module are connected to the database and the server respectively.

[0240] Image acquisition module: used to acquire ultrasound images of patients; pre-process the ultrasound images by denoising, standardizing and tilt correction to obtain a set of images to be extracted;

[0241] Feature extraction module: used to construct a gray-level co-occurrence matrix based on the image set to be extracted; based on the gray-level co-occurrence matrix, the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance corresponding to the images in the image set to be extracted are calculated to obtain the texture features of the image set to be extracted; the shape features of the image set to be extracted are extracted using the OpenCV library to obtain the patient's ultrasound image features;

[0242] Model training module: used to obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set;

[0243] Preprocess and extract features of the original picture set to obtain a reference picture set;

[0244] Using the reference image set as a training set for computer model training, an ultrasound image diagnosis model is constructed;

[0245] Medical pre-diagnosis module: used to substitute the patient's ultrasound image features into the ultrasound diagnosis model to obtain the patient's disease name; prescribe medicine for the patient based on the disease name.

[0246] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technical personnel in this field according to actual conditions. For example, if there are weight coefficients and proportional coefficients, their set sizes are to quantify each parameter to obtain a specific value, which is convenient for subsequent comparison. Regarding the size of the weight coefficient and the proportional coefficient, it is sufficient as long as it does not affect the proportional relationship between the parameter and the quantized value.

[0247] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An ultrasonic diagnostic auxiliary method based on machine learning, characterized in that: The method comprises: Step S1: Obtaining ultrasound images of the patient; performing preprocessing of denoising, standardization and tilt correction on the ultrasound images to obtain a set of images to be extracted; Step S2: construct a gray-level co-occurrence matrix according to the image set to be extracted; calculate the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance corresponding to the images in the image set to be extracted based on the gray-level co-occurrence matrix to obtain the texture features of the image set to be extracted; use the OpenCV library to extract the shape features of the image set to be extracted to obtain the patient's ultrasound image features; Step S3: Obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set; Preprocess and extract features of the original picture set to obtain a reference picture set; Using the reference image set as a training set for computer model training, an ultrasound image diagnosis model is constructed; Step S4: Substitute the patient's ultrasonic image features into the ultrasonic diagnostic model to obtain the patient's disease name; and prescribe a prescription for the patient based on the disease name.

2. The method for assisting ultrasonic diagnosis based on machine learning according to claim 1, characterized in that: The specific steps of step S1 are as follows: Step S11: Obtain an ultrasound image of the patient as an initial image; Count the number of initial images, denoted as pn; Step S12: the first initial picture is recorded as picture A; Perform denoising, enhancement and normalization preprocessing on the image A to obtain the image 1 to be extracted; Step S13: Repeat the same steps of preprocessing the first initial picture, and preprocess the second to pnth initial pictures to obtain the to-be-extracted pictures 2 to pn; Take the to-be-extracted pictures 1 to to-be-extracted pictures pn as the to-be-extracted picture set and proceed to step S2.

3. The method for assisting ultrasonic diagnosis based on machine learning according to claim 2, characterized in that: The specific steps of step S12 are as follows: Step S121: performing denoising processing on the image A to obtain a filtered image of the image A; Get the pixels of image A, recorded as w×l; w represents the pixel length and l represents the pixel width; Use the Pillow library function to get the RGB value of each pixel in image A and get the RGB matrix: Among them, X 11 , represents the RGB code corresponding to the pixel in the first row and first column of image A; X 12 , represents the RGB code corresponding to the pixel in the first row and second column of image A; and so on, X wl , represents the RGB code corresponding to the pixel in the wth row and the lth column in the image A; Step S122: Calculate X 11 ~X wl The average value is recorded as aX; calculate X 11 ~X wl The standard deviation of is denoted as μX; The order of the convolution kernel is recorded as J; compare the size of w and l to determine the value of J; If w ≥ l, let the square root of w be gw, and continue to take the square root of w until the value of gw is between (0,10), and use gw as the value of J; If w < l, let the square root of l be gl, and continue to take the square root of l until the value of lw is between (0,10), and gl is used as the value of J; Step S123: performing Gaussian filtering on the image A to obtain a filtered image of the image A; Step S124: normalize and tilt-correct the filtered image of picture A to obtain picture 1 to be extracted.

4. The method for assisting ultrasonic diagnosis based on machine learning according to claim 3, characterized in that: The specific steps of step S124 are as follows: Step S1241: The filtered image of picture A is recorded as filtered image A; Read the maximum and minimum RGB values ​​of each pixel in the filtered image A, denoted as X MAX and X MIN ; Calculate the average RGB value of each pixel in the filtered image A, denoted as X AVE ; Step S1242: Read the RGB value of each pixel in the filtered image A, denoted as CO (1,1) , CO (1,2) ~CO (w,l) ; Among them, CO (1,1) represents the RGB value corresponding to the pixel at (1,1) in the filtered image A; CO (1,2) represents the RGB value corresponding to the pixel at (1,2) in the filtered image A; and so on, CO (w,w) Represents the RGB value corresponding to the pixel at (w, l) in the filtered image A; Step S1243: Normalize the pixel at (1,1) in the filtered image A and calculate CO (1,1) The standardized value of (1,1) ; Determine CO (1,1) Is it in X MAX ~X AVE between; Step S1244: Repeat the calculation of CO (1,1) The same procedure as the normalized values ​​was used to calculate CO (1,2) ~CO (w,l) The standardized value of is used to complete the standardization of image A and obtain the standardized image A; Step S1245: Use an edge detection algorithm to detect the image edge of the standardized image A; detect the image edge through Hough transform, calculate the rotation angle of the image, and use the rotation transformation instruction to correct the tilt of the image so that the image direction of the standardized image A is aligned with the horizontal direction to obtain the image 1 to be extracted.

5. The method for assisting ultrasonic diagnosis based on machine learning according to claim 2, characterized in that: The specific steps of step S2 are as follows: Step S21: constructing a gray-level co-occurrence matrix according to the image set to be extracted; Based on the gray-level co-occurrence matrix, the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance of each image are calculated to obtain the texture features of the image set to be extracted; Step S22: using the OpenCV library to extract shape features of the images in the image set to be extracted; Step S221: Use the imread() function to read each picture in the picture set to be extracted; Use the Canny edge detection algorithm to find the edges of each image in the image set to be extracted; Step S222: Locate the picture according to the edge of each picture, and use the cv2.findContours function to find the contour of each picture in the picture set to be extracted; For each contour, use the arcLength() function to calculate the length of the contour; Use the contourArea() function to calculate the area of ​​the contour; Use the boundingRect() function to calculate the bounding box of the contour; Use the approxPolyDP() function to calculate the approximate polygon of the contour; Step S223: Summarize the contour length, contour area, contour bounding box and contour approximate polygon of each picture in the picture set to be extracted as the shape feature of the picture set to be extracted; Step S23: Summarize the texture features and shape features of the image set to be extracted as the patient's ultrasound image features, and proceed to step S3.

6. The method for assisting ultrasonic diagnosis based on machine learning according to claim 5, characterized in that: The specific steps of step S21 are as follows: Step S211: extracting texture features of the image set to be extracted; Define a 256×256 empty matrix as the initial gray-level co-occurrence matrix; With 1 as the distance between pixel pairs, 0° as the direction, and 256 levels as grayscale, the greycomatrix function is used to scan the image set to be extracted, fill in the initial grayscale co-occurrence matrix, and obtain the grayscale co-occurrence matrix of the image set to be extracted, which is recorded as co-occurrence matrix A; Step S212: Count the number of pixel pairs in the co-occurrence matrix A, denoted as pip; Count the number of occurrences of each pixel pair in the co-occurrence matrix A, recorded as occ1, occ2~occ pip ; Among them, occ1 represents the number of occurrences of the first pixel pair in the co-occurrence matrix A; occ2 represents the number of occurrences of the second pixel pair in the co-occurrence matrix A; and so on. pip represents the number of occurrences of the pip-th pixel pair in the co-occurrence matrix A; Step S213: Calculate occ1 to occ pip The sum of is denoted as aocc; Calculate the contrast, dissimilarity, homogeneity coefficient, angular second moment and correlation coefficient of the first pixel pair; Step S2131: Assume the first pixel pair is (i, j); wherein i and j both represent grayscale values, and the value range of i and j is 0 to 255; Step S2132: Calculate the homogeneity coefficient of the first pixel pair, denoted as hom(1); Step S214: Repeat the same steps of calculating the contrast, dissimilarity, homogeneity coefficient, angular second moment and correlation coefficient of the first pixel pair, calculate the contrast of the second to pip-th pixel pairs, and obtain con(2) to con(pip).

7. The method for assisting ultrasonic diagnosis based on machine learning according to claim 6, characterized in that: The subsequent steps of step S215 are as follows: Step S216: Calculate the angular mean square difference of the first pixel pair; Step S2161: Count the number of occurrences of the first pixel pair in each picture in the picture set to be extracted, recorded as oc1, oc2~oc pn ; Wherein, pn represents the number of pictures in the picture set to be extracted; oc1 represents the number of times the first pixel pair appears in the first picture to be extracted in the picture set to be extracted; oc2 represents the number of times the first pixel pair appears in the second image to be extracted in the image set to be extracted; By analogy, oc pn Indicates the number of times the first pixel pair appears in the pnth picture to be extracted in the picture set to be extracted; oc1~oc pn The sum is occ1; Step S2162: Calculate the total grayscale value of all pixels in each image in the image set to be extracted, denoted as hn1, hn2~hn pn ; Among them, hn1 represents the total grayscale value of all pixels in the first picture in the picture set to be extracted; hn2 represents the total grayscale value of all pixels in the second picture in the picture set to be extracted; By analogy, hn pn Represents the total grayscale value of all pixels in the pnth image in the image set to be extracted; Step S2163: Calculate the angular mean square difference of the first pixel pair, denoted asm(1); hn i Represents the total grayscale value of all pixels in the i-th image in the image set to be extracted; Step S217: Repeat the same steps of calculating the diagonal average square difference of the first pixel, and calculate the diagonal average square differences of the second to pip-th pixels to obtain asm(2) to asm(pip); Step S218: Summarize the contrast, dissimilarity, homogeneity coefficient, angular second-order moment and correlation coefficient of the picture set to be extracted and the diagonal mean square difference corresponding to the 1st to pipth pixels as the texture features of the picture set to be extracted.

8. An ultrasonic diagnosis assistance system based on machine learning, applicable to any one of the ultrasonic diagnosis assistance methods based on machine learning in claims 1-7, characterized in that: The system comprises: Image acquisition module: used to acquire ultrasound images of patients; pre-process the ultrasound images by denoising, standardizing and tilt correction to obtain a set of images to be extracted; Feature extraction module: used to construct a gray-level co-occurrence matrix based on the image set to be extracted; based on the gray-level co-occurrence matrix, the contrast, dissimilarity, homogeneity coefficient, angular second-order moment, correlation coefficient and angular mean variance corresponding to the images in the image set to be extracted are calculated to obtain the texture features of the image set to be extracted; the shape features of the image set to be extracted are extracted using the OpenCV library to obtain the patient's ultrasound image features; Model training module: used to obtain medical record information in the target hospital; extract ultrasound images of all symptoms in the medical record information as the original image set; Preprocess and extract features of the original picture set to obtain a reference picture set; Using the reference image set as a training set for computer model training, an ultrasound image diagnosis model is constructed; Medical pre-diagnosis module: used to substitute the patient's ultrasound image features into the ultrasound diagnosis model to obtain the patient's disease name; prescribe medicine for the patient based on the disease name.

Images