Image feature extraction method and device of ultrasonic image, equipment and medium

Abstract

The invention provides an image feature extraction method and device of an ultrasonic image, equipment and a medium, and belongs to the field of medical image.The method comprises the steps that a breast ultrasonic image is obtained; determining a target area containing echo features; extracting effective features of the target area; acquiring blood flow richness characteristics; global feature fusion is conducted on the effective features and the blood flow richness features, and target fusion features are obtained in combination with the blood flow proportion. By extracting the multi-scale features of the original breast ultrasonic image with the marked area, the positions of the echo features can be accurately determined, the effective features of the target area can be further extracted, the edge image of the echo features can be determined, the boundary definition of the echo feature image can be improved, then the blood flow richness features can be extracted, and the accuracy of the breast ultrasonic image can be improved. And the effective features and the blood flow richness features are fused to obtain image features of the ultrasonic images at multiple angles, so that the accuracy of the image features is further improved.

Description

Technical Field

[0001] The present invention belongs to the field of medical imaging technology, and in particular relates to an image feature extraction method, device, equipment and storage medium for ultrasonic imaging. Background Art

[0002] Breast BI-RADS 4 categories are two commonly used diagnostic criteria in breast ultrasound examinations, which are used to identify the characteristics, risks, and recommended follow-up treatments of nodules in the breast. However, the two commonly used diagnostic criteria for breast BI-RADS 4 categories in the prior art are not unified, which means that identification according to the commonly used diagnostic criteria cannot accurately reflect the true situation of benign and malignant breast BI-RADS 4 categories. In addition, due to individual differences and different stages of disease development, the imaging performance of breast tissue of different patients may also be different. Different hospitals have different classifications based on images alone, and different doctors have different categories based on images combined with experience. This requires doctors to combine more clinical experience, pathological results and other factors for comprehensive analysis when interpreting images, which further increases the difficulty of identification. Among them, the image processing results of breast ultrasound images are a reference indicator for pathological results.

[0003] In the prior art, image processing of breast ultrasound images usually adopts manually defined feature extraction algorithms, which require doctors to manually select and extract image features of areas where tumors may exist based on their experience and knowledge. However, since breast ultrasound images usually have some fuzzy and atypical features, such as unclear boundaries, irregular shapes, calcification, etc., if feature extraction is performed using the prior art, doctors may need to repeatedly observe and compare multiple images to determine the tumor features in the ultrasound images. In addition, the features of benign and malignant lesions in breast ultrasound images are very complex. The prior art image processing of breast ultrasound images relies too much on the experience and knowledge of doctors, and is easily affected by individual differences and subjective judgments of doctors, resulting in limited reliability and consistency of the results of image feature extraction of areas where tumors may exist, and there may even be a problem of poor accuracy of the extracted features. Summary of the invention

[0004] In order to solve the problem of inaccurate image feature extraction of ultrasound images, the present invention provides an ultrasound image feature extraction method, device, computer equipment and storage medium.

[0005] In order to achieve the above object, the present invention provides the following technical solutions:

[0006] The present invention provides an image feature extraction method for ultrasound images, comprising: acquiring a breast ultrasound image; extracting multi-scale features of the breast ultrasound image, and determining a target area containing echo features according to the multi-scale features; acquiring grayscale feature matrix values, LBP histograms and edge features of the target area, and using the grayscale feature matrix values, LBP histograms and edge features as effective features; extracting an echo feature image in the target area, and acquiring a blood flow richness feature of the echo feature image; and performing global feature fusion on the effective features and the blood flow richness features to obtain a target fusion feature.

[0007] Optionally, after acquiring the breast ultrasound image, the breast ultrasound image is also preprocessed, including: acquiring the original breast ultrasound image; performing grayscale normalization and size adjustment on the original breast ultrasound image, and performing size standardization, denoising and contrast adjustment to obtain a preprocessed image of the breast ultrasound image; and extracting multi-scale features of the preprocessed image.

[0008] Optionally, extracting multi-scale features of the breast ultrasound image, and determining a target area containing echo features based on the multi-scale features includes: extracting features of the breast ultrasound image through a convolutional neural network to obtain a multi-scale feature map; generating multiple candidate areas suspected of having echo features corresponding to the multi-scale feature map; for each generated candidate area, identifying it according to the corresponding scale features to obtain multiple detection frames; eliminating redundant detection frames with an overlap greater than or equal to a preset overlap through a non-maximum suppression algorithm, and determining a target area containing echo features based on the remaining detection frames.

[0009] Optionally, obtaining the grayscale feature matrix value, LBP histogram and edge features of the target area includes: determining the grayscale feature matrix value of the target area through a grayscale co-occurrence matrix; defining a neighborhood structure in the target area, for each pixel in the target area, comparing its grayscale value with the grayscale value of its neighboring pixel points, generating a binary code, converting the binary codes of all pixels into decimal numbers to form an LBP histogram; and extracting the edge features of the target area through a pre-trained artificial neural network model.

[0010] Optionally, extracting the echo characteristic image in the target area includes: using image segmentation technology to identify and extract the echo characteristic image in the target area, determining the edge image of the echo characteristic image, and then obtaining the image coordinates of the edge image; extracting the area corresponding to the image coordinates to obtain the echo characteristic image.

[0011] Optionally, determining the edge image of the echo feature image includes: using image segmentation technology to identify and extract echo feature areas, performing Gaussian smoothing on the echo feature areas to obtain processing areas; respectively determining the grayscale differences in the horizontal and vertical directions of each pixel in the processing area, and determining the edge strength of the pixel through the grayscale differences; filtering each pixel through a non-maximum suppression algorithm based on the edge strength; performing double threshold detection and edge connection on the filtering results to determine the edge image of the echo feature image.

[0012] Optionally, obtaining the image coordinates of the edge image includes: dividing the edge image into multiple grids, each grid having multiple bounding boxes; determining the confidence of each bounding box, adjusting the bounding box according to the confidence, and then obtaining the adjusted bounding box coordinates; converting the bounding box coordinates to obtain the image coordinates of the edge image.

[0013] Secondly, the present invention also provides an image feature extraction device for ultrasonic images, comprising:

[0014] An acquisition module, used for acquiring breast ultrasound images;

[0015] A processing module is used to extract multi-scale features of the breast ultrasound image, and determine a target area containing echo features according to the multi-scale features; obtain grayscale feature matrix values, LBP histograms and edge features of the target area, and use the grayscale feature matrix values, LBP histograms and edge features as effective features; extract echo feature images in the target area, and obtain blood flow richness features of the echo feature images;

[0016] The fusion module is used to perform global feature fusion on the effective feature and the blood flow richness feature to obtain a target fusion feature.

[0017] In addition, the present invention also provides a computer-readable storage medium, wherein the storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned method for extracting image features of ultrasound images is implemented.

[0018] In addition, the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for extracting image features of ultrasound images when executing the program.

[0019] The image feature extraction method of ultrasound images provided by the present invention has the following beneficial effects:

[0020] First, the multi-scale features of the breast ultrasound image are extracted, and the target area containing echo feature information is determined through the multi-scale features, which is conducive to accurately determining the location of the echo feature. On this basis, the effective features of the target area are further extracted, and the edge image of the echo feature is determined through image segmentation technology, which is conducive to improving the boundary clarity of the echo feature image. Then, the blood flow richness feature is obtained according to the intercepted echo feature image, and the blood flow richness feature is fused with the above-mentioned effective features to obtain ultrasound image features at multiple angles, further improving the accuracy of the image features. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiment of the present invention and its design scheme, the following briefly introduces the drawings required for this embodiment. The drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 A schematic flow chart of an image feature extraction method for ultrasonic images provided by the present invention.

[0023] Figure 2 A block diagram of an image feature extraction device for ultrasonic images provided by the present invention.

[0024] Figure 3 A schematic diagram of a computer device for implementing an image feature extraction method for ultrasonic images provided by the present invention. DETAILED DESCRIPTION

[0025] In order to enable those skilled in the art to better understand the technical solution of the present invention and implement it, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and cannot be used to limit the scope of protection of the present invention.

[0026] The technical solutions provided by various embodiments of the present invention are described in detail below in conjunction with the accompanying drawings.

[0027] Figure 1 A schematic diagram of a method for extracting image features of an ultrasound image provided by the present invention is shown in FIG. Figure 1 As shown, the following steps are included:

[0028] S101, obtaining breast ultrasound images.

[0029] The breast ultrasound image of the breast ultrasound image is a breast ultrasound image with annotations of suspected echo features; the annotations are annotations made by the doctor on the original breast ultrasound image based on his / her own experience.

[0030] Specifically, the original breast ultrasound image may be obtained first, and the original breast ultrasound image may be preprocessed to obtain the breast ultrasound image. The preprocessing includes: grayscale normalization and size adjustment of the original breast ultrasound image, and size standardization, denoising and contrast adjustment.

[0031] For example, the original breast ultrasound image is subjected to grayscale normalization and size adjustment, including but not limited to size standardization, denoising, and contrast adjustment, to enhance the quality and contrast of the image; normalization and flipping of the original breast ultrasound image can better solve the problem of brightness change and enhance the image quality, match better accuracy, and reduce the error rate. Secondly, the ultrasound image is subjected to grayscale normalization and other processing, which is converted into a grayscale image matrix, and the distribution of pixel values ​​is adjusted to make the image contrast more uniform and highlight the features of interest. Normalize the data to [0, 1]:

[0032]

[0033] Where max(x) and min(x) are the maximum and minimum values ​​of the data respectively. Use the normalization formula to process each pixel in the image to obtain the normalized pixel value.

[0034] S102: extracting multi-scale features of the breast ultrasound image, and determining a target area containing echo features according to the multi-scale features.

[0035] Specifically, the breast ultrasound image can be feature extracted through a convolutional neural network to obtain a multi-scale feature map; corresponding to the multi-scale feature map, multiple candidate areas suspected of having echo features are generated; for each generated candidate area, identification is performed according to the corresponding scale feature to obtain multiple detection frames; redundant detection frames with an overlap greater than or equal to a preset overlap are eliminated through a non-maximum suppression algorithm, and the target area where the echo features are located is determined based on the remaining detection frames.

[0036] After preprocessing the original breast ultrasound image, there will be significant differences in grayscale between the echo feature tissue and the surrounding normal tissue. Using the ROI (regions of interest) grayscale histogram, these differences can be clearly displayed through the grayscale histogram, providing global grayscale information. Using the ROI regional grid map, regional block analysis or texture analysis is performed, and the region of interest is divided into several small squares to reveal local heterogeneity and detailed features, and the ROI is determined to mark the region of interest.

[0037] For example, the input breast ultrasound image size is 640x640, and features are extracted through a convolutional neural network to obtain a feature map. A dense prediction module is introduced to perform dense prediction on the high-dimensional feature map. A region generation network is used to generate a series of candidate regions ROI from the feature map. The feature pyramid network structure is used to fuse feature maps of different scales. For each generated ROI, the extracted feature set is used to identify the target in the image, and a non-maximum suppression algorithm is used to eliminate redundant ROIs with a high degree of overlap to obtain the final processing result.

[0038] S103, obtaining the grayscale feature matrix value, LBP histogram and edge feature of the target area, and taking the grayscale feature matrix value, LBP histogram and edge feature as effective features.

[0039] Specifically, the grayscale feature matrix value of the target area is determined by the grayscale co-occurrence matrix, the grayscale co-occurrence matrix is ​​determined according to the pixel interval of the target area, and multiple first effective features are extracted from the grayscale co-occurrence matrix; a neighborhood structure is defined in the target area, for each pixel in the target area, its grayscale value is compared with the grayscale value of its neighboring pixel points, a binary code is generated, and the binary codes of all pixels are converted into decimal numbers to form an LBP histogram; the edge features of the target area are extracted by a pre-trained artificial neural network model.

[0040] For example, in the first step, the optimal threshold is found by calculating the variance value between classes.

[0041] Find a threshold T and divide all the pixels of the image into two categories. One category has pixel values ​​less than or equal to T, which is determined as the background area, and the other category has pixel values ​​greater than T, which is determined as the foreground area. When the inter-class variance of the two categories reaches the maximum value, the T value is considered to be the most appropriate threshold.

[0042] The percentage of background pixels is:

[0043]

[0044] The proportion of foreground pixels is:

[0045]

[0046] Calculate the average value of each category and the average gray value of the background:

[0047]

[0048] Average gray value of foreground:

[0049]

[0050] Find the threshold under the maximum inter-class variance in turn and return the calculated threshold. The inter-class variance is as follows:

[0051] g=w1*(μ-μ1) 2 +w2*(μ-μ2) 2

[0052] Among them, Sum is the sum of pixels, N1 is the number of all pixels in the background area, w1 is the ratio of the number of pixels in the background area to the total number of pixels in the image, and μ1 is the average pixel value; N2 is the number of all pixels in the foreground area, w2 is the ratio of the number of pixels in the foreground area to the total number of pixels in the image, and μ2 is the average pixel value; μ is the average pixel value of 0-255. Then analyze the shape, size, and connectivity of the target object to extract the feature set related to the target.

[0053] Step 2: Use the object detection framework of the image model to perform feature detection.

[0054] The target detection framework of the image model can be D2-Net, and the specific detection process is as follows:

[0055]

[0056] Define the feature point (i, j), the maximum index in the feature vector of the pixel point (i, j) is k, and the maximum eigenvalue is in D k The local area is also the maximum value.

[0057] Step 3: Feature extraction based on gray-level co-occurrence matrix, local binary pattern and artificial neural network.

[0058] Calculate the gray level co-occurrence matrix of the target area, extract various statistics from the gray level co-occurrence matrix, and use these features for subsequent echo feature classification or recognition tasks; define a neighborhood structure such as a 3x3 neighborhood in the target area, compare its gray value with the gray value of its neighboring pixels for each pixel in the image, generate a binary code, convert the binary codes of all pixels into decimal numbers to form a local binary pattern histogram, and use the local binary pattern histogram as a feature vector for subsequent echo feature classification or recognition tasks; construct an artificial neural network model, which usually includes an input layer, a hidden layer, and an output layer, and train the artificial neural network model through a sample image. The sample image includes a preprocessed sample breast ultrasound image and its corresponding edge features. The sample breast ultrasound image is used as input to obtain a predicted feature, and the artificial neural network model is trained with the goal of minimizing the difference between the edge feature and the predicted feature, and the edge features of the target area are extracted through the trained artificial neural network model.

[0059] S104: extracting the echo characteristic image in the target area, and obtaining the blood flow richness feature of the echo characteristic image.

[0060] Specifically, an image segmentation technique is used to identify and extract the echo characteristic image in the target area, determine the edge image of the echo characteristic image, and then obtain the image coordinates of the edge image; extract the area corresponding to the image coordinates to obtain the echo characteristic image.

[0061] In one embodiment, the echo feature region is subjected to Gaussian smoothing to obtain a processing region; the grayscale difference of each pixel in the processing region in the horizontal direction and the vertical direction is determined respectively, and the edge strength of the pixel is determined by the grayscale difference; each pixel is filtered by a non-maximum suppression algorithm according to the edge strength; the filtering result is subjected to double threshold detection and edge connection to determine the edge image of the echo feature image. The edge image is divided into multiple grids, each grid having multiple bounding boxes; the confidence of each bounding box is determined, and the bounding box is adjusted according to the confidence to obtain the adjusted bounding box coordinates; the bounding box coordinates are converted to obtain the image coordinates of the edge image.

[0062] For example, the first step is to detect the boundary or area of ​​the echo feature image.

[0063] Image segmentation technology is used to identify and extract echo feature images, and Gaussian smoothing is performed.

[0064]

[0065] Analysis shows that:

[0066] f(x,y)=f(x)*f(y)

[0067] Perform a two-dimensional Gaussian filter on the image and set the image template radius to r, then we have

[0068]

[0069] Among them, the coordinates of the center pixel are x and y are the offsets relative to the center point, * represents the convolution operator, f(x, y) represents the value of the two-dimensional Gaussian function at the point (x, y), σ is the standard deviation of the Gaussian function, and Gaussian blur is applied to remove noise and reduce the recognition of false edges.

[0070] The gradient magnitude and direction are then calculated using the edge detection method of the differential operator.

[0071] The pixel values within the region are calculated using a 3×3 template, and edge detection is achieved by using the difference generated from the gray values of the pixels within a specific region. The differential operator in the horizontal direction is denoted as Fx, and the differential operator in the vertical direction is denoted as Fy. After the convolution operation, the gray differences in the horizontal and vertical directions can be calculated using two templates respectively, and the gradient magnitudes in the horizontal and vertical directions can be obtained.

[0072] Template in the horizontal direction:

[0073]

[0074] Template in the vertical direction:

[0075]

[0076] For each pixel in the input image, convolution operations are performed with the two templates respectively to calculate the gray differences in the horizontal and vertical directions. Then, the sum of the absolute values of the gray differences in the two directions is used as the edge intensity of this pixel.

[0077] Subsequently, non-maximum suppression is carried out. Non-maximum suppression is performed along the direction of the gradient magnitude. In the first step, the orientation of the gradient is calculated first, and then it is compared whether the gradient value in the current direction of the current pixel is the maximum value in this direction within the 3×3 region. In order to better compare the magnitudes of the gradients. First, it is assumed that the gradient changes are continuous between adjacent pixels along the vertical and horizontal directions. Then, the directions of the gradient magnitudes in the X and Y directions of this pixel are compared. If Fx < Fy (and Fx and Fy are in the same direction), set the weight And weight the two pairs of points in this direction in the relevant neighborhood. Finally, non-maximum suppression operation is performed on the gradient value of this point and the weighted gradient values of the other two pairs of points.

[0078] Subsequently, double-threshold detection and edge connection are carried out. Two thresholds are set, namely TL and TH, where those greater than TH are all detected as edges, and those lower than TL are all detected as non-edges. For the intermediate pixels, if they are adjacent to the pixels determined to be edges, they are determined to be edges; otherwise, they are non-edges. Calculate Fx in the x direction and Fy in the y direction respectively, and use the following formula to obtain the final edge image.

[0079]

[0080] Step 2: Analyze the boundaries or regions.

[0081] Resize the image and divide it into S×S grids. Each grid has B bounding boxes, and the center of the bounding box is in the corresponding grid. Each grid is responsible for predicting B bounding boxes. The size is reflected by the rectangular box, so each bounding box must have a width w and a height h. Knowing the center coordinates x and y of the bounding box is equivalent to the position of the bounding box.

[0082] Set each grid to generate B bounding boxes, that is, predict B prediction boxes, represented by Pr(Object). If there is an object in the bounding box, it is 1, otherwise it is 0. It is reflected by the intersection of the box where the object is actually located or the manually labeled box and the predicted bounding box. The closer the predicted box is to the manually labeled box, the larger the intersection of the two ratios is, and the more accurate it is. Set the confidence CS, which is expressed by the following formula:

[0083] CS=Pr(Object)*IoU

[0084] When there is no object in this bounding box, Pr(Object) is 0 and the confidence is 0; when there is an object, Pr(Object) is 1 and the confidence is equal to the intersection-union ratio.

[0085] Step 3: Extract the boundary coordinate points of the area to obtain the precise coordinate values ​​of the echo features.

[0086] The prediction box is fine-tuned to get the anchor box. The position of the anchor box is fixed. Set the coordinates of the center of the anchor box region:

[0087] center_x=cx+0.5

[0088] center_y=cy+0.5

[0089] The center coordinates of the prediction box are generated in the following way:

[0090] bx=cx+σ(tx)

[0091] by=cy+σ(ty)

[0092] Where tx and ty are real numbers, and σ(x) is the defined activation function, which is defined as follows:

[0093]

[0094] The function value of the activation function is between 0 and 1. When tx = ty = 0, bx = cx + 0.5, by = cy + 0.5, the center of the prediction box coincides with the center of the anchor box. The size of the anchor box is ph, pw, and the size of the prediction box is:

[0095] bh=phe th

[0096] bw=pwe tw

[0097] Then we get the coordinates of the prediction box (t x , t y , t w , t h ).

[0098] In addition, the target area can be gray-classified to obtain multiple pixel point groups; the classification gray-scale value that produces the maximum variance between the multiple pixel point groups is determined, and the classification gray-scale value is used as the segmentation threshold of the target area; the blood flow richness is quantified according to the segmentation threshold, and then the blood flow proportion is determined.

[0099] For example, the captured target area is preprocessed, filtered, denoised, enhanced, etc. to improve the detection accuracy of the blood flow signal.

[0100] Then, threshold segmentation, region growing and morphological operations are performed to detect blood flow signals in the image. The grayscale value when the maximum variance is generated between the grayscale classified pixel clusters is used as the overall segmentation threshold of the image. The image grayscale set is set to S = (1, 2, 3, ... i, ... L), and the number of pixels with grayscale level i is set to ni. Then the total number of pixels in the image is After normalization, the number of pixels is obtained:

[0101]

[0102] Among them, P is the grayscale histogram, and t is the threshold for separating two regions. The area ratio of region 1 and region 2 that can be separated by t is calculated from the histogram, and the average grayscale of the entire image and region 1 is The grayscale of area 2 is Average grayscale of the entire image Among them, G is the gray level of the image. Then the relationship between the average gray value of the whole image and the average gray value of area 1 and area 2 is:

[0103] u=u1θ1+u2θ2

[0104] The variance between regions can be obtained by the following expression:

[0105]

[0106] The threshold T is determined from this:

[0107]

[0108] The threshold T is then used to quantify the blood flow richness, and the quantitative parameters of the blood flow area are compared with the parameters of the entire breast area to calculate the blood flow ratio.

[0109] S105. Perform global feature fusion on the effective feature and the blood flow richness feature to obtain a target fusion feature.

[0110] Exemplarily, use a centralized feature pyramid to capture global long-distance dependencies, achieve comprehensive and differentiated feature representations, and implement the effective feature and the blood flow richness feature on the top layer of the feature pyramid.

[0111] Set LVC as an encoder with an inherent dictionary, and the inherent codebook is B = {b1, b2,..., bK}, where N = H × W is the total spatial number of the input feature, where H and W respectively represent the spatial sizes of the height and width of the feature map, learn a set of scaling factors S = {s1, s2,..., sK} for the visual center, and use a set of scale factors s to map xi and bk to the corresponding position information in sequence. The information of the entire image regarding the k-th codeword can be calculated as follows:

[0112]

[0113] where, x i is the i-th pixel point, bk is the k-th learnable visual codeword, and sK is the K-th scale factor. x i - bk is the information regarding the position of each pixel point relative to the codeword, and K is the total number of visual centers. Use φ to fuse all e k , where φ includes a BN layer, a ReLU layer, and an average pooling layer, then obtain the complete information of the entire image regarding K codewords:

[0114]

[0115] After obtaining the output of the codebook, further send e to a fully connected layer and a 3×3 convolutional layer to determine the scaling factor coefficients, and then obtain the target fusion feature. Perform channel-wise addition between the input blood flow richness feature X in and the scaling factor coefficients, and the expression is: Z = X in ⊕ (δ(Conv 3×3 (e))).

[0116] where, δ is the activation function, ⊕ is channel-wise addition, and finally obtain the output target fusion feature, as shown in the following expression:

[0117] LVC(X in ) = X in ⊕ Z

[0118] The target fusion feature obtained by the present invention can assist medical staff in performing relevant diagnoses regarding the breast.

[0119] By adopting the above method, the multi-scale features of the breast ultrasound image are first extracted, and the target area containing echo feature information is determined through the multi-scale features, which is conducive to accurately determining the location of the echo feature. On this basis, the effective features of the target area are further extracted, and the edge image of the echo feature is determined through image segmentation technology, which is conducive to improving the boundary clarity of the echo feature image. Then, the blood flow richness feature is obtained according to the intercepted echo feature image, and the blood flow richness feature is fused with the above effective features to obtain ultrasound image features at multiple angles, further improving the accuracy of the image features.

[0120] When identifying breast BI-RADS Category 4, doctors can use the target fusion features obtained by the technical solution of the present invention as a reference factor for pathological results when interpreting images, and conduct a comprehensive analysis based on more clinical experience, pathological results and other factors to reduce the difficulty of identification.

[0121] In addition, in order to further illustrate the present invention, the present invention also provides a possible implementation of an ultrasonic image feature extraction method, which is as follows:

[0122] S1: The annotated image is subjected to denoising and rotation processing, followed by binarization processing based on the RGB color space. The color extraction interval threshold is set to H: [38, 77], S: [45, 255], V: [50, 255]. The color of each pixel is within the set threshold range. If so, the pixel value is set to 255 (white), otherwise it is set to 0 (black).

[0123] S2: A region generation network is used to generate a series of candidate regions ROI from the feature map. The FPN structure is used. For each generated ROI, the extracted feature set is used to identify the target in the image. The non-maximum suppression algorithm is used to eliminate redundant detection frames with high overlap to obtain the final detection result.

[0124] S3: Find the optimal threshold T by calculating the inter-class variance, and find the threshold under the maximum inter-class variance in turn. The inter-class variance g = w1*(μ-μ1) 2 +w2*(μ-μ2) 2 , analyzes the characteristics of the target object such as shape, size and connectivity, extracts the feature set related to the target, and then extracts the features based on gray-level co-occurrence matrix, local binary pattern and artificial neural network.

[0125] S4: Use image segmentation technology to identify and extract echo feature areas, obtain semantic segmentation maps, and obtain binary mask maps. Perform Gaussian smoothing, using the expression f(x,y)=f(x)*f(y), where f(x,y) represents the value of the two-dimensional Gaussian function at the point (x,y). Use a 3x3 template to calculate the pixel values ​​in the area, and use the difference in the grayscale values ​​of the pixels in a specific area to achieve edge detection. Use edge detection using differential operators to calculate the gradient amplitude and direction, followed by non-maximum suppression, and perform non-maximum suppression along the degree-preserving direction. The first step is to calculate the direction of the gradient. Use The final edge image is obtained by the formula, and finally the boundary coordinate points of the area are extracted to obtain the precise coordinate values ​​of the echo features (0.495139, 0.431424, 0.173611, 0.192708).

[0126] S5: Define the interception area and determine the coordinates of the intercepted area (t x ,t y ,t w ,t h ), convert the coordinates into pixel coordinates, intercept the corresponding area from the original image, and save it as a new image.

[0127] S6: Filter, denoise, enhance, etc. the captured new image to improve the detection accuracy of blood flow signals, calculate the relationship between the average grayscale values ​​of the entire image u=u1θ1+u2θ2, and use the variance between regions as:

[0128]

[0129] Calculating the threshold Then, to quantify the blood flow richness, the quantitative parameters of the blood flow area were compared with the parameters of the entire breast area, and the blood flow proportion was calculated.

[0130] S7: Using a centralized feature pyramid, the image to be processed is input into the convolutional neural network, and multi-scale feature maps are extracted through different layers of the convolutional neural network. Starting from the feature map of the highest layer, the upsampled feature map is fused with the feature map of the next layer by additive fusion. This process is repeated until the required lowest layer is reached to obtain the target fused feature.

[0131] Figure 2 A block diagram of an ultrasonic image feature extraction device provided by the present invention, such as Figure 2 As shown, including:

[0132] The acquisition module 201 is used to acquire breast ultrasound images.

[0133] The processing module 202 is used to extract the multi-scale features of the breast ultrasound image, and determine the target area containing echo features based on the multi-scale features; obtain the grayscale feature matrix value, LBP histogram and edge features of the target area, and use the grayscale feature matrix value, LBP histogram and edge features as effective features; extract the echo feature image in the target area, and obtain the blood flow richness feature of the echo feature image.

[0134] The fusion module 203 is used to perform global feature fusion on the effective feature and the blood flow richness feature to obtain a target fusion feature.

[0135] By using the above-mentioned device, multi-scale features of the breast ultrasound image are first extracted, and the target area containing echo feature information is determined through the multi-scale features, which is conducive to accurately determining the location of the echo feature. On this basis, the effective features of the target area are further extracted, and the edge image of the echo feature is determined through image segmentation technology, which is conducive to improving the boundary clarity of the echo feature image. Thereafter, the blood flow richness feature is obtained based on the intercepted echo feature image, and the blood flow richness feature is fused with the above-mentioned effective features to obtain ultrasound image features at multiple angles, thereby further improving the accuracy of the image features.

[0136] This specification also provides a computer-readable storage medium, which stores a computer program, which can be used to execute the above Figure 1 An image feature extraction method for ultrasound images is provided.

[0137] This manual also provides Figure 3 The structural diagram of the computer device shown in FIG. Figure 3 As shown in the figure, at the hardware level, the computer device includes a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include other hardware required for the business. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to achieve the above Figure 1 An image feature extraction method for ultrasound images is provided.

[0138] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0139] The present invention is described with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as a combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the flowchart. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0140] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0141] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0142] It should be noted that the above specific implementation methods can enable those skilled in the art to more fully understand the invention, but do not limit the invention in any way. Therefore, although the present specification and embodiments have described the invention in detail, those skilled in the art should understand that the invention can still be modified or replaced by equivalents; and all technical solutions and improvements that do not deviate from the spirit and scope of the invention are included in the protection scope of the patent for the invention. Any figure mark in the claims should not be regarded as limiting the claims involved.

Claims

1. A method for extracting image features of ultrasonic images, characterized in that: include: Obtain breast ultrasound images; Extracting multi-scale features of the breast ultrasound image, and determining a target area containing echo features based on the multi-scale features; Obtaining grayscale feature matrix values, LBP histogram and edge features of the target area, and using the grayscale feature matrix values, LBP histogram and edge features as valid features; Extracting the echo characteristic image in the target area, and obtaining the blood flow richness feature of the echo characteristic image; The effective features and the blood flow richness features are globally fused to obtain target fusion features.

2. The method according to claim 1, characterized in that The step of extracting multi-scale features of the breast ultrasound image comprises: Grayscale normalization and size adjustment are performed on the breast ultrasound image, and size standardization, denoising and contrast adjustment are performed to obtain a preprocessed image of the breast ultrasound image; Multi-scale features of the preprocessed image are extracted.

3. The method according to claim 1, characterized in that Determining the target area containing echo features according to the multi-scale features comprises: Corresponding to the multi-scale feature map, generating a plurality of candidate regions suspected to have echo features; For each generated candidate region, identify it according to the corresponding scale feature to obtain multiple detection boxes; The redundant detection frames with overlap greater than or equal to the preset overlap are eliminated through the non-maximum suppression algorithm, and the target area containing echo features is determined based on the remaining detection frames.

4. The method according to claim 1, characterized in that: The step of obtaining the grayscale feature matrix value, LBP histogram and edge feature of the target area includes: Determine the grayscale feature matrix value of the target area through the grayscale co-occurrence matrix; A neighborhood structure is defined in the target area, and for each pixel in the target area, its grayscale value is compared with the grayscale value of its neighboring pixel points to generate a binary code, and the binary codes of all pixels are converted into decimal numbers to form an LBP histogram; The edge features of the target area are extracted by a pre-trained artificial neural network model.

5. The method according to claim 1, characterized in that Extracting the echo characteristic image in the target area includes: Using image segmentation technology to identify and extract the echo characteristic image in the target area, determine the edge image of the echo characteristic image, and then obtain the image coordinates of the edge image; The region corresponding to the image coordinates is extracted to obtain an echo characteristic image.

6. The method according to claim 5, characterized in that Determining the edge image of the echo characteristic image comprises: Perform Gaussian smoothing on the extracted echo feature image to obtain the processing area; Determine the grayscale difference of each pixel in the processing area in the horizontal direction and the vertical direction respectively, and determine the edge strength of the pixel through the grayscale difference; According to the edge strength, each pixel is filtered by a non-maximum suppression algorithm; Double threshold detection and edge connection are performed on the filtering result to determine the edge image of the echo feature image.

7. The method according to claim 6, characterized in that Acquiring the image coordinates of the edge image includes: Dividing the edge image into a plurality of grids, each grid having a plurality of bounding boxes; Determining a confidence level of each bounding box, and adjusting the bounding box according to the confidence level to obtain adjusted bounding box coordinates; The bounding box coordinates are transformed to obtain image coordinates of the edge image.

8. An image feature extraction device for ultrasonic images, characterized in that: include: An acquisition module, used for acquiring breast ultrasound images; A processing module is used to extract multi-scale features of the breast ultrasound image, and determine a target area containing echo features according to the multi-scale features; obtain grayscale feature matrix values, LBP histograms and edge features of the target area, and use the grayscale feature matrix values, LBP histograms and edge features as effective features; extract echo feature images in the target area, and obtain blood flow richness features of the echo feature images; The fusion module is used to perform global feature fusion on the effective feature and the blood flow richness feature to obtain a target fusion feature.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method described in any one of claims 1 to 7 is implemented.

Images