Orthopedic ultrasound image segmentation method and system based on machine learning

Abstract

The invention discloses an orthopaedic ultrasound image segmentation method and system based on machine learning. The method comprises the steps of data acquisition, basic preprocessing, edge information enhancement, feature coding and decoding and ultrasound image segmentation. The invention relates to the technical field of ultrasonic image segmentation, in particular to an orthopedic ultrasonic image segmentation method and system based on machine learning, and the method comprises the steps: carrying out the edge information enhancement through employing a method for generating middle-valley ultrasonic parameter and entropy ultrasonic parameter images, and highlighting the edge information; a feature coding and decoding structure is combined with a network structure of a feature fusion module, a machine learning method is adopted, feature coding and decoding are carried out based on the ultrasonic parameter image data, and the feature fusion performance is improved; and the channel perception fusion unit is adopted to perform feature fusion optimization, so that the feature fusion performance in the feature map is improved, and high-quality ultrasonic image features are extracted.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image segmentation, and in particular to an orthopedic ultrasonic image segmentation method and system based on machine learning. Background Art

[0002] The orthopedic ultrasound image segmentation method based on machine learning is a process of using machine learning algorithms to classify pixels in orthopedic ultrasound images according to different objects or structures. Orthopedic ultrasound image segmentation aims to separate and locate different tissue structures such as bones, joints, and muscles in the image. This method helps to improve the understanding and application of orthopedic ultrasound images and further promote progress in the field of orthopedics.

[0003] However, in the existing orthopedic ultrasound image segmentation methods, there is a technical problem that the ultrasound imaging modes are multi-modal, multi-level and multi-type, and thus a method that can fuse multiple types of features is needed; in the existing orthopedic ultrasound image segmentation methods, there is a technical problem that the segmentation performance of single-modal images is low, while the feature encoding and decoding and feature fusion aggregation of multi-modal feature images are difficult; in the existing orthopedic ultrasound image segmentation methods, there is a technical problem that in the process of multi-modal feature map fusion, simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy. Summary of the invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an orthopedic ultrasound image segmentation method and system based on machine learning. In view of the technical problem that in the existing orthopedic ultrasound image segmentation methods, there are multi-modal, multi-level and multi-type characteristics of ultrasound imaging modes, and thus a method that can fuse multiple types of features is needed, this scheme creatively adopts the method of generating mid-valley ultrasound parameter and entropy ultrasound parameter image to enhance edge information, highlighting the edge information and improving the accuracy of subsequent segmentation tasks; in view of the fact that in the existing orthopedic ultrasound image segmentation methods, the segmentation performance of single-modal images is low, and the feature encoding of multi-modal feature images is To address the technical problem of high difficulty in aggregation of decoding and feature fusion, this solution creatively adopts a feature encoding and decoding structure combined with a network structure of a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data, thereby improving the performance of feature fusion; in view of the technical problem that in the existing orthopedic ultrasound image segmentation method, simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy during the fusion of multimodal feature maps, this solution creatively adopts a channel-aware fusion unit to optimize feature fusion, thereby improving the performance of feature fusion in feature maps, extracting high-quality ultrasound image features, and optimizing the overall process of image segmentation.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides an orthopedic ultrasound image segmentation method based on machine learning, the method comprising the following steps:

[0006] Step S1: data collection;

[0007] Step S2: basic preprocessing;

[0008] Step S3: edge information enhancement;

[0009] Step S4: feature encoding and decoding;

[0010] Step S5: Ultrasonic image segmentation.

[0011] Furthermore, in step S1, the data acquisition is used to acquire the orthopedic ultrasonic image to be segmented, specifically, to acquire the original data of the orthopedic ultrasonic image from the ultrasonic imaging system.

[0012] Furthermore, in step S2, the basic preprocessing is used to perform data preprocessing on the original ultrasonic image, specifically, filtering, denoising and normalizing the original data of the orthopedic ultrasonic image to obtain preprocessed ultrasonic image data.

[0013] Further, in step S3, the edge information enhancement is used to enhance the edge information of the ultrasound image by generating an ultrasound parameter image, specifically by generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data, performing edge information enhancement to obtain ultrasound image data, including the following steps:

[0014] Step S31: mid-valley parameter image generation, which is used to generate distribution statistics describing ultrasound images and echo intensity, and obtain echo feature information of different image areas, specifically including the following steps:

[0015] Step S311: construct the probability density function of the mid-valley parameter, and the calculation formula is: ;

[0016] Wherein, f(·) is the probability density function of the mid-valley parameter, r is the distance variable used to represent the distance from the pixel point in the preprocessed ultrasound image data to the image center, m is the shape parameter, is the Euler gamma function, is the scaling parameter, exp(·) is the natural exponential function, and U(·) is the unit step function;

[0017] Step S312: construct shape parameters, the calculation formula is: ;

[0018] Wherein, m is a shape parameter, E(·) is an expected function, and R is an ultrasonic backscattering envelope, wherein the ultrasonic backscattering envelope R is used to represent the intensity value of the echo signal at different distances;

[0019] Step S313: generating a mid-valley parameter image, specifically generating a mid-valley parameter image by constructing the probability density function and the shape parameters, and obtaining ultrasonic mid-valley parameter image data;

[0020] Step S32: generating an entropy parameter image for enhancing the texture information of the ultrasound image, specifically comprising the following steps:

[0021] Step S321: entropy value calculation, specifically, using a sliding window algorithm to obtain local signal information from the beamforming analog signal data, and calculating the entropy value by constructing an entropy value probability density function. The entropy value calculation formula is: ;

[0022] Where H is the entropy value, x is the pixel value in the preprocessed ultrasound image data, and x max is the maximum pixel value, xmin is the minimum pixel value, and f(·) is the probability density function;

[0023] Step S322: assigning a new pixel value, specifically assigning the central pixel value of the square window in the sliding window algorithm to the value of the entropy value H;

[0024] Step S323: Window movement, specifically, moving the window in the sliding window algorithm within the data range, repeating the operations described in steps S321 and S322, and generating an ultrasound entropy parameter image;

[0025] Step S324: generating an ultrasonic entropy parameter image, specifically generating the ultrasonic entropy parameter image by calculating the entropy value, assigning the new pixel value and moving the window, to obtain ultrasonic entropy parameter image data;

[0026] Step S33: edge information enhancement, specifically, performing edge information enhancement on the ultrasound image by generating the mid-valley parameter image and the entropy parameter image to obtain ultrasound parameter image data.

[0027] Further, in step S4, the feature encoding and decoding is used to perform feature fusion and extract multimodal image features, specifically using a feature encoding and decoding structure combined with a network structure of a feature fusion module, and using a machine learning method, based on the ultrasound parameter image data, to perform feature encoding and decoding to obtain ultrasound image feature data;

[0028] The feature encoding and decoding structure is combined with the network structure of the feature fusion module, including a feature encoder sub-block, a feature fusion sub-block and a feature decoder sub-block;

[0029] The feature encoder sub-block is used to extract features of the multimodal image, and specifically includes an input layer and four encoding layers;

[0030] The feature fusion sub-block is used to fuse features of the same dimension, and specifically includes a maximum fusion unit, a sum fusion unit, a convolution fusion unit, and a channel perception fusion unit;

[0031] The feature decoder sub-block is used to restore the fused features into high-level semantic features, and specifically includes four decoding layers and one output layer;

[0032] The step of using a feature encoding and decoding structure combined with a network structure of a feature fusion module and a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data includes:

[0033] Step S41: constructing a feature encoder sub-block, comprising the following steps:

[0034] Step S411: constructing a basic structure of an image encoder, specifically using a pre-trained ResNet-34 network as an image encoder, deleting the average pooling layer and the fully connected layer of the ResNet-34 network, and constructing an input layer and four encoding layers. The calculation formula for constructing the basic structure of the image encoder is: ;

[0035] Where Q(·) is the coding layer representation function, X is the layer index, Q1 is the first coding layer, Q2 is the second coding layer, Q3 is the third coding layer, and Q4 is the fourth coding layer;

[0036] Step S412: Generate a multi-modal five-stage feature map, specifically a two-dimensional ultrasound modal feature map, a mid-valley ultrasound parameter modal feature map, and an entropy ultrasound parameter modal feature map, and the calculation formula is: ;

[0037] In the formula, F i is a multimodal five-stage feature map, i is the feature map stage index, and the value range of i is {1, 2, 3, 4, 5}, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram;

[0038] Step S413: feature coding, specifically, performing feature coding by constructing the basic structure of the image encoder and generating the multimodal five-stage feature map to obtain the ultrasonic image coding feature;

[0039] Step S42: constructing a feature fusion sub-block, including the following steps:

[0040] Step S421: construct the maximum fusion unit, the calculation formula is: ;

[0041] In the formula, is the fused feature map output by the maximum fusion unit, i is the feature map stage index, max(·) is the maximum value function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram;

[0042] Step S422: construct a summing and fusion unit, and the calculation formula is: ;

[0043] In the formula, is the fused feature map output by the summation fusion unit, i is the feature map stage index, is the 2D ultrasound weighted parameter, is the two-dimensional ultrasonic modal characteristic diagram, is the mid-valley weighting parameter, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy weighting parameter, is the entropy ultrasound parameter modal characteristic diagram;

[0044] Step S423: construct a convolution fusion unit, and the calculation formula is: ;

[0045] In the formula, is the concatenated feature map, i is the feature map stage index, is the fused feature map output by the convolutional fusion unit, Concat(·) is the concatenation operator, Conv 3×3 (·) is the convolution operator corresponding to a convolutional layer of size 3×3, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram;

[0046] Step S424: construct a channel-aware fusion unit, specifically, by adopting the global average pooling operation and the global maximum pooling operation to obtain the global channel information of the cascade feature map, and using the shared multi-layer perception layer to generate channel parameters, using the channel parameters and adopting channel-aware fusion to generate a fused feature map, specifically including the following steps:

[0047] Step S4241: construct channel parameters, the calculation formula is: ;

[0048] In the formula, is the average pooling channel coefficient, is the maximum pooling channel coefficient, i is the feature map stage index, M r (·) is a shared multi-layer perceptual operation function, which is used to construct the output channel of the fully connected layer, where r is a balancing coefficient with a value of 4, which is used to balance performance and computational loss, P avg (·) is the global average pooling operation function, P max (·) is the global maximum pooling operation function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram, [·:·] is the concatenation operator;

[0049] Step S4242: Calculate the global channel attention, the calculation formula is: ;

[0050] In the formula, A i is the global channel attention, is a S-shaped function, is the average pooling channel coefficient, is the maximum pooling channel coefficient, is the element-wise sum operator;

[0051] Step S4243: Calculate the fusion feature map, the calculation formula is: ;

[0052] Step S425: feature fusion, specifically, performing feature fusion by constructing the maximum fusion unit, the summation fusion unit, the convolution fusion unit, and the channel perception fusion unit to obtain a fusion feature map. ;

[0053] Step S43: constructing a feature decoder sub-block, including the following steps:

[0054] Step S431: construct a decoding layer, the decoding layer is used to include two convolutional layers of size 1×1 and one deconvolutional layer of size 3×3. The calculation formula for constructing the decoding layer is: ;

[0055] Where D j is the decoding layer identifier, j is the decoding layer index, where j ranges from {1, 2, 3, 4}, Conv 1×1 (·) is the convolution operator corresponding to the convolutional layer of size 1×1, DC 3×3 (·) is the convolution operator corresponding to the deconvolution layer of size 3×3, D j-1 is the decoding layer identifier of layer j-1, is the element-wise sum operator, It is the fusion feature map corresponding to the 6-j stage;

[0056] Step S432: construct an output layer, the output layer is used to calculate the final output value, including two convolutional layers of size 3×3 and one deconvolutional layer of size 4×4. The calculation formula for constructing the output layer is: ;

[0057] In the formula, is the codec output value, used to segment the image, Conv 3×3 (·) is the convolution operator corresponding to the convolutional layer of size 3×3, DC 4×4 (·) is the convolution operator corresponding to the deconvolution layer of size 4×4, D 4 is the layer 4 decoding layer identifier, is the element-wise sum operator, It is the fusion feature map of the first stage;

[0058] Step S44: feature encoding and decoding model training, specifically, performing feature encoding and decoding model training through the feature encoder sub-block, the feature fusion sub-block and the feature decoder sub-block to obtain a feature encoding and decoding model Model CA ;

[0059] Step S45: feature information processing, specifically using the feature encoding and decoding model Model CA , based on the ultrasonic parameter image data, feature encoding and decoding are performed to obtain ultrasonic image feature data.

[0060] Furthermore, in step S5, the ultrasonic image segmentation is used to perform ultrasonic image segmentation based on ultrasonic image information, specifically using the feature information processing method in the feature encoding and decoding and the machine learning neural network model to perform ultrasonic image segmentation to obtain orthopedic ultrasonic image segmentation data.

[0061] The present invention provides an orthopedic ultrasound image segmentation system based on machine learning, comprising a data acquisition module, a basic preprocessing module, an edge information enhancement module, a feature encoding and decoding module and an ultrasound image segmentation module;

[0062] The data acquisition module is used for data acquisition, obtains the original data of the orthopedic ultrasonic image through acquisition, and sends the original data of the orthopedic ultrasonic image to the basic preprocessing module;

[0063] The basic preprocessing module is used for basic data preprocessing, obtains preprocessed ultrasound image data through basic data preprocessing, and sends the preprocessed ultrasound image data to the edge information enhancement module;

[0064] The edge information enhancement module is used for edge information enhancement, obtains ultrasound parameter image data through edge information enhancement, and sends the ultrasound parameter image data to the feature encoding and decoding module;

[0065] The feature encoding and decoding module is used for feature encoding and decoding, obtains ultrasonic image feature data through feature encoding and decoding, and sends the ultrasonic image feature data to the ultrasonic image segmentation module;

[0066] The ultrasonic image segmentation module is used for ultrasonic image segmentation, and obtains orthopedic ultrasonic image segmentation data through ultrasonic image segmentation.

[0067] The beneficial effects achieved by the present invention using the above scheme are as follows: (1) In view of the technical problem that the existing orthopedic ultrasound image segmentation methods have the characteristics of multi-modality, multi-level and multi-type of ultrasound imaging modes, and thus require a method that can integrate multiple types of features, this scheme creatively adopts the method of generating mid-valley ultrasound parameter and entropy ultrasound parameter images to enhance edge information, highlighting the edge information and improving the accuracy of subsequent segmentation tasks;

[0068] (2) In view of the technical problems that the segmentation performance of single-modality images is low and the aggregation of feature encoding and decoding and feature fusion of multi-modality feature images is difficult in the existing orthopedic ultrasound image segmentation methods, this solution creatively adopts a network structure that combines a feature encoding and decoding structure with a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data, thereby improving the performance of feature fusion;

[0069] (3) In the existing orthopedic ultrasound image segmentation methods, there is a technical problem that simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy during the fusion of multimodal feature maps. This scheme creatively uses a channel-aware fusion unit to optimize feature fusion, improves the performance of feature fusion in feature maps, extracts high-quality ultrasound image features, and optimizes the overall process of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] Figure 1 A schematic diagram of a flow chart of an orthopedic ultrasound image segmentation method based on machine learning provided by the present invention;

[0071] Figure 2 A schematic diagram of an orthopedic ultrasound image segmentation system based on machine learning provided by the present invention;

[0072] Figure 3 This is a schematic diagram of the process of enhancing feature information in step S3;

[0073] Figure 4 This is a schematic diagram of the feature encoding and decoding process of step S4;

[0074] Figure 5 A schematic diagram of the process of constructing a feature fusion sub-block in step S42.

[0075] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0077] In the description of the present invention, it should be understood that terms such as “upper”, “lower”, “front”, “back”, “left”, “right”, “top”, “bottom”, “inside” and “outside” indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0078] Example 1, see Figure 1 The present invention provides an orthopedic ultrasound image segmentation method based on machine learning, the method comprising the following steps: Step S1: data collection;

[0079] Step S2: basic preprocessing;

[0080] Step S3: edge information enhancement;

[0081] Step S4: feature encoding and decoding;

[0082] Step S5: Ultrasonic image segmentation.

[0083] Example 2, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S1, the data acquisition is used to acquire the orthopedic ultrasound image to be segmented, specifically, to acquire the original data of the orthopedic ultrasound image from the ultrasound imaging system;

[0084] In step S2, the basic preprocessing is used to perform data preprocessing on the original ultrasonic image, specifically, filtering, denoising and normalizing the original data of the orthopedic ultrasonic image to obtain preprocessed ultrasonic image data.

[0085] Example 3, see Figure 1 , Figure 2 and Figure 3 This embodiment is based on the above embodiment. In step S3, the edge information enhancement is used to enhance the edge information of the ultrasound image by generating an ultrasound parameter image. Specifically, the edge information is enhanced by generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data to obtain ultrasound image data, including the following steps:

[0086] Step S31: mid-valley parameter image generation, which is used to generate distribution statistics describing ultrasound images and echo intensity, and obtain echo feature information of different image areas, specifically including the following steps:

[0087] Step S311: construct the probability density function of the mid-valley parameter, and the calculation formula is: ;

[0088] Wherein, f(·) is the probability density function of the mid-valley parameter, r is the distance variable used to represent the distance from the pixel point in the preprocessed ultrasound image data to the image center, m is the shape parameter, is the Euler gamma function, is the scaling parameter, exp(·) is the natural exponential function, and U(·) is the unit step function;

[0089] Step S312: construct shape parameters, the calculation formula is: ;

[0090] Wherein, m is a shape parameter, E(·) is an expected function, and R is an ultrasonic backscattering envelope, wherein the ultrasonic backscattering envelope R is used to represent the intensity value of the echo signal at different distances;

[0091] Step S313: generating a mid-valley parameter image, specifically generating a mid-valley parameter image by constructing the probability density function and the shape parameters, and obtaining ultrasonic mid-valley parameter image data;

[0092] Step S32: generating an entropy parameter image for enhancing the texture information of the ultrasound image, specifically comprising the following steps:

[0093] Step S321: entropy value calculation, specifically, using a sliding window algorithm to obtain local signal information from the beamforming analog signal data, and calculating the entropy value by constructing an entropy value probability density function. The entropy value calculation formula is: ;

[0094] Where H is the entropy value, x is the pixel value in the preprocessed ultrasound image data, and x max is the maximum pixel value, xmin is the minimum pixel value, and f(·) is the probability density function;

[0095] Step S322: assigning a new pixel value, specifically assigning the central pixel value of the square window in the sliding window algorithm to the value of the entropy value H;

[0096] Step S323: Window movement, specifically, moving the window in the sliding window algorithm within the data range, repeating the operations described in steps S321 and S322, and generating an ultrasound entropy parameter image;

[0097] Step S324: generating an ultrasonic entropy parameter image, specifically generating the ultrasonic entropy parameter image by calculating the entropy value, assigning the new pixel value and moving the window, to obtain ultrasonic entropy parameter image data;

[0098] Step S33: edge information enhancement, specifically, performing edge information enhancement on the ultrasound image by generating the mid-valley parameter image and the entropy parameter image to obtain ultrasound parameter image data.

[0099] By performing the above operations, in view of the technical problem that in the existing orthopedic ultrasound image segmentation methods, the ultrasound imaging modes are multi-modal, multi-level and multi-type, and a method that can fuse multiple types of features is needed, this scheme creatively adopts the method of generating mid-valley ultrasound parameter and entropy ultrasound parameter images to enhance edge information, highlighting the edge information and improving the accuracy of subsequent segmentation tasks.

[0100] Example 4, see Figure 1 , Figure 2 , Figure 4 and Figure 5 This embodiment is based on the above embodiment. In step S4, the feature encoding and decoding is used to perform feature fusion and extract multimodal image features. Specifically, a feature encoding and decoding structure is combined with a network structure of a feature fusion module, and a machine learning method is used to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data;

[0101] The feature encoding and decoding structure is combined with the network structure of the feature fusion module, including a feature encoder sub-block, a feature fusion sub-block and a feature decoder sub-block;

[0102] The feature encoder sub-block is used to extract features of the multimodal image, and specifically includes an input layer and four encoding layers;

[0103] The feature fusion sub-block is used to fuse features of the same dimension, and specifically includes a maximum fusion unit, a sum fusion unit, a convolution fusion unit, and a channel perception fusion unit;

[0104] The feature decoder sub-block is used to restore the fused features into high-level semantic features, and specifically includes four decoding layers and one output layer;

[0105] The step of using a feature encoding and decoding structure combined with a network structure of a feature fusion module and a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data includes:

[0106] Step S41: constructing a feature encoder sub-block, comprising the following steps:

[0107] Step S411: constructing a basic structure of an image encoder, specifically using a pre-trained ResNet-34 network as an image encoder, deleting the average pooling layer and the fully connected layer of the ResNet-34 network, and constructing an input layer and four encoding layers. The calculation formula for constructing the basic structure of the image encoder is: ;

[0108] Where Q(·) is the coding layer representation function, X is the layer index, Q1 is the first coding layer, Q2 is the second coding layer, Q3 is the third coding layer, and Q4 is the fourth coding layer;

[0109] Step S412: Generate a multi-modal five-stage feature map, specifically a two-dimensional ultrasound modal feature map, a mid-valley ultrasound parameter modal feature map, and an entropy ultrasound parameter modal feature map, and the calculation formula is: ;

[0110] In the formula, Fi is a multimodal five-stage feature map, i is the feature map stage index, and the value range of i is {1, 2, 3, 4, 5}, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram;

[0111] Step S413: feature coding, specifically, performing feature coding by constructing the basic structure of the image encoder and generating the multimodal five-stage feature map to obtain the ultrasonic image coding feature;

[0112] Step S42: constructing a feature fusion sub-block, including the following steps:

[0113] Step S421: construct the maximum fusion unit, the calculation formula is: ;

[0114] In the formula, is the fused feature map output by the maximum fusion unit, i is the feature map stage index, max(·) is the maximum value function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram;

[0115] Step S422: construct a summing and fusion unit, and the calculation formula is: ;

[0116] In the formula, is the fused feature map output by the summation fusion unit, i is the feature map stage index, is the 2D ultrasound weighted parameter, is the two-dimensional ultrasonic modal characteristic diagram, is the mid-valley weighting parameter, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy weighting parameter, is the entropy ultrasound parameter modal characteristic diagram;

[0117] Step S423: construct a convolution fusion unit, and the calculation formula is: ;

[0118] In the formula, is the concatenated feature map, i is the feature map stage index, is the fused feature map output by the convolutional fusion unit, Concat(·) is the concatenation operator, Conv 3×3 (·) is the convolution operator corresponding to a convolutional layer of size 3×3, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram;

[0119] Step S424: construct a channel-aware fusion unit, specifically, by adopting the global average pooling operation and the global maximum pooling operation to obtain the global channel information of the cascade feature map, and using the shared multi-layer perception layer to generate channel parameters, using the channel parameters and adopting channel-aware fusion to generate a fused feature map, specifically including the following steps:

[0120] Step S4241: construct channel parameters, the calculation formula is: ;

[0121] In the formula, is the average pooling channel coefficient, is the maximum pooling channel coefficient, i is the feature map stage index, M r (·) is a shared multi-layer perceptual operation function, which is used to construct the output channel of the fully connected layer, where r is a balancing coefficient with a value of 4, which is used to balance performance and computational loss, P avg (·) is the global average pooling operation function, P max (·) is the global maximum pooling operation function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram, [·:·] is the concatenation operator;

[0122] Step S4242: Calculate the global channel attention, the calculation formula is: ;

[0123] In the formula, A i is the global channel attention, is a S-shaped function, is the average pooling channel coefficient, is the maximum pooling channel coefficient, is the element-wise sum operator;

[0124] Step S4243: Calculate the fusion feature map, the calculation formula is: ;

[0125] Step S425: feature fusion, specifically, performing feature fusion by constructing the maximum fusion unit, the summation fusion unit, the convolution fusion unit, and the channel perception fusion unit to obtain a fusion feature map. ;

[0126] Step S43: constructing a feature decoder sub-block, including the following steps:

[0127] Step S431: construct a decoding layer, the decoding layer is used to include two convolutional layers of size 1×1 and one deconvolutional layer of size 3×3. The calculation formula for constructing the decoding layer is: ;

[0128] Where D j is the decoding layer identifier, j is the decoding layer index, where j ranges from {1, 2, 3, 4}, Conv 1×1 (·) is the convolution operator corresponding to the convolutional layer of size 1×1, DC 3×3 (·) is the convolution operator corresponding to the deconvolution layer of size 3×3, D j-1 is the decoding layer identifier of layer j-1, is the element-wise sum operator, It is the fusion feature map corresponding to the 6-j stage;

[0129] Step S432: construct an output layer, the output layer is used to calculate the final output value, including two convolutional layers of size 3×3 and one deconvolutional layer of size 4×4. The calculation formula for constructing the output layer is: ;

[0130] In the formula, is the codec output value, used to segment the image, Conv 3×3 (·) is the convolution operator corresponding to the convolutional layer of size 3×3, DC 4×4 (·) is the convolution operator corresponding to the deconvolution layer of size 4×4, D 4 is the layer 4 decoding layer identifier, is the element-wise sum operator, It is the fusion feature map of the first stage;

[0131] Step S44: feature encoding and decoding model training, specifically, performing feature encoding and decoding model training through the feature encoder sub-block, the feature fusion sub-block and the feature decoder sub-block to obtain a feature encoding and decoding model Model CA ;

[0132] Step S45: feature information processing, specifically using the feature encoding and decoding model Model CA , based on the ultrasonic parameter image data, feature encoding and decoding are performed to obtain ultrasonic image feature data.

[0133] By performing the above operations, in order to solve the technical problems that the segmentation performance of single-modal images is low and the aggregation difficulty of feature encoding and decoding and feature fusion of multi-modal feature images is high in the existing orthopedic ultrasonic image segmentation methods, this solution creatively adopts a feature encoding and decoding structure combined with a network structure of a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasonic parameter image data, thereby improving the performance of feature fusion; in order to solve the technical problems that in the existing orthopedic ultrasonic image segmentation methods, simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy in the process of multi-modal feature map fusion, this solution creatively adopts a channel-aware fusion unit to perform feature fusion optimization, thereby improving the performance of feature fusion in feature maps, refining high-quality ultrasonic image features, and optimizing the overall process of image segmentation.

[0134] Example 5, see Figure 1 and Figure 2 In step S5, the ultrasonic image segmentation is used to perform ultrasonic image segmentation based on the ultrasonic image information, specifically using the feature information processing method in the feature encoding and decoding and the machine learning neural network model to perform ultrasonic image segmentation to obtain orthopedic ultrasonic image segmentation data.

[0135] Example 6, see Figure 1 and Figure 2 , this embodiment is based on the above embodiment, and the present invention provides an orthopedic ultrasound image segmentation system based on machine learning, including a data acquisition module, a basic preprocessing module, an edge information enhancement module, a feature encoding and decoding module and an ultrasound image segmentation module;

[0136] The data acquisition module is used for data acquisition, obtains the original data of the orthopedic ultrasonic image through acquisition, and sends the original data of the orthopedic ultrasonic image to the basic preprocessing module;

[0137] The basic preprocessing module is used for basic data preprocessing, obtains preprocessed ultrasound image data through basic data preprocessing, and sends the preprocessed ultrasound image data to the edge information enhancement module;

[0138] The edge information enhancement module is used for edge information enhancement, obtains ultrasound parameter image data through edge information enhancement, and sends the ultrasound parameter image data to the feature encoding and decoding module;

[0139] The feature encoding and decoding module is used for feature encoding and decoding, obtains ultrasonic image feature data through feature encoding and decoding, and sends the ultrasonic image feature data to the ultrasonic image segmentation module;

[0140] The ultrasonic image segmentation module is used for ultrasonic image segmentation, and obtains orthopedic ultrasonic image segmentation data through ultrasonic image segmentation.

[0141] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0142] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

[0143] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for segmenting orthopedic ultrasound images based on machine learning, characterized in that: The method comprises the following steps: Step S1: data collection; Step S2: basic preprocessing; Step S3: edge information enhancement; Step S4: feature encoding and decoding; Step S5: ultrasonic image segmentation; In step S1, the data acquisition is used to acquire the orthopedic ultrasound image to be segmented, specifically, to acquire the original data of the orthopedic ultrasound image from the ultrasound imaging system; In step S2, the basic preprocessing is used to perform data preprocessing on the original ultrasonic image, specifically, filtering, denoising and normalizing the original data of the orthopedic ultrasonic image to obtain preprocessed ultrasonic image data; In step S3, the edge information enhancement is used to enhance the edge information of the ultrasound image by generating an ultrasound parameter image, specifically by generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data, performing edge information enhancement to obtain ultrasound image data; In step S4, the feature encoding and decoding is used to perform feature fusion and extract multimodal image features, specifically using a feature encoding and decoding structure combined with a network structure of a feature fusion module, and using a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data; The step of using a network structure combining a feature encoding and decoding structure with a feature fusion module and a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data includes: step S41: constructing a feature encoder sub-block; step S42: constructing a feature fusion sub-block; step S43: constructing a feature decoder sub-block; step S44: feature encoding and decoding model training; step S45: feature information processing; In step S5, the ultrasonic image segmentation is used to perform ultrasonic image segmentation based on ultrasonic image information, specifically using the feature information processing method in the feature encoding and decoding and the machine learning neural network model to perform ultrasonic image segmentation to obtain orthopedic ultrasonic image segmentation data.

2. The orthopedic ultrasound image segmentation method based on machine learning according to claim 1, characterized in that: In step S3, the method of generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data and performing edge information enhancement to obtain ultrasound image data includes the following steps: Step S31: generating a mid-valley parameter image; Step S32: generating an entropy parameter image; Step S33: Edge information enhancement.

3. The orthopedic ultrasound image segmentation method based on machine learning according to claim 2, characterized in that: In step S31, the mid-valley parameter image is generated to generate distribution statistics describing the ultrasound image and echo intensity and obtain echo feature information of different image regions, specifically including the following steps: Step S311: construct the probability density function of the mid-valley parameter, and the calculation formula is: ； Wherein, f(·) is the probability density function of the mid-valley parameter, r is the distance variable used to represent the distance from the pixel point in the preprocessed ultrasound image data to the image center, m is the shape parameter, is the Euler gamma function, is the scaling parameter, exp(·) is the natural exponential function, and U(·) is the unit step function; Step S312: construct shape parameters, the calculation formula is: ； Wherein, m is a shape parameter, E(·) is an expected function, and R is an ultrasonic backscattering envelope, wherein the ultrasonic backscattering envelope R is used to represent the intensity value of the echo signal at different distances; Step S313: generating a mid-valley parameter image, specifically generating a mid-valley parameter image by constructing the probability density function and the shape parameters, and obtaining ultrasonic mid-valley parameter image data.

4. The orthopedic ultrasound image segmentation method based on machine learning according to claim 3, characterized in that: In step S32, the entropy parameter image is generated to enhance the texture information of the ultrasound image, specifically comprising the following steps: Step S321: entropy value calculation, specifically, using a sliding window algorithm to obtain local signal information from the beamforming analog signal data, and calculating the entropy value by constructing an entropy value probability density function. The entropy value calculation formula is: ； Where H is the entropy value, x is the pixel value in the preprocessed ultrasound image data, and x max is the maximum pixel value, xmin is the minimum pixel value, and f(·) is the probability density function; Step S322: assigning a new pixel value, specifically assigning the central pixel value of the square window in the sliding window algorithm to the value of the entropy value H; Step S323: Window movement, specifically, moving the window in the sliding window algorithm within the data range, repeating the operations described in steps S321 and S322, and generating an ultrasound entropy parameter image; Step S324: generating an ultrasonic entropy parameter image, specifically generating the ultrasonic entropy parameter image by calculating the entropy value, assigning the new pixel value and moving the window, to obtain ultrasonic entropy parameter image data; In step S33, the edge information is enhanced by generating the mid-valley parameter image and the entropy parameter image to enhance the edge information of the ultrasound image and obtain ultrasound parameter image data.

5. The orthopedic ultrasound image segmentation method based on machine learning according to claim 4, characterized in that: In step S4, the feature encoding and decoding structure is combined with the network structure of the feature fusion module. It includes a feature encoder sub-block, a feature fusion sub-block and a feature decoder sub-block; The feature encoder sub-block is used to extract features of the multimodal image, and specifically includes an input layer and four encoding layers; The feature fusion sub-block is used to fuse features of the same dimension, and specifically includes a maximum fusion unit, a sum fusion unit, a convolution fusion unit, and a channel perception fusion unit; The feature decoder sub-block is used to restore the fused features into high-level semantic features, and specifically includes four decoding layers and one output layer.

6. The orthopedic ultrasound image segmentation method based on machine learning according to claim 5, characterized in that: In step S41, constructing the feature encoder sub-block includes the following steps: Step S411: constructing a basic structure of an image encoder, specifically using a pre-trained ResNet-34 network as an image encoder, deleting the average pooling layer and the fully connected layer of the ResNet-34 network, and constructing an input layer and four encoding layers. The calculation formula for constructing the basic structure of the image encoder is: ； Where Q(·) is the coding layer representation function, X is the layer index, Q1 is the first coding layer, Q2 is the second coding layer, Q3 is the third coding layer, and Q4 is the fourth coding layer; Step S412: Generate a multi-modal five-stage feature map, specifically a two-dimensional ultrasound modal feature map, a mid-valley ultrasound parameter modal feature map, and an entropy ultrasound parameter modal feature map, and the calculation formula is: ； In the formula, F i is a multimodal five-stage feature map, i is the feature map stage index, and the value range of i is {1, 2, 3, 4, 5}, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram; Step S413: feature coding, specifically, performing feature coding by constructing the basic structure of the image encoder and generating the multimodal five-stage feature map to obtain the ultrasonic image coding feature.

7. The orthopedic ultrasound image segmentation method based on machine learning according to claim 6, characterized in that: In step S42, constructing the feature fusion sub-block includes the following steps: Step S421: construct the maximum fusion unit, the calculation formula is: ； In the formula, is the fused feature map output by the maximum fusion unit, i is the feature map stage index, max(·) is the maximum value function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram; Step S422: construct a summing and fusion unit, and the calculation formula is: ； In the formula, is the fused feature map output by the summation fusion unit, i is the feature map stage index, is the 2D ultrasound weighted parameter, is the two-dimensional ultrasonic modal characteristic diagram, is the mid-valley weighting parameter, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy weighting parameter, is the entropy ultrasound parameter modal characteristic diagram; Step S423: construct a convolution fusion unit, and the calculation formula is: ； In the formula, is the concatenated feature map, i is the feature map stage index, is the fused feature map output by the convolutional fusion unit, Concat(·) is the concatenation operator, Conv 3×3 (·) is the convolution operator corresponding to a convolutional layer of size 3×3, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram; Step S424: construct a channel-aware fusion unit, specifically, by adopting the global average pooling operation and the global maximum pooling operation to obtain the global channel information of the cascade feature map, and using the shared multi-layer perception layer to generate channel parameters, and using the channel parameters and channel-aware fusion to generate a fused feature map, specifically including the following steps: Step S4241: construct channel parameters, the calculation formula is: ； In the formula, is the average pooling channel coefficient, is the maximum pooling channel coefficient, i is the feature map stage index, M r (·) is a shared multi-layer perceptual operation function, which is used to construct the output channel of the fully connected layer, where r is a balancing coefficient with a value of 4, which is used to balance performance and computational loss, P avg (·) is the global average pooling operation function, P max (·) is the global maximum pooling operation function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the middle valley, is the entropy ultrasound parameter modal characteristic diagram, [·:·] is the concatenation operator; Step S4242: Calculate the global channel attention, the calculation formula is: ； In the formula, A i is the global channel attention, is a S-shaped function, is the average pooling channel coefficient, is the maximum pooling channel coefficient, is the element-wise sum operator; Step S4243: Calculate the fusion feature map, the calculation formula is: ； Step S425: feature fusion, specifically, performing feature fusion by constructing the maximum fusion unit, the summation fusion unit, the convolution fusion unit, and the channel perception fusion unit to obtain a fusion feature map. .

8. The orthopedic ultrasound image segmentation method based on machine learning according to claim 7, characterized in that: In step S43, constructing the feature decoder sub-block includes the following steps: Step S431: construct a decoding layer, the decoding layer is used to include two convolutional layers of size 1×1 and one deconvolutional layer of size 3×3. The calculation formula for constructing the decoding layer is: ； Where D j is the decoding layer identifier, j is the decoding layer index, where j ranges from {1, 2, 3, 4}, Conv 1×1 (·) is the convolution operator corresponding to the convolutional layer of size 1×1, DC 3×3 (·) is the convolution operator corresponding to the deconvolution layer of size 3×3, D j-1 is the decoding layer identifier of layer j-1, is the element-wise sum operator, It is the fusion feature map corresponding to the 6-j stage; Step S432: construct an output layer, the output layer is used to calculate the final output value, including two convolutional layers of size 3×3 and one deconvolutional layer of size 4×4. The calculation formula for constructing the output layer is: ； In the formula, is the codec output value, used to segment the image, Conv 3×3 (·) is the convolution operator corresponding to the convolutional layer of size 3×3, DC 4×4 (·) is the convolution operator corresponding to the deconvolution layer of size 4×4, D 4 is the layer 4 decoding layer identifier, is the element-wise sum operator, It is the fusion feature map of the first stage; In step S44, the feature encoding and decoding model training is specifically performed by constructing the feature encoder sub-block, the constructing the feature fusion sub-block and the constructing the feature decoder sub-block to obtain the feature encoding and decoding model Model CA ; In step S45, the feature information processing is specifically performed using the feature encoding and decoding model Model CA , based on the ultrasonic parameter image data, feature encoding and decoding are performed to obtain ultrasonic image feature data.

9. An orthopedic ultrasonic image segmentation system based on machine learning, used to implement an orthopedic ultrasonic image segmentation method based on machine learning as claimed in any one of claims 1 to 8, characterized in that: It includes a data acquisition module, a basic preprocessing module, an edge information enhancement module, a feature encoding and decoding module and an ultrasound image segmentation module.

10. The orthopedic ultrasound image segmentation system based on machine learning according to claim 9, characterized in that: The data acquisition module is used for data acquisition, obtains raw data of orthopedic ultrasonic images through acquisition, and sends the raw data of orthopedic ultrasonic images to the basic preprocessing module; The basic preprocessing module is used for basic data preprocessing, obtains preprocessed ultrasound image data through basic data preprocessing, and sends the preprocessed ultrasound image data to the edge information enhancement module; The edge information enhancement module is used for edge information enhancement, obtains ultrasound parameter image data through edge information enhancement, and sends the ultrasound parameter image data to the feature encoding and decoding module; The feature encoding and decoding module is used for feature encoding and decoding, obtains ultrasonic image feature data through feature encoding and decoding, and sends the ultrasonic image feature data to the ultrasonic image segmentation module; The ultrasonic image segmentation module is used for ultrasonic image segmentation, and obtains orthopedic ultrasonic image segmentation data through ultrasonic image segmentation.

Images