A method and system for orthopedic ultrasound image segmentation based on machine learning

Abstract

The present invention discloses a method and system for orthopedic ultrasonic image segmentation based on machine learning, the method comprising data acquisition, basic preprocessing, edge information enhancement, feature encoding and decoding, and ultrasonic image segmentation. The present invention relates to the technical field of ultrasonic image segmentation, and specifically refers to a method and system for orthopedic ultrasonic image segmentation based on machine learning. The present invention adopts a method of generating mid-valley ultrasonic parameters and entropy ultrasonic parameter images to enhance edge information, thereby highlighting edge information; adopts a network structure combining a feature encoding and decoding structure with a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasonic parameter image data, thereby improving the performance of feature fusion; adopts a channel-aware fusion unit to perform feature fusion optimization, thereby improving the performance of feature fusion in the feature map and refining high-quality ultrasonic image features.

Description

Technical Field

[0001] The present invention relates to the technical field of ultrasonic image segmentation, and in particular to an orthopedic ultrasonic image segmentation method and system based on machine learning. Background Art

[0002] The orthopedic ultrasound image segmentation method based on machine learning is a process of using machine learning algorithms to classify pixels in orthopedic ultrasound images according to different objects or structures. Orthopedic ultrasound image segmentation aims to separate and locate different tissue structures such as bones, joints, and muscles in the image. This method helps to improve the understanding and application of orthopedic ultrasound images and further promote progress in the field of orthopedics.

[0003] However, in the existing orthopedic ultrasound image segmentation methods, there is a technical problem that the ultrasound imaging mode has multi-modal, multi-level and multi-type characteristics, and thus a method that can fuse multiple types of features is needed; in the existing orthopedic ultrasound image segmentation methods, there is a technical problem that the segmentation performance of single-modal images is low, while the feature encoding and decoding and feature fusion aggregation of multi-modal feature images are difficult; in the existing orthopedic ultrasound image segmentation methods, there is a technical problem that in the process of multi-modal feature map fusion, simple aggregation cannot extract effective features, and normalized fusion will lead to information redundancy. Summary of the Invention

[0004] In view of the above situation, in order to overcome the defects of the prior art, the present invention provides an orthopedic ultrasound image segmentation method and system based on machine learning. In view of the technical problem that in the existing orthopedic ultrasound image segmentation method, there is a multi-modal, multi-level and multi-type feature of ultrasound imaging modes, and thus a method that can fuse multiple types of features is needed, this solution creatively adopts the method of generating mid-valley ultrasound parameter and entropy ultrasound parameter image to enhance edge information, highlighting the edge information and improving the accuracy of subsequent segmentation tasks; in view of the technical problem that in the existing orthopedic ultrasound image segmentation method, there is a low segmentation performance of single-modal images, and the feature encoding of multi-modal feature images is To address the technical problem of high difficulty in aggregation of decoding and feature fusion, this solution creatively adopts a feature encoding and decoding structure combined with a network structure of a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data, thereby improving the performance of feature fusion; in view of the technical problem in the existing orthopedic ultrasound image segmentation method that in the process of multimodal feature map fusion, simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy, this solution creatively adopts a channel-aware fusion unit to perform feature fusion optimization, thereby improving the performance of feature fusion in the feature map, refining high-quality ultrasound image features, and optimizing the overall process of image segmentation.

[0005] The technical solution adopted by the present invention is as follows: The present invention provides an orthopedic ultrasound image segmentation method based on machine learning, which includes the following steps:

[0006] Step S1: data collection;

[0007] Step S2: basic preprocessing;

[0008] Step S3: edge information enhancement;

[0009] Step S4: feature encoding and decoding;

[0010] Step S5: Ultrasound image segmentation.

[0011] Furthermore, in step S1, the data acquisition is used to acquire the orthopedic ultrasound image to be segmented, specifically to acquire the original data of the orthopedic ultrasound image from the ultrasound imaging system.

[0012] Furthermore, in step S2, the basic preprocessing is used to perform data preprocessing on the original ultrasound image, specifically, performing filtering, denoising and normalization operations on the original data of the orthopedic ultrasound image to obtain preprocessed ultrasound image data.

[0013] Furthermore, in step S3, the edge information enhancement is used to enhance the edge information of the ultrasound image by generating an ultrasound parameter image, specifically by generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data, and performing edge information enhancement to obtain ultrasound image data, including the following steps:

[0014] Step S31: mid-valley parameter image generation, which is used to generate distribution statistics describing the ultrasound image and echo intensity, and obtain echo feature information of different image regions, specifically including the following steps:

[0015] Step S311: Construct the probability density function of the mid-valley parameter, and the calculation formula is:

[0016] ;

[0017] Wherein, f(·) is the probability density function of the mid-valley parameter, r is the distance variable used to represent the distance from the pixel point in the preprocessed ultrasound image data to the image center, m is the shape parameter, is the Euler gamma function, is the scaling parameter, exp(·) is the natural exponential function, and U(·) is the unit step function;

[0018] Step S312: Construct shape parameters, the calculation formula is:

[0019] ;

[0020] Where m is the shape parameter, E(·) is the expectation function, and R is the ultrasonic backscattering envelope, which is used to represent the intensity value of the echo signal at different distances;

[0021] Step S313: generating a mid-valley parameter image, specifically generating the mid-valley parameter image by constructing the probability density function and the shape parameters, and obtaining ultrasound mid-valley parameter image data;

[0022] Step S32: generating an entropy parameter image for enhancing the texture information of the ultrasound image, specifically comprising the following steps:

[0023] Step S321: Entropy calculation, specifically using a sliding window algorithm to obtain local signal information from the beamformed analog signal data, and calculating the entropy value by constructing an entropy probability density function. The entropy value calculation formula is:

[0024] ;

[0025] Where H is the entropy value, x is the pixel value in the preprocessed ultrasound image data, and x max is the maximum pixel value, xmin is the minimum pixel value, and f(·) is the probability density function;

[0026] Step S322: assigning a new pixel value, specifically assigning the center pixel value of the square window in the sliding window algorithm to the value of the entropy value H;

[0027] Step S323: Window movement, specifically, moving the window in the sliding window algorithm within the data range, repeating the operations described in steps S321 and S322 to generate an ultrasound entropy parameter image;

[0028] Step S324: generating an ultrasound entropy parameter image, specifically generating the ultrasound entropy parameter image by calculating the entropy value, assigning the new pixel value, and moving the window, to obtain ultrasound entropy parameter image data;

[0029] Step S33: edge information enhancement, specifically, performing edge information enhancement on the ultrasound image by generating the mid-valley parameter image and the entropy parameter image to obtain ultrasound parameter image data.

[0030] Furthermore, in step S4, the feature encoding and decoding is used to perform feature fusion and extract multimodal image features, specifically using a feature encoding and decoding structure combined with a network structure of a feature fusion module, and using a machine learning method to perform feature encoding and decoding based on the ultrasonic parameter image data to obtain ultrasonic image feature data;

[0031] The feature encoding and decoding structure is combined with the network structure of the feature fusion module, including a feature encoder sub-block, a feature fusion sub-block and a feature decoder sub-block;

[0032] The feature encoder sub-block is used to extract features of multimodal images, and specifically includes an input layer and four encoding layers;

[0033] The feature fusion sub-block is used to fuse features of the same dimension, specifically including a maximum fusion unit, a summation fusion unit, a convolution fusion unit and a channel-aware fusion unit;

[0034] The feature decoder sub-block is used to restore the fused features into high-level semantic features, and specifically includes four decoding layers and one output layer;

[0035] The step of using a feature encoding and decoding structure combined with a network structure of a feature fusion module and a machine learning method to perform feature encoding and decoding based on the ultrasonic parameter image data to obtain ultrasonic image feature data includes:

[0036] Step S41: constructing a feature encoder sub-block, including the following steps:

[0037] Step S411: Constructing the basic structure of the image encoder. Specifically, a pre-trained ResNet-34 network is used as the image encoder, the average pooling layer and the fully connected layer of the ResNet-34 network are deleted, and an input layer and four encoding layers are constructed. The calculation formula for constructing the basic structure of the image encoder is:

[0038] ;

[0039] Where Q(·) is the coding layer representation function, X is the layer index, Q1 is the first coding layer, Q2 is the second coding layer, Q3 is the third coding layer, and Q4 is the fourth coding layer;

[0040] Step S412: Generate a multimodal five-stage feature map, specifically a two-dimensional ultrasound modal feature map, a mid-valley ultrasound parameter modal feature map, and an entropy ultrasound parameter modal feature map. The calculation formula is:

[0041] ;

[0042] Where, F i is a multimodal five-stage feature map, i is the feature map stage index, and the value range of i is {1, 2, 3, 4, 5}, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram;

[0043] Step S413: feature coding, specifically, performing feature coding by constructing the basic structure of the image encoder and generating the multimodal five-stage feature map to obtain the ultrasonic image coding feature;

[0044] Step S42: constructing a feature fusion sub-block, including the following steps:

[0045] Step S421: Construct the maximum fusion unit, the calculation formula is:

[0046] ;

[0047] Where, is the fusion feature map output by the maximum fusion unit, i is the feature map stage index, max(·) is the maximum value function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram;

[0048] Step S422: Construct a summing and fusion unit, and the calculation formula is:

[0049] ;

[0050] Where, is the fused feature map output by the sum fusion unit, i is the feature map stage index, is the two-dimensional ultrasound weighting parameter, is the two-dimensional ultrasonic modal characteristic diagram, is the mid-valley weighting parameter, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy weighting parameter, is the entropy ultrasound parameter modal characteristic diagram;

[0051] Step S423: Construct a convolution fusion unit. The calculation formula is:

[0052] ;

[0053] Where, is the concatenated feature map, i is the feature map stage index, is the fusion feature map output by the convolution fusion unit, Concat(·) is the concatenation operator, Conv 3×3 (·) is the convolution operator corresponding to a convolutional layer of size 3×3, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram;

[0054] Step S424: constructing a channel-aware fusion unit, specifically by using global average pooling operations and global maximum pooling operations to obtain global channel information of the cascade feature map, and using a shared multi-layer perception layer to generate channel parameters, and using the channel parameters and channel-aware fusion to generate a fused feature map, specifically including the following steps:

[0055] Step S4241: Construct channel parameters, the calculation formula is:

[0056] ;

[0057] Where, is the average pooling channel coefficient, is the maximum pooling channel coefficient, i is the feature map stage index, M r (·) is a shared multi-layer perception operation function, which is used to construct the output channel of the fully connected layer. Among them, r is a balance coefficient with a value of 4, which is used to balance performance and computational loss. avg (·) is the global average pooling operation function, P max (·) is the global maximum pooling operation function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasonic parameter modal characteristic diagram, [·:·] is the concatenation operator;

[0058] Step S4242: Calculate the global channel attention. The calculation formula is:

[0059] ;

[0060] Where A i is the global channel attention, is a sigmoid function, is the average pooling channel coefficient, is the maximum pooling channel coefficient, is the element-wise sum operator;

[0061] Step S4243: Calculate the fusion feature map. The calculation formula is:

[0062] ;

[0063] Step S425: Feature fusion, specifically, performing feature fusion by constructing the maximum fusion unit, the summation fusion unit, the convolution fusion unit, and the channel perception fusion unit to obtain a fusion feature map. ;

[0064] Step S43: constructing a feature decoder sub-block, including the following steps:

[0065] Step S431: Construct a decoding layer, wherein the decoding layer is used to combine the two convolutional layers of size 1×1 and one deconvolutional layer of size 3×3. The calculation formula for constructing the decoding layer is:

[0066] ;

[0067] Where D j is the decoding layer identifier, j is the decoding layer index, where j ranges from {1, 2, 3, 4}, Conv 1×1 (·) is the convolution operator corresponding to the convolution layer of size 1×1, DC 3×3 (·) is the convolution operator corresponding to the deconvolution layer of size 3×3, D j-1 is the decoding layer identifier of layer j-1, is the element-wise sum operator, is the fusion feature map corresponding to the 6-j stage;

[0068] Step S432: construct an output layer. The output layer is used to calculate the final output value and includes two 3×3 convolutional layers and one 4×4 deconvolutional layer. The calculation formula for constructing the output layer is:

[0069] ;

[0070] Where, Is the codec output value, used to segment the image, Conv 3×3 (·) is the convolution operator corresponding to the convolutional layer of size 3×3, DC 4×4 (·) is the convolution operator corresponding to the deconvolution layer of size 4×4, D 4 is the layer 4 decoding layer identifier, is the element-wise sum operator, It is the fusion feature map of the first stage;

[0071] Step S44: Feature encoding and decoding model training, specifically, by constructing the feature encoder sub-block, the feature fusion sub-block and the feature decoder sub-block, performing feature encoding and decoding model training to obtain the feature encoding and decoding model Model CA ;

[0072] Step S45: feature information processing, specifically using the feature encoding and decoding model Model CA , based on the ultrasonic parameter image data, feature encoding and decoding are performed to obtain ultrasonic image feature data.

[0073] Furthermore, in step S5, the ultrasound image segmentation is used to perform ultrasound image segmentation based on ultrasound image information, specifically using the feature information processing method in the feature encoding and decoding and the machine learning neural network model to perform ultrasound image segmentation to obtain orthopedic ultrasound image segmentation data.

[0074] The present invention provides an orthopedic ultrasound image segmentation system based on machine learning, which includes a data acquisition module, a basic preprocessing module, an edge information enhancement module, a feature encoding and decoding module, and an ultrasound image segmentation module;

[0075] The data acquisition module is used for data acquisition, obtains the original data of the orthopedic ultrasound image through acquisition, and sends the original data of the orthopedic ultrasound image to the basic preprocessing module;

[0076] The basic preprocessing module is used for basic data preprocessing, obtains preprocessed ultrasound image data through basic data preprocessing, and sends the preprocessed ultrasound image data to the edge information enhancement module;

[0077] The edge information enhancement module is used for edge information enhancement, obtains ultrasound parameter image data through edge information enhancement, and sends the ultrasound parameter image data to the feature encoding and decoding module;

[0078] The feature encoding and decoding module is used for feature encoding and decoding, obtains ultrasonic image feature data through feature encoding and decoding, and sends the ultrasonic image feature data to the ultrasonic image segmentation module;

[0079] The ultrasonic image segmentation module is used for ultrasonic image segmentation, and obtains orthopedic ultrasonic image segmentation data through ultrasonic image segmentation.

[0080] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0081] (1) In view of the technical problem that the existing orthopedic ultrasound image segmentation methods have the characteristics of multi-modality, multi-level and multi-type of ultrasound imaging patterns, and thus require a method that can integrate multiple types of features, this solution creatively adopts the method of generating mid-valley ultrasound parameter and entropy ultrasound parameter images to enhance edge information, highlighting the edge information and improving the accuracy of subsequent segmentation tasks;

[0082] (2) In view of the technical problems in existing orthopedic ultrasound image segmentation methods, such as low segmentation performance of single-modal images and high difficulty in feature encoding and decoding and feature fusion aggregation of multi-modal feature images, this solution creatively adopts a network structure that combines a feature encoding and decoding structure with a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data, thereby improving the performance of feature fusion;

[0083] (3) In view of the technical problems in the existing orthopedic ultrasound image segmentation methods, simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy during the fusion of multimodal feature maps, this solution creatively uses a channel-aware fusion unit to optimize feature fusion, improves the performance of feature fusion in feature maps, extracts high-quality ultrasound image features, and optimizes the overall process of image segmentation. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Figure 1 A schematic diagram of a flow chart of an orthopedic ultrasound image segmentation method based on machine learning provided by the present invention;

[0085] Figure 2 A schematic diagram of an orthopedic ultrasound image segmentation system based on machine learning provided by the present invention;

[0086] Figure 3 This is a flow chart of feature information enhancement in step S3;

[0087] Figure 4 This is a schematic diagram of the feature encoding and decoding process in step S4;

[0088] Figure 5 A schematic diagram of the process of constructing a feature fusion sub-block in step S42.

[0089] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0090] The technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments; based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0091] In the description of the present invention, it should be understood that terms such as "upper", "lower", "front", "back", "left", "right", "top", "bottom", "inside" and "outside" indicating directions or positional relationships are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they should not be understood as limiting the present invention.

[0092] Example 1, see Figure 1 The present invention provides an orthopedic ultrasound image segmentation method based on machine learning, which includes the following steps:

[0093] Step S1: data collection;

[0094] Step S2: basic preprocessing;

[0095] Step S3: edge information enhancement;

[0096] Step S4: feature encoding and decoding;

[0097] Step S5: Ultrasound image segmentation.

[0098] Example 2, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. In step S1, the data acquisition is used to acquire the orthopedic ultrasound image to be segmented, specifically, to obtain the original data of the orthopedic ultrasound image from the ultrasound imaging system;

[0099] In step S2, the basic preprocessing is used to perform data preprocessing on the original ultrasound image, specifically, filtering, denoising and normalization operations are performed on the original data of the orthopedic ultrasound image to obtain preprocessed ultrasound image data.

[0100] Example 3, see Figure 1 、 Figure 2 and Figure 3 This embodiment is based on the above embodiment. In step S3, the edge information enhancement is used to enhance the edge information of the ultrasound image by generating an ultrasound parameter image. Specifically, the edge information is enhanced by generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data to obtain ultrasound image data. The method includes the following steps:

[0101] Step S31: mid-valley parameter image generation, which is used to generate distribution statistics describing the ultrasound image and echo intensity, and obtain echo feature information of different image regions, specifically including the following steps:

[0102] Step S311: Construct the probability density function of the mid-valley parameter, and the calculation formula is:

[0103] ;

[0104] Wherein, f(·) is the probability density function of the mid-valley parameter, r is the distance variable used to represent the distance from the pixel point in the preprocessed ultrasound image data to the image center, m is the shape parameter, is the Euler gamma function, is the scaling parameter, exp(·) is the natural exponential function, and U(·) is the unit step function;

[0105] Step S312: Construct shape parameters, the calculation formula is:

[0106] ;

[0107] Where m is the shape parameter, E(·) is the expectation function, and R is the ultrasonic backscattering envelope, which is used to represent the intensity value of the echo signal at different distances;

[0108] Step S313: generating a mid-valley parameter image, specifically generating the mid-valley parameter image by constructing the probability density function and the shape parameters, and obtaining ultrasound mid-valley parameter image data;

[0109] Step S32: generating an entropy parameter image for enhancing the texture information of the ultrasound image, specifically comprising the following steps:

[0110] Step S321: Entropy calculation, specifically using a sliding window algorithm to obtain local signal information from the beamformed analog signal data, and calculating the entropy value by constructing an entropy probability density function. The entropy value calculation formula is:

[0111] ;

[0112] Where H is the entropy value, x is the pixel value in the preprocessed ultrasound image data, and x max is the maximum pixel value, xmin is the minimum pixel value, and f(·) is the probability density function;

[0113] Step S322: assigning a new pixel value, specifically assigning the center pixel value of the square window in the sliding window algorithm to the value of the entropy value H;

[0114] Step S323: Window movement, specifically, moving the window in the sliding window algorithm within the data range, repeating the operations described in steps S321 and S322 to generate an ultrasound entropy parameter image;

[0115] Step S324: generating an ultrasound entropy parameter image, specifically generating the ultrasound entropy parameter image by calculating the entropy value, assigning the new pixel value, and moving the window, to obtain ultrasound entropy parameter image data;

[0116] Step S33: edge information enhancement, specifically, performing edge information enhancement on the ultrasound image by generating the mid-valley parameter image and the entropy parameter image to obtain ultrasound parameter image data.

[0117] By performing the above operations, in order to solve the technical problem that in the existing orthopedic ultrasound image segmentation methods, ultrasound imaging modes are multi-modal, multi-level and multi-type, and a method that can fuse multiple types of features is needed, this scheme creatively adopts the method of generating mid-valley ultrasound parameter and entropy ultrasound parameter images to enhance edge information, highlighting the edge information and improving the accuracy of subsequent segmentation tasks.

[0118] Example 4, see Figure 1 、 Figure 2 、 Figure 4 and Figure 5 This embodiment is based on the above embodiment. In step S4, the feature encoding and decoding is used to perform feature fusion and extract multimodal image features. Specifically, a feature encoding and decoding structure is combined with a network structure of a feature fusion module, and a machine learning method is used to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data.

[0119] The feature encoding and decoding structure is combined with the network structure of the feature fusion module, including a feature encoder sub-block, a feature fusion sub-block and a feature decoder sub-block;

[0120] The feature encoder sub-block is used to extract features of multimodal images, and specifically includes an input layer and four encoding layers;

[0121] The feature fusion sub-block is used to fuse features of the same dimension, specifically including a maximum fusion unit, a summation fusion unit, a convolution fusion unit and a channel-aware fusion unit;

[0122] The feature decoder sub-block is used to restore the fused features into high-level semantic features, and specifically includes four decoding layers and one output layer;

[0123] The step of using a feature encoding and decoding structure combined with a network structure of a feature fusion module and a machine learning method to perform feature encoding and decoding based on the ultrasonic parameter image data to obtain ultrasonic image feature data includes:

[0124] Step S41: constructing a feature encoder sub-block, including the following steps:

[0125] Step S411: Constructing the basic structure of the image encoder. Specifically, a pre-trained ResNet-34 network is used as the image encoder, the average pooling layer and the fully connected layer of the ResNet-34 network are deleted, and an input layer and four encoding layers are constructed. The calculation formula for constructing the basic structure of the image encoder is:

[0126] ;

[0127] Where Q(·) is the coding layer representation function, X is the layer index, Q1 is the first coding layer, Q2 is the second coding layer, Q3 is the third coding layer, and Q4 is the fourth coding layer;

[0128] Step S412: Generate a multimodal five-stage feature map, specifically a two-dimensional ultrasound modal feature map, a mid-valley ultrasound parameter modal feature map, and an entropy ultrasound parameter modal feature map. The calculation formula is:

[0129] ;

[0130] Where, F i is a multimodal five-stage feature map, i is the feature map stage index, and the value range of i is {1, 2, 3, 4, 5}, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram;

[0131] Step S413: feature coding, specifically, performing feature coding by constructing the basic structure of the image encoder and generating the multimodal five-stage feature map to obtain the ultrasonic image coding feature;

[0132] Step S42: constructing a feature fusion sub-block, including the following steps:

[0133] Step S421: Construct the maximum fusion unit, the calculation formula is:

[0134] ;

[0135] Where, is the fusion feature map output by the maximum fusion unit, i is the feature map stage index, max(·) is the maximum value function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram;

[0136] Step S422: Construct a summing and fusion unit, and the calculation formula is:

[0137] ;

[0138] Where, is the fused feature map output by the sum fusion unit, i is the feature map stage index, is the two-dimensional ultrasound weighting parameter, is the two-dimensional ultrasonic modal characteristic diagram, is the mid-valley weighting parameter, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy weighting parameter, is the entropy ultrasound parameter modal characteristic diagram;

[0139] Step S423: Construct a convolution fusion unit. The calculation formula is:

[0140] ;

[0141] Where, is the concatenated feature map, i is the feature map stage index, is the fusion feature map output by the convolution fusion unit, Concat(·) is the concatenation operator, Conv 3×3 (·) is the convolution operator corresponding to a convolutional layer of size 3×3, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram;

[0142] Step S424: constructing a channel-aware fusion unit, specifically by using global average pooling operations and global maximum pooling operations to obtain global channel information of the cascade feature map, and using a shared multi-layer perception layer to generate channel parameters, and using the channel parameters and channel-aware fusion to generate a fused feature map, specifically including the following steps:

[0143] Step S4241: Construct channel parameters, the calculation formula is:

[0144] ;

[0145] Where, is the average pooling channel coefficient, is the maximum pooling channel coefficient, i is the feature map stage index, M r (·) is a shared multi-layer perception operation function, which is used to construct the output channel of the fully connected layer. Among them, r is a balance coefficient with a value of 4, which is used to balance performance and computational loss. avg (·) is the global average pooling operation function, P max (·) is the global maximum pooling operation function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasonic parameter modal characteristic diagram, [·:·] is the concatenation operator;

[0146] Step S4242: Calculate the global channel attention. The calculation formula is:

[0147] ;

[0148] Where A iis the global channel attention, is a sigmoid function, is the average pooling channel coefficient, is the maximum pooling channel coefficient, is the element-wise sum operator;

[0149] Step S4243: Calculate the fusion feature map. The calculation formula is:

[0150] ;

[0151] Step S425: Feature fusion, specifically, performing feature fusion by constructing the maximum fusion unit, the summation fusion unit, the convolution fusion unit, and the channel perception fusion unit to obtain a fusion feature map. ;

[0152] Step S43: constructing a feature decoder sub-block, including the following steps:

[0153] Step S431: Construct a decoding layer, wherein the decoding layer is used to combine the two convolutional layers of size 1×1 and one deconvolutional layer of size 3×3. The calculation formula for constructing the decoding layer is:

[0154] ;

[0155] Where D j is the decoding layer identifier, j is the decoding layer index, where j ranges from {1, 2, 3, 4}, Conv 1×1 (·) is the convolution operator corresponding to the convolution layer of size 1×1, DC 3×3 (·) is the convolution operator corresponding to the deconvolution layer of size 3×3, D j-1 is the decoding layer identifier of layer j-1, is the element-wise sum operator, is the fusion feature map corresponding to the 6-j stage;

[0156] Step S432: construct an output layer. The output layer is used to calculate the final output value and includes two 3×3 convolutional layers and one 4×4 deconvolutional layer. The calculation formula for constructing the output layer is:

[0157] ;

[0158] Where, Is the codec output value, used to segment the image, Conv 3×3 (·) is the convolution operator corresponding to the convolutional layer of size 3×3, DC 4×4 (·) is the convolution operator corresponding to the deconvolution layer of size 4×4, D4 is the layer 4 decoding layer identifier, is the element-wise sum operator, It is the fusion feature map of the first stage;

[0159] Step S44: Feature encoding and decoding model training, specifically, by constructing the feature encoder sub-block, the feature fusion sub-block and the feature decoder sub-block, performing feature encoding and decoding model training to obtain the feature encoding and decoding model Model CA ;

[0160] Step S45: feature information processing, specifically using the feature encoding and decoding model Model CA , based on the ultrasonic parameter image data, feature encoding and decoding are performed to obtain ultrasonic image feature data.

[0161] By performing the above operations, in order to address the technical problems in existing orthopedic ultrasound image segmentation methods, such as low segmentation performance of single-modal images and high difficulty in aggregation of feature encoding and decoding and feature fusion of multi-modal feature images, this solution creatively adopts a feature encoding and decoding structure combined with a network structure of a feature fusion module, and adopts a machine learning method to perform feature encoding and decoding based on the ultrasound parameter image data, thereby improving the performance of feature fusion; in order to address the technical problems in existing orthopedic ultrasound image segmentation methods, such as simple aggregation cannot extract effective features and normalized fusion will lead to information redundancy during the fusion of multi-modal feature maps, this solution creatively adopts a channel-aware fusion unit to perform feature fusion optimization, thereby improving the performance of feature fusion in feature maps, extracting high-quality ultrasound image features, and optimizing the overall process of image segmentation.

[0162] Example 5, see Figure 1 and Figure 2 In step S5, the ultrasonic image segmentation is used to perform ultrasonic image segmentation based on the ultrasonic image information, specifically using the feature information processing method in the feature encoding and decoding and the machine learning neural network model to perform ultrasonic image segmentation to obtain orthopedic ultrasonic image segmentation data.

[0163] Example 6, see Figure 1 and Figure 2 This embodiment is based on the above embodiment. The present invention provides an orthopedic ultrasound image segmentation system based on machine learning, including a data acquisition module, a basic preprocessing module, an edge information enhancement module, a feature encoding and decoding module, and an ultrasound image segmentation module;

[0164] The data acquisition module is used for data acquisition, obtains the original data of the orthopedic ultrasound image through acquisition, and sends the original data of the orthopedic ultrasound image to the basic preprocessing module;

[0165] The basic preprocessing module is used for basic data preprocessing, obtains preprocessed ultrasound image data through basic data preprocessing, and sends the preprocessed ultrasound image data to the edge information enhancement module;

[0166] The edge information enhancement module is used for edge information enhancement, obtains ultrasound parameter image data through edge information enhancement, and sends the ultrasound parameter image data to the feature encoding and decoding module;

[0167] The feature encoding and decoding module is used for feature encoding and decoding, obtains ultrasonic image feature data through feature encoding and decoding, and sends the ultrasonic image feature data to the ultrasonic image segmentation module;

[0168] The ultrasonic image segmentation module is used for ultrasonic image segmentation, and obtains orthopedic ultrasonic image segmentation data through ultrasonic image segmentation.

[0169] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or apparatus.

[0170] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

[0171] The present invention and its embodiments are described above. This description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if a person skilled in the art is inspired by this and, without departing from the purpose of the present invention, designs structures and embodiments similar to this technical solution without inventiveness, they shall fall within the scope of protection of the present invention.

Claims

1. A method for orthopedic ultrasound image segmentation based on machine learning, comprising the following steps: Step S1: data acquisition, used to acquire the orthopedic ultrasound image to be segmented and obtain the original data of the orthopedic ultrasound image; Step S2: basic preprocessing, performing filtering, denoising and normalization operations on the raw data of the orthopedic ultrasound image to obtain preprocessed ultrasound image data; Step S3: edge information enhancement, by generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data, performing edge information enhancement to obtain ultrasound parameter image data; Step S4: feature encoding and decoding, which is used to perform feature fusion and extract multimodal image features. Specifically, a feature encoding and decoding structure is combined with a network structure of a feature fusion module, and a machine learning method is used to perform feature encoding and decoding based on the ultrasound parameter image data to obtain ultrasound image feature data. Specifically, it includes: Step S41: constructing a feature encoder sub-block; Step S42: constructing a feature fusion sub-block; Step S43: constructing a feature decoder sub-block; Step S44: training a feature encoding and decoding model; Step S45: processing feature information; Step S5: Ultrasonic image segmentation, for performing ultrasonic image segmentation based on ultrasonic image feature data to obtain orthopedic ultrasonic image segmentation data; Step S41 specifically includes: step S411: constructing a basic structure of an image encoder, specifically using a pre-trained ResNet-34 network as an image encoder, deleting the average pooling layer and the fully connected layer of the ResNet-34 network, and constructing an input layer and four encoding layers; step S412: generating a multimodal five-stage feature map, specifically a two-dimensional ultrasound modal feature map, a mid-valley ultrasound parameter modal feature map, and an entropy ultrasound parameter modal feature map; step S413: feature encoding, specifically performing feature encoding by constructing the basic structure of the image encoder and generating the multimodal five-stage feature map to obtain ultrasound image coding features; Step S42 specifically includes: the feature fusion sub-block is used to fuse features of the same dimension, specifically including a maximum fusion unit, a sum fusion unit, a convolution fusion unit and a channel-aware fusion unit; Step S43 specifically includes: step S431: constructing a decoding layer, wherein the decoding layer includes two convolutional layers of size 1×1 and one deconvolutional layer of size 3×3; step S432: constructing an output layer, wherein the output layer is used to calculate the final output value, and includes two convolutional layers of size 3×3 and one deconvolutional layer of size 4×4; In step S44, the feature encoding and decoding model training is specifically performed by constructing the feature encoder sub-block, the constructing the feature fusion sub-block and the constructing the feature decoder sub-block to obtain the feature encoding and decoding model Model CA ; In step S45, the feature information processing is specifically performed using the feature encoding and decoding model Model CA , based on the ultrasonic parameter image data, feature encoding and decoding are performed to obtain ultrasonic image feature data.

2. The orthopedic ultrasound image segmentation method based on machine learning according to claim 1, characterized in that: In step S3, the process of generating a mid-valley parameter image and an entropy parameter image based on the preprocessed ultrasound image data and performing edge information enhancement to obtain ultrasound parameter image data includes the following steps: Step S31: generating a mid-valley parameter image; Step S32: generating an entropy parameter image; Step S33: Edge information enhancement.

3. The orthopedic ultrasound image segmentation method based on machine learning according to claim 2, characterized in that: In step S31, the mid-valley parameter image is generated to generate distribution statistics describing the ultrasound image and echo intensity, and obtain echo feature information of different image regions, specifically including the following steps: Step S311: Construct the probability density function of the mid-valley parameter, and the calculation formula is: Wherein, f(·) is the probability density function of the mid-valley parameter, r is the distance variable used to represent the distance from the pixel point in the preprocessed ultrasound image data to the image center, m is the shape parameter, Γ(·) is the Euler gamma function, Ω is the scaling parameter, exp(·) is the natural exponential function, and U(·) is the unit step function; Step S312: Construct shape parameters, the calculation formula is: Where m is the shape parameter, E(·) is the expectation function, and R is the ultrasonic backscattering envelope, which is used to represent the intensity value of the echo signal at different distances; Step S313: generating a mid-valley parameter image, specifically generating the mid-valley parameter image by constructing a probability density function and the shape parameters, and obtaining ultrasound mid-valley parameter image data.

4. The orthopedic ultrasound image segmentation method based on machine learning according to claim 3, characterized in that: In step S32, the entropy parameter image is generated to enhance the texture information of the ultrasound image, specifically comprising the following steps: Step S321: Entropy calculation, specifically using a sliding window algorithm to obtain local signal information from the beamformed analog signal data, and calculating the entropy value by constructing an entropy probability density function. The entropy value calculation formula is: Where H is the entropy value, x is the pixel value in the preprocessed ultrasound image data, and x max is the maximum pixel value, xmin is the minimum pixel value, and f(·) is the probability density function; Step S322: assigning a new pixel value, specifically assigning the center pixel value of the square window in the sliding window algorithm to the value of the entropy value H; Step S323: Window movement, specifically, moving the window in the sliding window algorithm within the data range, repeating the operations described in steps S321 and S322 to generate an ultrasound entropy parameter image; Step S324: generating an ultrasound entropy parameter image, specifically generating the ultrasound entropy parameter image by calculating the entropy value, assigning the new pixel value, and moving the window, to obtain ultrasound entropy parameter image data; In step S33, the edge information is enhanced by generating the mid-valley parameter image and the entropy parameter image to enhance the edge information of the ultrasound image and obtain ultrasound parameter image data.

5. The orthopedic ultrasound image segmentation method based on machine learning according to claim 4, characterized in that: In step S411, the calculation formula for constructing the basic structure of the image encoder is: Q(X) = {Q1, Q2, Q3, Q4}; Where Q(·) is the coding layer representation function, X is the layer index, Q1 is the first coding layer, Q2 is the second coding layer, Q3 is the third coding layer, and Q4 is the fourth coding layer; In step S412, the calculation formula for generating the multimodal five-stage feature map is: Where, F i is a multimodal five-stage feature map, i is the feature map stage index, and the value range of i is {1, 2, 3, 4, 5}, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasonic parameter modal characteristic diagram.

6. The orthopedic ultrasound image segmentation method based on machine learning according to claim 5, characterized in that: In step S42, constructing the feature fusion sub-block includes the following steps: Step S421: Construct the maximum fusion unit, the calculation formula is: Where, is the fusion feature map output by the maximum fusion unit, i is the feature map stage index, max(·) is the maximum value function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram; Step S422: Construct a summing and fusion unit, and the calculation formula is: Where, is the fusion feature map output by the sum fusion unit, i is the feature map stage index, α is the two-dimensional ultrasound weighting parameter, is the two-dimensional ultrasonic modal characteristic diagram, β is the mid-valley weighting parameter, is the modal characteristic diagram of the mid-valley ultrasonic parameters, γ is the entropy weighting parameter, is the entropy ultrasound parameter modal characteristic diagram; Step S423: Construct a convolution fusion unit. The calculation formula is: Where, is the concatenated feature map, i is the feature map stage index, is the fused feature map output by the convolutional fusion unit, Concat(·) is the concatenation operator, Conv3×3(·) is the convolution operator corresponding to the convolutional layer of size 3×3, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram; Step S424: constructing a channel-aware fusion unit, specifically by using global average pooling operations and global maximum pooling operations to obtain global channel information of the cascade feature map, and using a shared multi-layer perception layer to generate channel parameters, and using the channel parameters and channel-aware fusion to generate a fused feature map, specifically including the following steps: Step S4241: Construct channel parameters, the calculation formula is: Where, is the average pooling channel coefficient, is the maximum pooling channel coefficient, i is the feature map stage index, M r (·) is a shared multi-layer perception operation function, which is used to construct the output channel of the fully connected layer. Among them, r is a balance coefficient with a value of 4, which is used to balance performance and computational loss. avg (·) is the global average pooling operation function, P max (·) is the global maximum pooling operation function, is the two-dimensional ultrasonic modal characteristic diagram, is the modal characteristic diagram of the ultrasonic parameters in the valley, is the entropy ultrasound parameter modal characteristic diagram, [·:·] is the concatenation operator; Step S4242: Calculate the global channel attention. The calculation formula is: Where A i is the global channel attention, σ(·) is the S-type function, is the average pooling channel coefficient, is the maximum pooling channel coefficient, is the element-wise sum operator; Step S4243: Calculate the fusion feature map. The calculation formula is: Step S425: Feature fusion, specifically, performing feature fusion by constructing the maximum fusion unit, the summation fusion unit, the convolution fusion unit, and the channel perception fusion unit to obtain a fusion feature map.

7. The orthopedic ultrasound image segmentation method based on machine learning according to claim 6, characterized in that: In step S431, the calculation formula for constructing the decoding layer is: Where D j is the decoding layer identifier, j is the decoding layer index, where j ranges from {1, 2, 3, 4}, Conv 1×1 (·) is the convolution operator corresponding to the convolution layer of size 1×1, DC 3×3 (·) is the convolution operator corresponding to the deconvolution layer of size 3×3, D j-1 is the decoding layer identifier of layer j-1, is the element-wise sum operator, is the fusion feature map corresponding to the 6-j stage; In step S432, the calculation formula for constructing the output layer is: Where, F output Is the codec output value, used to segment the image, Conv 3×3 (·) is the convolution operator corresponding to the convolutional layer of size 3×3, DC 4×4 (·) is the convolution operator corresponding to the deconvolution layer of size 4×4, D 4 is the layer 4 decoding layer identifier, is the element-wise sum operator, It is the fused feature map of the first stage.

8. A machine learning-based orthopedic ultrasound image segmentation system, configured to implement the machine learning-based orthopedic ultrasound image segmentation method according to any one of claims 1 to 7, characterized in that: It includes a data acquisition module, a basic preprocessing module, an edge information enhancement module, a feature encoding and decoding module and an ultrasound image segmentation module.

9. The orthopedic ultrasound image segmentation system based on machine learning according to claim 8, characterized in that: The data acquisition module is used for data acquisition, obtains the original data of the orthopedic ultrasound image through acquisition, and sends the original data of the orthopedic ultrasound image to the basic preprocessing module; The basic preprocessing module is used for basic data preprocessing, obtains preprocessed ultrasound image data through basic data preprocessing, and sends the preprocessed ultrasound image data to the edge information enhancement module; The edge information enhancement module is used for edge information enhancement, obtains ultrasound parameter image data through edge information enhancement, and sends the ultrasound parameter image data to the feature encoding and decoding module; The feature encoding and decoding module is used for feature encoding and decoding, obtains ultrasonic image feature data through feature encoding and decoding, and sends the ultrasonic image feature data to the ultrasonic image segmentation module; The ultrasonic image segmentation module is used for ultrasonic image segmentation, and obtains orthopedic ultrasonic image segmentation data through ultrasonic image segmentation.

Images