Ultrasonic image recognition model and ultrasonic image recognition method

Abstract

The present application is applicable to the field of ultrasonic image recognition technology and provides an ultrasonic image recognition model and ultrasonic image recognition method. The ultrasonic image recognition model includes a residual module, a pyramid-like decomposition module, a probability pooling module, a channel attention module, a first feature image adjustment module, a second feature image adjustment module, and an image recognition module. The residual module receives an ultrasonic image, and its output end is respectively connected to the pyramid-like decomposition module, the first feature image adjustment module, and the second feature image adjustment module. The pyramid-like decomposition module is connected to the probability pooling module, the probability pooling module is connected to the channel attention module, the channel attention module is connected to the first feature image adjustment module, the first feature image adjustment module is connected to the second feature image adjustment module, and the second feature image adjustment module is connected to the image recognition module. The image recognition module outputs the recognition result of the ultrasonic image. The present application can improve the accuracy of ultrasonic image recognition.

Description

Technical Field

[0001] The present application belongs to the technical field of ultrasonic image recognition, and in particular relates to an ultrasonic image recognition model and an ultrasonic image recognition method. Background Art

[0002] Ultrasonic testing analyzes an object's internal condition by receiving and imaging the echoes of sound waves of a certain frequency at interfaces with differing dielectric densities. Ultrasonic nondestructive testing, due to its advantages such as dynamic detection, high sensitivity, high controllability, and economic efficiency, has long held an irreplaceable position in numerous fields, including industry, medicine, and civil engineering. However, ultrasonic images are formed by sampling, converting, and assigning values to echoes of a fixed center frequency. This results in a small number of channels and high levels of noise, resulting in a low amount of effective information. Furthermore, since ultrasonic image interpretation requires specialized expertise in various fields, relying not only on individual experience but also on accuracy and efficiency that struggle to meet the demands of social development.

[0003] In recent years, artificial intelligence technologies, particularly neural networks, have continuously achieved breakthroughs and are entering a period of rapid development, with their application in various industries steadily deepening. Therefore, the introduction of artificial intelligence methods into ultrasound image recognition and classification tasks is of great significance, as it will aid professional decision-making and alleviate numerous related difficulties. However, due to the significant differences between the ultrasound imaging process and conventional optical imaging, directly using convolutional neural networks struggles to extract sufficient features from ultrasound images, resulting in suboptimal results. Consequently, the accuracy of existing ultrasound image recognition methods needs to be improved. Summary of the Invention

[0004] The present application provides an ultrasound image recognition model and an ultrasound image recognition method, which can solve the problem of low recognition accuracy of current ultrasound image recognition methods.

[0005] In a first aspect, the present application provides an ultrasound image recognition model, comprising a residual module, a pyramid-like decomposition module, a probability pooling module, a channel attention module, a first feature image adjustment module, a second feature image adjustment module, and an image recognition module;

[0006] The residual module is used to perform convolution processing on the input ultrasound image to obtain an initial multi-channel feature image corresponding to the input ultrasound image;

[0007] A pyramid-like decomposition module is used to perform pyramid decomposition on the feature image corresponding to each channel in the initial multi-channel feature image to obtain multiple feature sub-images corresponding to each channel; the scales of the multiple feature sub-images are different from each other;

[0008] The probability pooling module is used to perform probability pooling on each of the multiple feature sub-images to obtain the initial weight coefficient of each feature sub-image. Probabilistic pooling means that the probabilities of the pooling methods corresponding to different feature sub-images are different. The pooling methods include maximum pooling, average pooling, minimum pooling, and random pooling, and the probabilities follow the standard normal distribution.

[0009] The channel attention module is used to calculate the channel weight of each channel according to the initial weight coefficient and activate the channel weights of all channels to obtain the channel weight matrix of the initial multi-channel feature image;

[0010] A first feature image adjustment module is used to multiply the channel weight matrix and the initial multi-channel feature image to obtain an intermediate multi-channel feature image;

[0011] A second feature image adjustment module is used to add the intermediate multi-channel feature image and the initial multi-channel feature image to obtain a final multi-channel feature image;

[0012] The image recognition module is used to recognize the final multi-channel feature image and obtain the recognition result of the final multi-channel feature image.

[0013] Optionally, the input end of the residual module receives the ultrasound image, the output end of the residual module is respectively connected to the input end of the pyramid decomposition module, the first input end of the first feature image adjustment module and the first input end of the second feature image adjustment module, the output end of the pyramid decomposition module is connected to the input end of the probability pooling module, the output end of the probability pooling module is connected to the input end of the channel attention module, the output end of the channel attention module is connected to the second input end of the first feature image adjustment module, the output end of the first feature image adjustment module is connected to the second input end of the second feature image adjustment module, the output end of the second feature image adjustment module is connected to the input end of the image recognition module, and the output end of the image recognition module outputs the recognition result of the ultrasound image.

[0014] Optionally, pyramid decomposition is performed on the feature image corresponding to each channel in the initial multi-channel feature image to obtain multiple feature sub-images corresponding to each channel, including:

[0015] For each channel’s corresponding feature image, perform the following steps:

[0016] Step i, taking the feature image as the initial image, copying the initial image to obtain an intermediate image;

[0017] Step ii, intercepting the outermost circle pixels of the intermediate image with a preset step size to obtain the final image;

[0018] In step iii, if the scale of the final image is greater than or equal to the preset scale threshold, the final image is used as the initial image in step i and the process returns to step i. Otherwise, the initial image and all the final images obtained by executing step iii are used as multiple feature sub-images corresponding to each channel.

[0019] Optionally, the scale threshold is calculated as

[0020]

[0021]

[0022] E=(min(M,N)-2)mod2*T

[0023] Among them, M x ,N x Both represent scale thresholds, E represents the judgment condition for scale threshold calculation, T represents the step size, M and N both represent the scale of the initial image, mod represents the remainder, and |·| represents the absolute value.

[0024] Optionally, the channel weight of each channel is calculated based on the initial weight coefficient, and the channel weights of all channels are subjected to one-dimensional convolution processing to obtain the channel weight matrix of the initial multi-channel feature image, including:

[0025] For each channel, add up all the initial weight coefficients corresponding to the channel to obtain the channel weight;

[0026] Use one-dimensional convolution to convolve the channel weights of all channels to obtain the intermediate channel weight matrix; if the total number of channels is less than or equal to 128, the convolution kernel of the one-dimensional convolution is set to 5, otherwise the convolution kernel of the one-dimensional convolution is set to 7;

[0027] The activation function is used to activate the intermediate channel weight matrix to obtain the channel weight matrix.

[0028] In a second aspect, the present application provides an ultrasound image recognition method, comprising:

[0029] Acquire an ultrasound image training set; the ultrasound image training set includes multiple ultrasound images of known types;

[0030] Inputting multiple ultrasound images of known types into an ultrasound image recognition model one by one to obtain a recognition result for each ultrasound image; the ultrasound image recognition model is the ultrasound image recognition model described above;

[0031] Calculate the loss of the ultrasound image recognition model according to the recognition result and the type of each ultrasound image, and perform backpropagation on the ultrasound image recognition model according to the loss until the ultrasound image recognition model is fitted, thereby obtaining a trained ultrasound image recognition model;

[0032] The ultrasonic image to be recognized is input into the trained ultrasonic image recognition model to obtain a recognition result of the ultrasonic image to be recognized.

[0033] Optionally, multiple ultrasound images of known types are input into the ultrasound image recognition model one by one to obtain a recognition result for each ultrasound image, including:

[0034] By calculating the formula

[0035]

[0036] Get the output probability P of each type k Among them, P k represents the output probability of type k, k = 1, 2, ..., Q, Q represents the total number of types of ultrasound images in the ultrasound image training set, z k represents the output value of the kth node. The number of nodes corresponds to the number of types. j represents the jth node, n represents the number of output nodes, and softmax(·) represents the activation function.

[0037] The type with the highest output probability is taken as the recognition result.

[0038] Optionally, based on the recognition result and the type of each ultrasound image, a loss of the ultrasound image recognition model is calculated, and backpropagation is performed on the ultrasound image recognition model according to the loss until the ultrasound image recognition model is fitted, thereby obtaining a trained ultrasound image recognition model, including:

[0039] For each ultrasound image, the formula

[0040] FL=-a k (1-p k ) γ log(p k )

[0041] The loss FL of the ultrasound image recognition model is obtained; where a k represents the weight of type k, p k represents the output probability of type k, γ represents the focusing parameter, Y∈[0,5];

[0042] The ultrasound image recognition model is back-propagated according to the loss EL until the loss value of the new ultrasound image recognition model obtained by back-propagation is less than the preset loss threshold. The new ultrasound image recognition model is then fitted to obtain the trained ultrasound image recognition model. The momentum of the back-propagation is set to 0.9, the learning rate of the back-propagation is obtained using the thermal learning rate strategy, and the weight attenuation coefficient of the L1 regularization of the back-propagation is set to 5e-4.

[0043] The above solution of the present application has the following beneficial effects:

[0044] The ultrasound image recognition model provided in this application uses a pyramid-like decomposition module to perform pyramid decomposition on the feature image corresponding to each channel, which can make the subsequently extracted features more accurate, thereby improving the accuracy of ultrasound image recognition. The probability pooling module is used to probabilistically pool each feature sub-image, which can prevent overfitting when the feature image is pooled. Through multiple pooling methods, important local features can be selectively extracted, and the feature image can be recognized based on these important local features, which can significantly improve the accuracy of ultrasound image recognition.

[0045] Other beneficial effects of the present application will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0047] Figure 1 A schematic diagram of the structure of an ultrasound image recognition model provided in one embodiment of the present application;

[0048] Figure 2 A schematic diagram of the structure of a residual module provided in one embodiment of the present application;

[0049] Figure 3 A schematic diagram of the structure of a characteristic image gold character provided in one embodiment of the present application;

[0050] Figure 4 A flowchart of ultrasound image recognition provided in one embodiment of the present application. DETAILED DESCRIPTION

[0051] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0052] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0053] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0054] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0055] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0056] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0057] In response to the problem of low recognition accuracy of current ultrasound image recognition models, the present application provides an ultrasound image recognition model and an ultrasound image recognition method. The ultrasound image recognition model uses a pyramid-like decomposition module to perform pyramid decomposition on the feature image corresponding to each channel, which can make the subsequently extracted features more accurate, thereby improving the accuracy of ultrasound image recognition. The probability pooling module is used to probabilistically pool each feature sub-image, which can prevent overfitting during feature image pooling. Through multiple pooling methods, important local features can be selectively extracted, and feature images can be recognized based on these important local features, which can significantly improve the accuracy of ultrasound image recognition.

[0058] The ultrasound image recognition model provided in this application is exemplified below.

[0059] like Figure 1 As shown, the ultrasound image recognition model provided by this application includes a residual module (such as Figure 1 11), pyramid-like decomposition module (as shown in Figure 1 12), probability pooling module (as shown in Figure 1 13), channel attention module (as shown in Figure 1 14), the first feature image adjustment module (as shown in Figure 1 15), the second feature image adjustment module (as shown in Figure 1 16) and the image recognition module (as shown in Figure 1 17).

[0060] For the sake of convenience, in the embodiments of the present application, only the functions of the modules in the ultrasound image recognition model are described. The residual module, pyramid-like decomposition module, probability pooling module, channel attention module, first feature image adjustment module, second feature image adjustment module and image recognition module are regarded as a group of functional units. The ultrasound image recognition model also includes multiple groups of the above functional units.

[0061] The connection relationship between the modules is as follows:

[0062] The input end of the residual module receives the ultrasound image, the output end of the residual module is respectively connected to the input end of the pyramid decomposition module, the first input end of the first feature image adjustment module and the first input end of the second feature image adjustment module, the output end of the pyramid decomposition module is connected to the input end of the probability pooling module, the output end of the probability pooling module is connected to the input end of the channel attention module, the output end of the channel attention module is connected to the second input end of the first feature image adjustment module, the output end of the first feature image adjustment module is connected to the second input end of the second feature image adjustment module, the output end of the second feature image adjustment module is connected to the input end of the image recognition module, and the output end of the image recognition module outputs the recognition result of the ultrasound image.

[0063] Each module is described below with examples.

[0064] The residual module is used to perform convolution processing on the input ultrasound image to obtain an initial multi-channel feature image corresponding to the input ultrasound image.

[0065] For example, Figure 2 As shown, in one embodiment of the present application, the residual module includes a first convolutional layer (such as Figure 2 21), the first normalization layer (as shown in Figure 2 22), the first activation layer (as shown in Figure 2 23), the second convolutional layer (as shown in Figure 2 24), the second normalization layer (as shown in Figure 2 25), the second activation layer (as shown in Figure 2 26), the third convolutional layer (as shown in Figure 2 27), the third standardization layer (as shown in Figure 2 28), the above layers are connected in sequence. The first convolutional layer is a 1*1 convolutional layer, the second convolutional layer is a 3*3 convolutional layer, and the third convolutional layer is a 1*1 convolutional layer. The stride of the second convolutional layer is set to 1. This can reduce the loss of pixel correlation information caused by stride length.

[0066] It should be noted that, in order to reduce redundant parameters, in the embodiments of the present application, the bias terms of each convolutional layer are removed.

[0067] The pyramid-like decomposition module is used to perform pyramid decomposition on the feature image corresponding to each channel in the initial multi-channel feature image to obtain multiple feature sub-images corresponding to each channel.

[0068] The scales of the multiple feature sub-images are different from each other.

[0069] The pyramid-like decomposition module is used to perform pyramid decomposition on the feature image corresponding to each channel in the initial multi-channel feature image to obtain multiple feature sub-images corresponding to each channel. The specific process is as follows:

[0070] In step i, the feature image is used as the initial image, and the initial image is copied to obtain an intermediate image.

[0071] Step ii: intercept the outermost circle pixels of the intermediate image with a preset step size to obtain the final image.

[0072] In step iii, if the scale of the final image is greater than or equal to the preset scale threshold, the final image is used as the initial image in step i and the process returns to step i. Otherwise, the initial image and all the final images obtained by executing step iii are used as multiple feature sub-images corresponding to each channel. The calculation formula for the scale threshold is:

[0073]

[0074]

[0075] E=(min(M,N)-2)mod2*T

[0076] Among them, M x ,N xBoth represent scale thresholds, E represents the judgment condition for scale threshold calculation, T represents the step size, M and N both represent the scale of the initial image, mod represents the remainder, and |·| represents the absolute value.

[0077] In addition, the formula

[0078]

[0079] The total number of feature sub-images Q is obtained. It should be understood that all feature sub-images are arranged in order from small to large scales, and a feature sub-image pyramid can be obtained, specifically as follows: Figure 3 shown.

[0080] When performing pyramid decomposition on the feature image, the formula

[0081] P C =C*P Q

[0082]

[0083] Get the number of pixels P after pyramid decomposition C , where C represents the number of channels of the feature image, P Q Represents the number of pixels contained in a single-channel feature image, and k represents the kth feature sub-image.

[0084] The probability pooling module is used to perform probability pooling on each feature sub-image in the multiple feature sub-images to obtain an initial weight coefficient of each feature sub-image.

[0085] The above-mentioned probability pooling indicates that the probabilities of the pooling methods corresponding to different feature sub-images are different from each other. The pooling methods include maximum pooling, average pooling, minimum pooling and random pooling, and the probabilities obey the standard normal distribution.

[0086] Because the same pooling method produces similar results on centrally symmetric feature images, using a unified pooling method can easily lead to over-amplification of local features, which not only covers other features but also makes it difficult to extract multi-dimensional information. In view of this, this application uses different pooling methods for feature sub-images of different scales after pyramid decomposition.

[0087] Specifically, the probabilities of different pooling methods are set to follow a standard normal distribution. For example, the probability of the pooling method distributed in the interval (μ-σ, μ+σ) (global maximum pooling) is 68%, the probability of the pooling method distributed in the interval (-∞, μ-σ) (global average pooling) is 16%, and the probability of the pooling method distributed in the interval (μ+σ, +∞) (global minimum pooling) is 12%. When located on μ-σ or μ+σ, global random pooling is used with a probability of 4%. In practical applications, global maximum pooling and global average pooling are preferred because global maximum pooling can retain the representative information of the feature image, while global average pooling can blur the information of the feature image.

[0088] The channel attention module is used to calculate the channel weight of each channel based on the initial weight coefficient, and perform one-dimensional convolution on the channel weights of all channels to obtain the channel weight matrix of the initial multi-channel feature image.

[0089] In an embodiment of the present application, the channel attention module calculates the channel weight of each channel according to the initial weight coefficient, and performs one-dimensional convolution processing on the channel weights of all channels to obtain the channel weight matrix of the initial multi-channel feature image. The specific process is as follows:

[0090] For each channel, all the initial weight coefficients corresponding to the channel are added together to obtain the channel weight.

[0091] For example, by calculating the formula W i =w i,1 +w i,2 +...+w i,j +...+w i,Q , get the channel weight W i Among them, W i represents the channel weight of the i-th channel, w i,j Represents the jth initial weight coefficient of the i-th channel.

[0092] The channel weights of all channels are convolved using one-dimensional convolution to obtain the intermediate channel weight matrix. If the total number of channels is less than or equal to 128, the convolution kernel of the one-dimensional convolution is set to 5; otherwise, the convolution kernel of the one-dimensional convolution is set to 7.

[0093] It is worth mentioning that setting the convolution kernel of the one-dimensional convolution according to the total number of channels is helpful for dealing with complex changes in ultrasound images. At the same time, the one-dimensional convolution performs convolution processing on the channel weights of all channels, which can extract the dependencies between adjacent channels and make the channel weight matrix more accurate.

[0094] The activation function is used to activate the intermediate channel weight matrix to obtain the channel weight matrix.

[0095] For example, the activation function is expressed as S(x)=1 / (1+e -x ).

[0096] It should be noted that each row in the channel weight matrix corresponds to a channel weight.

[0097] The first feature image adjustment module is used to multiply the channel weight matrix and the initial multi-channel feature image to obtain an intermediate multi-channel feature image

[0098] This part essentially performs weighted processing on the feature image of each channel. Each element in the channel weight matrix represents the weight value of the corresponding channel. By multiplying it with the initial multi-channel feature image, the importance of each channel can be adjusted, highlighting or suppressing the feature information of different channels. This allows the extraction of more discriminative and important features.

[0099] The second feature image adjustment module is used to add the intermediate multi-channel feature image and the initial multi-channel feature image to obtain a final multi-channel feature image.

[0100] This step can enhance the expressiveness and robustness of the feature image. Through the addition operation, the feature information in the intermediate multi-channel feature image can be fused with the original information in the initial multi-channel feature image, thereby obtaining a richer and more accurate feature representation.

[0101] The image recognition module is used to recognize the final multi-channel feature image and obtain the recognition result of the final multi-channel feature image.

[0102] In an embodiment of the present application, the image recognition module is a fully connected layer. When performing feature image recognition, each neuron in the fully connected layer will output a probability, and the ultrasound image type corresponding to the neuron with the highest probability will be used as the recognition result; wherein, the neurons and ultrasound image types correspond one to one, that is, each neuron corresponds to one ultrasound image type.

[0103] It can be seen that the ultrasound image recognition model provided in this application uses a pyramid-like decomposition module to perform pyramid decomposition on the feature image corresponding to each channel, which can make the subsequently extracted features more accurate, thereby improving the accuracy of ultrasound image recognition. The probability pooling module is used to perform probability pooling on each feature sub-image, which can prevent overfitting during feature image pooling. Through multiple pooling methods, important local features can be selectively extracted, and feature images can be recognized based on these important local features, which can significantly improve the accuracy of ultrasound image recognition.

[0104] The ultrasound image recognition method provided in this application is exemplified below.

[0105] like Figure 4As shown, the ultrasound image recognition method provided by this application includes the following steps:

[0106] Step 41: Obtain an ultrasound image training set.

[0107] The ultrasound image training set includes multiple ultrasound images of known types.

[0108] The following is an exemplary description of the process of obtaining an ultrasound image training set.

[0109] In step 41.1, 1,500 ultrasound images are obtained from the ultrasound image open dataset to form an ultrasound image training set.

[0110] In step 41.2, experts in related fields identify the ultrasound image training set and determine the type of each ultrasound image.

[0111] For example, the ultrasound images in the ultrasound image training set are identified as ultrasound images corresponding to the liver, the spleen, the kidney, the intestine, the gallbladder, the bladder, the calf muscle, and the breast. Accordingly, the types of ultrasound images include the liver, spleen, kidney, intestine, gallbladder, bladder, calf muscle, and breast.

[0112] Step 41.3: perform data amplification on the classified ultrasound images.

[0113] In the embodiments of the present application, the data amplification methods that can be used include:

[0114] Mode 1, rotation at any angle within 0° to 30°;

[0115] Method 2: randomly use one of up-down or left-right flipping;

[0116] Method 3: Enlarge the image to 1.5 times the original aspect ratio and crop it to the original size;

[0117] Method 4: crop a small portion of the original image and then enlarge it to its original size;

[0118] Method 5: Randomly select a pixel area of 20*20 to 50*50 in the image and fill all the pixels in the area with the pixel mean value of the area;

[0119] Method 6: randomly using one of the Sobel operator, Roberts operator, Prewitt operator (a differential operator for image edge detection), and Laplacian operator to sharpen the ultrasound image;

[0120] Method 7: randomly adding one of Gaussian noise, salt and pepper noise, or gamma noise to the ultrasound image.

[0121] Step 41.4: scaling the ultrasound image after data amplification.

[0122] Specifically, the size of each category of ultrasound images can be scaled to 64*64, and then all ultrasound images of the same size are standardized so that the mean of any pixel value in each ultrasound image is 0 and the standard deviation is 1. The calculation formula of this process is P stand =(P S -μ) / σ, where P s Represents the original pixel value, P stand Represents the new pixel value after standardization, μ represents the mean value of the pixel, and σ represents the standard deviation of the pixel.

[0123] In step 42, multiple ultrasound images of known types are input into the ultrasound image recognition model one by one to obtain a recognition result for each ultrasound image.

[0124] The ultrasound image recognition model is the ultrasound image recognition model mentioned above.

[0125] The process of step 42 (inputting multiple ultrasound images of known types into the ultrasound image recognition model one by one to obtain a recognition result for each ultrasound image) is exemplarily described below.

[0126] Specifically, through the calculation formula

[0127]

[0128] Get the output probability P of each type k .

[0129] Among them, P k represents the output probability of type k, k = 1, 2, ..., Q, Q represents the total number of types of ultrasound images in the ultrasound image training set, z k represents the output value of the kth node. The number of nodes corresponds to the number of types. j represents the jth node, n represents the number of output nodes, and softmax(·) represents the activation function.

[0130] For example, in one embodiment of the present application, the output probability of the ultrasound image recognition model is

[0131]

[0132] The type with the highest output probability is taken as the recognition result. That is, the input ultrasound image is recognized as an ultrasound image corresponding to the calf muscle.

[0133] Step 43 , calculating the loss of the ultrasound image recognition model according to the recognition result and the type of each ultrasound image, and performing backpropagation on the ultrasound image recognition model according to the loss until the ultrasound image recognition model is fitted, thereby obtaining a trained ultrasound image recognition model.

[0134] Specifically, for each ultrasound image, the formula

[0135] FL=-a k (1-p k ) γ log(p k )

[0136] The loss FL of the ultrasound image recognition model is obtained; where a k represents the weight of type k, p k represents the output probability of type k, γ represents the focusing parameter, γ∈[0,5];

[0137] The ultrasound image recognition model is back-propagated according to the loss FL until the loss value of the new ultrasound image recognition model obtained by back-propagation is less than the preset loss threshold. The new ultrasound image recognition model is then fitted to obtain the trained ultrasound image recognition model. The momentum of the back-propagation is set to 0.9, the learning rate of the back-propagation is obtained using the thermal learning rate strategy, and the weight attenuation coefficient of the L1 regularization of the back-propagation is set to 5e-4.

[0138] In step 44 , the ultrasound image to be recognized is input into the trained ultrasound image recognition model to obtain a recognition result of the ultrasound image to be recognized.

[0139] The ultrasound image to be identified is an ultrasound image of uncertain type.

[0140] In an embodiment of the present application, after the ultrasound image recognition model is trained, in order to further improve the accuracy of the ultrasound image recognition model, the effect of the ultrasound image recognition model can be measured by calculating the average accuracy, precision, sensitivity and F-1 score (an indicator used in statistics to measure the accuracy of a binary classification model) corresponding to the recognition results, and the ultrasound image recognition model with the best effect (highest average accuracy, highest precision, highest sensitivity and highest F-1 score) is taken as the final ultrasound image recognition model.

[0141] The calculation formula for the average accuracy is as follows:

[0142]

[0143] The accuracy is calculated as follows:

[0144]

[0145] The sensitivity is calculated as follows:

[0146]

[0147] The formula for calculating the F-1 score is as follows:

[0148]

[0149] Among them, represents the number of samples correctly predicted as positive, represents the number of samples incorrectly predicted as positive, represents the number of samples incorrectly predicted as negative, and represents the number of samples correctly predicted as negative.

[0150] The comparison of the ultrasonic image recognition model and ultrasonic image recognition method provided by this application with the recognition effects of other existing technologies is shown in the following table:

[0151]

[0152]

[0153] It can be seen from the results in the above table that the ultrasound image recognition model and ultrasound image recognition method provided by this application are more advanced than other existing technologies, and the recognition effect of ultrasound images is more accurate.

[0154] The above is a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles described in the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An ultrasound image recognition model, characterized in that: The ultrasound image recognition model includes a residual module, a pyramid-like decomposition module, a probability pooling module, a channel attention module, a first feature image adjustment module, a second feature image adjustment module and an image recognition module; The residual module is used to perform convolution processing on the input ultrasound image to obtain an initial multi-channel feature image corresponding to the input ultrasound image; The pyramid-like decomposition module is used to perform pyramid decomposition on the feature image corresponding to each channel in the initial multi-channel feature image to obtain multiple feature sub-images corresponding to each channel; the scales of the multiple feature sub-images are different from each other; The probability pooling module is used to perform probability pooling on each of the multiple feature sub-images to obtain an initial weight coefficient for each feature sub-image; the probability pooling indicates that the probabilities of pooling methods corresponding to different feature sub-images are different, and the pooling methods include maximum pooling, average pooling, minimum pooling and random pooling, and the probabilities obey a standard normal distribution; The channel attention module is used to calculate the channel weight of each channel according to the initial weight coefficient, and perform one-dimensional convolution processing on the channel weights of all channels to obtain the channel weight matrix of the initial multi-channel feature image; The first feature image adjustment module is configured to multiply the channel weight matrix and the initial multi-channel feature image to obtain an intermediate multi-channel feature image; The second feature image adjustment module is configured to add the intermediate multi-channel feature image and the initial multi-channel feature image to obtain a final multi-channel feature image; The image recognition module is used to recognize the final multi-channel feature image to obtain a recognition result of the final multi-channel feature image.

2. The ultrasound image recognition model according to claim 1, characterized in that: The input end of the residual module receives the ultrasound image, the output end of the residual module is respectively connected to the input end of the pyramid-like decomposition module, the first input end of the first feature image adjustment module, and the first input end of the second feature image adjustment module, the output end of the pyramid-like decomposition module is connected to the input end of the probability pooling module, the output end of the probability pooling module is connected to the input end of the channel attention module, the output end of the channel attention module is connected to the second input end of the first feature image adjustment module, the output end of the first feature image adjustment module is connected to the second input end of the second feature image adjustment module, the output end of the second feature image adjustment module is connected to the input end of the image recognition module, and the output end of the image recognition module outputs the recognition result of the ultrasound image.

3. The ultrasound image recognition model according to claim 1, characterized in that: The step of performing pyramid decomposition on the feature image corresponding to each channel in the initial multi-channel feature image to obtain a plurality of feature sub-images corresponding to each channel includes: For each channel’s corresponding feature image, perform the following steps: Step i, taking the characteristic image as the initial image, and copying the initial image to obtain an intermediate image; Step ii, intercepting the outermost circle pixels of the intermediate image with a preset step size to obtain a final image; In step iii, if the scale of the final image is greater than or equal to the preset scale threshold, the final image is used as the initial image in step i, and the process returns to step i; otherwise, the initial image and all the final images obtained by executing step iii are used as the multiple feature sub-images corresponding to each channel.

4. The ultrasound image recognition model according to claim 3, characterized in that: The calculation formula of the scale threshold is: E=(min(M,N)-2)mod 2*T Among them, M x ,N x Both represent scale thresholds, E represents the judgment condition for scale threshold calculation, T represents the step size, M and N both represent the scale of the initial image, mod represents the remainder, and |·| represents the absolute value.

5. The ultrasound image recognition model according to claim 4, characterized in that: The channel weight of each channel is calculated according to the initial weight coefficient, and the channel weights of all channels are subjected to one-dimensional convolution processing to obtain the channel weight matrix of the initial multi-channel feature image, including: For each channel, all initial weight coefficients corresponding to the channel are added together to obtain the channel weight; Perform convolution on the channel weights of all channels using one-dimensional convolution to obtain an intermediate channel weight matrix; wherein, if the total number of channels is less than or equal to 128, the convolution kernel of the one-dimensional convolution is set to 5; otherwise, the convolution kernel of the one-dimensional convolution is set to 7; The intermediate channel weight matrix is activated by using an activation function to obtain the channel weight matrix.

6. An ultrasonic image recognition method, characterized in that: include: Acquire an ultrasound image training set; the ultrasound image training set includes a plurality of ultrasound images of known types; Inputting the plurality of known types of ultrasound images into the ultrasound image recognition model one by one to obtain a recognition result of each ultrasound image; the ultrasound image recognition model is the ultrasound image recognition model according to any one of claims 1 to 5; Calculating the loss of the ultrasound image recognition model according to the recognition result and the type of each ultrasound image, and performing backpropagation on the ultrasound image recognition model according to the loss until the ultrasound image recognition model is fitted, thereby obtaining a trained ultrasound image recognition model; The ultrasonic image to be recognized is input into the trained ultrasonic image recognition model to obtain a recognition result of the ultrasonic image to be recognized.

7. The ultrasonic image recognition method according to claim 6, characterized in that: Inputting the plurality of ultrasound images of known types into the ultrasound image recognition model one by one to obtain a recognition result for each ultrasound image includes: By calculating the formula Get the output probability P of each type k Among them, P k represents the output probability of type K, k = 1, 2, ..., Q, Q represents the total number of types of ultrasound images in the ultrasound image training set, z k represents the output value of the kth node, j represents the jth node, n represents the number of output nodes, and softmax(·) represents the activation function; The type with the highest output probability is taken as the recognition result.

8. The ultrasonic image recognition method according to claim 7, characterized in that: The step of calculating the loss of the ultrasound image recognition model according to the recognition result and the type of each ultrasound image, and performing backpropagation on the ultrasound image recognition model according to the loss until the ultrasound image recognition model is fitted to obtain a trained ultrasound image recognition model, comprising: For each ultrasound image, the formula FL=-a k (1-p k ) γ log(p k ) The loss FL of the ultrasound image recognition model is obtained; wherein, a k represents the weight of type k, p k represents the output probability of type k, γ represents the focusing parameter, γ∈[0,5]; The ultrasonic image recognition model is back-propagated according to the loss FL until the loss value of the new ultrasonic image recognition model obtained by back-propagation is less than a preset loss threshold, and the new ultrasonic image recognition model is determined to be fitted to obtain the trained ultrasonic image recognition model; wherein, the momentum of back-propagation is set to 0.9, the learning rate of back-propagation is obtained using a thermal learning rate strategy, and the weight attenuation coefficient of the L1 regularization of back-propagation is set to 5e-4.

Images