Intelligent detection method and device for musculoskeletal ultrasound imaging

Abstract

The present invention provides a method and device for intelligent detection of musculoskeletal ultrasound images, and relates to the field of image processing technology. The method comprises: obtaining edge feature maps corresponding to each frame of ultrasound image in a musculoskeletal ultrasound video; for each frame of ultrasound image, determining a multiscale feature map set of the edge feature map corresponding to the ultrasound image and a multiscale feature map set of the corresponding historical frame image based on a multiscale feature encoding network; determining a target feature map of the ultrasound image based on the multiscale feature map set of the edge feature map and the multiscale feature map set of the historical frame image; determining whether a specific tissue is included in the target feature map based on a preset tissue classifier, or, if a specific tissue is included in the target feature map based on a preset bounding box predictor, determining the size, location information and confidence of the specific tissue. The present invention reduces the limitations of subjective judgment of medical personnel, improves the accuracy of judgment, and can identify subtle tissue structures that are easily overlooked by the naked eye.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method and device for intelligent detection of musculoskeletal ultrasonic images. Background Art

[0002] Musculoskeletal UltraSound (MSKUS) is an ultrasound diagnostic technology used in the musculoskeletal system, which is different from the traditional ultrasound technology used in the fields of abdomen, heart, obstetrics and gynecology. After a long period of development, musculoskeletal ultrasound has become the main imaging technology for detecting the musculoskeletal system alongside X-ray, CT and MRI, and is widely used in professional fields such as rehabilitation, orthopedic surgery, neurosurgery, pain medicine, sports medicine and rheumatology.

[0003] In traditional musculoskeletal ultrasound imaging, medical staff generally perform manual identification of musculoskeletal ultrasound images to identify muscles, tendons, nerves and other tissues in musculoskeletal ultrasound images. Traditional musculoskeletal ultrasound imaging is limited by the subjective judgment of medical staff. Medical staff will suffer from visual fatigue after working for a long time, which affects the accuracy of judgment. And because musculoskeletal ultrasound images are dynamically changing, medical staff may easily overlook subtle tissue structures when identifying with the naked eye.

[0004] Currently, there is an urgent need for a musculoskeletal ultrasound imaging detection method that can quickly and efficiently detect tissues such as muscles, tendons, and nerves in musculoskeletal ultrasound images. Summary of the invention

[0005] The present invention provides an intelligent detection method and device for musculoskeletal ultrasonic imaging, which is used to solve the problem in the prior art that musculoskeletal ultrasonic imaging detection is limited by the subjective judgment of medical personnel. When medical personnel are visually fatigued, the accuracy of judgment is affected, and subtle defects in tissue structure are easily overlooked. The present invention can reduce the limitations of medical personnel's subjective judgment, improve the accuracy of musculoskeletal system judgment, and can identify subtle tissue structures that are easily overlooked by the naked eye.

[0006] The present invention provides a method for intelligent detection of muscle-bone ultrasonic images, comprising:

[0007] Obtain edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video;

[0008] For each frame of the ultrasound image, determining a multiscale feature map set of an edge feature map corresponding to the ultrasound image and a multiscale feature map set of a corresponding historical frame image based on a multiscale feature encoding network;

[0009] Determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0010] Determine whether the target feature map contains specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0011] According to a method for intelligent detection of musculoskeletal ultrasound images provided by the present invention, the step of obtaining edge feature maps corresponding to each frame of ultrasound image in a musculoskeletal ultrasound video comprises:

[0012] Acquire each frame of ultrasound image in the musculoskeletal ultrasound video;

[0013] Image processing is performed on each frame of the ultrasound image to obtain an edge feature map corresponding to each frame of the ultrasound image. The image processing includes: grayscale processing, Gaussian denoising processing, frame gradient calculation, non-maximum suppression and double threshold boundary selection.

[0014] According to a method for intelligent detection of musculoskeletal ultrasonic images provided by the present invention, the image processing is performed on each frame of the ultrasonic image to obtain an edge feature map corresponding to each frame of the ultrasonic image, including:

[0015] For each frame of the ultrasonic image, grayscale processing and Gaussian denoising processing are performed on the ultrasonic image to obtain an initial edge feature map corresponding to the ultrasonic image;

[0016] Determine the gradient value of each pixel in the initial edge feature map respectively;

[0017] Comparing the gradient value of each pixel on the initial edge feature map with the gradient values ​​of pixels adjacent to the pixel in the positive gradient direction and the gradient values ​​of pixels adjacent to the pixel in the negative gradient direction, respectively, to determine a plurality of edge pixels, and suppressing the gradient values ​​of pixels other than the edge pixels to 0;

[0018] Based on a preset double threshold, determining strong edge pixels, weak edge pixels and other pixels in each of the edge pixels, and suppressing the gradient values ​​of the other pixels to 0;

[0019] When the strong edge pixel does not exist in the pixels adjacent to the weak edge pixel, suppressing the gradient value of the weak edge pixel to 0;

[0020] In the case where the strong edge pixel exists in pixels adjacent to the weak edge pixel, determining the weak edge pixel as a true edge pixel;

[0021] An edge feature map of the ultrasound image is determined based on the strong edge pixels and the true edge pixels.

[0022] According to a method for intelligent detection of musculoskeletal ultrasound images provided by the present invention, the multi-scale feature coding network is a dual-path multi-scale feature coding network, and the dual-path multi-scale feature coding network includes a global multi-scale feature coding network and an adjacent multi-scale feature coding network;

[0023] The method of determining a set of multiscale feature maps of edge feature maps corresponding to the ultrasound image and a set of multiscale feature maps of corresponding historical frame images based on a multiscale feature coding network includes:

[0024] Determine a multi-scale feature map set of the edge feature map corresponding to the ultrasound image based on the dual-path multi-scale feature encoding network;

[0025] Based on the global multi-scale feature encoding network, determining a preset number of first historical frame images corresponding to the ultrasound image, and determining a set of multi-scale feature maps of the first historical frame images;

[0026] Based on the adjacent multi-scale feature encoding network, determining a preset number of second historical frame images adjacent to the ultrasound image, and determining a multi-scale feature map set of the second historical frame images;

[0027] A multi-scale feature map set of the historical frame image corresponding to the ultrasound image is determined based on the multi-scale feature map set of the first historical frame image and the multi-scale feature map set of the second historical frame image.

[0028] According to a method for intelligent detection of musculoskeletal ultrasound images provided by the present invention, the multi-scale feature map set based on the edge feature map and the multi-scale feature map set of the historical frame image determines the target feature map of the ultrasound image, including:

[0029] Determine a target multi-scale feature map set based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0030] Encoding the target multi-scale feature map set based on a preset encoder;

[0031] The encoded target multi-scale feature map set is decoded based on a preset decoder and preset learning query parameters to determine the target feature map of the ultrasound image.

[0032] According to a method for intelligent detection of muscle-bone ultrasonic images provided by the present invention, the method further comprises:

[0033] Determine a multi-scale attention mechanism integrating edge perception based on a preset query feature and a sampling point corresponding to the preset query feature;

[0034] The edge-aware multi-scale attention mechanism is integrated into the initial encoder and the initial decoder respectively to obtain a preset encoder corresponding to the initial encoder and a preset decoder corresponding to the initial decoder.

[0035] According to a musculoskeletal ultrasonic imaging intelligent detection method provided by the present invention, the preset tissue classifier and the preset bounding box predictor are respectively obtained by training based on the following methods:

[0036] Get the training dataset;

[0037] Iteratively training an initial tissue classifier based on the training data set, and optimizing parameters of the initial tissue classifier based on a first loss function to obtain the preset tissue classifier;

[0038] An initial bounding box predictor is iteratively trained based on the training data set, and parameters of the initial bounding box predictor are optimized based on a second loss function and a generalized IoU loss function to obtain the preset bounding box predictor.

[0039] The present invention also provides a musculoskeletal ultrasonic imaging intelligent detection device, comprising:

[0040] An acquisition module, used to acquire edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video;

[0041] A first determination module is used to determine, for each frame of the ultrasound image, a multi-scale feature map set of an edge feature map corresponding to the ultrasound image and a multi-scale feature map set of a corresponding historical frame image based on a multi-scale feature coding network;

[0042] A second determination module is used to determine the target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0043] A classification and recognition module determines whether the target feature map contains a specific tissue based on a preset tissue classifier, or, when it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determines the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0044] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the intelligent detection method for musculoskeletal ultrasonic imaging as described above is implemented.

[0045] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the intelligent detection method for musculoskeletal ultrasonic imaging as described in any one of the above is implemented.

[0046] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the intelligent detection method for musculoskeletal ultrasonic imaging as described above is implemented.

[0047] The present invention provides a method and device for intelligent detection of musculoskeletal ultrasound images. First, edge feature maps corresponding to each frame of ultrasound image in a musculoskeletal ultrasound video are obtained. Then, for each frame of ultrasound image, a multiscale feature map set of edge feature maps corresponding to the ultrasound image and a multiscale feature map set of corresponding historical frame images are determined based on a multiscale feature encoding network. Then, a target feature map of the ultrasound image is determined based on the multiscale feature map set of the edge feature map and the multiscale feature map set of the historical frame images. A preset tissue classifier is used to determine whether a specific tissue is included in the target feature map. Alternatively, when a preset bounding box predictor is used to determine whether a specific tissue is included in the target feature map, the size, location information and confidence of the specific tissue are determined based on the preset bounding box predictor. The technical solution of the present invention reduces the limitations of subjective judgment of medical personnel, improves the accuracy of judgment of the musculoskeletal system, and can identify subtle tissue structures that are easily overlooked by the naked eye. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0049] Figure 1 It is a schematic diagram of the process of the intelligent detection method of muscle-bone ultrasonic imaging provided by an embodiment of the present invention;

[0050] Figure 2 is a schematic diagram of the structure of a musculoskeletal ultrasonic imaging intelligent detection device provided by an embodiment of the present invention;

[0051] Figure 3 It is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In view of the fact that the musculoskeletal ultrasound imaging detection in the prior art is limited by the subjective judgment of medical personnel, the medical personnel may be visually fatigued, which affects the accuracy of judgment and may easily ignore subtle tissue structure problems, the present invention provides a musculoskeletal ultrasound imaging intelligent detection method. Figure 1 FIG. 1 is a flow chart of a method for intelligent detection of muscle-bone ultrasound images provided by an embodiment of the present invention. Figure 1 As shown, the musculoskeletal ultrasonic imaging intelligent detection method includes the following steps:

[0054] Step 110: Obtain edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video.

[0055] In one embodiment, the step of obtaining edge feature maps corresponding to respective frames of ultrasound images in the musculoskeletal ultrasound video includes:

[0056] Acquire each frame of ultrasound image in the musculoskeletal ultrasound video;

[0057] Image processing is performed on each frame of the ultrasound image to obtain an edge feature map corresponding to each frame of the ultrasound image. The image processing includes: grayscale processing, Gaussian denoising processing, frame gradient calculation, non-maximum suppression and double threshold boundary selection.

[0058] Specifically, a musculoskeletal ultrasound video can be obtained, and then each frame of ultrasound image in the musculoskeletal ultrasound video can be processed using an edge extraction network to obtain edge feature maps corresponding to each frame of ultrasound image. Exemplarily, a Canny edge extraction method based on a graphics processing unit (Graphics Processing Unit) can be used for image processing, and the image processing can include: grayscale processing, Gaussian denoising processing, frame gradient calculation, non-maximum suppression, and double threshold boundary selection.

[0059] In the above embodiment, each frame of ultrasound image in the musculoskeletal ultrasound video is first obtained, and then each frame of ultrasound image is processed separately to obtain an edge feature map corresponding to each frame of ultrasound image, thereby avoiding the interference of noise points and making the edges in the edge feature map clearer. In addition, the image processing method has a small amount of calculation and can obtain the edge feature map corresponding to each frame of ultrasound image in a relatively short time.

[0060] In one embodiment, performing image processing on each frame of the ultrasound image to obtain an edge feature map corresponding to each frame of the ultrasound image includes:

[0061] For each frame of the ultrasonic image, grayscale processing and Gaussian denoising processing are performed on the ultrasonic image to obtain an initial edge feature map corresponding to the ultrasonic image;

[0062] Determine the gradient value of each pixel in the initial edge feature map respectively;

[0063] Comparing the gradient value of each pixel on the initial edge feature map with the gradient values ​​of pixels adjacent to the pixel in the positive gradient direction and the gradient values ​​of pixels adjacent to the pixel in the negative gradient direction, respectively, to determine a plurality of edge pixels, and suppressing the gradient values ​​of pixels other than the edge pixels to 0;

[0064] Based on a preset double threshold, determining strong edge pixels, weak edge pixels and other pixels in each of the edge pixels, and suppressing the gradient values ​​of the other pixels to 0;

[0065] When the strong edge pixel does not exist in the pixels adjacent to the weak edge pixel, suppressing the gradient value of the weak edge pixel to 0;

[0066] In the case where the strong edge pixel exists in pixels adjacent to the weak edge pixel, determining the weak edge pixel as a true edge pixel;

[0067] An edge feature map of the ultrasound image is determined based on the strong edge pixels and the true edge pixels.

[0068] Specifically, for each frame of ultrasound image, since the ultrasound image obtained by the direct-connected ultrasound device is a RGB three-channel color video frame, and a single-channel grayscale video frame is sufficient to provide boundary information, the ultrasound image can be grayed to reduce redundant data. Exemplarily, when the ultrasound image is grayed, the weighted conversion coefficients of the RGB three channels are 0.299, 0.587, and 0.114, respectively.

[0069] In order to minimize the influence of noise on the edge extraction result, the noise can be filtered to prevent false detection caused by noise. Therefore, the gray-scaled ultrasound image can be subjected to Gaussian denoising based on a Gaussian filter to obtain an initial edge feature map corresponding to the ultrasound image. Exemplarily, a Gaussian filter can be used to convolve the gray-scaled ultrasound image, and the process can be represented by the following mathematical expression:

[0070]

[0071] The size of the Gaussian filter is 7 times 7, and the variance σ = 1.4. G[i,j] represents the pixel with coordinates (i,j), b = 3 represents the filter coefficient, π represents the circumference, and e represents a natural constant.

[0072] After obtaining the initial edge feature map, the Sobel operator can be used to determine the gradient value of each pixel in the initial edge feature map. For example, the Sobel operator can be used to perform convolution operations on the initial edge feature map in the horizontal and vertical directions to obtain two gradient components. The gradient value can be used to determine whether each pixel is inside a specific tissue or at the boundary of a specific tissue. After determining the gradient value of each pixel in the initial edge feature map, for each pixel on the initial edge feature map, the gradient value of the pixel can be compared with the gradient value of the pixel adjacent to the pixel in the positive gradient direction and the gradient value of the pixel adjacent to the pixel in the negative gradient direction. When the gradient value of the pixel is greater than the gradient value of the pixel adjacent to the pixel in the positive gradient direction, and the gradient value of the pixel is greater than the gradient value of the pixel adjacent to the pixel in the negative gradient direction, the pixel can be determined to be an edge pixel, and the edge pixel is determined to be a pixel at the boundary of a specific tissue. Further, multiple edge pixels can be determined, and the gradient values ​​of pixels other than edge pixels can be suppressed to 0, and pixels whose gradient values ​​are suppressed to 0 are determined to be pixels inside a specific tissue.

[0073] Further, a preset double threshold value can be adaptively selected, and the preset double threshold value can include a preset gradient value high threshold value and a preset gradient value low threshold value, and the preset gradient value high threshold value is greater than the preset gradient value low threshold value. When the gradient value of the edge pixel is greater than or equal to the preset gradient value high threshold value, the edge pixel is determined to be a strong edge pixel. When the gradient value of the edge pixel is greater than the preset gradient value low threshold value and less than the preset gradient value high threshold value, the edge pixel is determined to be a weak edge pixel. When the gradient value of the edge pixel is less than or equal to the gradient value low threshold value, the edge pixel is determined to be other pixels, and the gradient values ​​of other pixels can be suppressed to 0. Among them, the weak edge pixel may be a real edge pixel, or it may be due to noise or color change that the pixel is determined to be a weak edge pixel. Generally speaking, when a weak edge pixel is connected to a strong edge pixel, the weak edge pixel is a real edge pixel. Based on this, it can be determined whether there is a strong edge pixel in the pixels adjacent to the weak edge pixel. It is easy to understand that the number of pixels adjacent to the weak edge pixel depends on the position of the weak edge pixel in the edge feature map. When there is no strong edge pixel among the pixels adjacent to the weak edge pixel, the pixel may be determined as a weak edge pixel due to noise or color change, and the gradient value of the weak edge pixel can be suppressed to 0; when there is a strong edge pixel among the pixels adjacent to the weak edge pixel, the weak edge pixel is determined as a true edge pixel.

[0074] Finally, the edge feature map of the ultrasound image can be determined based on the strong edge pixels and the true edge pixels.

[0075] In the above embodiment, redundant data is reduced by graying the ultrasound image, and the computational overhead is reduced. The interference of noise points is avoided by Gaussian denoising, so that the edges in the edge feature map are clearer. The preset double threshold is adaptively selected, which can be applied to different images. In addition, the image processing method has a small amount of computation and can obtain the edge feature maps corresponding to each frame of ultrasound image in a short time.

[0076] Step 120: for each frame of the ultrasound image, based on a multi-scale feature encoding network, determine a multi-scale feature map set of the edge feature map corresponding to the ultrasound image and a multi-scale feature map set of the corresponding historical frame images.

[0077] In one embodiment, the multi-scale feature encoding network is a dual-path multi-scale feature encoding network, and the dual-path multi-scale feature encoding network includes a global multi-scale feature encoding network and an adjacent multi-scale feature encoding network;

[0078] The method of determining a set of multiscale feature maps of edge feature maps corresponding to the ultrasound image and a set of multiscale feature maps of corresponding historical frame images based on a multiscale feature coding network includes:

[0079] Determine a multi-scale feature map set of the edge feature map corresponding to the ultrasound image based on the dual-path multi-scale feature encoding network;

[0080] Based on the global multi-scale feature encoding network, determining a preset number of first historical frame images corresponding to the ultrasound image, and determining a set of multi-scale feature maps of the first historical frame images;

[0081] Based on the adjacent multi-scale feature encoding network, determining a preset number of second historical frame images adjacent to the ultrasound image, and determining a multi-scale feature map set of the second historical frame images;

[0082] A multi-scale feature map set of the historical frame image corresponding to the ultrasound image is determined based on the multi-scale feature map set of the first historical frame image and the multi-scale feature map set of the second historical frame image.

[0083] Specifically, the multi-scale feature coding network may be a dual-path multi-scale feature coding network, and the dual-path multi-scale feature coding network may include a global multi-scale feature coding network and an adjacent multi-scale feature coding network. For each frame of ultrasound image, a multi-scale feature map set of edge feature maps corresponding to the ultrasound image may be determined based on the dual-path multi-scale feature coding network. It is also possible to randomly determine a preset number of first historical frame images corresponding to the ultrasound image based on the global multi-scale feature coding network, the first historical frame image being the ultrasound image before the ultrasound image in the same musculoskeletal ultrasound video for any ultrasound image, for example, for the 88th ultrasound image in a certain musculoskeletal ultrasound video, a preset number of ultrasound images may be randomly selected from the 1st to 87th ultrasound images as the first historical frame image, or a preset number of ultrasound images may be extracted from the 1st to 87th ultrasound images based on a fixed time interval as the first historical frame image, it is easy to understand that for an ultrasound image in a certain musculoskeletal ultrasound video, when the number of ultrasound images before the ultrasound image is less than the preset number, the ultrasound image may be supplemented and determined as the first historical frame image, and a multi-scale feature map set of the first historical frame image may be determined. It is also possible to determine a preset number of second historical frame images adjacent to the ultrasound image based on the adjacent multi-scale feature encoding network. The second historical frame image is, for any ultrasound image, an ultrasound image that is located before and adjacent to the ultrasound image in the same musculoskeletal ultrasound video. For example, for the 88th ultrasound image in a certain musculoskeletal ultrasound video, when the preset number is 3, the 85th, 86th and 87th frames can be selected as the second historical frame images. It is easy to understand that for an ultrasound image in a certain musculoskeletal ultrasound video, when the number of ultrasound images before the ultrasound image is less than the preset number, the ultrasound image can be supplemented and determined as the second historical frame image, and a multi-scale feature map set of the second historical frame image can be determined. Furthermore, the multi-scale feature map set of the historical frame image corresponding to the ultrasound image can be determined based on the multi-scale feature map set of the first historical frame image and the multi-scale feature map set of the second historical frame image.

[0084] For example, it can be used Represents a set of multi-scale feature maps, where l represents the number of layers of the multi-scale feature map, and L represents the total number of layers of the multi-scale feature map. in represents the real number space, C represents the number of channels, H l represents the height of the multi-scale feature map of the lth layer, W l Represents the width of the multi-scale feature map of the lth layer. For the ultrasound image of the tth frame, the multi-scale feature map set F of the edge feature map corresponding to the ultrasound image of the tth frame can be determined based on the dual-path multi-scale feature encoding network. tBased on the global multi-scale feature encoding network, the first historical frame image r1, r2, r3 corresponding to the ultrasound image of the tth frame is randomly determined, and the multi-scale feature map set F of the first historical frame image is determined r1 , F r2 , F r3 Based on the adjacent multi-scale feature encoding network, the second historical frame images n1, n2, n3 adjacent to the ultrasound image of the t-th frame are determined, and the multi-scale feature map set F1, F2, F3 of the second historical frame images is determined. Further, the multi-scale feature map set F1, F2, F3, F1, F2, F3 of the historical frame images corresponding to the ultrasound image of the t-th frame can be determined.

[0085] In the above embodiment, based on the multi-scale feature encoding network, a multi-scale feature map set of the edge feature map corresponding to the ultrasound image and a multi-scale feature map set of the corresponding historical frame image are determined, which lays a foundation for determining the target feature map.

[0086] Step 130: Determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image.

[0087] In one embodiment, the determining the target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image includes:

[0088] Determine a target multi-scale feature map set based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0089] Encoding the target multi-scale feature map set based on a preset encoder;

[0090] The encoded target multi-scale feature map set is decoded based on a preset decoder and preset learning query parameters to determine the target feature map of the ultrasound image.

[0091] Specifically, for the ultrasound image of the tth frame, the multi-scale feature map set F based on the edge feature map can be t And the multi-scale feature map set F1, F2, F3, F r1 , F r2 , F r3 Determine the target multi-scale feature map set F1, F2, F3, F t , F r1 , F r2 , F r3 , and then the target multi-scale feature map set F1, F2, F3, F can be obtained based on the preset encoder EMSTA-Encoder t , Fr1 , F r2 , F r3 Encode and obtain the encoded target multi-scale feature map set E′ n1 , E′ n2 , E′ n3 , E′ t , E′ r1 , E′ r2 , E′ r3 , we can further use the preset decoder EMSTA-Decoder and the preset learning query parameter Q to encode the target multi-scale feature map set E′ n1 , E′ n2 , E′ n3 , E′ t , E′ r1 , E′ r2 , E′ r3 Decode and determine the target feature map P of the ultrasound image of the tth frame t .in, N represents the number of categories, E′ n1 , E′ n2 , E′ n3 The coefficient for decoding stacking can be 0.4, E′ r1 , E′ r2 , E′ r3 The coefficient for decoding stacking can be 0.4, E′ t The factor when decoding stacks can be 0.2.

[0092] In the above embodiment, a target feature map of the ultrasound image is determined based on a multi-scale feature map set of edge feature maps and a multi-scale feature map set of historical frame images, which lays a foundation for the subsequent preset tissue classifier and preset bounding box predictor to identify and judge characteristic tissues in the ultrasound image.

[0093] Step 140: Determine whether the target feature map contains specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0094] Specifically, it is possible to determine whether a specific tissue is included in the target feature map based on a preset tissue classifier, and the specific tissue may be a tissue structure such as muscle, tendon, bone, and nerve, for example, the long axis of the median nerve, the short axis of the median nerve, the flexor digitorum profundus, the flexor digitorum superficialis, the pronator quadratus, the flexor carpi radialis, the palmaris longus, the brachioradialis, and the transverse carpal ligament. Alternatively, it is also possible to determine whether the target feature map contains a specific tissue based on a preset bounding box predictor, and in the case that the target feature map contains a specific tissue, it is also possible to determine the size, location information, and confidence of the specific tissue based on the preset bounding box predictor, wherein the confidence represents the reliability of the size and location information of the specific tissue determined by the bounding box predictor.

[0095] The present invention provides a method for intelligent detection of musculoskeletal ultrasound images. First, edge feature maps corresponding to each frame of ultrasound image in a musculoskeletal ultrasound video are obtained. Then, for each frame of ultrasound image, a multiscale feature map set of edge feature maps corresponding to the ultrasound image and a multiscale feature map set of corresponding historical frame images are determined based on a multiscale feature encoding network. Then, a target feature map of the ultrasound image is determined based on the multiscale feature map set of the edge feature map and the multiscale feature map set of the historical frame images. A preset tissue classifier is used to determine whether a specific tissue is included in the target feature map. Alternatively, when a preset bounding box predictor is used to determine whether a specific tissue is included in the target feature map, the size, location information and confidence of the specific tissue are determined based on the preset bounding box predictor. The technical solution of the present invention reduces the limitations of subjective judgment of medical personnel, improves the accuracy of judgment of the musculoskeletal system, and can identify subtle tissue structures that are easily overlooked by the naked eye.

[0096] In one embodiment, the musculoskeletal ultrasonic imaging intelligent detection method further includes:

[0097] Determine a multi-scale attention mechanism integrating edge perception based on a preset query feature and a sampling point corresponding to the preset query feature;

[0098] The edge-aware multi-scale attention mechanism is integrated into the initial encoder and the initial decoder respectively to obtain a preset encoder corresponding to the initial encoder and a preset decoder corresponding to the initial decoder.

[0099] Specifically, the query feature z for querying the multi-scale feature map set can be preset: q And the preset query feature z q The corresponding sampling point p q , and then p q It can also represent multi-scale feature maps The sampling point p obtained by sampling qThe coordinates of the edge-aware multi-scale spatial-temporal attention mechanism can be determined, which can be expressed by the following expression:

[0100]

[0101] Among them, m represents the multi-head index, k represents the sampling point index, and W m represents the linear layer, A tlqk represents the attention weight of the sampling point, Δp tlqk Represents the sample offset of the sampling point, Δp tlqk Based on the preset query feature z q Prediction obtained, φ l Indicates normalization. tlqk By setting the preset query feature z q Feeding it to the linear layer and the SoftMax layer to predict, we can get the sum of the attention weights of the sampling points to be equal to 1, that is

[0102] The multi-scale attention mechanism integrating edge perception can be further integrated into the initial encoder, and a cascade structure can be used to stack five EMSTA layers and five FNN layers (Feedforward Neural Network) in the initial encoder, so as to obtain a preset encoder corresponding to the initial encoder.

[0103] The multi-scale attention mechanism integrating edge perception can also be integrated into the initial decoder. Six self-attention layers, six LSTDA layers (Long Short-Term Dependence Adaptive Layer) and six FNN layers can be stacked in the initial decoder to obtain a preset decoder corresponding to the initial decoder.

[0104] In the above embodiment, compared with the standard deformable attention, the edge-aware multi-scale attention makes full use of the long spatial information within the frame, the temporal information across frames, and the edge information, and integrates the edge-aware multi-scale attention mechanism into the initial encoder and the initial decoder respectively, and obtains the preset encoder corresponding to the initial encoder and the preset decoder corresponding to the initial decoder, which reduces the computational complexity of the encoding process and facilitates the deep feature extraction of the decoding process.

[0105] In one embodiment, the preset tissue classifier and the preset bounding box predictor are respectively trained based on the following methods:

[0106] Get the training dataset;

[0107] Iteratively training an initial tissue classifier based on the training data set, and optimizing parameters of the initial tissue classifier based on a first loss function to obtain the preset tissue classifier;

[0108] An initial bounding box predictor is iteratively trained based on the training data set, and parameters of the initial bounding box predictor are optimized based on a second loss function and a generalized IoU loss function to obtain the preset bounding box predictor.

[0109] Specifically, a certain number of data sets can be obtained, and some of the data sets can be used as training data sets. The data sets can include feature maps and their corresponding labels. The feature maps and the target feature maps of the ultrasound images obtained by the aforementioned musculoskeletal ultrasound imaging intelligent detection method can be the same type of images. After obtaining the training data set, the initial tissue classifier can be iteratively trained based on the training data set, and the parameters of the initial tissue classifier can be optimized based on the first loss function to obtain a preset tissue classifier, wherein the first loss function can be determined by the negative log-likelihood method.

[0110] After obtaining the training data set, the initial bounding box predictor can be iteratively trained based on the feature map and its corresponding label in the training data set, and the parameters of the initial bounding box predictor can be optimized based on a linear combination of the second loss function and the generalized IoU loss function (GeneralizedIoU Loss) to obtain a preset bounding box predictor, wherein the intersection of the predicted value and the actual value of the initial bounding box predictor can be determined, and the union of the predicted value and the actual value of the initial bounding box predictor can be determined, and the ratio of the intersection to the union is used as the second loss function.

[0111] In the above embodiment, a preset tissue classifier and a preset bounding box predictor are obtained through training, so that the preset tissue classifier can more accurately determine whether a specific tissue is contained in a target feature map, and the preset bounding box predictor can more accurately determine whether a specific tissue is contained in a target feature map, and determine the size, location information and confidence of the specific tissue.

[0112] Optionally, a data set other than the training data set in the data set may be used as a test data set.

[0113] Optionally, the performance of the tissue classifier can be comprehensively evaluated and compared using two indicators, average precision and recall. The output results of the tissue classifier can be divided into positive samples (Positive) and negative samples (Negative), and the number of True Positives (TP) correctly classified as positive samples can be determined, that is, the number of instances that are actually positive samples and classified as positive samples by the tissue classifier. The number of False Positives (FP) incorrectly classified as positive samples can also be determined, that is, the number of instances that are actually negative samples but are classified as positive samples by the tissue classifier. The number of False Negatives (FN) incorrectly classified as negative samples can also be determined, that is, the number of instances that are actually positive samples but are classified as negative samples by the tissue classifier. The number of True Negatives (TN) correctly classified as negative samples can also be determined, that is, the number of instances that are actually negative samples and are classified as negative samples by the tissue classifier. The performance of the tissue classifier can then be determined according to the following values:

[0114] Precision: refers to the ratio of the predicted value to the true value, that is, TP / (TP+FP). The larger the value, the better the performance of the tissue classifier. 1 is the ideal state.

[0115] Recall: Recall rate refers to the proportion of all True samples predicted as Positive, that is, TP / (TP+FN). The larger the value, the better the performance of the tissue classifier, and 1 is the ideal state.

[0116] PR curve: (Precision-Recall) curve. The more convex the curve is to the upper right in the two-dimensional coordinates, the better the performance of the tissue classifier.

[0117] AP: The area under the PR curve. AP stands for Average Precision. The larger the value, the better the performance of the tissue classifier.

[0118] The musculoskeletal ultrasonic imaging intelligent detection device provided by the present invention is described below. The musculoskeletal ultrasonic imaging intelligent detection device described below and the musculoskeletal ultrasonic imaging intelligent detection method described above can be referenced to each other.

[0119] Figure 2 FIG. 1 is a schematic diagram of the structure of the intelligent detection device for muscle-bone ultrasonic imaging provided by an embodiment of the present invention. Figure 2 As shown, the musculoskeletal ultrasonic imaging intelligent detection device 200 includes:

[0120] An acquisition module 210 is used to acquire edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video;

[0121] A first determination module 220 is used to determine, for each frame of the ultrasound image, a multi-scale feature map set of an edge feature map corresponding to the ultrasound image and a multi-scale feature map set of a corresponding historical frame image based on a multi-scale feature coding network;

[0122] A second determination module 230 is used to determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0123] The classification and identification module 240 is used to determine whether the target feature map contains specific tissue based on a preset tissue classifier, or, when it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0124] In one embodiment, the acquisition module 210 is specifically used for:

[0125] Acquire each frame of ultrasound image in the musculoskeletal ultrasound video;

[0126] Image processing is performed on each frame of the ultrasound image to obtain an edge feature map corresponding to each frame of the ultrasound image. The image processing includes: grayscale processing, Gaussian denoising processing, frame gradient calculation, non-maximum suppression and double threshold boundary selection.

[0127] In one embodiment, the acquisition module 210 is further configured to:

[0128] For each frame of the ultrasonic image, grayscale processing and Gaussian denoising processing are performed on the ultrasonic image to obtain an initial edge feature map corresponding to the ultrasonic image;

[0129] Determine the gradient value of each pixel in the initial edge feature map respectively;

[0130] Comparing the gradient value of each pixel on the initial edge feature map with the gradient values ​​of pixels adjacent to the pixel in the positive gradient direction and the gradient values ​​of pixels adjacent to the pixel in the negative gradient direction, respectively, to determine a plurality of edge pixels, and suppressing the gradient values ​​of pixels other than the edge pixels to 0;

[0131] Based on a preset double threshold, determining strong edge pixels, weak edge pixels and other pixels in each of the edge pixels, and suppressing the gradient values ​​of the other pixels to 0;

[0132] When the strong edge pixel does not exist in the pixels adjacent to the weak edge pixel, suppressing the gradient value of the weak edge pixel to 0;

[0133] In the case where the strong edge pixel exists in pixels adjacent to the weak edge pixel, determining the weak edge pixel as a true edge pixel;

[0134] An edge feature map of the ultrasound image is determined based on the strong edge pixels and the true edge pixels.

[0135] In one embodiment, the multi-scale feature coding network is a dual-path multi-scale feature coding network, which includes a global multi-scale feature coding network and an adjacent multi-scale feature coding network. The first determination module 220 is specifically used to:

[0136] The method of determining a set of multiscale feature maps of edge feature maps corresponding to the ultrasound image and a set of multiscale feature maps of corresponding historical frame images based on a multiscale feature coding network includes:

[0137] Determine a multi-scale feature map set of the edge feature map corresponding to the ultrasound image based on the dual-path multi-scale feature encoding network;

[0138] Based on the global multi-scale feature encoding network, determining a preset number of first historical frame images corresponding to the ultrasound image, and determining a set of multi-scale feature maps of the first historical frame images;

[0139] Based on the adjacent multi-scale feature encoding network, determining a preset number of second historical frame images adjacent to the ultrasound image, and determining a multi-scale feature map set of the second historical frame images;

[0140] A multi-scale feature map set of the historical frame image corresponding to the ultrasound image is determined based on the multi-scale feature map set of the first historical frame image and the multi-scale feature map set of the second historical frame image.

[0141] In one embodiment, the second determining module 230 is specifically configured to:

[0142] Determine a target multi-scale feature map set based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0143] Encoding the target multi-scale feature map set based on a preset encoder;

[0144] The encoded target multi-scale feature map set is decoded based on a preset decoder and preset learning query parameters to determine the target feature map of the ultrasound image.

[0145] In one embodiment, the musculoskeletal ultrasonic imaging intelligent detection device further includes an integrated module, which is specifically used for:

[0146] Determine a multi-scale attention mechanism integrating edge perception based on a preset query feature and a sampling point corresponding to the preset query feature;

[0147] The edge-aware multi-scale attention mechanism is integrated into the initial encoder and the initial decoder respectively to obtain a preset encoder corresponding to the initial encoder and a preset decoder corresponding to the initial decoder.

[0148] In one embodiment, the musculoskeletal ultrasonic imaging intelligent detection device further includes a training module, which is specifically used for:

[0149] Get the training dataset;

[0150] Iteratively training an initial tissue classifier based on the training data set, and optimizing parameters of the initial tissue classifier based on a first loss function to obtain the preset tissue classifier;

[0151] An initial bounding box predictor is iteratively trained based on the training data set, and parameters of the initial bounding box predictor are optimized based on a second loss function and a generalized IoU loss function to obtain the preset bounding box predictor.

[0152] The present invention provides an intelligent detection device for musculoskeletal ultrasonic images. First, edge feature maps corresponding to each frame of ultrasonic image in a musculoskeletal ultrasonic video are obtained. Then, for each frame of ultrasonic image, a multiscale feature map set of edge feature maps corresponding to the ultrasonic image and a multiscale feature map set of corresponding historical frame images are determined based on a multiscale feature encoding network. Then, a target feature map of the ultrasonic image is determined based on the multiscale feature map set of the edge feature map and the multiscale feature map set of the historical frame images. A preset tissue classifier is used to determine whether a specific tissue is included in the target feature map. Alternatively, when a preset bounding box predictor is used to determine whether a specific tissue is included in the target feature map, the size, location information and confidence of the specific tissue are determined based on the preset bounding box predictor. The technical solution of the present invention reduces the limitations of subjective judgment of medical personnel, improves the accuracy of judgment of the musculoskeletal system, and can identify subtle tissue structures that are easily overlooked by the naked eye.

[0153] Figure 3 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the intelligent detection method of muscle-bone ultrasound imaging, which includes:

[0154] Obtain edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video;

[0155] For each frame of the ultrasound image, determining a multiscale feature map set of an edge feature map corresponding to the ultrasound image and a multiscale feature map set of a corresponding historical frame image based on a multiscale feature encoding network;

[0156] Determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0157] Determine whether the target feature map contains specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0158] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0159] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the muscle-bone ultrasonic imaging intelligent detection method provided by the above methods, the method includes:

[0160] Obtain edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video;

[0161] For each frame of the ultrasound image, determining a multiscale feature map set of an edge feature map corresponding to the ultrasound image and a multiscale feature map set of a corresponding historical frame image based on a multiscale feature encoding network;

[0162] Determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0163] Determine whether the target feature map contains specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0164] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the muscle-bone ultrasonic imaging intelligent detection method provided by the above methods, the method comprising:

[0165] Obtain edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video;

[0166] For each frame of the ultrasound image, determining a multiscale feature map set of an edge feature map corresponding to the ultrasound image and a multiscale feature map set of a corresponding historical frame image based on a multiscale feature encoding network;

[0167] Determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image;

[0168] Determine whether the target feature map contains specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor.

[0169] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0170] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent detection of muscle-bone ultrasonic images, characterized in that: include: Obtain edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video; For each frame of the ultrasound image, determining a multiscale feature map set of an edge feature map corresponding to the ultrasound image and a multiscale feature map set of a corresponding historical frame image based on a multiscale feature encoding network; Determine a target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image; Determine whether the target feature map contains a specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains a specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor; The multi-scale feature encoding network is a dual-path multi-scale feature encoding network, and the dual-path multi-scale feature encoding network includes a global multi-scale feature encoding network and an adjacent multi-scale feature encoding network; The method of determining a set of multiscale feature maps of edge feature maps corresponding to the ultrasound image and a set of multiscale feature maps of corresponding historical frame images based on a multiscale feature coding network includes: Determine a multi-scale feature map set of the edge feature map corresponding to the ultrasound image based on the dual-path multi-scale feature encoding network; Based on the global multi-scale feature encoding network, determining a preset number of first historical frame images corresponding to the ultrasound image, and determining a set of multi-scale feature maps of the first historical frame images; Based on the adjacent multi-scale feature encoding network, determining a preset number of second historical frame images adjacent to the ultrasound image, and determining a multi-scale feature map set of the second historical frame images; A multi-scale feature map set of the historical frame image corresponding to the ultrasound image is determined based on the multi-scale feature map set of the first historical frame image and the multi-scale feature map set of the second historical frame image.

2. The intelligent detection method of muscle-bone ultrasonic imaging according to claim 1, characterized in that: The step of obtaining edge feature maps corresponding to respective frames of ultrasound images in the musculoskeletal ultrasound video comprises: Acquire each frame of ultrasound image in the musculoskeletal ultrasound video; Image processing is performed on each frame of the ultrasound image to obtain an edge feature map corresponding to each frame of the ultrasound image. The image processing includes: grayscale processing, Gaussian denoising processing, frame gradient calculation, non-maximum suppression and double threshold boundary selection.

3. The intelligent detection method of muscle-bone ultrasonic imaging according to claim 2, characterized in that: The performing image processing on each frame of the ultrasonic image to obtain an edge feature map corresponding to each frame of the ultrasonic image includes: For each frame of the ultrasonic image, grayscale processing and Gaussian denoising processing are performed on the ultrasonic image to obtain an initial edge feature map corresponding to the ultrasonic image; Determine the gradient value of each pixel in the initial edge feature map respectively; Comparing the gradient value of each pixel on the initial edge feature map with the gradient values ​​of pixels adjacent to the pixel in the positive gradient direction and the gradient values ​​of pixels adjacent to the pixel in the negative gradient direction, respectively, to determine a plurality of edge pixels, and suppressing the gradient values ​​of pixels other than the edge pixels to 0; Based on a preset double threshold, determining strong edge pixels, weak edge pixels and other pixels in each of the edge pixels, and suppressing the gradient values ​​of the other pixels to 0; When the strong edge pixel does not exist in the pixels adjacent to the weak edge pixel, suppressing the gradient value of the weak edge pixel to 0; In the case where the strong edge pixel exists in pixels adjacent to the weak edge pixel, determining the weak edge pixel as a true edge pixel; An edge feature map of the ultrasound image is determined based on the strong edge pixels and the true edge pixels.

4. The intelligent detection method for muscle-bone ultrasonic imaging according to any one of claims 1 to 3, characterized in that: The determining the target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image comprises: Determine a target multi-scale feature map set based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image; Encoding the target multi-scale feature map set based on a preset encoder; The encoded target multi-scale feature map set is decoded based on a preset decoder and preset learning query parameters to determine the target feature map of the ultrasound image.

5. The intelligent detection method of muscle-bone ultrasonic imaging according to claim 4, characterized in that: The method further comprises: Determine a multi-scale attention mechanism integrating edge perception based on a preset query feature and a sampling point corresponding to the preset query feature; The edge-aware multi-scale attention mechanism is integrated into the initial encoder and the initial decoder respectively to obtain a preset encoder corresponding to the initial encoder and a preset decoder corresponding to the initial decoder.

6. The intelligent detection method of muscle-bone ultrasonic imaging according to claim 5, characterized in that: The preset tissue classifier and the preset bounding box predictor are respectively obtained by training based on the following methods: Get the training dataset; Iteratively training an initial tissue classifier based on the training data set, and optimizing parameters of the initial tissue classifier based on a first loss function to obtain the preset tissue classifier; An initial bounding box predictor is iteratively trained based on the training data set, and parameters of the initial bounding box predictor are optimized based on a second loss function and a generalized IoU loss function to obtain the preset bounding box predictor.

7. A muscle-bone ultrasonic imaging intelligent detection device, characterized in that: include: An acquisition module, used to acquire edge feature maps corresponding to each frame of ultrasound image in the musculoskeletal ultrasound video; A first determination module is used to determine, for each frame of the ultrasound image, a multi-scale feature map set of an edge feature map corresponding to the ultrasound image and a multi-scale feature map set of a corresponding historical frame image based on a multi-scale feature coding network; A second determination module is used to determine the target feature map of the ultrasound image based on the multi-scale feature map set of the edge feature map and the multi-scale feature map set of the historical frame image; A classification and recognition module, used to determine whether the target feature map contains a specific tissue based on a preset tissue classifier, or, if it is determined that the target feature map contains the specific tissue based on a preset bounding box predictor, determine the size, location information and confidence of the specific tissue based on the preset bounding box predictor; The multi-scale feature coding network is a dual-path multi-scale feature coding network, which includes a global multi-scale feature coding network and an adjacent multi-scale feature coding network. The first determining module is specifically used to: Determine a multi-scale feature map set of the edge feature map corresponding to the ultrasound image based on the dual-path multi-scale feature encoding network; Based on the global multi-scale feature encoding network, determining a preset number of first historical frame images corresponding to the ultrasound image, and determining a set of multi-scale feature maps of the first historical frame images; Based on the adjacent multi-scale feature encoding network, determining a preset number of second historical frame images adjacent to the ultrasound image, and determining a multi-scale feature map set of the second historical frame images; A multi-scale feature map set of the historical frame image corresponding to the ultrasound image is determined based on the multi-scale feature map set of the first historical frame image and the multi-scale feature map set of the second historical frame image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the musculoskeletal ultrasonic imaging intelligent detection method as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the intelligent detection method of musculoskeletal ultrasonic imaging as claimed in any one of claims 1 to 6 is implemented.

Images