Method, device, equipment and medium for rapid segmentation of breast lesion features in medical videos

Abstract

The present invention discloses a method, device, equipment and medium for rapid segmentation of breast lesion features in medical videos, and relates to the field of artificial intelligence technology. The method comprises: aligning the features of a first feature map to obtain a second feature map; collecting local context features of the second feature map based on a first deep convolutional layer and a first point-by-point convolutional layer to obtain a third feature map; and enhancing the features of the third feature map based on a second point-by-point convolutional layer and a third point-by-point convolutional layer to obtain a fourth feature map. Based on the deep convolutional layer and the point-by-point convolutional layer, the amount of data during feature collection can be reduced, thereby improving the reasoning speed. At the same time, after feature alignment, the local context features of the second feature map are collected by combining the first deep convolutional layer and the first point-by-point convolutional layer; the features of the third feature map are enhanced by combining the second point-by-point convolutional layer and the third point-by-point convolutional layer, which can significantly improve the accuracy of recognition.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method, device, equipment and medium for rapid segmentation of breast lesion features in medical videos. Background Art

[0002] Ultrasound videos are full of complex semantic data, providing a valuable resource for the segmentation of breast lesions. However, a key obstacle is that the computational complexity of existing models is too high, resulting in slow inference speed. This problem poses an important challenge to the widespread application of these models and limits their practicality in a wider range. Therefore, how to reduce the computational complexity of feature extraction and improve the inference speed while maintaining accuracy is a technical problem that needs to be solved in this field. Summary of the invention

[0003] The embodiments of the present invention provide a method, device, equipment and medium for rapid segmentation of breast lesion features in medical videos, aiming to solve the problems of large amount of calculation and low accuracy in the prior art of segmentation of breast lesion features in medical videos.

[0004] In a first aspect, an embodiment of the present invention provides a method for rapid segmentation of breast lesion features in medical videos. The device for rapid segmentation of breast lesion features in medical videos includes an encoder and a decoder connected to each other. The encoder includes a Stem module, a first downsampling module, a first feature extraction module, a second downsampling module, and a second feature extraction module connected in sequence; the decoder includes a first upsampling module, a third feature extraction module, a second upsampling module, a fourth feature extraction module, and a third upsampling module connected in sequence. The method is applied to the feature extraction module, the feature extraction module includes the first feature extraction module, the second feature extraction module, the third feature extraction module, and the fourth feature extraction module. The method includes:

[0005] Receiving a first feature map, and performing feature alignment on the first feature map to obtain a second feature map, wherein the first feature map is obtained based on frame image acquisition in an ultrasound video;

[0006] Based on the preset first depth convolution layer and the first point-by-point convolution layer, local context features are collected on the second feature map to obtain a third feature map;

[0007] Based on the preset second point-by-point convolution layer and the third point-by-point convolution layer, feature enhancement is performed on the third feature map to obtain a fourth feature map.

[0008] A further technical solution is that the step of performing feature alignment on the first feature map to obtain a second feature map includes:

[0009] Performing a convolution operation on the first feature map through a preset second depth convolution layer to obtain a fifth feature map;

[0010] The fifth feature map is superimposed on the first feature map to obtain the second feature map.

[0011] A further technical solution is that the local context feature acquisition is performed on the second feature map based on the preset first depth convolution layer and the first point-by-point convolution layer to obtain a third feature map, including:

[0012] Performing a convolution operation on the second feature map through the first depth convolution layer to obtain a sixth feature map;

[0013] Performing a convolution operation on the sixth feature map through the first point-by-point convolution layer to obtain the third feature map.

[0014] A further technical solution is that the local context feature acquisition is performed on the second feature map based on the preset first depth convolution layer and the first point-by-point convolution layer to obtain a third feature map, and further includes:

[0015] Before performing the convolution operation on the second feature map through the first deep convolution layer, performing batch normalization processing on the second feature map;

[0016] Before performing the convolution operation on the sixth feature map through the first point-by-point convolution layer, batch normalization is performed on the sixth feature map.

[0017] A further technical solution is that the third feature map is enhanced based on the preset second point-by-point convolution layer and the third point-by-point convolution layer to obtain a fourth feature map, including:

[0018] Superimposing the third feature map and the second feature map to obtain a seventh feature map;

[0019] Performing a feature-enhanced convolution operation on the seventh feature map through the second point-by-point convolution layer to obtain an eighth feature map, wherein a dimension of the eighth feature map is greater than a dimension of the seventh feature map;

[0020] Activating the eighth feature map by a preset activation function to obtain a ninth feature map;

[0021] Performing a feature-reducing convolution operation on the ninth feature map through the third point-by-point convolution layer to obtain the fourth feature map, wherein the dimension of the fourth feature map is equal to the dimension of the seventh feature map.

[0022] A further technical solution is that the number of convolution kernels in the second point-by-point convolution layer is greater than the number of channels of the frame image; and the number of convolution kernels in the third point-by-point convolution layer is equal to the number of channels of the frame image.

[0023] In a second aspect, the present invention provides a device for rapid segmentation of breast lesion features in medical videos, the device comprising an encoder and a decoder connected to each other, the encoder comprising a Stem module, a first down-sampling module, a first feature extraction module, a second down-sampling module and a second feature extraction module connected in sequence; the decoder comprising a first up-sampling module, a third feature extraction module, a second up-sampling module, a fourth feature extraction module and a third up-sampling module connected in sequence, wherein the feature extraction module is used to execute the method described in the first aspect, the feature extraction module comprising the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module.

[0024] Its further technical solution is that the Stem module includes a third downsampling module, a maximum pooling module and a TempLP module, the third downsampling module is used to perform a downsampling operation on the frame image currently inputted into the ultrasound video; the maximum pooling module is used to perform a maximum pooling operation on the output data of the third downsampling module to obtain the features of the currently inputted frame image; the TempLP module is used to fuse the features of the currently inputted frame image with the features of the frame image before the currently inputted frame image, and output them to the first downsampling module.

[0025] In a third aspect, an embodiment of the present invention further provides a computer device, which includes a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the above method when executing the computer program.

[0026] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program can implement the above method when executed by a processor.

[0027] The embodiment of the present invention provides a method, device, equipment and medium for rapid segmentation of breast lesion features in medical videos. The method includes: receiving a first feature map, performing feature alignment on the first feature map to obtain a second feature map, wherein the first feature map is obtained based on frame image acquisition in an ultrasound video; performing local context feature acquisition on the second feature map based on a preset first depth convolution layer and a first point-by-point convolution layer to obtain a third feature map; performing feature enhancement on the third feature map based on a preset second point-by-point convolution layer and a third point-by-point convolution layer to obtain a fourth feature map. On the one hand, based on the deep convolution layer and the point-by-point convolution layer, the amount of data during feature collection can be reduced, thereby improving the reasoning speed. At the same time, the applicant has found that after feature alignment, the local context features of the second feature map are collected by combining the first deep convolution layer and the first point-by-point convolution layer; further, the third feature map is enhanced by combining the second point-by-point convolution layer and the third point-by-point convolution layer, which can significantly improve the accuracy of recognition. After measurement, by applying the technical solution of the embodiment of the present invention, the Dice coefficient reaches 83.1, the Jaccard coefficient reaches 74.28, the precision rate reaches 85.67%, the recall rate reaches 87.19, and the FPS can reach more than 200, thereby achieving high processing efficiency while ensuring high accuracy. In summary, the present invention can achieve fast and high-precision segmentation in terminals with limited computing resources, with a wider application scenario and lower computing power cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0029] Figure 1 A schematic block diagram of a device for rapid segmentation of breast lesion features in medical video provided by an embodiment of the present invention;

[0030] Figure 2 A schematic diagram of a flow chart of a method for rapid segmentation of breast lesion features in medical videos provided by an embodiment of the present invention;

[0031] Figure 3 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0032] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0034] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0035] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0036] As used in this specification and the appended claims, the term "if" may be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if [described condition or event] is detected" may be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0037] The embodiment of the present invention proposes a medical video breast lesion feature fast segmentation device, which includes an encoder and a decoder connected to each other. The encoder includes a Stem module, a first downsampling module, a first feature extraction module, a second downsampling module and a second feature extraction module connected in sequence; the decoder includes a first upsampling module, a third feature extraction module, a second upsampling module, a fourth feature extraction module and a third upsampling module connected in sequence. The frame image in the ultrasound video passes through the encoder and decoder in the medical video breast lesion feature fast segmentation device in sequence to obtain a segmentation result. The Stem module includes a third downsampling module, a maximum pooling module and a TempLP module.

[0038] Furthermore, the first down-sampling module, the second down-sampling module and the third down-sampling module are used to perform down-sampling operations; the first up-sampling module, the second up-sampling module and the third up-sampling module are used to perform up-sampling operations; the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module are used to collect features.

[0039] Furthermore, the third downsampling module is used to perform a downsampling operation on the frame image currently inputted into the ultrasound video; the maximum pooling module is used to perform a maximum pooling operation on the output data of the third downsampling module to obtain the features of the currently inputted frame image; the TempLP module is used to fuse the features of the currently inputted frame image with the features of the frame image before the currently inputted frame image, and output them to the first downsampling module.

[0040] Furthermore, in the context of ultrasound video processing, the TempLP module provides a unique solution in improving the efficiency of ultrasound video processing, especially in sharing shallow semantic representations along the temporal dimension. Guided by the principle of temporal locality in data access patterns, the TempLP module proactively predicts future memory requirements. The TempLP module utilizes shallow semantics of future adjacent frames as pre-fetched representative data values. By dynamically updating the cache memory with breast lesion features, repeated memory accesses to similar semantic representations are reduced, thereby reducing the latency of pattern retrieval and the overall storage requirements. The TempLP module significantly accelerates the overall processing speed and improves the efficiency of the system during ultrasound video sequence processing.

[0041] At the same time, the TempLP module performs cache management and performs conscious cache clearing operations when new ultrasound video data enters the Stem module. This process ensures that temporal consistency is maintained and effectively eliminates possible interference from previously cached, irrelevant representations. Maintaining a clean cache enables the system to better predict future memory requirements and optimize memory resource management. In the field of ultrasound video processing, adhering to temporal consistency is crucial to avoid confusion in shallow semantic representations, thereby enhancing the reliability of all subsequent stages of data processing. Therefore, through the comprehensive mechanism of the Stem module - combining downsampling operations, maximum pooling operations and the TempLP module, an efficient system that can perform both feature extraction and inter-frame information transmission in ultrasound video processing is achieved.

[0042] The specific operation process of the TempLP module is as follows:

[0043] Assume that the current frame is t, the next frame of the current frame of the video is t+1, and so on.

[0044] Assume that the semantic feature of the current frame is f, the semantic feature of the next frame is f+1, and so on.

[0045] Step 1.1: Send the current video frame t to the TempLP module to obtain the shallow semantic feature f of the current frame. And store this feature in the STEM module.

[0046] Step 1.2: Perform network prediction to obtain the segmentation result of the current frame.

[0047] Step 2.1: Take the input image at time t+1 and extract the shallow semantic feature f stored in the STEM module at time t.

[0048] Step 2.2: The video at time t+1 is sent to the TempLP module to obtain the shallow semantic feature f+1 at time t+1.

[0049] Step 2.3: The shallow semantic features f and f+1 are fused to obtain the updated shallow semantic features f'. The meaning is that the previous frame and the current frame are fused to obtain features containing time information.

[0050] Step 2.4: The updated shallow semantic features f' are stored in the STEM module.

[0051] Step 2.5: Perform network prediction to obtain the segmentation result at time t+1.

[0052] Step 3.1: Take the input image at time t+2 and extract the shallow semantic features f' stored in the STEM module at time t+1.

[0053] Step 3.2: The video at time t+2 is sent to the TempLP module to obtain the shallow semantic feature f+2 at time t+2.

[0054] Step 3.3: The shallow semantic feature f' is fused with f+2. The updated shallow semantic feature f' is obtained. The meaning is that the previous frame and the current frame are fused to obtain a feature containing time information.

[0055] Step 3.4: The updated shallow semantic features f” are stored in the STEM module.

[0056] Step 3.5: Perform network prediction to obtain the segmentation result at time t+2.

[0057] The subsequent video frames are processed in the same way, and finally the segmentation result of the entire video is obtained.

[0058] In one embodiment, see Figure 1 , Figure 1 This is a schematic diagram of the structure of a medical video breast lesion feature rapid segmentation device proposed by the present invention. Figure 1In the figure, Stem represents the Stem module, DownSamp in the Stem module represents the third downsampling module, MaxPooling represents the maximum pooling module, and TempLP represents the TempLP module. DownSamp.1 represents the first downsampling module, DownSamp.2 represents the second downsampling module, UpSamp.1 represents the first upsampling module, UpSamp.2 represents the second upsampling module, UpSamp.3 represents the third upsampling module, BasB l ock1 represents the first feature extraction module, BasB l ock2 represents the second feature extraction module, BasB l ock3 represents the third feature extraction module, and BasB l ock4 represents the fourth feature extraction module.

[0059] Furthermore, the feature extraction modules (i.e., the first feature extraction module, the second feature extraction module, the third feature extraction module, and the fourth feature extraction module) include an alignment module, a local context integrator, and an adaptive semantic enhancer. Figure 1 In the figure, ALI is the alignment module, LC I is the local context integrator, and ASE is the adaptive semantic enhancer. DW Conv is the deep convolution layer, PW Conv is the point-by-point convolution layer, Batch Norm refers to batch normalization, GELU refers to the GELU activation function, and Swi sh refers to the Swi sh activation function.

[0060] See also Figure 2 , Figure 2 It is a flowchart of a method for rapid segmentation of breast lesion features in medical videos provided by an embodiment of the present invention. The method is applied to the feature extraction modules in the device for rapid segmentation of breast lesion features in medical videos, specifically the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module. The alignment module, the local context integrator and the adaptive semantic enhancer in the feature extraction module are used to perform the following steps S1-S3 respectively. Figure 2 As shown, the method comprises the following steps:

[0061] S1, receiving a first feature map, and performing feature alignment on the first feature map to obtain a second feature map, wherein the first feature map is obtained based on frame image acquisition in an ultrasound video.

[0062] In a specific implementation, a first feature map is received, and the first feature map is obtained based on frame image acquisition in the ultrasound video. Specifically, the first feature map can be output by a previous module of the feature extraction module, for example, for the first feature extraction module, its corresponding first feature map is output by the Stem module.

[0063] The first feature map is feature aligned to obtain a second feature map. After feature alignment, the collected features can be made smoother, thereby improving the accuracy of subsequent breast lesion segmentation.

[0064] In one embodiment, the above step of “performing feature alignment on the first feature map to obtain the second feature map” specifically includes:

[0065] S11: Perform a convolution operation on the first feature map through a preset second depth convolution layer to obtain a fifth feature map.

[0066] In a specific implementation, the first feature map is convolved by a preset second depth convolution layer to obtain a fifth feature map. In a specific operation, when the convolution operation is performed by the second depth convolution layer, the size of the convolution kernel is set to 3 and the padding size is set to 1, so that the dimension of the fifth feature map collected is the same as that of the first feature map, that is, the number of features is the same.

[0067] S12: Superimpose features of the fifth feature map and the first feature map to obtain the second feature map.

[0068] In a specific implementation, the fifth feature map is feature-superimposed on the first feature map to obtain the second feature map. The fifth feature map is obtained by convolving the first feature map through the second deep convolution layer. The purpose of feature alignment can be achieved by superimposing the fifth feature map with the first feature map, so that the features of the obtained second feature map are smoother, thereby improving the accuracy of subsequent breast lesion segmentation.

[0069] S2: Based on the preset first depth convolution layer and the first point-by-point convolution layer, local context features are collected on the second feature map to obtain a third feature map.

[0070] In a specific implementation, a first deep convolutional layer and a first point-by-point convolutional layer are combined to collect local context features of the second feature map to obtain a third feature map, which contains more detailed local context features, thereby improving the accuracy of subsequent breast lesion segmentation.

[0071] Specifically, the first depth convolution layer performs a separate convolution operation on each channel of the second feature map, while the first point-by-point convolution layer combines linear features while maintaining the original spatial dimension. The first depth convolution layer and the first point-by-point convolution layer are batch normalized using the BatchNorm function to improve stability and performance. The first depth convolution layer uses a filter size of 3x3 and a padding size of 1 to maintain the spatial resolution of the input second feature map. Finally, residual connections are introduced to facilitate the flow of information and achieve effective feature learning and representation.

[0072] In one embodiment, the above step of “collecting local context features on the second feature map based on the preset first depth convolution layer and the first point-by-point convolution layer to obtain a third feature map” specifically includes:

[0073] S21, performing a convolution operation on the second feature map through the first deep convolution layer to obtain a sixth feature map.

[0074] In a specific implementation, before the second feature map is convolved by the first deep convolutional layer, the second feature map is batch normalized, for example, the batch normalization is performed using the BatchNorm function. The second feature map that has been batch normalized is input into the first deep convolutional layer, and a convolution operation is performed to obtain a sixth feature map. The first deep convolutional layer uses a filter size of 3x3, and a padding size of 1 is used to maintain the spatial resolution of the input second feature map. The dimension of the sixth feature map is the same as that of the second feature map, thereby ensuring the reliability of feature acquisition, avoiding excessive feature loss, and improving the accuracy of subsequent breast lesion segmentation.

[0075] S22: Perform a convolution operation on the sixth feature map through the first point-by-point convolution layer to obtain the third feature map.

[0076] In a specific implementation, before the sixth feature map is convolved by the first point-by-point convolution layer, the sixth feature map is batch normalized, for example, the batch normalization is performed using the BatchNorm function. The sixth feature map that has undergone batch normalization is input into the first point-by-point convolution layer, and the first point-by-point convolution layer performs a convolution operation on the sixth feature map to obtain the third feature map. The dimension of the third feature map is the same as that of the sixth feature map, thereby ensuring the reliability of feature acquisition, avoiding excessive feature loss, and improving the accuracy of subsequent breast lesion segmentation.

[0077] By combining the first depth convolution layer and the first point-by-point convolution layer, on the one hand, the amount of calculation can be reduced, and on the other hand, the accurate collection of local context features can be ensured.

[0078] S3: Perform feature enhancement on the third feature map based on a preset second point-by-point convolution layer and a third point-by-point convolution layer to obtain a fourth feature map.

[0079] In a specific implementation, the third feature map is enhanced by combining the second point-by-point convolution layer and the third point-by-point convolution layer to obtain the fourth feature map. The feature enhancement method can improve the accuracy of subsequent breast lesion segmentation.

[0080] Specifically, in one embodiment, the above step of “enhancing the third feature map based on the preset second point-by-point convolution layer and the third point-by-point convolution layer to obtain the fourth feature map” specifically includes:

[0081] S31, superimposing the third feature map and the second feature map to obtain a seventh feature map.

[0082] In a specific implementation, the third feature map is superimposed on the second feature map to obtain the seventh feature map. After superposition, the seventh feature map incorporates the features of the third feature map and the second feature map. When the features are subsequently enhanced, the features of the third feature map and the second feature map can be used simultaneously to avoid feature loss, thereby making feature acquisition more accurate and improving the accuracy of subsequent breast lesion segmentation.

[0083] S32, performing a feature enhancement convolution operation on the seventh feature map through the second point-by-point convolution layer to obtain an eighth feature map, wherein the dimension of the eighth feature map is greater than the dimension of the seventh feature map.

[0084] In a specific implementation, the second point-by-point convolution layer performs a feature enhancement convolution operation on the seventh feature map to obtain an eighth feature map. Through feature enhancement, the dimension of the eighth feature map can be made larger than the dimension of the seventh feature map, that is, more features can be collected. The specific implementation method is to adjust the number of convolution kernels of the second point-by-point convolution layer when performing the convolution operation. The number of convolution kernels in the second point-by-point convolution layer is greater than the number of channels of the frame image. For example, the number of channels of the frame image is C, and the number of convolution kernels in the second point-by-point convolution layer can be set to 4C, and C is usually 3. By increasing the number of convolution kernels, the number of features collected can be increased, so that the dimension of the eighth feature map can be made larger than the dimension of the seventh feature map.

[0085] S33, activating the eighth feature map through a preset activation function to obtain a ninth feature map.

[0086] In a specific implementation, the eighth feature map is activated by a preset activation function to obtain a ninth feature map. The activation function specifically includes a GELU activation function and a Swi sh activation function.

[0087] In the medical video segmentation task, the present invention tailors specific activation functions for semantic features at different levels. Shallow semantics focuses on low-level feature representations, such as the texture and edges of breast lesions, which are directly related to the original attributes of the input data. These features are learned and propagated through simple nonlinear functions, among which the GELU activation function is used. The GELU activation function has smooth nonlinear characteristics and effectively captures and propagates these low-level features (second downsampling / upsampling stage), helping the network to better learn and convey these subtleties. On the other hand, deep semantics involves the combination of multiple nonlinear transformations and input data to include more abstract, high-level features. These features may represent complex patterns, concepts, or semantic information, such as the structure and semantic category of breast lesions. In order to efficiently learn and represent these abstract high-level features, we choose the Swi sh activation function for deep semantics (third downsampling stage and first upsampling stage) because it has stronger nonlinearity and is well suited for learning and representing these abstract high-level attributes. The Swi sh activation function performs well in capturing and propagating these complex features, thereby enhancing the network's expressive power in deep semantic representation.

[0088] By using appropriate activation functions for different levels of semantics, such as GELU activation function and Swi sh activation function, the network can effectively learn and represent shallow and deep semantics, thereby improving the network's expressiveness, generalization ability, and the overall effect and accuracy of medical video segmentation tasks.

[0089] S34, performing a feature reduction convolution operation on the ninth feature map through the third point-by-point convolution layer to obtain the fourth feature map, wherein the dimension of the fourth feature map is equal to the dimension of the seventh feature map.

[0090] In a specific implementation, the third point-by-point convolution layer performs a convolution operation to reduce the features of the ninth feature map, so as to obtain the fourth feature map. By reducing the features, the dimension of the fourth feature map can be equal to the dimension of the seventh feature map, that is, the number of features is reduced, thereby reducing the amount of subsequent calculations. The specific implementation method is to adjust the number of convolution kernels of the third point-by-point convolution layer when performing the convolution operation. The number of convolution kernels in the third point-by-point convolution layer is equal to the number of channels of the frame image. For example, the number of channels of the frame image is C, and the number of convolution kernels in the third point-by-point convolution layer can be set to C, and C is usually 3. By reducing the number of convolution kernels, the number of features collected can be reduced, so that the dimension of the fourth feature map is equal to the dimension of the seventh feature map.

[0091] The technical solution of an embodiment of the present invention receives a first feature map, performs feature alignment on the first feature map to obtain a second feature map, wherein the first feature map is obtained based on frame image acquisition in an ultrasound video; performs local context feature acquisition on the second feature map based on a preset first depth convolution layer and a first point-by-point convolution layer to obtain a third feature map; performs feature enhancement on the third feature map based on a preset second point-by-point convolution layer and a third point-by-point convolution layer to obtain a fourth feature map. On the one hand, based on the deep convolution layer and the point-by-point convolution layer, the amount of data during feature collection can be reduced, thereby improving the reasoning speed. At the same time, the applicant has found that after feature alignment, the local context features of the second feature map are collected by combining the first deep convolution layer and the first point-by-point convolution layer; further, the third feature map is enhanced by combining the second point-by-point convolution layer and the third point-by-point convolution layer, which can significantly improve the accuracy of recognition. After measurement, by applying the technical solution of the embodiment of the present invention, the Di ce coefficient reaches 83.1, the Jaccard coefficient reaches 74.28, the precision rate reaches 85.67%, the recall rate reaches 87.19, and the FPS can reach more than 200, thereby achieving high processing efficiency while ensuring high accuracy. In summary, the present invention can achieve fast and high-precision segmentation in terminals with limited computing resources, with a wider application scenario and lower computing power cost.

[0092] It should be noted that the Di ce coefficient is used to measure the similarity between two samples, and is particularly used to evaluate the quality of image segmentation. The Jaccard coefficient, also known as the intersection-over-union ratio, is used to measure the overlap between two sets and is usually used to measure sample similarity. Precision refers to the ratio of correctly predicted positive examples to the total number of predicted positive examples, and is often used to evaluate the performance of classification tasks. Recall refers to the ratio of correctly predicted positive examples to the total number of actual positive examples, and is used to evaluate the model's ability to capture positive examples. FPS (Frames Per Second) is used to measure the processing speed of video or image processing tasks.

[0093] The above-mentioned breast lesion segmentation device can be implemented in the form of a computer program. The computer program can be used in Figure 3 Runs on the computer device shown.

[0094] See also Figure 3 , Figure 35 is a schematic block diagram of a computer device provided in an embodiment of the present application. The computer device 500 may be a terminal or a server, wherein the terminal may be an electronic device with communication functions such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The server may be an independent server or a server cluster composed of multiple servers.

[0095] The computer device 500 includes a processor 502 , a memory, and a network interface 505 connected via a system bus 501 , wherein the memory may include a non-volatile storage medium 503 and an internal memory 504 .

[0096] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, the processor 502 can execute a method for fast segmentation of breast lesion features in medical videos.

[0097] The processor 502 is used to provide computing and control capabilities to support the operation of the entire computer device 500 .

[0098] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a method for rapid segmentation of breast lesion features in medical videos.

[0099] The network interface 505 is used to communicate with other devices over the network. Those skilled in the art will appreciate that the above structure is only a block diagram of a portion of the structure related to the present application solution, and does not constitute a limitation on the computer device 500 to which the present application solution is applied. The specific computer device 500 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0100] The processor 502 is used to run the computer program 5032 stored in the memory to implement the steps of a method for rapid segmentation of breast lesion features in medical videos proposed in any of the above embodiments.

[0101] It should be understood that in the embodiment of the present application, the processor 502 may be a central processing unit (CPU), and the processor 502 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0102] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above embodiment can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiment of the above method.

[0103] Therefore, the present invention also provides a storage medium. The storage medium may be a computer-readable storage medium. The storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of a method for rapid segmentation of breast lesion features in a medical video proposed in any of the above embodiments.

[0104] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk, or an optical disk, etc., which can store program codes. The computer-readable storage medium can be non-volatile or volatile.

[0105] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0106] In the several embodiments provided by the present invention, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of each unit is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0107] The steps in the method of the embodiment of the present invention can be adjusted in order, combined and deleted according to actual needs. The units in the device of the embodiment of the present invention can be combined, divided and deleted according to actual needs. In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0108] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention.

[0109] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0110] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

[0111] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for rapid segmentation of breast lesion features in medical videos, characterized in that: The medical video breast lesion feature fast segmentation device comprises an encoder and a decoder connected to each other, wherein the encoder comprises a Stem module, a first downsampling module, a first feature extraction module, a second downsampling module and a second feature extraction module connected in sequence; the decoder comprises a first upsampling module, a third feature extraction module, a second upsampling module, a fourth feature extraction module and a third upsampling module connected in sequence, the method is applied to the feature extraction module, the feature extraction module comprises the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module, and the method comprises: Receiving a first feature map, and performing feature alignment on the first feature map to obtain a second feature map, wherein the first feature map is obtained based on frame image acquisition in an ultrasound video; Based on the preset first depth convolution layer and the first point-by-point convolution layer, local context features are collected on the second feature map to obtain a third feature map; Performing feature enhancement on the third feature map based on a preset second point-by-point convolution layer and a third point-by-point convolution layer to obtain a fourth feature map; The step of performing feature enhancement on the third feature map based on the preset second point-by-point convolution layer and the third point-by-point convolution layer to obtain a fourth feature map includes: Superimposing the third feature map and the second feature map to obtain a seventh feature map; Performing a feature-enhanced convolution operation on the seventh feature map through the second point-by-point convolution layer to obtain an eighth feature map, wherein a dimension of the eighth feature map is greater than a dimension of the seventh feature map; Activating the eighth feature map by a preset activation function to obtain a ninth feature map; Performing a feature-reducing convolution operation on the ninth feature map through the third point-by-point convolution layer to obtain the fourth feature map, wherein the dimension of the fourth feature map is equal to the dimension of the seventh feature map.

2. The method for rapid segmentation of breast lesion features in medical videos according to claim 1, characterized in that: The performing feature alignment on the first feature map to obtain a second feature map includes: Performing a convolution operation on the first feature map through a preset second depth convolution layer to obtain a fifth feature map; The fifth feature map is superimposed on the first feature map to obtain the second feature map.

3. The method for rapid segmentation of breast lesion features in medical video according to claim 1, characterized in that: The collecting local context features of the second feature map based on the preset first depth convolution layer and the first point-by-point convolution layer to obtain a third feature map includes: Performing a convolution operation on the second feature map through the first depth convolution layer to obtain a sixth feature map; Performing a convolution operation on the sixth feature map through the first point-by-point convolution layer to obtain the third feature map.

4. The method for rapid segmentation of breast lesion features in medical videos according to claim 3, characterized in that: The method further includes: collecting local context features of the second feature map based on the preset first depth convolution layer and the first point-by-point convolution layer to obtain a third feature map; Before performing the convolution operation on the second feature map through the first deep convolution layer, performing batch normalization processing on the second feature map; Before performing the convolution operation on the sixth feature map through the first point-by-point convolution layer, batch normalization is performed on the sixth feature map.

5. The method for rapid segmentation of breast lesion features in medical videos according to claim 1, characterized in that: The number of convolution kernels in the second point-by-point convolution layer is greater than the number of channels of the frame image; and the number of convolution kernels in the third point-by-point convolution layer is equal to the number of channels of the frame image.

6. A medical video breast lesion feature rapid segmentation device, characterized in that: The medical video breast lesion feature rapid segmentation device includes an encoder and a decoder connected to each other, the encoder includes a Stem module, a first down-sampling module, a first feature extraction module, a second down-sampling module and a second feature extraction module connected in sequence; the decoder includes a first up-sampling module, a third feature extraction module, a second up-sampling module, a fourth feature extraction module and a third up-sampling module connected in sequence, wherein the feature extraction module is used to execute the method described in any one of claims 1-5, and the feature extraction module includes the first feature extraction module, the second feature extraction module, the third feature extraction module and the fourth feature extraction module.

7. The medical video breast lesion feature rapid segmentation device according to claim 6, characterized in that: The Stem module includes a third downsampling module, a maximum pooling module and a TempLP module. The third downsampling module is used to perform a downsampling operation on the frame image currently inputted by the ultrasound video; the maximum pooling module is used to perform a maximum pooling operation on the output data of the third downsampling module to obtain the features of the currently inputted frame image; the TempLP module is used to fuse the features of the currently inputted frame image with the features of the frame image before the currently inputted frame image, and output them to the first downsampling module.

8. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the method according to any one of claims 1 to 5 when executing the computer program.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 5 can be implemented.

Images