Pneumothorax prediction system and method, electronic equipment and medium
By designing a pneumothoracic prediction system including data acquisition, vector cutting, spatial coding, time coding and classification head, the pneumothoracic prediction method lacks time dimension modeling and low processing efficiency in the prior art, and efficient pneumothoracic prediction is achieved.
Patent Information
- Application Number
- CN202411619025.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-13
- Publication Date
- 2025-03-07
AI Technical Summary
Existing pneumothoracic prediction methods lack the time-dimensional modeling ability, have inefficient processing, and rely on manual analysis, with limited efficiency and accuracy.
A pneumothoracic prediction system is designed to obtain video detection data of chest ultrasound images through the data acquisition module. The vector acquisition module cuts the video frame into feature vectors. The spatial encoding module and the time encoding module perform spatial encoding and time encoding respectively to generate timing features, and finally use the classification head to perform pneumothoracic prediction.
The system can effectively retain the time information of video detection data, improve the efficiency of pneumothorax prediction, realize the automatic prediction of pneumothorax, and reduce manual operation errors.
Smart Images
Figure CN119580147A_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of medical imaging and computer-aided technology, and relates to a pneumothorax prediction system, and in particular to a pneumothorax prediction system, method, electronic equipment and medium. Background Art
[0002] Pneumothorax is a common chest disease, usually caused by air leaking from the lungs into the chest cavity, causing lung collapse, which may cause breathing difficulties or even endanger life. Traditional methods for diagnosing pneumothorax mainly rely on static images, such as X-rays or CT scans. Although these methods are effective, they have certain limitations in dynamic monitoring and rapid diagnosis. In recent years, ultrasound imaging has been increasingly used in the diagnosis of pneumothorax due to its advantages of non-invasive, real-time and dynamic observation. However, the diagnostic process of ultrasound imaging usually requires expert doctors to analyze each frame of the image one by one, which not only consumes a lot of time, but is also limited by the doctor's experience and judgment. In order to improve the efficiency and accuracy of diagnosis, researchers have begun to explore the use of computer vision and machine learning technologies to automatically analyze ultrasound video data. In this regard, models based on deep learning have shown significant potential. In particular, the Transformer model has unique advantages in processing sequence data.
[0003] However, although convolutional neural networks (CNNs) perform well in image classification and object detection, when processing video data, complex network architectures are usually required to capture time series information. Summary of the invention
[0004] In view of the above-mentioned shortcomings of the prior art, the purpose of the present application is to provide a pneumothorax prediction system, method, electronic device and medium to solve the problems of the pneumothorax prediction method in the prior art lacking modeling capability in the time dimension and low processing efficiency.
[0005] In a first aspect, the present application provides a pneumothorax prediction system, which includes: a data acquisition module for acquiring video detection data of chest ultrasound images; a vector acquisition module for cutting and processing video frames of the video detection data to obtain feature vectors corresponding to the video frames; a spatial encoding module for using a spatial encoder to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features; a temporal encoding module for using a temporal encoder to perform temporal encoding processing on the spatial features to obtain time series features; and a result acquisition module for using a classification head to predict the time series features to obtain a pneumothorax prediction result.
[0006] In the present application, the vector acquisition module is used to cut the video frames of the video detection data to obtain the feature vectors corresponding to the video frames. The spatial coding module and the temporal coding module are used to perform spatial coding and temporal coding processing in sequence using the spatial encoder and the temporal encoder to obtain the time series features. The result acquisition module is used to obtain the pneumothorax prediction results of the time series features using the classification head. This pneumothorax prediction system can retain the time information of the video detection data, directly perform prediction analysis on the video detection data of the chest ultrasound image, and improve the efficiency of pneumothorax prediction.
[0007] In an implementation of the first aspect, cutting and processing the video frame of the video detection data to obtain a feature vector corresponding to the video frame includes: processing the video detection data to obtain continuous frames and the number of continuous frames of the video detection data; cutting the video detection data and using the number of continuous frames to obtain the number of cuts in any dimension; mapping the video detection data to another dimension and performing position encoding to obtain a feature vector corresponding to the video frame.
[0008] In an implementation of the first aspect, cutting the video detection data includes: dividing the video detection data to obtain multi-dimensional continuous frames; and cutting each frame of the multi-dimensional continuous frames to obtain a plurality of three-dimensional cut blocks.
[0009] In an implementation of the first aspect, a spatial encoder is used to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features, including: classifying the feature vectors corresponding to the video frames by time to obtain the feature vectors at each moment; and using the spatial encoder to perform spatial encoding processing on the feature vectors at the same moment to obtain the spatial features at any moment.
[0010] In an implementation of the first aspect, performing time encoding processing on the spatial features using a time encoder to obtain a time series feature includes: performing time encoding processing on the spatial features using the same time encoder to obtain the time series feature.
[0011] In an implementation of the first aspect, the deep learning model includes the temporal encoder, the spatial encoder and the classification head, and training the deep learning model includes: acquiring a video detection data set of chest ultrasound images; performing three-dimensional filtering processing on the video detection data set using a central frame initialization strategy to obtain a video detection training data set; and training the deep learning model using the video detection training data set.
[0012] In this implementation, a central frame initialization strategy is used to perform three-dimensional filtering on the video detection data set, which reduces the feature dimension, reduces the complexity of model calculation, and improves the speed of pneumothorax prediction.
[0013] In an implementation of the first aspect, the spatial encoder and the temporal encoder include a self-attention mechanism, a residual connection, and a feedforward neural network, the spatial encoder is used to perform self-attention calculations on the feature vectors corresponding to the input video frames, and the temporal encoder is used to perform self-attention calculations on the input spatial features.
[0014] In a second aspect, the present application provides a pneumothorax prediction method, which includes: obtaining video detection data of chest ultrasound images; cutting video frames of the video detection data to obtain feature vectors corresponding to the video frames; using a spatial encoder to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features; using a temporal encoder to perform temporal encoding processing on the spatial features to obtain time series features; and using a classification head to predict the time series features to obtain a pneumothorax prediction result.
[0015] In a third aspect, the present application provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program stored in the memory, so that the electronic device performs a pneumothorax prediction method as described in any one of the first aspects.
[0016] In a fourth aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the pneumothorax prediction method described in any one of the first aspects. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1A Shown is a schematic diagram of an application scenario of the pneumothorax prediction method described in this application.
[0018] Figure 1B Shown are structural diagrams of the client-cloud interaction scenarios in these implementations.
[0019] Figure 2 Shown is a schematic flow chart of the pneumothorax prediction method described in an embodiment of the present application.
[0020] Figure 3 Shown is a schematic flow chart of the pneumothorax prediction method described in an embodiment of the present application.
[0021] Figure 4 Shown is a schematic flow chart of the pneumothorax prediction method described in an embodiment of the present application.
[0022] Figure 5Shown is a schematic flow chart of the pneumothorax prediction method described in an embodiment of the present application.
[0023] Figure 6 Shown is a schematic flow chart of the pneumothorax prediction method described in an embodiment of the present application.
[0024] Figure 7 Shown is a schematic diagram of the structure of the pneumothorax prediction system described in an embodiment of the present application.
[0025] Figure 8 Shown is a schematic diagram of the structure of an electronic device described in an embodiment of the present application.
[0026] Component number description
[0027] 1 Pneumothorax prediction device
[0028] 11 Video detection equipment
[0029] 12 Local Processor
[0030] 13 Display Terminal
[0031] 2-end-cloud interactive system
[0032] 20 Terminal
[0033] 21 Cloud Server
[0034] 100 Pneumothorax Prediction System
[0035] 110 Data acquisition module
[0036] 120 Vector acquisition module
[0037] 130 Spatial Coding Module
[0038] 140 Time coding module
[0039] 150 Result Acquisition Module
[0040] 800 Electronic equipment
[0041] 810 Memory
[0042] 820 processor
[0043] 830 Display
[0044] Steps S11 to S15
[0045] Steps S121 to S123
[0046] Steps S1221 to S1222
[0047] Steps S131 to S132
[0048] Steps S21 to S23 DETAILED DESCRIPTION
[0049] The following describes the embodiments of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. The present application can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.
[0050] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present application, and thus the drawings only show components related to the present application rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed at will, and the component layout may also be more complicated.
[0051] In the embodiments of the present application, the words "optionally" or "for example" represent examples, illustrations or descriptions. Any embodiment or design described as "optionally" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "optionally" or "for example" is intended to present related concepts in a specific way.
[0052] In the embodiments of the present application, "at least one" refers to one or more, and "plurality" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can represent: a, b, c, ab, ac, bc or abc, where a, b, c can be single or multiple.
[0053] Although there are many advanced technologies and methods in the field of pneumothorax diagnosis, the existing technologies still face the following major problems in practical applications: First, traditional pneumothorax diagnosis methods, such as X-rays and CT scans, usually require multiple steps such as image acquisition, processing, analysis and interpretation. These steps are not only time-consuming, but also rely on the doctor's experience and judgment, resulting in low efficiency and susceptibility to subjective factors in the diagnostic process. Second, X-rays and CT scans usually provide static images and cannot capture the dynamic changes of pneumothorax in the time dimension. This limits the monitoring of the development process of pneumothorax and may also lead to missed or misdiagnosis of early pneumothorax. Third, most current methods still require manual intervention or preprocessing steps to locate the location of pneumothorax lesions or extract features. These preprocessing steps not only increase the complexity of the system, but also limit the automation level and adaptability of the system. Fourth, 3D CNN uses a three-dimensional convolution kernel to directly convolve in the spatial and temporal dimensions, while capturing temporal and spatial information in the video. However, 3D CNN faces greater challenges when processing longer videos. The model needs to process all frames in the entire video, so in long videos, the computing resource consumption of 3D CNN increases significantly, and a large amount of memory and computing power is required. This not only prolongs the training and inference time, but also causes the model to be less efficient when processing very long video sequences. Finally, Vision Transformer (ViT) is a classic model of the Transformer structure in the field of image processing. However, ViT is essentially designed for image processing and lacks native modeling capabilities for the time dimension, so it encounters significant drawbacks when processing videos.
[0054] At least to address the above-mentioned problems, embodiments of the present application provide a pneumothorax prediction method, system, electronic device and medium, wherein the pneumothorax prediction method comprises: acquiring video detection data of chest ultrasound images; cutting and processing video frames of the video detection data to obtain feature vectors corresponding to the video frames; using a spatial encoder to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features; using a temporal encoder to perform temporal encoding processing on the spatial features to obtain time series features; and using a classification head to predict the time series features to obtain a pneumothorax prediction result.
[0055] In the embodiment of the present application, the video frame of the video detection data is cut and processed to obtain the feature vector corresponding to the video frame. The spatial encoder and the temporal encoder are used to perform spatial encoding and temporal encoding processing in sequence to obtain the time series features. The pneumothorax prediction result of the time series features is obtained using a classification head. This pneumothorax prediction method can retain the time information of the video detection data, directly perform prediction analysis on the video detection data of the chest ultrasound image, and improve the efficiency of pneumothorax prediction.
[0056] Figure 1AThe pneumothorax prediction device 1 can be used to implement the pneumothorax prediction method provided in the embodiment of the present application, but the application scenario of the pneumothorax prediction method provided in the embodiment of the present application is not limited to Figure 1A The pneumothorax prediction device 1 shown. Figure 1A As shown, the pneumothorax prediction device 1 includes a video detection device 11, a local processor 12 and a display terminal 13. The pneumothorax prediction method provided in the embodiment of the present application can be applied to the local processor 12.
[0057] in, Figure 1A The local processor 12 in the embodiment may be a local processor or a local processor cluster composed of multiple local processors or a cloud computing center, etc., which is not limited to the specifics herein. Figure 1A Only one video detection device 11, one local processor 12 and one display terminal 13 are shown, but it should be understood that Figure 1A The examples are only used to understand the present solution, and the specific numbers of local processors 12 and display terminals 13 should be flexibly determined based on actual conditions.
[0058] In some other implementations, the pneumothorax prediction device 1 may not include the display terminal 13, but only include a local processor 12 with a display function and a video detection device 11. The pneumothorax prediction method provided in the embodiment of the present application can be applied to the local processor 12. The local processor 12 with a display function may include a tablet computer, a laptop computer, a PDA, a mobile phone, a personal computer (PC) and a voice interaction device, or a monitoring device, a face recognition device, etc., which is not limited here.
[0059] In still other implementations, the pneumothorax prediction method described in the present application may be applied to a terminal-cloud interaction scenario. Figure 1B The following is a schematic diagram of the structure of the client-cloud interaction scenario in these implementations. Figure 1B As shown, the terminal-cloud interaction system 2 includes a terminal 20 and a cloud server 21. The terminal 20 and the cloud server 21 can communicate with each other, and the communication method is not limited to wired or wireless.
[0060] Among them, the terminal 20 can be mobile or fixed, for example, the terminal 20 can be a wireless terminal or a wired terminal. The wireless terminal can refer to a device with wireless transceiver function, which can be deployed indoors, outdoors and in industrial workshops. The terminal 20 can be a mobile phone, a tablet computer, a laptop computer, etc., which is not limited here. The cloud server 21 can include one or more servers, or one or more processing nodes, or one or more virtual machines running on the server. The cloud server 21 can also be called a server cluster, a management platform, a data processing center, etc., which is not limited in the embodiments of the present application.
[0061] The technical solutions in the embodiments of the present application will be described in detail below in conjunction with the drawings in the embodiments of the present application.
[0062] The following embodiments of the present application provide a method for predicting pneumothorax, for example, by Figure 1A The local processor 12 shown or Figure 1B It is implemented by the cloud server 21 shown. Figure 2 The flowchart of the pneumothorax prediction method described in the embodiment of the present application is shown as follows: Figure 2 As shown, the pneumothorax prediction method includes steps S11 to S15.
[0063] Step S11, obtaining video detection data of chest ultrasound images.
[0064] Optionally, video detection data of chest ultrasound images are collected, wherein the video detection data of chest ultrasound images includes ultrasound examination video data in different time periods and under different conditions.
[0065] Step S12: cutting the video frame of the video detection data to obtain a feature vector corresponding to the video frame.
[0066] Optionally, a cutting process is used to convert the video detection data into blocks suitable for transformer processing.
[0067] Step S13, using a spatial encoder to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features.
[0068] Optionally, the spatial encoder is a transformer encoder. The spatial feature is an output result of a feature vector corresponding to the video frame after being processed by the spatial encoder.
[0069] Step S14, using a time encoder to perform time encoding processing on the spatial features to obtain time series features.
[0070] Optionally, the temporal encoder is a transformer encoder. The temporal features are output results of the temporal encoder encoding and splicing the spatial features.
[0071] Step S15, using the classification head to predict the time series features to obtain a pneumothorax prediction result.
[0072] Optionally, the classification head is an MLP multi-layer perceptron, and the time series feature T is sent to the classification head for prediction to obtain the probability of pneumothorax. The obtained pneumothorax prediction result is expressed as:
[0073] Y = MLP(T).
[0074] In some possible implementations, video detection data of chest ultrasound images are collected, and the video detection data of chest ultrasound images includes ultrasound examination video data in different time periods and under different conditions. The video frames of the video detection data are cut to obtain feature vectors corresponding to the video frames. The feature vectors corresponding to each of the video frames are spatially encoded using a spatial encoder to obtain spatial features. The spatial features are temporally encoded using a temporal encoder to obtain time series features. The time series features are predicted using a classification head to obtain a pneumothorax prediction result.
[0075] In an embodiment of the present application, the video frames of the video detection data are cut and processed to obtain feature vectors corresponding to the video frames. The spatial encoder and the temporal encoder are used to construct a deep learning model, and the deep learning model is a ViViT (Video vision transformer) model. The spatial encoder and the temporal encoder are used to perform spatial encoding and temporal encoding processing in sequence to obtain time series features. The pneumothorax prediction result of the time series features is obtained using a classification head. This pneumothorax prediction method can retain the time information of the video detection data, directly perform prediction and analysis on the video detection data of the chest ultrasound image, improve the efficiency of pneumothorax prediction, realize the automatic prediction of pneumothorax, and reduce manual operation errors.
[0076] Figure 3 The flowchart of the pneumothorax prediction method described in the embodiment of the present application is shown as follows: Figure 3 As shown, the step S12 includes steps S121 to S123.
[0077] Step S121, processing the video detection data to obtain continuous frames and the number of continuous frames of the video detection data.
[0078] Step S122, cutting the video detection data, and obtaining the number of cuts in any dimension using the number of consecutive frames.
[0079] Step S123: Map the video detection data to another dimension and perform position encoding to obtain a feature vector corresponding to the video frame.
[0080] In some possible implementations, the video detection data is processed to obtain the video detection data including T frames, each frame having an image dimension of H×W×C, that is, height×width×channel, and the input video detection data The video detection data is divided into several pipelines, each pipeline includes T consecutive frames. Any frame image in the T consecutive frames of the video detection data is further divided into blocks of fixed size. The block of any pipeline is represented as The number of cuts in any dimension is expressed as:
[0081]
[0082] Among them, nt, nh and nw are the number of cuts in the three dimensions of frame number, height and width respectively.
[0083] The video detection data x i The linear layer is sent to map to another dimension d and position encoded p to obtain a feature vector corresponding to the video frame. The feature vector is expressed as:
[0084] z=[z cls ,E x1 ,E x2 ,...E xn ]+p,
[0085] Among them, p is the learnable position code, z cls is the vector of category encoding, and E is the encoding weight.
[0086] Figure 4 The flowchart of the pneumothorax prediction method described in the embodiment of the present application is shown as follows: Figure 4 As shown, the step S122 includes steps S1221 to S1222.
[0087] Step S1221, dividing the video detection data to obtain multi-dimensional continuous frames.
[0088] Step S1222: cutting each frame of the multi-dimensional continuous frames to obtain a plurality of three-dimensional cutting blocks.
[0089] In some possible implementations, the video detection data is divided into a number of pipelines, each pipeline includes a certain number of continuous frames, and each of the multidimensional continuous frames is cut into patches of a fixed size. The patch blocks suitable for transformer processing obtained in this way retain the temporal and spatial information in the video detection data.
[0090] Figure 5 The flowchart of the pneumothorax prediction method described in the embodiment of the present application is shown as follows: Figure 5 As shown, the step S13 includes steps S131 to S132.
[0091] Step S131, classifying the feature vectors corresponding to the video frames by time, and obtaining the feature vectors at each moment.
[0092] Step S132: perform spatial encoding processing on the feature vectors at the same time by using the spatial encoder to obtain the spatial features at any time.
[0093] In some possible implementations, the spatial encoder only performs interactive feature extraction on feature vectors with the same time index, that is, the same number of frames. Assuming that the feature vectors corresponding to the input video frames include T frames, T spatial encoders are used to encode the feature vectors at each moment. The spatial encoder is used to perform spatial encoding processing on the feature vectors at each moment to obtain the spatial features at each moment. The calculation formula for obtaining the spatial features at each moment is:
[0094] S i =S_Encoder i (z i ),i∈(1,T),
[0095] Among them, S i is the output of the i-th spatial encoder, S_Encoder is the spatial encoder, z i is the feature vector of the image at the i-th moment.
[0096] In the embodiment of the present application, a spatial encoder is used to process spatial dimension information, which further reduces the computational cost and improves the processing speed.
[0097] In one embodiment of the present application, performing time encoding processing on the spatial features using a time encoder to obtain a temporal feature includes: performing time encoding processing on the spatial features using the same time encoder to obtain the temporal feature.
[0098] In some possible implementations, for the spatial feature S obtained by using T spatial encoders i, and then splice them and send them to a time encoder. Use a time encoder to perform time encoding processing on each of the spatial features to obtain the time series features. The calculation formula of the time series features is:
[0099] T=T_Encoder(concat(S i )),
[0100] Wherein, T is the output of the time encoder, T_Encoder is the time encoder, and concat is the concatenation process.
[0101] In the embodiment of the present application, a time encoder is used to process time dimension information, which further reduces the computing cost and improves the processing speed.
[0102] Figure 6 The flowchart of the pneumothorax prediction method described in the embodiment of the present application is shown as follows: Figure 6 As shown, the deep learning model includes the temporal encoder, the spatial encoder and the classification head, and training the deep learning model includes steps S21 to S23.
[0103] Step S21, obtaining a video detection data set of chest ultrasound images.
[0104] Optionally, video detection data of chest ultrasound images are collected, and the video detection data of chest ultrasound images includes ultrasound examination video data in different time periods and under different conditions. A video detection data set of chest ultrasound images is constructed based on the video detection data of the chest ultrasound images, and the constructed video detection data set is annotated at the video level, that is, whether the entire video detection data contains pneumothorax is annotated. During the annotation process, in order to ensure high quality and high credibility of the annotation, the annotated data is strictly reviewed and proofread to ensure the accuracy and consistency of the annotation.
[0105] Step S22: Perform three-dimensional filtering on the video detection data set using a central frame initialization strategy to obtain a video detection training data set.
[0106] Step S23: training the deep learning model using the video detection training data set.
[0107] In some possible implementations, in order to utilize the VIT pre-trained weights, a 3D filter is used to generate training vectors using a central frame initialization strategy. Compared to traditional filters that use transformation matrices, 3D filters use a central frame initialization strategy to generate training vectors. The transformation matrix is retained at the positions, and the remaining positions are initialized to 0 to obtain a video detection training data set. The deep learning model is trained using the video detection training data set. The training vector is expressed as:
[0108] E=[0,...E image ,...0],
[0109] Among them, E image is the linear transformation matrix.
[0110] In some other possible implementations, video detection data of chest ultrasound images are collected, and a validation data set is formed using the video detection data of chest ultrasound images, and the performance of the deep learning model is evaluated using the validation data set.
[0111] In the embodiment of the present application, the video detection data set is subjected to three-dimensional filtering processing using the central frame initialization strategy, and the training data set obtained in this way can be used to perform model training and fine-tuning using VIT and training weights as well as the video detection data, thereby reducing the feature dimension, reducing the complexity of model calculation, and improving the speed of pneumothorax prediction. The deep learning model can capture the dynamic changes of the video detection data, identify the characteristics of pneumothorax in the time dimension, and improve the ability to predict pneumothorax.
[0112] In one embodiment of the present application, the spatial encoder and the temporal encoder include a self-attention mechanism, a residual connection and a feedforward neural network, the spatial encoder is used to perform self-attention calculation on the feature vector corresponding to the input video frame, and the temporal encoder is used to perform self-attention calculation on the input spatial features.
[0113] In some possible implementations, assuming the input is Z, the self-attention calculation formula is:
[0114]
[0115] Among them, W K , W V and W Q They are used to generate linear transformation matrices for query, key, and value, respectively. is a scaling factor used to prevent the dot product value from being too large, A is the attention score matrix, and O is the output matrix.
[0116] After the residual connection, the self-attention output is expressed as:
[0117] O S =Z+O.
[0118] Then O S It is sent to the feedforward neural network, and the calculation formula of the feedforward neural network is:
[0119] O F =LayerNorm(MLP(O S )),
[0120] Z out =O F +O S
[0121] Among them, LayerNorm represents layer normalization, MLP is a multi-layer perceptron, O F is the intermediate output, Z out is the final output.
[0122] Figure 7 The structure diagram of the pneumothorax prediction system described in the embodiment of the present application is shown as follows: Figure 7 As shown, the pneumothorax prediction system 100 includes a data acquisition module 110 , a vector acquisition module 120 , a space encoding module 130 , a time encoding module 140 and a result acquisition module 150 .
[0123] The data acquisition module 110 is used to acquire video detection data of chest ultrasound images.
[0124] The vector acquisition module 120 is used to perform segmentation processing on the video frame of the video detection data to obtain a feature vector corresponding to the video frame.
[0125] The spatial encoding module 130 is used to perform spatial encoding processing on the feature vectors corresponding to each of the video frames using a spatial encoder to obtain spatial features.
[0126] The time encoding module 140 is used to perform time encoding processing on the spatial features using a time encoder to obtain time series features.
[0127] The result acquisition module 150 is used to predict the time series features using the classification head to obtain a pneumothorax prediction result.
[0128] In some possible implementations, the data acquisition module 110 is used to collect video detection data of chest ultrasound images, and the video detection data of chest ultrasound images includes ultrasound examination video data under different time periods and different conditions. The vector acquisition module 120 is used to cut the video frames of the video detection data to obtain feature vectors corresponding to the video frames. The spatial encoding module 130 is used to perform spatial encoding processing on the feature vectors corresponding to each of the video frames using a spatial encoder to obtain spatial features. The time encoding module 140 is used to perform time encoding processing on the spatial features using a time encoder to obtain time series features. The result acquisition module 150 is used to predict the time series features using a classification head to obtain a pneumothorax prediction result.
[0129] In the embodiment of the present application, the vector acquisition module 120 is used to cut the video frame of the video detection data to obtain the feature vector corresponding to the video frame. The spatial encoding module 130 and the temporal encoding module 140 are used to use the spatial encoder and the temporal encoder to perform spatial encoding and temporal encoding processing in sequence to obtain time series features. The result acquisition module 150 is used to use the classification head to obtain the pneumothorax prediction result of the time series features. This pneumothorax prediction system 100 can retain the time information of the video detection data, directly perform prediction analysis on the video detection data of the chest ultrasound image, and improve the efficiency of pneumothorax prediction.
[0130] In the several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of modules / units is only a logical function division. There may be other division methods in actual implementation, such as multiple modules or units can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules or units, which can be electrical, mechanical or other forms.
[0131] The modules / units described as separate components may or may not be physically separated, and the components displayed as modules / units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules / units may be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in the various embodiments of the present application may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.
[0132] Those of ordinary skill in the art should further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0133] An embodiment of the present application also provides an electronic device. Figure 8 The structure diagram of the electronic device 800 described in the embodiment of the present application is shown. Figure 8 As shown, in this embodiment, the electronic device 800 includes a memory 810 and a processor 820 .
[0134] The memory 810 is used to store computer programs; preferably, the memory 810 includes: ROM, RAM, disk, USB flash drive, memory card or CD-ROM and other media that can store program codes.
[0135] Specifically, the memory 810 may include a computer system readable medium in the form of a volatile memory, such as a random access memory (RAM) and / or a cache memory. The electronic device 800 may further include other removable / non-removable, volatile / non-volatile computer system storage media. The memory 810 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present application.
[0136] The processor 820 is connected to the memory 810 and is used to execute the computer program stored in the memory 810 so that the electronic device 800 executes the pneumothorax prediction method described in any embodiment of the present application.
[0137] Optionally, the processor 820 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor 802 may be a microprocessor or any conventional processor, etc. In combination with the steps of the accessory optimization method provided in the embodiment of the present invention, it can be directly embodied as being executed by a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory, and the processor reads the information in the memory and completes the steps of the aforementioned method in combination with its hardware.
[0138] Optionally, the electronic device 800 in this embodiment may further include a display 830. The display 830 is communicatively connected with the memory 810 and the processor 820, and is used to display a graphical user interface (GUI) interaction interface related to the pneumothorax prediction method described in the embodiment of the present application.
[0139] The present application also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the pneumothorax prediction method described in any embodiment of the present application is implemented.
[0140] The descriptions of the processes or structures corresponding to the above-mentioned figures have different emphases. For parts that are not described in detail in a certain process or structure, please refer to the relevant descriptions of other processes or structures.
[0141] The above embodiments are merely illustrative of the principles and effects of the present application and are not intended to limit the present application. Anyone familiar with the technology may modify or change the above embodiments without violating the spirit and scope of the present application. Therefore, all equivalent modifications or changes made by a person of ordinary skill in the art without departing from the spirit and technical ideas disclosed in the present application shall still be covered by the claims of the present application.
Claims
1. A pneumothorax prediction system, characterized in that: include: A data acquisition module, used to acquire video detection data of chest ultrasound images; A vector acquisition module, used for cutting the video frame of the video detection data to obtain a feature vector corresponding to the video frame; A spatial coding module, used for performing spatial coding processing on the feature vectors corresponding to each of the video frames using a spatial encoder to obtain spatial features; A time coding module, used for performing time coding processing on the spatial features using a time encoder to obtain time series features; The result acquisition module is used to predict the time series features using the classification head to obtain a pneumothorax prediction result.
2. The pneumothorax prediction system according to claim 1, characterized in that: The video frame of the video detection data is cut and processed to obtain a feature vector corresponding to the video frame, including: Processing the video detection data to obtain continuous frames and the number of continuous frames of the video detection data; Cutting the video detection data, and obtaining the number of cuts in any dimension using the number of consecutive frames; The video detection data is mapped to another dimension and position-encoded to obtain a feature vector corresponding to the video frame.
3. The pneumothorax prediction system according to claim 2, characterized in that: Cutting the video detection data includes: Dividing the video detection data to obtain multi-dimensional continuous frames; Each of the multi-dimensional continuous frames is cut to obtain a plurality of three-dimensional cut blocks.
4. The pneumothorax prediction system according to claim 1, characterized in that: Using a spatial encoder to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features includes: Classifying the feature vectors corresponding to the video frames by time to obtain the feature vectors at each moment; The spatial encoder is used to perform spatial encoding processing on the feature vectors at the same time to obtain the spatial features at any time.
5. The pneumothorax prediction system according to claim 1, characterized in that: The spatial features are subjected to time encoding processing by using a time encoder to obtain time series features, including: The spatial features are time-encoded using the same time encoder to obtain the temporal features.
6. The pneumothorax prediction system according to claim 1, characterized in that: The deep learning model includes the temporal encoder, the spatial encoder and the classification head, and training the deep learning model includes: Obtain a video detection dataset of chest ultrasound images; Performing three-dimensional filtering processing on the video detection data set by using a central frame initialization strategy to obtain a video detection training data set; The deep learning model is trained using the video detection training data set.
7. The pneumothorax prediction system according to claim 1, characterized in that: The spatial encoder and the temporal encoder include a self-attention mechanism, a residual connection and a feedforward neural network. The spatial encoder is used to perform self-attention calculation on the feature vector corresponding to the input video frame, and the temporal encoder is used to perform self-attention calculation on the input spatial feature.
8. A method for predicting pneumothorax, characterized in that: include: Acquire video detection data of chest ultrasound images; Cutting the video frame of the video detection data to obtain a feature vector corresponding to the video frame; Using a spatial encoder to perform spatial encoding processing on the feature vectors corresponding to each of the video frames to obtain spatial features; Performing time encoding processing on the spatial features using a time encoder to obtain time series features; The classification head is used to predict the time series features to obtain a pneumothorax prediction result.
9. An electronic device, characterized in that: The electronic device comprises: Memory for storing computer programs; A processor, wherein the processor is used to execute the computer program stored in the memory so that the electronic device executes the pneumothorax prediction method as described in claim 8.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the pneumothorax prediction method according to claim 8 is implemented.