Pulmonary Embolism Recognition Device, Terminal Device, and Storage Medium Based on Parameter Sharing

Abstract

An embodiment of the present invention discloses a pulmonary embolism recognition device, a terminal device, and a storage medium based on parameter sharing. The device inputs a to-be-detected image into a pre-trained first feature extractor to obtain a first feature; inputs the first feature into a pre-trained first classification network to obtain the probability that the to-be-detected image contains a pulmonary embolism; inputs the to-be-detected image with a probability reaching a first threshold into a pre-trained region detection network to detect the position of the pulmonary embolism and intercept the corresponding region image; inputs the region image into a pre-trained second feature extractor to obtain a second feature; fuses the first feature and the second feature to obtain a mixed feature; inputs the mixed feature into a pre-trained second classification network and a third classification network respectively to confirm the nature type and position type of the pulmonary embolism. By combining the commonness and characteristics of the feature recognition results corresponding to different lesion information dimensions, the accuracy of the recognition and detection results is improved.

Description

Technical Field

[0001] Embodiments of the present invention relate to the technical field of image processing, and particularly to a pulmonary embolism recognition device, a terminal device, and a storage medium based on parameter sharing. Background Art

[0002] Pulmonary embolism is a disease caused by the obstruction of the pulmonary artery. When one feels tense and painful with each breath, it may indicate a serious or even life-threatening condition. Therefore, timely diagnosis and correct treatment can significantly reduce the mortality rate.

[0003] The clinical symptoms and signs of pulmonary embolism lack specificity, and it is prone to misdiagnosis and missed diagnosis in clinical practice. Moreover, doctors spend a lot of time and energy in diagnosing pulmonary embolism, so there is an easy situation of overdiagnosis by doctors. If machine learning can be used to assist in more accurate diagnosis and recognition, it can not only greatly reduce the energy of doctors, but also be more effective for the management and treatment of patients.

[0004] Currently, CT (Computed Tomography) pulmonary angiography is the most common type of medical image for evaluating patients with pulmonary embolism. A CT scan consists of hundreds of images, and these images need to be carefully examined to identify blood clots in the pulmonary artery. With the increasing use of imaging, the time limit of radiologists may lead to diagnostic delays. How to use chest CT pulmonary angiography images (hereinafter referred to as CT images) and machine learning technology to more accurately identify pulmonary embolism is an urgent problem to be solved. Especially to improve the diagnostic efficiency, more specific judgment results for pulmonary embolism are needed.

[0005] When the inventors verified the existing solutions for identifying pulmonary embolism in chest CT pulmonary angiography images through machine learning, they found that most of the solutions for predicting pulmonary embolism based on artificial intelligence only simply predict whether there is a suspected pulmonary embolism in chest CT pulmonary angiography images, but cannot provide in-depth and accurate information for pulmonary embolism recognition. Summary of the Invention

[0006] The present invention provides a pulmonary embolism recognition device, a terminal device, and a storage medium based on parameter sharing to solve the technical problem of insufficient in-depth and accurate information for pulmonary embolism recognition in the prior art.

[0007] In a first aspect, an embodiment of the present invention provides a pulmonary embolism recognition device based on parameter sharing, including:

[0008] A first extraction unit, configured to input a to-be-tested image into a pre-trained first feature extractor to obtain a first feature;

[0009] A probability calculation unit, configured to input the first feature into a pre-trained first classification network to obtain the probability that the image to be tested contains pulmonary embolism;

[0010] A region screenshot unit, configured to input the image to be tested whose probability reaches a first threshold into a pre-trained region detection network to detect the location of pulmonary embolism and intercept the corresponding region image;

[0011] A second feature extraction unit, configured to input the region image into a pre-trained second feature extractor to obtain a second feature;

[0012] A feature fusion unit, configured to fuse the first feature and the second feature to obtain a mixed feature;

[0013] A comprehensive judgment unit, configured to input the mixed feature into a pre-trained second classification network and a third classification network respectively to confirm the nature type and location type of pulmonary embolism.

[0014] In a second aspect, an embodiment of the present invention further provides a terminal device, including:

[0015] One or more processors;

[0016] A memory, configured to store one or more programs;

[0017] When the one or more programs are executed by the one or more processors, the one or more processors implement the pulmonary embolism recognition based on parameter sharing as described in the first aspect.

[0018] In a third aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, it implements the pulmonary embolism recognition based on parameter sharing as described in the first aspect.

[0019] The above-mentioned pulmonary embolism recognition device, terminal device and storage medium based on parameter sharing input the image to be measured into a pre-trained first feature extractor to obtain a first feature; input the first feature into a pre-trained first classification network to obtain the probability that the image to be measured contains pulmonary embolism; input the image to be measured with a probability reaching a first threshold into a pre-trained region detection network to detect the location of pulmonary embolism and intercept the corresponding region image; input the region image into a pre-trained second feature extractor to obtain a second feature; fuse the first feature and the second feature to obtain a mixed feature; input the mixed feature into a pre-trained second classification network and a third classification network respectively to confirm the nature type and location type of pulmonary embolism. By means of hierarchical detection and feature fusion, the possible lesion areas in the image to be measured, as well as the type and location of the lesions, are confirmed. The hierarchical relationship and shared feature information between different lesion information dimensions are mined by a multi-level classification scheme, and the commonness and particularity of the feature recognition results are combined, improving the accuracy of the recognition and detection results. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 FIG. is a flowchart of a pulmonary embolism recognition method based on parameter sharing provided by an embodiment of the present invention;

[0021] Figure 2 FIG. is a schematic diagram of an image processing process of a pulmonary embolism recognition method based on parameter sharing provided by an embodiment of the present invention;

[0022] Figure 3 FIG. is a schematic diagram of the structure of the first feature extractor in an embodiment of the present invention;

[0023] Figure 4 FIG. is a schematic diagram of the structure of the first classification network in an embodiment of the present invention;

[0024] Figure 5 FIG. is a schematic diagram of the structure of a pulmonary embolism recognition device based on parameter sharing provided by an embodiment of the present invention;

[0025] Figure 6 FIG. is a schematic diagram of the structure of a terminal device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0026] The present invention will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are used to explain the present invention, rather than limiting the present invention. Additionally, it should be noted that, for the sake of convenience of description, only parts related to the present invention are shown in the drawings, rather than all the structures.

[0027] It should be noted that due to space limitations, the description in the specification of this application does not exhaust all optional implementation manners. After reading the description of this application, those skilled in the art should be able to think that as long as the technical features do not conflict with each other, any combination of technical features can constitute an optional implementation manner.

[0028] For example, in one implementation manner of the embodiment, a technical feature is described: the first feature extractor includes 5 convolutional layers. In another implementation manner of the embodiment, another technical feature is described: the first classification network includes 2 convolutional layers and 1 fully connected layer. After reading the description of this application, those skilled in the art should be able to think that the implementation manner with both of these two features is also an optional implementation manner, that is, in the specific implementation process, the first feature extractor includes 5 convolutional layers, and at the same time the first classification network includes 2 convolutional layers and 1 fully connected layer.

[0029] The following will describe each embodiment in detail.

[0030] Figure 1 It is a flowchart of a pulmonary embolism recognition method based on parameter sharing provided for an embodiment of the present invention. The pulmonary embolism recognition method based on parameter sharing provided in the embodiment can be executed by an operating device corresponding to the pulmonary embolism recognition method based on parameter sharing. The operating device can be implemented in software and / or hardware. The operating device can be composed of two or more physical entities, or can be composed of one physical entity.

[0031] In this solution, first, the problem-related data is defined and the model is described using mathematical symbols. For the basic training data, use to represent the CT images of the data set and the category labels corresponding to the images, where represents the number of CT images included in the data set, represents the th CT image, is the label indicating whether there is a pulmonary embolism in the CT image, that is, indicates that the image contains a pulmonary embolism. Similarly, is the label of the pulmonary embolism category in the CT image, that is, respectively represent no pulmonary embolism, chronic pulmonary embolism, and acute pulmonary embolism. is the label of the pulmonary embolism location in the CT image, that is, respectively represent no pulmonary embolism, the pulmonary embolism is located on the left, in the middle, and on the right. The final task of the model after training based on the data set is: given any CT image , predict whether it contains an embolism. If it contains one, it is also necessary to predict the specific category (acute / chronic) and location (left / middle / right). Based on the above problem-related definitions and model descriptions, this solution passesFigure 1 It is implemented by the relevant steps below. In the actual process of pulmonary embolism recognition, the processing process of the image to be measured is basically similar to that of the sample image. The training of each model is completed based on the sample image as a whole. After the training of the model output is stable, the image to be measured is input into each model to complete the corresponding prediction. In the subsequent description, the training process of each model is mainly described, and the process of predicting and recognizing the image to be measured in each trained model can refer to the corresponding training process.

[0032] Specifically, referring to Figure 1 , the pulmonary embolism recognition method based on parameter sharing specifically includes:

[0033] Step S101: Input the image to be measured into a pre-trained first feature extractor to obtain the first feature.

[0034] For the data flow changes in the detailed processing process of this solution, please refer to Figure 2 . As Figure 2 shown, there are , …, in total n samples. First, use the first feature extractor to extract the features of the input sample , . represents the set of features extracted from all samples, that is, the set of the first features obtained in the training stage.

[0035] The CT images in the training set are 2D images of standard size (such as 512×512, 256×256). The CT images are input into the first feature extractor as shown in Figure 3 . In the first feature extractor shown in Figure 3 , it includes 5 convolutional layers. The first convolutional layer is a convolutional layer with a convolutional kernel size of 3×3, a channel number of 24, and a stride of 1×1. The second convolutional layer consists of two parts, namely a convolutional block with a convolutional kernel size of 3×3, a channel number of 32, and a stride of 1×1, and a max-pooling layer with a convolutional kernel size of 3×3 and a stride of 2×2. The third to fifth layers are similar in structure to the second layer, except that the channel numbers of the third and fourth layers are both 64, and the channel number of the fifth layer is 32. After passing through this network, the feature vector corresponding to each sample is finally output, with a size of .

[0036] Step S102: Input the first feature into a pre-trained first classification network to obtain the probability that the image to be measured contains pulmonary embolism.

[0037] Further referring to Figure 2 , the feature After the first classification network , which is used to calculate the probability of pulmonary embolism in each CT image . If is greater than the first threshold (assuming the first threshold is β ), it is considered that the sample contains embolism, and as described above, 0 or 1 is used to identify no pulmonary embolism and having pulmonary embolism, and is used to represent the features corresponding to the samples predicted by the first classification network to contain pulmonary embolism. In the subsequent steps, the filtered features will pass through the second classification network and the third classification network to respectively predict the category and location of pulmonary embolism.

[0038] In this solution, the first classification network used to calculate the probability of pulmonary embolism in each CT image is trained based on the following minimized loss function:

[0039]

[0040] where represents the label of whether there is pulmonary embolism in the sample in the training set, represents the total number of samples in the training set, represents the sample the probability of containing pulmonary embolism.

[0041] The network structure of the first classification network is as Figure 4 shown. It consists of 2 convolutional layers and 1 fully connected layer. The first convolutional layer contains two parts, namely a convolutional block with a convolutional kernel size of 3×3, 64 channels, and a stride of 1×1, and a max pooling layer with a convolutional kernel size of 3×3 and a stride of 2×2. The second convolutional layer is similar to the first convolutional layer, except that the number of channels is 32, and finally a fully connected layer is connected. The first feature obtained in step S101 is passed through the first classification network, and finally the prediction result of the probability of each CT image containing pulmonary embolism is obtained.

[0042] Step S103: Input the test image whose probability reaches the first threshold into the pre-trained region detection network to detect the location of pulmonary embolism and intercept the corresponding regional image.

[0043] When specifically implementing the detection of regional images, it can be achieved through R-CNN (Region-Convolutional Neural Networks, region-based convolutional neural network) series algorithms (such as R-CNN, Fast R-CNN, Faster R-CNN, etc.), or it can also be achieved through other detection methods. For example, CenterNet can be used as the detection network for regional images.

[0044] If CenterNet is used as the region detection network, it is specifically trained through the following loss function:

[0045]

[0046] Among them, N represents the number of pulmonary embolisms, M is the number of center points of the predicted output regional image, represents the probability that each center point is predicted to belong to a pulmonary embolism, and respectively represent the weight sparsity of the offset of the center point and the weight coefficient of the target size loss of the regional image, and respectively represent the predicted center point offset and the true center point offset, and represent the predicted target size of the regional image and the true target size of the regional image, represents a hyperparameter.

[0047] In this regional detection network using CenterNet, the training input is a sample picture containing pulmonary embolisms. As an anchor-free detection algorithm, the output of CenterNet is a probability prediction P for each point as the center point of the target (pulmonary embolism), and a direct prediction of the center point offset offset and the target size size. As shown in the formula of the loss function above, the classification loss uses the original focal loss, that is, the first term on the right side of the above formula, As a hyperparameter, it generally takes a value of 2. The regression loss uses L1 loss, that is, the second and third terms on the right side of the above formula.

[0048] As Figure 2 shown, assuming that after the screening of the first classification network, among n samples, and reach the first threshold, the CT pictures of these two samples are respectively passed through the pre-trained regional detection network to obtain the central positions of suspected pulmonary embolisms ( x, y) coordinates of the center position of the suspected embolism in each CT image, and taking the coordinates as the center point, a regional image of a preset size (such as 48×48, 60×60, etc.) is intercepted from the corresponding CT image to obtain the Instance set. In a CT image, the number of center positions that may be detected may be more than one. Therefore, there may be multiple regional images intercepted corresponding to each center position in each CT image.

[0049] Step S104: Input the regional image into a pre-trained second feature extractor to obtain the second feature.

[0050] The regional image intercepted in step S103 is a 2D image of standard size, and the structure of the second feature extractor for recognizing the regional image is the same as that of the first feature extractor, that is, it also includes 5 convolutional layers. The first convolutional layer is a convolutional layer with a convolutional kernel size of 3×3, 24 channels, and a stride of 1×1. The second convolutional layer contains two parts, namely a convolutional block with a convolutional kernel size of 3×3, 32 channels, and a stride of 1×1, and a max-pooling layer with a convolutional kernel size of 3×3 and a stride of 2×2. The third to fifth layers are similar to the second layer in structure, except that the number of channels is 64. After passing through this network, the feature vector corresponding to each sample is finally output , with a size of .

[0051] In the specific implementation process, the second feature extractor can be trained based on the following minimized loss function:

[0052]

[0053] Among them, and respectively represent the property label and position label of the sample in the training set, represents the number of samples selected according to the first threshold, represents the first feature of the i th sample selected, represents the second feature of the i th sample selected, represents the mixed feature of the i th sample selected, The second classification network, represents the third classification network. In this embodiment, to distinguish from the object of feature extraction, the selected sample image is represented as , and the regional image intercepted from the selected sample image is represented as It should be noted that the above second feature extractor realizes the feature extraction of the regional image through an optional training method. In the specific implementation process, other existing image feature extraction schemes can also be used for feature extraction of the regional image, such as SIFT (Scale-invariant feature transform), HOG (Histogram of Oriented Gradient), etc.

[0054] Step S105: Fuse the first feature and the second feature to obtain a mixed feature.

[0055] The first feature and the second feature are mixed in the following way:

[0056]

[0057] Among them, represents the mixed feature, represents the first feature, represents the second feature, represents the combination of two feature vectors. The specific mixing strategy can be the concatenation of two feature vectors, addition fusion, element-wise multiplication, etc. In the actual mixing strategy adopted, as long as the two feature vectors are fused into a set of data for feature description through a certain calculation method.

[0058] Step S106: Input the mixed feature into the pre-trained second classification network and third classification network respectively to confirm the nature type and location type of the pulmonary embolism.

[0059] The second classification network is trained based on the following minimization loss function:

[0060]

[0061] Among them, represents the number of samples selected according to the first threshold, represents the sample in the training set 's nature label, represents the second classification network, represents the mixed feature of the selected samples.

[0062] The third classification network is trained based on the following minimization loss function:

[0063] Among them, represents the number of samples selected according to the first threshold, represents the sample in the training set 's location label, Represents the third classification network, Represents the combined features of the selected samples.

[0064] The first feature extractor is trained based on minimizing the following loss function:

[0065]

[0066] Where, Represents the number of samples in the training set, Represents the number of samples selected according to the first threshold, Represents the i th sample's first feature, Represents the i th sample's second feature, , and respectively represent the disease label, property label, and location label of the sample in the training set, Represents the first classification network, Represents the second classification network, Represents the third classification network, Represents the combined features of the selected samples.

[0067] The structures of the second classification network and the third classification network are similar to that of the first classification network, and they are both composed of 2 convolutional layers and 1 fully connected layer. The first convolutional layer consists of two parts, namely a convolutional block with a convolutional kernel size of 3×3, a channel number of 64, and a stride of 1×1, and a max pooling layer with a convolutional kernel size of 3×3 and a stride of 2×2. The second convolutional layer is similar to the first convolutional layer, except that the channel number is 32, and finally a fully connected layer is connected. The main differences between the first classification network, the second classification network, and the third classification network lie in that the structures of the input layer and output layer of each network are correspondingly adjusted according to the needs of data input and classification output.

[0068] It can be seen from the limitations of steps S104 - S106 that the first classification network, the second classification network, and the third classification network all share the parameters of the first feature extractor. The second classification network and the third classification network also share the parameters of the second feature extractor, and synchronously update the parameters of the first feature extractor and the second feature extractor according to the output results of each classifier. Each network uses multiple losses under different classification tasks to learn the shared features and establishes the correlation relationship between different classification tasks.

[0069] In the existing solutions, only a single judgment is made on CT images. If multiple judgments are to be realized, multiple models are required for classification prediction. In this solution, through the correlation relationship between each classification task, a single model is used to perform multi-level classification simultaneously, which is beneficial to exploring the hierarchical relationship and shared feature information between different categories.

[0070] Generally speaking, in a hierarchical manner, first predict whether there is a pulmonary embolism in the image to be measured. If there is a pulmonary embolism, further judge the type and location of the pulmonary embolism. Usually, low-level features (such as shape) of the image to be measured can be captured in the lower layers of the neural network. Therefore, a feature extractor is first used for preliminary feature extraction. And high-level features of the image can be extracted in the higher layers of the neural network. Therefore, different branches are connected to the feature extractor to specifically classify specific tasks. By embedding the hierarchical structure of the categories into the network model, the hierarchical network can make more interpretable predictions, and at the same time enhance the accuracy of the final classification result. Moreover, the parameters of the feature extractor at the bottom layer are shared, which is equivalent to each task's loss updating the parameters of the feature extractor. This is beneficial for the feature extractor to learn the common features between different tasks, while the classification networks of different branches are used to learn the characteristic features of their respective tasks. Combining the commonality and characteristics is also beneficial for enhancing the accuracy of the final classification result.

[0071] As described above, the image to be measured is input into a pre-trained first feature extractor to obtain the first feature; the first feature is input into a pre-trained first classification network to obtain the probability that the image to be measured contains a pulmonary embolism; the image to be measured with a probability reaching the first threshold is input into a pre-trained region detection network to detect the location of the pulmonary embolism and intercept the corresponding region image; the region image is input into a pre-trained second feature extractor to obtain the second feature; the first feature and the second feature are fused to obtain a mixed feature; the mixed feature is respectively input into a pre-trained second classification network and a third classification network to confirm the nature type and location type of the pulmonary embolism. By means of hierarchical detection and feature fusion, the possible lesion area, the type and location of the lesion in the image to be measured are confirmed. The hierarchical relationship and shared feature information between different lesion information dimensions are explored with a multi-level classification scheme. Combining the commonality and characteristics of the feature recognition results improves the accuracy of the recognition and detection results.

[0072] Figure 5 This is a schematic structural diagram of a pulmonary embolism recognition device based on parameter sharing provided by an embodiment of the present invention. Refer to Figure 5 As shown in the figure, the pulmonary embolism recognition device based on parameter sharing includes: a first extraction unit 210, a probability calculation unit 220, a region screenshot unit 230, a second extraction unit 240, a feature fusion unit 250, and a comprehensive judgment unit 260.

[0073] Among them, the first extraction unit 210 is configured to input the image to be measured into a pre-trained first feature extractor to obtain a first feature; the probability calculation unit 220 is configured to input the first feature into a pre-trained first classification network to obtain the probability that the image to be measured contains pulmonary embolism; the region screenshot unit 230 is configured to input the image to be measured with a probability reaching a first threshold into a pre-trained region detection network to detect the position of pulmonary embolism and intercept the corresponding region image; the second extraction unit 240 is configured to input the region image into a pre-trained second feature extractor to obtain a second feature; the feature fusion unit 250 is configured to fuse the first feature and the second feature to obtain a mixed feature; the comprehensive judgment unit 260 is configured to input the mixed feature into a pre-trained second classification network and a third classification network respectively to confirm the nature type and position type of pulmonary embolism.

[0074] Based on the above embodiments, the first classification network is trained based on the following minimized loss function:

[0075]

[0076] Among them, represents the label of whether there is pulmonary embolism in the sample in the training set represents the total number of samples in the training set, represents the sample the probability of containing pulmonary embolism.

[0077] Based on the above embodiments, the region detection network is trained through the following loss function:

[0078]

[0079] Among them, N represents the number of pulmonary embolisms, M is the number of the center points of the predicted output region image, represents the probability that each center point is predicted to belong to pulmonary embolism, and respectively represent the weight sparsity of the offset of the center point and the weight coefficient of the target size loss of the region image, and respectively represent the predicted center point offset and the true center point offset, and represent the predicted target size of the region image and the true target size of the region image, represents a hyperparameter.

[0080] Based on the above embodiments, the first feature and the second feature are mixed in the following manner:

[0081] ​

[0082] Among them, represents a mixed feature, represents the first feature, represents the second feature, represents mixing two feature vectors.

[0083] Based on the above embodiments, the second feature extractor is trained based on the following minimized loss function:

[0084]

[0085] Among them, and respectively represent the property label and the position label of the sample in the training set, represents the number of samples screened according to the first threshold, represents the first feature of the i th sample screened, represents the second feature of the i th sample screened, represents the mixed feature of the i th sample screened, The second classification network, represents the third classification network.

[0086] Based on the above embodiments, the second classification network is trained based on the following minimized loss function:

[0087]

[0088] Among them, represents the number of samples screened according to the first threshold, represents the property label of the sample in the training set, represents the second classification network, represents the mixed feature of the screened samples.

[0089] Based on the above embodiments, the third classification network is trained based on the following minimized loss function:

[0090]

[0091] Among them, represents the number of samples screened according to the first threshold, represents the position label of the sample in the training set, represents the third classification network, represents the mixed feature of the screened samples.

[0092] Based on the above embodiments, the first feature extractor is trained based on the following minimized loss function:

[0093]

[0094] wherein, represents the number of samples in the training set, represents the number of samples filtered according to the first threshold, represents the first feature of the i th sample filtered out, represents the second feature of the i th sample filtered out, , and respectively represent the disease label, property label and location label of the sample in the training set, represents the first classification network, represents the second classification network, represents the third classification network, represents the mixed feature of the filtered samples.

[0095] The pulmonary embolism recognition device based on parameter sharing provided by the embodiments of the present invention is included in the terminal device and can be used to execute any one of the pulmonary embolism recognition methods based on parameter sharing provided in the above embodiments, and has corresponding functions and beneficial effects.

[0096] It should be noted that in the embodiments of the above-mentioned pulmonary embolism recognition device based on parameter sharing, the included units and modules are only divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of mutual distinction and do not limit the protection scope of the present invention.

[0097] Figure 6 FIG. is a schematic structural diagram of a terminal device provided by an embodiment of the present invention, and this terminal device is a specific hardware presentation solution of the operation device of the intelligent interaction tablet described above. As Figure 6 shown, this terminal device includes a processor 310, a memory 320, an input device 330, an output device 340, and a communication device 350; the number of processors 310 in the terminal device can be one or more, Figure 6 taking one processor 310 as an example; the processor 310, memory 320, input device 330, output device 340, and communication device 350 in the terminal device can be connected through a bus or other means, Figure 6 taking connection through a bus as an example.

[0098] The memory 320, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and modules, such as the program instructions / modules corresponding to the pulmonary embolism recognition method based on parameter sharing in the embodiments of the present invention (for example, the first extraction unit 210, probability calculation unit 220, region screenshot unit 230, second extraction unit 240, feature fusion unit 250, and comprehensive judgment unit 260 in the pulmonary embolism recognition device based on parameter sharing). By running the software programs, instructions, and modules stored in the memory 320, the processor 310 executes various functional applications and data processing of the terminal device, that is, implements the above-mentioned pulmonary embolism recognition method based on parameter sharing.

[0099] The memory 320 may mainly include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the terminal device and the like. In addition, the memory 320 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage devices. In some instances, the memory 320 may further include a memory remotely set relative to the processor 310, and these remote memories can be connected to the terminal device through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0100] The input device 330 can be used to receive input digital or character information, and generate key signal inputs related to the user settings and function controls of the terminal device. The output device 340 may include display devices such as a display screen.

[0101] The above terminal device includes a pulmonary embolism recognition device based on parameter sharing, which can be used to execute any pulmonary embolism recognition method based on parameter sharing, and has corresponding functions and beneficial effects.

[0102] The embodiments of the present invention further provide a storage medium containing computer-executable instructions, and the computer-executable instructions are used to perform related operations in the pulmonary embolism recognition method based on parameter sharing provided in any embodiment of the present application when executed by a computer processor, and have corresponding functions and beneficial effects.

[0103] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product.

[0104] Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The present application is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory capable of guiding the computer or other programmable data processing devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks. These computer program instructions can also be loaded onto the computer or other programmable data processing devices, such that a series of operation steps are executed on the computer or other programmable devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable devices provide steps for implementing the functions specified in the flow Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0105] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory. The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0106] A computer-readable medium includes both permanent and non-permanent, removable and non-removable media and can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0107] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising the element.

[0108] Note that the above is only a preferred embodiment of the present invention and the applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, re-adjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in more detail through the above embodiments, the present invention is not limited to the above embodiments. Without departing from the concept of the present invention, more other equivalent embodiments can be included, and the scope of the present invention is determined by the scope of the appended claims.

Claims

1. A pulmonary embolism recognition device based on parameter sharing, characterized in that Including: A first extraction unit for inputting an image to be measured into a pre-trained first feature extractor to obtain a first feature; A probability calculation unit for inputting the first feature into a pre-trained first classification network to obtain the probability that the image to be measured contains pulmonary embolism; A region screenshot unit for inputting an image to be measured with a probability reaching a first threshold into a pre-trained region detection network to detect the location of pulmonary embolism and intercept the corresponding region image; A second extraction unit for inputting the region image into a pre-trained second feature extractor to obtain a second feature; A feature fusion unit for fusing the first feature and the second feature to obtain a mixed feature; A comprehensive judgment unit for inputting the mixed feature into a pre-trained second classification network and a third classification network respectively to confirm the nature type and location type of pulmonary embolism; Wherein, the first classification network is trained based on the following minimized loss function: Among them, represents the label of whether there is pulmonary embolism in the training set samples , represents the total number of samples in the training set, represents the sample with the probability of containing pulmonary embolism.

2. The pulmonary embolism recognition device according to claim 1, wherein, The region detection network is trained through the following loss function: Where N represents the number of pulmonary embolisms, and M represents the number of center points of the predicted output regional image. represents the probability that each center point is predicted to belong to a pulmonary embolism. and represent the weight sparsity of the offset of the center point and the weight coefficient of the target size loss of the regional image, respectively. and represent the predicted center point offset and the true center point offset, respectively. and represent the predicted target size of the regional image and the true target size of the regional image. represents a hyperparameter.

3. The pulmonary embolism recognition device according to claim 1, wherein, The first feature and the second feature are mixed in the following manner: Among them, represents a mixed feature, represents a first feature, represents a second feature, represents mixing two feature vectors.

4. The pulmonary embolism recognition device according to claim 1, wherein, The second feature extractor is trained based on the following minimized loss function: Among them, and respectively represent the property label and the position label of the sample in the training set, represents the number of samples selected according to the first threshold, represents the first feature of the i th selected sample, represents the second feature of the i th selected sample, represents the mixed feature of the i th selected sample, the second classification network, represents the third classification network.

5. The pulmonary embolism recognition device according to claim 1, characterized in that, The second classification network is trained based on the following minimized loss function: Among them, represents the number of samples selected according to the first threshold, represents the samples in the training set with property labels, represents the second classification network, represents the mixed features of the selected samples.

6. The pulmonary embolism recognition device according to claim 1, wherein The third classification network is trained based on the following minimized loss function: Among them, represents the number of samples selected according to the first threshold, represents the samples in the training set of the position label, represents the third classification network, represents the mixed features of the selected samples.

7. The pulmonary embolism recognition device according to claim 1, characterized in that, The first feature extractor is trained based on the following minimized loss function: Among them, represents the number of samples in the training set, represents the number of samples selected according to the first threshold, represents the i th first feature of the selected sample, represents the i th second feature of the selected sample, , and respectively represent the disease label, property label and position label of the sample in the training set, represents the first classification network, represents the second classification network, represents the third classification network, represents the mixed feature of the selected sample.

Images