Model training method of joint dataset, x-ray film identification method and related equipment

Abstract

The present disclosure provides a model training method of a joint data set, an X sheet identification method and related equipment. Different categories of data sets are combined during model training, thereby expanding the sample categories. Then, the network parameters of the feature extraction network corresponding to each category of data set are updated using the common loss value of the combined data set, which is beneficial to considering other categories of data sets when the feature extraction network corresponding to different categories of data sets is used, thereby improving the generality and accuracy of the identification model obtained by training. And according to the preset demand set by the user, the probability vector is analyzed to obtain the to-be-identified result of the to-be-identified X sheet. Not only can the to-be-identified X sheet be accurately identified, but also the actual needs of the user can be met, and the flexibility is high.

Description

Technical Field

[0001] This disclosure relates to the field of image recognition technology, and in particular to a model training method for a joint dataset, a method for recognizing X-ray images, and related equipment. Background Technology

[0002] Chest X-rays are currently the most common examination method for screening and diagnosing pleural diseases. In routine clinical practice, the screening and diagnosis of chest X-ray images rely on the experience of professional physicians, which is time-consuming and prone to subjective assessment errors. Currently, automated chest X-ray screening mainly consists of three interrelated tasks: anomaly identification, disease classification within anomalies, and lesion region localization within diseases. Although existing algorithms can generally determine whether a chest X-ray is normal or abnormal, the performance of the anomaly identification task is limited due to the low specificity of chest X-rays and the low distinguishability between and within classes. Furthermore, in real-world clinical scenarios, positive samples are scarce, or even specific diseases are lacking, making effective supervised training impossible. Summary of the Invention

[0003] The purpose of this disclosure is to provide a model training method for joint datasets, an X-ray recognition method, and related equipment. During model training, a common loss value is used to update the network parameters of the feature extraction networks corresponding to each category of datasets. This is beneficial for the feature extraction networks corresponding to different categories of datasets to take into account other categories of datasets when used, thereby improving the versatility of the trained recognition model.

[0004] To achieve the above objectives, this disclosure adopts the following technical solution: a model training method for a joint dataset, comprising:

[0005] Multiple first datasets of different categories are obtained, and the first training images contained in each first dataset are adjusted to a preset format to obtain the second datasets corresponding to each first dataset.

[0006] Each of the second datasets is used as training data and input into the neural network for model training. During the model training process, the joint loss function is used to calculate the loss value of each of the second datasets after they are combined, and the network parameters of the feature extraction network corresponding to each of the second datasets are updated using the loss value. The process is iterated until the model converges to obtain the recognition model.

[0007] This disclosure also provides a method for identifying X-ray images, comprising:

[0008] Obtain the X-ray image to be identified;

[0009] The X to be identified is input into the identification model to predict the probability vector, wherein the identification model is trained by the model training method of the joint dataset as described above, and the training data used by the identification model is the image obtained by X-ray scanning.

[0010] According to the preset requirements, the negative probability and positive probability corresponding to the X-ray image to be identified are obtained by parsing from the probability vector;

[0011] The identification result of the X-ray film to be identified is determined based on the relationship between the negative probability and the positive probability.

[0012] This disclosure also provides a model training apparatus for a joint dataset, comprising:

[0013] The first acquisition module is used to acquire multiple first datasets of different categories, and adjust the first training images contained in each first dataset to a preset format to obtain the second datasets corresponding to each first dataset.

[0014] The training module is used to input each of the second datasets as training data into the neural network for model training. During the model training process, the joint loss function is used to calculate the loss value of each of the second datasets after they are combined, and the network parameters of the feature extraction network corresponding to each of the second datasets are updated using the loss. The process is iterated until the model converges to obtain the recognition model.

[0015] This disclosure also provides an X-ray film identification device, comprising:

[0016] The second acquisition module is used to acquire the X-ray image to be identified;

[0017] The prediction module is used to input the X to be identified into the recognition model and predict the probability vector, wherein the recognition model is trained by the model training method of the joint dataset as described above, and the training data used by the recognition model is the image obtained by X-ray scanning.

[0018] The parsing module is used to parse the negative probability and positive probability corresponding to the X-ray image to be identified from the probability vector according to preset requirements;

[0019] The determination module is used to determine the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability.

[0020] This disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0021] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0022] This disclosure provides a model training method for joint datasets, an X-ray image recognition method, and related equipment. During model training, multiple first datasets of different categories are first acquired, and the first training images contained in each first dataset are adjusted to a preset format to obtain second datasets corresponding to each first dataset. Then, each second dataset is input as training data into a neural network for model training. During model training, a joint loss function is used to calculate the loss value of the combined second datasets, and this loss value is used to update the network parameters of the feature extraction networks corresponding to each second dataset. This process is iterated until the model converges, resulting in a recognition model. In application, the X-ray to be recognized is input into the recognition model to predict a probability vector. Then, according to preset requirements, the negative and positive probabilities corresponding to the X-ray to be recognized are parsed from the probability vector. Finally, the recognition result of the X-ray to be recognized is determined based on the relationship between the negative and positive probabilities. In this disclosure, datasets of different categories are combined during model training to expand the sample categories. Then, the network parameters of the feature extraction network corresponding to each category of dataset are updated using the common loss value of the combined dataset. This is beneficial for the feature extraction network corresponding to different categories of datasets to take into account other categories of datasets when using them, thereby improving the generality and accuracy of the trained recognition model. Attached Figure Description

[0023] Figure 1 This is a schematic diagram of the steps of a model training method for a joint dataset in one embodiment of this disclosure;

[0024] Figure 2 This is a schematic diagram of the steps of an X-ray identification method in one embodiment of the present disclosure;

[0025] Figure 3 This is an overall structural block diagram of a model training device for a joint dataset in one embodiment of this disclosure;

[0026] Figure 4 This is an overall structural block diagram of the X-ray identification device in one embodiment of the present disclosure;

[0027] Figure 5 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present disclosure.

[0028] The realization of the purpose, functional features and advantages of this disclosure will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0030] Reference Figure 1 One embodiment of this disclosure provides a model training method for a joint dataset, comprising:

[0031] S1: Obtain multiple first datasets of different categories, and adjust the first training images contained in each first dataset to a preset format to obtain the second datasets corresponding to each first dataset;

[0032] S2: Input each of the second datasets as training data into the neural network for model training. During the model training process, use the joint loss function to calculate the loss value of each of the second datasets after combination, and use the loss value to update the network parameters of the feature extraction network corresponding to each of the second datasets respectively. Iterate in a loop until the model converges to obtain the recognition model.

[0033] In this embodiment, the training system acquires multiple pre-prepared first datasets of different categories, each of which is collected in the same way. This embodiment uses two different categories of first datasets, VIN-CXR (VinBigData Chest X-ray) and SIIM-CXR, as examples. In practical applications, more categories of first datasets can be used for model training; the selection depends on actual needs and is not specifically limited here. The first training images contained in the VIN-CXR and SIIM-CXR datasets are both X-ray images obtained through X-ray scanning, and both are related to human chest X-rays. Specifically, the first dataset, VIN-CXR, contains one category of normal images and 14 categories of disease images, labeled as follows: No findings (normal) 0, Aortic enlargement 1, Atelectasis 2, Calcification 3, Cardiomegaly 4, Consolidation 5, Interstitial lung disease 6, Infiltration 7, Lung Opacity 8, Nodule / Mass 9, Other lesion 10, Pleural effusion 11, Pleural Thickening 12, Pneumothorax 13, and Pulmonary fibrosis 14. The first dataset, SIIM-CXR, contains one class of normal images and three classes of disease images, labeled as: Negative for Pneumonia (0), Typical Appearance (1), Indeterminate Appearance (2), and Atypical Appearance (3). During model training, regardless of the number of classes in the first dataset, all images are uniformly converted to one-hot class encoding. For example, in the first datasets VIN-CXR and SIIM-CXR, a single first training image in VIN-CXR corresponds to a 15-dimensional vector of 0s and 1s, while a single first training image in SIIM-CXR corresponds to a 4-dimensional vector of 0s and 1s.For example, if the first training image in the first dataset VIN-CXR is a normal image, its corresponding category vector is: [1,0,0,0,0,0,0,0,0,0,0,0,0,0,0], with a length of 15 and the first flag bit being 1; if the first training image is a Consolidation image, its corresponding category vector is: [0,0,0,0,0,1,0,0,0,0,0,0,0,0,0], with a length of 15 and the 6th flag bit being 1; if the first training image contains multiple diseases, such as Consolidation and Pulmonary fibrosis, the corresponding category vector is [0,0,0,0,0,1,0,0,0,0,0,0,0,0,1], with the 6th and 15th flag bits being 1. In the first dataset SIIM-CXR, a normal first training image has a category vector of [1,0,0,0] with a length of 4, and the first flag bit is 1. If it is an Atypical Appearance image, the category vector is [0,0,0,1], and the fourth flag bit is 1. Because the last three categories in this dataset are exclusive, there will not be two disease categories at the same time. That is, the category vector corresponding to a single first training image will not have two 1s.

[0034] Before inputting the first training images contained in each first dataset into the neural network for model training, the training system adjusts each first training image to a preset size and standardizes the pixel values ​​of each first training image within a preset range. This adjusts the first training images of each first dataset to a preset format, resulting in a second dataset corresponding to each first dataset (the second training images in the second dataset are obtained by adjusting the first training images to a preset format). The training system then inputs each second dataset as training data into the neural network for model training. Specifically, this neural network includes multiple feature extraction networks, multiple first fully connected layers, and one fully connected layer. The number of feature extraction networks and the number of first fully connected layers are consistent with the number of categories in the first dataset, and each feature extraction network corresponds to a single first fully connected layer. The outputs of the multiple first fully connected layers are all connected to the input of the second fully connected layer. During model training, each second dataset for a single category is processed by a single feature extraction network and a first fully connected layer (e.g., the first dataset VIN-CXR is processed by feature extraction network A and first fully connected layer A; the first dataset SIIM-CXR is processed by feature extraction network B and first fully connected layer B). First, the second training images contained in each second dataset are input into their respective feature extraction networks for feature extraction, resulting in feature maps for each second training image. Then, each feature map is input into its corresponding first fully connected layer and mapped to a high-dimensional vector, resulting in a first vector for each feature map; the first vectors output by different first fully connected layers have the same length. The first vectors are concatenated at the input of the second fully connected layer to obtain a second vector. The concatenated second vector is then input into the second fully connected layer and mapped to a low-dimensional vector, resulting in a third vector. The third vector is probabilized using a softmax function to obtain multiple probability values, the number of which is consistent with the length of the third vector (i.e., these probability values ​​represent the probability of each vector value in the third vector, with each vector value representing a training image of a single category). Finally, a joint loss function is used to calculate the loss on the probabilized third vector, obtaining the loss value corresponding to the joint processing of all datasets. This loss is then fed back to each feature extraction network to update the network parameters. This model training process is iterated until the model converges, resulting in a recognition model with ideal performance.

[0035] In this embodiment, different categories of datasets are combined during model training to expand the sample categories. Then, the common loss value of the combined datasets is used to update the network parameters of the feature extraction networks corresponding to each category of datasets. This is beneficial for the feature extraction networks corresponding to different categories of datasets to take into account other categories of datasets when using them, thereby improving the generality and accuracy of the trained recognition model.

[0036] Furthermore, the neural network comprises multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. The number of feature extraction networks and the number of first fully connected layers are consistent with the number of categories in the first dataset. Each feature extraction network is connected to a single first fully connected layer. During model training, the step of calculating the joint loss value of each second dataset using a classification loss function includes:

[0037] S201: Input the second training images of each second dataset into their respective feature extraction networks to extract features, and obtain the feature maps corresponding to each second training image.

[0038] S202: Each feature map is input into its corresponding first fully connected layer and mapped to a high-dimensional vector to obtain a first vector corresponding to each feature map, wherein the lengths of the first vectors are the same.

[0039] S203: Concatenate the first vectors to obtain the second vector;

[0040] S204: Input the second vector into the second fully connected layer and map it into a low-dimensional vector to obtain the third vector;

[0041] S205: Use the softmax function to probabilize the third vector, and use the joint loss function to calculate the loss of the probabilized third vector to obtain the loss value.

[0042] In this embodiment, the neural network includes multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. A single category of the second dataset corresponds to a single feature extraction network and a single first fully connected layer. The feature extraction network is used to extract image features from the second training images and can use popular networks such as DenseNet-121, ResNet-50, esNet18, and MobileNetV3. This embodiment uses the second dataset categories VIN-CXR and SIIM-CXR as examples. For VIN-CXR, the feature extraction network is DenseNet-121, and for SIIM-CXR, it is ResNet-50 (in actual applications, the category of the feature extraction network used for different dataset categories can be selected according to actual needs, and no specific restrictions are made here). The second dataset VIN-CXR is input into the DenseNet-121 feature extraction network for feature extraction, resulting in a corresponding feature map A. The size of feature map A is (1024, 8, 8), i.e., 1024 channels, with a height and width of 8. The second dataset, SIIM-CXR, is input into the ResNet-50 feature extraction network for feature extraction, resulting in feature map B. Feature map B has a size of (2048, 8, 8), meaning its height and width are both 8, and it has 2048 channels. Then, feature maps A and B are simultaneously input into their respective first fully connected layers and mapped to high-dimensional vectors, resulting in first vectors A and B with the same length of 512. Next, first vectors A and B are concatenated to obtain a second vector of length 1024. This concatenated second vector is input into the second fully connected layer and mapped to a low-dimensional vector, resulting in a third vector with a length of 18. This third vector corresponds to the total number of categories in the VIN-CXR and SIIM-CXR datasets (in both datasets VIN-CXR and SIIM-CXR, the first category is the normal case, which can be merged). By merging datasets of different categories, the coverage of negative samples is expanded, and the coverage of positive sample categories is increased. The third vector is probabilized using the softmax function, resulting in 18 probability values ​​(each representing the probability of occurrence of a vector value at a corresponding position within the third vector). Finally, a joint loss function is used to calculate the loss on the probabilized third vector, yielding the combined loss value for each dataset. Preferably, the joint loss function is BCELoss, which calculates the binary cross entropy between the predicted and true classes.

[0043] Furthermore, the step of adjusting the first training images contained in each of the first datasets to a preset format to obtain the second datasets corresponding to each of the first datasets includes:

[0044] S101: Adjust each of the first training images according to a preset image size, and standardize the pixel values ​​of each of the first training images to a preset range to obtain the second training images corresponding to each of the first training images.

[0045] In this embodiment, the training system adjusts each first training image according to a preset image size and standardizes the pixel values ​​of each first training image to a preset range, thereby obtaining a second training image corresponding to each first training image. Preferably, the training system adjusts the pixel values ​​of each first training image to (3, 256, 256), i.e., 3 channels, width 256, and height 256. Furthermore, the pixel values ​​of the first training images are standardized to [0, 1]. After completing the preset image size adjustment and pixel value standardization, the first training images are transformed into second training images.

[0046] Reference Figure 2 This disclosure also provides an embodiment of a method for identifying X-ray images, comprising:

[0047] A1: Obtain the X-ray image to be identified;

[0048] A2: Input the X to be identified into the identification model to predict the probability vector, wherein the identification model is trained by the model training method of the joint dataset as described above, and the training data used by the identification model is the image obtained by X-ray scanning;

[0049] A3: Based on the preset requirements, the negative probability and positive probability corresponding to the X-ray image to be identified are obtained by parsing from the probability vector;

[0050] A4: Based on the relationship between the negative probability and the positive probability, determine the identification result of the X-ray film to be identified.

[0051] In this embodiment, the recognition model trained by the joint dataset model training method is deployed on the hardware device of the recognition system. During application, the recognition system retrieves a pre-taken X-ray image to be recognized and inputs it into the recognition model to obtain a probability vector. This probability vector contains one negative vector value and multiple positive vector values. The number of positive vector values ​​is the same as the number of disease image categories contained in the dataset used during model training (i.e., the first dataset). For example, if the datasets used during model training are VIN-CXR and SIIM-CXR, these two datasets contain a total of 17 disease image categories. Therefore, the probability vector obtained by the trained recognition model contains 17 positive vector values ​​(as can be seen from the embodiments of steps S201-S205 above, the categories of normal images are merged, so the probability vector only contains one negative vector value). Based on the user's preset requirements, the recognition system parses the negative and positive probabilities corresponding to the X-ray image to be recognized from the probability vector output by the model. Specifically, the recognition system uses the negative vector value (located in the first position of the probability vector) as the negative probability of the X-ray image to be recognized and retrieves the preset requirements. If the preset requirement is to reduce the false diagnosis rate, the positive vector value with the highest value is selected from all positive vector values ​​as the positive probability of the X-ray to be identified. If the preset requirement is to reduce the false negative rate, the positive vector value with the lowest value is selected from all positive vector values ​​as the positive probability of the X-ray to be identified. The identification system analyzes the relationship between the negative and positive probabilities of the X-ray to be identified. If the negative probability is greater than the positive probability, the identification result of the X-ray to be identified is determined to be negative (i.e., normal); if the negative probability is less than the positive probability, the identification result of the X-ray to be identified is determined to be positive (i.e., abnormal).

[0052] In this embodiment, the recognition model can output probability vectors (where the vector values ​​of the probability vectors are probability values) for the categories of the X-ray images to be recognized, corresponding to the datasets used during training. This ensures the comprehensiveness and accuracy of the recognition of the X-ray images. Furthermore, the recognition system parses the probability vectors according to the user's preset requirements to obtain the recognition results for the X-ray images. This not only achieves accurate recognition of the X-ray images but also meets the user's actual needs, offering high flexibility.

[0053] Furthermore, the probability vector includes one negative vector value and multiple positive vector values, and the sum of the probabilities of the negative vector value and all positive vector values ​​in the probability vector is 1. The step of parsing the negative and positive probabilities corresponding to the X-ray image to be identified from the probability vector according to preset requirements includes:

[0054] A301: Use the negative vector value as the negative probability of the X-ray image to be identified, and retrieve the preset requirements;

[0055] A302: If the preset requirement is to reduce the misdiagnosis rate, then the positive vector value with the maximum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified;

[0056] A303: If the preset requirement is to reduce the false negative rate, then the positive vector value with the minimum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified.

[0057] In this embodiment, the probability vector output by the model contains one negative vector value and multiple positive vector values, and the sum of the probabilities of the negative vector value and all positive vector values ​​is 1. Taking the first dataset, which used two categories during model training—VIN-CXR and SIIM-CXR—as an example, the length of the probability vector predicted by the recognition model is 18. The probability vector value at the first position (corresponding to the image category labeling in the dataset during model training) is a negative vector value, representing the probability that the X-ray image to be identified is negative. Therefore, the recognition system uses the negative vector value as the negative probability of the X-ray image to be identified. The vector values ​​at positions 2 to 18 are all positive vector values, representing the probability that the X-ray image to be identified is positive. If the user's preset requirement is to reduce the false diagnosis rate, then the probability that the X-ray image to be identified is positive should be increased. Therefore, the vector value with the maximum value is selected from all positive vector values ​​as the positive probability of the X-ray image to be identified. If the user's preset requirement is to reduce the false negative rate, then the probability of the X-ray being identified as negative should be increased. Therefore, the positive vector value with the minimum value is selected from each positive vector value as the probability of the X-ray being positive.

[0058] Furthermore, the step of determining the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability includes:

[0059] A401: Determine whether the negative probability is greater than the positive probability;

[0060] A402: If the negative probability is greater than the positive probability, then the identification result is determined to be negative;

[0061] A403: If the negative probability is less than the positive probability, then the identification result is determined to be positive.

[0062] In this embodiment, the identification system determines whether the negative probability of the X-ray image to be identified is greater than the positive probability. If the negative probability is greater than the positive probability, the identification result of the X-ray image to be identified is determined to be negative (i.e., normal). If the negative probability is less than the positive probability, the identification result of the X-ray image to be identified is determined to be positive.

[0063] Reference Figure 3This disclosure provides a model training apparatus for a joint dataset, comprising:

[0064] The first acquisition module 1 is used to acquire multiple first datasets of different categories, and adjust the first training images contained in each first dataset to a preset format to obtain the second datasets corresponding to each first dataset.

[0065] Training module 2 is used to input each of the second datasets as training data into the neural network for model training. During the model training process, the joint loss function is used to calculate the loss value of each of the second datasets after combination, and the network parameters of the feature extraction network corresponding to each of the second datasets are updated using the loss. The process is iterated until the model converges to obtain the recognition model.

[0066] Furthermore, the neural network includes multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. The number of feature extraction networks and the number of first fully connected layers are consistent with the number of categories in the first dataset. Each feature extraction network is connected to a single first fully connected layer. The training module 2 includes:

[0067] The extraction unit is used to input the second training images of each second dataset into their respective corresponding feature extraction networks to extract features and obtain feature maps corresponding to each second training image.

[0068] The first mapping unit is used to input each of the feature maps into their respective first fully connected layers and map them into high-dimensional vectors to obtain a first vector corresponding to each of the feature maps, wherein the lengths of the first vectors are the same.

[0069] The splicing unit is used to splice the first vectors together to obtain the second vector;

[0070] The second mapping unit is used to map the second vector input to the second fully connected layer into a low-dimensional vector to obtain the third vector;

[0071] The calculation unit is used to probabilize the third vector using the softmax function and to calculate the loss of the probabilized third vector using the joint loss function to obtain the loss value.

[0072] Furthermore, the first acquisition module 1 includes:

[0073] The adjustment unit is used to adjust each of the first training images according to a preset image size and to standardize the pixel values ​​of each of the first training images to a preset range, so as to obtain the second training images corresponding to each of the first training images.

[0074] In this embodiment, each module and unit in the model training device for the joint dataset is used to perform each step in the model training method for the joint dataset described above. The specific implementation process is not described in detail here.

[0075] Reference Figure 4 This disclosure provides an X-ray film recognition device, comprising:

[0076] The second acquisition module 3 is used to acquire the X-ray image to be identified;

[0077] Prediction module 4 is used to input the X to be identified into the recognition model and predict the probability vector, wherein the recognition model is trained by the model training method of the joint dataset as described above, and the training data used by the recognition model is the image obtained by X-ray scanning.

[0078] The parsing module 5 is used to parse the negative probability and positive probability corresponding to the X-ray image to be identified from the probability vector according to preset requirements;

[0079] The determination module 6 is used to determine the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability.

[0080] Furthermore, the probability vector includes one negative vector value and multiple positive vector values, and the sum of the probabilities of the negative vector value and all positive vector values ​​in the probability vector is 1. The parsing module 5 includes:

[0081] The selection unit is used to use the negative vector value as the negative probability of the X-ray image to be identified, and to retrieve the preset requirements;

[0082] The first screening unit is used to select the positive vector value with the maximum value from each of the positive vector values ​​as the positive probability of the X-ray to be identified if the preset requirement is to reduce the misdiagnosis rate.

[0083] The second screening unit is used to select the positive vector value with the minimum value from the positive vector values ​​as the positive probability of the X-ray to be identified if the preset requirement is to reduce the false negative rate.

[0084] Furthermore, the determination module 6 includes:

[0085] The judgment unit is used to determine whether the negative probability is greater than the positive probability;

[0086] The first determination unit is used to determine that the identification result is negative if the negative probability is greater than the positive probability.

[0087] The second determination unit is used to determine that the identification result is positive if the negative probability is less than the positive probability.

[0088] In this embodiment, each module and unit in the X-ray identification device is used to perform the corresponding steps in the X-ray identification method described above, and the specific implementation process is not described in detail here.

[0089] This disclosure provides a model training device for joint datasets and an X-ray image recognition device. During model training, multiple first datasets of different categories are firstly acquired, and the first training images contained in each first dataset are adjusted to a preset format to obtain second datasets corresponding to each first dataset. Then, each second dataset is input as training data into a neural network for model training. During model training, a joint loss function is used to calculate the loss value of the combined second datasets, and this loss value is used to update the network parameters of the feature extraction networks corresponding to each second dataset. This process is iterated until the model converges, resulting in a recognition model. In application, the X-ray to be recognized is input into the recognition model, and a probability vector is predicted. Then, according to preset requirements, the negative and positive probabilities corresponding to the X-ray to be recognized are parsed from the probability vector. Finally, the recognition result of the X-ray to be recognized is determined based on the relationship between the negative and positive probabilities. In this disclosure, datasets of different categories are combined during model training to expand the sample categories. Then, the network parameters of the feature extraction network corresponding to each category of dataset are updated using the common loss value of the combined dataset. This is beneficial for the feature extraction network corresponding to different categories of datasets to take into account other categories of datasets when using them, thereby improving the generality and accuracy of the trained recognition model.

[0090] Reference Figure 5 This disclosure also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as a first dataset. The network interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a model training method for a joint dataset and an X-ray image recognition method.

[0091] The processor described above executes the following steps for training the model on the joint dataset:

[0092] S1: Obtain multiple first datasets of different categories, and adjust the first training images contained in each first dataset to a preset format to obtain the second datasets corresponding to each first dataset;

[0093] S2: Input each of the second datasets as training data into the neural network for model training. During the model training process, use the joint loss function to calculate the loss value of each of the second datasets after combination, and use the loss value to update the network parameters of the feature extraction network corresponding to each of the second datasets respectively. Iterate in a loop until the model converges to obtain the recognition model.

[0094] Furthermore, the neural network comprises multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. The number of feature extraction networks and the number of first fully connected layers are consistent with the number of categories in the first dataset. Each feature extraction network is connected to a single first fully connected layer. During model training, the step of calculating the joint loss value of each second dataset using a classification loss function includes:

[0095] S201: Input the second training images of each second dataset into their respective feature extraction networks to extract features, and obtain the feature maps corresponding to each second training image.

[0096] S202: Each feature map is input into its corresponding first fully connected layer and mapped to a high-dimensional vector to obtain a first vector corresponding to each feature map, wherein the lengths of the first vectors are the same.

[0097] S203: Concatenate the first vectors to obtain the second vector;

[0098] S204: Input the second vector into the second fully connected layer and map it into a low-dimensional vector to obtain the third vector;

[0099] S205: Use the softmax function to probabilize the third vector, and use the joint loss function to calculate the loss of the probabilized third vector to obtain the loss value.

[0100] Furthermore, the step of adjusting the first training images contained in each of the first datasets to a preset format to obtain the second datasets corresponding to each of the first datasets includes:

[0101] S101: Adjust each of the first training images according to a preset image size, and standardize the pixel values ​​of each of the first training images to a preset range to obtain the second training images corresponding to each of the first training images.

[0102] The processor described above performs the following steps in the X-ray identification method:

[0103] A1: Obtain the X-ray image to be identified;

[0104] A2: Input the X to be identified into the identification model to predict the probability vector, wherein the identification model is trained by the model training method of the joint dataset as described above, and the training data used by the identification model is the image obtained by X-ray scanning;

[0105] A3: Based on the preset requirements, the negative probability and positive probability corresponding to the X-ray image to be identified are obtained by parsing from the probability vector;

[0106] A4: Based on the relationship between the negative probability and the positive probability, determine the identification result of the X-ray film to be identified.

[0107] Furthermore, the probability vector includes one negative vector value and multiple positive vector values, and the sum of the probabilities of the negative vector value and all positive vector values ​​in the probability vector is 1. The step of parsing the negative and positive probabilities corresponding to the X-ray image to be identified from the probability vector according to preset requirements includes:

[0108] A301: Use the negative vector value as the negative probability of the X-ray image to be identified, and retrieve the preset requirements;

[0109] A302: If the preset requirement is to reduce the misdiagnosis rate, then the positive vector value with the maximum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified;

[0110] A303: If the preset requirement is to reduce the false negative rate, then the positive vector value with the minimum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified.

[0111] Furthermore, the step of determining the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability includes:

[0112] A401: Determine whether the negative probability is greater than the positive probability;

[0113] A402: If the negative probability is greater than the positive probability, then the identification result is determined to be negative;

[0114] A403: If the negative probability is less than the positive probability, then the identification result is determined to be positive.

[0115] An embodiment of this disclosure also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a model training method for a joint dataset and a method for recognizing X-ray films. The model training method for the joint dataset specifically includes:

[0116] S1: Obtain multiple first datasets of different categories, and adjust the first training images contained in each first dataset to a preset format to obtain the second datasets corresponding to each first dataset;

[0117] S2: Input each of the second datasets as training data into the neural network for model training. During the model training process, use the joint loss function to calculate the loss value of each of the second datasets after combination, and use the loss value to update the network parameters of the feature extraction network corresponding to each of the second datasets respectively. Iterate in a loop until the model converges to obtain the recognition model.

[0118] Furthermore, the neural network comprises multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. The number of feature extraction networks and the number of first fully connected layers are consistent with the number of categories in the first dataset. Each feature extraction network is connected to a single first fully connected layer. During model training, the step of calculating the joint loss value of each second dataset using a classification loss function includes:

[0119] S201: Input the second training images of each second dataset into their respective feature extraction networks to extract features, and obtain the feature maps corresponding to each second training image.

[0120] S202: Each feature map is input into its corresponding first fully connected layer and mapped to a high-dimensional vector to obtain a first vector corresponding to each feature map, wherein the lengths of the first vectors are the same.

[0121] S203: Concatenate the first vectors to obtain the second vector;

[0122] S204: Input the second vector into the second fully connected layer and map it into a low-dimensional vector to obtain the third vector;

[0123] S205: Use the softmax function to probabilize the third vector, and use the joint loss function to calculate the loss of the probabilized third vector to obtain the loss value.

[0124] Furthermore, the step of adjusting the first training images contained in each of the first datasets to a preset format to obtain the second datasets corresponding to each of the first datasets includes:

[0125] S101: Adjust each of the first training images according to a preset image size, and standardize the pixel values ​​of each of the first training images to a preset range to obtain the second training images corresponding to each of the first training images.

[0126] The specific method for identifying the X-ray film is as follows:

[0127] A1: Obtain the X-ray image to be identified;

[0128] A2: Input the X to be identified into the identification model to predict the probability vector, wherein the identification model is trained by the model training method of the joint dataset as described above, and the training data used by the identification model is the image obtained by X-ray scanning;

[0129] A3: Based on the preset requirements, the negative probability and positive probability corresponding to the X-ray image to be identified are obtained by parsing from the probability vector;

[0130] A4: Based on the relationship between the negative probability and the positive probability, determine the identification result of the X-ray film to be identified.

[0131] Furthermore, the probability vector includes one negative vector value and multiple positive vector values, and the sum of the probabilities of the negative vector value and all positive vector values ​​in the probability vector is 1. The step of parsing the negative and positive probabilities corresponding to the X-ray image to be identified from the probability vector according to preset requirements includes:

[0132] A301: Use the negative vector value as the negative probability of the X-ray image to be identified, and retrieve the preset requirements;

[0133] A302: If the preset requirement is to reduce the misdiagnosis rate, then the positive vector value with the maximum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified;

[0134] A303: If the preset requirement is to reduce the false negative rate, then the positive vector value with the minimum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified.

[0135] Furthermore, the step of determining the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability includes:

[0136] A401: Determine whether the negative probability is greater than the positive probability;

[0137] A402: If the negative probability is greater than the positive probability, then the identification result is determined to be negative;

[0138] A403: If the negative probability is less than the positive probability, then the identification result is determined to be positive.

[0139] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this disclosure and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0140] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, first object, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, first object, or method. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, first object, or method that includes that element.

[0141] The above description is only a preferred embodiment of this disclosure and does not limit the patent scope of this disclosure. Any equivalent structural or procedural changes made based on the content of this disclosure and its drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A method for training a model using a joint dataset, wherein, include: Multiple first datasets of different categories are obtained, and the first training images contained in each first dataset are adjusted to a preset format to obtain the second datasets corresponding to each first dataset. Each of the second datasets is used as training data and input into the neural network for model training. During the model training process, the joint loss function is used to calculate the loss value of the second datasets after they are combined, and the network parameters of the feature extraction network corresponding to each of the second datasets are updated using the loss value. The process is iterated until the model converges to obtain the recognition model. The neural network includes multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. The number of feature extraction networks and the number of first fully connected layers are the same as the number of categories in the first dataset. Each feature extraction network is connected to a single first fully connected layer. The step of calculating the joint loss value of each of the second datasets using a classification loss function during model training includes: The second training images of each second dataset are respectively input into their respective feature extraction networks for feature extraction, thereby obtaining the feature maps corresponding to each second training image. Each feature map is input into its corresponding first fully connected layer and mapped to a high-dimensional vector to obtain a first vector corresponding to each feature map, wherein the lengths of the first vectors are the same. The first vectors are concatenated to obtain the second vector; The second vector is input into the second fully connected layer and mapped to a low-dimensional vector to obtain the third vector; The third vector is probabilized using the softmax function, and the loss is calculated using the joint loss function to obtain the loss value.

2. The model training method for the joint dataset according to claim 1, wherein, The step of adjusting the first training images contained in each of the first datasets to a preset format to obtain the second datasets corresponding to each of the first datasets includes: Each of the first training images is adjusted according to a preset image size, and the pixel values ​​of each of the first training images are standardized to a preset range to obtain the second training image corresponding to each of the first training images.

3. A method for identifying X-ray images, wherein, include: Obtain the X-ray image to be identified; The X to be identified is input into the identification model to predict the probability vector, wherein the identification model is trained by the model training method of the joint dataset as described in any one of claims 1-2, and the training data used by the identification model is an image obtained by X-ray scanning; According to the preset requirements, the negative probability and positive probability corresponding to the X-ray image to be identified are obtained by parsing from the probability vector; The identification result of the X-ray film to be identified is determined based on the relationship between the negative probability and the positive probability.

4. The X-ray image identification method according to claim 3, wherein, The probability vector contains one negative vector value and multiple positive vector values. The sum of the probabilities of the negative vector value and all positive vector values ​​in the probability vector is 1. The step of parsing the negative and positive probabilities corresponding to the X-ray image to be identified from the probability vector according to preset requirements includes: The negative vector value is used as the negative probability of the X-ray image to be identified, and the preset requirements are retrieved; If the preset requirement is to reduce the misdiagnosis rate, then the positive vector value with the maximum value is selected from each of the positive vector values ​​as the positive probability of the X-ray to be identified. If the preset requirement is to reduce the false negative rate, then the positive vector value with the minimum value is selected from all the positive vector values ​​as the positive probability of the X-ray to be identified.

5. The X-ray image identification method according to claim 3, wherein, The step of determining the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability includes: Determine whether the negative probability is greater than the positive probability; If the negative probability is greater than the positive probability, then the identification result is determined to be negative. If the negative probability is less than the positive probability, then the identification result is determined to be positive.

6. A model training apparatus for a joint dataset, wherein, include: The first acquisition module is used to acquire multiple first datasets of different categories, and adjust the first training images contained in each first dataset to a preset format to obtain the second datasets corresponding to each first dataset. The training module is used to input each of the second datasets as training data into the neural network for model training. During the model training process, the joint loss function is used to calculate the loss value of each of the second datasets after they are combined, and the network parameters of the feature extraction network corresponding to each of the second datasets are updated using the loss. The process is iterated until the model converges to obtain the recognition model. The neural network includes multiple feature extraction networks, multiple first fully connected layers, and one second fully connected layer. The number of feature extraction networks and the number of first fully connected layers are the same as the number of categories in the first dataset. Each feature extraction network is connected to a single first fully connected layer. The step of calculating the joint loss value of each of the second datasets using a classification loss function during model training includes: The second training images of each second dataset are respectively input into their respective feature extraction networks for feature extraction, thereby obtaining the feature maps corresponding to each second training image. Each feature map is input into its corresponding first fully connected layer and mapped to a high-dimensional vector to obtain a first vector corresponding to each feature map, wherein the lengths of the first vectors are the same. The first vectors are concatenated to obtain the second vector; The second vector is input into the second fully connected layer and mapped to a low-dimensional vector to obtain the third vector; The third vector is probabilized using the softmax function, and the loss is calculated using the joint loss function to obtain the loss value.

7. An X-ray image recognition device, wherein, include: The second acquisition module is used to acquire the X-ray image to be identified; The prediction module is used to input the X to be identified into the recognition model and predict the probability vector, wherein the recognition model is trained by the model training method of the joint dataset according to any one of claims 1-2, and the training data used by the recognition model is an image obtained by X-ray scanning. The parsing module is used to parse the negative probability and positive probability corresponding to the X-ray image to be identified from the probability vector according to preset requirements; The determination module is used to determine the identification result of the X-ray film to be identified based on the relationship between the negative probability and the positive probability.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein... When the processor executes the computer program, it implements the steps of the model training method for the joint dataset according to any one of claims 1-2 and / or the steps of the X-ray image recognition method according to any one of claims 3-5.

9. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the steps of the model training method for the joint dataset according to any one of claims 1-2 and / or the steps of the X-ray image recognition method according to any one of claims 3-5.

Images