Pet X-ray posture classification method, device and equipment based on deep learning
Through deep learning methods, deep convolutional neural network model is trained to realize automatic classification of pet X-ray postures, solving the problem of low diagnosis and misdiagnosis caused by pet doctors' need to read videos manually, and improving the diagnostic accuracy and efficiency of pet medical care .
Patent Information
- Application Number
- CN202210065098.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-01-20
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2042-01-20
AI Technical Summary
In the field of pet medical care, there are not many researches on the classification of pet X-ray postures based on deep learning, which leads to pet doctors needing to manually read the film to classify positive lateral images, resulting in low diagnostic efficiency, long time-consuming and prone to missed diagnosis and misdiagnosis.
Deep learning methods are adopted to design deep convolutional neural network structures through data preparation, labeling, augmentation and model construction, and to train classification models to realize automatic classification of pet X-ray poses, reducing doctors’ reading time and work intensity.
It improves the efficiency of pet doctors' reading of films, reduces missed diagnosis and misdiagnosis, and realizes the rapid and accurate classification of pet X-ray postures, assisting doctors in making more accurate diagnosis.
Smart Images

Figure CN114445658B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence medical treatment and medical imaging diagnosis, and specifically relates to a pet X-ray posture classification method, device and equipment based on deep learning. Background Art
[0002] DR (Digital Radiography) system (its shooting principle is as follows Figure 6 The DR is a direct digital X-ray photography system. It uses digital photosensitive plates to collect data based on the principle of ordinary radiation projection, and then forms an image after comprehensive analysis and arrangement by a computer. The X-ray digital images taken by DR have higher spatial resolution, larger dynamic range, and higher image quality. It is a commonly used detection device in many large hospitals.
[0003] There are two types of postures for pets when taking DR images: frontal and lateral. Frontal means that the pet faces the radiation probe from the front or back, presenting a two-dimensional plane image with the front and back overlapping; lateral means that the pet faces the radiation probe from the right or left side, and the radiation penetrates from one side of the body to the other side.
[0004] In the field of medical diagnosis based on deep learning, convolutional neural networks are usually used to extract image features, then classify the features and predict the state of the image.
[0005] Currently, in the field of human CT medicine, convolutional neural networks can be used to train classification models, and the classification models can predict and classify the left and right hands of the human body.
[0006] At present, in the field of pet medicine, X-rays are one of the most commonly used means of diagnosing pet diseases. X-rays are usually taken by an X-ray photography system (DR). Due to the high resolution of pet X-rays taken by DR, the size of the pet is not fixed, the image density difference is small, foreign matter, and image labels are missing. There are not many studies on pet X-ray posture classification based on deep learning. Therefore, pet doctors need to manually read the films to classify and screen the frontal and lateral images to diagnose diseases.
[0007] Usually when taking X-rays of pets, the pets are photographed in frontal and side positions. X-rays taken in different pet positions correspond to the diagnosis of diseases in different parts of the body.
[0008] Veterinarians diagnose related diseases by viewing the frontal or lateral images. For example, veterinarians can see symptoms such as abdominal edema and pleural effusion by observing the lateral image; and symptoms such as spinal injuries by observing the frontal image.
[0009] When diagnosing diseases based on manual X-rays, veterinarians are required to read the films to classify and select the frontal and lateral images. This process will take a long time when there is a large amount of case image data. In addition, there is a lot of repetitive work in the manual film reading process. The diagnostic efficiency of veterinarians is low and it is easy to miss or misdiagnose. Summary of the invention
[0010] In order to solve the defects and shortcomings in the above-mentioned prior art, the present invention provides a pet X-ray posture classification method, device and equipment based on deep learning that uses a deep learning method to assist doctors in classifying frontal and lateral images. The classification has high accuracy and reduces missed diagnoses and misdiagnoses. It can reduce the pet doctor's reading time, reduce the doctor's workload, and effectively improve the doctor's reading efficiency, thereby improving the diagnostic efficiency.
[0011] The technical solution of the present invention is a pet X-ray posture classification method based on deep learning, comprising the following steps:
[0012] 1) Data preparation: collect pet X-ray images taken by DR equipment in clinical practice, and filter out images that cannot be used for posture classification, such as images whose posture cannot be distinguished and the objects are not pets;
[0013] 2) Data labeling: Generate corresponding label files for the collected pet image labeling categories, where the label categories are frontal and lateral views;
[0014] 3) Data augmentation: augment the labeled pet image data in a preset manner;
[0015] 4) Posture classification model construction: Design the deep convolutional neural network structure, network parameters, loss function, and optimizer according to the preset method, and then send the pet image as a sample into the network training classification model to train the classification model;
[0016] 5) Pet posture prediction: Input pet images into the classification model and output the predicted pet posture.
[0017] Preferably, the preset data augmentation method in step 3) includes:
[0018] a. Rotate the original image 90, 180, or 270 degrees clockwise;
[0019] b. Mirror and flip the original image vertically;
[0020] c. Rotate the flipped image clockwise by 90, 180, and 270 degrees;
[0021] d. Randomly rotate and shake the original image with the center of the image as the center, and the shaking amplitude is within 10 degrees up and down;
[0022] e. Normalize the image data after being augmented by a, b, c, and d;
[0023] f. Randomly shuffle the image data after step e.
[0024] Preferably, the preset posture classification model construction parameter settings in step 4) include:
[0025] a. The convolutional neural network model structure includes the image feature extraction stage and the classification stage. The feature extraction stage uses the first six stages of the pre-trained model detnet59 as the backbone network to extract the image features of pet images. The classification stage includes the FC layer of detnet59 plus a binary classification fully connected layer for frontal and lateral classification;
[0026] b. The training parameters of the model include setting the number of training rounds epoch = 100, the training batch size batch_size = 16, the learning rate learning_rate = 0.0001, the optimizer uses the SGD algorithm, and the network uses the softmax + cross entropy loss function CE and then weighted as the loss function L(X, y);
[0027] L(X,y)=-W p ylog(P(Y=1|X))-W n (1-y)log(P(Y=0|X))
[0028] Among them, y represents the prediction result of the deep convolutional network on the pet's posture, P(Y=1|X) represents the probability that the network predicts the pet's posture to be upright, and P(Y=0|X) represents the probability that the network predicts the pet's posture to be sideways. W p and W n They respectively represent the proportion of positive examples (front position) and negative examples (side position) in the samples.
[0029] A pet X-ray posture classification device based on deep learning is applied to a pet X-ray posture classification method based on deep learning, including a module component for training a posture classification model and a module component for predicting pet posture, wherein the module component for training the posture classification model includes a sample acquisition module, a sample annotation module, a sample preprocessing module and a classification model training module; the module component for predicting pet posture includes a pet image acquisition module, a pet image preprocessing module, a pet image recognition module and a pet image prediction module.
[0030] Preferably, wherein:
[0031] A sample acquisition module is used to acquire several pet X-ray images taken by DR equipment;
[0032] The sample annotation module is used to obtain the pet posture labels corresponding to the pet X-ray images; the labels are divided into positive examples (frontal position) and negative examples (lateral position), and then a variety of data enhancement methods are used to expand the samples to obtain the training set data;
[0033] Sample preprocessing module: used to preprocess the pet images input to the network. First, the images are normalized and then cropped to a resolution of 512×512. At the same time, the labels of the training data are digitally encoded, with positive examples marked as 1 and negative examples marked as 0.
[0034] The classification model training module is used to input the preprocessed pet posture training data into the deep convolutional neural network to determine the type of pet posture; the model parameters of the deep convolutional neural network are determined after tuning and training based on the training samples obtained from the sample labeling module; the pet posture types include frontal position and lateral position.
[0035] Preferably, the classification model training module further includes:
[0036] a. Model training submodule, which is connected to the sample preprocessing module, is used to extract n images from the training set data, send them to the classification network for forward propagation, and obtain the predicted probability of the pet's posture, where n is the preset batch_size and the predicted probability is the probability that the pet's posture is in the correct position;
[0037] b. Loss calculation submodule, which connects the model training submodule and the sample preprocessing module and calculates the loss value in the following way;
[0038]
[0039] Among them, N = 16 is the batch_size, y i represents the prediction result of the posture of the i-th pet picture in this batch, P(Y=1|X) represents the probability that the network predicts the pet posture to be upright, and P(Y=0|X) represents the probability that the network predicts the pet posture to be sideways. W p and W n Respectively represent the proportion of positive examples (front position) and negative examples (side position) in the samples of this batch;
[0040] c. The parameter update submodule is connected to the loss calculation submodule and uses the SGD gradient descent algorithm to pass the loss back to update the neural network parameters;
[0041] d. The training control submodule is connected to the parameter update submodule, the loss calculation submodule, and the model training submodule, and is used to control the network training process to end when the loss value is lower than the threshold.
[0042] Preferably, wherein:
[0043] A pet image acquisition module, used to acquire the pet X-ray to be predicted;
[0044] A pet image preprocessing module, connected to the pet image acquisition module, is used to preprocess the pet image to be predicted, and perform standardized cropping and normalization processing on the image;
[0045] The pet image recognition module is connected to the classification model training module and the pet image preprocessing module respectively, and is used to input the pet image to be predicted into the trained deep convolutional neural network to obtain the probability that the pet posture in the image is in the correct position;
[0046] The pet image prediction module is connected to the pet image recognition module and is used to make a judgment based on the probability value of the pet posture being in the upright position; if the probability of the pet posture being in the upright position is not less than the preset separation threshold between the upright position and the side position, the pet posture is judged to be in the upright position; otherwise, the pet posture is judged to be in the side position.
[0047] A pet X-ray posture classification device based on deep learning includes a memory and a processor, wherein the memory is used to store a computer program of an X-ray posture classification algorithm based on deep learning; the processor executes the computer program to implement the pet X-ray posture classification method based on deep learning as described above.
[0048] The beneficial effects of the present invention are as follows:
[0049] 1. The present invention starts from the actual clinical diagnosis process of pet medical care, and aims to solve the problem that pet doctors need to classify and screen pet postures when reading films, which results in a long time and low reading efficiency. A method and device for classifying pet X-ray postures based on deep learning is invented.
[0050] 2. The present invention achieves the effect of predicting the posture of pet X-rays using deep neural networks by organizing, labeling, and training a posture classification model based on a convolutional neural network for pet X-ray data.
[0051] 3. The present invention can improve the efficiency of pet doctors in reading films by quickly determining the pet's posture category and providing the pet doctor with the desired posture image, and can also assist the pet doctor in making a more accurate posture classification diagnosis.
[0052] 4. The present invention can automatically screen out lateral pet images by computer, and has a better image for the abdominal area segmentation task of pet X-rays based on deep learning.
[0053] 5. The present invention can also be used by pet doctors to filter out images with the same pet posture when searching for similar images, thereby improving the accuracy of the pet doctors in searching for similar images. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 This is a flow chart of the pet X-ray posture classification method based on deep learning in the present invention;
[0055] Figure 2 It is a module schematic diagram of a pet X-ray posture classification device based on deep learning in the present invention;
[0056] Figure 3 This is a diagram of the pet X-ray posture classification network architecture of the present invention;
[0057] Figure 4 This is a schematic diagram of a pet's frontal image in the present invention;
[0058] Figure 5 This is a schematic diagram of a pet side image in the present invention;
[0059] Figure 6 This is a schematic diagram of the DR system mentioned in the background technology of the present invention. DETAILED DESCRIPTION
[0060] The present invention will be further described in detail below in conjunction with the accompanying drawings, but this does not limit the protection scope of the present invention.
[0061] like Figure 1 As shown, a pet X-ray posture classification method based on deep learning includes the following steps:
[0062] 1) Data preparation: collect pet X-ray images taken by DR equipment in clinical practice, and filter out images that cannot be used for posture classification, such as images whose posture cannot be distinguished and the objects are not pets;
[0063] 2) Data labeling: Generate corresponding label files for the collected pet image labeling categories, where the label categories are frontal and lateral views;
[0064] 3) Data augmentation: augment the labeled pet image data in a preset manner;
[0065] 4) Posture classification model construction: Design the deep convolutional neural network structure, network parameters, loss function, and optimizer according to the preset method, and then send the pet image as a sample into the network training classification model to train the classification model;
[0066] 5) Pet posture prediction: Input pet images into the classification model and output the predicted pet posture.
[0067] The preset data augmentation methods in step 3) include:
[0068] a. Rotate the original image 90, 180, or 270 degrees clockwise;
[0069] b. Mirror and flip the original image vertically;
[0070] c. Rotate the flipped image clockwise by 90, 180, and 270 degrees;
[0071] d. Randomly rotate and shake the original image with the center of the image as the center, and the shaking amplitude is within 10 degrees up and down;
[0072] e. Normalize the image data after being augmented by a, b, c, and d;
[0073] f. Randomly shuffle the image data after step e.
[0074] The preset posture classification model construction parameter settings in step 4) include:
[0075] a. The convolutional neural network model structure includes the image feature extraction stage and the classification stage. The feature extraction stage uses the first six stages of the pre-trained model detnet59 as the backbone network to extract the image features of pet images. The classification stage includes the FC layer of detnet59 plus a binary classification fully connected layer for frontal and lateral classification;
[0076] b. The training parameters of the model include setting the number of training rounds epoch = 100, the training batch size batch_size = 16, the learning rate learning_rate = 0.0001, the optimizer uses the SGD algorithm, and the network uses the softmax + cross entropy loss function CE and then weighted as the loss function L(X, y);
[0077] L(X,y)=-W p ylog(P(Y=1|X))-W n (1-y)log(P(Y=0|X))
[0078] Among them, y represents the prediction result of the deep convolutional network on the pet's posture, P(Y=1|X) represents the probability that the network predicts the pet's posture to be upright, and P(Y=0|X) represents the probability that the network predicts the pet's posture to be sideways. W p and W n They respectively represent the proportion of positive examples (front position) and negative examples (side position) in the samples.
[0079] like Figure 2-5As shown, a pet X-ray posture classification device based on deep learning includes a module component for training a posture classification model and a module component for predicting a pet posture, wherein the module component for training the posture classification model includes a sample acquisition module, a sample annotation module, a sample preprocessing module and a classification model training module; the module component for predicting a pet posture includes a pet image acquisition module, a pet image preprocessing module, a pet image recognition module and a pet image prediction module.
[0080] in:
[0081] A sample acquisition module is used to acquire several pet X-ray images taken by DR equipment;
[0082] The sample annotation module is used to obtain the pet posture labels corresponding to the pet X-ray images; the labels are divided into positive examples (frontal position) and negative examples (lateral position), and then a variety of data enhancement methods are used to expand the samples to obtain the training set data;
[0083] Sample preprocessing module: used to preprocess the pet images input to the network. First, the images are normalized and then cropped to a resolution of 512×512. At the same time, the labels of the training data are digitally encoded, with positive examples marked as 1 and negative examples marked as 0.
[0084] The classification model training module is used to input the preprocessed pet posture training data into the deep convolutional neural network to determine the type of pet posture; the model parameters of the deep convolutional neural network are determined after tuning and training based on the training samples obtained from the sample labeling module; the pet posture types include frontal position and lateral position.
[0085] Among them, the classification model training module also includes:
[0086] a. Model training submodule, which is connected to the sample preprocessing module, is used to extract n images from the training set data, send them to the classification network for forward propagation, and obtain the predicted probability of the pet's posture, where n is the preset batch_size and the predicted probability is the probability that the pet's posture is in the correct position;
[0087] b. Loss calculation submodule, which connects the model training submodule and the sample preprocessing module and calculates the loss value in the following way;
[0088]
[0089] Among them, N = 16 is the batch_size, y i represents the prediction result of the posture of the i-th pet picture in this batch, P(Y=1|X) represents the probability that the network predicts the pet posture to be upright, and P(Y=0|X) represents the probability that the network predicts the pet posture to be sideways. W p and W n Respectively represent the proportion of positive examples (front position) and negative examples (side position) in the samples of this batch;
[0090] c. The parameter update submodule is connected to the loss calculation submodule and uses the SGD gradient descent algorithm to pass the loss back to update the neural network parameters;
[0091] d. The training control submodule is connected to the parameter update submodule, the loss calculation submodule, and the model training submodule, and is used to control the network training process to end when the loss value is lower than the threshold.
[0092] in:
[0093] A pet image acquisition module, used to acquire the pet X-ray to be predicted;
[0094] A pet image preprocessing module, connected to the pet image acquisition module, is used to preprocess the pet image to be predicted, and perform standardized cropping and normalization processing on the image;
[0095] The pet image recognition module is connected to the classification model training module and the pet image preprocessing module respectively, and is used to input the pet image to be predicted into the trained deep convolutional neural network to obtain the probability that the pet posture in the image is in the correct position;
[0096] The pet image prediction module is connected to the pet image recognition module and is used to make a judgment based on the probability value of the pet posture being in the upright position; if the probability of the pet posture being in the upright position is not less than the preset separation threshold between the upright position and the side position, the pet posture is judged to be in the upright position; otherwise, the pet posture is judged to be in the side position.
[0097] The present invention also provides a pet X-ray posture classification device based on deep learning, comprising a memory and a processor, wherein the memory is used to store a computer program of an X-ray posture classification algorithm based on deep learning; the processor executes the computer program to implement a pet X-ray posture classification method based on deep learning.
[0098] The main innovative features of the present invention are as follows:
[0099] A method for classifying and predicting the posture of pet X-rays based on deep learning is invented. In pet medical clinical diagnosis, a device for AI posture classification of pet X-rays taken by DR equipment is invented. It is used to assist pet doctors in reading films. In order to replace the need to classify pet images when reading films manually, the present invention proposes for the first time a classification network model based on deep learning to classify and predict the posture of pet images.
Claims
1. A pet X-ray posture classification method based on deep learning, Features: It includes the following steps: Data preparation: collect pet X-ray images taken by DR equipment in clinic and filter out images that cannot be used for posture classification; Data labeling: Generate corresponding label files for the collected pet image labeling categories, where the label categories are frontal and lateral views; Data augmentation: augment the labeled pet image data in a preset way; Posture classification model construction: Design the deep convolutional neural network structure, network parameters, loss function, and optimizer according to the preset method, and then send the pet images as samples to the network training classification model to train the classification model; Pet pose prediction: Input a pet image to the classification model, and the predicted pet pose will be output; The preset posture classification model construction parameter settings in step 4) include: a. The convolutional neural network model structure includes the image feature extraction stage and the classification stage. The feature extraction stage uses the first six stages of the pre-trained model detnet59 as the backbone network to extract the image features of pet images. The classification stage includes the FC layer of detnet59 plus a binary classification fully connected layer for frontal and lateral classification; b. The training parameters of the model include setting the number of training rounds epoch=100, the training batch size batch_size=16, the learning rate learning_rate=0.0001, the optimizer uses the SGD algorithm, and the network uses the softmax+cross entropy loss function CE and then weighted as the loss function ; ; Among them, y represents the prediction result of the deep convolutional network on the pet's posture, represents the probability that the network predicts the pet's posture is correct, represents the probability that the network predicts the pet's posture to be sideways, , , and They respectively represent the proportion of positive examples (front position) and negative examples (side position) in the samples.
2. A pet X-ray posture classification method based on deep learning according to claim 1, Features: The preset data augmentation methods in step 3) include: a. Rotate the original image 90, 180, or 270 degrees clockwise; b. Mirror and flip the original image vertically; c. Rotate the flipped image clockwise by 90, 180, and 270 degrees; d. Randomly rotate and shake the original image with the center of the image as the center, and the shaking amplitude is within 10 degrees up and down; e. Normalize the image data after being augmented by a, b, c, and d; f. Randomly shuffle the image data after step e.
3. A pet X-ray posture classification device based on deep learning, applied to the pet X-ray posture classification method based on deep learning as described in any one of claims 1-2, Features: It includes module components for training posture classification models and module components for predicting pet postures, wherein the module components for training posture classification models include sample acquisition modules, sample labeling modules, sample preprocessing modules and classification model training modules; the module components for predicting pet postures include pet image acquisition modules, pet image preprocessing modules, pet image recognition modules and pet image prediction modules.
4. A pet X-ray posture classification device based on deep learning according to claim 3 , Features: in: A sample acquisition module is used to acquire several pet X-ray images taken by DR equipment; The sample annotation module is used to obtain the pet posture labels corresponding to the pet X-ray images; the labels are divided into positive examples (frontal position) and negative examples (lateral position), and then a variety of data enhancement methods are used to expand the samples to obtain the training set data; Sample preprocessing module: used to preprocess the pet images input to the network. First, the images are normalized and then cropped to a resolution of 512×512. At the same time, the labels of the training data are digitally encoded, with positive examples marked as 1 and negative examples marked as 0. The classification model training module is used to input the preprocessed pet posture training data into the deep convolutional neural network to determine the type of pet posture; the model parameters of the deep convolutional neural network are determined after tuning and training based on the training samples obtained from the sample labeling module; the pet posture types include frontal position and lateral position.
5. A pet X-ray posture classification device based on deep learning according to claim 4 , Features: in, The classification model training module also includes: a. Model training submodule, which is connected to the sample preprocessing module, is used to extract n images from the training set data, send them to the classification network for forward propagation, and obtain the predicted probability of the pet's posture, where n is the preset batch_size and the predicted probability is the probability that the pet's posture is in the correct position; b. Loss calculation submodule, which connects the model training submodule and the sample preprocessing module and calculates the loss value in the following way; ; Among them, N=16 is the batch_size size, represents the predicted result of the pose of the i-th pet picture in this batch, represents the probability that the network predicts the pet's posture is correct, represents the probability that the network predicts the pet's posture to be sideways, , , and Respectively represent the proportion of positive examples (front position) and negative examples (side position) in the samples of this batch; c. The parameter update submodule is connected to the loss calculation submodule and uses the SGD gradient descent algorithm to pass the loss back to update the neural network parameters; d. The training control submodule is connected to the parameter update submodule, the loss calculation submodule, and the model training submodule, and is used to control the network training process to end when the loss value is lower than the threshold.
6. A pet X-ray posture classification device based on deep learning according to claim 3, It is characterized in that: A pet image acquisition module, used to acquire the pet X-ray to be predicted; A pet image preprocessing module, connected to the pet image acquisition module, is used to preprocess the pet image to be predicted, and perform standardized cropping and normalization processing on the image; The pet image recognition module is connected to the classification model training module and the pet image preprocessing module respectively, and is used to input the pet image to be predicted into the trained deep convolutional neural network to obtain the probability that the pet posture in the image is in the correct position; A pet image prediction module, connected to the pet image recognition module, is used to make a determination based on the probability value of the pet's posture being in the correct position; If the probability that the pet's posture is in the upright position is not less than a preset separation threshold between the upright position and the side position, the pet's posture is determined to be in the upright position; otherwise, the pet's posture is determined to be in the side position.
7. A pet X-ray posture classification device based on deep learning, Features: It includes a memory and a processor, wherein the memory is used to store a computer program of an X-ray posture classification algorithm based on deep learning; the processor executes the computer program to implement the pet X-ray posture classification method based on deep learning as described in any one of claims 1-2.
