A method, system and readable storage medium for expanding and enhancing deep learning training data
By generating new training data based on the classification and linear superposition method based on ray energy spectrum, the shortcomings of deep learning training data augmentation methods in the existing technology for radiation therapy dose calculations are solved, and the generalization ability and new data processing ability of the model are improved. Avoid overfitting.
Patent Information
- Application Number
- CN202111541880.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-12-16
AI Technical Summary
The data augmentation method used in image processing for deep learning training lacks effectiveness for radiation therapy dose calculation, which may lead to errors in new data or non-substantial increase, which cannot effectively compensate for the 'blank area' of the training set, affecting the model Generalization ability.
New training data is generated by classification and linear superposition based on ray energy spectrum, and the original data is classified and randomly selected using the data classification module and data training module. New training data is generated according to the linear superposition formula. This process is repeated to generate more A large number of training samples meet dosage principles and characteristics.
Effectively generate training data with new features, make up for the 'blank area' of the original training set, improve the generalization ability of the model and the processing ability of the new data, and avoid overfitting.
Smart Images

Figure CN114298177B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data enhancement technology, and in particular to a method, system and readable storage medium suitable for expanding and enhancing deep learning training data. Background Art
[0002] The biggest challenge in applying neural networks in the medical field is the lack of a large amount of labeled data. Using large-scale training sets that cover all situations as much as possible can improve the generalizability of the model and avoid the model's insufficient performance when encountering new data. However, unlike natural image datasets, medical imaging datasets are generally very small. The main reason is that obtaining anonymous and accurately labeled medical imaging data is a very laborious and difficult task. Similarly, in radiotherapy dose calculation, it is also difficult to traverse and obtain all dose distributions of different intensities and shapes. Therefore, the use of effective data augmentation technology fills the "blank area" of the training set and ensures that the trained model has generalization ability and is applicable to a variety of conditions. This is the key to whether deep learning can be implemented in dose calculation.
[0003] At present, the deep learning training for image processing uses technical enhancement, which includes: 1) Cropping. According to the size of the input data matrix, the original data is randomly cut to obtain different parts of data as new training data. 2) Scaling. The image is reduced or enlarged to generate new data. Each time new data is generated, a random reduction and enlargement ratio is generated within a preset scaling range, and the original image is reduced or enlarged. 3) Shift. The image is shifted horizontally and vertically to generate new data. Each time new data is generated, the horizontal and vertical translation distances are randomly generated within a preset translation distance range to generate new training data. 4) Rotation. The image is rotated to generate new data. Each time new data is generated, a certain set point is used as the rotation center, and a certain angle is randomly rotated within a preset rotation angle range to generate new data. 5) Flipping. The image is flipped horizontally or vertically to generate new data. Each time new data is generated, it is randomly determined whether to flip the horizontal or vertical position to generate new data.
[0004] Existing technologies do not effectively add useful new types of data to dose calculations based on the principles and characteristics of dosimetry, and may even generate erroneous new data, such as: 1) Cutting, because the dose at a certain point in dose calculation includes the contribution of surrounding scattered rays, so the cutting operation may cause the loss of some scattering information. 2) Scaling, particle interaction and migration are related to the actual physical distance, such as the inverse square law of distance. Simple scaling without modifying the corresponding value will cause the information reflected in this part to be incorrect. 3) Translation, rotation, and flipping, because when the relative positions of particles and medium distribution are fixed, the dose distribution generated is also fixed. Simple translation, rotation, and flipping operations do not generate new data, such as not covering the generation of new doses under different energy spectra, different beam intensities, and shapes, and do not generate substantially new data. Therefore, the present invention aims to provide a data enhancement method that can overcome the above-mentioned defects. Summary of the invention
[0005] The purpose of the present invention is to overcome the shortcomings of the prior art and provide a method, system and readable storage medium suitable for expanding and enhancing deep learning training data.
[0006] The object of the present invention is achieved by the following technical solution: A method for expanding and enhancing deep learning training data, applied to a data training server connected to multiple user devices, the method comprising at least the following steps: S1, obtaining original training data, and classifying the original training data through a data classification module based on the ray energy spectrum of the original training data, wherein the ray energy spectrum S with the same characteristic is classified into k The original training data are classified into the same category; S2, through the first data training module, based on the first random selection method, from the data with the same characteristic energy spectrum S k S3, through a second data training module, based on a second random selection method, select M second data from the first new data, and generate second new data from the M data in a second linear superposition manner; S4, repeat steps S2 and S3 to generate training samples.
[0007] Preferably, the first linear superposition method includes at least the following steps: S201, naming the N randomly selected data as (T 1,k ,D 1,k ), (T 2,k ,D 2,k )……(T N,k ,D N,k ); S202, the N data are linearly superimposed based on the first superposition formula to obtain the first new data (T' k ,D' k), the first superposition formula is Among them, T is the input data of the training set, D is the target data of the training set, and c i is the first stacking ratio, wherein the first stacking ratio c i is a random number between 0 and 1, i is the number of training steps, and the first stacking ratio c i satisfy
[0008] Preferably, the second linear superposition method includes at least the following steps: S301, naming the M randomly selected data as (T'1, D'1), (T'2, D'2) ... (T' M ,D' M ); S302, the M data are linearly superimposed based on a second superposition formula to obtain the second new data (T", D"), and the second superposition formula is Among them, d k is the second stacking ratio, wherein the second stacking ratio d k is a random number between 0 and 1, k is the number of training steps, and the second stacking ratio d k satisfy
[0009] Preferably, the number N of the first data is less than S k The total amount of original data in the classified data, the amount of secondary data M is less than S k Classification type.
[0010] Preferably, when steps S2 and S3 are repeated to generate training samples, the number N of the first data and the number M of the second data are configured to be randomly set, and the first stacking ratio c i and the second stacking ratio d k All are configured as random settings.
[0011] The present invention has the following advantages:
[0012] The present invention is a technology for enhancing training data during the training process by applying deep learning to dose calculation. It achieves the effect of effectively randomly generating training data with new features by classifying and superimposing according to the radiation energy, radiation intensity and shape based on the dosimetric principle and the characteristic of linear superposition, further effectively filling the "blank area" of the original training set, and achieving the purpose of improving the generalization ability of the training model and the ability to process new data to avoid overfitting. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 It is a flow chart of the expansion and enhancement method applicable to deep learning training data of the present invention;
[0014] Figure 2 It is a schematic diagram of the modular structure of the expansion and enhancement system of the present invention.
[0015] In the figure, 1-data training server, 2-user equipment, 3-data classification module, 4-first data training module, 5-second data training module. DETAILED DESCRIPTION
[0016] The present invention is further described below in conjunction with the accompanying drawings, and the protection scope of the present invention is not limited to the following:
[0017] like Figure 1 As shown, the present application provides a method for expanding and enhancing deep learning training data, which at least includes the following steps:
[0018] S1, obtaining original training data, and classifying the original training data based on the ray energy spectrum of the original training data, wherein the ray energy spectrum S with the same characteristic is classified into k The original training data are classified into the same category;
[0019] Specifically, the original training data is obtained according to the following steps:
[0020] S101, converting the image information into a medium material and electron density distribution map. The image is a CT image, and the HU value of the CT image is converted into an electron density distribution image according to the HU-electron density conversion curve of the machine that collects the CT image.
[0021] S102, according to the beam conditions and the material and electron density distribution of the medium provided by the image, calculate the amount of interaction between the initial incident photon and the medium TERMA. This implementation is calculated for monoenergetic. For a photon line with monoenergetic energy E, from the source To calculation point The TERMA calculation formula is: In the formula as well as In the grid and The attenuation coefficient of photons is related to the photon energy and the medium. The mass attenuation coefficient is obtained from the National Institute of Standards and Technology (NIST). for The medium density at is the beam flux distribution.
[0022] S103, training target data. The training target data is obtained by Monte Carlo dose algorithm, paired with input data during calculation, and generated under the same CT and the same radiation energy and distribution.
[0023] S104, training data generation conditions. The training data is generated under the following conditions: (1) The radiation energy is generated by randomly selecting an energy spectrum from the three medical linear accelerator beam energy spectra of 4MV, 10MV, and 24MV. (2) By randomly setting the field conditions, the field range is in the crossline direction, X1: -20cm~15cm, X2: -15cm~20cm, inline direction, Y1: -20cm~10cm, Y2: -10cm~20cm. The isocenter is randomly located in the phantom. According to the field conditions and the electron density distribution, the corresponding TERMA distribution and the dose distribution calculated by the target dose algorithm are calculated.
[0024] Preferably, having the same characteristic energy spectrum S k The original training data can be classified into the same category as follows: according to the generated original training data ray energy spectrum 4MV, 10MV, 24MV, the original training data is divided into three categories, represented by S1, S2, and S3 respectively.
[0025] S2, based on the first random selection method, from the energy spectrum S with the same characteristic k N first data are selected from the original training data, and the N first data are used to generate first new data in a first linear superposition manner.
[0026] Specifically, for the original training data with the same ray energy spectrum, five first data (T 1,k ,D 1,k ), (T 2,k ,D 2,k )……(T 5,k ,D 5,k ) Generate the first new data (T') according to the first linear superposition method k ,D' k ). Each energy spectrum generates one first new data, a total of three first new data, which are represented by (T'1, D'1), (T'2, D'2), and (T'3, D'3) respectively.
[0027] Preferably, the first linear superposition method includes at least the following steps:
[0028] S201, name the N randomly selected data as (T 1,k ,D 1,k ), (T 2,k ,D 2,k )……(T N,k ,DN,k );
[0029] S202, the N data are linearly superimposed based on a first superposition formula to obtain the first new data (T' k ,D' k ), the first superposition formula is Among them, T is the input data of the training set, D is the target data of the training set, and c i is the first stacking ratio, wherein the first stacking ratio c i is a random number between 0 and 1, i is the number of training steps, and the first stacking ratio c i satisfy
[0030] S3, based on a second random selection method, select M second data from the first new data, and generate second new data from the M data in a second linear superposition method.
[0031] Specifically, the first new data (T' k ,D' k ), three second data (T'1, D'1), (T'2, D'2), (T'3, D'3) are randomly selected, and the above three second data are used to generate second new data (T", D") in a second linear superposition manner.
[0032] Preferably, the second linear superposition method includes at least the following steps:
[0033] S301, the M randomly selected data are named (T'1, D'1), (T'2, D'2) ... (T' M ,D' M );
[0034] S302, the M data are linearly superimposed based on a second superposition formula to obtain the second new data (T", D"), where the second superposition formula is: Among them, d k is the second stacking ratio, wherein the second stacking ratio d k is a random number between 0 and 1, k is the number of training steps, and the second stacking ratio d k satisfy
[0035] S4, repeat steps S2 and S3 to generate training samples.
[0036] Specifically, the generated second new data (T", D") is sent to the network training process, and steps S2 and S3 are repeated to continue generating new data until the training is completed.
[0037] Preferably, the number N of the first data is less than Sk The total amount of original data in the classified data, the amount of secondary data M is less than S k When steps S2 and S3 are repeated to generate training samples, the number N of the first data and the number M of the second data are configured to be randomly set, and the first stacking ratio c i and the second stacking ratio d k All are configured as random settings.
[0038] Preferably, the enhancement technology for training data in the training process can be applied to model training including but not limited to convolutional neural network frameworks, etc. Different incident ray energy spectra include but are not limited to beam energy spectra at different acceleration voltages of existing medical linear accelerators, manually fitted different characteristic energy spectra, and also single energy spectra, etc. The dose calculation result as the training target can be obtained through any treatment planning system or software with dose calculation function, such as pencil beam algorithm, barrel convolution algorithm, Monte Carlo simulation, etc.
[0039] Preferably, Figure 2 As shown, the present application also provides an expansion and enhancement system suitable for deep learning training data, which at least includes a data training server 1 and several user devices 2. The data training server 1 is used to provide original training data. The data training server can transmit the trained training data to the user device 2 for application by the user device 2. The expansion and enhancement system also includes a data classification module 3, a first data training module 4 and a second data training module 5. The data classification module 3 is used to execute step S1. The first data training module 4 is used to execute step S2. The second data training module 5 is used to execute step S3.
[0040] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.
Claims
1. A method for augmenting deep learning training data, characterized in that: Applied to a data training server in communication connection with a plurality of user devices, the expansion and enhancement method comprises at least the following steps: S1, obtaining original training data, and classifying the original training data based on the ray energy spectrum of the original training data through a data classification module (3), wherein the data with the same characteristic energy spectrum S k The original training data are classified into the same category; S2, through the first data training module (4), based on the first random selection method, from the energy spectrum S with the same characteristic k N first data are selected from the original training data, and the N first data are used to generate first new data in a first linear superposition manner; the first linear superposition manner includes at least the following steps: S201, name the N randomly selected data as (T 1 ,k ,D 1 ,k )(T 2 ,k ,D 2 ,k )……(T N ,k ,D N ,k ); S202, the N data are linearly superimposed based on a first superposition formula to obtain the first new data (T' k ,D' k ), the first superposition formula is Among them, T is the input data of the training set, D is the target data of the training set, and c i is the first stacking ratio, wherein the first stacking ratio c i is a random number between 0 and 1, i is the number of training steps, and the first stacking ratio ci satisfies ; S3, through the second data training module (5), based on the second random selection method, M second data are selected from the first new data, and the M data are used to generate second new data in a second linear superposition method; the second linear superposition method includes at least the following steps: S301, the M randomly selected data are named (T'1, D'1), (T'2, D'2) ... (T' M ,D' M ); S302, the M data are linearly superimposed based on a second superposition formula to obtain the second new data (T", D"), where the second superposition formula is: Among them, d k is the second stacking ratio, wherein the second stacking ratio d k is a random number between 0 and 1, k is the number of training steps, and the second stacking ratio d k satisfy ; S4, repeating steps S2 and S3 to generate training samples; in the case of repeating steps S2 and S3 to generate training samples, the number N of the first data and the number M of the second data are configured as random settings, and the first superposition ratio c i and the second stacking ratio d k All are configured as random settings.
2. The method for expanding and enhancing deep learning training data according to claim 1, characterized in that: The number N of the first data is less than S k The total amount of original data in the classified data, the amount of secondary data M is less than S k Classification type.
3. A system for augmenting and enhancing deep learning training data, characterized in that: It includes a data training server and a plurality of user devices that are communicatively connected to the data training server, and the data training server is used to execute the expansion and enhancement method according to any one of claims 1-2.
4. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed, executes the expansion and enhancement method according to any one of claims 1 to 2.
Citation Information
Patent Citations
Multi-spectral image classification method based on threshold self-adaption and convolutional neural network
CN107392130A
three-dimensional medical image registration method based on a U-NET neural network
CN109872332A
X-ray mammary gland image deep learning classification method
CN110232396A
Cerebral microhemorrhage automatic detection method and system based on deep learning
CN110956634A
Method and device for simulating radiation dose field
CN111239791A
