A difficult sample mining method, system, terminal and storage medium
By constructing point cloud shape models and template shapes, using training loss values to update reward expectations, mining difficult samples from medical image data, solving the problem of category imbalance in the existing technology, and improving the accuracy and speed of medical image segmentation models.
Patent Information
- Application Number
- CN202111156861.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-09-30
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-09-30
AI Technical Summary
In the organ segmentation task of three-dimensional medical images, existing random sampling strategies lead to excessive sampling of categories with large voxel proportions, resulting in poor results in small organ segmentation, and difficult to identify anatomical area boundaries and inter-individual differences. Large structural segmentation has the problem of missed or missed detection.
By building a point cloud shape model, select template shapes containing multiple label types, use training loss values to update the reward expectation value of template shapes, and mine difficult samples from medical image data for training based on the reward value distribution, reducing memory and computing resources Consumption, scientifically evaluate the degree of sample difficulty and fully explore representative samples of difficult problems.
It effectively alleviates the problem of category imbalance in model training, improves segmentation accuracy and model fitting speed, and ensures accurate segmentation of the boundaries of small organs and organs with low contrast.
Smart Images

Figure CN113920079B_ABST
Abstract
Description
Technical Field
[0001] The present application belongs to the field of medical image processing technology, and in particular, relates to a difficult sample mining method, system, terminal and storage medium. Background Art
[0002] In the organ segmentation task based on 3D medical images such as CT (Computed Tomography) or MR (Magnetic Resonance), it is usually necessary to crop 3D patches (i.e., 3D local parts in the original 3D medical image) from the image as samples to train the segmentation model. When sampling the original 3D medical image in the form of 3D patches, there is a problem of sample selection. The commonly used random sampling strategy usually faces the following two problems: First, the sampling probabilities of different categories are different, and the sampling probability of categories with a large voxel ratio will be greater, so that the final trained model has a better segmentation effect on background areas or large organs (such as the liver), but often misses the segmentation of small organs (such as the adrenal gland); second, it does not pay enough attention to anatomical areas that are difficult to identify, such as low-contrast organ boundaries (such as the pancreas) or anatomical structures with large differences between individuals (such as the stomach), so that the final trained model has missed or misdetected the segmentation of these areas. Introducing a difficult sample mining strategy during the training process can effectively alleviate the above problems. That is, during the training process, samples that are difficult to segment for the current model are mined from the training set, forcing the model to learn the features of these difficult samples, thereby improving the overall segmentation accuracy of the model.
[0003] Existing difficult sample mining solutions include:
[0004] First, in each training batch, a large number of samples are randomly selected for forward propagation to calculate the loss, and then the samples with larger losses are selected as difficult samples, and these difficult samples are used for back propagation to update the model parameters. In this scheme, due to the large memory of 3Dpatch samples, only a small number of samples can be selected in each training batch, that is, the batch size is small, and it is not comprehensive to select samples with larger losses as difficult samples from a small number of candidate samples; in addition, the forward propagation of 3Dpatch is computationally intensive, and only selecting some samples for back propagation will result in many useless calculations, which is not very suitable for the field of medical image segmentation.
[0005] Second, maintain a sampling probability for each sample. During the training process, update the sampling probability based on the number of times the sample is sampled and the average loss value. Samples with fewer sampling times or larger average loss values have a higher sampling probability. In this scheme, since the size of the original three-dimensional medical images is usually large, there are a lot of 3D patch samples that can be mined from the training set. Maintaining and updating a sampling probability for each sample during the training process requires high memory and computing resources, and is also not very suitable for the field of medical image segmentation. Summary of the invention
[0006] The present application provides a difficult sample mining method, system, terminal and storage medium, aiming to solve at least one of the above-mentioned technical problems in the prior art to a certain extent.
[0007] In order to solve the above problems, this application provides the following technical solutions:
[0008] A difficult sample mining method, comprising:
[0009] Collect medical image data including K types of labels;
[0010] Constructing a point cloud shape model according to the labels of the medical image data, and selecting a point cloud shape model containing K types of labels as a template shape;
[0011] Randomly cutting a set number of training samples from the medical image data, inputting the training samples into a segmentation model for training, and after a batch of training is completed, updating the expected reward value of the template shape according to the training loss value of the current batch of training samples;
[0012] When the training batch of the segmentation model reaches a set number of times, difficult samples are mined from the medical image data according to the distribution of expected rewards on the template shape, and the segmentation model is trained according to the difficult samples.
[0013] The technical solution adopted by the embodiment of the present application also includes: constructing a point cloud shape model according to the label of the medical image data includes:
[0014] For each label L in the medical image data i (i∈[1,N]), N is the number of medical image data, and the organ corresponding to each label is meshed by the marching cube algorithm to obtain the surface mesh of each organ;
[0015] All surface meshes are downsampled by the decimation algorithm, and adjacent meshes with similar normal vectors are approximately merged into one mesh;
[0016] The vertices of the merged surface mesh are extracted, and a multi-organ point cloud shape model is formed according to the vertices.
[0017] The technical solution adopted by the embodiment of the present application also includes: the selecting a point cloud shape model containing K types of labels as a template shape includes:
[0018] Assume that based on label L i The resulting point cloud shape model is S i , where the shape model of the kth organ is In S i∈[1,N] Randomly select a shape model containing K types of labels as the template shape S m .
[0019] The technical solution adopted by the embodiment of the present application also includes: constructing a point cloud shape model according to the labels of the medical image data, and selecting a point cloud shape model containing K types of labels as a template shape also includes:
[0020] For the point cloud shape model S i For each point in, the coordinates are maintained separately and the normal vector;
[0021] For the template shape S m For each point in the, respectively maintain the point index, the organ category of the point, the adjacent point, the sampling frequency c, the reward average And the expected reward value v.
[0022] The technical solution adopted by the embodiment of the present application also includes: constructing a point cloud shape model according to the labels of the medical image data, and selecting a point cloud shape model containing K types of labels as a template shape also includes:
[0023] The corresponding relationship between the template shape and the point clouds of all organs in each point cloud shape model is calculated based on the elastic point cloud registration algorithm;
[0024] The point cloud correspondence calculation method includes:
[0025] The template shape S m The point cloud of the kth organ in With the i-th point cloud shape model S i The point cloud of the kth organ in Perform registration to obtain the mapping matrix between the two point clouds
[0026] The point cloud of each organ in each point cloud shape model is registered with the point cloud of the corresponding organ in the template shape. After (N-1)×K registrations, a point cloud mapping matrix of all organs in the template shape and the point cloud shape model is obtained.
[0027] The technical solution adopted by the embodiment of the present application also includes: randomly cutting a set number of training samples from the medical image data and inputting the training samples into the segmentation model for training specifically:
[0028] A training sample is cropped from the i-th medical image data, and a point cloud shape model S is obtained according to the cropping coordinate range of the training sample. i Retrieve the point set contained in the training sample from the point cloud
[0029] For each point in the point set Get the training sample voxels within the radius r and calculate the mean training loss value of all voxels to get the point The reward value obtained in this round of training
[0030] According to the template shape S m With point cloud shape model S i The mapping matrix between Get point set In S m The corresponding point set in
[0031] Use the reward value obtained in this round of training Update Point The sampling frequency c maintained in i , average reward And the expected reward value v i :
[0032]
[0033] c i ←c i +1
[0034]
[0035] In the above formula, the reward average is updated in a weighted average manner β is the weight, and the expected reward value v is calculated using the upper confidence interval algorithm i .
[0036] The technical solution adopted by the embodiment of the present application also includes: mining difficult samples from the medical image data according to the distribution of the expected value of the reward on the template shape includes:
[0037] For the template shape S m Each point q in i , calculate the expected average reward value of the point and its adjacent points, and get iThe average reward expectation of the local anatomical structure centered on And in S m Middle front 2×B large Randomly select a point q opt , get the value of point q opt The point set {q1,q2,…,q n}; Where B is the preset batch size;
[0038] Select an image I arbitrarily from the medical image data i , and obtain image I i Point cloud shape model S i , through the mapping matrix Get the point set {q1,q2,…,q n}In S i The corresponding point set {p1,p2,…,p n};
[0039] According to the point set {q1,q2,…,q n}The expected value of reward v i with {p1,p2,…,p n}'s coordinates Calculate the point set {p1,p2,…,p n}In S i The geometric center of
[0040]
[0041] Calculate {p1,p2,…,p n The normal vector δ of each point in i The average value of the normal vector And the geometric center Along or against the mean normal vector The direction of translation is
[0042] by Centered on the image I i Difficult samples to collect.
[0043] Another technical solution adopted by the embodiment of the present application is: a difficult sample mining system, comprising:
[0044] Data collection module: used to collect medical image data including K types of labels;
[0045] Model building module: used to build a point cloud shape model according to the labels of the medical image data, and select a point cloud shape model containing K types of labels as a template shape;
[0046] The first model training module is used to randomly cut a set number of training samples from the medical image data, input the training samples into the segmentation model for training, and update the reward expectation value of the template shape according to the training loss value of the current batch of training samples after a batch of training is completed;
[0047] A sample mining module: used for mining difficult samples from the medical image data according to the expected value distribution of rewards on the template shape after the training batch of the segmentation model reaches a set number of times;
[0048] The second model training module is used to train the segmentation model according to the difficult samples.
[0049] Another technical solution adopted by the embodiment of the present application is: a terminal, the terminal includes a processor and a memory coupled to the processor, wherein:
[0050] The memory stores program instructions for implementing the difficult sample mining method;
[0051] The processor is used to execute the program instructions stored in the memory to control difficult sample mining.
[0052] Another technical solution adopted by the embodiment of the present application is: a storage medium storing program instructions executable by a processor, wherein the program instructions are used to execute the difficult sample mining method.
[0053] Compared with the prior art, the beneficial effects of the embodiments of the present application are as follows: the difficult sample mining method, system, terminal and storage medium of the embodiments of the present application align the point cloud between the template shape and all the point cloud shape models in the training set image, and build the association between the training set images about the anatomical structure, so that only the attributes in the template shape need to be maintained and updated in the model training, which greatly reduces the consumption of memory and computing resources. By abstracting the difficulty of the training sample into an estimable reward value, the UCB algorithm is used to iteratively update the reward value expectation of the template shape based on the number of sampling times and the historical loss value during model training, and the difficult samples in the training set image are mined according to the reward value distribution in the template shape, so as to more scientifically evaluate the difficulty of the sample, and more fully mine the difficult samples that are more representative (difficult to segment) for the segmentation model, effectively alleviate the class imbalance problem in model training, improve model accuracy and accelerate model fitting. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 is a flowchart of a difficult sample mining method according to an embodiment of the present application;
[0055] Figure 2Schematic diagram of template shape maintenance in an embodiment of the present application; wherein (a) is a schematic diagram of point cloud shape model maintenance, and (b) is a schematic diagram of template shape maintenance;
[0056] Figure 3 This is a graph of the training loss function under different sampling strategies;
[0057] Figure 4 Schematic diagram of the segmentation effect of the segmentation model on the test set data under different sampling strategies, where (a), (b), (c), and (d) respectively show the anatomical structures of the right adrenal gland, the boundary of the esophagus and stomach, the pancreas, and the portal vein and splenic vein;
[0058] Figure 5 A schematic diagram of the structure of a difficult sample mining system according to an embodiment of the present application;
[0059] Figure 6 This is a schematic diagram of the terminal structure of an embodiment of the present application;
[0060] Figure 7 A schematic diagram of the structure of a storage medium according to an embodiment of the present application. DETAILED DESCRIPTION
[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.
[0062] See also Figure 1 , is a flow chart of the difficult sample mining method of the embodiment of the present application. The difficult sample mining method of the embodiment of the present application comprises the following steps:
[0063] S10: Collect medical image data with multiple organ labels, and randomly divide the medical image data into training set, validation set and test set;
[0064] In this step, the medical image data is abdominal CT data, and may also be medical image data of other parts.
[0065] S20: constructing point cloud shape models including multiple organ types according to each label in the training set image, and arbitrarily selecting a point cloud shape model including all organ types as a template shape;
[0066] In this step, assuming that the number of training set images is N, and the types of organ labels include K, the construction process of the point cloud shape model specifically includes:
[0067] S21: For each label L in the training set i(i∈[1,N]), mesh the surface of each organ using the MC (Marching Cubes) algorithm, and then downsample the surface meshes using the decimation algorithm, merging multiple adjacent meshes with similar normal vectors into one mesh to reduce the number of surface meshes;
[0068] S22: extract the vertices of the surface mesh after downsampling of each organ respectively, form a multi-organ point cloud shape model according to the vertices of the surface mesh, and arbitrarily select a shape model containing all organs from the multi-organ point cloud shape model as a template shape; specifically: assume that based on the label L i The resulting multi-organ point cloud shape model is S i , where the shape model of the kth organ is In S i∈[1,N] Randomly select a shape model containing all K organ types as the template shape S m ;
[0069] S23: For each point cloud shape model S i For each point in, the coordinates are maintained separately and normal vectors, etc.; for the template shape S m For each point in the, respectively maintain the point index, the organ category of the point, the adjacent point, the sampling frequency c, the reward average Reward value expectation v and other attributes;
[0070] like Figure 2 The following is a schematic diagram of template shape maintenance in an embodiment of the present application. (a) is a schematic diagram of point cloud shape model maintenance, where the normal vector of a point is obtained by averaging the normal vectors of all adjacent grids; (b) is a schematic diagram of template shape maintenance.
[0071] S30: calculating the point cloud correspondence between the template shape and all organs in each point cloud shape model based on the elastic point cloud registration algorithm;
[0072] In this step, the point cloud correspondence calculation method specifically includes the following steps:
[0073] S31: Based on the elastic point cloud registration algorithm, the template shape S m The point cloud of the kth organ in With the i-th point cloud shape model S i The point cloud of the kth organ in Perform registration to obtain the mapping matrix between the two point clouds
[0074] S32: Perform the above-mentioned registration on the point cloud of each organ in each point cloud shape model and the point cloud of the corresponding organ in the template shape (the template shape does not need to be registered with itself). After (N-1)×K registrations, a point cloud mapping matrix of the template shape and all organs in the point cloud shape model is obtained.
[0075] S40: randomly cutting a certain number of training samples from the training set images, inputting the training samples into the segmentation model for training, and after a batch of training is completed, updating the reward expectation value of the template shape according to the training loss value of the current batch of training samples;
[0076] In this step, the training process of model training based on training samples is specifically as follows:
[0077] S41: Crop the training sample from the i-th training set image, and extract the training sample from the point cloud shape model S according to the cropping coordinate range of the training sample. i Retrieve the point set contained in the training sample from the point cloud
[0078] S42: For each point in the point set Get the training sample voxels within the radius r and calculate the mean training loss value of all voxels to get the point The reward value obtained in this round of training The parameter value of the radius r can be preset according to the actual situation, and is preferably set to 5 in the embodiment of the present application;
[0079] S43: According to the template shape S m With point cloud shape model S i The mapping matrix between Get point set In S m The corresponding point set in
[0080] S44: Use the reward value obtained in this round of training Update Point The sampling frequency c maintained in i , average reward And the expected reward value v i :
[0081]
[0082] c i ←c i +1 (2)
[0083]
[0084] Among them, formula (1) updates the reward average value in a weighted average manner The weight β can be preset according to the actual application. Formula (3) calculates the expected reward value v through the UCB (upper confidence bound) algorithm i .
[0085] S50: Determine whether the model training batch is greater than the set parameter T. If so, execute S60; otherwise, re-execute S40;
[0086] S60: mining difficult samples from the training set images according to the expected reward value distribution on the template shape, inputting the difficult samples into the segmentation model for training, and after a batch of training is completed, updating the expected reward value in the template shape according to the training loss value of the difficult samples in the current batch;
[0087] In this step, the training process of model training based on difficult samples is as follows:
[0088] S61: For template shape S m Each point q in i , calculate the expected average reward value of the point and its adjacent points, and get i The average reward expectation of the local anatomical structure centered on And in S m The first 2×B (B is the default batch size) is large Randomly select a point q opt , get the value of point q opt The point set {q1,q2,…,q n};
[0089] S62: Select an image I from the training set images at random i , and obtain image I i Point cloud shape model S i , through the pre-computed mapping matrix Get the point set {q1,q2,…,q n}In S i The corresponding point set {p1,p2,…,p n};
[0090] S63: According to the point set {q1,q2,…,q n}The expected value of reward v i with {p1,p2,…,p n}'s coordinates Calculate the point set {p1,p2,…,p n}In S i The geometric center of
[0091]
[0092] S64: Calculate {p1,p2,…,p n The normal vector δ of each point in i The average value of the normal vector And the geometric center Along or against the mean normal vector The direction of translation is by Centered on image I i The translation distance can be randomly set according to the actual application, and the setting condition of the translation distance is: after the translation, Centered on image I i The coordinate range of the difficult samples collected in the experiment must include {p1,p2,…,p n}.
[0093] S70: Determine whether the model training meets the termination condition. If yes, execute S80; otherwise, execute S60 again.
[0094] In this step, the termination condition of model training is: the loss function value no longer decreases or the training batches reach the predetermined number.
[0095] S80: Export the trained segmentation model and verify and evaluate the segmentation model through the validation set and test set.
[0096] In order to verify the feasibility and effectiveness of the difficult sample mining method of the embodiment of the present application, the difficult sample mining method of the embodiment of the present application is compared with the random sampling strategy (Heuristic), OHEM (Online Hard Examples Mining) sampling strategy and RUCB (Relaxed Upper Confidence Bound) sampling strategy through experiments. Figure 3 As shown in , it is the training loss function curve under different sampling strategies. Figure 3 It can be seen that the sampling strategy proposed in the embodiment of the present application enables the model to obtain the lowest loss function value with the least number of training batches, that is, it enables the model to fit towards a better global solution at a faster speed. Figure 4The figure is a schematic diagram of the segmentation effect of the segmentation model on the test set data under different sampling strategies, where (a), (b), (c), and (d) respectively show the anatomical structures of the right adrenal gland, the boundary between the esophagus and the stomach, the pancreas, and the portal vein and splenic vein. By comparing the segmentation results achieved by different sampling strategies with the label column (GT), it can be seen that the other three sampling strategies have certain omissions and mis-segments for small organs, low-contrast organ boundaries, and different organs with similar grayscale and structure, while the segmentation results of the sampling strategy proposed in the embodiment of the present application are closest to the label, and can segment the anatomical structure most completely and accurately.
[0097] Based on the above, the difficult sample mining method of the embodiment of the present application aligns the point cloud between the template shape and all the point cloud shape models in the training set image, and constructs the association between the training set images about the anatomical structure, so that only the attributes in the template shape need to be maintained and updated during model training, which greatly reduces the consumption of memory and computing resources. By abstracting the difficulty of the training sample into an estimable reward value, during model training, the UCB algorithm is used to iteratively update the reward value expectation of the template shape based on the number of sampling times and the historical loss value, and the difficult samples in the training set image are mined according to the reward value distribution in the template shape, so as to more scientifically evaluate the difficulty of the sample, and more fully mine the difficult samples that are more representative (difficult to segment) for the segmentation model, effectively alleviating the class imbalance problem in model training, improving model accuracy and accelerating model fitting.
[0098] See also Figure 5 , is a schematic diagram of the structure of the difficult sample mining system of the embodiment of the present application. The difficult sample mining system 40 of the embodiment of the present application includes:
[0099] Data collection module 41: used to collect medical image data including K types of labels;
[0100] Model building module 42: used to build a point cloud shape model according to the labels of the medical image data, and select a point cloud shape model containing K types of labels as a template shape;
[0101] The first model training module 43 is used to randomly cut a set number of training samples from the medical image data, input the training samples into the segmentation model for training, and after a batch of training is completed, update the reward expectation value of the template shape according to the training loss value of the current batch of training samples;
[0102] Sample mining module 44: used for mining difficult samples from medical image data according to the expected value distribution of rewards on template shapes after the training batch of the segmentation model reaches a set number of times;
[0103] The second model training module 45 is used to train the segmentation model according to difficult samples.
[0104] See also Figure 6 , is a schematic diagram of a terminal structure of an embodiment of the present application. The terminal 50 includes a processor 51 and a memory 52 coupled to the processor 51.
[0105] The memory 52 stores program instructions for implementing the above-mentioned difficult sample mining method.
[0106] The processor 51 is used to execute program instructions stored in the memory 52 to control difficult sample mining.
[0107] The processor 51 may also be referred to as a CPU (Central Processing Unit). The processor 51 may be an integrated circuit chip having the ability to process signals. The processor 51 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
[0108] See also Figure 7 , which is a schematic diagram of the structure of the storage medium of the embodiment of the present application. The storage medium of the embodiment of the present application stores a program file 61 that can implement all the above methods, wherein the program file 61 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or terminal devices such as a computer, a server, a mobile phone, and a tablet.
[0109] The above description of the disclosed embodiments enables professionals and technicians in the field to implement or use the present application. Various modifications to these embodiments will be apparent to professionals and technicians in the field, and the general principles defined in this application can be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown in the present application, but will conform to the widest range consistent with the principles and novel features disclosed in the present application.
Claims
1. A difficult sample mining method, characterized in that: include: Collect medical image data including K types of labels; Constructing a point cloud shape model according to the labels of the medical image data, and selecting a point cloud shape model containing K types of labels as a template shape; Randomly cutting a set number of training samples from the medical image data, inputting the training samples into a segmentation model for training, and after a batch of training is completed, updating the expected reward value of the template shape according to the training loss value of the current batch of training samples; When the training batch of the segmentation model reaches a set number of times, mining difficult samples from the medical image data according to the expected value distribution of rewards on the template shape, and training the segmentation model according to the difficult samples; The step of constructing a point cloud shape model according to the label of the medical image data comprises: For each label L in the medical image data i (i∈[1, N]), N is the number of medical image data, and the organ corresponding to each label is meshed by the marching cube algorithm to obtain the surface mesh of each organ; All surface meshes are downsampled by the decimation algorithm, and adjacent meshes with similar normal vectors are approximately merged into one mesh; extracting vertices of the merged surface meshes, and forming a multi-organ point cloud shape model according to the vertices; The step of randomly cutting a set number of training samples from the medical image data and inputting the training samples into the segmentation model for training is specifically as follows: A training sample is cropped from the i-th medical image data, and a point cloud shape model S is obtained according to the cropping coordinate range of the training sample. i Retrieve the point set contained in the training sample from the point cloud For each point in the point set Get the training sample voxels within the radius r and calculate the mean training loss value of all voxels to get the point The reward value obtained in this round of training According to the template shape S m With point cloud shape model S i The mapping matrix between Get point set In S m The corresponding point set in Use the reward value obtained in this round of training Update Point The sampling frequency c maintained in i , average reward And the expected reward value v i : c i ←c i +1 In the above formula, the reward average is updated in a weighted average manner β is the weight, and the expected reward value v is calculated using the upper confidence interval algorithm i .
2. The difficult sample mining method according to claim 1, characterized in that: The step of selecting a point cloud shape model containing K types of labels as a template shape includes: Assume that based on label L i The resulting point cloud shape model is S i , where the shape model of the kth organ is In S i∈[1,N] Randomly select a shape model containing K label types as the template shape S m .
3. The difficult sample mining method according to claim 2, characterized in that: The step of constructing a point cloud shape model according to the labels of the medical image data and selecting a point cloud shape model containing K types of labels as a template shape further includes: For the point cloud shape model S i For each point in, the coordinates are maintained separately and the normal vector; For the template shape S m For each point in the, respectively maintain the point index, the organ category of the point, the adjacent point, the sampling frequency c, the reward average And the expected reward value v.
4. The difficult sample mining method according to claim 3, characterized in that: The step of constructing a point cloud shape model according to the labels of the medical image data and selecting a point cloud shape model containing K types of labels as a template shape further includes: The corresponding relationship between the template shape and the point clouds of all organs in each point cloud shape model is calculated based on the elastic point cloud registration algorithm; The point cloud correspondence calculation method includes: The template shape S m The point cloud of the kth organ in With the i-th point cloud shape model S i The point cloud of the kth organ in Perform registration to obtain the mapping matrix between the two point clouds The point cloud of each organ in each point cloud shape model is registered with the point cloud of the corresponding organ in the template shape. After (N-1)×K registrations, a point cloud mapping matrix of all organs in the template shape and the point cloud shape model is obtained.
5. The difficult sample mining method according to claim 4, characterized in that: The mining of difficult samples from the medical image data according to the reward expectation value distribution on the template shape comprises: For the template shape S m Each point q in i , calculate the expected average reward value of the point and its adjacent points, and get i The average reward expectation of the local anatomical structure centered on And in S m Middle front 2×B large Randomly select a point q opt , get the value of point q opt The point set {q1, q2, ..., q n }; Where B is the preset batch size; Select an image I arbitrarily from the medical image data i , and obtain image I i Point cloud shape model S i , through the mapping matrix Get the point set {q1, q2, ..., q n }In S i The corresponding point set {p1, p2, ..., p n }; According to the point set {q1, q2, ..., q n }The expected value of reward v i with {p1, p2, ..., p n }'s coordinates Compute the point set {p1, p2, ..., p n }In S i The geometric center of Calculate {p1, p2, ..., p n The normal vector δ of each point in i The average value of the normal vector And the geometric center Along or against the mean normal vector The direction of translation is by Centered on the image I i Difficult samples to collect.
6. A difficult sample mining system using the difficult sample mining method according to claim 1, characterized in that: include: Data collection module: used to collect medical image data including K types of labels; Model building module: used to build a point cloud shape model according to the labels of the medical image data, and select a point cloud shape model containing K types of labels as a template shape; The first model training module is used to randomly cut a set number of training samples from the medical image data, input the training samples into the segmentation model for training, and update the reward expectation value of the template shape according to the training loss value of the current batch of training samples after a batch of training is completed; A sample mining module: used for mining difficult samples from the medical image data according to the expected value distribution of rewards on the template shape after the training batch of the segmentation model reaches a set number of times; The second model training module is used to train the segmentation model according to the difficult samples.
7. A terminal, characterized in that: The terminal includes a processor and a memory coupled to the processor, wherein: The memory stores program instructions for implementing the difficult sample mining method according to any one of claims 1 to 5; The processor is used to execute the program instructions stored in the memory to control difficult sample mining.
8. A storage medium, characterized in that: Program instructions executable by a processor are stored, and the program instructions are used to execute the difficult sample mining method according to any one of claims 1 to 5.
