Training method, device and equipment of pre-training model applied to three-dimensional image

Abstract

Embodiments of the present application disclose a training method and device of a pre-training model applied to a three-dimensional image and equipment, and belong to the technical field of artificial intelligence. The method comprises the following steps: repeating the following steps until a preset condition is reached: determining a first feature vector of a to-be-processed mask image based on a first to-be-trained model, determining a second feature vector of a to-be-processed three-dimensional image; obtaining a third feature vector of each other three-dimensional image; updating the parameters of the first to-be-trained model according to the first feature vector, the second feature vector and the third feature vectors; and the first to-be-trained model after the preset condition is reached can extract a feature map of the three-dimensional image. The first feature vector and the second feature vector of the same to-be-processed three-dimensional image form a positive sample pair, the first feature vector of the to-be-processed three-dimensional image and the third feature vector of the other three-dimensional image form a negative sample pair, and the parameters of the first to-be-trained model are updated. The obtained model can accurately obtain a feature map of the three-dimensional image.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a training method, apparatus, and device for a pre-trained model applied to three-dimensional images. Background Technology

[0002] With the development of artificial intelligence technology, it is possible to recognize 3D images. For example, 3D medical images can be recognized to obtain image information, recognition results, and so on.

[0003] In existing technologies, 3D images can be processed using a self-supervised approach to obtain a trained self-supervised model. This model can then be used to determine the feature maps of the 3D images. Each 3D image used in the training process is obtained after undergoing different data augmentation. Subsequent 3D image recognition is then performed based on these feature maps.

[0004] However, in existing technologies, the 3D images of different objects under the same target object are very similar, and even if data augmentation is performed on each 3D image, the resulting 3D images are still very similar. In the above methods, only different data augmentation is performed on each 3D image, and the augmented 3D images are directly applied to the self-supervised learning process. It is impossible to obtain a good self-supervised model based on these 3D images. The feature map of the 3D image obtained by the self-supervised model is not accurate, and the feature map cannot be used in the subsequent image recognition process. Summary of the Invention

[0005] This application provides a training method, apparatus, and device for pre-trained models applied to 3D images. It addresses the problem that existing self-supervised training methods cannot obtain a good self-supervised model based on augmented 3D images, and the feature maps of the resulting 3D images obtained from the self-supervised model are inaccurate. The technical solution is as follows:

[0006] In a first aspect, embodiments of this application provide a training method for a pre-trained model applied to three-dimensional images, the method comprising:

[0007] Obtain the 3D image to be processed, the mask image corresponding to the 3D image to be processed, and at least one other 3D image different from the 3D image to be processed, and repeat the following steps until the preset conditions are met:

[0008] The first feature vector of the mask image to be processed is determined based on the first model to be trained, and the second feature vector of the three-dimensional image to be processed is determined; and the third feature vector of each of the other three-dimensional images is obtained, wherein the third feature vector represents the global features of the other three-dimensional images;

[0009] Update the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors;

[0010] Among them, the first model to be trained after meeting the preset conditions is used to extract the feature map of the three-dimensional image to be analyzed.

[0011] In one feasible implementation, updating the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors includes:

[0012] A first calculation parameter is determined based on the first feature vector, the second feature vector, and the preset hyperparameter coefficients; and a second calculation parameter is determined based on the first feature vector, each of the third feature vectors, and the preset hyperparameter coefficients.

[0013] A decision function is established based on the first calculation parameter and the second calculation parameter;

[0014] The decision function is subjected to gradient backpropagation to update the parameters of the first model to be trained.

[0015] In one feasible implementation, the second feature vector is obtained by processing the three-dimensional image to be processed based on the second model to be trained; after updating the parameters of the first model to be trained according to the first feature vector, the second feature vector, and each of the third feature vectors, the method further includes:

[0016] Update the parameters of the second model to be trained based on the updated first model to be trained;

[0017] The second training model, after meeting the preset conditions, is used to extract the feature map of the three-dimensional image to be analyzed.

[0018] In one feasible implementation, updating the parameters of the second training model based on the updated first training model includes:

[0019] The parameters of the second training model are updated based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

[0020] In one feasible implementation, acquiring the three-dimensional image to be processed and the mask image to be processed corresponding to the three-dimensional image to be processed includes:

[0021] The three-dimensional image to be processed is acquired, and the three-dimensional image to be processed is preprocessed to obtain a local image corresponding to the three-dimensional image to be processed.

[0022] Based on the 3D image to be processed and the local image, a mask image to be processed is determined, wherein the size of the mask image to be processed is the same as the size of the 3D image to be processed.

[0023] In one feasible implementation, preprocessing the three-dimensional image to be processed to obtain a local image corresponding to the three-dimensional image to be processed includes:

[0024] The three-dimensional image to be processed is segmented to obtain multiple segmented images;

[0025] Determine the variance of each segmented image and remove segmented images with variance less than a preset threshold to obtain filtered segmented images.

[0026] One of the segmented images after filtering is randomly selected as the local image.

[0027] In one feasible implementation, determining the first feature vector of the mask image to be processed based on the first model to be trained includes:

[0028] Based on the first training model, feature extraction processing is performed on the mask image to be processed to obtain the feature map of the mask image to be processed.

[0029] The feature map of the mask image to be processed is subjected to mapping transformation to obtain the first feature vector.

[0030] In one feasible implementation, the size of the first feature map is greater than or equal to the total number of the plurality of segmented images.

[0031] In one feasible implementation, determining the first feature vector of the mask image to be processed based on the first model to be trained includes:

[0032] The feature map of the mask image to be processed is determined based on the first training model, and the feature map of the local image corresponding to the three-dimensional image to be processed is determined based on the first training model; wherein, the mask image to be processed is obtained based on the three-dimensional image to be processed and the local image, and the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed;

[0033] The feature map of the mask image to be processed is mapped and transformed based on the first multilayer perceptron network model to obtain the first sub-vector, and the feature map of the local image is mapped and transformed based on the second multilayer perceptron network model to obtain the second sub-vector.

[0034] The first feature vector is determined based on the first sub-vector and the second sub-vector.

[0035] In one feasible implementation, determining the first feature vector based on the first sub-vector and the second sub-vector includes:

[0036] Based on the third multilayer perceptron network model, the first sub-vector and the second sub-vector are subjected to recovery mapping projection transformation to obtain the first feature vector;

[0037] Alternatively, feature concatenation can be performed on the first sub-vector and the second sub-vector to obtain the first feature vector.

[0038] In one feasible implementation, determining the second feature vector of the three-dimensional image to be processed includes:

[0039] Based on the second training model, feature extraction processing is performed on the three-dimensional image to be processed to obtain a second feature map;

[0040] The second feature map is mapped and transformed based on the fourth multilayer perceptron network model to obtain the second feature vector.

[0041] In one feasible implementation, the first model to be trained is a convolutional network model or a deep learning model; the second model to be trained for processing the three-dimensional image to be processed is a convolutional network model or a deep learning model.

[0042] Secondly, embodiments of this application provide a method for extracting feature maps from a three-dimensional image, the method comprising:

[0043] A 3D image to be analyzed is acquired, and the 3D image to be analyzed is input into a feature extraction model to obtain a feature map of the 3D image to be analyzed; wherein, the feature extraction model is a first training model provided by the first aspect after the preset conditions are met;

[0044] Based on the feature map of the three-dimensional image to be analyzed, the recognition result of the three-dimensional image to be analyzed is determined.

[0045] In one feasible implementation, determining the recognition result of the three-dimensional image to be analyzed based on the feature map of the three-dimensional image to be analyzed includes:

[0046] Based on the feature map of the three-dimensional image to be analyzed, the feature vector of the three-dimensional image to be analyzed is determined;

[0047] The feature vector of the three-dimensional image to be analyzed is input into a preset recognition model to obtain the recognition result of the three-dimensional image to be analyzed;

[0048] The identification result can be any one of the following: the category of the three-dimensional image to be analyzed, the characteristics of the three-dimensional image to be analyzed, or the structural segmentation result of the three-dimensional image to be analyzed.

[0049] Thirdly, embodiments of this application provide a training apparatus for a pre-trained model applied to three-dimensional images, the apparatus comprising:

[0050] The first acquisition unit is used to acquire the three-dimensional image to be processed and the mask image to be processed corresponding to the three-dimensional image to be processed.

[0051] The second acquisition unit is used to acquire at least one other three-dimensional image that is different from the three-dimensional image to be processed.

[0052] An execution unit is used to repeat the following units until a preset condition is met:

[0053] The first determining unit is used to determine the first feature vector of the mask image to be processed based on the first training model;

[0054] The second determining unit is used to determine the second feature vector of the three-dimensional image to be processed;

[0055] The third acquisition unit is used to acquire the third feature vector of each of the other three-dimensional images, wherein the third feature vector represents the global features of the other three-dimensional images;

[0056] The first update unit is used to update the parameters of the first model to be trained based on the first feature vector, the second feature vector and each of the third feature vectors.

[0057] Among them, the first model to be trained after meeting the preset conditions is used to extract the feature map of the three-dimensional image to be analyzed.

[0058] In one feasible implementation, the first updating unit includes:

[0059] The first determining module is used to determine a first calculation parameter based on the first feature vector, the second feature vector, and preset hyperparameter coefficients; and to determine a second calculation parameter based on the first feature vector, each of the third feature vectors, and the preset hyperparameter coefficients.

[0060] A module is established to establish a decision function based on the first calculation parameters and the second calculation parameters;

[0061] An update module is used to perform gradient backpropagation processing on the decision function to update the parameters of the first model to be trained.

[0062] In one feasible implementation, the second feature vector is obtained by processing the three-dimensional image to be processed based on the second training model; the apparatus further includes:

[0063] The second update unit is used to update the parameters of the second model to be trained based on the updated first model to be trained, after the first update unit updates the parameters of the first model to be trained based on the first feature vector, the second feature vector and each of the third feature vectors.

[0064] The second training model, after meeting the preset conditions, is used to extract the feature map of the three-dimensional image to be analyzed.

[0065] In one feasible implementation, the second updating unit is specifically used for:

[0066] The parameters of the second training model are updated based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

[0067] In one feasible implementation, the first acquisition unit includes:

[0068] The acquisition module is used to acquire the three-dimensional image to be processed, preprocess the three-dimensional image to be processed, and obtain a local image corresponding to the three-dimensional image to be processed.

[0069] The second determining module is used to determine the mask image to be processed based on the three-dimensional image to be processed and the local image, wherein the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed.

[0070] In one feasible implementation, the acquisition module is specifically used for:

[0071] The three-dimensional image to be processed is segmented to obtain multiple segmented images;

[0072] Determine the variance of each segmented image and remove segmented images with variance less than a preset threshold to obtain filtered segmented images.

[0073] One of the segmented images after filtering is randomly selected as the local image.

[0074] In one feasible implementation, the first determining unit includes:

[0075] The first processing module is used to perform feature extraction processing on the mask image to be processed based on the first model to be trained, so as to obtain the feature map of the mask image to be processed.

[0076] The second processing module is used to perform mapping transformation processing on the feature map of the mask image to be processed to obtain the first feature vector.

[0077] In one feasible implementation, the size of the first feature map is greater than or equal to the total number of the plurality of segmented images.

[0078] In one feasible implementation, the first determining unit includes:

[0079] The third determining module is used to determine the feature map of the mask image to be processed based on the first training model, and to determine the feature map of the local image corresponding to the three-dimensional image to be processed based on the first training model; wherein the mask image to be processed is obtained based on the three-dimensional image to be processed and the local image, and the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed.

[0080] The third processing module is used to perform mapping transformation processing on the feature map of the mask image to be processed based on the first multilayer perceptron network model to obtain the first sub-vector, and to perform mapping transformation processing on the feature map of the local image based on the second multilayer perceptron network model to obtain the second sub-vector.

[0081] The fourth determining module is used to determine the first feature vector based on the first sub-vector and the second sub-vector.

[0082] In one feasible implementation, the fourth determining module is specifically used for:

[0083] Based on the third multilayer perceptron network model, the first sub-vector and the second sub-vector are subjected to recovery mapping projection transformation to obtain the first feature vector;

[0084] Alternatively, feature concatenation can be performed on the first sub-vector and the second sub-vector to obtain the first feature vector.

[0085] In one feasible implementation, the second determining unit includes:

[0086] The fifth determining module is used to perform feature extraction processing on the three-dimensional image to be processed based on the second training model to obtain a second feature map;

[0087] The fourth processing module is used to perform mapping transformation processing on the second feature map based on the fourth multilayer perceptron network model to obtain the second feature vector.

[0088] In one feasible implementation, the first model to be trained is a convolutional network model or a deep learning model; the second model to be trained for processing the three-dimensional image to be processed is a convolutional network model or a deep learning model.

[0089] Fourthly, embodiments of this application provide a feature map extraction device for a three-dimensional image, the device comprising:

[0090] The acquisition unit is used to acquire the 3D image to be analyzed.

[0091] A processing unit is used to input the three-dimensional image to be analyzed into a feature extraction model to obtain a feature map of the three-dimensional image to be analyzed; wherein, the feature extraction model is the first training model in the device provided by the third party after reaching the preset conditions;

[0092] The determining unit is used to determine the recognition result of the three-dimensional image to be analyzed based on the feature map of the three-dimensional image to be analyzed.

[0093] In one feasible implementation, the determining unit is specifically used for:

[0094] Based on the feature map of the three-dimensional image to be analyzed, the feature vector of the three-dimensional image to be analyzed is determined;

[0095] The feature vector of the three-dimensional image to be analyzed is input into a preset recognition model to obtain the recognition result of the three-dimensional image to be analyzed;

[0096] The identification result can be any one of the following: the category of the three-dimensional image to be analyzed, the characteristics of the three-dimensional image to be analyzed, or the structural segmentation result of the three-dimensional image to be analyzed.

[0097] Fifthly, embodiments of this application provide an electronic device, including: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and execute the method provided in the first aspect, or the computer program adapted to be loaded by the processor and execute the method provided in the second aspect.

[0098] In a sixth aspect, embodiments of this application provide a computer storage medium storing a plurality of instructions, the instructions being adapted for a processor to load and execute the method provided in the first aspect, or the instructions being adapted for a processor to load and execute the method provided in the second aspect.

[0099] In a seventh aspect, embodiments of this application provide a computer program product comprising: a computer program stored in a readable storage medium, wherein at least one processor of an electronic device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the electronic device to perform the method provided in the first aspect, or the at least one processor executes the computer program to cause the electronic device to perform the method provided in the second aspect.

[0100] In this embodiment, a three-dimensional image to be processed, a mask image corresponding to the three-dimensional image to be processed, and at least one other three-dimensional image different from the three-dimensional image to be processed are obtained. A positive sample combination is formed based on the three-dimensional image to be processed and the mask image to be processed, and a negative sample combination is formed based on the three-dimensional image to be processed and each other three-dimensional image. Then, the following steps are repeated until a preset condition is met: a first feature vector of the mask image to be processed is determined based on a first training model, and a second feature vector of the three-dimensional image to be processed is determined; and a third feature vector of each other three-dimensional image is obtained, the third feature vector representing the other three-dimensional image. The global features of the image are analyzed; the first and second feature vectors of the same 3D image to be processed are combined into a positive sample pair (the first feature vector of the mask image to be processed and the second feature vector of the 3D image to be processed are combined into a positive sample pair), and the first feature vector of the 3D image to be processed and the third feature vector of each other 3D image are combined into a negative sample pair (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vector of each other 3D image); the parameters of the first training model are updated based on the positive sample pairs and each negative sample pair; the first training model after reaching the preset conditions is used to extract the feature map of the 3D image to be analyzed. A positive sample pair is formed by combining the first feature vector and the second feature vector of the same 3D image to be processed (the first feature vector of the mask image to be processed and the second feature vector of the 3D image to be processed are combined into a positive sample pair). The similarity between the first feature vector and the second feature vector of the same 3D image is high. A negative sample pair is formed by combining the first feature vector of the 3D image to be processed and the third feature vector of each other 3D image (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vector of each other 3D image). The similarity between the first feature vector and the third feature vector of different 3D images is low. The parameters of the first training model are updated based on the positive sample pairs and each negative sample pair. Then, the model is trained based on a small number of unlabeled samples. This can solve various defects of training models by data augmentation and improve the learning ability of the model. A good first training model can be obtained based on the positive sample pairs and each negative sample pair. The feature map of the 3D image obtained by the model is accurate and can be used in the subsequent image recognition process. Furthermore, the first training model obtained in this embodiment does not learn information such as data noise, and the feature map of the 3D image obtained based on the first training model does not include information such as data noise. Attached Figure Description

[0101] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0102] Figure 1 A flowchart illustrating a training method for a pre-trained model applied to 3D images, provided as an embodiment of this application;

[0103] Figure 2 A flowchart illustrating another training method for a pre-trained model applied to three-dimensional images, provided as an embodiment of this application;

[0104] Figure 3 A schematic diagram of the three-dimensional image to be processed provided in an embodiment of this application;

[0105] Figure 4 A schematic diagram of a partial image provided in an embodiment of this application;

[0106] Figure 5 A schematic diagram of the mask image to be processed provided in an embodiment of this application;

[0107] Figure 6 A schematic diagram of the training process of the pre-trained model provided in the embodiments of this application. Figure 1 ;

[0108] Figure 7 A flowchart illustrating another training method for a pre-trained model applied to three-dimensional images, provided in an embodiment of this application;

[0109] Figure 8 A schematic diagram of the training process of the pre-trained model provided in the embodiments of this application. Figure 2 ;

[0110] Figure 9 A numerical schematic diagram of the decision function for the training process provided in the embodiments of this application;

[0111] Figure 10 A numerical schematic diagram of the decision function for the analysis process provided in the embodiments of this application;

[0112] Figure 11 A flowchart illustrating a method for extracting feature maps from a three-dimensional image, as provided in an embodiment of this application;

[0113] Figure 12 A schematic diagram of the structure of a training device for a pre-trained model applied to a three-dimensional image, provided in an embodiment of this application;

[0114] Figure 13A schematic diagram of the structure of another training device for a pre-trained model applied to three-dimensional images, provided in an embodiment of this application;

[0115] Figure 14 A schematic diagram of the structure of a feature map extraction device for a three-dimensional image provided in an embodiment of this application;

[0116] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0117] Figure 16 This application provides a schematic diagram of the structure of another electronic device. Detailed Implementation

[0118] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0119] It should be understood that the described embodiments are merely some, not all, of the embodiments in this application. All other embodiments obtained by those skilled in the art based on the embodiments in this application without inventive effort are within the scope of protection of this application.

[0120] In the following description, when referring to the accompanying drawings, the same numbers in different drawings denote the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0121] In the description of this application, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this application according to the specific circumstances. Furthermore, in the description of this application, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0122] The electronic device involved in this application includes a USB interface, a memory, a processor, and buttons. The processor is connected to the USB interface, the memory, and the buttons. The USB interface is a USB device used to connect to a display device. The electronic device connects to the display device (e.g., a laptop) via the USB interface. The electronic device can be combined with a USB cable to form a custom-designed special USB cable, or it can exist as a combination of a USB dongle with the electronic device and a standard USB cable. The electronic device can be powered directly through the USB interface of the display device. One purpose of the memory is to store specific programs or program downloaders required for image processing. One purpose of the processor is to load and control the specific programs or program downloaders stored in the memory. The buttons are used to trigger the processor to generate relevant control commands. For example, after a button is clicked, the processor receives the operation data sent by the button and generates a corresponding command based on the operation data.

[0123] With the development of artificial intelligence technology, it is possible to recognize 3D images. For example, 3D medical images can be recognized to obtain image information, recognition results, and so on.

[0124] For example, deep learning models can be used to perform structural segmentation on 3D medical images of the brain to obtain segmentation results. Deep learning models can also be used to perform detection processing on 3D medical images of the lungs to obtain lung node detection results. Furthermore, deep learning models can be used to perform anomaly recognition processing on 3D medical images of the chest to obtain recognition results.

[0125] Before performing recognition processing on 3D images, it is necessary to determine the feature maps of the 3D images. This requires obtaining a model to determine the feature maps. In one example, when the dataset of the target object is small, a random initialization training process is needed to obtain the model for determining the feature maps. For instance, datasets of 3D medical images are relatively small, thus requiring a random initialization training process for the deep learning model to obtain the model for determining the feature maps of the 3D images.

[0126] However, the above methods suffer from a lack of datasets, which leads to high requirements for the amount of labeled data and parameter tuning strategies during training, and makes the training process quite difficult. As a result, the accuracy of the model used to determine the feature map of a 3D image is limited, the generalization ability is poor, and so on. The resulting model cannot accurately determine the feature map of a 3D image.

[0127] In one example, unlabeled 3D images can be input into an initial model for training, based on constructing discriminative or generative proxy tasks, to obtain a model for determining the feature maps of 3D images. For instance, based on contrastive learning or image restoration learning, unlabeled 3D images can be input into an initial model for training, resulting in a model for determining the feature maps of 3D images. However, in the above methods, the feature maps output by the trained model may include information such as data noise; or, it may be impossible to obtain a good self-supervised model based on the 3D images of each object, and the feature maps of the 3D images obtained by the self-supervised model are inaccurate and cannot be used in subsequent image recognition processes.

[0128] For example, an unsupervised pre-trained model for 3D images can be built based on image restoration methods. This involves augmenting the 3D image data to simulate image corruption, and then training an encoder-decoder model for 3D image restoration. This results in a model used to determine the feature maps of the 3D image. However, because image restoration is the primary task, the trained model focuses more on details, making it prone to learning information such as data noise. Consequently, the output feature maps will contain this noise. Furthermore, the model training process is limited by manually designed data augmentation combinations, making it difficult to achieve automatic learning from data structure information and knowledge.

[0129] For another example, 3D images can be processed using a self-supervised approach to obtain a trained self-supervised model. This model can then be used to determine the feature maps of the 3D images. During training, each 3D image is obtained by performing different data augmentations on the same 3D image. Subsequent 3D image recognition is then performed based on these feature maps. However, in this approach, each 3D image used during training is obtained by performing different data augmentations on each 3D image. Different 3D images of the same target object are very similar, and even after data augmentation on each 3D image, the resulting images are still very similar (for example, augmenting an image of a cat will still result in an image similar to the original cat image). Therefore, this approach merely performs different data augmentations on each 3D image and directly applies the augmented images to the self-supervised learning process. It cannot obtain a good self-supervised model based on these images, and the feature maps obtained from the resulting self-supervised model are inaccurate and cannot be used for subsequent image recognition.

[0130] For example, in the case of 3D medical images, the similarity of 3D medical images of the same body part from different people is very high. For instance, the grayscale space and structural content of chest images from different people are similar. Furthermore, the original 3D medical image of the same person and the 3D medical image after data augmentation of the same person are also very similar. Therefore, in the process of self-supervised learning directly based on these images, the resulting self-supervised model cannot accurately determine the feature maps.

[0131] This application provides a training method, apparatus, and device for pre-trained models applied to 3D images to solve the above-mentioned problems. These are described below.

[0132] It should be noted that, due to space limitations, this application specification does not exhaustively list all possible implementation methods. Those skilled in the art, after reading this application specification, should be able to deduce that, as long as the technical features do not contradict each other, any combination of technical features can constitute an optional implementation method. The following provides a detailed description of each embodiment.

[0133] Figure 1 A flowchart illustrating a training method for a pre-trained model applied to 3D images, as provided in this application embodiment, is shown below. Figure 1 As shown, the method provided in this embodiment includes the following steps:

[0134] S101. Obtain the three-dimensional image to be processed, the mask image to be processed corresponding to the three-dimensional image to be processed, and at least one other three-dimensional image different from the three-dimensional image to be processed.

[0135] For example, the execution subject of this embodiment can be an electronic device, a terminal device, a server, a training device or device for an item recommendation model, or other devices or devices that can execute the scheme of this embodiment, without limitation.

[0136] First, acquire the 3D image to be processed. This can be received from other devices or read from a storage medium. For example, the 3D image to be processed is a 3D medical image. The dimensions of the 3D image are D × H × W; where D, H, and W are all positive integers greater than or equal to 1.

[0137] Based on the 3D image to be processed, a mask image of the 3D image to be processed is generated; this mask image is called the mask image to be processed. Furthermore, the size of the mask image to be processed is the same as the size of the 3D image to be processed. For example, the mask image to be processed is a mask image directly generated from the 3D image to be processed; or, the mask image to be processed is generated from the 3D image to be processed that lacks local regions.

[0138] At least one other 3D image also needs to be acquired; each other 3D image is a different 3D image from the one being processed; and each other 3D image has the same size as the one being processed. Other 3D images can be received from other devices or read from storage media. For example, the 3D image to be processed is a 3D medical image; the other 3D images are also 3D medical images.

[0139] For example, the dimensions of the 3D image to be processed are D×H×W, the dimensions of the mask image to be processed are D×H×W, and the dimensions of other 3D images are D×H×W.

[0140] S102. Repeat the following steps S103-S104 until the preset conditions are met; wherein, the first model to be trained after the preset conditions are met is used to extract the feature map of the three-dimensional image to be analyzed.

[0141] For example, the following steps S103-S104 are repeated until a preset condition is met. The preset condition is: the number of repetitions is greater than or equal to a preset number of times, or the parameters of the first model to be trained no longer change.

[0142] S103. Based on the first model to be trained, determine the first feature vector of the mask image to be processed, and determine the second feature vector of the three-dimensional image to be processed; and obtain the third feature vector of each other three-dimensional image, wherein the third feature vector represents the global features of the other three-dimensional images.

[0143] In one example, the first model to be trained is a convolutional network model or a deep learning model; the second model to be trained is a convolutional network model or a deep learning model.

[0144] For example, the mask image to be processed is input into a first training model, and the mask image is processed based on the first training model to obtain a feature map of the mask image to be processed; a first feature vector is generated based on the feature map of the mask image to be processed. At this time, the first feature vector represents the global features of the mask image to be processed, and the first feature vector also represents the global features of the three-dimensional image to be processed.

[0145] The 3D image to be processed is input into a second training model. The second training model processes the 3D image to obtain a feature map of the 3D image to be processed. A second feature vector is generated based on the feature map of the 3D image to be processed. At this time, the second feature vector represents the global features of the 3D image to be processed.

[0146] Each other 3D image is input into the second training model to obtain a feature map of each other 3D image; a third feature vector of each other 3D image is generated based on the feature map of each other 3D image. The third feature vector at this time represents the global features of the other 3D images.

[0147] Both the first and second training models are image feature encoders. The first training model is a convolutional network model or a deep learning model; the second training model is also a convolutional network model or a deep learning model.

[0148] For example, the first model to be trained is a symmetric 3D convolutional network model, with no limit on the number of convolutional units or the number of convolutional layers. The second model to be trained is also a symmetric 3D convolutional network model, with no limit on the number of convolutional units or the number of convolutional layers.

[0149] S104. Update the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors.

[0150] For example, a positive sample pair is formed by combining the first feature vector and the second feature vector, and each negative sample pair is formed by combining the first feature vector and each third feature vector. It is understood that a positive sample pair includes the first feature vector and the second feature vector from the same 3D image to be processed (the positive sample pair includes the first feature vector of the mask image to be processed and the second feature vector of the 3D image to be processed); a negative sample pair includes the first feature vector of the 3D image to be processed and the third feature vectors of other 3D images (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vectors of other 3D images).

[0151] Then, based on the positive sample pairs and each negative sample pair, the parameters of the first model to be trained are updated using gradient backpropagation.

[0152] Repeat steps S103-S104 until the preset conditions are met. Then, the first training model after the preset conditions are met can be used to extract feature maps from the 3D image to be analyzed.

[0153] In this embodiment, a 3D image to be processed, a mask image corresponding to the 3D image to be processed, and at least one other 3D image different from the 3D image to be processed are obtained. A positive sample combination is formed based on the 3D image to be processed and the mask image to be processed, and a negative sample combination is formed based on the 3D image to be processed and each other 3D image. Then, the following steps are repeated until a preset condition is met: a first feature vector of the mask image to be processed is determined based on a first training model, and a second feature vector of the 3D image to be processed is determined; and a third feature vector of each other 3D image is obtained, the third feature vector representing the other 3D image's... Global features; forming a positive sample pair by combining the first and second feature vectors of the same 3D image to be processed (forming a positive sample pair by combining the first feature vector of the mask image to be processed and the second feature vector of the 3D image to be processed); forming a negative sample pair by combining the first feature vector of the 3D image to be processed and the third feature vector of each other 3D image (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vector of each other 3D image); updating the parameters of the first training model based on the positive sample pairs and each negative sample pair; the first training model after reaching the preset conditions is used to extract the feature map of the 3D image to be analyzed. A positive sample pair is formed by combining the first feature vector and the second feature vector of the same 3D image to be processed (the first feature vector of the mask image to be processed and the second feature vector of the 3D image to be processed are combined into a positive sample pair). The similarity between the first feature vector and the second feature vector of the same 3D image is high. A negative sample pair is formed by combining the first feature vector of the 3D image to be processed and the third feature vector of each other 3D image (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vector of each other 3D image). The similarity between the first feature vector and the third feature vector of different 3D images is low. The parameters of the first training model are updated based on the positive sample pairs and each negative sample pair. Then, the model is trained based on a small number of unlabeled samples. This can solve various defects of training models by data augmentation and improve the learning ability of the model. A good first training model can be obtained based on the positive sample pairs and each negative sample pair. The feature map of the 3D image obtained by the model is accurate and can be used in the subsequent image recognition process. Furthermore, the first training model obtained in this embodiment does not learn information such as data noise, and the feature map of the 3D image obtained based on the first training model does not include information such as data noise.

[0154] Figure 2 A flowchart illustrating another training method for a pre-trained model applied to 3D images, as provided in this application embodiment, is shown below. Figure 2 As shown, the method provided in this embodiment includes the following steps:

[0155] S201. Obtain the three-dimensional image to be processed, the mask image to be processed corresponding to the three-dimensional image to be processed, and at least one other three-dimensional image different from the three-dimensional image to be processed.

[0156] In one example, step S201, when acquiring the 3D image to be processed and the corresponding mask image to be processed, specifically includes:

[0157] The process involves acquiring a 3D image to be processed, preprocessing the 3D image to obtain a local image corresponding to the 3D image to be processed, and determining a mask image to be processed based on the 3D image to be processed and the local image, wherein the size of the mask image to be processed is the same as the size of the 3D image to be processed.

[0158] In one example, "preprocessing the 3D image to be processed to obtain a local image corresponding to the 3D image to be processed" includes;

[0159] The 3D image to be processed is segmented to obtain multiple segmented images; the variance of each segmented image is determined, and segmented images with variance less than a preset threshold are removed to obtain filtered segmented images; one segmented image is randomly selected from the filtered segmented images to be the local image.

[0160] For example, the execution subject of this embodiment can be an electronic device, a terminal device, a server, a training device or device for an item recommendation model, or other devices or devices that can execute the scheme of this embodiment, without limitation.

[0161] First, acquire the 3D image to be processed and at least one other 3D image. This can be received from another device or read from a storage medium. The 3D image to be processed has the same dimensions as the other 3D images.

[0162] For example, the 3D image to be processed is a 3D medical image; the other 3D images are also 3D medical images. The size of the 3D image to be processed is D×H×W; the size of the other 3D images is also D×H×W; where D, H, and W are all positive integers greater than or equal to 1.

[0163] We need to select a region from the 3D image to be processed as a local image. We can first preprocess the 3D image to obtain a local image corresponding to it. In one example, the 3D image to be processed is divided into multiple regions, resulting in multiple segmented images. For example, if the size of the 3D image to be processed is D×H×W, dividing it into multiple segmented images of size p×p×p results in (D×H×W) / (p×p×p) segmented images, where p is a positive integer greater than or equal to 1, and p is less than or equal to D, p is less than or equal to H, and p is less than or equal to W. Then, we calculate the variance (image variance) of each segmented image; we discard segmented images with variances less than a preset threshold, thus filtering the regional information of the image; we obtain filtered segmented images, where the variance of each filtered segmented image is greater than or equal to the preset threshold, resulting in images with a larger amount of regional information; finally, we randomly select one segmented image from the filtered images as the local image.

[0164] Based on the 3D image to be processed and the local image, an image processing algorithm is used to obtain the mask image to be processed, wherein the size of the mask image to be processed is the same as the size of the 3D image to be processed.

[0165] In one example, Figure 3 This is a schematic diagram of the three-dimensional image to be processed provided in an embodiment of this application. The three-dimensional image to be processed is represented by the word "image". Figure 4 This is a schematic diagram of a partial image provided in an embodiment of this application. The partial image is represented by patch_image. The size of the partial image is the same as the size of the three-dimensional image to be processed. However, only one segmented image is retained in the partial image patch_image. The remaining parts of the partial image patch_image can be set to a preset value (for example, the remaining parts of the partial image patch_image are all set to 0). Figure 5 This is a schematic diagram of the mask image to be processed provided in an embodiment of this application. The mask image to be processed is represented by `masked_image`. The size of the mask image to be processed is the same as the size of the 3D image to be processed. The region in the mask image to be processed corresponding to the determined segmented image is set to a preset value (for example, the region in the mask image to be processed corresponding to the determined segmented image is set to 0). The other regions in the mask image to be processed remain the original 3D image to be processed. Wherein, `image = patch_image + masked_image`.

[0166] S202. Repeat the following steps S203-S207 until the preset conditions are met; wherein, the first model to be trained after the preset conditions are met is used to extract the feature map of the three-dimensional image to be analyzed.

[0167] For example, the following steps S203-S207 are repeated until a preset condition is met. The preset condition is: the number of repetitions is greater than or equal to a preset number. Alternatively, the preset condition is: the parameters of the first model to be trained no longer change, and the parameters of the second model to be trained no longer change.

[0168] S203. Determine the first feature vector of the mask image to be processed based on the first model to be trained.

[0169] In one example, step S203 includes:

[0170] Based on the first model to be trained, feature extraction processing is performed on the mask image to be processed to obtain the feature map of the mask image to be processed; the feature map of the mask image to be processed is then subjected to mapping transformation processing to obtain the first feature vector.

[0171] In one example, the size of the first feature map is greater than or equal to the total number of the multiple segmented images.

[0172] For example, Figure 6 A schematic diagram of the training process of the pre-trained model provided in the embodiments of this application. Figure 1 ,like Figure 6 As shown, the masked image to be processed, `masked_image`, is input into a first training model, which is an image feature encoder. This first training model can be a convolutional network model or a deep learning model. Then, feature extraction is performed on the masked image to be processed based on the first training model to obtain a feature map of the masked image. Next, the feature map of the masked image to be processed is input into a Multi-Layer Perceptron (MLP) model (which can be called a fifth MLP model). Based on this fifth MLP model, a mapping transformation is performed on the feature map of the masked image to be processed, outputting a first feature vector. Using a multi-layer perceptron network to perform the mapping transformation on the feature map of the masked image to be processed can isolate the adverse effects of the self-supervised training process provided in this embodiment on the vector.

[0173] In one example, the masked image to be processed, `masked_image`, is input into a first training model `f`. The first training model `f` is an image feature encoder; for example, it may be a Convolutional Neural Network (CNN) model, a Transformer model, or a combination of CNN and Transformer. If the first training model `f` is a CNN model, then the number of convolutional layers in `f` is the same as that in the ResNet34 model, but `f` uses three-dimensional convolution (3D convolution). For example, the network parameters of the first training model are shown in Table 1.

[0174] Table 1 Network parameters of the first model to be trained

[0175]

[0176] In Table 1, "Layer name" represents the name of the convolutional layer in the first model to be trained, which can have up to 6 convolutional layers. "Output_size" represents the spatial size of the feature map output by each convolutional layer of the first model to be trained. "33-layer" represents the kernel size and channel size of each convolutional layer in the first model to be trained. For example, the kernel size of convolutional layer Conv1 is 5×5×5, and the channel size of convolutional layer Conv1 is 16; the kernel size of convolutional layer Conv2_x is 3×3×3, and the channel size of convolutional layer Conv2_x is 32; the kernel size of convolutional layer Conv3_x is 3×3×3, and the channel size of convolutional layer Conv3_x is 64; the kernel size of convolutional layer Conv4_x is 3×3×3, and the channel size of convolutional layer Conv4_x is 128; the kernel size of convolutional layer Conv5_x is 3×3×3, and the channel size of convolutional layer Conv5_x is 256.

[0177] Based on the first training model f, feature extraction is performed on the masked image to be processed, resulting in a feature map of the masked image. The size of the masked image is D×H×W, and the size of the output feature map is D / s×H / s×W / s, where s is the total downsampling ratio and is a positive integer greater than or equal to 1. The size of the feature map is (D×H×W) / (s×s×s), where s is a positive integer greater than or equal to 1. The size of the feature map is greater than or equal to the total number of segmented images (D×H×W) / (p×p×p); s <= p, and p is an integer multiple of s. For example, s = p = 32. When p is an integer multiple of s, ROI Pooling can be performed based on the first training model f to obtain the feature code corresponding to each image region in the masked image. The feature code corresponding to each image region constitutes the feature map of the masked image. Then, the feature map of the mask image to be processed is input into the fifth multilayer perceptron network model. Based on the fifth multilayer perceptron network, the feature map of the mask image to be processed is mapped and transformed, and the first feature vector is output.

[0178] S204. Determine the second feature vector of the three-dimensional image to be processed.

[0179] In one example, step S204 includes:

[0180] The second feature map is obtained by extracting features from the 3D image to be processed based on the second training model; the second feature map is then mapped and transformed based on the fourth multilayer perceptron network model to obtain the second feature vector.

[0181] For example, such as Figure 6 As shown, the 3D image to be processed is input into a second training model, which is an image feature encoder. This second training model can be a convolutional network model or a deep learning model. Then, feature extraction is performed on the 3D image to be processed based on the second training model to obtain a feature map of the 3D image (referred to as the second feature map). Next, the feature map of the 3D image to be processed is input into a multilayer perceptron network model (which can be called a fourth multilayer perceptron network model). Based on this fourth multilayer perceptron network, a mapping transformation is performed on the feature map of the 3D image to be processed, outputting a second feature vector. The second feature vector represents the global features of the 3D image to be processed. Using a multilayer perceptron network to perform mapping transformation on the feature map of the 3D image to be processed can isolate the adverse effects of the self-supervised training process provided in this embodiment on the vector.

[0182] In one example, the 3D image to be processed, `image`, is input into a second training model `mf`. The second training model `mf` is an image feature encoder; for example, it could be a CNN model, a Transformer model, or a combination of CNN and Transformer. If the second training model `mf` is a CNN model, then the number of convolutional layers in `mf` is the same as that in the ResNet34 model, but it uses 3D convolution. For example, the network parameters of the second training model `mf` are shown in Table 1 above.

[0183] Based on the second training model mf, feature extraction is performed on the 3D image to be processed, resulting in a feature map of the 3D image to be processed. This feature map is called the second feature map. The size of the 3D image to be processed is D×H×W, and the size of the output feature map is D / s×H / s×W / s, where s is the total downsampling ratio and is a positive integer greater than or equal to 1. The size of the feature map is (D×H×W) / (s×s×s), where s is a positive integer greater than or equal to 1. The size of the feature map is greater than or equal to the total number of segmented images (D×H×W) / (p×p×p); s <= p, and p is an integer multiple of s. For example, s = p = 32. When p is an integer multiple of s, ROI Pooling can be performed based on the second training model mf to obtain the feature code corresponding to each image region in the 3D image to be processed. The feature code corresponding to each image region constitutes the feature map of the 3D image to be processed. Then, the feature map of the 3D image to be processed is input into the fourth multilayer perceptron network model. Based on the fourth multilayer perceptron network, the feature map of the 3D image to be processed is mapped and transformed to output the second feature vector.

[0184] S205. Obtain the third feature vector of each other 3D image. The third feature vector represents the global features of the other 3D images.

[0185] For example, each other 3D image is input into a second training model, which is an image feature encoder; the second training model is a convolutional network model or a deep learning model; then, feature extraction processing is performed on the other 3D images based on the second training model to obtain feature maps of the other 3D images; then, the feature maps of the other 3D images are input into a multilayer perceptron network model (which can be called a sixth multilayer perceptron network model), and the feature maps of the other 3D images are mapped and transformed based on the sixth multilayer perceptron network to output the third feature vector of each other 3D image. It can be seen that the third feature vector of the other 3D images represents the global features of the other 3D images.

[0186] Each third feature vector can be stored in a dynamic queue.

[0187] All of the above-mentioned multilayer perceptron network models can adopt existing MLP network models; for example, each of the above-mentioned multilayer perceptron network models can adopt an MLP network model including one hidden layer, or an MLP network model including multiple hidden layers. The MLP network model includes a first Norm layer, a first activation layer, a first linear layer, a second Norm layer, a second activation layer, and a second linear layer.

[0188] S206. Update the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors.

[0189] In one example, step S206 specifically includes:

[0190] Based on the first feature vector, the second feature vector, and the preset hyperparameter coefficients, the first calculation parameter is determined; and based on the first feature vector, each of the third feature vectors, and the preset hyperparameter coefficients, the second calculation parameter is determined; based on the first calculation parameter and the second calculation parameter, a decision function is established; gradient backpropagation processing is performed on the decision function to update the parameters of the first model to be trained.

[0191] For example, a positive sample pair is formed by combining the first feature vector and the second feature vector; each third feature vector is read from the dynamic queue described above, and each negative sample pair is formed by combining the first feature vector and each third feature vector. It can be seen that a positive sample pair includes the first feature vector and the second feature vector of the same 3D image to be processed (the positive sample pair includes the first feature vector of the mask image to be processed and the second feature vector of the 3D image to be processed); a negative sample pair includes the first feature vector of the 3D image to be processed and the third feature vector of other 3D images (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vector of other 3D images).

[0192] Based on positive sample pairs (which include the first and second eigenvectors) and preset hyperparameter coefficients, the first computational parameter exp(q·k) is determined. + / τ); where q is the first eigenvector, k + Let be the second feature vector, τ be the preset hyperparameter coefficients, and exp be an exponential function with base e. The second calculation parameter is determined based on each negative sample pair (each negative sample pair includes the first feature vector and each third feature vector) and the preset hyperparameter coefficients. Where q is the first eigenvector, k i Let τ be the i-th third feature vector, and τ be the preset hyperparameter coefficients; N is the total number of other images, and N is also the total number of third feature vectors (N is also the length of the dynamic queue mentioned above). N is a positive integer greater than or equal to 1, and i is a positive integer greater than or equal to 1 and less than or equal to N.

[0193] Then, a decision function is established based on the first and second calculation parameters. For decision function L q Gradient backpropagation is performed to update the parameters of the first model to be trained.

[0194] S207. The second feature vector is obtained by processing the 3D image to be processed based on the second training model; the parameters of the second training model are updated according to the updated first training model. The second training model, after meeting preset conditions, is used to extract the feature map of the 3D image to be analyzed.

[0195] In one example, step S207 specifically includes: updating the parameters of the second training model based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

[0196] For example, after updating the parameters of the first model to be trained, the parameters of the second model to be trained can be updated based on the new first model to be trained.

[0197] In one example, based on the updated parameters θ of the first model to be trained q The original parameters θ in the second model to be trained k And a preset update momentum m, to update the parameters of the second model to be trained; to obtain the updated parameters w of the second model to be trained. k =mθ k +(1-m)θ q .

[0198] It can be seen that the parameters of the second model to be trained are not updated by the gradient backpropagation method in the above training process, but are updated based on the updated parameters of the first model to be trained.

[0199] Repeat steps S203-S207 until the preset conditions are met. Then, the first training model after meeting the preset conditions can be used to extract feature maps from the 3D image to be analyzed; in this embodiment, the first training model after meeting the preset conditions is specifically used to extract feature maps from the mask image of the 3D image to be analyzed. The second training model after meeting the preset conditions is used to extract feature maps from the 3D image to be analyzed.

[0200] In this embodiment, based on the above embodiments, a 3D image to be processed and at least one other 3D image different from the 3D image to be processed are obtained; the 3D image to be processed is divided into multiple regions to obtain multiple segmented images; segmented images with variance less than a preset threshold are removed, thereby filtering the regional information of the image; a segmented image is randomly selected from the filtered segmented images as a local image. The following steps are repeated until the preset conditions are met: feature extraction processing is performed on the mask image to be processed based on the first training model to obtain the feature map of the mask image to be processed; mapping transformation processing is performed on the feature map of the mask image to be processed to obtain the first feature vector; feature extraction processing is performed on the 3D image to be processed based on the second training model to obtain the second feature map; mapping transformation processing is performed on the second feature map based on the fourth multilayer perceptron network model to obtain the second feature vector; the third feature vector of each other 3D image is obtained; the first feature vector and the second feature vector under the same 3D image to be processed are combined into a positive sample pair (the first feature vector of the mask image to be processed and the 3D image to be processed are combined into a positive sample pair). The first and second feature vectors of the image are used to form a positive sample pair. The similarity between the first and second feature vectors of the same 3D image is high. The first feature vector of the 3D image to be processed and the third feature vector of each other 3D image are used to form a negative sample pair (the negative sample pair includes the first feature vector of the mask image to be processed and the third feature vector of other 3D images). The similarity between the first and third feature vectors of different 3D images is low. Based on the positive sample pairs and each negative sample pair, the parameters of the first training model are updated. Then, the model is trained based on a small number of unlabeled samples. Finally, based on the updated parameters of the first training model, the parameters of the second training model are updated. This method can solve various defects in training models using data augmentation, improving the model's learning ability. A good first training model can be obtained based on the positive and negative sample pairs. The feature map of the mask image corresponding to the 3D image obtained by the first training model is accurate, and this feature map can be used in subsequent image recognition processes. Furthermore, based on the first training model, the second training model is updated. The feature map of the 3D image obtained by the second training model is accurate, and this feature map can be used in subsequent image recognition processes. Furthermore, the first training model obtained in this embodiment will not learn information such as data noise, and the feature map of the three-dimensional image obtained based on the first training model will not include information such as data noise.

[0201] Figure 7 A flowchart illustrating another training method for a pre-trained model applied to 3D images, as provided in this application embodiment, is shown below. Figure 7 As shown, the method provided in this embodiment includes the following steps:

[0202] S301. Obtain the three-dimensional image to be processed, the mask image to be processed corresponding to the three-dimensional image to be processed, and at least one other three-dimensional image different from the three-dimensional image to be processed.

[0203] In one example, step S301, when acquiring the 3D image to be processed and the corresponding mask image to be processed, specifically includes:

[0204] The process involves acquiring a 3D image to be processed, preprocessing the 3D image to obtain a local image corresponding to the 3D image to be processed, and determining a mask image to be processed based on the 3D image to be processed and the local image, wherein the size of the mask image to be processed is the same as the size of the 3D image to be processed.

[0205] In one example, "preprocessing the 3D image to be processed to obtain a local image corresponding to the 3D image to be processed" includes;

[0206] The 3D image to be processed is segmented to obtain multiple segmented images; the variance of each segmented image is determined, and segmented images with variance less than a preset threshold are removed to obtain filtered segmented images; one segmented image is randomly selected from the filtered segmented images to be the local image.

[0207] For example, the execution subject of this embodiment can be an electronic device, a terminal device, a server, a training device or device for an item recommendation model, or other devices or devices that can execute the scheme of this embodiment, without limitation.

[0208] This step can be referred to step S201 above, and will not be repeated here.

[0209] S302. Repeat the following steps S303-S309 until the preset conditions are met; wherein, the first model to be trained after the preset conditions are met is used to extract the feature map of the three-dimensional image to be analyzed.

[0210] For example, the following steps S303-S309 are repeated until a preset condition is met. The preset condition is: the number of repetitions is greater than or equal to a preset number. Alternatively, the preset condition is: the parameters of the first model to be trained no longer change, and the parameters of the second model to be trained no longer change.

[0211] S303. Determine the feature map of the mask image to be processed based on the first training model, and determine the feature map of the local image corresponding to the three-dimensional image to be processed based on the first training model; wherein, the mask image to be processed is obtained based on the three-dimensional image to be processed and the local image, and the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed.

[0212] In one example, the size of the feature map of the mask image to be processed is greater than or equal to the total number of the multiple segmented images; the size of the feature map of the local image corresponding to the 3D image to be processed is greater than or equal to the total number of the multiple segmented images.

[0213] For example, Figure 8 A schematic diagram of the training process of the pre-trained model provided in the embodiments of this application. Figure 2 ,like Figure 8 As shown, the masked image to be processed is input into the first training model, which is an image feature encoder. The first training model is a convolutional network model or a deep learning model. Then, based on the first training model, feature extraction processing is performed on the masked image to be processed to obtain the feature map of the masked image to be processed. The feature map of the masked image to be processed represents the global features of the three-dimensional image to be processed, excluding local images.

[0214] The local image patch_image is input into the first model to be trained. Based on the first model to be trained, the local image is processed to extract features and obtain the feature map of the local image. The feature map of the local image represents the local features of the 3D image to be processed.

[0215] S304. Based on the first multilayer perceptron network model, the feature map of the mask image to be processed is mapped and transformed to obtain the first sub-vector. Based on the second multilayer perceptron network model, the feature map of the local image is mapped and transformed to obtain the second sub-vector.

[0216] For example, the feature map of the mask image to be processed is input into a first multilayer perceptron network model. Based on the first multilayer perceptron network, the feature map of the mask image to be processed is subjected to mapping transformation processing, and a first sub-vector is output. The first sub-vector represents the global features of the 3D image to be processed, excluding local images. By using a multilayer perceptron network to perform mapping transformation processing on the feature map of the mask image to be processed, the adverse effects of the self-supervised training process provided in this embodiment on the vector can be isolated.

[0217] The feature map of the local image is input into the second multilayer perceptron network model. Based on the second multilayer perceptron network, the feature map of the local image is mapped and transformed, and a second sub-vector is output. The second sub-vector represents the local features of the 3D image to be processed (the feature map of the second sub-vector represents the local features of the local image). By using a multilayer perceptron network to map and transform the feature map of the local image, the adverse effects of the self-supervised training process provided in this embodiment on the vector can be isolated.

[0218] In one example, the masked image to be processed, `masked_image`, is input into a first training model `f`. The first training model `f` is an image feature encoder; for example, it may be a CNN model, a Transformer model, or a combination of CNN and Transformer. If the first training model `f` is a CNN model, then the number of convolutional layers in `f` is the same as that in the ResNet34 model, but `f` uses three-dimensional convolution (3D convolution). For example, the network parameters of the first training model are shown in Table 1 above.

[0219] Based on the first training model f, feature extraction is performed on the masked image to be processed, resulting in a feature map of the masked image. The size of the masked image is D×H×W, and the size of the output feature map is D / s×H / s×W / s, where s is the total downsampling ratio and is a positive integer greater than or equal to 1. The size of the feature map is (D×H×W) / (s×s×s), where s is a positive integer greater than or equal to 1. The size of the feature map is greater than or equal to the total number of segmented images (D×H×W) / (p×p×p); s <= p, and p is an integer multiple of s. For example, s = p = 32. When p is an integer multiple of s, ROI Pooling can be performed based on the first training model f to obtain the feature code corresponding to each image region in the masked image. The feature code corresponding to each image region constitutes the feature map of the masked image. Then, the feature map of the mask image to be processed is input into the first multilayer perceptron network model. Based on the first multilayer perceptron network, the feature map of the mask image to be processed is transformed, and the first sub-vector q is output. w First subvector q w This represents the global features of the 3D image to be processed, excluding local features. For example, qw = proj_m(proj(f(masked_image))); where masked_image is the mask image to be processed, f() represents the first training model to process the mask image, proj() represents the first multilayer perceptron network model to perform mapping transformation on the feature map of the mask image, and proj_m() represents the first sub-vector q obtained by performing another mapping transformation on the output vector of the first multilayer perceptron network model. w .

[0220] The local image `patch_image` is fed into a first training model `f`, identical to the first training model described above. Feature extraction is performed on the local image `patch_image` based on the first training model `f` to obtain a feature map of the local image. The size of the mask image to be processed is D×H×W, and the size of the output feature map of the mask image to be processed is D / s×H / s×W / s, where s is the total downsampling ratio and is a positive integer greater than or equal to 1. The size of the feature map of the local image is (D×H×W) / (s×s×s), where s is a positive integer greater than or equal to 1. The size of the feature map of the local image is greater than or equal to the total number of segmented images (D×H×W) / (p×p×p); s <= p, and p is an integer multiple of s. For example, s = p = 32. When p is an integer multiple of s, ROI Pooling can be performed based on the first training model `f` to obtain the feature code corresponding to each image region in the local image. The feature code corresponding to each image region constitutes the feature map of the local image. Then, the feature map of the local image is input into the second multilayer perceptron network model. Based on this second multilayer perceptron network, the feature map of the local image is mapped and transformed, and the second sub-vector qp is output. The second sub-vector qp represents the local features of the 3D image to be processed. For example, qp = proj_p(proj(f(patch_image))); where patch_image is the local image, f() represents the first training model to process the local image; proj() represents the second multilayer perceptron network model that performs the mapping and transformation of the feature map of the local image; proj_p() represents the second sub-vector qp obtained by performing another mapping and transformation on the vector output by the second multilayer perceptron network model. p .

[0221] The first and second multilayer perceptron network models can both be referred to as specific projection modules. These specific projection modules are used to generate feature vectors after mapping and transforming the image. Both the first and second multilayer perceptron network models can employ existing MLP network models; for example, the first and second multilayer perceptron network models can be MLP network models with one hidden layer, or MLP network models with multiple hidden layers.

[0222] S305. Determine the first eigenvector based on the first subvector and the second subvector.

[0223] In one example, step S305 includes the following implementation methods:

[0224] The first implementation of step S305 involves performing a recovery mapping projection transformation on the first sub-vector and the second sub-vector based on the third multilayer perceptron network model to obtain the first feature vector.

[0225] The second implementation of step S305 involves performing feature concatenation on the first sub-vector and the second sub-vector to obtain the first feature vector.

[0226] For example, the first sub-vector represents the global features of the 3D image to be processed, excluding local features, while the second sub-vector represents the local features of the 3D image to be processed. To obtain the complete features of the 3D image to be processed, a first feature vector needs to be generated based on the first and second sub-vectors. It is understood that in this embodiment, the first feature vector represents the global features of the 3D image to be processed. The first feature vector obtained based on the local image and the mask image to be processed ensures the learning effect of the trained model.

[0227] In one example, such as Figure 8 As shown, the first subvector q w Second subvector q p The input is fed into the third multilayer perceptron network model; based on the third multilayer perceptron network model, the first sub-vector q is processed. w Second subvector q p Perform a restoration mapping projection transformation to output the first feature vector q = pred(q w+ q p ); where pred() represents the restoration mapping projection transformation process on the vector. By performing restoration mapping projection transformation on the first sub-vector and the second sub-vector, the resulting first feature vector represents the complete features of the 3D image to be processed; and it makes the features more varied, and the features included in the first feature vector are richer.

[0228] The aforementioned third-layer perceptron network model is referred to as the prediction module; the prediction module is used to perform recovery mapping projection processing on the feature vectors. The third-layer perceptron network model can adopt existing MLP network models; for example, the third-layer perceptron network model can adopt an MLP network model with one hidden layer, or it can adopt an MLP network model with multiple hidden layers.

[0229] Alternatively, in another example, the first feature vector can be generated by directly concatenating the features of the first and second sub-vectors.

[0230] S306. Determine the second feature vector of the three-dimensional image to be processed.

[0231] In one example, step S306 includes:

[0232] The second feature map is obtained by extracting features from the 3D image to be processed based on the second training model; the second feature map is then mapped and transformed based on the fourth multilayer perceptron network model to obtain the second feature vector.

[0233] For example, such as Figure 8 As shown, the 3D image to be processed is input into a second training model, which is an image feature encoder. This second training model can be a convolutional network model or a deep learning model. Then, feature extraction is performed on the 3D image to be processed based on the second training model to obtain a feature map of the 3D image (referred to as the second feature map). Next, the feature map of the 3D image to be processed is input into a fourth multilayer perceptron network model. Based on the fourth multilayer perceptron network, the feature map of the 3D image to be processed is mapped and transformed to output a second feature vector. The second feature vector represents the global features of the 3D image to be processed. Using a multilayer perceptron network to perform mapping and transformation on the feature map of the 3D image to be processed can isolate the adverse effects of the self-supervised training process provided in this embodiment on the vector.

[0234] The fourth multilayer perceptron network model can adopt an existing MLP network model; for example, the above-mentioned multilayer perceptron network models can adopt an MLP network model with one hidden layer, or an MLP network model with multiple hidden layers.

[0235] This step can also be performed in step S204 above, which will not be repeated here.

[0236] S307. Obtain the third feature vector of each other three-dimensional image. The third feature vector represents the global features of the other three-dimensional images.

[0237] For example, each other 3D image is input into a second training model, which is an image feature encoder; the second training model is a convolutional network model or a deep learning model; then, feature extraction processing is performed on the other 3D images based on the second training model to obtain feature maps of the other 3D images; then, the feature maps of the other 3D images are input into a multilayer perceptron network model (which can be called a sixth multilayer perceptron network model), and the feature maps of the other 3D images are mapped and transformed based on the sixth multilayer perceptron network to output the third feature vector of each other 3D image. It can be seen that the third feature vector of the other 3D images represents the global features of the other 3D images.

[0238] Each third feature vector can be stored in a dynamic queue.

[0239] The sixth-layer perceptron network model can adopt existing MLP network models; for example, the sixth-layer perceptron network model can adopt an MLP network model with one hidden layer, or it can adopt an MLP network model with multiple hidden layers.

[0240] S308. Update the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors.

[0241] In one example, step S308 specifically includes:

[0242] Based on the first feature vector, the second feature vector, and the preset hyperparameter coefficients, the first calculation parameter is determined; and based on the first feature vector, each of the third feature vectors, and the preset hyperparameter coefficients, the second calculation parameter is determined; based on the first calculation parameter and the second calculation parameter, a decision function is established; gradient backpropagation processing is performed on the decision function to update the parameters of the first model to be trained.

[0243] For example, the first feature vector, the second feature vector, and each of the third feature vectors are normalized. After normalization, a positive sample pair is formed based on the first feature vector and the second feature vector in this embodiment; wherein, the first feature vector represents the global features of the three-dimensional image to be processed (the first feature vector represents the features of the mask image to be processed and the features of the local image), and the second feature vector also represents the global features of the three-dimensional image to be processed.

[0244] like Figure 8 As shown, each third feature vector is read from the dynamic queue, and each negative sample pair is combined based on the first feature vector and each third feature vector; wherein, the first feature vector represents the global features of the three-dimensional image to be processed (the first feature vector represents the features of the mask image to be processed and the features of the local image), and the third feature vector also represents the global features of other three-dimensional images.

[0245] It can be seen that positive sample pairs include the first and second feature vectors of the same 3D image to be processed; negative sample pairs include the first feature vector of the 3D image to be processed and the third feature vector of other 3D images.

[0246] Based on positive sample pairs (which include the first and second eigenvectors) and preset hyperparameter coefficients, the first computational parameter exp(q·k) is determined. + / τ); where q is the first eigenvector, k + Let be the second feature vector, τ be the preset hyperparameter coefficients, and exp be an exponential function with base e. The second calculation parameter is determined based on each negative sample pair (each negative sample pair includes the first feature vector and each third feature vector) and the preset hyperparameter coefficients. Where q is the first eigenvector, k i Let τ be the i-th third feature vector, and τ be the preset hyperparameter coefficients; N is the total number of other images, and N is also the total number of third feature vectors (N is also the length of the dynamic queue mentioned above). N is a positive integer greater than or equal to 1, and i is a positive integer greater than or equal to 1 and less than or equal to N.

[0247] Then, a decision function is established based on the first and second calculation parameters. For decision function L q Gradient backpropagation is performed to update the parameters of the first model to be trained. Thus, during training, the parameters of the first model to be trained are updated through gradient backpropagation. The decision function L... q It can also be called a loss function.

[0248] Specifically, only the parameters of the first training model used to process the masked image (masked_image) are updated; then, the parameters of the first training model used to process the local image (patch_image) are updated synchronously. It can be seen that these two training models are the same. This is because the local image contains a large number of image-free regions (e.g., a large number of zero-value regions), which are unfavorable for training and updating model parameters. Therefore, the same training model is used to extract feature maps from both the masked image and the local image, and the same first and second multilayer perceptron network models are used.

[0249] S309. The second feature vector is obtained by processing the 3D image to be processed based on the second training model; the parameters of the second training model are updated according to the updated first training model. The second training model, after meeting preset conditions, is used to extract the feature map of the 3D image to be analyzed.

[0250] In one example, step S309 specifically includes: updating the parameters of the second training model based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

[0251] For example, after updating the parameters of the first model to be trained, the parameters of the second model to be trained can be updated based on the new first model to be trained.

[0252] In one example, based on the updated parameters θ of the first model to be trained q The original parameters θ in the second model to be trained k And a preset update momentum m, to update the parameters of the second model to be trained; to obtain the updated parameters w of the second model to be trained. k =mθ k+(1-m)θ q .

[0253] It can be seen that the parameters of the second model to be trained are not updated by the gradient backpropagation method in the above training process, but are updated based on the updated parameters of the first model to be trained.

[0254] Repeat steps S303-S309 until the preset conditions are met. Then, the first training model after meeting the preset conditions can be used to extract feature maps of the 3D image to be analyzed. In this embodiment, the first training model after meeting the preset conditions is specifically used to extract feature maps of the mask image of the 3D image to be analyzed, and can also be used to extract feature maps of local images of the 3D image to be analyzed. The second training model after meeting the preset conditions is used to extract feature maps of the 3D image to be analyzed.

[0255] Figure 9 A numerical schematic diagram of the decision function for the training process provided in the embodiments of this application is shown below. Figure 9 As shown, the decision function is also the loss function. The first model to be trained is trained using training samples (which include the 3D image to be processed and other 3D images) to establish the decision function; the parameters of the first model to be trained are updated based on the decision function. Figure 8 The x-axis represents the value of the decision function (the unit of the decision function value is not restricted). Figure 8 The ordinate represents the number of iteration rounds, with each round consisting of multiple iterations. For example, medical 3D images of the chest can be used to construct training samples.

[0256] Figure 10 A numerical schematic diagram of the decision function for the analysis process provided in the embodiments of this application, as shown below. Figure 10 As shown, the decision function is also the loss function. The first training model is used to process the validation samples (which include the 3D images to be analyzed) to obtain the decision function. Figure 9 The x-axis represents the value of the decision function (the unit of the decision function value is not restricted). Figure 10 The ordinate represents the number of iteration rounds, with each round consisting of multiple iterations. For example, a 3D medical image of the chest can be used to construct the validation sample. The "analysis process" refers to processing the validation sample (which includes the 3D image to be analyzed) using either the first trained model or the second trained model.

[0257] In this embodiment, based on the above embodiments, a three-dimensional image to be processed and at least one other three-dimensional image different from the three-dimensional image to be processed are obtained; the three-dimensional image to be processed is divided into multiple regions to obtain multiple segmented images; segmented images with variance less than a preset threshold are removed, thereby filtering the regional information of the image; and a segmented image is randomly selected from the filtered segmented images as a local image. Repeat the following steps until the preset conditions are met: Determine the first sub-vector of the mask image to be processed based on the first training model and the first multilayer perceptron network model; determine the second sub-vector of the local image corresponding to the 3D image to be processed based on the first training model and the second multilayer perceptron network model; determine the first feature vector based on the first and second sub-vectors. In this embodiment, the first feature vector represents the global features of the 3D image to be processed; obtain the third feature vector of each other 3D image; form a positive sample pair by combining the first and second feature vectors under the same 3D image to be processed, where the similarity between the first and second feature vectors of the same 3D image is high; form each negative sample pair by combining the first feature vector of the 3D image to be processed and the third feature vector of each other 3D image, where the similarity between the first and third feature vectors of different 3D images is low; update the parameters of the first training model based on the positive sample pairs and each negative sample pair; then train the model based on a small number of unlabeled samples; and update the parameters of the second training model based on the updated parameters of the first training model. This approach addresses various shortcomings of data augmentation methods in training models, enhancing their learning capabilities. It yields a well-designed first training model based on positive and negative sample pairs. The feature map of the mask image corresponding to the 3D image obtained from this first training model is accurate and can be used in subsequent image recognition processes. Furthermore, a second training model is updated based on the first training model, resulting in an accurate feature map of the 3D image obtained from this second training model, which can also be used in subsequent image recognition processes. Moreover, the first training model obtained in this embodiment does not learn data noise or similar information, and the feature map of the 3D image obtained based on the first training model does not include data noise or similar information.

[0258] Figure 11 A flowchart illustrating a method for extracting feature maps from a three-dimensional image, as provided in this application embodiment, is shown below. Figure 11 As shown, the method provided in this embodiment includes the following steps:

[0259] S401. Obtain the 3D image to be analyzed.

[0260] For example, the execution subject of this embodiment can be an electronic device, a terminal device, a server, a training device or device for an item recommendation model, or other devices or devices that can execute the scheme of this embodiment, without limitation.

[0261] First, acquire the 3D image to be analyzed. This can be received from other devices or read from a storage medium. For example, the 3D image to be analyzed might be a 3D medical image.

[0262] S402. Input the three-dimensional image to be analyzed into the feature extraction model to obtain the feature map of the three-dimensional image to be analyzed; wherein, the feature extraction model is the first training model after reaching the preset conditions in the training method of the pre-trained model applied to the three-dimensional image.

[0263] For example, based on the above embodiments, a first training model with preset conditions has been obtained. The 3D image to be analyzed can then be input into this first training model to obtain a feature map of the 3D image. For instance, the 3D image to be analyzed can be directly input into the first training model with preset conditions to obtain a feature map; or, a mask image of the 3D image to be analyzed can be determined and input into the first training model with preset conditions to obtain a feature map.

[0264] Furthermore, a portion of the 3D image to be analyzed can be extracted to obtain a local image of the 3D image to be analyzed; the local image of the 3D image to be analyzed is then input into the first training model under preset conditions to obtain the feature map of the local image of the 3D image to be analyzed.

[0265] S403. Based on the feature map of the three-dimensional image to be analyzed, determine the recognition result of the three-dimensional image to be analyzed.

[0266] In one example, step S403 includes: determining the feature vector of the three-dimensional image to be analyzed based on the feature map of the three-dimensional image to be analyzed; inputting the feature vector of the three-dimensional image to be analyzed into a preset recognition model to obtain the recognition result of the three-dimensional image to be analyzed.

[0267] The recognition result can be any one of the following: the category of the 3D image to be analyzed, the characteristics of the 3D image to be analyzed, or the structural segmentation result of the 3D image to be analyzed.

[0268] For example, based on the needs of the scene, the feature map of the 3D image to be analyzed is processed to obtain the recognition result of the 3D image to be analyzed.

[0269] The analysis can be performed using a classification model on the feature map of the 3D image to determine its category. Alternatively, a feature recognition model can be used to analyze the feature map and determine the image's characteristics; for example, if the image is a 3D medical image, the characteristic could be determined as whether image abnormalities are present. Alternatively, a segmentation model can be used to analyze the feature map and determine the structural segmentation result of the 3D image.

[0270] In one example, the feature map of the 3D image to be analyzed is input into the MLP network model, and the feature vector of the 3D image to be analyzed is output; the feature vector of the 3D image to be analyzed is input into the preset recognition model to obtain the recognition result of the 3D image to be analyzed.

[0271] Furthermore, based on the above embodiments, a second training model under preset conditions has been obtained. The three-dimensional image to be analyzed can then be input into the second training model under preset conditions to obtain a feature map of the three-dimensional image to be analyzed (the feature map represents the global features of the three-dimensional image to be analyzed). Then, based on the feature map, the recognition result of the three-dimensional image to be analyzed is obtained.

[0272] Three-dimensional images of different human organs are input into an existing unsupervised model (e.g., a randomly initialized model) to obtain feature maps. Image segmentation is then performed based on these feature maps to obtain the segmentation accuracy for each human organ's three-dimensional image. As shown in Table 2, after image segmentation based on the feature maps obtained from the existing unsupervised model, the segmentation accuracy for the esophagus is 0.687, for the heart it is 0.919, for the trachea it is 0.842, and for the aorta it is 0.842, with an average accuracy of 0.835.

[0273] Three-dimensional images of different human organs are input into a first training model or a trained model to obtain feature maps. Image segmentation is then performed based on these feature maps to obtain the segmentation accuracy for each human organ's three-dimensional image. As shown in Table 2, after image segmentation based on the feature maps obtained from the first training model or the trained model provided in this embodiment, the segmentation accuracy for the esophagus is 0.745, for the heart it is 0.93, for the trachea it is 0.867, and for the aorta it is 0.911, with an average accuracy of 0.863. This shows that the average accuracy is improved by 0.028 compared to existing methods.

[0274] Table 2. Experimental Comparison Results

[0275]

[0276]

[0277] In this embodiment, the first or second trained model provided in the above embodiments is used to determine the feature map of the 3D image to be analyzed. Since the parameters of the first trained model are updated based on positive and negative sample pairs in the above embodiments, and the model is trained using a small number of unlabeled samples, various defects of training models using data augmentation methods can be solved, improving the model's learning ability. A good first trained model can be obtained based on positive and negative sample pairs, and the feature map of the 3D image obtained by the model is accurate and can be used in subsequent image recognition processes. Furthermore, the first trained model obtained in this embodiment does not learn information such as data noise, and the feature map of the 3D image obtained based on the first trained model does not include information such as data noise.

[0278] It should be noted that, due to space limitations, this application does not exhaustively list all possible implementation methods. As long as the features are not contradictory, they can be freely combined to form optional implementation methods of this application.

[0279] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0280] Figure 12 This is a schematic diagram of a training device for a pre-trained model applied to 3D images, provided as an embodiment of this application; this device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. Figure 12 As shown, the device includes: a first acquisition unit 31, a second acquisition unit 32, an execution unit 33, a first determination unit 34, a second determination unit 35, a third acquisition unit 36, and a first update unit 37.

[0281] The first acquisition unit 31 is used to acquire the three-dimensional image to be processed and the mask image to be processed corresponding to the three-dimensional image to be processed.

[0282] The second acquisition unit 32 is used to acquire at least one other three-dimensional image that is different from the three-dimensional image to be processed.

[0283] The execution unit 33 is used to repeat the process of the first determining unit 34, the second determining unit 35, the third obtaining unit 36, and the first updating unit 37 until the preset conditions are met.

[0284] The first determining unit 34 is used to determine the first feature vector of the mask image to be processed based on the first model to be trained.

[0285] The second determining unit 35 is used to determine the second feature vector of the three-dimensional image to be processed.

[0286] The third acquisition unit 36 ​​is used to acquire the third feature vector of each other three-dimensional image, and the third feature vector represents the global features of the other three-dimensional images.

[0287] The first update unit 37 is used to update the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors.

[0288] Among them, the first model to be trained after meeting the preset conditions is used to extract the feature map of the three-dimensional image to be analyzed.

[0289] For example, this embodiment can refer to the above embodiments, and will not be repeated here.

[0290] Figure 13 This is a schematic diagram of another training device for a pre-trained model applied to three-dimensional images, provided as an embodiment of this application; this device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. Figure 12 Based on the illustrated embodiments, as Figure 13 As shown, in this device, the first update unit 37 includes:

[0291] The first determining module 371 is used to determine the first calculation parameter based on the first feature vector, the second feature vector, and the preset hyperparameter coefficients; and to determine the second calculation parameter based on the first feature vector, each third feature vector, and the preset hyperparameter coefficients.

[0292] Module 372 is established to establish a decision function based on the first calculation parameter and the second calculation parameter.

[0293] The update module 373 is used to perform gradient backpropagation on the decision function to update the parameters of the first model to be trained.

[0294] In one example, the second feature vector is obtained by processing the 3D image to be processed based on the second model to be trained; the apparatus provided in this embodiment also includes:

[0295] The second update unit 41 is used to update the parameters of the second training model according to the updated first training model after the first update unit 37 updates the parameters of the first training model according to the first feature vector, the second feature vector and each third feature vector.

[0296] Among them, the second training model, after meeting the preset conditions, is used to extract the feature map of the three-dimensional image to be analyzed.

[0297] It is understood that the process of the first determining unit 34, the second determining unit 35, the third obtaining unit 36, the first updating unit 37, and the second updating unit 41 needs to be repeated until the preset conditions are met.

[0298] In one example, the second update unit 41 is specifically used for:

[0299] The parameters of the second training model are updated based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

[0300] In one example, the first acquisition unit 31 includes:

[0301] The acquisition module 311 is used to acquire the three-dimensional image to be processed, preprocess the three-dimensional image to be processed, and obtain a local image corresponding to the three-dimensional image to be processed.

[0302] The second determining module 312 is used to determine a mask image to be processed based on the three-dimensional image to be processed and the local image, wherein the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed.

[0303] In one example, module 311 is retrieved, specifically for:

[0304] The 3D image to be processed is segmented to obtain multiple segmented images; the variance of each segmented image is determined, and segmented images with variance less than a preset threshold are removed to obtain filtered segmented images; one segmented image is randomly selected from the filtered segmented images to be the local image.

[0305] In one example, the first determining unit 34 includes:

[0306] The first processing module 341 is used to perform feature extraction processing on the mask image to be processed based on the first model to be trained, so as to obtain the feature map of the mask image to be processed.

[0307] The second processing module 342 is used to perform mapping transformation on the feature map of the mask image to be processed, to obtain the first feature vector. In one example, the size of the first feature map is greater than or equal to the total number of the multiple segmented images.

[0308] Alternatively, the first determining unit 34 includes:

[0309] The third determining module 343 is used to determine the feature map of the mask image to be processed based on the first training model, and to determine the feature map of the local image corresponding to the three-dimensional image to be processed based on the first training model; wherein, the mask image to be processed is obtained based on the three-dimensional image to be processed and the local image, and the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed.

[0310] The third processing module 344 is used to perform mapping transformation processing on the feature map of the mask image to be processed based on the first multilayer perceptron network model to obtain the first sub-vector, and to perform mapping transformation processing on the feature map of the local image based on the second multilayer perceptron network model to obtain the second sub-vector.

[0311] The fourth determining module 345 is used to determine the first feature vector based on the first sub-vector and the second sub-vector.

[0312] In one example, the fourth determining module 345 is specifically used for:

[0313] The first feature vector is obtained by performing a restoration mapping projection transformation on the first and second sub-vectors based on the third multilayer perceptron network model; or, the first feature vector is obtained by performing feature concatenation on the first and second sub-vectors.

[0314] In one example, the second determining unit 35 includes:

[0315] The fifth determining module 351 is used to perform feature extraction processing on the three-dimensional image to be processed based on the second training model to obtain the second feature map.

[0316] The fourth processing module 352 is used to perform mapping transformation processing on the second feature map based on the fourth multilayer perceptron network model to obtain the second feature vector.

[0317] In one example, the first model to be trained is a convolutional network model or a deep learning model; the second model to be trained for processing the 3D image to be processed is a convolutional network model or a deep learning model.

[0318] For example, this embodiment can refer to the above embodiments, and will not be repeated here.

[0319] It should be noted that the training device for the pre-trained model of three-dimensional images provided in the above embodiments is only illustrated by the division of the above functional modules when executing the training method for the pre-trained model of three-dimensional images. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the embodiments of the training device for the pre-trained model of three-dimensional images provided in the above embodiments and the embodiments of the training method for the pre-trained model of three-dimensional images belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0320] Figure 14 This is a schematic diagram of a feature map extraction device for a three-dimensional image provided in an embodiment of this application; this device can be implemented as all or part of an electronic device through software, hardware, or a combination of both. Figure 14 As shown, the device includes: an acquisition unit 51, a processing unit 52, and a determination unit 53.

[0321] Acquisition unit 51 is used to acquire the three-dimensional image to be analyzed.

[0322] The processing unit 52 is used to input the three-dimensional image to be analyzed into the feature extraction model to obtain the feature map of the three-dimensional image to be analyzed; wherein the feature extraction model is the first training model in the device of any one of claims 15-26 after reaching the preset conditions.

[0323] The determining unit 53 is used to determine the recognition result of the three-dimensional image to be analyzed based on the feature map of the three-dimensional image to be analyzed.

[0324] In one example, unit 53 is defined as being used specifically for:

[0325] Based on the feature map of the 3D image to be analyzed, the feature vector of the 3D image to be analyzed is determined; the feature vector of the 3D image to be analyzed is input into the preset recognition model to obtain the recognition result of the 3D image to be analyzed.

[0326] The recognition result can be any one of the following: the category of the 3D image to be analyzed, the characteristics of the 3D image to be analyzed, or the structural segmentation result of the 3D image to be analyzed.

[0327] For example, this embodiment can refer to the above embodiments, and will not be repeated here.

[0328] It should be noted that the feature map extraction device for three-dimensional images provided in the above embodiments is only illustrated by the division of the above functional modules when executing the feature map extraction method for three-dimensional images. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the embodiments of the feature map extraction device for three-dimensional images provided in the above embodiments and the embodiments of the feature map extraction method for three-dimensional images belong to the same concept, and the implementation process is detailed in the method embodiments, which will not be repeated here.

[0329] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0330] Figure 15 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application, such as... Figure 15 As shown, the electronic device may include a processor 61 and a memory 62; wherein the memory 62 stores a computer program adapted to be loaded by the processor 61 and executed by the above-described method steps.

[0331] The electronic device may also include a transmitter 63 and a receiver 64.

[0332] Figure 16 A schematic diagram of another electronic device is provided for the embodiments of this application. For example... Figure 16 As shown, the electronic device 1000 may include: at least one processor 1001, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002.

[0333] The communication bus 1002 is used to realize the connection and communication between these components.

[0334] The user interface 1003 may include a display screen and a camera. Optionally, the user interface 1003 may also include a standard wired interface and a wireless interface.

[0335] The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface).

[0336] The processor 1001 may include one or more processing cores. The processor 1001 connects to various parts within the electronic device 1000 using various interfaces and lines. It executes various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in the memory 1005, and by calling data stored in the memory 1005. Optionally, the processor 1001 may be implemented using at least one hardware form selected from Digital Signal Processing (DSP), Field-Programmable Gate Array (FPGA), and Programmable Logic Array (PLA). The processor 1001 may integrate one or more of the following: a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), and a modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the content to be displayed on the screen; and the modem handles wireless communication. It is understood that the modem may also be implemented as a separate chip, without being integrated into the processor 1001.

[0337] The memory 1005 may include random access memory (RAM) or read-only memory. Optionally, the memory 1005 may include a non-transitory computer-readable storage medium. The memory 1005 can be used to store instructions, programs, code, code sets, or instruction sets. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-described method embodiments, etc.; the data storage area may store data involved in the above-described method embodiments, etc. Optionally, the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001. Figure 16 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and operating applications for electronic devices.

[0338] exist Figure 16In the electronic device 1000 shown, the user interface 1003 is mainly used to provide an input interface for the user and to obtain the user input data; while the processor 1001 can be used to call the electronic device operation application stored in the memory 1005 and specifically execute the methods provided in the above embodiments.

[0339] This application also provides a computer storage medium that can store multiple instructions. The instructions are adapted to be loaded by a processor and executed as described in the above embodiments. For details of the execution process, please refer to the specific description of the above embodiments, which will not be repeated here.

[0340] The device containing the storage medium can be an electronic device.

[0341] This application also provides a computer program product, which includes a computer program stored in a readable storage medium. At least one processor of a terminal device can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the terminal device to perform the solution provided in any of the above embodiments.

[0342] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0343] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0344] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0345] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0346] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0347] Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0348] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0349] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0350] The above are merely embodiments of this application and are not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for training a pre-trained model applied to a three-dimensional image, characterized in that, The method includes: Obtain the 3D image to be processed, the mask image corresponding to the 3D image to be processed, and at least one other 3D image different from the 3D image to be processed, and repeat the following steps until the preset conditions are met: The first feature vector of the mask image to be processed is determined based on the first model to be trained, and the second feature vector of the three-dimensional image to be processed is determined; and the third feature vector of each of the other three-dimensional images is obtained, wherein the third feature vector represents the global features of the other three-dimensional images; Update the parameters of the first model to be trained based on the first feature vector, the second feature vector, and each of the third feature vectors; Among them, the first training model after meeting the preset conditions is used to extract the feature map of the three-dimensional image to be analyzed. The step of determining the first feature vector of the mask image to be processed based on the first model to be trained includes: The feature map of the mask image to be processed is determined based on the first training model, and the feature map of the local image corresponding to the three-dimensional image to be processed is determined based on the first training model; wherein, the mask image to be processed is obtained based on the three-dimensional image to be processed and the local image, and the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed; The feature map of the mask image to be processed is mapped and transformed based on the first multilayer perceptron network model to obtain the first sub-vector, and the feature map of the local image is mapped and transformed based on the second multilayer perceptron network model to obtain the second sub-vector. The first feature vector is determined based on the first sub-vector and the second sub-vector.

2. The method according to claim 1, characterized in that, Based on the first feature vector, the second feature vector, and each of the third feature vectors, the parameters of the first model to be trained are updated, including: A first calculation parameter is determined based on the first feature vector, the second feature vector, and the preset hyperparameter coefficients; and a second calculation parameter is determined based on the first feature vector, each of the third feature vectors, and the preset hyperparameter coefficients. A decision function is established based on the first calculation parameter and the second calculation parameter; The decision function is subjected to gradient backpropagation to update the parameters of the first model to be trained.

3. The method according to claim 1, characterized in that, The second feature vector is obtained by processing the 3D image to be processed based on the second training model; after updating the parameters of the first training model according to the first feature vector, the second feature vector, and each of the third feature vectors, the method further includes: Update the parameters of the second model to be trained based on the updated first model to be trained; The second training model, after meeting the preset conditions, is used to extract the feature map of the three-dimensional image to be analyzed.

4. The method according to claim 3, characterized in that, Based on the updated first model to be trained, update the parameters of the second model to be trained, including: The parameters of the second training model are updated based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

5. The method according to claim 1, characterized in that, Acquiring the 3D image to be processed and the corresponding mask image to be processed includes: The three-dimensional image to be processed is acquired, and the three-dimensional image to be processed is preprocessed to obtain a local image corresponding to the three-dimensional image to be processed. Based on the 3D image to be processed and the local image, a mask image to be processed is determined, wherein the size of the mask image to be processed is the same as the size of the 3D image to be processed.

6. The method according to claim 5, characterized in that, Preprocessing the three-dimensional image to be processed to obtain a local image corresponding to the three-dimensional image to be processed includes: The three-dimensional image to be processed is segmented to obtain multiple segmented images; Determine the variance of each segmented image and remove segmented images with variance less than a preset threshold to obtain filtered segmented images. One of the segmented images after filtering is randomly selected as the local image.

7. The method according to claim 6, characterized in that, The size of the first feature vector is greater than or equal to the total number of the plurality of segmented images.

8. The method according to claim 1, characterized in that, Determining the first feature vector based on the first sub-vector and the second sub-vector includes: Based on the third multilayer perceptron network model, the first sub-vector and the second sub-vector are subjected to recovery mapping projection transformation to obtain the first feature vector; Alternatively, feature concatenation can be performed on the first sub-vector and the second sub-vector to obtain the first feature vector.

9. The method according to any one of claims 1-8, characterized in that, Determining the second feature vector of the 3D image to be processed includes: Based on the second training model, feature extraction processing is performed on the three-dimensional image to be processed to obtain a second feature map; The second feature map is mapped and transformed based on the fourth multilayer perceptron network model to obtain the second feature vector.

10. The method according to any one of claims 1-8, characterized in that, The first model to be trained is a convolutional network model or a deep learning model; the second model to be trained for processing the three-dimensional image to be processed is a convolutional network model or a deep learning model.

11. A method for extracting feature maps from a three-dimensional image, characterized in that, The method includes: A 3D image to be analyzed is acquired, and the 3D image to be analyzed is input into a feature extraction model to obtain a feature map of the 3D image to be analyzed; wherein, the feature extraction model is the first training model after reaching the preset condition in the method described in any one of claims 1-10; Based on the feature map of the three-dimensional image to be analyzed, the recognition result of the three-dimensional image to be analyzed is determined.

12. The method according to claim 11, characterized in that, Based on the feature map of the three-dimensional image to be analyzed, the recognition result of the three-dimensional image to be analyzed is determined, including: Based on the feature map of the three-dimensional image to be analyzed, the feature vector of the three-dimensional image to be analyzed is determined; The feature vector of the three-dimensional image to be analyzed is input into a preset recognition model to obtain the recognition result of the three-dimensional image to be analyzed; The identification result can be any one of the following: the category of the three-dimensional image to be analyzed, the characteristics of the three-dimensional image to be analyzed, or the structural segmentation result of the three-dimensional image to be analyzed.

13. A training device for a pre-trained model applied to three-dimensional images, characterized in that, The device includes: The first acquisition unit is used to acquire the three-dimensional image to be processed and the mask image to be processed corresponding to the three-dimensional image to be processed. The second acquisition unit is used to acquire at least one other three-dimensional image that is different from the three-dimensional image to be processed. An execution unit is used to repeat the following units until a preset condition is met: The first determining unit is used to determine the first feature vector of the mask image to be processed based on the first training model; The second determining unit is used to determine the second feature vector of the three-dimensional image to be processed; The third acquisition unit is used to acquire the third feature vector of each of the other three-dimensional images, wherein the third feature vector represents the global features of the other three-dimensional images; The first update unit is used to update the parameters of the first model to be trained based on the first feature vector, the second feature vector and each of the third feature vectors. Among them, the first training model after meeting the preset conditions is used to extract the feature map of the three-dimensional image to be analyzed. The first determining unit includes: The third determining module is used to determine the feature map of the mask image to be processed based on the first training model, and to determine the feature map of the local image corresponding to the three-dimensional image to be processed based on the first training model; wherein the mask image to be processed is obtained based on the three-dimensional image to be processed and the local image, and the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed. The third processing module is used to perform mapping transformation processing on the feature map of the mask image to be processed based on the first multilayer perceptron network model to obtain the first sub-vector, and to perform mapping transformation processing on the feature map of the local image based on the second multilayer perceptron network model to obtain the second sub-vector. The fourth determining module is used to determine the first feature vector based on the first sub-vector and the second sub-vector.

14. The apparatus according to claim 13, characterized in that, The first update unit includes: The first determining module is used to determine a first calculation parameter based on the first feature vector, the second feature vector, and preset hyperparameter coefficients; and to determine a second calculation parameter based on the first feature vector, each of the third feature vectors, and the preset hyperparameter coefficients. A module is established to establish a decision function based on the first calculation parameters and the second calculation parameters; An update module is used to perform gradient backpropagation processing on the decision function to update the parameters of the first model to be trained.

15. The apparatus according to claim 13, characterized in that, The second feature vector is obtained by processing the three-dimensional image to be processed based on the second training model; the device further includes: The second update unit is used to update the parameters of the second model to be trained based on the updated first model to be trained, after the first update unit updates the parameters of the first model to be trained based on the first feature vector, the second feature vector and each of the third feature vectors. The second training model, after meeting the preset conditions, is used to extract the feature map of the three-dimensional image to be analyzed.

16. The apparatus according to claim 15, characterized in that, The second update unit is specifically used for: The parameters of the second training model are updated based on the updated first training model, the original parameters in the second training model, and the preset update momentum.

17. The apparatus according to claim 13, characterized in that, The first acquisition unit includes: The acquisition module is used to acquire the three-dimensional image to be processed, preprocess the three-dimensional image to be processed, and obtain a local image corresponding to the three-dimensional image to be processed. The second determining module is used to determine the mask image to be processed based on the three-dimensional image to be processed and the local image, wherein the size of the mask image to be processed is the same as the size of the three-dimensional image to be processed.

18. The apparatus according to claim 17, characterized in that, The acquisition module is specifically used for: The three-dimensional image to be processed is segmented to obtain multiple segmented images; Determine the variance of each segmented image and remove segmented images with variance less than a preset threshold to obtain filtered segmented images. One of the segmented images after filtering is randomly selected as the local image.

19. The apparatus according to claim 18, characterized in that, The size of the first feature vector is greater than or equal to the total number of the plurality of segmented images.

20. The apparatus according to claim 13, characterized in that, The fourth determining module is specifically used for: Based on the third multilayer perceptron network model, the first sub-vector and the second sub-vector are subjected to recovery mapping projection transformation to obtain the first feature vector; Alternatively, feature concatenation can be performed on the first sub-vector and the second sub-vector to obtain the first feature vector.

21. The apparatus according to any one of claims 13-20, characterized in that, The second determining unit includes: The fifth determining module is used to perform feature extraction processing on the three-dimensional image to be processed based on the second training model to obtain a second feature map; The fourth processing module is used to perform mapping transformation processing on the second feature map based on the fourth multilayer perceptron network model to obtain the second feature vector.

22. The apparatus according to any one of claims 13-20, characterized in that, The first model to be trained is a convolutional network model or a deep learning model; the second model to be trained for processing the three-dimensional image to be processed is a convolutional network model or a deep learning model.

23. A device for extracting feature maps from a three-dimensional image, characterized in that, The device includes: The acquisition unit is used to acquire the 3D image to be analyzed. A processing unit is used to input the three-dimensional image to be analyzed into a feature extraction model to obtain a feature map of the three-dimensional image to be analyzed; wherein the feature extraction model is the first training model in the apparatus according to any one of claims 13-22 after reaching the preset conditions; The determining unit is used to determine the recognition result of the three-dimensional image to be analyzed based on the feature map of the three-dimensional image to be analyzed.

24. The apparatus according to claim 23, characterized in that, The determining unit is specifically used for: Based on the feature map of the three-dimensional image to be analyzed, the feature vector of the three-dimensional image to be analyzed is determined; The feature vector of the three-dimensional image to be analyzed is input into a preset recognition model to obtain the recognition result of the three-dimensional image to be analyzed; The identification result can be any one of the following: the category of the three-dimensional image to be analyzed, the characteristics of the three-dimensional image to be analyzed, or the structural segmentation result of the three-dimensional image to be analyzed.

25. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and executed as described in any one of claims 1-10, or the computer program adapted to be loaded by the processor and executed as described in any one of claims 11-12.

26. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions, the instructions being adapted to be loaded by a processor and executed as described in any one of claims 1-10, or the instructions being adapted to be loaded by a processor and executed as described in any one of claims 11-12.

27. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-10, or the computer program that, when executed by a processor, implements the method according to any one of claims 11-12.

Images