A multi-class glomerulus automatic segmentation method for renal biopsy images based on deep learning

Abstract

This invention discloses a deep learning-based method for automatic multi-class glomerulus segmentation in renal biopsy images, comprising the following steps: Step 1: Preprocessing of renal biopsy images; Step 2: Model training (encoder portion); Step 3: Result fusion; Step 4: Model testing. This method utilizes the Transform module and the VGG module in the encoder and decoder, respectively, to process the complete contextual information of glomeruli at a large scale while ensuring accurate glomerular contour segmentation. This method also divides the semantic segmentation process used in traditional techniques into a segmentation branch and a classification branch, ensuring the uniformity and integrity of the segmentation results for each glomerulus.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to a method for automatic segmentation of multiple glomeruli in renal puncture images based on deep learning. Background Art

[0002] Chronic kidney disease (CKD) is a serious threat to human health. CKD primarily occurs as a secondary or primary condition in the renal tubules, glomeruli, interstitium, and renal vasculature. Chronic kidney disease includes glomerulonephritis, lupus erythematosus nephritis, immunoglobulin A (IgA) nephropathy, and diabetic nephropathy, among others. During renal biopsy, the assessment and classification of different glomeruli within the kidney is crucial for defining the nature and pathological type of renal disease, and is crucial for determining treatment options and prognosis. In clinical practice, the quantification and analysis of glomeruli in renal biopsies is often performed manually by expert pathologists. However, due to the inherent subjectivity between pathologists and the time-consuming nature of manual analysis, the efficiency and reproducibility of kidney disease diagnosis are low. Therefore, the design of a computer-assisted system that can automatically and accurately segment glomeruli within the kidney is crucial.

[0003] With the widespread application of deep learning technology in the field of pathological images, accurate segmentation and classification of different types of glomeruli is the key to analyzing renal puncture images. For traditional semantic segmentation models, multi-class glomerular segmentation faces the following problems:

[0004] 1. Insufficient segmentation accuracy. Glomeruli are large in size, and traditional networks have difficulty processing the complete contextual information of a single glomerulus, resulting in the segmentation of different categories of regions within the same glomerulus.

[0005] 2. The number of labeled glomeruli is small and there is a serious imbalance between the number of samples of different types. Traditional semantic segmentation networks find it difficult to increase the weight of difficult-to-classify glomeruli during network training, resulting in the network being ultimately affected by categories with large sample sizes. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the defects of the prior art and provide a method for automatic segmentation of multiple glomeruli in renal puncture images based on deep learning.

[0007] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0008] In a second aspect, the present invention provides a method for automatic segmentation of multiple glomeruli in renal biopsy images based on deep learning, comprising:

[0009] Acquire a renal puncture image to be segmented;

[0010] Inputting the renal puncture image to be segmented into a trained multi-class glomerulus automatic segmentation model;

[0011] Obtaining a multi-class semantic segmentation result of the glomerulus according to the output of the multi-class glomerulus automatic segmentation model;

[0012] The multi-class glomerulus automatic segmentation model includes an encoder, a classification branch, a decoder and a segmentation branch;

[0013] The encoder includes a first Transform module, a second Transform module, a third Transform module, and a fourth Transform module connected in sequence, and a classification branch is added after the fourth Transform module to obtain a classification result for each glomerulus image;

[0014] The decoder includes a fourth VGG module, a third VGG module, a second VGG module, and a first VGG module connected in sequence, and a segmentation branch is added after the fourth VGG module to obtain a contour segmentation result of each glomerulus image;

[0015] The classification results and contour segmentation results of each glomerulus image are fused to obtain a complete glomerulus multi-class semantic segmentation result.

[0016] In some embodiments, the encoder processing includes:

[0017] Taking the renal puncture image as input, the renal puncture image is processed by Patch Partition to obtain the first token, and the first token is converted into the first feature map by the Token to Feature Map module;

[0018] The first token is processed by the first Transform module to obtain the second token, and the second token is converted into the second feature map by the Token to Feature Map module;

[0019] The second token is processed by the second Transform module to obtain the third token, and the third token is converted into the third feature map by the Token to Feature Map module;

[0020] The third token is processed by the third Transform module to obtain the fourth token, and the fourth token is converted into the fourth feature map by the Token to Feature Map module;

[0021] The fourth token is processed by the fourth Transform module to obtain the fifth token;

[0022] The processing of the decoder includes:

[0023] The fifth token and the fourth feature map are upsampled by the fourth VGG module and then concatenated to obtain the fourth upsampled feature map;

[0024] The fourth upsampled feature map and the third feature map are upsampled by the third VGG module and then concatenated to obtain a third upsampled feature map;

[0025] The third upsampled feature map and the second feature map are upsampled by the second VGG module and then concatenated to obtain a second upsampled feature map;

[0026] The second upsampled feature map and the first feature map are upsampled by the first VGG module and then spliced ​​to obtain the first upsampled feature map.

[0027] In some embodiments, the classification branch processing process includes:

[0028] Take the fifth token output by the fourth Transform module of the encoder as input and perform the normalization LayerNorm operation;

[0029] Apply average pooling to the standardized fifth token to make the token dimension 1, retaining the batch dimension and feature dimension;

[0030] Use a fully connected layer to map the feature dimension obtained by the average pooling operation to an n-dimensional vector, where n represents the number of glomerular categories;

[0031] Perform SoftMax operation on the n-dimensional vector to obtain the probability value of each class, that is, the classification result.

[0032] In some embodiments, the process of splitting the branch includes:

[0033] Taking the first upsampled feature map obtained by the first VGG module of the decoder as input, we first use a convolution operation with a convolution kernel size of 2x2 to map the number of channels of the first upsampled feature map to 2D, i.e., the background and glomerulus regions. Then, we perform a SoftMax operation on the mapped first upsampled feature map to obtain the probability values ​​of the background and glomerulus regions at each pixel, i.e., the contour segmentation result.

[0034] In some embodiments, acquiring a renal puncture image to be segmented includes:

[0035] The original renal puncture image was magnified 20 times, and each glomerulus was placed at the center and a 384*384 image block was cropped. The cropped 384*384 image block was normalized to obtain the renal puncture image to be segmented.

[0036] In some embodiments, the training method of the multi-class glomerulus automatic segmentation model includes:

[0037] Get the training dataset;

[0038] The training data set is input into a multi-class glomerulus automatic segmentation model, and training is performed in combination with a classification task loss function until a preset condition is met, thereby obtaining a trained multi-class glomerulus automatic segmentation model.

[0039] Furthermore, obtaining a training data set includes:

[0040] A1. Take the labeled original renal biopsy image at 20x magnification, center each glomerulus, and crop a 384*384 image patch.

[0041] A2. Normalize the data;

[0042] A3. Data augmentation methods include: one of horizontal flip, vertical flip, and transposition, random 90° rotation, elastic deformation, and grid distortion.

[0043] In some embodiments, the classification task loss function Loss includes:

[0044] Loss = α t (1-p) γ ·CE loss Among them, α t Take 0.5, γ take 2, CE loss represents the cross entropy loss of the correct category, and p represents the relative confidence of the correct category;

[0045] CE loss =-log(p t );

[0046]

[0047] where p t represents the predicted probability of the correct analogy, p t_max Indicates the maximum predicted probability value other than the correct category.

[0048] In a second aspect, the present invention provides a device for automatic segmentation of multiple glomeruli in renal puncture images based on deep learning, comprising a processor and a storage medium;

[0049] The storage medium is used to store instructions;

[0050] The processor is configured to operate according to the instructions to execute the steps of the method according to the first aspect.

[0051] In a third aspect, the present invention provides a storage medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when executed by a processor.

[0052] Compared with the prior art, the present invention has the following beneficial effects:

[0053] 1: The present invention uses the Transform module and the VGG module in the encoder and decoder respectively, which can process the complete context information of the glomerulus on a larger scale while ensuring the accuracy of glomerular contour segmentation. It divides the semantic segmentation process in traditional technology into segmentation branch and classification branch, ensuring the uniformity and integrity of the segmentation results of the same glomerulus.

[0054] 2: This paper proposes a new classification loss function scheme, which can improve the classification accuracy of the model under the condition of sample imbalance, improve the robustness and generalization ability of the model, improve the diagnostic efficiency of pathologists, and reduce the workload of doctors.

[0055] 3: The present invention is based on the proposed overall medical image processing solution, and combines it with the classification loss function for image samples to ultimately form an intelligent system that reduces the timing difference between process nodes. The output results can draw more accurate conclusions based on real-time images, rather than the feedback through complex databases or convolution optimization modules under the existing technology. The actual effect of being able to directly output conclusions based only on the actual image segmentation by medical personnel on site is also the current technical trend of medical image analysis. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0057] Figure 1 It is a schematic diagram of the overall process of the present invention;

[0058] Figure 2 It is a schematic diagram of the sample function processing flow of the present invention. DETAILED DESCRIPTION

[0059] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0060] Example 1

[0061] A deep learning-based method for automatic multi-class glomerulus segmentation in renal biopsy images, including:

[0062] Acquire a renal puncture image to be segmented;

[0063] Inputting the renal puncture image to be segmented into a trained multi-class glomerulus automatic segmentation model;

[0064] Obtaining a multi-class semantic segmentation result of the glomerulus according to the output of the multi-class glomerulus automatic segmentation model;

[0065] The multi-class glomerulus automatic segmentation model includes an encoder, a classification branch, a decoder and a segmentation branch;

[0066] The encoder includes a first Transform module, a second Transform module, a third Transform module, and a fourth Transform module connected in sequence, and a classification branch is added after the fourth Transform module to obtain a classification result for each glomerulus image;

[0067] The decoder includes a fourth VGG module, a third VGG module, a second VGG module, and a first VGG module connected in sequence, and a segmentation branch is added after the fourth VGG module to obtain a contour segmentation result of each glomerulus image;

[0068] The classification results and contour segmentation results of each glomerulus image are fused to obtain a complete glomerulus multi-class semantic segmentation result.

[0069] In some embodiments, the encoder processing includes:

[0070] Taking the renal puncture image as input, the renal puncture image is processed by Patch Partition to obtain the first token, and the first token is converted into the first feature map by the Token to Feature Map module;

[0071] The first token is processed by the first Transform module to obtain the second token, and the second token is converted into the second feature map by the Token to Feature Map module;

[0072] The second token is processed by the second Transform module to obtain the third token, and the third token is converted into the third feature map by the Token to Feature Map module;

[0073] The third token is processed by the third Transform module to obtain the fourth token, and the fourth token is converted into the fourth feature map by the Token to Feature Map module;

[0074] The fourth token is processed by the fourth Transform module to obtain the fifth token;

[0075] The processing of the decoder includes:

[0076] The fifth token and the fourth feature map are upsampled by the fourth VGG module and then concatenated to obtain the fourth upsampled feature map;

[0077] The fourth upsampled feature map and the third feature map are upsampled by the third VGG module and then concatenated to obtain a third upsampled feature map;

[0078] The third upsampled feature map and the second feature map are upsampled by the second VGG module and then concatenated to obtain a second upsampled feature map;

[0079] The second upsampled feature map and the first feature map are upsampled by the first VGG module and then spliced ​​to obtain the first upsampled feature map.

[0080] In some embodiments, the classification branch processing process includes:

[0081] Take the fifth token output by the fourth Transform module of the encoder as input and perform the normalization LayerNorm operation;

[0082] Apply average pooling to the standardized fifth token to make the token dimension 1, retaining the batch dimension and feature dimension;

[0083] Use a fully connected layer to map the feature dimension obtained by the average pooling operation to an n-dimensional vector, where n represents the number of glomerular categories;

[0084] Perform SoftMax operation on the n-dimensional vector to obtain the probability value of each class, that is, the classification result.

[0085] In some embodiments, the process of splitting the branch includes:

[0086] Taking the first upsampled feature map obtained by the first VGG module of the decoder as input, we first use a convolution operation with a convolution kernel size of 2x2 to map the number of channels of the first upsampled feature map to 2D, i.e., the background and glomerulus regions. Then, we perform a SoftMax operation on the mapped first upsampled feature map to obtain the probability values ​​of the background and glomerulus regions at each pixel, i.e., the contour segmentation result.

[0087] In some embodiments, acquiring a renal puncture image to be segmented includes:

[0088] The original renal puncture image was magnified 20 times, and each glomerulus was placed at the center and a 384*384 image block was cropped. The cropped 384*384 image block was normalized to obtain the renal puncture image to be segmented.

[0089] In some embodiments, the training method of the multi-class glomerulus automatic segmentation model includes:

[0090] Get the training dataset;

[0091] The training data set is input into a multi-class glomerulus automatic segmentation model, and training is performed in combination with a classification task loss function until a preset condition is met, thereby obtaining a trained multi-class glomerulus automatic segmentation model.

[0092] In some specific embodiments, Figure 1 and Figure 2 As shown in FIG, a multi-class glomerulus automatic segmentation method for renal puncture images based on deep learning includes the following steps:

[0093] Step 1. Preprocessing of renal biopsy images: The original images and pathologist-labeled data are used to divide the model into training and test sets. The training set is normalized and subjected to flipping and elastic deformation for data augmentation.

[0094] Step 2. Model training, encoder part: A four-layer Transform module is used. The tokens obtained in each layer are converted into feature maps using the Token-to-Feature Map module, which is used as the feature map for subsequent decoder operations. An additional classification branch is added to the last layer of the encoder to obtain the classification results for each glomerulus image.

[0095] A four-layer VGG module is used to fuse the sampling feature map with the feature map of the corresponding layer of the encoder. The decoder finally adds a segmentation branch to obtain the contour segmentation result of each glomerulus image. Finally, the classification result and the segmentation result are fused to obtain the complete semantic segmentation result of different types of glomeruli. The parameters of each part of the network are gradually obtained through training.

[0096] Step 3. Fuse the glomerular category information obtained by the classification branch in step 2 with the glomerular contour segmentation result obtained by the segmentation branch to obtain a complete glomerular multi-class semantic segmentation result;

[0097] Step 4. Model testing: Based on the model obtained in step 2, perform the same normalization operation on the test image and send it to the model test to obtain the corresponding results.

[0098] Furthermore, the data preprocessing in step 1 includes:

[0099] A1. At 20x magnification, center each glomerulus and crop a 384x384 image patch.

[0100] A2. Normalize the data;

[0101] A3. Data augmentation methods include: one of horizontal flip, vertical flip, and transposition, random 90° rotation, elastic deformation, and grid distortion.

[0102] Furthermore, the specific process of the classification branch and the segmentation branch in step 2 is as follows:

[0103] B1. Classification branch: The classification task uses the feature map obtained from the last encoder layer in step 2. First, perform a LayerNorm operation. Then, use an average pooling operation to compress the token dimension in the feature map to 1, retaining the batch dimension and feature dimension. Then, use a fully connected layer to map the feature dimension obtained in the previous step to an n-dimensional vector, where n represents the number of glomerulus categories. Finally, perform a SoftMax operation to obtain the probability value of each category.

[0104] B2. Segmentation branch: The segmentation task uses the feature map obtained by the last layer of the decoder in step 2. First, a convolution operation with a kernel size of 2x2 is used to map the number of channels of the feature map to n+1 dimensions, where n represents the number of glomerulus categories. Then, a SoftMax operation is performed to obtain the probability value of each category including the background at each pixel.

[0105] Furthermore, the classification task loss function that effectively handles the sample imbalance problem in step 2 includes the following steps:

[0106] C1. Based on the final classification task of the encoder in step 2, a SoftMax operation is used to obtain the predicted probability value of each class;

[0107] C2. Calculate the cross entropy loss of the correct category:

[0108] CE loss =-log(p t );

[0109] where p t represents the predicted probability of the correct analogy;

[0110] C3. Calculate the relative confidence of the correct category:

[0111]

[0112] where p t represents the predicted probability of the correct analogy, p t_max Indicates the maximum predicted probability value except for the correct category;

[0113] C4. Calculate the final classification task loss:

[0114] Loss = α t (1-p) γ ·CE loss ;

[0115] where α t In this method, γ is set to 0.5 and γ is set to 2.

[0116] Specifically, such as Figure 1 、 Figure 2 As shown, according to the above method flow, medical personnel only mark the required image range and combine it with the original image to form a model training set. They process the data and generate image construction under real-time conditions, and output the final result based on the combination of segmentation structure and classification results.

[0117] At Figure 2 In the image construction process, the corresponding weight values ​​are further set according to the particularity of medical images, such as α t As well as the value of γ, combined with the calculated function, the final output result does not need to be repeatedly verified and optimized, there is no need to build complex timing nodes, and the output conclusion has high accuracy.

[0118] Comparing the conclusions of the present invention with those of the prior art, such as the image acquisition and learning system formed based on the PACS system, a control group is formed for the conclusions it generates. By comparing the conclusions judged by the actual medical experience of medical personnel related to renal puncture imaging, the accuracy of the method of the present invention can be increased to 97% compared with the prior art method, which is basically the same as the conclusions of medical personnel related to renal puncture imaging, and the output efficiency is higher.

[0119] Example 2

[0120] The difference from Example 1 is that the encoder and decoder in step 2 can all use VGG modules, Transform modules or other "U"-shaped structures.

[0121] Example 3

[0122] The difference from Example 1 is that the classification task and the segmentation task are not used at the same time, and the semantic segmentation task is used directly.

[0123] Example 4

[0124] The difference from Example 1 is that the classification task loss function uses cross entropy loss and focal loss function.

[0125] Example 5

[0126] This embodiment provides a deep learning-based automatic segmentation device for multiple glomeruli in renal puncture images, including a processor and a storage medium;

[0127] The storage medium is used to store instructions;

[0128] The processor is configured to operate according to the instructions to execute the steps of the method according to any one of Embodiments 1 to 4.

[0129] Example 6

[0130] This embodiment provides a storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the methods described in Embodiments 1 to 4 are implemented.

[0131] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0132] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0133] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0134] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0135] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. A deep learning-based automatic segmentation method for multi-class glomeruli in renal biopsy images, characterized by: include: Acquiring a renal puncture image to be segmented; inputting the renal puncture image to be segmented into a trained multi-class glomerulus automatic segmentation model; Obtaining a multi-class semantic segmentation result of the glomerulus according to the output of the multi-class glomerulus automatic segmentation model; The multi-class glomerulus automatic segmentation model includes an encoder, a classification branch, a decoder and a segmentation branch; The processing process of the encoder includes: taking a renal puncture image as input, performing Patch Partition processing on the renal puncture image to obtain a first token, converting the first token into a first feature map by a Token to Feature Map module; processing the first token into a second token by a first Transform module, converting the second token into a second feature map by a Token to Feature Map module; processing the second token into a third token by a second Transform module, converting the third token into a third feature map by a Token to Feature Map module; processing the third token into a fourth token by a third Transform module, converting the fourth token into a fourth feature map by a Token to Feature Map module; processing the fourth token into a fifth token by a fourth Transform module; The processing process of the decoder includes: the fifth Token and the fourth feature map are upsampled by the fourth VGG module and then spliced ​​to obtain a fourth upsampled feature map; the fourth upsampled feature map and the third feature map are upsampled by the third VGG module and then spliced ​​to obtain a third upsampled feature map; the third upsampled feature map and the second feature map are upsampled by the second VGG module and then spliced ​​to obtain a second upsampled feature map; the second upsampled feature map and the first feature map are upsampled by the first VGG module and then spliced ​​to obtain a first upsampled feature map; The processing process of the classification branch includes: taking the fifth token output by the fourth Transform module of the encoder as input, performing a normalized LayerNorm operation; performing an average pooling operation on the normalized fifth token to make the token dimension 1, retaining the batch dimension and the feature dimension; using a fully connected layer to map the feature dimension obtained by the average pooling operation to an n-dimensional vector, where n represents the number of glomerulus categories; performing a SoftMax operation on the n-dimensional vector to obtain a probability value for each category, i.e., a classification result; The processing process of the segmentation branch includes: taking the first up-sampled feature map obtained by the first VGG module of the decoder as input, first using a convolution operation with a convolution kernel size of 2x2 to map the number of channels of the first up-sampled feature map to 2 dimensions, i.e., the background and glomerular regions, and then performing a SoftMax operation on the mapped first up-sampled feature map to obtain the probability value of the background and glomerular regions at each pixel, i.e., the contour segmentation result; The classification results and contour segmentation results of each glomerulus image are fused to obtain the complete glomerulus multi-class semantic segmentation results.

2. The method for automatic segmentation of multiple glomeruli in renal biopsy images based on deep learning according to claim 1, characterized in that: Acquiring the renal puncture image to be segmented includes: The original renal puncture image was magnified 20 times, and each glomerulus was placed at the center and a 384*384 image block was cropped. The cropped 384*384 image block was normalized to obtain the renal puncture image to be segmented.

3. The method for automatic segmentation of multiple glomeruli in renal biopsy images based on deep learning according to claim 1, characterized in that: The training method of the multi-class glomerulus automatic segmentation model includes: Get the training dataset; The training data set is input into a multi-class glomerulus automatic segmentation model, and training is performed in combination with a classification task loss function until a preset condition is met, thereby obtaining a trained multi-class glomerulus automatic segmentation model.

4. The method for automatic segmentation of multiple glomeruli in renal biopsy images based on deep learning according to claim 3, characterized in that: Obtaining a training dataset includes: A1. Take the labeled original renal biopsy image at 20x magnification, center each glomerulus, and crop a 384*384 image patch. A2. Normalize the data; A3. Data augmentation methods include: horizontal flip, vertical flip, transposition, random 90° rotation, elastic deformation, and grid distortion.

5. The method for automatic segmentation of multiple glomeruli in renal biopsy images based on deep learning according to claim 3, characterized in that: The classification task loss function ,include: ； in, Take 0.5, Take 2, represents the cross entropy loss of the correct category, Indicates the relative confidence of the correct category; ； ； in represents the predicted probability of the correct analogy, Indicates the maximum predicted probability value other than the correct category.

6. A deep learning-based automatic segmentation device for multi-class glomeruli in renal biopsy images, characterized by: including processors and storage media; The storage medium is used to store instructions; The processor is configured to operate according to the instructions to execute the steps of the method according to any one of claims 1 to 5.

7. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images