Training method of decomposition convolution model, lesion area prediction method and related equipment

Abstract

The present disclosure provides a training method of a decomposition convolution model, a lesion area prediction method and related equipment. In the model training process, the spectral standardization decomposition convolution layer of the convolutional neural network performs two convolution operations on the training data according to the two low-rank matrices of the spectral standardization decomposition convolution layer, the calculation complexity of single convolution is reduced, and the model parameters are reduced. Moreover, the values of the two low-rank matrices are subjected to spectral standardization, so as to ensure that the two low-rank matrices do not have excessively large singular values, effectively improve the generalization ability, and ensure the ideal performance of the trained decomposition convolution model. In application, the decomposition convolution model is deployed on an ontology hardware device with limited computing resources, the decomposition convolution model extracts a feature map of a case image, and the feature map is processed by a target detection model, so as to identify a lesion area and lesion information on the case image, the case data has high confidentiality, and the identification is accurate.

Description

Technical Field

[0001] This disclosure relates to the field of model training technology, and in particular to a training method for a decomposed convolutional model, a method for predicting lesion areas, and related equipment. Background Technology

[0002] AI-assisted medical image diagnosis utilizes artificial intelligence's image processing capabilities to achieve functions such as lesion detection, target organ segmentation, risk assessment, lesion type classification, and organ and tissue labeling and segmentation. This aims to meet clinical assistance needs such as lesion identification and labeling, disease type classification, 3D image reconstruction, and even automatic delineation of radiotherapy target areas, providing aids and references for doctors in image interpretation and delineation. Deploying AI-assisted medical image diagnosis systems on medical display devices will become a new trend in future products. However, due to the limited computing resources of medical display devices and the high computational cost of the convolutional network models upon which AI-assisted medical image diagnosis relies, combining the two presents a dilemma. Summary of the Invention

[0003] The purpose of this disclosure is to provide a training method for a decomposed convolutional model, a method for predicting lesion regions, and related equipment. The decomposed convolutional model trained based on spectral normalization decomposed convolutional layers has low computational overhead, can be deployed on hardware devices with limited computing resources, and has ideal performance.

[0004] To achieve the above objectives, this disclosure adopts the following technical solution: a training method for decomposing a convolutional model, comprising:

[0005] Obtain training data;

[0006] The training data is input into a convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices.

[0007] The model training process is repeated iteratively until the model converges, resulting in a decomposed convolutional model.

[0008] This disclosure also provides a method for predicting lesion areas, including:

[0009] Acquire case images;

[0010] The case image is input into a decomposition convolution model for feature extraction to obtain a feature map corresponding to the case image. The decomposition convolution model is trained by any of the decomposition convolution model training methods described above.

[0011] The feature map is input into the target detection model to identify and label the lesion areas and lesion information on the case image.

[0012] This disclosure also provides a training apparatus for decomposing a convolutional model, comprising:

[0013] The first acquisition module is used to acquire training data;

[0014] The training module is used to input the training data into the convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices.

[0015] The loop module is used to iterate through the model training steps until the model converges, resulting in a decomposed convolutional model.

[0016] This disclosure also provides a lesion area prediction device, comprising:

[0017] The second acquisition module is used to acquire case images;

[0018] The extraction module is used to input the case image into the decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image, wherein the decomposition convolution model is trained by any of the decomposition convolution model training methods described above;

[0019] The recognition module is used to input the feature map into the target detection model to identify and label the lesion areas and lesion information on the case image.

[0020] This disclosure also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the methods described above.

[0021] This disclosure also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described above.

[0022] This disclosure provides a training method for a decomposed convolutional model, a lesion region prediction method, and related equipment. During model training, the spectrally normalized decomposed convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices. This reduces the computational complexity of a single convolution, thereby reducing the number of model parameters. Furthermore, spectral normalization of the values ​​of the two low-rank matrices ensures that they do not contain excessively large singular values, effectively improving generalization ability and guaranteeing the ideal performance of the trained decomposed convolutional model. In application, the decomposed convolutional model is deployed on computationally limited hardware devices. The model extracts feature maps from case images, which are then processed by a target detection model to identify lesion regions and lesion information in the case images. This approach ensures high confidentiality of case data and accurate identification. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the steps of a training method for decomposing a convolutional model in one embodiment of this disclosure;

[0024] Figure 2 This is a schematic diagram of the steps of a lesion area prediction method in one embodiment of the present disclosure;

[0025] Figure 3 This is an overall structural block diagram of a training device for decomposing a convolutional model according to an embodiment of the present disclosure;

[0026] Figure 4 This is an overall structural block diagram of the lesion area prediction device in one embodiment of the present disclosure;

[0027] Figure 5 This is a schematic block diagram of the structure of a computer device according to an embodiment of the present disclosure.

[0028] The realization of the purpose, functional features and advantages of this disclosure will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this disclosure clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this disclosure.

[0030] Reference Figure 1 One embodiment of this disclosure provides a training method for decomposing a convolutional model, comprising:

[0031] S1: Obtain training data;

[0032] S2: Input the training data into the convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices.

[0033] S3: Iterate through the model training steps until the model converges, resulting in the decomposed convolution model.

[0034] Preferably, the two low-rank matrices are obtained by decomposing the weight matrix of the standard convolutional layer corresponding to the spectral normalization convolutional layer according to the convolution kernel parameters of the standard convolutional layer.

[0035] In this embodiment, the training system acquires user input or pre-stored training data, and then inputs the training data into a pre-constructed convolutional neural network for model training. Compared with conventional convolutional neural networks in the prior art, the convolutional neural network in this embodiment uses spectral normalization decomposition convolutional layers to replace the existing standard convolutional layers (i.e., all standard convolutional layers in the existing conventional convolutional neural network are replaced with spectral normalization decomposition convolutional layers to obtain the convolutional neural network of this embodiment). Specifically, the spectral normalization decomposition convolutional layer contains two low-rank matrices, which are obtained by decomposing the weight matrix of the corresponding standard convolutional layer according to the convolution kernel parameters of the standard convolutional layer. During model training, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices (the logic of the convolution operation is the same as that of the conventional convolution operation), thereby reducing the computational overhead (the computational overhead includes the memory and CPU / GPU of the hardware device corresponding to the training system). After introducing decomposition convolution into the network structure of the convolutional neural network, both the forward inference and backward propagation gradient updates of the convolution are based on the two low-rank matrices. For the two low-rank matrices in the decomposed convolution, excessively high singular values ​​can easily occur during model training, causing the convolution weights to be controlled by the direction of these singular values, which is detrimental to generalization performance. Good generalization performance means that the weights tend towards zero, but are not equal to zero; excessively high singular values ​​result in overly large weights, affecting generalization performance. Therefore, the spectral normalization decomposed convolution layer also performs spectral normalization on the values ​​of the two low-rank matrices during training to ensure that these two low-rank matrices do not have excessively large singular values, thereby helping to improve generalization performance and ensuring the ideal performance of the decomposed convolution model obtained after training. The training system iterates through the model training steps until the model converges and then stops training, obtaining the desired decomposed convolution model (in this embodiment, the training logic of the decomposed convolution model is the same as that of existing model training; the only difference lies in the processing logic of the spectral normalization decomposed convolution layer on the training data, therefore, the training actions other than the processing logic of the spectral normalization decomposed convolution layer are not described in detail).

[0036] In this embodiment, during model training, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices. This reduces the computational complexity of a single convolution, thereby reducing the number of model parameters. Furthermore, spectral normalization of the values ​​of the two low-rank matrices ensures that they do not contain excessively large singular values, effectively improving generalization ability and guaranteeing the ideal performance of the trained decomposition convolutional model.

[0037] Furthermore, the convolution kernel parameters include the number of input channels c of the convolution kernel. i Number of output channels c o And the kernel size k×k;

[0038] The weight matrix has a size of m×n, where m=c i ×k, n=c o ×k;

[0039] The two low-rank matrices are low-rank matrix P and low-rank matrix Q, where the size of low-rank matrix P is m×r and the size of low-rank matrix Q is r×n, where r represents the rank of low-rank matrix P and low-rank matrix Q, and r < 0. <min{m,n}。

[0040] In this embodiment, it is assumed that the weight matrix W of the 2D standard convolutional layer is of size m×n, where m=c i ×k, n=c o ×k, c i c represents the number of input channels of the convolution kernel. o The number of output channels of the convolutional layer is represented by k×k, and the kernel size is represented by k×k. Assume the input feature map F... i Size c i ×h i ×w i ; and its output feature map F o Size c o ×h o ×w o The forward computation process of standard convolution can be represented as: in This represents the circular convolution process. Considering a single convolution in the circular convolution process, its computational complexity is O(c). i c o k 2). The weight matrix W is decomposed into two low-rank matrices P and Q, that is, W≈P×Q, where the size of the low-rank matrix P is m×r, the size of the low-rank matrix Q is r×n, r represents the rank of the low-rank matrix P and the low-rank matrix Q, r<<min{m,n}, that is, the rank r of the low-rank matrix P and the low-rank matrix Q is much smaller than the minimum value of m and n, which plays a role in greatly reducing model parameters. After decomposing the weight matrix of the 2D standard convolution layer into two low-rank matrices, the spectral normalized decomposed convolution layer of this embodiment is obtained; in the forward inference process, the two low-rank matrices of the spectral normalized decomposed convolution layer are used to complete the convolution operation in two steps, thereby achieving the effect of reducing computational overhead. Specifically, the spectral normalized decomposed convolution layer decomposes the sequential operation of the k×k convolution kernel of the standard convolution layer into two operations of two 1×k convolution kernels, wherein the low-rank matrix Q is restructured into a convolution kernel r×c i ×1×k, and the matrix P is restructured into c o ×1×k×r, and then a convolution operation is performed respectively. The forward calculation process of decomposed convolution is expressed as: The computational complexity of a single convolution of the spectral normalized decomposed convolution layer is: o(c i kr+c o kr). It can be seen from this that the computational complexity of the spectral normalized decomposed convolution layer is greatly reduced compared with that of the standard convolution layer.

[0041] Further, the step of performing spectral normalization on the values of the two low-rank matrices comprises:

[0042] S201: Obtain a product matrix of the two low-rank matrices, and calculate the spectral norm of the product matrix;

[0043] S202: Perform spectral normalization on the values in the two low-rank matrices respectively based on the spectral norm, wherein the calculation formula for spectral normalization is: P represents the value of the low-rank matrix, σ represents the spectral norm, represents the value of the low-rank matrix after spectral normalization.

[0044] In this embodiment, the singular values of a matrix correspond to important implicit information of the matrix, and the importance is positively correlated with the magnitude of the singular values. Singular value decomposition decomposes matrix A into the sum of several rank-one matrices, as shown in the following formula: where the coefficient σ in front of each term on the right side of the equation is the singular value, and u and v represent the left and right column vectors corresponding to the singular value respectively. Any matrix can be expressed as the sum of a series of small matrices with rank 1, and the singular value measures the contribution of these rank 1 matrices to the original matrix, with larger singular values corresponding to greater contributions. The spectral norm is the maximum singular value of a matrix, that is σ max , the rank 1 matrix corresponding to the spectral norm contains the most important information in matrix A.

[0045] During model training, the training system first calculates the product matrix W of low-rank matrices P and Q. Then, by performing singular value decomposition on the product matrix W, it obtains the spectral norm (i.e., the maximum singular value) of W. With the spectral norm, the values ​​in the low-rank matrices P and Q can be spectrally normalized. Specifically, the formula for spectral normalization is: σ represents the spectral norm, and P represents the value of the low-rank matrix. The values ​​of the low-rank matrix P and Q represent the values ​​of the low-rank matrix after spectral normalization. The value of the low-rank matrix Q is represented by the spectral normalization. Spectral normalization of the values ​​of the two low-rank matrices ensures that they do not exhibit excessively large singular values, thereby improving generalization performance and effectively enhancing the performance of the decomposed convolutional model obtained after training.

[0046] Furthermore, the step of calculating the spectral norm of the product matrix includes:

[0047] S2011: Calculate the left and right column vectors corresponding to the singular values ​​of the product matrix, respectively, wherein the first formula for calculating the left column vector corresponding to the singular values ​​of the product matrix is: u n ←W·v n-1 The second formula for calculating the right column vector corresponding to the singular values ​​of the product matrix is: v n ←W·u n , where n is the power iteration number, W represents the product matrix, u represents the left column vector, and v represents the right column vector;

[0048] S2012: Call the third calculation formula, and substitute the left column vector and the right column vector into the third calculation formula to calculate the spectral norm, wherein the third calculation formula is:

[0049] In this embodiment, the training system uses the power iteration method to approximate the spectral norm of the product matrix W. Compared to obtaining the spectral norm by performing singular value decomposition on the product matrix W, this reduces computational overhead and the resource pressure on the hardware devices used for model training. Specifically, the training system calls the first and second calculation formulas to calculate the left and right column vectors corresponding to the singular values ​​of the product matrix, respectively. The first calculation formula is: u n ←W·v n-1 The second calculation formula is v n ←W·u n n is the power iteration number, W represents the product matrix, u represents the left column vector, and v represents the right column vector; when n equals 1, v n-1 can be a random vector (e.g., sampled from a Gaussian distribution). Then, the third formula is called, and the aforementioned left and right column vectors are substituted into the third formula. In the process, through iteration, an approximate value σ of the spectral norm is obtained, which is the maximum singular value.

[0050] Reference Figure 2 This disclosure also provides a method for predicting lesion areas, comprising:

[0051] A1: Obtain case images;

[0052] A2: Input the case image into the decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image, wherein the decomposition convolution model is trained by any of the decomposition convolution model training methods described above;

[0053] A3: Input the feature map into the target detection model to identify and label the lesion areas and lesion information on the case image.

[0054] In this embodiment, the prediction system acquires the case image (e.g., a chest X-ray) input by the user, and then inputs the case image into a decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image. The decomposition convolution model is trained using the aforementioned training method, resulting in fewer model parameters, lower computational complexity, while still maintaining ideal model performance. The prediction system inputs the feature map into a target detection model for processing, thereby identifying lesion regions on the case image and marking them on the image. The target detection model uses the case image as training data (the case image used as training data is labeled with case information) and is trained through deep learning. The trained target detection model can identify lesion regions and lesion information (e.g., disease type and corresponding probability) in the input image. Preferably, the decomposition convolution model and the target detection model are deployed on local hardware devices, such as medical display devices; because the decomposition convolution model has fewer model parameters, it can significantly reduce computational complexity, making it suitable for local hardware devices with limited computing resources. Furthermore, deploying the model on local hardware ensures the security of case data, avoids network transmission instability, and improves the speed of identifying lesion areas and lesion information in case images.

[0055] Furthermore, after the step of inputting the feature map into the target detection model to identify and label the lesion region and lesion information on the case image, the method includes:

[0056] A4: Search for corresponding treatment information based on the lesion information, and associate the treatment information with the lesion area;

[0057] A5: Output the case image, which is labeled with the treatment information, the lesion information, and the lesion area, to the display interface of the medical display device for display.

[0058] In this embodiment, the prediction system obtains treatment information (such as treatment methods, medications, and cure probabilities) corresponding to the predicted lesion information through online or local database searches, and then associates this treatment information with the lesion area. The prediction system then displays case images labeled with treatment information, lesion information, and the lesion area on the medical display device's interface, providing assistance and reference for doctors interpreting the images.

[0059] Reference Figure 3 An embodiment of this disclosure also provides a training apparatus for decomposing a convolutional model, comprising:

[0060] The first acquisition module 1 is used to acquire training data;

[0061] Training module 2 is used to input the training data into the convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices.

[0062] Loop module 3 is used to iterate through the model training steps until the model converges, resulting in a decomposed convolutional model.

[0063] Preferably, the two low-rank matrices are obtained by decomposing the weight matrix of the standard convolutional layer corresponding to the spectral normalization convolutional layer according to the convolution kernel parameters of the standard convolutional layer.

[0064] Preferably, the convolution kernel parameters include the number of input channels c of the convolution kernel. i Number of output channels c o And the kernel size k×k;

[0065] The weight matrix has a size of m×n, where m=c i ×k, n=c o ×k;

[0066] The two low-rank matrices are low-rank matrix P and low-rank matrix Q, where the size of low-rank matrix P is m×r and the size of low-rank matrix Q is r×n, where r represents the rank of low-rank matrix P and low-rank matrix Q, and r < 0. <min{m,n}。

[0067] Furthermore, the training module 2 includes:

[0068] A calculation unit is used to obtain the product matrix of the two low-rank matrices and calculate the spectral norm of the product matrix;

[0069] A spectral normalization unit is used to perform spectral normalization on the values ​​in the two low-rank matrices based on the spectral norm, wherein the formula for calculating spectral normalization is: P represents the value of the low-rank matrix, and σ represents the spectral norm. The value of the low-rank matrix after the spectrum is normalized.

[0070] Furthermore, the computing unit includes:

[0071] The first calculation subunit is used to calculate the left and right column vectors corresponding to the singular values ​​of the product matrix, respectively, wherein the first calculation formula for the left column vector corresponding to the singular values ​​of the product matrix is: u n ←W·v n-1 The second formula for calculating the right column vector corresponding to the singular values ​​of the product matrix is: v n ←W·u n , where n is the power iteration number, W represents the product matrix, u represents the left column vector, and v represents the right column vector;

[0072] The second calculation subunit is used to call the third calculation formula and substitute the left column vector and the right column vector into the third calculation formula to calculate the spectral norm, wherein the third calculation formula is:

[0073] In this embodiment, each module, unit, and sub-unit in the training device for decomposing the convolution model is used to perform each step in the training method for decomposing the convolution model described above. The specific implementation process is not described in detail here.

[0074] Reference Figure 4 An embodiment of this disclosure also provides a lesion area prediction device, comprising:

[0075] The second acquisition module 4 is used to acquire case images;

[0076] Extraction module 5 is used to input the case image into the decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image, wherein the decomposition convolution model is trained by any of the decomposition convolution model training methods described above;

[0077] The recognition module 6 is used to input the feature map into the target detection model to identify and label the lesion area and lesion information on the case image.

[0078] Furthermore, the lesion area prediction device also includes:

[0079] Search module 7 is used to search for corresponding treatment information based on the lesion information and associate the treatment information with the lesion area;

[0080] The display module 8 is used to output a case image labeled with the treatment information, the lesion information and the lesion area to the display interface of the medical display device for display.

[0081] In this embodiment, each module in the training device of the lesion area prediction device is used to perform the corresponding steps in the above-mentioned lesion area prediction method, and the specific implementation process is not described in detail here.

[0082] This embodiment provides a training device for a decomposed convolutional model and a lesion region prediction device. During model training, the spectrally normalized decomposed convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices. This reduces the computational complexity of a single convolution, thereby reducing the number of model parameters. Furthermore, spectral normalization of the values ​​of the two low-rank matrices ensures that they do not contain excessively large singular values, effectively improving generalization ability and guaranteeing the ideal performance of the trained decomposed convolutional model. In application, the decomposed convolutional model is deployed on computationally limited hardware devices. The model extracts feature maps from case images, which are then processed by a target detection model to identify lesion regions and lesion information in the case images. The case data maintains high confidentiality and provides accurate identification.

[0083] Reference Figure 5 This disclosure also provides a computer device, which may be a server, and its internal structure may be as follows: Figure 5 As shown, the computer device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data such as convolutional neural networks. The network interface allows communication with external terminals via a network connection. When executed by the processor, the computer program implements a training method for decomposing a convolutional model and a lesion region prediction method.

[0084] The processor described above executes the following steps of the training method for decomposing the convolutional model:

[0085] S1: Obtain training data;

[0086] S2: Input the training data into the convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices.

[0087] S3: Iterate through the model training steps until the model converges, resulting in the decomposed convolution model.

[0088] Preferably, the two low-rank matrices are obtained by decomposing the weight matrix of the standard convolutional layer corresponding to the spectral normalization convolutional layer according to the convolution kernel parameters of the standard convolutional layer.

[0089] Furthermore, the convolution kernel parameters include the number of input channels c of the convolution kernel. i Number of output channels c o And the kernel size k×k;

[0090] The weight matrix has a size of m×n, where m=c i ×k, n=c o ×k;

[0091] The two low-rank matrices are low-rank matrix P and low-rank matrix Q, where the size of low-rank matrix P is m×r and the size of low-rank matrix Q is r×n, where r represents the rank of low-rank matrix P and low-rank matrix Q, and r < 0. <min{m,n}。

[0092] Furthermore, the step of spectral normalization of the values ​​of the two low-rank matrices includes:

[0093] S201: Obtain the product matrix of the two low-rank matrices and calculate the spectral norm of the product matrix;

[0094] S202: Perform spectral normalization on the values ​​in the two low-rank matrices based on the spectral norm, wherein the formula for calculating spectral normalization is: P represents the value of the low-rank matrix, and σ represents the spectral norm. The value of the low-rank matrix after the spectrum is normalized.

[0095] Furthermore, the step of calculating the spectral norm of the product matrix includes:

[0096] S2011: Calculate the left and right column vectors corresponding to the singular values ​​of the product matrix, respectively, wherein the first formula for calculating the left column vector corresponding to the singular values ​​of the product matrix is: u n ←W·v n-1 The second formula for calculating the right column vector corresponding to the singular values ​​of the product matrix is: v n ←W·un , where n is the power iteration number, W represents the product matrix, u represents the left column vector, and v represents the right column vector;

[0097] S2012: Call the third calculation formula, and substitute the left column vector and the right column vector into the third calculation formula to calculate the spectral norm, wherein the third calculation formula is:

[0098] The processor described above performs the following steps in the lesion region prediction method:

[0099] A1: Obtain case images;

[0100] A2: Input the case image into the decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image, wherein the decomposition convolution model is trained by any of the decomposition convolution model training methods described above;

[0101] A3: Input the feature map into the target detection model to identify and label the lesion areas and lesion information on the case image.

[0102] Furthermore, after the step of inputting the feature map into the target detection model to identify and label the lesion region and lesion information on the case image, the method includes:

[0103] A4: Search for corresponding treatment information based on the lesion information, and associate the treatment information with the lesion area;

[0104] A5: Output the case image, which is labeled with the treatment information, the lesion information, and the lesion area, to the display interface of the medical display device for display.

[0105] One embodiment of this disclosure also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a training method for a decomposed convolution model and a lesion region prediction method. The training method for the decomposed convolution model specifically includes:

[0106] S1: Obtain training data;

[0107] S2: Input the training data into the convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices.

[0108] S3: Iterate through the model training steps until the model converges, resulting in the decomposed convolution model.

[0109] Preferably, the two low-rank matrices are obtained by decomposing the weight matrix of the standard convolutional layer corresponding to the spectral normalization convolutional layer according to the convolution kernel parameters of the standard convolutional layer.

[0110] Furthermore, the convolution kernel parameters include the number of input channels c of the convolution kernel. i Number of output channels c o And the kernel size k×k;

[0111] The weight matrix has a size of m×n, where m=c i ×k, n=c o ×k;

[0112] The two low-rank matrices are low-rank matrix P and low-rank matrix Q, where the size of low-rank matrix P is m×r and the size of low-rank matrix Q is r×n, where r represents the rank of low-rank matrix P and low-rank matrix Q, and r < 0. <min{m,n}。

[0113] Furthermore, the step of spectral normalization of the values ​​of the two low-rank matrices includes:

[0114] S201: Obtain the product matrix of the two low-rank matrices and calculate the spectral norm of the product matrix;

[0115] S202: Perform spectral normalization on the values ​​in the two low-rank matrices based on the spectral norm, wherein the formula for calculating spectral normalization is: P represents the value of the low-rank matrix, and σ represents the spectral norm. The value of the low-rank matrix after the spectrum is normalized.

[0116] Furthermore, the step of calculating the spectral norm of the product matrix includes:

[0117] S2011: Calculate the left and right column vectors corresponding to the singular values ​​of the product matrix, respectively, wherein the first formula for calculating the left column vector corresponding to the singular values ​​of the product matrix is: u n ←W·v n-1 The second formula for calculating the right column vector corresponding to the singular values ​​of the product matrix is: v n ←W·u n , where n is the power iteration number, W represents the product matrix, u represents the left column vector, and v represents the right column vector;

[0118] S2012: Call the third calculation formula, and substitute the left column vector and the right column vector into the third calculation formula to calculate the spectral norm, wherein the third calculation formula is:

[0119] The specific method for predicting the lesion area is as follows:

[0120] A1: Obtain case images;

[0121] A2: Input the case image into the decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image, wherein the decomposition convolution model is trained by any of the decomposition convolution model training methods described above;

[0122] A3: Input the feature map into the target detection model to identify and label the lesion areas and lesion information on the case image.

[0123] Furthermore, after the step of inputting the feature map into the target detection model to identify and label the lesion region and lesion information on the case image, the method includes:

[0124] A4: Search for corresponding treatment information based on the lesion information, and associate the treatment information with the lesion area;

[0125] A5: Output the case image, which is labeled with the treatment information, the lesion information, and the lesion area, to the display interface of the medical display device for display.

[0126] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the methods described above. Any references to memory, storage, databases, or other media provided in this disclosure and in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual-speed SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0127] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, first object, or method that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, apparatus, first object, or method. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, apparatus, first object, or method that includes that element.

[0128] The above description is only a preferred embodiment of this disclosure and does not limit the patent scope of this disclosure. Any equivalent structural or procedural changes made based on the content of this disclosure and its drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this disclosure.

Claims

1. A training method for decomposing a convolutional model, wherein, include: Acquire training data, wherein the training data is case images; The training data is input into a convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices. The training process is repeated iteratively until the model converges, resulting in a decomposed convolutional model. The singular values ​​of the matrix correspond to important information implicit in the matrix, and their importance is positively correlated with their magnitude. The step of spectral normalization of the values ​​of the two low-rank matrices includes: Obtain the product matrix of the two low-rank matrices, and calculate the spectral norm of the product matrix; Based on the spectral norm, the values ​​in the two low-rank matrices are spectrally normalized, and the formula for calculating spectral normalization is as follows: P represents the value of the low-rank matrix. Characterizing the spectral norm, The value of the low-rank matrix after the spectrum is normalized; The step of calculating the spectral norm of the product matrix includes: Calculate the left and right column vectors corresponding to the singular values ​​of the product matrix, respectively. The first formula for calculating the left column vector corresponding to the singular values ​​of the product matrix is: The second formula for calculating the right column vector corresponding to the singular values ​​of the product matrix is: Where n is the power iteration number, W represents the product matrix, u represents the left column vector, and v represents the right column vector. When n equals 1, It is a random vector; The third calculation formula is invoked, and the left column vector and the right column vector are substituted into the third calculation formula to calculate the spectral norm, wherein the third calculation formula is: Through iteration, an approximate value of the spectral norm is obtained. , which is the maximum singular value.

2. The training method for the decomposed convolution model according to claim 1, wherein, The two low-rank matrices are obtained by decomposing the weight matrix of the standard convolutional layer corresponding to the convolutional layer according to the convolution kernel parameters of the standard convolutional layer, based on the spectral normalization decomposition.

3. The training method for the decomposed convolution model according to claim 2, wherein, The convolution kernel parameters include the number of input channels c of the convolution kernel. i Number of output channels c o and kernel size ; The size of the weight matrix is ,in, , ; The two low-rank matrices are low-rank matrix P and low-rank matrix Q, wherein the size of low-rank matrix P is... The size of the low-rank matrix Q is r represents the rank of the low-rank matrix P and the low-rank matrix Q. .

4. A method for predicting lesion areas, wherein, include: Acquire case images; The case image is input into a decomposition convolution model for feature extraction to obtain a feature map corresponding to the case image. The decomposition convolution model is trained by the training method of any one of the decomposition convolution models described in claims 1-3. The feature map is input into the target detection model to identify and label the lesion areas and lesion information on the case image.

5. A training apparatus for decomposing a convolutional model, used to implement the training method for decomposing a convolutional model according to any one of claims 1-3, wherein, include: The first acquisition module is used to acquire training data; The training module is used to input the training data into the convolutional neural network for model training. During the training process, the spectral normalization decomposition convolutional layer of the convolutional neural network performs two convolution operations on the training data based on its two low-rank matrices, and performs spectral normalization on the values ​​of the two low-rank matrices. The loop module is used to iterate through the model training steps until the model converges, resulting in a decomposed convolutional model.

6. A lesion area prediction device for implementing the lesion area prediction method of claim 4, wherein, include: The second acquisition module is used to acquire case images; The extraction module is used to input the case image into the decomposition convolution model for feature extraction to obtain the feature map corresponding to the case image, wherein the decomposition convolution model is trained by the training method of any one of the decomposition convolution models in claims 1-3; The recognition module inputs the feature map into the target detection model to identify and label the lesion areas and lesion information on the case image.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein... When the processor executes the computer program, it implements the steps of the training method for the decomposed convolution model according to any one of claims 1 to 3 and the steps of the lesion region prediction method according to claim 4.

8. A computer-readable storage medium having a computer program stored thereon, wherein, When the computer program is executed by the processor, it implements the steps of the training method for the decomposed convolution model according to any one of claims 1 to 3 and the steps of the lesion area prediction method according to claim 4.

Images