CV-based post-thyroid-resection neck scar recovery state prediction method and equipment

Abstract

The invention belongs to the field of intelligent medical treatment, and particularly relates to a post-thyroid resection neck scar recovery state prediction method based on CV. The method comprises the following steps: acquiring postoperative scar image data of a to-be-detected person; inputting the postoperative scar image data into each branch of the multi-branch group convolutional neural network at the same time to obtain scar image feature mapping of each branch, and performing feature fusion on the scar image feature mapping of each branch to obtain multi-branch fused scar image features, performing full connection layer conversion on the multi-branch fused scar image features to obtain final scar image feature vectors; the final scar image feature vector is input into a classifier to obtain scar score classification, the classification category is the score of the scar image data, and the higher the score is, the higher the scar hyperplasia degree is. According to the method, the multi-branch group convolutional neural network is adopted, and the scar image features of different scales and directions are extracted through a plurality of convolution branches, so that the feature capture capability of the complex scar image is enhanced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medical treatment, and more specifically, to a method, device, medium and program product for predicting the recovery state of neck scar after thyroidectomy based on CV. Background Art

[0002] Postoperative scar formation is one of the common complications of many surgical operations. Despite the continuous advancement of medical technology, scar formation and proliferation still trouble many patients. With the popularization of medical trauma repair surgery, patients are increasingly concerned about postoperative recovery, especially the appearance of scars. Excessive scar proliferation not only affects the appearance, but may also cause functional disorders, bringing physical and psychological burdens to patients. Therefore, how to evaluate the recovery status of scars, especially the accurate evaluation of scar proliferation in different time periods after surgery, is an important topic in clinical practice.

[0003] At present, scar assessment mainly relies on the experience and visual inspection of clinicians, usually combined with a standardized scoring system. However, manual assessment methods are highly subjective and have inconsistent assessment standards, which can easily lead to deviations and errors in the assessment results. In addition, traditional scoring systems are mostly based on two-dimensional images or direct observation, which cannot fully reflect complex information such as scar depth, texture, pigmentation, etc., and therefore cannot provide sufficient basis for scar treatment and intervention.

[0004] The prior art has the following deficiencies:

[0005] 1. Traditional convolutional neural networks are usually unable to effectively handle transformations such as rotation and translation in scar images, which leads to insufficient robustness to image features. Especially when dealing with scars with irregular shapes or complex textures, the features extracted by the model are often not accurate enough, affecting the evaluation results of the scar recovery status.

[0006] 2. In the existing technology, feature extraction is usually a fixed convolution operation, lacking a dynamic adjustment mechanism, which leads to insufficient performance of the model when processing multi-scale and multi-directional scar images and difficulty in capturing key feature information at different scales.

[0007] 3. The parameter update method of traditional convolutional neural networks is usually linear and lacks the characteristics of nonlinear dynamics. As a result, when processing complex scar images, the nonlinear expression ability of the model is limited, the training efficiency is low, and the ability to explore the feature space of scar images is insufficient.

[0008] 4. In the existing classification method based on support vector machine, the fixed allocation of feature weights easily leads to the model's inability to distinguish important features from irrelevant features. Especially when processing highly complex scar image feature vectors obtained through feature extraction, the model is prone to misjudgment or overfitting, and the accuracy and robustness of the classification are limited. Summary of the invention

[0009] In view of the above problems, the present invention provides a CV-based method for predicting the recovery status of neck scar after thyroidectomy, which uses computer vision (CV) to perform data processing and feature extraction on images to capture information in image data, and proposes improvements to the prediction model, thereby constructing a prediction model for the recovery status of neck scar after thyroidectomy that is suitable for clinical use.

[0010] The present application (first aspect) discloses a method for predicting the recovery state of neck scar after thyroidectomy based on CV, comprising:

[0011] Acquiring postoperative scar image data of the subject;

[0012] The postoperative scar image data is simultaneously input into each branch of the multi-branch convolutional neural network to obtain scar image feature maps of each branch, the scar image feature maps of each branch are subjected to feature fusion to obtain multi-branch fused scar image features, and the multi-branch fused scar image features are subjected to full connection layer conversion to obtain the final scar image feature vector;

[0013] The final scar image feature vector is input into a classifier to obtain a scar score classification, wherein the classification category is a score for the scar image data, and a higher score indicates a higher degree of scar hyperplasia.

[0014] Furthermore, the scar image feature maps extracted by each branch are expressed as:

[0015] F b =Ress(T b )

[0016] Among them, F b is the scar image feature map of the b-th branch, T b Represents the convolutional features extracted by the b-th branch; Ress() is an adaptive hybrid activation function;

[0017] Optionally, the adaptive hybrid activation function is expressed as:

[0018] F b =Ress(T b )=α·tanh(T b )+β·ReLU(T b )+γ·ELU(T b )

[0019] Among them, F b is the scar image feature map of the b-th branch, T bRepresents the convolutional features extracted by the b-th branch; Ress() is an adaptive hybrid activation function, α is the first learnable parameter of the adaptive hybrid activation function, β is the second learnable parameter of the adaptive hybrid activation function, γ is the third learnable parameter of the adaptive hybrid activation function, and satisfies α+β+γ=1; tanh() is a hyperbolic tangent function; ReLU() is a rectified linear unit function; ELU() is an exponential linear unit function.

[0020] Furthermore, the step of simultaneously inputting the postoperative scar image data into each branch of the multi-branch convolutional neural network to obtain scar image feature maps of each branch includes: convolving the postoperative scar image data with the group convolution kernel of each branch to obtain extracted convolution features, and mapping the extracted convolution features to obtain the scar image feature map;

[0021] Optionally, the extracted convolutional features are expressed as:

[0022] T b =G b *X+bc b

[0023] Among them, T b represents the convolutional features extracted by the b-th branch, G b is the group convolution kernel of the b-th branch of the group convolutional neural network, X represents the postoperative scar image data, bc b It represents the bias of the bth branch when multi-branch features are fused, and * represents the convolution operation;

[0024] Optionally, the group convolution kernel of each branch is a linear combination of multiple sub-convolution kernels;

[0025] Optionally, the group convolution kernel of each branch is expressed as:

[0026]

[0027] In the formula, G b represents the group convolution kernel of each branch, K is the number of sub-kernels of the b-th branch group convolution; g b,k is the kth subconvolution kernel in the bth branch; w b,k is the weight coefficient of the kth subconvolution kernel in the bth branch;

[0028] Optionally, in the group convolutional neural network of each branch, the convolution operation is expressed as:

[0029]

[0030] In the formula, is the nth in the bth branch pr Convolution result of scale convolution kernel, npr Represents the scale of the convolution kernel, that is, the step size of the convolution kernel; M is the size of the convolution kernel; is the postoperative scar image data at position i center -The pixel value at position m, where m represents the index within the scale variation range of the convolution kernel, i center Indicates the index of the center pixel of the current convolution.

[0031] Furthermore, the calculation method of the final scar image feature vector representation is:

[0032] Z=Sig(ω FC FIN n +bc FC )

[0033] Where Z is the final scar image feature vector of postoperative scar image data; Sig() is the Sigmoid activation function; ω FC and bc FC are the weights and biases of the fully connected layer respectively;

[0034] Optionally, the multi-branch fused scar image feature is expressed as:

[0035] FIN=φ(F 1 ,F 2 ,…,F b ,…F B )

[0036] Among them, FIN represents the scar image features of multi-branch fusion; φ() represents the function of feature cascade; F 1 is the scar image feature map of the first branch, F b is the scar image feature map of the b-th branch, where b∈[1,B], and B represents the total number of branches of the multi-branch group convolutional neural network.

[0037] Furthermore, the parameters in the multi-branch group convolutional neural network are obtained through training, and the training process includes: collecting a postoperative scar image dataset of patients in the test set;

[0038] annotating the postoperative scar image dataset,

[0039] The set of postoperative scar image data sequences are sequentially input into a multi-branch group convolutional neural network with initial parameters to extract the corresponding final scar image feature vector;

[0040] Based on the final scar image feature vector, classification is performed to obtain classification labels, scar image feature extraction losses are calculated based on the classification labels and the annotations of the postoperative scar image data set, and parameters in the multi-branch group convolutional neural network are obtained by iteratively optimizing the scar image feature extraction loss training;

[0041] Optionally, the calculation method of the scar image feature extraction loss is expressed as:

[0042]

[0043] Where, L n represents the scar image feature extraction loss of the nth sample, Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n The corresponding category, Y n is the annotation of the nth sample in the postoperative scar image dataset; is the multi-classification cross entropy loss of the nth sample; λ1 is the weight coefficient of the first scar image feature extraction loss, λ2 is the weight coefficient of the second scar image feature extraction loss, and λ3 is the weight coefficient of the third scar image feature extraction loss.

[0044] Further, the parameters are obtained by optimizing the training using reinforcement learning, and the strategy function of the reinforcement learning is π(a b |s n ), the policy function represents the state s n Next select action a b The probability that action a b The action for the b-th branch includes adjusting the convolution kernel parameters and / or weights, expressed as:

[0045]

[0046] In the formula, s n is the environmental state of the nth sample, including the feature extraction result of the current scar image; a is all possible actions in the current environment;

[0047] The value function of the reinforcement learning task is expressed as:

[0048]

[0049] In the formula, V(s n ) is the value function, estimated in state s n The expected reward under R n is an immediate reward, reflecting the effect of scar image feature extraction; γ RL is the discount factor; is the expected function;

[0050] Preferably, the instant reward of the reinforcement learning is calculated based on the accuracy of scar image feature extraction and the complexity of the model;

[0051] Preferably, the instant reward calculation method of the reinforcement learning is expressed as:

[0052]

[0053] In the formula, μ RL is the first trade-off coefficient, γ RL is the second trade-off coefficient; Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n The corresponding category, Y n is the label of the nth sample in the postoperative scar image dataset; || || represents the square of the L2 norm; w b is the weight vector of the b-th branch;

[0054] Optionally, a comprehensive loss function is obtained based on iterative optimization of the scar image feature extraction loss and reinforcement learning reward calculation, and the parameters in the multi-branch group convolutional neural network are obtained by iterative optimization of the comprehensive loss function training;

[0055] Optionally, the comprehensive loss function is expressed as:

[0056]

[0057] Where Loss′ is the comprehensive loss function; L n represents the scar image feature extraction loss of the nth sample; λ p is the trade-off coefficient of the comprehensive loss function;

[0058] Optionally, the classifier is any one or more of the following: support vector machine, logistic regression, convolutional network, ensemble learning;

[0059] Optionally, the classifier is obtained through training, and the training process includes: collecting a postoperative scar image dataset of patients in a test set; annotating the postoperative scar image dataset, sequentially inputting the set of postoperative scar image data sequences into a multi-branch group convolutional neural network to extract corresponding multi-branch fused scar image features, inputting the multi-branch fused scar image features into an initial classifier to obtain a predicted label, calculating the classifier loss based on the predicted label and the annotation of the postoperative scar image dataset, and training the classifier by iteratively optimizing the classifier loss;

[0060] Optionally, the classifier adopts a support vector machine.

[0061] Optionally, the classifier loss also includes a penalty term for scar image features fused by multiple branches;

[0062] Optionally, the classifier loss is expressed as:

[0063]

[0064] In the formula, y i represents the labeling score of the i-th sample input to the support vector machine, Z i,j The final scar image feature vector Z represents the i-th input image i,j The jth eigenvalue of j is the feature weight of the jth eigenvalue of the final scar image feature vector extracted by the multi-branch group convolutional neural network input to the support vector machine; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; max() is the maximum value function; f() is the classification decision function of the support vector machine, C u is the penalty coefficient of the support vector machine; θ represents the kernel function parameter of the support vector machine;

[0065] Optionally, the kernel function parameters and penalty coefficients of the support vector machine are updated as follows:

[0066]

[0067] Where η svm is the learning rate of the support vector machine; θ(t) is the parameter of the kernel function of the tth iteration, and θ(t+1) is the parameter of the kernel function of the t+1th iteration; C u (t) is the penalty coefficient of the support vector machine at the tth iteration, C u (t+1) is the penalty coefficient of the support vector machine at the t+1th iteration; L svm () is the loss function of the support vector machine, is the gradient of the kernel function parameters; is the gradient of the penalty coefficient of the support vector machine.

[0068] Furthermore, the parameter update process of the multi-branch group convolutional neural network is based on the chaotic mapping mechanism of nonlinear dynamics;

[0069] Optionally, in the parameter updating process of the chaotic mapping mechanism based on nonlinear dynamics, a chaotic mapping function is used to generate a sequence with chaotic characteristics, and the calculation method is expressed as:

[0070] Xcs(t+1)=ρ ch Xcs(t)(1-Xcs(t))+δ ch sin(πXcs(t))

[0071] Where Xcs(t) is the chaotic variable of the t-th iteration, Xcs(t+1) is the chaotic variable of the t+1-th iteration; ρch is the control parameter of the chaotic mapping; δ ch is the modulation parameter to control the complexity of the chaotic sequence;

[0072] Optionally, the parameters of the multi-branch group convolutional neural network are updated based on the gradient information of the loss function using a back propagation algorithm based on chaotic variables;

[0073] Optionally, the updating method is expressed as:

[0074]

[0075] Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable of the tth iteration, G b (t), G b (t+1) represents the G of the tth and t+1th iterations respectively. b , G b represents the group convolution kernel of each branch, w b (t), w b (t+1) represents the w of the tth and t+1th iterations respectively. b , w b represents the weight vector of the bth branch; bc b (t), bc b (t+1) represents the bc of the tth and t+1th iterations respectively. b , bc b Represents the bias of the b-th branch when multi-branch features are fused.

[0076] The second aspect of the present application discloses a postoperative scar recovery state prediction system based on computer vision, comprising:

[0077] Acquisition module: used to acquire postoperative scar image data of the subject to be tested;

[0078] Feature extraction module: used for simultaneously inputting the postoperative scar image data of the subject into each branch of the multi-branch convolutional neural network to obtain scar image feature maps of each branch, fusing the scar image feature maps of each branch to obtain multi-branch fused scar image features, and converting the multi-branch fused scar image features through a fully connected layer to obtain the final scar image feature vector;

[0079] Prediction module: The final scar image feature vector is input into a classifier to obtain a scar score classification, where the classification category is the score of the scar image data, and a higher score indicates a higher degree of scar hyperplasia.

[0080] The third aspect of the present application discloses a computer device, which includes: a memory and a processor; the memory is used to store program instructions; the processor is used to call the program instructions, and when the program instructions are executed, it is used to execute the steps of the above method.

[0081] A fourth aspect of the present application discloses a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the steps of the above method when executed by a processor.

[0082] A fifth aspect of the present application discloses a computer program product, including a computer program, which implements the steps of the above method when executed by a processor.

[0083] This application has the following beneficial effects:

[0084] 1. A multi-branch group convolutional neural network is adopted. Through group convolution operations, the model can maintain equivariance under image transformations such as rotation and translation, which improves the robustness of the model to scar image features. Through multiple convolution branches, scar image features of different scales and directions are extracted, enhancing the feature capture ability of complex scar images.

[0085] 2. A reinforcement learning strategy is used to adjust the weights of scar image feature extraction. The intelligent agent is used to learn the optimal strategy and dynamically adjust the parameters of each branch in the multi-branch network, so that the model can adaptively focus on the most discriminative image features and enhance the model's feature extraction capabilities for complex images.

[0086] 3. Adopting the chaotic mapping mechanism, chaos theory is used in the parameter update process of the convolutional neural network to improve the nonlinear expression and exploration capabilities of the model, enhance the capture of complex scar features, and improve training efficiency and model generalization ability.

[0087] 4. The support vector machine algorithm with penalty term is used to enhance the sensitivity of feature selection. Combined with the complexity of scar image features, the penalty term effectively prevents the model from overfitting. At the same time, the feature weights are adaptively adjusted, so that the model can perform scar scoring and classification more accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0089] Figure 1 is a schematic diagram of a method flow chart provided by the first aspect of an embodiment of the present invention;

[0090] Figure 2 is a schematic diagram of a program product provided by the second aspect of an embodiment of the present invention;

[0091] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention;

[0092] Figure 4 is a schematic diagram of the architecture of an exemplary computing device provided by an embodiment of the present invention;

[0093] Figure 5 is a schematic diagram of a storage medium provided by an embodiment of the present invention;

[0094] Figure 6 is a schematic diagram of image annotation provided by an embodiment of the present invention;

[0095] Figure 7 is a schematic diagram of image annotation provided by an embodiment of the present invention;

[0096] Figure 8 A data flow chart for inputting postoperative scar image data to obtaining a predicted label provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0097] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0098] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The sequence numbers of the operations, such as S101, S102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

[0099] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0100] Figure 1The present invention provides a flow chart of a method for predicting the recovery state of neck scar after thyroidectomy based on CV evaluation, specifically, the method comprises the following steps:

[0101] S101: Acquire postoperative scar image data of a subject;

[0102] S102: Input the postoperative scar image data into a multi-branch group convolutional neural network to extract and obtain a final scar image feature vector; input the postoperative scar image data of the subject into each branch of the multi-branch convolutional neural network at the same time to obtain scar image feature maps of each branch, and obtain multi-branch fused scar image features through feature fusion of the scar image feature maps of each branch, and obtain the final scar image feature vector through conversion of the multi-branch fused scar image features through a fully connected layer:

[0103] S103: The final scar image feature vector is input into a classifier to obtain a scar score classification, where the classification category is a score for the scar image data, ranging from 1 to 10, corresponding to 10 integer category labels, and a higher score indicates a higher degree of scar hyperplasia.

[0104] The present application proposes a postoperative scar recovery status assessment method based on a computer vision algorithm, and implements scar recovery status assessment using a scar recovery status assessment model, wherein the scar recovery status assessment model is obtained through training, and specific training data collection is derived from clinical trials, specifically, data is obtained from three experimental groups: a control group (only Mepiform), a photoelectric group (Mepiform + photoelectric treatment), and a botulinum toxin group (Mepiform + botulinum toxin injection);

[0105] The collection method was to take 3D photos and dermoscopic photos of each patient at 2 weeks, 1 month, 3 months, and 6 months after surgery;

[0106] After the collected data is annotated, it is stored in a digital format, specifically including image data and text scores. The text score is to score each scar image data through manual evaluation, ranging from 1 to 10, and the higher the score, the higher the degree of scar hyperplasia ( Figure 6 , Figure 7 ); the storage format of scar image data is JPEG, and the resolution is 1024×768 pixels.

[0107] The scar recovery status assessment model is a multi-branch group convolutional neural network, and the model is trained by supervised training;

[0108] Group Equivariant Convolutional Neural Networks (G-CNNs) is a network structure that expands the traditional convolutional neural network. By using the concepts in group theory, the network has equivariance under specific transformations (such as rotation, translation, etc.). The equivariance means that the output of the network remains unchanged for specific transformations of the input, thereby improving the robustness of the network to these transformations.

[0109] The multi-branch group convolutional neural network is a plurality of parallel convolutional branches ( Figure 8 As shown in the figure, each branch extracts scar image feature information of different scales and directions of scar image data through group convolution operation. In addition, the weights and parameters of each branch are dynamically adjusted by using a hierarchical strategy, so that the model can adaptively focus on the most discriminative scar image features, thereby improving the accuracy and robustness of scar image feature extraction.

[0110] Specifically, the training process of the multi-branch group convolutional neural network algorithm is as follows:

[0111] 1. Set the structure of the multi-branch group convolutional neural network, including the number of branches, the size and number of group convolution kernels of each branch, and the configuration of the fully connected layer. Specifically, suppose there are B convolutional branches in total, and the group convolution kernel of the bth branch of the multi-branch group convolutional neural network is G b , the weight of the fully connected layer of the b-th branch of the multi-branch group convolutional neural network is W b , the bias of the fully connected layer of the bth branch of the multi-branch group convolutional neural network is bc b , and b=1,2,…,B.

[0112] 2. Input scar image data into the model. Each image is simultaneously input into all convolution branches of the multi-branch group convolutional neural network. Specifically, let the nth scar image data be represented as X n , and n = 1, 2, ..., N, N is the total number of training samples.

[0113] 3. Multi-branch group convolutional neural network Each convolution branch extracts scar image features of different levels and scales through group convolution layer, activation function and pooling layer structure. The local and global information of the image can be captured through group convolution operation. Specifically, for scar image data, the way to extract scar image features through each convolution branch is expressed as:

[0114] F b,n =Ress(G b *X n +bc b )

[0115] In the formula, F n,bis the scar image feature map extracted from the nth sample in the bth branch; Ress() is an adaptive hybrid activation function; * indicates a convolution operation.

[0116] Furthermore, the group convolution kernel is a linear combination of multiple sub-convolution kernels, which is used to capture scar image features of different scales and directions. The implementation method is expressed as:

[0117]

[0118] Where K is the number of sub-kernels of the group convolution; w b,k is the weight coefficient of the kth subconvolution kernel in the bth branch, obtained through training; g b,k is the kth subconvolution kernel in the bth branch, and its initialization can use different filters (such as Gaussian kernel, Laplacian kernel, etc.) to capture different scar image feature patterns.

[0119] Furthermore, in the multi-branch group convolutional neural network, the convolution operation process is calculated as follows:

[0120]

[0121] In the formula, is the nth in the bth branch pr Convolution result of scale convolution kernel, n pr Represents the scale of the convolution kernel, that is, the step size of the convolution kernel; M is the size of the convolution kernel; For the input scar image at position i center -The pixel value at position m, where m represents the index within the scale variation range of the convolution kernel, i center Indicates the index of the center pixel of the current convolution.

[0122] Furthermore, the purpose of the adaptive hybrid activation function is to enhance the nonlinear expression ability, and the calculation method is expressed as:

[0123] Ress(x p )=α p tanh(x p )+β p ReLU(x p )+γ p ·ELU(x p )

[0124] In the formula, x p is the input value of the adaptive hybrid activation function; α p is the first learnable parameter of the adaptive hybrid activation function, β p is the second learnable parameter of the adaptive hybrid activation function, γ p The third learnable parameter of the adaptive hybrid activation function satisfies αp +β p +γ p =1; tanh() is the hyperbolic tangent function; ReLU() is the rectified linear unit function; ELU() is the exponential linear unit function.

[0125] 4. The scar image features extracted by each branch are fused. The fusion method can be scar image feature splicing, weighted summation, feature cascade, etc., to form a comprehensive scar image feature representation, which is expressed as:

[0126] FIN n =φ(F 1,n ,F 2,n ,…,F 3,n ,…F b,n )

[0127] Where FIN n is the scar image feature of the nth sample after multi-branch fusion; φ() represents the function of feature fusion; F 1,n is the scar image feature map of the first branch of the nth training set sample, F b,n is the scar image feature map of the b-th branch of the n-th training set sample, where b∈[1,B], B represents the total number of branches of the multi-branch group convolutional neural network, and n∈[1,N], N represents the total number of postoperative scar images input into the training set.

[0128] 5. Using the hierarchical strategy of reinforcement learning, the agent dynamically adjusts the weights and parameters of each branch according to the effect of the current scar image feature extraction. The agent updates the strategy to optimize the model performance by observing the results of scar image feature extraction and the preset reward function;

[0129] The agent is an entity that can take actions in a specific environment to maximize a certain cumulative reward in reinforcement learning. The goal of the agent is to learn a strategy that tells it which action to take in a given state to maximize the long-term accumulated reward.

[0130] Specifically, define the policy function as π(a b |s n ) The policy function represents the state s n Next select action a b The probability that action a b The action for the b-th branch may include adjusting the convolution kernel parameters, weights, etc., which can be expressed as:

[0131]

[0132] In the formula, s nis the environmental state of the nth sample, including the feature extraction result of the current scar image; a is all possible actions in the current environment;

[0133] The value function of the reinforcement learning task is expressed as:

[0134]

[0135] In the formula, s n is the environmental state of the nth sample, including the feature extraction result of the current scar image.

[0136] Furthermore, according to the reinforcement learning strategy, the value function of the feature extraction task is:

[0137]

[0138] In the formula, v(s n ) is the value function, estimated in state s n The expected reward under R n is an immediate reward, reflecting the effect of scar image feature extraction; γ RL is the discount factor; is the expected function;

[0139] In one embodiment, γ p Set to 0.3.

[0140] In one embodiment, the immediate reward of reinforcement learning takes into account the accuracy of scar image feature extraction and the complexity of the model, and the calculation method is expressed as:

[0141]

[0142] In the formula, μ RL is the first trade-off coefficient, μ RL is the second trade-off coefficient; Z n is the final scar image feature representation of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n The corresponding category, Y n is the label of the nth sample in the postoperative scar image dataset; || ||∥ represents the square of the L2 norm; w b For b p The weight vector of each branch is composed of all w b,k composition;

[0143] Optionally, a comprehensive loss function is obtained based on iterative optimization of the scar image feature extraction loss and reinforcement learning reward calculation, and the parameters in the multi-branch group convolutional neural network are obtained by iterative optimization of the comprehensive loss function training;

[0144] Furthermore, when the policy function is updated, its parameters are updated using a policy gradient method based on entropy regularization to promote policy exploration, expressed as:

[0145]

[0146] In the formula, θ RL is the policy function π(a|s n ) parameters; is the policy function π(a|s n ) is the gradient of the parameters; ← is the parameter update operation; η RL is the learning rate of the policy function; κ RL is the entropy regularization coefficient; HE(π) is the entropy of the strategy. Preferably, κ RL is set to 0.2. Preferably, η RL Set to 0.001.

[0147] Furthermore, the calculation method of the entropy of the strategy is expressed as:

[0148]

[0149] Where a is the action of all branches, representing the feature extraction operation of each convolution branch of the multi-branch group convolutional neural network.

[0150] 6. The fused scar image features are passed through the fully connected layer to obtain the final scar image feature vector, which is expressed as:

[0151] Z n =Sig(ω FC FIN n +bc FC )

[0152] In the formula, Z n For nth p The final scar image feature vector of samples; Sig() is the Sigmoid activation function; and are the weights and biases of the feature fusion fully connected layer, respectively.

[0153] 7. Define a comprehensive loss function, which is used to evaluate the overall performance of the model, including scar image feature extraction error and reinforcement learning reward items. The calculation method is expressed as:

[0154] In one embodiment, λ R Set to 0.3.

[0155] In one embodiment, the calculation method of scar image feature extraction loss is expressed as:

[0156]

[0157] Where, L n represents the scar image feature extraction loss of the nth sample, Z n is the final scar image feature representation of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n The corresponding category, Y n is the annotation of the nth sample in the postoperative scar image dataset; For nth p The cross entropy loss of samples; λ1 is the weight coefficient of the first scar image feature extraction loss, λ2 is the weight coefficient of the second scar image feature extraction loss, and λ3 is the weight coefficient of the third scar image feature extraction loss.

[0158] Optionally, the comprehensive loss function is expressed as:

[0159]

[0160] Where Loss′ is the comprehensive loss function; L n represents the scar image feature extraction loss of the nth sample; λ R is the trade-off coefficient of the comprehensive loss function;

[0161] 8. In each convolution branch of the multi-branch group convolutional neural network, based on the chaotic mapping mechanism of nonlinear dynamics, chaos theory is applied to the parameter update process of the multi-branch group convolutional neural network to enhance the nonlinear expression ability and training efficiency of the model, so that the convolution kernel parameters present a chaotic distribution, increase the ability to explore the scar image feature space, and improve the capture effect of the scar image features of complex scars. Specifically, the chaotic mapping function is used to generate a sequence with chaotic characteristics, and the calculation method is expressed as:

[0162] Xcs(t+1)=ρ ch Xcs(t)(1-Xcs(t))+δ ch sin(πXcs(t))

[0163] Where Xcs(t) is the chaotic variable of the t-th iteration, Xcs(t+1) is the chaotic variable of the t+1-th iteration; ρ ch is the control parameter of the chaotic mapping, satisfying 3.5699456<ρ ch ≤4;δ ch is the modulation parameter that controls the complexity of the chaotic sequence.

[0164] 9. Based on chaotic variables, the back propagation algorithm is used to update the parameters of the multi-branch group convolutional neural network according to the gradient information of the loss function. Specifically, the update method is expressed as:

[0165]

[0166] Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable of the tth iteration, G b (t), G b (t+1) represents the G of the tth and t+1th iterations respectively. b , G b represents the group convolution kernel of each branch, W b (t), W b (t+1) represents the W of the tth and t+1th iterations respectively. b , W b represents the weight vector of the bth branch; bc b (t), bc b (t+1) represents the bc of the tth and t+1th iterations respectively. b , bc b Indicates the bias of the bth branch when multi-branch features are fused;

[0167] Preferably, n sep Set to 0.004.

[0168] In some embodiments, the update is represented by:

[0169]

[0170] Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable of the tth iteration, G b (t), G b (t+1) represents the G of the tth and t+1th iterations respectively. b , G b represents the group convolution kernel of each branch, W b (t), W b (t+1) represents the W of the tth and t+1th iterations respectively. b , W b represents the weight vector of the bth branch; bc b (t), bc b (t+1) represents the bc of the tth and t+1th iterations respectively. b , bc b Represents the bias of the b-th branch when multi-branch features are fused.

[0171] 10. Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations or the change curve of the comprehensive loss function reaches a convergence state. Preferably, the preset maximum number of iterations is set to 1000 times.

[0172] Further, the scar image features extracted by the multi-branch group convolutional neural network are input into the classifier for training the classifier model, and the classifier model is used to obtain a scar image data score output according to the scar image features extracted by the multi-branch group convolutional neural network, ranging from 1 to 10, corresponding to 10 integer category labels, and the higher the score, the higher the degree of scar hyperplasia;

[0173] The classifier model adopts a penalty-term-based support vector machine algorithm. On the basis of the support vector machine algorithm, a penalty term for scar image feature vectors extracted by a multi-branch group convolutional neural network is adopted to enhance the model's sensitivity to feature selection, thereby improving classification accuracy.

[0174] Specifically, the training process of the penalty-based support vector machine algorithm is as follows:

[0175] 1. Initialize the kernel function parameters and penalty coefficients of the support vector machine. Set the kernel function parameters of the support vector machine to θ and the penalty coefficient of the support vector machine to C. u , the initialization method is expressed as:

[0176] θ=randn(0,1)·σ θ

[0177]

[0178] In the formula, randn(0,1) represents the uniform distribution in the interval (0,1); θ represents the parameter of the kernel function; σ θ is the initial standard deviation of the kernel parameters; C u is the penalty coefficient of the support vector machine, which is determined by the target variable Y u Initialize with the inverse of the variance of u is the target variable, representing the category vector of the sample labels of the scar image features extracted by the multi-branch group convolutional neural network input into the support vector machine; var() is the variance function.

[0179] 2. Assume that the scar image feature vector extracted by the multi-branch group convolutional neural network is Z, and the gradient descent method is used to optimize the parameters of the support vector machine. That is, in each iterative training process of the support vector machine, the loss of the current support vector machine model is calculated, and the parameters of the kernel function and the penalty term coefficient are updated according to the loss. The update method is expressed as:

[0180]

[0181] Where η svm is the learning rate of the support vector machine; θ(t) is the parameter of the kernel function of the tth iteration, and θ(t+1) is the parameter of the kernel function of the t+1th iteration; C u (t) is the penalty coefficient of the support vector machine at the tth iteration, C u (t+1) is the penalty coefficient of the support vector machine at the t+1th iteration; L svm () is the loss function of the support vector machine, is the gradient of the kernel function parameters; is the gradient of the penalty coefficient of the support vector machine. Preferably, η svm Set to 0.01.

[0182] In one embodiment, the loss function of the support vector machine adopts a loss calculation method with a penalty term, which is expressed as:

[0183]

[0184] In the formula, y i represents the labeling score of the i-th sample input to the support vector machine, Z i,j The final scar image feature vector Z represents the i-th input image i,j The jth eigenvalue of ; d is the dimension of scar image feature samples extracted by a single multi-branch group convolutional neural network; w j is the feature weight of the jth eigenvalue of the final scar image feature vector extracted by the multi-branch group convolutional neural network input to the support vector machine; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; max() is the maximum value function; f() is the classification decision function of the support vector machine, C u is the penalty coefficient of the support vector machine; θ represents the kernel function parameter of the support vector machine.

[0185] 3. Adopt an adaptive feature weight adjustment mechanism to dynamically adjust the weight of each feature according to its contribution in the classification task, enhance the sensitivity of the model to key features, and improve the classification accuracy. Specifically, for the feature weights of the feature vectors of the scar image feature samples extracted by the single multi-branch group convolutional neural network input to the support vector machine, at the beginning of model training, a weight is assigned to each feature, which is initialized to 1, indicating that all features are considered equally important in the initial state, expressed as:

[0186]

[0187] In the formula, is the feature weight of the jth eigenvalue of the feature vector of the scar image feature sample extracted by the single multi-branch group convolutional neural network that is initially input into the support vector machine.

[0188] Furthermore, the adjustment of feature weights is based on the contribution of the features to the model classification boundary, and a gradient-based method is used to update the weights, which is expressed as:

[0189]

[0190] Where η tz is the learning rate for updating feature weights; w j (t+1) is the feature weight of the j-th eigenvalue of the feature vector of the scar image feature sample extracted by the single multi-branch group convolutional neural network input to the support vector machine at the t+1th iteration.

[0191] 4. Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0192] After the multi-branch group convolutional neural network and the penalty-based support vector machine are trained, the trained multi-branch group convolutional neural network and the penalty-based support vector machine are used to extract scar image features and evaluate scar recovery status;

[0193] In one embodiment, the collected original scar image data is input into a trained multi-branch group convolutional neural network for feature extraction, and further, the extracted scar image features are input into a penalty-based support vector machine for classification, thereby obtaining a classification result. In this embodiment, the classification category is a score for the scar image data, ranging from 1 to 10, corresponding to 10 integer category labels, and a higher score indicates a higher degree of scar hyperplasia.

[0194] Note: The training set is used in the model training process, and the image to be evaluated is input when the trained model is used. When the superscript n in the expression in the above embodiment is included, it means the nth image or nth sample input during training, for example:

[0195] Image data X represents the image input when using the model. n Represents the nth sample during training.

[0196] F 1 represents the scar image feature map of the first branch corresponding to the input image when the model is applied, F 1,n Represents the scar image feature map of the first branch corresponding to the nth sample during training.

[0197] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention, such as Figure 3 As shown, the device may include: one or more processors, and one or more memories; wherein the memories store computer-readable codes, and when the computer-readable codes are run by the one or more processors, the method described above may be executed.

[0198] The processor in this embodiment can be an integrated circuit chip with signal processing capabilities. The above processor can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The disclosed methods, operations and logic block diagrams in the embodiments of the present disclosure can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can be an X86 architecture or an ARM architecture.

[0199] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the disclosed embodiments are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0200] For example, the method or device according to the embodiment of the present disclosure may also be implemented by Figure 4 The architecture of the computing device 3000 shown in FIG. Figure 4 As shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as ROM 3030 or hard disk 3070, may store various data or files used for processing and / or communication of the method provided by the present disclosure and program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 The architecture shown is only exemplary and can be omitted according to actual needs when implementing different devices. Figure 4 One or more components of a computing device are shown.

[0201] The embodiment of the present invention also provides a computer-readable storage medium, such as Figure 5 As shown, it is a schematic diagram of a storage medium provided by an embodiment of the present invention, and a computer readable instruction 4010 is stored on the computer storage medium 4020. When the computer readable instruction 4010 is executed by the processor, the method according to the embodiment of the present disclosure described with reference to the above figures can be executed. The computer readable storage medium in the embodiment of the present disclosure can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. The non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM) or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory. It should be noted that the memory of the methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0202] The present disclosure also provides a computer program product or a computer program, which implements the steps of the above method when executed by a processor, such as Figure 2 As shown, the computer program product or computer program comprises:

[0203] Acquisition module 201: used to acquire postoperative scar image data of a subject;

[0204] Feature extraction module 202: for simultaneously inputting the postoperative scar image data of the subject into each branch of the multi-branch convolutional neural network to obtain scar image feature maps of each branch, fusing the scar image feature maps of each branch to obtain multi-branch fused scar image features, and converting the multi-branch fused scar image features through a fully connected layer to obtain the final scar image feature vector;

[0205] Prediction module 203: inputs the final scar image feature vector into a classifier to obtain a scar score classification, where the classification category is the score of the scar image data, and a higher score indicates a higher degree of scar hyperplasia.

[0206] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, a program segment, or a part of a code, and the module, program segment, or a part of the code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0207] In general, various example embodiments of the present disclosure may be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Certain aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device. When various aspects of the disclosed embodiments are illustrated or described as block diagrams, flow charts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein may be implemented in hardware, software, firmware, dedicated circuits or logic, general purpose hardware or controllers or other computing devices, or some combination thereof as non-limiting examples.

[0208] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0210] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0212] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. It should be understood by those skilled in the art that various modifications and combinations may be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A method for predicting the recovery state of neck scar after thyroidectomy based on CV, characterized in that: The method comprises: obtaining postoperative scar image data of a subject to be tested; The postoperative scar image data is simultaneously input into each branch of the multi-branch group convolutional neural network to obtain scar image feature maps of each branch, the scar image feature maps of each branch are subjected to feature fusion to obtain multi-branch fused scar image features, and the multi-branch fused scar image features are subjected to full connection layer conversion to obtain a final scar image feature vector; The final scar image feature vector is input into a classifier to obtain a scar score classification, wherein the classification category is a score for the scar image data, and a higher score indicates a higher degree of scar hyperplasia.

2. The method for predicting the recovery state of neck scar after thyroidectomy based on CV according to claim 1, characterized in that: The scar image feature maps extracted by each branch are expressed as: F b =Ress(T b ) Among them, F b is the scar image feature map of the b-th branch, T b Represents the convolutional features extracted by the b-th branch; Ress() is an adaptive hybrid activation function; Optionally, the adaptive hybrid activation function is expressed as: F b =Ress(T b )=α·tanh(T b )+β·ReLU(T b )+γ·ELU(T b ) Among them, F b is the scar image feature map of the b-th branch, T b Represents the convolutional features extracted by the b-th branch; Ress() is an adaptive hybrid activation function, α is the first learnable parameter of the adaptive hybrid activation function, β is the second learnable parameter of the adaptive hybrid activation function, γ is the third learnable parameter of the adaptive hybrid activation function, and satisfies α+β+γ=1; tanh() is a hyperbolic tangent function; ReLU() is a rectified linear unit function; ELU() is an exponential linear unit function.

3. The method for predicting the recovery state of neck scar after thyroidectomy based on CV according to claim 1, characterized in that: The step of simultaneously inputting the postoperative scar image data into each branch of the multi-branch group convolutional neural network to obtain scar image feature maps of each branch comprises: convolving the postoperative scar image data with the group convolution kernel of each branch to obtain extracted convolution features, and mapping the extracted convolution features to obtain the scar image feature map; Optionally, the extracted convolutional features are expressed as: T b =G b *X+bc b Among them, T b represents the convolutional features extracted by the b-th branch, G b is the group convolution kernel of the b-th branch of the group convolutional neural network, X represents the postoperative scar image data, bc b It represents the bias of the bth branch when multi-branch features are fused, and * represents the convolution operation; Optionally, the group convolution kernel of each branch is a linear combination of multiple sub-convolution kernels; Optionally, the group convolution kernel of each branch is expressed as: In the formula, G b represents the group convolution kernel of each branch, K is the number of sub-kernels of the b-th branch group convolution; g b,k is the kth subconvolution kernel in the bth branch; w b,k is the weight coefficient of the kth subconvolution kernel in the bth branch; Optionally, in the group convolutional neural network of each branch, the convolution operation is expressed as: In the formula, is the nth in the bth branch pr Convolution result of scale convolution kernel, n pr Represents the scale of the convolution kernel, that is, the step size of the convolution kernel; M is the size of the convolution kernel; is the postoperative scar image data at position i center -The pixel value at position m, where m represents the index within the scale variation range of the convolution kernel, i center Indicates the index of the center pixel of the current convolution.

4. The method for predicting the recovery state of neck scar after thyroidectomy based on CV according to claim 1, characterized in that: The final scar image feature vector is calculated as follows: Z=Sig(ω FC THE END n +bc FC ) Where Z is the final scar image feature vector of postoperative scar image data; Sig() is the Sigmoid activation function; ω FC and bc FC are the weights and biases of the fully connected layer respectively; Optionally, the multi-branch fused scar image feature is expressed as: FIN=φ(F 1 ,F 2 ,…,F b ,…F B ) Among them, FIN represents the scar image features of multi-branch fusion; φ() represents the function of feature cascade; F 1 is the scar image feature map of the first branch, F b is the scar image feature map of the b-th branch, where b∈[1,B], and B represents the total number of branches of the multi-branch group convolutional neural network.

5. The method for predicting the recovery state of neck scar after thyroidectomy based on CV according to claim 1, characterized in that: The parameters in the multi-branch group convolutional neural network are obtained through training, and the training process includes: collecting a postoperative scar image data set of patients in the test set; annotating the postoperative scar image dataset, The postoperative scar image data set is input with the multi-branch group convolutional neural network with initial parameters to extract the corresponding final scar image feature vector; Based on the final scar image feature vector, classification is performed to obtain classification labels, scar image feature extraction losses are calculated based on the classification labels and the annotations of the postoperative scar image data set, and parameters in the multi-branch group convolutional neural network are obtained by iteratively optimizing the scar image feature extraction loss training; Optionally, the calculation method of the scar image feature extraction loss is expressed as: Where, L n represents the scar image feature extraction loss of the nth sample, Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n The corresponding category, Y n is the annotation of the nth sample in the postoperative scar image dataset; is the multi-classification cross entropy loss of the nth sample; λ1 is the weight coefficient of the first scar image feature extraction loss, λ2 is the weight coefficient of the second scar image feature extraction loss, and λ3 is the weight coefficient of the third scar image feature extraction loss.

6. The method for predicting the recovery state of neck scar after thyroidectomy based on CV according to claim 5, characterized in that: Reinforcement learning is used to optimize the training to obtain the parameters of the multi-branch group convolutional neural network. The strategy function of the reinforcement learning is π(a b |s n ), the policy function represents the state s n Next select action a b The probability that action a b The action for the b-th branch includes adjusting the convolution kernel parameters and / or weights, expressed as: In the formula, s n is the environmental state of the nth sample, including the feature extraction result of the current scar image; a is all possible actions in the current environment; The value function of the reinforcement learning task is expressed as: In the formula, V(s n ) is the value function, estimated in state s n The expected reward under R n is an immediate reward, reflecting the effect of scar image feature extraction; γ RL is the discount factor; is the expected function; Optionally, the instant reward of the reinforcement learning is calculated based on the accuracy of scar image feature extraction and the complexity of the model; Optionally, the instant reward calculation method of the reinforcement learning is expressed as: In the formula, μ RL is the first trade-off coefficient, γ RL is the second trade-off coefficient; Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n The corresponding category, Y n is the label of the nth sample in the postoperative scar image dataset; || || represents the square of the L2 norm; w b is the weight vector of the b-th branch; Optionally, a comprehensive loss function is obtained based on iterative optimization of the scar image feature extraction loss and reinforcement learning reward calculation, and the parameters in the multi-branch group convolutional neural network are obtained by iterative optimization of the comprehensive loss function training; Optionally, the comprehensive loss function is expressed as: Where Loss′ is the comprehensive loss function; L n represents the scar image feature extraction loss of the nth sample; λ p is the trade-off coefficient of the comprehensive loss function; Optionally, the classifier is any one or more of the following: support vector machine, logistic regression, convolutional network, ensemble learning; Optionally, the classifier is obtained through training, and the training process includes: collecting a postoperative scar image dataset of patients in a test set; annotating the postoperative scar image dataset, sequentially inputting the postoperative scar image dataset into a multi-branch group convolutional neural network to extract corresponding multi-branch fused scar image features, inputting the multi-branch fused scar image features into an initial classifier to obtain predicted labels, calculating the classifier loss based on the predicted labels and the annotations of the postoperative scar image dataset, and obtaining the classifier by iteratively optimizing the classifier loss; Optionally, the classifier adopts a support vector machine. Optionally, the classifier loss also includes a penalty term for scar image features fused by multiple branches; Optionally, the classifier loss is expressed as: In the formula, y i represents the labeling score of the i-th sample input to the support vector machine, Z i,j The final scar image feature vector Z represents the i-th input image i,j The jth eigenvalue of j is the feature weight of the jth eigenvalue of the final scar image feature vector extracted by the multi-branch group convolutional neural network input to the support vector machine; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; max() is the maximum value function; f() is the classification decision function of the support vector machine, C u is the penalty coefficient of the support vector machine; θ represents the kernel function parameter of the support vector machine; Optionally, the kernel function parameters and penalty coefficients of the support vector machine are updated as follows: Where η svm is the learning rate of the support vector machine; θ(t) is the parameter of the kernel function of the tth iteration, and θ(t+1) is the parameter of the kernel function of the t+1th iteration; C u (t) is the penalty coefficient of the support vector machine at the tth iteration, C u (t+1) is the penalty coefficient of the support vector machine at the t+1th iteration; L svm () is the loss function of the support vector machine, is the gradient of the kernel function parameters; is the gradient of the penalty coefficient of the support vector machine.

7. The method according to claim 1, characterized in that: The parameter update process of multi-branch group convolutional neural network is based on the chaotic mapping mechanism of nonlinear dynamics; Optionally, in the parameter updating process of the chaotic mapping mechanism based on nonlinear dynamics, a chaotic mapping function is used to generate a sequence with chaotic characteristics, and the calculation method is expressed as: Xcs(t+1)=ρ ch Xcs(t)(1-Xcs(t))+δ ch sin(πXcs(t)) Where Xcs(t) is the chaotic variable of the t-th iteration, Xcs(t+1) is the chaotic variable of the t+1-th iteration; ρ ch is the control parameter of the chaotic mapping; δ ch is the modulation parameter to control the complexity of the chaotic sequence; Optionally, the parameters of the multi-branch group convolutional neural network are updated based on the gradient information of the loss function using a back propagation algorithm based on chaotic variables; Optionally, the updating method of the parameters in the branch group convolutional neural network is expressed as: Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable of the tth iteration, G b (t), G b (t+1) represents the G of the tth and t+1th iterations respectively. b , G b represents the group convolution kernel of each branch, w b (t), w b (t+1) represents the w of the tth and t+1th iterations respectively. b , w b represents the weight vector of the bth branch; bc b (t), bc b (t+1) represents the bc of the tth and t+1th iterations respectively. b , bc b Represents the bias of the b-th branch when multi-branch features are fused.

8. A computer device, characterized in that: The device comprises: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images