CV-based Prediction Method and Device for the Recovery Status of Neck Scars after Thyroidectomy

Abstract

The present invention belongs to the field of intelligent medicine, and particularly relates to a method for predicting the recovery status of neck scars after thyroidectomy based on CV. The method includes obtaining postoperative scar image data of the subject to be tested; the postoperative scar image data is simultaneously input into each branch of the multi-branch group convolutional neural network to obtain scar image feature maps of each branch, the scar image feature maps of each branch are subjected to feature fusion to obtain multi-branch fused scar image features, and the multi-branch fused scar image features are converted through a fully connected layer to obtain the final scar image feature vector; the final scar image feature vector is input into a classifier to obtain a scar score classification, and the classification categories are the scores for the scar image data, and the higher the score, the higher the degree of scar hyperplasia. This application uses a multi-branch group convolutional neural network, and through multiple convolutional branches, it extracts scar image features of different scales and directions, enhancing the ability to capture features of complex scar images.

Description

Technical Field

[0001] The present invention relates to the field of intelligent medicine, and more specifically, to a method, device, medium, and program product for predicting the recovery status of neck scars after thyroidectomy based on CV. Background Art

[0002] The formation of postoperative scars is one of the common complications of many surgical operations. Despite the continuous progress of medical technology, the formation and hyperplasia of scars still trouble many patients. With the popularization of medical trauma repair surgeries, patients' attention to postoperative recovery, especially the appearance of scars, has increased day by day. The excessive hyperplasia of scars not only affects the appearance but may also lead to functional disorders, bringing physical and psychological burdens to patients. Therefore, how to evaluate the recovery status of scars, especially accurately evaluate the scar hyperplasia situation at different postoperative time periods, is an important topic in clinical practice.

[0003] Currently, the evaluation of scars mainly relies on the experience of clinicians and visual inspection, usually combined with a standardized scoring system. However, the manual evaluation method has problems such as strong subjectivity and inconsistent evaluation criteria, which easily lead to deviations and errors in evaluation results. In addition, traditional scoring systems are mostly based on two-dimensional images or direct observation, and cannot comprehensively reflect complex information such as the depth, texture, and pigmentation of scars. Therefore, it is difficult to provide sufficient basis for scar treatment and intervention.

[0004] The prior art has the following deficiencies:

[0005] 1. Traditional convolutional neural networks usually cannot effectively handle transformations such as rotation and translation in scar images, which results in insufficient robustness of their image features. Especially when dealing with scars with irregular shapes or complex textures, the features extracted by the model are often not precise enough, affecting the evaluation results of the scar recovery status.

[0006] 2. In the prior art, feature extraction is usually a fixed convolutional operation, lacking a mechanism for dynamic adjustment, resulting in insufficient performance of the model when dealing with multi-scale and multi-directional scar images and being difficult to capture key feature information at different scales.

[0007] 3. The parameter update method of traditional convolutional neural networks is usually linear, lacking the characteristics of non-linear dynamics, resulting in limited non-linear expression ability of the model when dealing with complex scar images, low training efficiency, and insufficient exploration ability for the scar image feature space.

[0008] 4. In the existing classification method based on support vector machines, the fixed allocation method of feature weights easily leads to the model being unable to distinguish important features from irrelevant features. Especially when dealing with the highly complex scar image feature vectors obtained through feature extraction, the model is prone to misjudgment or overfitting, and the accuracy and robustness of classification are limited. Summary of the Invention

[0009] In view of the above problems, the present invention provides a method for predicting the recovery status of neck scars after thyroidectomy based on CV, which uses computer vision (CV) to process image data and extract features to capture information in the image data, and improves the model of the prediction model, thereby constructing a prediction model for the recovery status of neck scars after thyroidectomy suitable for clinical use.

[0010] The present application (in the first aspect) discloses a method for predicting the recovery status of neck scars after thyroidectomy based on CV, including:[[]]

[0011] Obtaining postoperative scar image data of the person to be tested;

[0012] The postoperative scar image data is simultaneously input into each branch of the multi-branch convolutional neural network to obtain scar image feature maps of each branch. The scar image feature maps of each branch are fused through feature fusion to obtain multi-branch fused scar image features. The multi-branch fused scar image features are converted through a fully connected layer to obtain the final scar image feature vector;

[0013] The final scar image feature vector is input into a classifier to obtain a scar score classification. The classification category is the score of the scar image data. The higher the score, the higher the degree of scar hyperplasia.

[0014] Further, the scar image feature maps extracted by each branch are expressed as:

[0015] F b = Ress(T b )

[0016] where F b is the scar image feature map of the b-th branch, and T b represents the convolutional features extracted by the b-th branch; Ress() is an adaptive hybrid activation function;

[0017] Optionally, the adaptive hybrid activation function is expressed as:

[0018] F b = Ress(T b ) = α·tanh(T b ) + β·ReLU(T b ) + γ·ELU(T b )

[0019] where F b is the scar image feature map of the b-th branch, and T bDenote the convolutional features extracted by the b-th branch; Ress() is an adaptive hybrid activation function, α is the first learnable parameter of the adaptive hybrid activation function, β is the second learnable parameter of the adaptive hybrid activation function, γ is the third learnable parameter of the adaptive hybrid activation function, and α + β + γ = 1; tanh() is the hyperbolic tangent function; ReLU() is the rectified linear unit function; ELU() is the exponential linear unit function.

[0020] Further, the step of inputting the postoperative scar image data into each branch of the multi-branch convolutional neural network to obtain the scar image feature maps of each branch includes: the postoperative scar image data is convolved by the group convolutional kernels of each branch to obtain the extracted convolutional features, and the extracted convolutional features are mapped to obtain the scar image feature maps;

[0021] Optionally, the extracted convolutional features are expressed as:

[0022] T b =G b *X+bc b

[0023] where T b represents the convolutional features extracted by the b-th branch, G b is the group convolutional kernel of the group convolutional neural network of the b-th branch, X represents the postoperative scar image data, bc b represents the bias of the b-th branch during multi-branch feature fusion, and * represents the convolution operation;

[0024] Optionally, the group convolutional kernels of each branch are a linear combination of multiple sub-convolutional kernels;

[0025] Optionally, the group convolutional kernels of each branch are expressed as:

[0026]

[0027] In the formula, G b represents the group convolutional kernels of each branch, K is the number of sub-kernels of the group convolution of the b-th branch; g b,k is the k-th sub-convolutional kernel in the b-th branch; w b,k is the weight coefficient of the k-th sub-convolutional kernel in the b-th branch;

[0028] Optionally, in the group convolutional neural network of each branch, the convolution operation is expressed as:

[0029]

[0030] In the formula, is the convolution result of the n pr -th scale convolutional kernel in the b-th branch, npr denotes the scale of the convolutional kernel, i.e., the stride of the convolutional kernel; M is the size of the convolutional kernel; is the pixel value of the postoperative scar image data at position i center -m, where m represents the index within the scale change range of the convolutional kernel, and i center represents the index of the central pixel of the current convolution.

[0031] Furthermore, the calculation method of the final scar image feature vector representation is:

[0032] Z = Sig(ω FC FIN n + bc FC )

[0033] In the formula, Z is the final scar image feature vector of the postoperative scar image data; Sig() is the Sigmoid activation function; ω FC and bc FC are the weight and bias of the fully connected layer, respectively;

[0034] Optionally, the scar image feature representation of the multi-branch fusion is:

[0035] FIN = φ(F 1 , F 2 , …, F b , … F B )

[0036] where FIN represents the scar image feature of the multi-branch fusion; φ() represents the function of feature concatenation; F 1 is the scar image feature map of the first branch, and F b is the scar image feature map of the b-th branch, where b ∈ [1, B], and B represents the total number of branches of the multi-branch group convolutional neural network.

[0037] Furthermore, the parameters in the multi-branch group convolutional neural network are obtained through training, and the training process includes: collecting the postoperative scar image data sets of the patients in the test set;

[0038] annotating the postoperative scar image data sets,

[0039] successively inputting the set of the postoperative scar image data sequences into the multi-branch group convolutional neural network with initial parameters to extract the corresponding final scar image feature vectors;

[0040] obtaining the classification labels based on the classification of the final scar image feature vectors, calculating the scar image feature extraction loss based on the classification labels and the annotations of the postoperative scar image data sets, and training the parameters in the multi-branch group convolutional neural network by iteratively optimizing the scar image feature extraction loss;

[0041] Optionally, the calculation method of the scar image feature extraction loss is expressed as:

[0042]

[0043] In the formula, L n represents the scar image feature extraction loss of the nth sample, and Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output is the category corresponding to Z n , and Y n is the annotation of the nth sample in the postoperative scar image dataset; is the multi-class cross-entropy loss of the nth sample; λ1 is the weight coefficient of the first scar image feature extraction loss, λ2 is the weight coefficient of the second scar image feature extraction loss, and λ3 is the weight coefficient of the third scar image feature extraction loss.

[0044] Furthermore, the parameters are obtained by using reinforcement learning for optimization training. The policy function of the reinforcement learning is π(a b |s n ), and the policy function represents the probability of selecting the action a n in the state s b , and the action a b is the action for the bth branch, including adjusting the convolution kernel parameters and / or weights, and is expressed as:

[0045]

[0046] In the formula, s n is the environmental state of the nth sample, including the current scar image feature extraction result; a is all possible actions in the current environment;

[0047] The value function of the reinforcement learning task is expressed as:

[0048]

[0049] In the formula, V(s n ) is the value function, estimating the expected reward in the state s n ; R n is the immediate reward, reflecting the effect of scar image feature extraction; γ RL is the discount factor; is the expected function;

[0050] Preferably, the immediate reward of the reinforcement learning is calculated based on the accuracy of scar image feature extraction and the complexity of the model;

[0051] Preferably, the immediate reward calculation method of the reinforcement learning is expressed as:

[0052]

[0053] In the formula, μ RL is the first trade-off coefficient, and γ RL is the second trade-off coefficient; Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, which outputs the category corresponding to Z n , Y n is the annotation of the nth sample in the postoperative scar image dataset; || || represents the square of the L2 norm; w b is the weight vector of the bth branch;

[0054] Optionally, based on the iterative optimization of the scar image feature extraction loss and the reward calculation of the reinforcement learning, a comprehensive loss function is obtained, and the parameters in the multi-branch group convolution neural network are trained by iteratively optimizing the comprehensive loss function;

[0055] Optionally, the comprehensive loss function is expressed as:

[0056]

[0057] In the formula, Loss′ is the comprehensive loss function; L n represents the scar image feature extraction loss of the nth sample; λ p is the trade-off coefficient of the comprehensive loss function;

[0058] Optionally, the classifier is any one or several of the following: support vector machine, Logistic regression, convolutional network, ensemble learning;

[0059] Optionally, the classifier is obtained through training. The training process includes: collecting the postoperative scar image dataset of the patients in the test set; annotating the postoperative scar image dataset, and the set of the postoperative scar image data sequences is sequentially input into the multi-branch group convolution neural network to extract the corresponding multi-branch fused scar image features. The multi-branch fused scar image features are input into the initial classifier to obtain the predicted labels. Based on the predicted labels and the annotations of the postoperative scar image dataset, the classifier loss is calculated, and the classifier is trained by iteratively optimizing the classifier loss;

[0060] Optionally, the classifier uses a support vector machine,

[0061] Optionally, the classifier loss further includes a penalty term for the multi-branch fused scar image features;

[0062] Optionally, the classifier loss is expressed as:

[0063]

[0064] where y i represents the annotation score of the i-th sample input to the support vector machine, and Z i,j represents the j-th eigenvalue of the final scar image feature vector Z i,j of the i-th input image; w j is the feature weight of the j-th eigenvalue of the final scar image feature vector extracted by the multi-branch group convolutional neural network input to the support vector machine; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; max() is the maximum value function; f() is the classification decision function of the support vector machine, and C u is the penalty term coefficient of the support vector machine; θ represents the kernel function parameter of the support vector machine;

[0065] Optionally, the update method of the kernel function parameter and the penalty term coefficient of the support vector machine is:

[0066]

[0067] where η svm is the learning rate of the support vector machine; θ(t) is the parameter of the kernel function at the t-th iteration, and θ(t + 1) is the parameter of the kernel function at the (t + 1)-th iteration; C u (t) is the penalty coefficient of the support vector machine at the t-th iteration, and C u (t + 1) is the penalty coefficient of the support vector machine at the (t + 1)-th iteration; L svm () is the loss function of the support vector machine, is the gradient with respect to the parameter of the kernel function; is the gradient with respect to the penalty coefficient of the support vector machine.

[0068] Furthermore, the parameter update process of the multi-branch group convolutional neural network is based on the chaotic mapping mechanism of nonlinear dynamics;

[0069] Optionally, in the parameter update process based on the chaotic mapping mechanism of nonlinear dynamics, let the chaotic mapping function be used to generate a sequence with chaotic characteristics, and the calculation method is expressed as:

[0070] Xcs(t + 1) = ρ ch Xcs(t)(1 - Xcs(t)) + δ ch sin(πXcs(t))

[0071] where Xcs(t) is the chaotic variable at the t-th iteration, and Xcs(t + 1) is the chaotic variable at the (t + 1)-th iteration; ρch is the control parameter of the chaotic map; δ ch is the modulation parameter that controls the complexity of the chaotic sequence;

[0072] Optionally, based on the chaotic variable, use the backpropagation algorithm to update the parameters of the multi-branch group convolutional neural network according to the gradient information of the loss function;

[0073] Optionally, the update method is expressed as:

[0074]

[0075] where η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable at the t-th iteration, G b (t), G b (t + 1) respectively represent G at the t-th and (t + 1)-th iterations b , G b represents the group convolution kernels of each branch, w b (t), w b (t + 1) respectively represent w at the t-th and (t + 1)-th iterations b , w b represents the weight vector of the b-th branch; bc b (t), bc b (t + 1) respectively represent bc at the t-th and (t + 1)-th iterations b , bc b represents the bias of the b-th branch during multi-branch feature fusion.

[0076] The second aspect of this application discloses a postoperative scar recovery status prediction system based on computer vision, including:

[0077] An acquisition module: used to acquire the postoperative scar image data of the person to be tested;

[0078] A feature extraction module: used to input the postoperative scar image data of the person to be tested into each branch of the multi-branch convolutional neural network to obtain the scar image feature maps of each branch, the scar image feature maps of each branch are fused through feature fusion to obtain the multi-branch fused scar image features, and the multi-branch fused scar image features are converted through a fully connected layer to obtain the final scar image feature vector;

[0079] A prediction module: input the final scar image feature vector into a classifier to obtain a scar score classification, and the classification category is the score of the scar image data. The higher the score, the higher the degree of scar hyperplasia.

[0080] A third aspect of the present application discloses a computer device, the device comprising: a memory and a processor; the memory is used for storing program instructions; the processor is used for calling the program instructions, and when the program instructions are executed, it is used for executing the steps of the above-mentioned method.

[0081] A fourth aspect of the present application discloses a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-mentioned method.

[0082] A fifth aspect of the present application discloses a computer program product, comprising a computer program, and when the computer program is executed by a processor, it implements the steps of the above-mentioned method.

[0083] The present application has the following beneficial effects:

[0084] 1. By adopting a multi-branch group convolutional neural network, through group convolution operations, the model can maintain equivariance under image transformations such as rotation and translation, improving the robustness of the model to scar image features. Through multiple convolutional branches, scar image features of different scales and directions are extracted, enhancing the ability to capture features of complex scar images.

[0085] 2. By adopting a reinforcement learning strategy to adjust the weights for scar image feature extraction, the agent dynamically adjusts the parameters of each branch in the multi-branch network by learning the optimal strategy, enabling the model to adaptively focus on the most discriminative image features and enhancing the model's ability to extract features of complex images.

[0086] 3. By adopting a chaotic mapping mechanism, the chaotic theory is used to improve the non-linear expression ability and exploration ability of the model during the parameter update process of the convolutional neural network, enhancing the capture of complex scar features and improving the training efficiency and model generalization ability.

[0087] 4. By adopting a support vector machine algorithm with a penalty term to enhance the sensitivity to feature selection, combined with the complexity of scar image features, the penalty term effectively prevents the model from overfitting, and at the same time adaptively adjusts the feature weights, enabling the model to more accurately perform scar score classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0089] Figure 1 It is a schematic flowchart of the method provided in the first aspect of the embodiment of the present invention;

[0090] Figure 2 It is a schematic diagram of a program product provided in the second aspect of the embodiment of the present invention;

[0091] Figure 3 It is a schematic diagram of a computer device provided in the embodiment of the present invention;

[0092] Figure 4 It is a schematic diagram of the architecture of an exemplary computing device provided in the embodiment of the present invention;

[0093] Figure 5 It is a schematic diagram of a storage medium provided in the embodiment of the present invention;

[0094] Figure 6 It is a schematic diagram of an image annotation provided in the embodiment of the present invention;

[0095] Figure 7 It is a schematic diagram of an image annotation provided in the embodiment of the present invention;

[0096] Figure 8 It is a data flow diagram for inputting postoperative scar image data to obtain predicted labels provided in the embodiment of the present invention. Detailed implementation manners

[0097] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0098] In some processes described in the specification, claims and above-mentioned drawings of the present invention, there are multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order in which they appear in this article or may be executed in parallel. The serial numbers of the operations, such as S101, S102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions such as "first" and "second" in this article are used to distinguish different messages, devices, modules, etc., and do not represent a sequence, nor do they limit that "first" and "second" are of different types.

[0099] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0100] Figure 1It is a schematic flowchart of a method for predicting the recovery status of neck scars after thyroidectomy based on evaluation based on CV provided by an embodiment of the present invention. Specifically, the method includes the following steps:

[0101] S101: Obtain the postoperative scar image data of the person to be tested;

[0102] S102: Input the postoperative scar image data into a multi-branch group convolutional neural network to extract the final scar image feature vector; the postoperative scar image data of the person to be tested is simultaneously input into each branch of the multi-branch convolutional neural network to obtain the scar image feature maps of each branch, the scar image feature maps of each branch are fused through feature fusion to obtain the multi-branch fused scar image features, and the multi-branch fused scar image features are converted through a fully connected layer to obtain the final scar image feature vector:

[0103] S103: Input the final scar image feature vector into a classifier to obtain a scar score classification. The classification categories are the scores of the scar image data, ranging from 1 to 10, corresponding to 10 integer category labels. The higher the score, the higher the degree of scar hyperplasia.

[0104] This application proposes a method for evaluating the recovery status of postoperative scars based on computer vision algorithms, and uses a scar recovery status evaluation model to achieve scar recovery status evaluation. The scar recovery status evaluation model is obtained through training. Specifically, the training data is collected from clinical trials. Specifically, data is obtained from three experimental groups: a control group (only applying Mederma), a photoelectric group (applying Mederma + photoelectric treatment), and a botulinum toxin group (applying Mederma + injecting botulinum toxin);

[0105] The collection method is to take 3D photos and dermoscopic photos of each patient at 2 weeks, 1 month, 3 months, and 6 months after surgery;

[0106] After the collected data is labeled, it is stored in digital format, specifically including image data and text scores. The text scores are obtained by manually evaluating each scar image data, ranging from 1 to 10. The higher the score, the higher the degree of scar hyperplasia ( Figure 6 、 Figure 7 ); The storage format of the scar image data is JPEG, with a resolution of 1024×768 pixels.

[0107] The scar recovery status evaluation model is a multi-branch group convolutional neural network, and the model is trained through supervised training;

[0108] Group Equivariant Convolutional Neural Networks (abbreviated as G-CNNs) is a network structure that extends traditional convolutional neural networks. By utilizing concepts in group theory, the network has equivariance under specific transformations (such as rotation, translation, etc.). The equivariance means that the output of the network remains unchanged for specific transformations of the input, thereby improving the robustness of the network to these transformations;

[0109] The multi-branch group convolutional neural network is multiple parallel convolutional branches ( Figure 8 as shown), and each branch extracts scar image feature information at different scales and directions of the scar image data through group convolution operations. In addition, a hierarchical strategy is used to dynamically adjust the weights and parameters of each branch, enabling the model to adaptively focus on the most discriminative scar image features, thereby improving the accuracy and robustness of scar image feature extraction.

[0110] Specifically, the training process of the multi-branch group convolutional neural network algorithm is as follows:

[0111] 1. Set the structure of the multi-branch group convolutional neural network, including the number of branches, the size and number of group convolutional kernels of each branch, and the configuration of the fully connected layer. Specifically, let there be a total of B convolutional branches, the group convolutional kernel of the b-th branch of the multi-branch group convolutional neural network be G b , the weight of the fully connected layer of the b-th branch of the multi-branch group convolutional neural network be W b , the bias of the fully connected layer of the b-th branch of the multi-branch group convolutional neural network be bc b , and b = 1, 2,..., B.

[0112] 2. Input the scar image data into the model, and each image is simultaneously input into all convolutional branches of the multi-branch group convolutional neural network. Specifically, let the n-th scar image data be represented as X n , and n = 1, 2,..., N, where N is the total number of training samples.

[0113] 3. Each convolutional branch of the multi-branch group convolutional neural network extracts scar image features at different levels and scales through the group convolutional layer, activation function, and pooling layer structure. Through group convolution operations, local and global information of the image can be captured. Specifically, for the scar image data, the way to extract scar image features through each convolutional branch is expressed as:

[0114] F b,n = Ress(G b *X n +bc b )

[0115] In the formula, F n,bThe scar image feature map extracted for the nth sample in the bth branch; Ress() is an adaptive hybrid activation function; * represents the convolution operation.

[0116] Furthermore, the group convolution kernel is a linear combination of multiple sub-convolution kernels, used to capture scar image features at different scales and directions, and the implementation method is expressed as:

[0117]

[0118] In the formula, K is the number of sub-kernels of the group convolution; w b,k is the weight coefficient of the kth sub-convolution kernel in the bth branch, obtained through training; g b,k is the kth sub-convolution kernel in the bth branch, and its initialization can adopt different filters (such as Gaussian kernel, Laplace kernel, etc.) to capture different scar image feature patterns.

[0119] Furthermore, in the multi-branch group convolution neural network, the calculation method of the convolution operation process is expressed as:

[0120]

[0121] In the formula, is the convolution result of the nth pr scale convolution kernel in the bth branch, n pr represents the scale of the convolution kernel, that is, the stride of the convolution kernel; M is the size of the convolution kernel; is the pixel value of the input scar image at position i center -m, m represents the index within the scale change range of the convolution kernel, i center represents the index of the central pixel of the current convolution.

[0122] Furthermore, the purpose of the adaptive hybrid activation function is to enhance the non-linear expression ability, and the calculation method is expressed as:

[0123] Ress(x p ) = α p ·tanh(x p ) + β p ·ReLU(x p ) + γ p ·ELU(x p )

[0124] In the formula, x p is the input value of the adaptive hybrid activation function; α p is the first learnable parameter of the adaptive hybrid activation function, β p is the second learnable parameter of the adaptive hybrid activation function, γ p The third learnable parameter of the adaptive hybrid activation function, and satisfies αp +β p +γ p = 1; tanh() is the hyperbolic tangent function; ReLU() is the rectified linear unit function; ELU() is the exponential linear unit function.

[0125] 4. The scar image features extracted from each branch are fused. The fusion methods can be methods such as scar image feature splicing, weighted summation, feature concatenation, etc., to form a comprehensive scar image feature representation, expressed as:

[0126] FIN n = φ(F 1,n , F 2,n , …, F 3,n , …F b,n )

[0127] In the formula, FIN n is the scar image feature of the multi-branch fusion of the nth sample; φ() represents the feature fusion function; F 1,n is the scar image feature mapping of the first branch of the nth training set sample, and F b,n is the scar image feature mapping of the bth branch of the nth training set sample, where b ∈ [1, B], B represents the total number of branches of the multi-branch group convolutional neural network, and n ∈ [1, N], N represents the total number of postoperative scar images input in the training set.

[0128] 5. Using the hierarchical strategy of reinforcement learning, the agent dynamically adjusts the weights and parameters of each branch according to the current effect of scar image feature extraction. The agent observes the results of scar image feature extraction and a preset reward function, and updates the policy to optimize the model performance;

[0129] The agent is an entity that can take actions in a specific environment in reinforcement learning to maximize a certain cumulative reward. The goal of the agent is to learn a policy that can tell it which action to take in a given state to maximize the long-term cumulative reward.

[0130] Specifically, define the policy function as π(a b |s n ) The policy function represents the probability of selecting action a n in state s b , and action a b is the action for the bth branch, which may include adjusting convolution kernel parameters, weights, etc., expressed as:

[0131]

[0132] In the formula, s nThe environmental state of the nth sample, including the current scar image feature extraction result; a is all possible actions in the current environment;

[0133] The value function of the reinforcement learning task is expressed as:

[0134]

[0135] In the formula, s n is the environmental state of the nth sample, including the current scar image feature extraction result.

[0136] Furthermore, according to the reinforcement learning strategy, the value function of the feature extraction task is:

[0137]

[0138] In the formula, v(s n ) is the value function, estimating the expected reward in state s n ; R n is the immediate reward, reflecting the effect of scar image feature extraction; γ RL is the discount factor; is the expected function;

[0139] In one embodiment, γ p is set to 0.3.

[0140] In one embodiment, the immediate reward of the reinforcement learning takes into account the accuracy of scar image feature extraction and the complexity of the model, and the calculation method is expressed as:

[0141]

[0142] In the formula, μ RL is the first trade-off coefficient, μ RL is the second trade-off coefficient; Z n is the final scar image feature representation of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, outputting the category corresponding to Z n , Y n is the annotation of the nth sample in the postoperative scar image dataset; || ||∥ represents the square of the L2 norm; w b is the weight vector of the b p th branch, composed of all w b,k ;

[0143] Optionally, based on iterative optimization of the scar image feature extraction loss and the reward of the reinforcement learning, a comprehensive loss function is calculated, and the parameters in the multi-branch group convolutional neural network are trained by iterative optimization of the comprehensive loss function;

[0144] Furthermore, when updating the policy function, its parameters are updated using the entropy-regularized policy gradient method to promote exploration of the policy, which is expressed as:

[0145]

[0146] In the formula, θ RL is the parameter of the policy function π(a|s n ); is the gradient with respect to the parameter of the policy function π(a|s n ); ← represents the parameter update operation; η RL is the learning rate of the policy function; κ RL is the entropy regularization coefficient; HE(π) is the entropy of the policy. Preferably, κ RL is set to 0.2. Preferably, η RL is set to 0.001.

[0147] Furthermore, the calculation method of the entropy of the policy is expressed as:

[0148]

[0149] In the formula, a is the action of all branches, representing the feature extraction operation of each convolutional branch of the multi-branch group convolutional neural network.

[0150] 6. The fused scar image features are passed through a fully connected layer to obtain the final scar image feature vector, which is expressed as:

[0151] Z n = Sig(ω FC FIN n + bc FC )

[0152] In the formula, Z n is the final scar image feature vector of the n p th sample; Sig() is the Sigmoid activation function; and are the weight and bias of the feature fusion fully connected layer respectively.

[0153] 7. Define a comprehensive loss function, which is used to evaluate the overall performance of the model, including the scar image feature extraction error and the reward term of reinforcement learning. The calculation method is expressed as:

[0154] In one embodiment, λ R is set to 0.3.

[0155] In one embodiment, the calculation method of the scar image feature extraction loss is expressed as:

[0156]

[0157] In the formula, L n represents the scar image feature extraction loss of the nth sample, and Z n is the final scar image feature representation of the nth sample in the postoperative scar image dataset; the Softmax() function is the activation function, and the output Z n corresponds to the category, and Y n is the annotation of the nth sample in the postoperative scar image dataset; is the cross-entropy loss of the nth p sample; λ1 is the weight coefficient of the first scar image feature extraction loss, λ2 is the weight coefficient of the second scar image feature extraction loss, and λ3 is the weight coefficient of the third scar image feature extraction loss.

[0158] Optionally, the comprehensive loss function is expressed as:

[0159]

[0160] In the formula, Loss′ is the comprehensive loss function; L n represents the scar image feature extraction loss of the nth sample; λ R is the trade-off coefficient of the comprehensive loss function;

[0161] 8. In each convolutional branch of the multi-branch group convolutional neural network, based on the chaotic mapping mechanism of nonlinear dynamics, chaos theory is applied to the parameter update process of the multi-branch group convolutional neural network to enhance the nonlinear expression ability and training efficiency of the model, make the convolutional kernel parameters show a chaotic distribution, increase the exploration ability of the scar image feature space, and improve the capture effect of the scar image features of complex scars. Specifically, let the chaotic mapping function be used to generate a sequence with chaotic characteristics, and the calculation method is expressed as:

[0162] Xcs(t + 1) = ρ ch Xcs(t)(1 - Xcs(t)) + δ ch sin(πXcs(t))

[0163] In the formula, Xcs(t) is the chaotic variable of the tth iteration, and Xcs(t + 1) is the chaotic variable of the (t + 1)th iteration; ρ ch is the control parameter of the chaotic mapping, satisfying 3.5699456 < ρ ch ≤ 4; δ ch is the modulation parameter, which controls the complexity of the chaotic sequence.

[0164] 9. Based on the chaotic variable, use the backpropagation algorithm to update the parameters of the multi-branch group convolutional neural network according to the gradient information of the loss function. Specifically, the update method is expressed as:

[0165]

[0166] Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable at the t-th iteration, G b (t), G b (t + 1) respectively represent G at the t-th and (t + 1)-th iterations b , G b represents the group convolution kernel of each branch, W b (t), W b (t + 1) respectively represent W at the t-th and (t + 1)-th iterations b , W b represents the weight vector of the b-th branch; bc b (t), bc b (t + 1) respectively represent bc at the t-th and (t + 1)-th iterations b , bc b represents the bias of the b-th branch during multi-branch feature fusion;

[0167] Preferably, η sep is set to 0.004.

[0168] In some embodiments, the update method is expressed as:

[0169]

[0170] Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable at the t-th iteration, G b (t), G b (t + 1) respectively represent G at the t-th and (t + 1)-th iterations b , G b represents the group convolution kernel of each branch, W b (t), W b (t + 1) respectively represent W at the t-th and (t + 1)-th iterations b , W b represents the weight vector of the b-th branch; bc b (t), bc b (t + 1) respectively represent bc at the t-th and (t + 1)-th iterations b , bc b represents the bias of the b-th branch during multi-branch feature fusion.

[0171] 10. Repeat and iterate the above steps until the preset iteration stop condition is met, which indicates that the model training is completed. In one embodiment, the preset iteration stop condition is reaching the preset maximum number of iterations or the change curve of the comprehensive loss function reaches the convergence state. Preferably, the preset maximum number of iterations is set to 1000 times.

[0172] Further, the scar image features extracted by the multi-branch group convolutional neural network are input into a classifier for training the classifier model. The classifier model is used to obtain the scar image data score output according to the scar image features extracted by the multi-branch group convolutional neural network, with a range of 1-10, corresponding to 10 integer category labels. The higher the score, the higher the degree of scar hyperplasia.

[0173] The classifier model adopts a support vector machine algorithm based on a penalty term. On the basis of the support vector machine algorithm, a penalty term for the scar image feature vector extracted by the multi-branch group convolutional neural network is adopted to enhance the sensitivity of the model to feature selection, thereby improving the classification accuracy.

[0174] Specifically, the training process of the support vector machine algorithm based on the penalty term is as follows:

[0175] 1. Initialize the kernel function parameters and penalty term coefficients of the support vector machine. Set the kernel function parameter of the support vector machine as θ, and the penalty coefficient of the support vector machine as C u , and the initialization method is expressed as:

[0176] θ = randn(0,1)·σ θ

[0177]

[0178] In the formula, randn(0,1) represents the uniform distribution in the interval (0,1); θ represents the parameter of the kernel function; σ θ is the initial standard deviation of the kernel parameter; C u is the penalty coefficient of the support vector machine, which is initialized by the reciprocal of the variance of the target variable Y u ; Y u is the target variable, which represents the category vector of the sample labels of the scar image features extracted by the multi-branch group convolutional neural network input into the support vector machine; var() is the variance function.

[0179] 2. Let the scar image feature vector extracted by the multi-branch group convolutional neural network be Z. Use the gradient descent method to optimize the parameters of the support vector machine. That is, in the process of training the support vector machine in each iteration, the loss of the current support vector machine model will be calculated, and the kernel function parameters and penalty term coefficients will be updated according to the loss. The update method is expressed as:

[0180]

[0181] Where η svm is the learning rate of the support vector machine; θ(t) is the parameter of the kernel function at the t-th iteration, and θ(t + 1) is the parameter of the kernel function at the (t + 1)-th iteration; C u (t) is the penalty coefficient of the support vector machine at the t-th iteration, and C u (t + 1) is the penalty coefficient of the support vector machine at the (t + 1)-th iteration; L svm () is the loss function of the support vector machine, is the gradient with respect to the parameter of the kernel function; is the gradient with respect to the penalty coefficient of the support vector machine. Preferably, η svm is set to 0.01.

[0182] In one embodiment, the loss function of the support vector machine adopts a loss calculation method with a penalty term, which is expressed as:

[0183]

[0184] Where,, y i represents the annotation score of the i-th sample input to the support vector machine, and Z i,j represents the j-th eigenvalue of the final scar image feature vector Z i,j of the i-th input image; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; w j is the feature weight of the j-th eigenvalue of the final scar image feature vector extracted by the multi-branch group convolutional neural network input to the support vector machine; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; max() is the maximum value function; f() is the classification decision function of the support vector machine, and C u is the penalty term coefficient of the support vector machine; θ represents the kernel function parameter of the support vector machine.

[0185] 3. Adopt an adaptive feature weight adjustment mechanism to dynamically adjust the weight of each feature according to its contribution in the classification task, enhance the sensitivity of the model to key features, and improve the classification accuracy. Specifically, for the feature weight of the eigenvalue of the feature vector of the scar image feature sample extracted by a single multi-branch group convolutional neural network input to the support vector machine, at the beginning of model training, a weight is assigned to each feature, initialized to 1, indicating that all features are considered equally important in the initial state, which is expressed as:

[0186]

[0187] Where It is the feature weight of the j-th eigenvalue of the feature vector of the scar image feature sample extracted by the initial input to the single multi-branch group convolutional neural network of the support vector machine.

[0188] Furthermore, the adjustment of the feature weight is based on the contribution degree of the feature to the classification boundary of the model, and a gradient-based method is used to update the weight, which is expressed as:

[0189]

[0190] In the formula, η tz is the learning rate for updating the feature weight; w j (t + 1) is the feature weight of the j-th eigenvalue of the feature vector of the scar image feature sample extracted by the single multi-branch group convolutional neural network input to the support vector machine at the (t + 1)-th iteration.

[0191] 4. Repeat the above steps iteratively until the preset iteration stop condition is satisfied, which means the model training is completed. In one embodiment, the preset iteration stop condition is to reach the preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0192] After the multi-branch group convolutional neural network and the penalty-based support vector machine are trained, the trained multi-branch group convolutional neural network and the penalty-based support vector machine are used for feature extraction of scar images and evaluation of scar recovery status;

[0193] In one embodiment, the collected original scar image data is input into the trained multi-branch group convolutional neural network for feature extraction. Further, the extracted scar image features are input into the penalty-based support vector machine for classification, and then the classification result is obtained. In this embodiment, the classification categories are scores for the scar image data, ranging from 1 to 10, corresponding to 10 integer category labels. The higher the score, the higher the degree of scar hyperplasia.

[0194] Note: The training set is used during the model training process. When the trained model is used, the image to be evaluated is input. In the above embodiments, when the superscript in the expression includes n, it represents the n-th image or the n-th sample input during training. For example:

[0195] The image data X represents the image input when using the model, and X n represents the n-th sample during training.

[0196] F 1 represents the scar image feature mapping of the first branch corresponding to the input image when the model is applied, and F 1,n represents the scar image feature mapping of the first branch corresponding to the n-th sample during training.

[0197] Figure 3 is a schematic diagram of a computer device provided by an embodiment of the present invention. As Figure 3 shown, the device may include: one or more processors, and one or more memories; wherein, computer-readable code is stored in the memory, and when the computer-readable code is run by the one or more processors, the methods described above can be executed.

[0198] The processor in this embodiment may be an integrated circuit chip with signal processing capabilities. The above-mentioned processor may be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, operations and logic block diagrams disclosed in the embodiments of the present disclosure. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc., and may be of the X86 architecture or the ARM architecture.

[0199] Generally speaking, the various exemplary embodiments of the present disclosure may be implemented in hardware or special circuits, software, firmware, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that can be executed by a controller, a microprocessor or other computing devices. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts or using some other graphical representation, it will be understood that the blocks, devices, systems, technologies or methods described herein may be implemented as non-limiting examples in hardware, software, firmware, special circuits or logic, general hardware or controllers or other computing devices, or some combination thereof.

[0200] For example, the method or device according to the embodiments of the present disclosure may also be implemented by means of Figure 4 the architecture of the computing device 3000 shown. As Figure 4 shown, the computing device 3000 may include a bus 3010, one or more CPUs 3020, a read-only memory (ROM) 3030, a random access memory (RAM) 3040, a communication port 3050 connected to a network, an input / output component 3060, a hard disk 3070, etc. The storage device in the computing device 3000, such as the ROM 3030 or the hard disk 3070, may store various data or files used for the processing and / or communication of the methods provided by the present disclosure and the program instructions executed by the CPU. The computing device 3000 may also include a user interface 3080. Of course, Figure 4 the architecture shown is only exemplary, and when implementing different devices, one or more components shown in the Figure 4 computing device may be omitted according to actual needs.

[0201] An embodiment of the present invention also provides a computer-readable storage medium, as Figure 5 shown, which is a schematic diagram of the storage medium provided by the embodiment of the present invention. Computer-readable instructions 4010 are stored on the computer storage medium 4020. When the computer-readable instructions 4010 are run by a processor, the method according to the embodiments of the present disclosure described with reference to the above drawings can be executed. The computer-readable storage medium in the embodiments of the present disclosure may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus random access memory (DR RAM). It should be noted that the memories of the methods described herein are intended to include, but are not limited to, these and any other suitable types of memories. It should be noted that the memories of the methods described herein are intended to include, but are not limited to, these and any other suitable types of memories.

[0202] An embodiment of the present disclosure also provides a computer program product or a computer program, which, when executed by a processor, implements the steps of the above method, as Figure 2 shown, the computer program product or the computer program includes:

[0203] An acquisition module 201: configured to acquire postoperative scar image data of a subject to be measured;

[0204] A feature extraction module 202: configured to input the postoperative scar image data of the subject to be measured into each branch of the multi-branch convolutional neural network to obtain scar image feature maps of each branch, fuse the scar image feature maps of each branch to obtain a multi-branch fused scar image feature, and convert the multi-branch fused scar image feature through a fully connected layer to obtain the final scar image feature vector;

[0205] A prediction module 203: configured to input the final scar image feature vector into a classifier to obtain a scar score classification, and the classification category is the score of the scar image data, and the higher the score, the higher the degree of scar hyperplasia.

[0206] It should be noted that the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, as well as the combination of blocks in the block diagram and / or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0207] In general, the various example embodiments of the present disclosure can be implemented in hardware or dedicated circuits, software, firmware, logic, or any combination thereof. Some aspects can be implemented in hardware, while other aspects can be implemented in firmware or software that can be executed by a controller, a microprocessor, or other computing devices. When aspects of the embodiments of the present disclosure are illustrated or described as block diagrams, flowcharts, or using some other graphical representation, it will be understood that the blocks, devices, systems, techniques, or methods described herein can be implemented as non-limiting examples in hardware, software, firmware, dedicated circuits or logic, general hardware or controllers or other computing devices, or some combination thereof.

[0208] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein.

[0209] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings, direct couplings, or communication connections shown or discussed with each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can be in electrical, mechanical, or other forms.

[0210] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed across multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0211] In addition, each functional unit in various embodiments of the present invention may be integrated in a processing unit, may exist separately as individual physical units, or two or more units may be integrated in one unit. The above integrated units may be implemented in the form of hardware or in the form of software functional units.

[0212] The exemplary embodiments of the present disclosure described in detail above are merely illustrative and not restrictive. Those skilled in the art should understand that various modifications and combinations can be made to these embodiments or their features without departing from the principles and spirit of the present disclosure, and such modifications should fall within the scope of the present disclosure.

Claims

1. A method for predicting the recovery status of neck scars after thyroidectomy based on CV, characterized in that, The method includes: obtaining postoperative scar image data of a person to be tested; The postoperative scar image data is simultaneously input into each branch of a multi-branch group convolutional neural network to obtain scar image feature maps of each branch. The scar image feature maps of each branch are subjected to feature fusion to obtain multi-branch fused scar image features. The multi-branch fused scar image features are converted through a fully connected layer to obtain a final scar image feature vector. The postoperative scar image data is convolved through the group convolutional kernels of each branch to obtain extracted convolutional features, and the extracted convolutional features are mapped to obtain the scar image feature maps; The extracted convolutional features are expressed as: T b = G b * X + bc b Among them, T b represents the convolutional features extracted by the b-th branch, G b is the group convolution kernel of the group convolutional neural network of the b-th branch, X represents the postoperative scar image data, bc b represents the bias of the b-th branch during multi-branch feature fusion, and * represents the convolution operation; The group convolutional kernels of each branch are linear combinations of multiple sub-convolutional kernels; The group convolutional kernels of each branch are expressed as: where G b represents the group convolution kernel of each branch, K is the number of sub-kernels of the group convolution of the b-th branch; g b,k is the k-th sub-convolution kernel in the b-th branch; w b,k is the weight coefficient of the k-th sub-convolution kernel in the b-th branch; Optionally, in the group convolutional neural network of each branch, the convolution operation is expressed as: In the formula, is the convolution result of the n pr -th scale convolution kernel in the b-th branch, where n pr represents the scale of the convolution kernel, i.e., the stride of the convolution kernel; M is the size of the convolution kernel; is the pixel value of the postoperative scar image data at position i center -m, where m represents the index within the scale change range of the convolution kernel, and i center represents the index of the central pixel of the current convolution; The scar image feature maps extracted by each branch are expressed as: F b = Ress(T b ) Among them, F b is the scar image feature map of the b-th branch, and T b represents the convolutional features extracted by the b-th branch; Ress( ) is an adaptive hybrid activation function; The adaptive hybrid activation function is expressed as: F b = Ress(T b ) = α·tanh(T b ) + β·ReLU(T b ) + γ·ELU(T b ) Among them, F b is the scar image feature map of the b-th branch, and T b represents the convolutional features extracted by the b-th branch; Ress( ) is an adaptive hybrid activation function, α is the first learnable parameter of the adaptive hybrid activation function, β is the second learnable parameter of the adaptive hybrid activation function, γ is the third learnable parameter of the adaptive hybrid activation function, and α + β + γ = 1; tanh( ) is the hyperbolic tangent function; ReLU( ) is the rectified linear unit function; ELU( ) is the exponential linear unit function; the scar image feature represented by the multi-branch fusion is: FIN = φ(F 1 , F 2 , …, F b , … F B ) Among them, FIN represents the scar image features of multi-branch fusion; φ( ) represents the function of feature concatenation; F 1 is the scar image feature mapping of the first branch, F b is the scar image feature mapping of the b-th branch, where b ∈ [1, B], and B represents the total number of branches of the multi-branch group convolutional neural network; The final scar image feature vector is input into a classifier to obtain a scar score classification. The classification categories are scores for the scar image data, and the higher the score, the higher the degree of scar hyperplasia.

2. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The calculation method of the final scar image feature vector is: Z = Sig(ω FC FIN n +bc FC ) Wherein, Z is the final scar image feature vector of the postoperative scar image data; Sig( ) is the Sigmoid activation function; ω FC and bc FC are the weight and bias of the fully connected layer respectively, and n represents the nth image or the nth sample input during training.

3. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The parameters in the multi-branch group convolutional neural network are obtained through training. The training process includes: collecting a postoperative scar image dataset of patients in a test set; Annotating the postoperative scar image dataset, The postoperative scar image dataset is input into the multi-branch group convolutional neural network with initial parameters to extract the corresponding final scar image feature vector; After classification based on the final scar image feature vector, a classification label is obtained. Based on the classification label and the annotation of the postoperative scar image dataset, a scar image feature extraction loss is calculated. The parameters in the multi-branch group convolutional neural network are trained by iteratively optimizing the scar image feature extraction loss.

4. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The calculation method of the scar image feature extraction loss is expressed as: Where N is the total number of training samples, and L n represents the scar image feature extraction loss of the nth sample, and Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax( ) function is the activation function, and the output Z n corresponds to the category, and Y n is the annotation of the nth sample in the postoperative scar image dataset; is the multi-class cross-entropy loss of the nth sample; λ1 is the weight coefficient of the first scar image feature extraction loss, λ2 is the weight coefficient of the second scar image feature extraction loss, and λ3 is the weight coefficient of the third scar image feature extraction loss.

5. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, characterized in that, Optimize and train to obtain the parameters in the multi-branch group convolutional neural network using reinforcement learning, and the policy function of the reinforcement learning is π(a|s n ), the policy function represents the probability of selecting action a in state s n , and action a b is the action for the b-th branch, including adjusting the convolutional kernel parameters and / or weights, and a is the action for all branches, which is expressed as: where s n is the environmental state of the nth sample, including the current scar image feature extraction result; a is the action of all branches; The value function of the reinforcement learning task is expressed as: where, V(s n ) is the value function, estimating the expected reward in state s n ; R n is the immediate reward, reflecting the effect of scar image feature extraction; γ RL is the discount factor; is the expected function.

6. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 5, wherein The immediate reward of the reinforcement learning is calculated based on the accuracy of scar image feature extraction and the complexity of the model.

7. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 6, characterized in that, The calculation method of the immediate reward of the reinforcement learning is expressed as: where μ RL is the first trade-off coefficient, and γ RL is the second trade-off coefficient; Z n is the final scar image feature vector of the nth sample in the postoperative scar image dataset; the Softmax( ) function is the activation function, which outputs the category corresponding to Z n , and Y n is the annotation of the nth sample in the postoperative scar image dataset; || || represents the square of the L2 norm; w b is the weight vector of the bth branch, and B represents the total number of branches of the multi-branch group convolutional neural network.

8. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 6, wherein Based on iteratively optimizing the scar image feature extraction loss and the reward of the reinforcement learning, a comprehensive loss function is calculated. The parameters in the multi-branch group convolutional neural network are trained by iteratively optimizing the comprehensive loss function.

9. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 8, characterized in that, The comprehensive loss function is expressed as: where Loss′ is the comprehensive loss function; L n represents the loss of scar image feature extraction for the nth sample; λ p is the trade-off coefficient of the comprehensive loss function.

10. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein, The classifier is any one or more of the following: support vector machine, Logistic regression, convolutional network, ensemble learning.

11. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, characterized in that, The classifier is obtained through training. The training process includes: collecting the postoperative scar image dataset of the patients in the test set; annotating the postoperative scar image dataset, sequentially inputting the postoperative scar image dataset into a multi-branch group convolutional neural network to extract the corresponding multi-branch fused scar image features, inputting the multi-branch fused scar image features into an initial classifier to obtain the predicted labels, calculating the classifier loss based on the predicted labels and the annotations of the postoperative scar image dataset, and obtaining the classifier by iteratively optimizing the classifier loss.

12. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein, The classifier adopts a support vector machine.

13. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The classifier loss further includes a penalty term for the multi-branch fused scar image features.

14. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The classifier loss is expressed as: where y i represents the annotation score of the i-th sample input to the support vector machine, and Z i,j represents the j-th eigenvalue of the final scar image feature vector Z i of the i-th input image; w j is the feature weight of the j-th eigenvalue of the final scar image feature vector extracted by the multi-branch group convolutional neural network input to the support vector machine; d is the dimension of the scar image feature sample extracted by a single multi-branch group convolutional neural network; max( ) is the maximum value function; f( ) is the classification decision function of the support vector machine, and C u is the penalty term coefficient of the support vector machine; θ represents the kernel function parameter of the support vector machine.

15. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 14, wherein The update methods for the kernel function parameter and the penalty term coefficient of the support vector machine are: Where, η svm is the learning rate of the support vector machine; θ(t) is the parameter of the kernel function at the t-th iteration, and θ(t + 1) is the parameter of the kernel function at the (t + 1)-th iteration; C u (t) is the penalty coefficient of the support vector machine at the t-th iteration, and C u (t + 1) is the penalty coefficient of the support vector machine at the (t + 1)-th iteration; L svm () is the loss function of the support vector machine, is the gradient with respect to the parameter of the kernel function; is the gradient with respect to the penalty coefficient of the support vector machine.

16. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The parameter update process of the multi-branch group convolutional neural network is based on the chaotic mapping mechanism of nonlinear dynamics.

17. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 16, wherein In the parameter update process based on the chaotic mapping mechanism of nonlinear dynamics, assume that the chaotic mapping function is used to generate a sequence with chaotic characteristics, and the calculation method is expressed as: Xcs(t + 1) = ρ ch Xcs(t)(1 - Xcs(t)) + δ ch sin(πXcs(t)) Wherein, Xcs(t) is the chaotic variable of the t-th iteration, and Xcs(t + 1) is the chaotic variable of the (t + 1)-th iteration; ρ ch is the control parameter of the chaotic map; δ ch is the modulation parameter, which controls the complexity of the chaotic sequence.

18. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 17, wherein, Based on the chaotic variables, use the backpropagation algorithm to update the parameters of the multi-branch group convolutional neural network according to the gradient information of the loss function.

19. The method for predicting the recovery status of neck scars after thyroidectomy based on CV according to claim 1, wherein The update method for the parameters in the branch group convolutional neural network is expressed as: Among them, η sep is the learning rate of the multi-branch group convolutional neural network, Xcs(t) is the chaotic variable at the t-th iteration, G b (t), G b (t + 1) respectively represent G at the t-th and (t + 1)-th iterations b , G b represents the group convolution kernels of each branch, w b (t), w b (t + 1) respectively represent w at the t-th and (t + 1)-th iterations b , w b represents the weight vector of the b-th branch; bc b (t), bc b (t + 1) respectively represent bc at the t-th and (t + 1)-th iterations b , bc b represents the bias of the b-th branch during multi-branch feature fusion.

20. A computer device, characterized in that, The device includes: a memory and a processor; the memory is used to store a computer program; the processor executes the computer program to implement the steps of the method according to any one of claims 1-19.

21. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-19.

22. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the method according to any one of claims 1-19.

Images