Intelligent auxiliary inquiry medical large model training method and system

Abstract

The invention discloses an intelligent auxiliary inquiry medical large model training method and system, and relates to the technical field of artificial intelligence auxiliary medical treatment, and the method comprises the following steps: S1, collecting medical inquiry data; s2, data annotation and cleaning; s3, constructing a medical large model; s4, optimizing the medical pre-training model; s5, task specific fine tuning; and S6, designing an inquiry system. The model trained by the intelligent auxiliary inquiry medical large model training method can efficiently capture important information when a large number of complex medical languages are processed, and the ability of understanding and reasoning the symptoms of a patient is improved; according to the invention, intelligent on-time suggestion generation can be carried out based on real-time data, questioning and adjustment can be carried out according to symptom information provided by a patient, and the doctor seeing efficiency and accuracy are greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence-assisted medical technology, and specifically to a large-scale model training method and system for intelligent-assisted medical consultation. Background Art

[0002] In the modern medical environment, with the increase in the number of patients and the limited medical resources, the workload of doctors has gradually increased, and the traditional consultation model faces many challenges. Especially in the initial diagnosis stage, the communication between doctors and patients is inefficient, and the patient's description of symptoms may not be comprehensive or accurate, resulting in an increased risk of misdiagnosis or missed diagnosis. In addition, when doctors deal with a large number of patients in a limited time, it is difficult to ensure the quality and depth of each consultation.

[0003] The rapid development of natural language processing technology in recent years, especially the rise of large-scale language models based on the Transformer architecture, has provided new opportunities for the research and development of intelligent medical consultation systems.

[0004] Many traditional intelligent medical consultation systems rely mainly on predefined rules or fixed dialogue templates, with a single data source that is difficult to cover a wide range of medical knowledge, resulting in limited model applicability and insufficient diagnostic capabilities. Traditional medical consultation systems based on keyword matching or simple statistical models have difficulty handling complex semantic and reasoning tasks, lack accurate understanding of patients' diverse expressions, and often cannot provide personalized diagnostic recommendations. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method and system for training a large model of intelligent assisted medical consultation to solve the problems existing in the prior art, improve the efficiency and accuracy of consultation, and enhance the applicability and intelligence of the model. To achieve the above objectives, the present invention is implemented through the following technical solutions: a method and system for training a large model of intelligent assisted question-answering medicine, comprising the following steps: Step S1, medical consultation data collection; Step S2: data labeling and cleaning; Step S3, construction of a large medical model; it includes the following steps of a feedforward network optimization algorithm: A1. Initialize the feedforward network parameters by the following formula: ; ; in, represents the initial weight matrix of the feedforward network, Represents the initial bias vector of the feedforward network; represents the weight matrix of the feedforward network, represents the bias vector of the feedforward network; is the dimension of the feature space input to the feedforward network, is the feature dimension input to the feedforward network; The function is used to generate random numbers with standard normal distribution; A2. The input data is forward propagated through the feedforward network, and the output feature vector is: ; is the Sigmoid activation function; and They are The weights and biases of the feedforward network for the iteration; is the feature vector input to the feedforward network; is the output feature vector of the feedforward network; A3. Using L1 regularization technology to sparse features, the loss function of the feedforward network is expressed as: ; is the target output of the feedforward network, is the loss function of the feedforward network, is the L2 norm, which represents the sum of squared errors; is the L1 norm; is the regularization coefficient of the feedforward network; is the weighted normalized output of the feedforward network. The weighted normalized output of the layer is expressed as ; Many elements of the weight vector are forced to zero by the following formula, ; in is the first Row and Elements of a column; The parameters of the feedforward network are updated as follows, ; ; ; ;in, is the learning rate of the feedforward network; and are the gradients of the weights and biases of the feedforward network respectively; is the symbol of partial derivative; For the The weights of the feedforward network for the iteration, For the The bias of the feedforward network for the iteration; For the The weights of the feedforward network for the iteration, For the The bias of the feedforward network for the iteration; Step S4, medical pre-training model optimization; Step S5, fine-tuning specific system tasks; Step S6: Question and answer system design.

[0006] Preferably, the medical consultation data in step S1 comes from the following channels: Real electronic medical records that have been desensitized and contain the patient's basic information and chief complaint; Records of conversations between doctors and patients, including patient symptoms and doctor recommendations; Clinical guidelines, teaching materials, and medical records issued by medical institutions; A simulated medical consultation dialogue generated based on rules or templates of medical theory.

[0007] Preferably, the data labeling and cleaning in step S2 is achieved by: Label the collected raw data with symptom descriptions, syndrome differentiation, pathogenesis analysis, and medical terms; Clean the data to remove noise and irrelevant content and standardize inconsistent formats.

[0008] Preferably, the medical pre-training model is optimized as follows: Fine-tune the selected pre-trained model for the domain; load the collected medical consultation data; pre-train on a large scale using self-supervised learning techniques; predict masked vocabulary or sentence structures to master specific language patterns and knowledge in the medical field.

[0009] Preferably, the query, key, column, and value vectors of the multi-head attention mechanism sublayer are divided into multiple smaller heads. For each head, the dot product of the query and key vectors is calculated to obtain an attention score, and the Softmax function is used to process the attention score, where the attention weight of the i-th head is expressed as:

[0010] For the The attention weight of each head, For the The query vector of the head, For the The key vector of the head, For the The transpose of the key vector of the head, is the dimension of the input features, is the number of attention heads, For the The value vector of the heads; Use the obtained attention weights to perform weighted summation on the value vector to obtain the output vector; The output vectors of all heads are concatenated together to generate the output of the final multi-head self-attention layer.

[0011] Preferably, the layer output is evaluated by normalization, expressed as:

[0012] is a normalization function; For the feed-forward network The output of the layer; It is a feed-forward network. Normalized features of layers; The weight adjustment coefficient of the feedforward network data is calculated as:

[0013] is the Softmax function; represents the weight matrix of the feedforward network, represents the bias vector of the feedforward network; For the feed-forward network The weight adjustment coefficient of the layer; The feature weighting is performed based on the weight adjustment coefficient, which is expressed as:

[0014] For the feed-forward network Weighted normalized output of the layer; For the feed-forward network Dynamic adjustment factor of the layer.

[0015] Preferably, the dynamic adjustment factor adaptively amplifies or suppresses the output of the layer according to the intensity of the gradient, and the calculation method of the dynamic adjustment factor is expressed as:

[0016] is the loss function of the feedforward network. The gradient of the output of the layer.

[0017] Preferably, the adjustment method of the regularization coefficient of the feedforward network is expressed as:

[0018] is the regularization coefficient of the initial feedforward network, is the regularization decay rate of the feedforward network, is the current iteration number, is the total number of iterations of the feedforward network.

[0019] Preferably, an extreme learning machine classification algorithm based on adaptive meta-topology is used to verify the task.

[0020] In a preferred embodiment of the present invention, a system for training a large model of intelligent assisted medical consultation is provided, characterized in that the intelligent question-answering module includes the following functions: The system can automatically generate question-and-answer dialogues that conform to medical logic, covering the entire process from symptom description to disease classification; The system can answer users’ medical questions and provide professional medical knowledge related to symptoms and treatments; Based on the current state of the conversation, the system intelligently asks further questions, clarifies the type of disease corresponding to the symptoms, and provides suggestions for the diagnosis and treatment path.

[0021] The present invention provides a method and system for training a large model of intelligent assisted medical consultation. Compared with the prior art, it has the following beneficial effects: Through the multi-head attention mechanism and sparse feature selection algorithm of the large medical model, the model can efficiently capture important information when processing a large amount of complex medical language, and improve the understanding and reasoning ability of patient symptoms; The system can generate intelligent medical advice based on real-time data, and can conduct follow-up questions and adjustments based on the symptom information provided by the patient, greatly improving the efficiency and accuracy of medical treatment; Through the combination of pre-training and fine-tuning, the model can not only master the general knowledge in the medical field, but also effectively cope with the challenges of specific tasks, and has strong generalization ability and task adaptability. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a functional block diagram of the video monitoring system of the present invention; Figure 2 Schematic diagram of the neural network architecture of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] See also Figure 1-Figure 2 The present invention provides a method and system technical solution for training a large model of intelligent assisted medical consultation: a method and system for training a large model of intelligent assisted medical consultation, comprising the following steps: S1, medical consultation data collection; S2, data labeling and cleaning; S3, construction of large medical models; S4, optimization of medical pre-training models; Step S5, task-specific fine-tuning: Based on the pre-training, fine-tuning is performed for specific tasks such as symptom extraction, diagnosis suggestion generation, and medical knowledge question answering; Step S6, consultation system design: using real-time reasoning, the trained Transformer model is used to generate doctor consultation suggestions based on patient input information. The system gives dynamic feedback based on the reasoning results, and provides diagnosis and treatment suggestions and possible diagnosis paths.

[0025] The medical consultation data is collected through the following channels: Obtain de-sensitized electronic medical records containing basic patient information, chief complaints, and other content from cooperating TCM hospitals or TCM clinics; Cooperate with TCM medical institutions to obtain records of conversations between doctors and patients, including patient symptoms, doctor's advice, etc., from online or offline TCM consultation platforms; Collect clinical guidelines, teaching materials, and medical records issued by authoritative TCM institutions; Use rules or templates based on TCM theory to generate simulated consultation dialogues that cover common TCM symptoms and ensure logic and completeness; The data labeling and cleaning are specifically carried out in the following ways: Label the collected raw data with symptom description, syndrome differentiation, pathogenesis analysis, and TCM terminology; Clean the data to remove noise and irrelevant content and standardize inconsistent formats to ensure data quality and model training results; The method for constructing the medical macro model: The Transformer algorithm model is used as the basic architecture of the medical big model. It has the ability to process long texts and capture semantic information, which is suitable for the complex semantic understanding and reasoning requirements of medical consultation dialogue scenarios. The Transformer network consists of an encoder and a decoder. The encoder maps the input features into vector representations, and the decoder decodes them into target features. The encoder consists of N identical encoding layers, each of which contains a multi-head attention mechanism sublayer and a feedforward network sublayer, and each layer is followed by a residual connection and normalization operation; The multi-head attention mechanism sublayer is implemented through the multi-head self-attention mechanism, which is responsible for learning the importance of positions and combining information to obtain a global context representation to improve performance and generalization. Its implementation includes generating specific vectors, splitting into heads, calculating dot products to obtain attention scores, and processing scores with Softmax to obtain weight operations; The feed-forward network sublayer consists of two fully connected layers and an activation function, which performs nonlinear transformation on the vector to improve the model capability; The decoder consists of N identical decoding layers, each of which contains two multi-head attention mechanism sublayers and one feedforward network mechanism sublayer. The second multi-head attention mechanism sublayer realizes information interaction through the multi-head cross attention mechanism, and the output of each sublayer of the encoder and decoder is the result of layer normalization and residual connection. The specific method of optimizing the medical pre-training model is as follows: Fine-tune the selected pre-trained model in the field, load the medical consultation data collected in the early stage, and use self-supervised learning technology to pre-train on a large scale, so that the model can gradually master the specific language patterns and knowledge in the medical field by predicting the masked vocabulary or sentence structure; The specific method of task-specific fine-tuning is as follows: Based on pre-training, fine-tuning is performed for specific tasks such as symptom extraction, diagnosis suggestion generation, and medical knowledge question answering; Combining classification tasks such as disease prediction with generation tasks such as question dialogue generation, we improve generalization capabilities under the multi-task learning framework and perform medical logic verification on the generated dialogue content to ensure output reliability. For the classification task verification, an extreme learning machine classification algorithm based on adaptive meta-topology is used. The algorithm consists of an input layer, a hidden layer of adaptive meta-topology, and an output layer. The input layer receives vectorized medical data, the hidden layer can dynamically adjust the neuron connection mode, and the output layer is responsible for outputting the classification results. The meta-path concept is used to capture the high-order association of features to improve learning ability. Based on pre-training, fine-tuning is performed for specific tasks (such as symptom extraction, diagnosis suggestion generation, and medical knowledge question and answer). Combining classification tasks (such as disease prediction) and generation tasks (such as question dialogue generation), the model improves generalization capabilities under the multi-task learning framework, performs medical logic verification on the generated dialogue content, and ensures the medical reliability of the output results.

[0026] For the verification of classification tasks, the present invention adopts an extreme learning machine classification algorithm based on adaptive meta-topology. Traditional extreme learning machines use fixed random weights and biases and cannot fully adapt to complex medical data. The present invention adopts an adaptive meta-topology structure so that the neuron connections in the hidden layer can be dynamically adjusted according to the characteristics of the data, thereby enhancing the flexibility of the model.

[0027] The extreme learning machine classification algorithm based on adaptive meta-topology consists of an input layer, a hidden layer of adaptive meta-topology, and an output layer. The input layer receives vectorized medical data, the hidden layer adopts an adaptive meta-topology structure, and can dynamically adjust the connection mode of neurons according to data characteristics. The output layer is responsible for outputting classification results. By adopting the concept of meta-path, the model can capture high-order associations between features and improve the learning ability of complex data.

[0028] Specifically, the training process of the extreme learning machine classification algorithm based on adaptive meta-topology is as follows: Initialize the parameters of the extreme learning machine model and assume that the vectorized medical data is , the corresponding label is . For the The feature vector of the samples, For the The feature vector of the samples, For the The classification label corresponding to the feature vector of each sample is For the The classification label corresponding to the feature vector of each sample is is the number of samples input to the extreme learning machine.

[0029] Furthermore, the weight matrix and bias vector from the input layer to the hidden layer of the extreme learning machine are randomly initialized, expressed as:

[0030]

[0031] is the weight of the extreme learning machine, is the bias of the extreme learning machine; is the input layer of the extreme learning machine nodes and the hidden layer The initial weight of each node; is the hidden layer of the extreme learning machine The bias of the node.

[0032] The design of the medical consultation system includes: Design the system interaction logic based on the typical medical consultation system, including patient symptom input, doctor diagnosis suggestion output, and follow-up questioning; Using real-time reasoning, the trained Transformer model generates doctor consultation suggestions based on patient input information. The system gives dynamic feedback based on the reasoning results, providing diagnosis and treatment suggestions and possible diagnosis paths. Includes the following functional modules: Automatic question generation module: The system can automatically generate a medically logical question dialogue, covering the entire process from symptom description to diagnostic suggestions; Disease prediction and diagnosis support module: Based on the symptom information input by the patient, the system automatically analyzes potential diseases and provides diagnostic suggestions to assist doctors in making decisions; Medical knowledge question and answer module: The system can answer users' medical questions and provide professional medical knowledge related to symptoms and treatments; Intelligent follow-up questioning and treatment path recommendation module: The system can intelligently raise further questions based on the current conversation status, gradually narrow the diagnosis scope, and provide treatment path recommendations; In the step of building a large medical model, the multi-head attention mechanism sublayer is implemented as follows: The input features are transformed through three different linear transformation layers to generate "query", "key" and "value" vectors respectively. The linear transformation is achieved by multiplying the weight matrix. Split the generated query, key, column, and value vectors into multiple smaller parts, namely "heads", so that the model can process multiple different representation subspaces in parallel to capture the diversity of the input data; For each head, the dot product of the query and key vectors is calculated to obtain an attention score, which reflects the relative importance of different positions in the sequence. In order to stabilize the value of the attention score, the score is divided by a scaling factor, which can be the square root of the key vector dimension. The attention scores are processed using the Softmax function to obtain the attention weights for each position. The Softmax function ensures that all weights add up to 1, so that it can be processed as a probability distribution, where the attention weight of the i-th head is expressed as:

[0033] For the The attention weight of each head, For the The query vector of the head, For the The key vector of the head, For the The transpose of the key vector of the head, is the dimension of the input features, is the number of attention heads, For the The value vector of the heads; Use the obtained attention weights to perform weighted summation on the value vector to obtain the output vector; The output vectors of all heads are concatenated together and then passed through a linear transformation layer to generate the output of the final multi-head self-attention layer; In the step of constructing the medical large model, the feedforward network sublayer consists of two fully connected layers and an activation function, which is used to perform nonlinear transformation on the vector at each position, and its implementation method is as follows: The input vector is first linearly transformed by the first fully connected layer; Then a ReLU activation function is used for nonlinear transformation; Finally, a linear transformation is performed through the second fully connected layer to improve the model's expressiveness and generalization capabilities, allowing the model to better handle complex input data; In the step of building a large medical model, the feedforward network of the Transformer network adopts a feedforward network optimization algorithm based on sparse feature selection of a hyperplane, which includes the following steps: A1. Initialization phase: First, initialize the feedforward network parameters. The initialization method is expressed as:

[0034]

[0035] represents the initial weight matrix of the feedforward network, Represents the initial bias vector of the feedforward network; represents the weight matrix of the feedforward network, represents the bias vector of the feedforward network; is the dimension of the feature space input to the feedforward network, is the feature dimension input to the feedforward network; The function is used to generate random numbers with a standard normal distribution to ensure diversity in initial weights and biases.

[0036] A2. In each iterative training process, the input data is forward propagated through the feedforward network. The output of each layer is determined by the output of the previous layer and the parameters of the current layer. The output converted features are expressed as:

[0037] is the Sigmoid activation function; and They are The weights and biases of the feedforward network for the iteration; is the feature vector input to the feedforward network; is the output feature vector of the feedforward network.

[0038] A3. In the process of feature conversion, L1 regularization technology is used to achieve feature sparsification, which helps to remove redundant and unimportant features and retain the features that are most beneficial to subsequent tasks, expressed as:

[0039] is the target output of the feedforward network, is the loss function of the feedforward network, is the L2 norm, which represents the sum of squared errors; is the L1 norm, which is used to achieve sparse weights; is the regularization coefficient of the feedforward network, which regulates the degree of sparsity; is the weighted normalized output of the feedforward network. The weighted normalized output of the layer is expressed as .

[0040] In the data feature extraction model training stage, traditional methods mainly focus on static feature processing and lack the adaptability to dynamic data environment changes. The present invention adjusts the importance of weights of each layer in the feedforward network through a dynamic weight redistribution strategy, and dynamically optimizes the feedforward network structure according to the changes in data flow, thereby improving the model's adaptability to changing data characteristics and prediction accuracy. Specifically, For the feed-forward network The output of the layer is evaluated by normalizing the layer output, expressed as:

[0041] It is a normalization function used to evaluate the activity or information content of the layer output; It is a feed-forward network. Normalized features of the layer.

[0042] Furthermore, the weight adjustment coefficient of the feedforward network data is calculated, which is expressed as:

[0043] is the Softmax function; and are the weights and biases of DWRN, which are used to generate the weight adjustment coefficients of each layer from the evaluation indicators; For the feed-forward network Weight adjustment factor for the layer.

[0044] Furthermore, feature weighting is performed based on the weight adjustment coefficient, which is expressed as:

[0045] For the feed-forward network Weighted normalized output of the layer; For the feed-forward network Dynamic adjustment factor of the layer.

[0046] Furthermore, the dynamic adjustment factor adaptively amplifies or suppresses the output of the layer according to the strength of the gradient, enhancing the sensitivity of the model to important features. The calculation method of the dynamic adjustment factor is expressed as:

[0047] is the loss function of the feedforward network. The gradient of the output of the layer.

[0048] Furthermore, the role of the regularization term of the weight sparsification is to enhance the generalization ability of the model by causing many elements of the weight vector to approach zero. The calculation method is expressed as:

[0049] in is the first Row and By summing the absolute values ​​of each weight element, the model is forced to reduce unnecessary parameter complexity while maintaining the expression of key features.

[0050] In one embodiment, to meet the needs of different training stages, the regularization coefficient of the feedforward network is based on an adaptive adjustment mechanism of the training error, which decays exponentially as the iteration progresses, aiming to provide strong regularization in the early stage of training to prevent overfitting, and gradually relax in the later stage of training, allowing the model to adjust weights more freely to refine the fitting data. The adjustment method is expressed as:

[0051] is the regularization coefficient of the initial feedforward network, is the regularization decay rate of the feedforward network, is the current iteration number, is the total number of iterations of the feedforward network; According to the gradient calculated by the loss function, the feedforward network parameters are adjusted through the back propagation algorithm to optimize the hyperplane position and feedforward network weights to reduce the prediction error and improve the generalization ability of the model. The parameter update method of the feedforward network is expressed as:

[0052]

[0053]

[0054]

[0055] is the learning rate of the feedforward network, which determines the step size of parameter update; and are the gradients of the weights and biases of the feedforward network respectively; is the symbol of partial derivative; For the The weights of the feedforward network for the iteration, For the The bias of the feedforward network for the iteration; For the The weights of the feedforward network for the iteration, For the The bias of the feedforward network for the iteration; Repeat the above steps until the preset stop iteration condition is met, indicating that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times; Based on pre-training, fine-tuning is performed for specific tasks (such as symptom extraction, diagnosis suggestion generation, and medical knowledge question and answer). Combining classification tasks (such as disease prediction) and generation tasks (such as question dialogue generation), the model improves generalization capabilities under the multi-task learning framework, performs medical logic verification on the generated dialogue content, and ensures the medical reliability of the output results.

[0056] For the verification of classification tasks, the present invention adopts an extreme learning machine classification algorithm based on adaptive meta-topology. The traditional extreme learning machine adopts fixed random weights and biases, which cannot fully adapt to complex medical data. The present invention adopts an adaptive meta-topology structure, so that the neuron connections of the hidden layer can be dynamically adjusted according to the characteristics of the data, thereby enhancing the flexibility of the model. The extreme learning machine classification algorithm based on adaptive meta-topology consists of an input layer, a hidden layer of adaptive meta-topology, and an output layer. The input layer receives vectorized medical data, the hidden layer adopts an adaptive meta-topology structure, and can dynamically adjust the connection mode of neurons according to data characteristics. The output layer is responsible for outputting classification results. By adopting the concept of meta-path, the model can capture high-order associations between features and improve the learning ability of complex data.

[0057] Specifically, the training process of the extreme learning machine classification algorithm based on adaptive meta-topology is as follows: Initialize the parameters of the extreme learning machine model and assume that the vectorized medical data is , the corresponding label is . For the The feature vector of the samples, For the The feature vector of the samples, For the The classification label corresponding to the feature vector of each sample is For the The classification label corresponding to the feature vector of each sample is is the number of samples input to the extreme learning machine.

[0058] Furthermore, the weight matrix and bias vector from the input layer to the hidden layer of the extreme learning machine are randomly initialized, expressed as:

[0059]

[0060] is the weight of the extreme learning machine, is the bias of the extreme learning machine; is the input layer of the extreme learning machine nodes and the hidden layer The initial weight of each node; is the hidden layer of the extreme learning machine The bias of each node; According to the meta-path information, the topological structure of the hidden layer is adaptively adjusted. Specifically, the connection mode between neurons is dynamically changed through the feature association in the meta-path, the weight of important features is increased, and the influence of redundant features is reduced. Suppose the connection matrix is , is the connection matrix Line The elements of the column represent the connection strength, and the calculation method is expressed as:

[0061] is the activation function that controls the connection strength; For the Features and hidden layer nodes in the metapath The associated function in .

[0062] Furthermore, let the input of the activation function that controls the connection strength be , and its calculation method is expressed as:

[0063] is the gain parameter of the activation function. Preferably, Set to 2.

[0064] Furthermore, the calculation method of the correlation function is expressed as:

[0065] is an indicator function, which is 1 if the condition is met, otherwise it is 0; Indicates the first condition, specifically ; Indicates the second condition, specifically ; is the order of the higher-order term; Metapath No. order interaction coefficient; and Respectively Features A feature importance measure.

[0066] Furthermore, the calculation method of feature importance measure is expressed as:

[0067] For the The sample eigenvalues; For the The average value of the features.

[0068] In the hidden layer of the extreme learning machine, the weights and biases of the neurons are randomly generated, but under the constraints of the adaptive meta-topology, the generated parameters are ensured to conform to the characteristic associations described by the meta-path. Characterizing the constraints of the adaptive meta-topology, the weights and biases of the hidden layer of the extreme learning machine are generated as follows:

[0069]

[0070] is the weight matrix of the adjusted extreme learning machine; is the bias of the adjusted extreme learning machine; Represents element-wise multiplication.

[0071] The medical data is input into the model and passed to the hidden layer through the input layer. The hidden layer uses the adaptive meta-topology structure to perform nonlinear mapping on the input data and generate the hidden layer output. The calculation method of the hidden layer output is expressed as:

[0072] is the hidden layer output of the extreme learning machine; is the hidden layer activation function.

[0073] Furthermore, let the input of the hidden layer activation function be , the calculation method of the hidden layer activation function is expressed as:

[0074] Since the parameters of the hidden layer have been determined, the least squares method is used to solve the weight of the output layer so that the output of the model is as close as possible to the actual classification label. The classification label is a preset data label. The labeling method is to assign the corresponding classification category to the collected medical data. Suppose the output of the extreme learning machine is , the target label is , the calculation method of the extreme learning machine loss is expressed as:

[0075] is the loss of the extreme learning machine; is the output layer weight matrix; is the regularization parameter; For the The importance of a sample.

[0076] Furthermore, the least squares method is used to solve the optimal output layer weight matrix, which is expressed as:

[0077] is the identity matrix; for The transpose of is the diagonal weight matrix of the importance of the sample, whose elements are .

[0078] Furthermore, considering the importance of the sample, the sample weight vector is defined as , For the sample weight vector, For the The sample weight vector is calculated as follows:

[0079] For the The error of samples in the previous iteration; is the sample weight adjustment parameter. Preferably, Set to 0.2.

[0080] Use the extreme learning machine model to predict the training data and calculate the error between the predicted result and the actual label. The error is used to evaluate the model performance and provide a basis for the next adjustment. The output of the extreme learning machine model is expressed as:

[0081] Furthermore, the error is calculated based on the output of the extreme learning machine model, which is expressed as:

[0082] is the training error of the extreme learning machine; For the The output of samples; is the actual classification label of the th sample; is the loss function of the extreme learning machine; is the coefficient of the sum function combination; is the sum function combination weight. Preferably, Set to 0.1.

[0083] Furthermore, the multi-kernel function feature calculation method is used to extract multi-angle features of the medical data input into the extreme learning machine to obtain more comprehensive features that are conducive to classification. The calculation method of the hybrid kernel matrix of the extreme learning machine is defined as:

[0084] is the mixing kernel matrix of the extreme learning machine; For the The kernel matrix calculated by the kernel function; For the Kernel functions, including Gaussian kernel, polynomial kernel, and Laplace kernel; is the combination coefficient of the kernel function, satisfying ,and .

[0085] Furthermore, the calculation method of the Gaussian kernel function is expressed as:

[0086] is the Gaussian kernel function; represents the Frobenius norm; is the Gaussian kernel function parameter, is the training parameter, and is obtained through training.

[0087] And, the calculation method of the polynomial kernel function is expressed as:

[0088] is the polynomial kernel function; is the first polynomial kernel function parameter, is the second polynomial kernel function parameter, is the training parameter, and is obtained through training.

[0089] And, the calculation method of the Laplace kernel function is expressed as:

[0090] is the Laplace kernel function; is the Laplace kernel function parameter, is the training parameter, obtained through training.

[0091] According to the error feedback, the adaptive meta-topology structure and hidden layer parameters are adjusted. Specifically, the weights of the features involved in the high error samples are increased, the meta-path is reconstructed, the connection mode of the hidden layer is optimized, and the meta-path weight is updated as follows:

[0092] is the weight of the meta-path before updating, is the weight of the updated meta-path; is the learning rate of the extreme learning machine. Preferably, Set to 0.001.

[0093] Furthermore, the hidden layer weights of the extreme learning machine are updated, expressed as:

[0094] is the weight matrix of the adjusted extreme learning machine before updating; is the updated adjusted extreme learning machine weight matrix.

[0095] Repeat the above steps until the preset stop iteration condition is met, which means that the model training is completed. In one embodiment, the preset stop iteration condition is reaching a preset maximum number of iterations. Preferably, the preset maximum number of iterations is set to 1000 times.

[0096] When the performance of the extreme learning machine model meets the requirements, the adaptive meta-topology structure of the hidden layer and the weight of the output layer are fixed to form the final classifier model.

[0097] Meanwhile, the contents not described in detail in this specification belong to the prior art known to those skilled in the art.

[0098] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0099] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for training a large model of intelligent assisted medical consultation, comprising the following steps: Step S1, medical consultation data collection; Step S2: data labeling and cleaning; Step S3, construction of a large medical model, which includes the following steps of a feedforward network optimization algorithm: A1. Initialize the feedforward network parameters; A2, input data is forward propagated through the feedforward network; A3, use L1 regularization technique to sparse features; Step S4, medical pre-training model optimization; Step S5, fine-tuning specific system tasks; Step S6: Question and answer system design.

2. The intelligent assisted medical consultation model training method according to claim 1 is characterized in that: The medical consultation data in step S1 comes from the following channels: Real electronic medical records that have been desensitized and contain the patient's basic information and chief complaint; Records of conversations between doctors and patients, including patient symptoms and doctor recommendations; Clinical guidelines, teaching materials, and medical records issued by medical institutions; A simulated medical consultation dialogue generated based on rules or templates of medical theory.

3. The method for training a large model of intelligent assisted medical consultation according to claim 1, characterized in that: The data labeling and cleaning in step S2 is achieved by: Label the collected raw data with symptom descriptions, syndrome differentiation, pathogenesis analysis, and medical terms; Clean the data to remove noise and irrelevant content and standardize inconsistent formats.

4. The intelligent assisted medical consultation model training method according to claim 1 is characterized in that: In step A1, the feedforward network parameters are initialized by the following formula: ； ； in, represents the initial weight matrix of the feedforward network, Represents the initial bias vector of the feedforward network; represents the weight matrix of the feedforward network, represents the bias vector of the feedforward network; is the dimension of the feature space input to the feedforward network, is the feature dimension input to the feedforward network; The function is used to generate random numbers with a standard normal distribution.

5. The intelligent assisted medical consultation model training method according to claim 1 is characterized in that: In step S3, the output feature vector of the feedforward network is: ; is the Sigmoid activation function; and They are The weights and biases of the feedforward network for the iteration; is the feature vector input to the feedforward network; is the output feature vector of the feedforward network; the loss function of the feedforward network is expressed as: ; is the target output of the feedforward network, is the loss function of the feedforward network, is the L2 norm, which represents the sum of squared errors; is the L1 norm; is the regularization coefficient of the feedforward network; is the weighted normalized output of the feedforward network. The weighted normalized output of the layer is expressed as ; Many elements of the weight vector are forced to zero by the following formula, ; in is the first Row and Elements of a column; The parameters of the feedforward network are updated as follows: ; ; ; ;in, is the learning rate of the feedforward network; and are the gradients of the weights and biases of the feedforward network respectively; is the symbol of partial derivative; For the The weights of the feedforward network for the iteration, For the The bias of the feedforward network for the iteration; For the The weights of the feedforward network for the iteration, For the The bias of the feed-forward network for the iteration .

6. The intelligent assisted medical consultation model training method according to claim 1 is characterized in that: The medical pre-training model optimization method is as follows: Fine-tune the selected pre-trained model for the domain; load the collected medical consultation data; pre-train on a large scale using self-supervised learning techniques; predict masked vocabulary or sentence structures to master specific language patterns and knowledge in the medical field.

7. The method for training a large model of intelligent assisted medical consultation according to claim 6, characterized in that: The query, key, column, and value vectors of the multi-head attention mechanism sublayer are divided into multiple smaller heads. For each head, the dot product of the query and key vectors is calculated to obtain the attention score. The Softmax function is used to process the attention score, where the attention weight of the i-th head is expressed as: , For the The attention weight of each head, For the The query vector of the head, For the The key vector of the head, For the The transpose of the key vector of the head, is the dimension of the input features, is the number of attention heads, For the The value vector of the heads; Use the obtained attention weights to perform weighted summation on the value vector to obtain the output vector; The output vectors of all heads are concatenated together to generate the output of the final multi-head self-attention layer.

8. The intelligent assisted medical consultation model training method according to claim 1 is characterized in that: The layer output is evaluated by normalization, expressed as: , is a normalization function; For the feed-forward network The output of the layer; It is a feed-forward network. Normalized features of layers; The weight adjustment coefficient of the feedforward network data is calculated as: , is the Softmax function; represents the weight matrix of the feedforward network, represents the bias vector of the feedforward network; For the feed-forward network The weight adjustment coefficient of the layer; The feature weighting is performed based on the weight adjustment coefficient, which is expressed as: , For the feed-forward network Weighted normalized output of the layer; For the feed-forward network Dynamic adjustment factor of the layer.

9. The method for training a large model of intelligent assisted medical consultation according to claim 8, characterized in that: The dynamic adjustment factor adaptively amplifies or suppresses the output of the layer according to the strength of the gradient. The calculation method of the dynamic adjustment factor is expressed as: , is the loss function of the feedforward network. The gradient of the output of the layer.

10. A system for executing the intelligent assisted medical consultation large model training method as claimed in claim 1, characterized in that: Includes intelligent question-answering module, including the following functions: The system can automatically generate question-and-answer dialogues that conform to medical logic, covering the entire process from symptom description to disease classification; The system can answer users’ medical questions and provide professional medical knowledge related to symptoms and treatments; Based on the current state of the conversation, the system intelligently asks further questions, clarifies the type of disease corresponding to the symptoms, and provides suggestions for the diagnosis and treatment path.

Images