Interaction-based model performance improvement method and device, equipment and storage medium

Abstract

The invention relates to a model training technology, and discloses an interaction-based model performance improvement method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: selecting medical pictures in a medical picture set one by one as target medical pictures, and generating the medical text description of the target medical pictures through a proxy model; obtaining target problem feedback corresponding to the target medical picture and the medical text; extracting a target agency reward and a target medical answer from the target question feedback; and collecting the target medical answer, the target medical picture, the medical text and the target agent reward into a target training data set, grouping an agent model by using all the target training data to carry out iterative training to obtain a standard agent model, and ending model training. The method can be applied to the field of medical picture analysis, the model accuracy and the model training speed of the medical picture inquiry model can be optimized, and the model training efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of behavior model training, and in particular to an interaction-based model performance improvement method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the development of artificial intelligence technology, artificial intelligence models are being used more and more in medical applications. As an important application of artificial intelligence in medical care, intelligent medical consultation uses neural network models to analyze possible symptoms and problems based on acquired medical images. It is widely used in major medical institutions. In order to improve the accuracy and efficiency of intelligent medical consultation, it is necessary to improve the performance of the model.

[0003] Most existing model performance improvement technologies are based on performance improvement methods of manually expanded data sets, that is, more medical data sets are generated through manual labeling methods, and the newly generated medical data sets are used to further strengthen the model training. In practical applications, performance improvement methods based on manually expanded data sets require manual work to be performed in large quantities, with high requirements on time and labor costs, and are unable to achieve continuous performance improvement, which may lead to low efficiency in model training. Summary of the invention

[0004] The present invention provides an interactive model performance improvement method, device and computer-readable storage medium, the main purpose of which is to solve the problem of low efficiency during model training.

[0005] To achieve the above object, the present invention provides an interactive model performance improvement method, comprising:

[0006] Selecting medical images in the medical atlas one by one as target medical images, encoding the target medical images into target medical vectors using the encoder model in the proxy model, and encoding the target medical vectors into medical text descriptions using the text generation model of the proxy model;

[0007] Publishing the target medical image and the medical text as a target medical question to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal;

[0008] Extracting target interaction information and target interaction text from the target question feedback, and calculating a target agent reward using the agent model and the target interaction information;

[0009] Performing part-of-speech tagging, entity recognition, and syntactic analysis operations on the target interactive text in sequence to obtain a target annotated text, and performing semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer;

[0010] The target medical answer, the target medical image, the medical text and the target agent reward are collected into a target training data group, and the agent model is iteratively trained using all the target training data groups to obtain a standard agent model, thereby ending the model training.

[0011] Optionally, encoding the target medical image into a target medical vector using an encoder model in the proxy model includes:

[0012] Using the encoder model in the proxy model to sequentially perform multi-level convolution and multi-level pooling operations on the target medical image to obtain target medical features;

[0013] Performing feature flattening and full connection operations on the target medical features in sequence to obtain target feature encoding;

[0014] Performing latent variable analysis on the target feature encoding to obtain latent distribution features;

[0015] The target latent feature is re-parameterized according to the latent distribution feature to obtain a target medical vector.

[0016] Optionally, encoding the target medical vector into a medical text description using the text generation model of the proxy model includes:

[0017] Using the text generation model of the proxy model, adding a position code to the target medical vector code to obtain a position medical code;

[0018] Performing attention encoding on the position medical code by using a self-attention mechanism to obtain an attention medical code;

[0019] Performing multi-level residual connection and full connection operations on the attention medical code to obtain a standard medical code;

[0020] A linear activation operation is performed on the standard medical coding to obtain a medical text description.

[0021] Optionally, calculating a target proxy reward by using the proxy model and the target interaction information includes:

[0022] extracting interaction frequency features and interaction result features from the target interaction information respectively;

[0023] The interaction frequency feature and the interaction result feature are spliced ​​into a target interaction feature, and a target interaction reward corresponding to the target interaction feature is extracted using a preset bisection model;

[0024] Extracting the image confidence corresponding to the target interaction information from the proxy model, and using the image confidence as the target knowledge reward;

[0025] The target agent reward of the agent model is calculated according to the target interaction reward and the target knowledge reward using the following balanced reward formula:

[0026] R=α·R interaction +(1-α)·R knowledge ,0≤α≤1

[0027] Where R refers to the target agent reward, α is the confrontation coefficient preset in the balance reward formula, and R interaction refers to the target interaction reward, R knowledge refers to the target knowledge reward.

[0028] Optionally, the step of sequentially performing part-of-speech tagging, entity recognition, and syntactic analysis on the target interactive text to obtain a target annotated text includes:

[0029] Performing text segmentation and stop word filtering operations on the target interactive text in sequence to obtain an interactive word sequence;

[0030] Performing a vectorization operation on the interactive word sequence to obtain an interactive word vector sequence;

[0031] Performing part-of-speech encoding on the interactive word vector sequence to obtain an interactive part-of-speech feature set;

[0032] Performing entity detection on the interactive word vectors in the interactive word vector sequence one by one to obtain an entity word vector set;

[0033] Performing a graph convolution operation on the interactive word vector sequence to obtain interactive syntactic features;

[0034] splicing the interactive word sequences into primary interactive text, and performing part-of-speech mapping on the primary interactive text using the interactive part-of-speech feature set to obtain a part-of-speech interactive text;

[0035] Using the entity word vector set to perform entity tagging on the part-of-speech interaction text to obtain entity interaction text;

[0036] The entity interaction text is syntactically annotated using the interaction syntactic features to obtain a target annotated text.

[0037] Optionally, the using the text parsing model of the proxy model to perform semantic analysis on the target annotated text to obtain a target medical answer includes:

[0038] Using the text parsing model of the proxy model to perform semantic role labeling on the target standard text to obtain a primary semantic text;

[0039] Extracting semantic relations from the primary semantic text by using entity association;

[0040] converting the primary semantic text into semantic features according to the semantic relationship;

[0041] The semantic features are activated and transcoded to obtain the target medical answer.

[0042] Optionally, the iterative training of the proxy model using all target training data sets to obtain a standard proxy model includes:

[0043] Gather all target training data sets into a training data set;

[0044] Aggregating the target proxy rewards for each target training data set in the training data set into a proxy reward set;

[0045] Performing threshold filtering on the proxy reward set to obtain a standard proxy reward set;

[0046] Filtering the target training data set corresponding to the standard proxy reward set from the training data set to form a standard training data set;

[0047] The proxy model is iteratively trained using the standard training data set to obtain a standard proxy model.

[0048] In order to solve the above problems, the present invention also provides an interactive model performance improvement device, the device comprising:

[0049] A question generation module, used for selecting medical images in the medical atlas one by one as target medical images, encoding the target medical images into target medical vectors using the encoder model in the proxy model, and encoding the target medical vectors into medical text descriptions using the text generation model of the proxy model;

[0050] A feedback acquisition module, used for publishing the target medical image and the medical text as a target medical question to a user terminal, and acquiring target question feedback corresponding to the target medical question from the user terminal;

[0051] An incentive calculation module, used to extract target interaction information and target interaction text from the target question feedback, and calculate a target agent reward using the agent model and the target interaction information;

[0052] A semantic analysis module, used to perform part-of-speech tagging, entity recognition and syntactic analysis operations on the target interactive text in sequence to obtain a target annotated text, and perform semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer;

[0053] The model training module is used to collect the target medical answer, the target medical image, the medical text and the target agent reward into a target training data group, use all the target training data groups to iteratively train the agent model, obtain a standard agent model, and end the model training.

[0054] In order to solve the above problem, the present invention further provides an electronic device, the electronic device comprising:

[0055] at least one processor; and,

[0056] a memory communicatively connected to the at least one processor; wherein,

[0057] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the above-mentioned interaction-based model performance improvement method.

[0058] In order to solve the above problems, the present invention also provides a computer-readable storage medium, in which at least one computer program is stored. The at least one computer program is executed by a processor in an electronic device to implement the above-mentioned interaction-based model performance improvement method.

[0059] The embodiment of the present invention selects medical images in a medical atlas as target medical images one by one, uses an encoder model in a proxy model to encode the target medical images into target medical vectors, and uses a text generation model of the proxy model to encode the target medical vectors into medical text descriptions. Descriptive questions can be generated for medical images to facilitate subsequent interaction with users. By publishing the target medical images and the medical text as target medical questions to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal, human-computer interaction can be achieved, the testing and training accuracy of the model can be further expanded, and the fault tolerance of the model can be improved. By extracting target interaction information and target interaction text from the target question feedback, and using the proxy model and the target interaction information to calculate the target proxy reward, the target proxy reward can be used to measure the effectiveness of each interaction with the user, thereby improving the interactivity of the model.

[0060] By sequentially performing part-of-speech tagging, entity recognition, and syntactic analysis operations on the target interactive text, a target annotated text is obtained, and a text parsing model of the proxy model is used to perform semantic analysis on the target annotated text to obtain a target medical answer, which can assist the computer in understanding the user's reply to the target medical question, expand the training data set, and improve the accuracy of the model. By aggregating the target medical answer, the target medical image, the medical text, and the target proxy reward into a target training data group, the proxy model is iteratively trained using all target training data groups to obtain a standard proxy model, and the type and size of the data set can be expanded in combination with user feedback, and random events can be added to the data set, thereby automatically improving the accuracy and training speed of the model, thereby improving the model performance. Therefore, the interactive model performance improvement method, device, electronic device, and computer-readable storage medium proposed in the present invention can solve the problem of low efficiency when performing model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of a flow chart of an interactive model performance improvement method provided by an embodiment of the present invention;

[0062] Figure 2 A schematic diagram of a process for extracting a target medical vector provided by an embodiment of the present invention;

[0063] Figure 3 A schematic diagram of a process for generating a medical text description provided by an embodiment of the present invention;

[0064] Figure 4 A functional module diagram of an interactive model performance improvement device provided by an embodiment of the present invention;

[0065] Figure 5 A schematic diagram of the structure of an electronic device for implementing the interaction-based model performance improvement method provided in one embodiment of the present invention.

[0066] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0067] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0068] The embodiment of the present application provides a method for improving model performance based on interaction. The execution subject of the interactive model performance improvement method includes but is not limited to at least one of the electronic devices such as the server, the terminal, etc. that can be configured to execute the method provided by the embodiment of the present application. In other words, the interactive model performance improvement method can be executed by software or hardware installed on the terminal device or the server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, etc. The server can be an independent server, or it can be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (Content Delivery Network, CDN), and big data and artificial intelligence platforms.

[0069] Reference Figure 1 FIG. 1 is a flow chart of an interactive model performance improvement method according to an embodiment of the present invention. In this embodiment, the interactive model performance improvement method includes:

[0070] S1. Select medical images in the medical atlas one by one as target medical images, use the encoder model in the proxy model to encode the target medical images into target medical vectors, and use the text generation model of the proxy model to encode the target medical vectors into medical text descriptions.

[0071] In an embodiment of the present invention, the medical atlas is an atlas containing multiple medical images, each of which refers to various image types collected, processed and applied in the medical field, such as X-ray films, CT scan images (Computed Tomography, CT for short), MRI images (Magnetic Resonance Imaging, MRI for short), ultrasound images and nuclear magnetic resonance imaging (Nuclear Magnetic Resonance, NMR for short), etc.

[0072] In detail, the proxy model consists of an encoder model, a text generation model and a text parsing model. The encoder model is used to extract medical features of medical images, the text generation model is used to generate medical questions describing the medical images based on the medical features, and the text parsing model is used to parse the user's answers to the medical images and the medical questions.

[0073] In detail, the target medical vector is a vector containing multiple features of the medical image, such as tissue density features, texture features, shape features and tumor marker features in the medical image. The tissue density features are used to distinguish tissues such as bones, muscles and fat. The texture features are used to describe the smoothness of the tissue, grayscale changes and texture patterns. The shape features are used to describe the size and contour shape of tissues and organs. The tumor marker features are used to detect and analyze tumor markers.

[0074] Specifically, the encoder model can be an information maximizing variational autoencoder (InfoVAE for short), which is an improved version of the traditional variational autoencoder.

[0075] In the embodiment of the present invention, refer to Figure 2 As shown, the encoder model in the proxy model is used to encode the target medical image into a target medical vector, including:

[0076] S21, using the encoder model in the proxy model to sequentially perform multi-level convolution and multi-level pooling operations on the target medical image to obtain target medical features;

[0077] S22, performing feature flattening and full connection operations on the target medical features in sequence to obtain target feature codes;

[0078] S23, performing latent variable analysis on the target feature coding to obtain potential distribution features;

[0079] S24. Re-parameter sampling is performed on the target potential feature according to the potential distribution feature to obtain a target medical vector.

[0080] In detail, the multiple convolutional layers and pooling layers of the encoder model can be used to perform multi-level convolution and multi-level pooling operations on the target medical image in sequence to obtain target medical features. The tf.reshape function of TensorFlow or the view function in PyTorch can be used to perform feature flattening operations on the target medical features. The fully connected layer can be used to perform fully connected operations on the target medical features.

[0081] Specifically, the latent variable distribution layer (Latent Distribution Layer) of the encoder model can be used to perform latent variable analysis on the target feature encoding to obtain latent distribution characteristics, which include latent variables, means and variances of latent variables. Reparameter sampling of the target latent features according to the latent distribution characteristics to obtain a target medical vector refers to extracting a latent vector from the target latent features as a target medical vector according to the latent distribution characteristics using the reparameterization technique.

[0082] In detail, the text generation model can be a recurrent neural network model (RNN) or a Transformer model. The recurrent neural network model can capture long-term dependencies in sequence data by introducing recurrent connections in the network, allowing information to be transmitted between different time steps. The Transformer model can achieve parallel computing by using a self-attention mechanism to capture dependencies in the sequence.

[0083] In detail, the medical text description is a descriptive text question for the target medical image, such as "is there a tumor in the image", "which part of the human body does the image represent", etc.

[0084] For details, refer to Figure 3 As shown, the text generation model using the proxy model encodes the target medical vector into a medical text description, including:

[0085] S31, using the text generation model of the proxy model to add a position code to the target medical vector code to obtain a position medical code;

[0086] S32, using a self-attention mechanism to perform attention encoding on the position medical code to obtain an attention medical code;

[0087] S33, performing multi-level residual connection and full connection operations on the attention medical code to obtain a standard medical code;

[0088] S34. Perform a linear activation operation on the standard medical code to obtain a medical text description.

[0089] In an embodiment of the present invention, the position code is a code used to record the position information of each feature in the target medical vector code. The sine function and cosine function can be used to add the position code to the target medical vector code to obtain the position medical code. The self-attention mechanism is a mechanism for capturing the relationship between each position in a sequence.

[0090] In detail, the residual connection is a technology used to strengthen neural network training and solve the degradation problem of deep networks, and the linear activation is a common neural network activation function.

[0091] In an embodiment of the present invention, by selecting medical images in a medical atlas as target medical images one by one, using the encoder model in the proxy model to encode the target medical images into target medical vectors, and using the text generation model of the proxy model to encode the target medical vectors into medical text descriptions, descriptive questions can be generated for the medical images, thereby facilitating subsequent interaction with users.

[0092] S2. Publishing the target medical image and the medical text as a target medical question to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal.

[0093] In an embodiment of the present invention, the target medical problem refers to a graphic problem including the target medical picture and the medical text, the user terminal may be a medical social network group used to communicate and answer some medical questions, and the target problem feedback refers to the interaction data recorded by the user terminal with each user on the target medical text.

[0094] In the embodiment of the present invention, obtaining the target problem feedback corresponding to the target medical problem from the user terminal refers to using a keyword retrieval method to retrieve the record log data of the target medical problem from the user terminal according to the keyword of the target medical problem.

[0095] In an embodiment of the present invention, by publishing the target medical image and the medical text as target medical problems to the user terminal and obtaining target problem feedback corresponding to the target medical problem from the user terminal, human-computer interaction can be achieved, further expanding the testing and training accuracy of the model and improving the fault tolerance of the model.

[0096] S3. Extract target interaction information and target interaction text from the target question feedback, and calculate a target agent reward using the agent model and the target interaction information.

[0097] In an embodiment of the present invention, the target interaction information refers to the interaction data of each user for the target medical problem, such as answer data and ignore data within a limited time, and the target interaction text refers to the user's interactive answer text for the target medical problem.

[0098] Specifically, the target interaction information and target interaction text can be extracted from the target question feedback using a keyword matching method or a data format matching method. For example, the target interaction information is log data, the data format is timestamp data, and the target interaction text is text data, the data format is text data.

[0099] In an embodiment of the present invention, the target proxy reward is calculated using the proxy model and the target interaction information, including: extracting interaction frequency features and interaction result features from the target interaction information respectively; splicing the interaction frequency features and the interaction result features into target interaction features, and extracting the target interaction reward corresponding to the target interaction features using a preset dichotomizer model; extracting the image confidence corresponding to the target interaction information from the proxy model, and using the image confidence as the target knowledge reward; and calculating the target proxy reward of the proxy model according to the target interaction reward and the target knowledge reward using the following balanced reward formula:

[0100] R=α·R interaction +(1-α)·R knowledge ,0≤α≤1

[0101] Where R refers to the target agent reward, α is the confrontation coefficient preset in the balance reward formula, and R interaction refers to the target interaction reward, R knowledge refers to the target knowledge reward.

[0102] In detail, the target interaction reward is an activity coefficient reward for interactive data, and the target knowledge reward is a coefficient reward for the difficulty and importance of the target medical problem.

[0103] In detail, the interaction frequency feature refers to the number of interactions between the user and the target medical problem per unit time in the target interaction information, and the interaction result feature refers to the result when the user interacts with the target medical problem, including no reply, posting a reply, and likes.

[0104] Specifically, the bipartite model can be a support vector machine (SVM) or a decision tree model, the target interaction reward is a coefficient between 0 and 1, the interaction keyword refers to the section keyword selected by the user during interaction, such as tumor, cancer, etc., and generating the interaction weight according to the interaction keyword refers to weight mapping the interaction keyword to obtain the interaction weight.

[0105] Specifically, the image confidence corresponding to the target interaction information can be extracted from the proxy model using a Bayesian neural network or a Monte Carlo method, wherein the Bayesian neural network is a prior distribution and the Monte Carlo distribution method is a posterior distribution.

[0106] In an embodiment of the present invention, by extracting target interaction information and target interaction text from the target question feedback, and calculating the target agent reward using the agent model and the target interaction information, the target agent reward can be used to measure the effectiveness of each interaction with the user, thereby improving the interactivity of the model.

[0107] S4. Perform part-of-speech tagging, entity recognition and syntactic analysis on the target interactive text in sequence to obtain a target annotated text, and use the text parsing model of the proxy model to perform semantic analysis on the target annotated text to obtain a target medical answer.

[0108] In the embodiment of the present invention, the target annotated text refers to the target interactive text annotated with part of speech, structure, entity and syntax. The target annotated text can facilitate the computer to understand the interactive text information of the user.

[0109] In an embodiment of the present invention, the target interactive text is sequentially subjected to part-of-speech tagging, entity recognition, and syntactic analysis operations to obtain a target labeled text, including: sequentially performing text segmentation and stop word filtering operations on the target interactive text to obtain an interactive word sequence; performing vectorization operations on the interactive word sequence to obtain an interactive word vector sequence; performing part-of-speech encoding on the interactive word vector sequence to obtain an interactive part-of-speech feature set; performing entity detection on the interactive word vectors in the interactive word vector sequence one by one to obtain an entity word vector set; performing graph convolution operations on the interactive word vector sequence to obtain interactive syntactic features; splicing the interactive word sequence into a primary interactive text, performing part-of-speech mapping on the primary interactive text using the interactive part-of-speech feature set to obtain a part-of-speech interactive text; performing entity tagging on the part-of-speech interactive text using the entity word vector set to obtain an entity interactive text; performing syntactic tagging on the entity interactive text using the interactive syntactic features to obtain a target labeled text.

[0110] In detail, the target interactive text may be segmented using a bidirectional maximum matching method or a forward maximum matching method, and the target interactive text may be filtered for stop words using a stop word list method to obtain an interactive word sequence.

[0111] The interactive word sequence can be vectorized using one-hot encoding or Word2vec encoding to obtain an interactive word vector sequence. The interactive word vector sequence can be part-of-speech encoded using a hidden Markov model or a conditional random field model to obtain an interactive part-of-speech feature set. The interactive word vectors in the interactive word vector sequence can be subjected to entity detection one by one using a bidirectional long short-term memory network or a Transformer model to obtain an entity word vector set. The interactive word vector sequence can be subjected to a graph convolution operation using a graph convolutional network (GCN) to obtain interactive syntactic features.

[0112] In detail, the text parsing model of the proxy model is used to perform semantic analysis on the target annotated text to obtain a target medical answer, including: using the text parsing model of the proxy model to perform semantic role annotation on the target standard text to obtain a primary semantic text; using entity association to extract semantic relations from the primary semantic text; converting the primary semantic text into semantic features based on the semantic relations; and activating and transcoding the semantic features to obtain a target medical answer.

[0113] In detail, semantic role annotation can be performed on word pairs with parts of speech marked in the target standard text to obtain primary semantic text, for example, words with verb parts of speech can be marked as predicates, arguments can be matched for each predicate, and semantic roles can be matched for each argument.

[0114] Specifically, the semantic relations include "belonging to relationship", "located in relationship" and "includes relationship", etc. The primary semantic text can be converted into semantic features according to the semantic relations using relational embedding models or graph neural networks, and the semantic features can be activated and transcoded using a trained Transformer model or a recurrent neural network model to obtain target medical answers.

[0115] In an embodiment of the present invention, the target interactive text is sequentially subjected to part-of-speech tagging, entity recognition, and syntactic analysis operations to obtain a target annotated text, and the target annotated text is semantically analyzed using the text parsing model of the proxy model to obtain a target medical answer. This can assist the computer in understanding the user's reply to the target medical question, expand the training data set, and improve the accuracy of the model.

[0116] S5. The target medical answer, the target medical image, the medical text and the target agent reward are collected into a target training data group, and the agent model is iteratively trained using all the target training data groups to obtain a standard agent model, thereby ending the model training.

[0117] In an embodiment of the present invention, the iterative training of the proxy model using all target training data groups to obtain a standard proxy model includes: aggregating all target training data groups into a training data set; aggregating the target proxy rewards of each target training data group in the training data set into a proxy reward set; performing threshold filtering on the proxy reward set to obtain a standard proxy reward set; screening out target training data groups corresponding to the standard proxy reward set from the training data set and aggregating them into a standard training data set; and iteratively training the proxy model using the standard training data set to obtain a standard proxy model.

[0118] In detail, the threshold filtering of the proxy reward set to obtain the standard proxy reward set refers to adding the target proxy rewards in the proxy reward set whose values ​​are greater than a preset reward threshold to the standard proxy reward set, and the proxy model can be iteratively trained using the standard training data set using a stochastic gradient descent algorithm or a fast gradient descent algorithm to obtain a standard proxy model.

[0119] In an embodiment of the present invention, the target medical answer, the target medical image, the medical text and the target agent reward are collected into a target training data group, and the agent model is iteratively trained using all the target training data groups to obtain a standard agent model. The type and size of the data set can be expanded in combination with user feedback, and random events can be added to the data set, thereby automatically improving the accuracy and training speed of the model, thereby improving the model performance.

[0120] The embodiment of the present invention selects medical images in a medical atlas as target medical images one by one, uses an encoder model in a proxy model to encode the target medical images into target medical vectors, and uses a text generation model of the proxy model to encode the target medical vectors into medical text descriptions. Descriptive questions can be generated for medical images to facilitate subsequent interaction with users. By publishing the target medical images and the medical text as target medical questions to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal, human-computer interaction can be achieved, the testing and training accuracy of the model can be further expanded, and the fault tolerance of the model can be improved. By extracting target interaction information and target interaction text from the target question feedback, and using the proxy model and the target interaction information to calculate the target proxy reward, the target proxy reward can be used to measure the effectiveness of each interaction with the user, thereby improving the interactivity of the model.

[0121] By sequentially performing part-of-speech tagging, entity recognition, and syntactic analysis operations on the target interactive text, a target annotated text is obtained, and a text parsing model of the proxy model is used to perform semantic analysis on the target annotated text to obtain a target medical answer, which can assist the computer in understanding the user's reply to the target medical question, expand the training data set, and improve the accuracy of the model. By aggregating the target medical answer, the target medical image, the medical text, and the target proxy reward into a target training data group, the proxy model is iteratively trained using all target training data groups to obtain a standard proxy model, and the type and size of the data set can be expanded in combination with user feedback, and random events can be added to the data set, thereby automatically improving the accuracy and training speed of the model, thereby improving the model performance. Therefore, the interactive model performance improvement method proposed in the present invention can solve the problem of low efficiency when performing model training.

[0122] like Figure 4 , which is a functional module diagram of an interactive model performance improvement device provided by an embodiment of the present invention.

[0123] The interactive model performance improvement device 100 of the present invention can be installed in an electronic device. According to the functions to be implemented, the interactive model performance improvement device 100 can include a question generation module 101, a feedback acquisition module 102, an incentive calculation module 103, a semantic analysis module 104 and a model training module 105. The module of the present invention can also be called a unit, which refers to a series of computer program segments that can be executed by an electronic device processor and can complete fixed functions, which are stored in the memory of the electronic device.

[0124] In this embodiment, the functions of each module / unit are as follows:

[0125] The question generation module 101 is used to select medical images in the medical atlas one by one as target medical images, encode the target medical images into target medical vectors using the encoder model in the proxy model, and encode the target medical vectors into medical text descriptions using the text generation model of the proxy model;

[0126] The feedback acquisition module 102 is used to publish the target medical image and the medical text as a target medical question to the user terminal, and obtain target question feedback corresponding to the target medical question from the user terminal;

[0127] The incentive calculation module 103 is used to extract target interaction information and target interaction text from the target question feedback, and calculate the target agent reward using the agent model and the target interaction information;

[0128] The semantic analysis module 104 is used to perform part-of-speech tagging, entity recognition and syntactic analysis operations on the target interactive text in sequence to obtain a target annotated text, and perform semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer;

[0129] The model training module 105 is used to collect the target medical answer, the target medical image, the medical text and the target agent reward into a target training data group, use all the target training data groups to iteratively train the agent model, obtain a standard agent model, and end the model training.

[0130] In detail, each module described in the interactive model performance improvement device 100 in the embodiment of the present invention is used in the same manner as described above. Figures 1 to 3 The technical means are the same as the interactive model performance improvement method described in , and can produce the same technical effects, so they will not be repeated here.

[0131] like Figure 5 , is a schematic diagram of the structure of an electronic device for implementing an interactive-based model performance improvement method provided by an embodiment of the present invention.

[0132] The electronic device 1 may include a processor 10, a memory 11, a communication bus 12, and a communication interface 13, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an interactive model performance enhancement program.

[0133] In some embodiments, the processor 10 may be composed of an integrated circuit, for example, a single packaged integrated circuit, or a plurality of packaged integrated circuits with the same or different functions, including one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and combinations of various control chips. The processor 10 is the control core (ControlUnit) of the electronic device, and uses various interfaces and lines to connect various components of the entire electronic device, and executes or executes programs or modules stored in the memory 11 (for example, executing interactive model performance improvement programs, etc.), and calls data stored in the memory 11 to execute various functions of the electronic device and process data.

[0134] The memory 11 includes at least one type of readable storage medium, and the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (for example, SD or DX memory, etc.), a magnetic memory, a disk, an optical disk, etc. In some embodiments, the memory 11 may be an internal storage unit of an electronic device, such as a mobile hard disk of the electronic device. In other embodiments, the memory 11 may also be an external storage device of an electronic device, such as a plug-in mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device. Further, the memory 11 may also include both an internal storage unit of the electronic device and an external storage device. The memory 11 can not only be used to store application software and various types of data installed in the electronic device, such as the code of an interactive model performance improvement program, but also can be used to temporarily store data that has been output or is to be output.

[0135] The communication bus 12 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is configured to realize connection and communication between the memory 11 and at least one processor 10, etc.

[0136] The communication interface 13 is used for communication between the above-mentioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is generally used to establish a communication connection between the electronic device and other electronic devices. The user interface may be a display (Display), an input unit (such as a keyboard (Keyboard)), and optionally, the user interface may also be a standard wired interface, a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, and an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. Among them, the display may also be appropriately referred to as a display screen or a display unit, which is used to display information processed in the electronic device and to display a visual user interface.

[0137] The figure only shows an electronic device with components. Those skilled in the art will understand that the structure shown in the figure does not constitute a limitation on the electronic device, and may include fewer or more components than shown in the figure, or combine certain components, or arrange the components differently.

[0138] For example, although not shown, the electronic device may also include a power source (such as a battery) for supplying power to each component. Preferably, the power source may be logically connected to the at least one processor 10 through a power management device, so that the power management device can realize functions such as charging management, discharging management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include a variety of sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

[0139] It should be understood that the embodiment is for illustration only and the scope of the patent application is not limited to this structure.

[0140] Selecting medical images in the medical atlas one by one as target medical images, encoding the target medical images into target medical vectors using the encoder model in the proxy model, and encoding the target medical vectors into medical text descriptions using the text generation model of the proxy model;

[0141] Publishing the target medical image and the medical text as a target medical question to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal;

[0142] Extracting target interaction information and target interaction text from the target question feedback, and calculating a target agent reward using the agent model and the target interaction information;

[0143] Performing part-of-speech tagging, entity recognition, and syntactic analysis operations on the target interactive text in sequence to obtain a target annotated text, and performing semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer;

[0144] The target medical answer, the target medical image, the medical text and the target agent reward are collected into a target training data group, and the agent model is iteratively trained using all the target training data groups to obtain a standard agent model, thereby ending the model training.

[0145] Specifically, the specific implementation method of the processor 10 for the above instructions can refer to the description of the relevant steps in the corresponding embodiment of the accompanying drawings, which will not be repeated here.

[0146] Furthermore, if the module / unit integrated in the electronic device 1 is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile. For example, the computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a mobile hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (ROM).

[0147] The present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor of an electronic device, the computer program can implement:

[0148] Selecting medical images in the medical atlas one by one as target medical images, encoding the target medical images into target medical vectors using the encoder model in the proxy model, and encoding the target medical vectors into medical text descriptions using the text generation model of the proxy model;

[0149] Publishing the target medical image and the medical text as a target medical question to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal;

[0150] Extracting target interaction information and target interaction text from the target question feedback, and calculating a target agent reward using the agent model and the target interaction information;

[0151] Performing part-of-speech tagging, entity recognition, and syntactic analysis operations on the target interactive text in sequence to obtain a target annotated text, and performing semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer;

[0152] The target medical answer, the target medical image, the medical text and the target agent reward are collected into a target training data group, and the agent model is iteratively trained using all the target training data groups to obtain a standard agent model, thereby ending the model training.

[0153] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.

[0154] The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0155] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional modules.

[0156] It is obvious to those skilled in the art that the present invention is not limited to the details of the above exemplary embodiments, and that the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present invention.

[0157] Therefore, no matter from which point of view, the embodiments should be regarded as illustrative and non-restrictive, and the scope of the present invention is limited by the appended claims rather than the above description, so it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the present invention. Any attached figure mark in the claims should not be regarded as limiting the claims involved.

[0158] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0159] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0160] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The multiple units or devices stated in the system embodiment can also be implemented by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

[0161] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present invention.

Claims

1. A method for improving model performance based on interaction, characterized in that: The method comprises: Selecting medical images in the medical atlas one by one as target medical images, encoding the target medical images into target medical vectors using the encoder model in the proxy model, and encoding the target medical vectors into medical text descriptions using the text generation model of the proxy model; Publishing the target medical image and the medical text as a target medical question to a user terminal, and obtaining target question feedback corresponding to the target medical question from the user terminal; Extracting target interaction information and target interaction text from the target question feedback, and calculating a target agent reward using the agent model and the target interaction information; Performing part-of-speech tagging, entity recognition, and syntactic analysis on the target interactive text in sequence to obtain a target annotated text, and performing semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer; The target medical answer, the target medical image, the medical text and the target agent reward are collected into a target training data group, and the agent model is iteratively trained using all the target training data groups to obtain a standard agent model, thereby ending the model training.

2. The interactive model performance improvement method according to claim 1, characterized in that: The step of encoding the target medical image into a target medical vector by using an encoder model in the proxy model includes: Using the encoder model in the proxy model to sequentially perform multi-level convolution and multi-level pooling operations on the target medical image to obtain target medical features; Performing feature flattening and full connection operations on the target medical features in sequence to obtain target feature encoding; Performing latent variable analysis on the target feature encoding to obtain latent distribution features; The target latent feature is re-parameterized according to the latent distribution feature to obtain a target medical vector.

3. The interactive model performance improvement method according to claim 1, characterized in that: The step of encoding the target medical vector into a medical text description using the text generation model of the proxy model includes: Using the text generation model of the proxy model, adding a position code to the target medical vector code to obtain a position medical code; Performing attention encoding on the position medical code by using a self-attention mechanism to obtain an attention medical code; Performing multi-level residual connection and full connection operations on the attention medical code to obtain a standard medical code; A linear activation operation is performed on the standard medical coding to obtain a medical text description.

4. The interactive model performance improvement method according to claim 1, characterized in that: The calculating a target proxy reward by using the proxy model and the target interaction information includes: extracting interaction frequency features and interaction result features from the target interaction information respectively; The interaction frequency feature and the interaction result feature are spliced ​​into a target interaction feature, and a target interaction reward corresponding to the target interaction feature is extracted using a preset bisection model; Extracting the image confidence corresponding to the target interaction information from the proxy model, and using the image confidence as the target knowledge reward; The target agent reward of the agent model is calculated according to the target interaction reward and the target knowledge reward using the following balanced reward formula: R=α·R interaction +(1-a)·R knowledge ,0≤α≤1 Where R refers to the target agent reward, α is the confrontation coefficient preset in the balance reward formula, and R interaction refers to the target interaction reward, R knowledge refers to the target knowledge reward.

5. The interactive model performance improvement method according to claim 1, characterized in that: The step of sequentially performing part-of-speech tagging, entity recognition, and syntactic analysis on the target interactive text to obtain a target annotated text includes: Performing text segmentation and stop word filtering operations on the target interactive text in sequence to obtain an interactive word sequence; Performing a vectorization operation on the interactive word sequence to obtain an interactive word vector sequence; Performing part-of-speech encoding on the interactive word vector sequence to obtain an interactive part-of-speech feature set; Performing entity detection on the interactive word vectors in the interactive word vector sequence one by one to obtain an entity word vector set; Performing a graph convolution operation on the interactive word vector sequence to obtain interactive syntactic features; splicing the interactive word sequences into primary interactive text, and performing part-of-speech mapping on the primary interactive text using the interactive part-of-speech feature set to obtain a part-of-speech interactive text; Using the entity word vector set to perform entity tagging on the part-of-speech interaction text to obtain entity interaction text; The entity interaction text is syntactically annotated using the interaction syntactic features to obtain a target annotated text.

6. The interactive model performance improvement method according to claim 1, characterized in that: The method of using the text parsing model of the proxy model to perform semantic analysis on the target annotated text to obtain a target medical answer includes: Using the text parsing model of the proxy model to perform semantic role labeling on the target standard text to obtain a primary semantic text; Extracting semantic relations from the primary semantic text by using entity association; converting the primary semantic text into semantic features according to the semantic relationship; The semantic features are activated and transcoded to obtain the target medical answer.

7. The interactive model performance improvement method according to claim 1, characterized in that: The iterative training of the proxy model using all target training data sets to obtain a standard proxy model includes: Gather all target training data sets into a training data set; Aggregating the target proxy rewards for each target training data set in the training data set into a proxy reward set; Performing threshold filtering on the proxy reward set to obtain a standard proxy reward set; Filtering the target training data set corresponding to the standard proxy reward set from the training data set to form a standard training data set; The proxy model is iteratively trained using the standard training data set to obtain a standard proxy model.

8. A model performance improvement device based on interaction, characterized in that: The device comprises: A question generation module, used for selecting medical images in the medical atlas one by one as target medical images, encoding the target medical images into target medical vectors using the encoder model in the proxy model, and encoding the target medical vectors into medical text descriptions using the text generation model of the proxy model; A feedback acquisition module, used for publishing the target medical image and the medical text as a target medical question to a user terminal, and acquiring target question feedback corresponding to the target medical question from the user terminal; An incentive calculation module, used to extract target interaction information and target interaction text from the target question feedback, and calculate a target agent reward using the agent model and the target interaction information; A semantic analysis module, used to perform part-of-speech tagging, entity recognition and syntactic analysis operations on the target interactive text in sequence to obtain a target annotated text, and perform semantic analysis on the target annotated text using the text parsing model of the proxy model to obtain a target medical answer; The model training module is used to collect the target medical answer, the target medical image, the medical text and the target agent reward into a target training data group, use all the target training data groups to iteratively train the agent model, obtain a standard agent model, and end the model training.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the interaction-based model performance improvement method as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the interaction-based model performance improvement method as described in any one of claims 1 to 7 is implemented.

Images