Intention matching method, system and device based on complex network semantics and medium

Abstract

The invention provides an intention matching method, system and device based on complex network semantics and a medium, and the method comprises the steps: collecting a question and answer data set of a demand field, and carrying out cleaning, entity linking and word embedding processing on question and answer data; the BERT model is adjusted and trained based on LoRA, and in each round of training, forward propagation, loss calculation, back propagation and BERT model parameter updating are carried out; obtaining problem data input by a user, performing processing and feature extraction on the problem data, inputting the problem data into the BERT model, and generating an optimal result of intention matching; the BERT model is periodically updated by collecting question and answer data input by a user. According to the method, the BERT model is finely adjusted by using the LoRA technology, so that the model is trained in a specific field, and the semantic intention recognition capability of the model in the specific field can be effectively enhanced.

Description

Technical Field

[0001] The present invention relates to the field of semantic recognition technology, and more specifically to an intention matching method, system, device and medium based on complex network semantics. Background Art

[0002] As enterprises fully embrace the wave of digital transformation, the importance of intelligent question-answering systems as a key bridge connecting users and enterprises and promoting internal communication and collaboration has become increasingly prominent. Empowered by big data and artificial intelligence technologies, especially the rapid development of deep learning models, intelligent question-answering systems have demonstrated unprecedented language processing capabilities, can accurately analyze user questions, and quickly generate relevant answers, greatly improving user experience and enterprise operational efficiency. These systems can not only handle daily, general information queries, but also provide professional knowledge answers in specific scenarios, becoming an accelerator for enterprise digital transformation.

[0003] However, despite the significant progress made in the technology of intelligent question-answering systems, their limitations are gradually becoming apparent in the face of complex and ever-changing practical application scenarios. When dealing with deep knowledge questions and answers in specific fields, due to the professionalism and complexity of domain knowledge, it is often difficult for the system to fully and accurately grasp the essence of the question, resulting in uneven quality of answers. In addition, users may use a variety of language forms when expressing their needs, and even contain complex network semantics and potential intentions, which places higher demands on the understanding ability of intelligent question-answering systems. When dealing with such complex semantics, the current system is still difficult to be completely accurate and sometimes misunderstandings or answers that deviate from the topic may occur. Summary of the invention

[0004] In view of the above problems, the purpose of the present invention is to provide an intent matching method, system, device and medium based on complex network semantics, and use LoRA (Low-Rank Adaptation) technology to fine-tune the BERT model so that the model can be trained in a specific field, which can effectively enhance the model's semantic intent recognition ability in a specific field.

[0005] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0006] In a first aspect, the present invention discloses an intention matching method based on complex network semantics, comprising the following steps:

[0007] S1: Collect question-answering datasets in the demand domain, clean the question-answering data, perform entity linking and word embedding processing;

[0008] S2: Adjust and train the BERT model based on LoRA. In each round of training, forward propagation, loss calculation, backpropagation and update of BERT model parameters are performed;

[0009] S3: Obtain the question data entered by the user, process and extract features from the question data, and input it into the BERT model to generate the best result for intent matching;

[0010] S4: Regularly update the BERT model by collecting question-answering data input by users.

[0011] Further, step S1 comprises:

[0012] Collect question-answering data sets based on known question-answering business content in the required field and external professional knowledge bases, and divide the question-answering data sets into training sets, validation sets, and test sets;

[0013] Identify entities in the text of the question-answering dataset and link the entities to corresponding entries in the professional knowledge base, and add corpus attributes to the corresponding entities in the training set, wherein the corpus attributes include context information, entity type, attributes, and related entities;

[0014] Perform data cleaning on the question and answer data of the question and answer dataset;

[0015] Use word embedding technology to vectorize question and answer data.

[0016] Further, step S2 comprises:

[0017] Import the BERT model, freeze all layers of the BERT model, and set the parameters of the LoRA layer to be trainable; input the question-answering data of the training set into the attention layer of the BERT model, and optimize the model weights by introducing a low-rank matrix into the attention layer of the BERT model;

[0018] Perform multiple rounds of BERT model training, perform forward propagation, calculate loss, back propagate, and update model parameters in each round of training, and save the model after each round of training for the next round of training;

[0019] Use the validation set to evaluate the performance of the adjusted BERT model;

[0020] After training is complete, save the BERT model and configuration file, and deploy the BERT model.

[0021] Furthermore, the method freezes all layers of the BERT model and sets the parameters of the LoRA layer to be trainable, including:

[0022] Loop through each parameter of the BERT model and set the requires_grad attribute of each parameter to False to freeze all layers of the BERT model;

[0023] Make the parameters of the LoRA layer of the BERT model trainable by setting the requires_grad attribute of the LoRA layer parameters to True.

[0024] Furthermore, the method of optimizing the model weight by introducing a low-rank matrix into the attention layer of the BERT model includes:

[0025] Define a weight matrix W of size m×n and decompose it into two low-rank matrices U and V, where the size of U is m×r and the size of V is r×n, where r is a positive integer less than m and n, such that: W≈UV;

[0026] Based on the low-rank matrix, the original attention score matrix S ij Updated to S i ′ j ;

[0027] The original attention score matrix S ij The calculation formula is:

[0028]

[0029] Among them, Q is the query vector in the question-answering data, K is the key vector, and d k is the dimension of the K-bond vector;

[0030] The updated attention score matrix S i ′ j The formula is:

[0031]

[0032] Among them, Q ′ =QU,K ′ =KV;

[0033] Based on the updated attention score matrix S i ′ j Use the softmax function to generate the attention weight matrix A ij ;

[0034] The attention weight matrix A ij The calculation formula is:

[0035]

[0036] The final attention output calculation formula is set as:

[0037] Attention(Q,K,J)=A·J;

[0038] Among them, J is the value vector and A is the attention weight matrix A ij .

[0039] Further, the forward propagation, loss calculation, back propagation and model parameter update include: forward propagation by executing command W_hat=torch.matmul(U,V); wherein torch is a PyTorch library, and the matmul method is used to calculate the matrix product of U and V;

[0040] Calculate the loss by executing the command loss = torch.norm(W-W_hat, p = 'fro'); where the norm method is used to calculate the difference between W and W_hat, and 'fro' uses the Frobenius norm;

[0041] Clear the accumulated gradient in the optimizer by executing the optimizer.zero_grad() command, and perform backpropagation by calling the loss.backward() command to calculate the gradient of the loss function with respect to the model parameters and use it to update the model parameters;

[0042] Update the BERT model parameters using the Adam optimizer.

[0043] Further, step S3 includes:

[0044] Get the question data entered by the user;

[0045] Perform sentence segmentation on the problem data, generate text data, and perform risk detection;

[0046] Split the text data into multiple words and phrases, convert each word into a word embedding vector, and perform sequence conversion to generate input data;

[0047] Load the trained BERT model, input the input data into the BERT model, and obtain the probability distribution of intent categories; the intent category with the highest probability is taken as the output result.

[0048] In a second aspect, the present invention further discloses an intention matching system based on complex network semantics, including: a data preparation module for collecting question and answer data sets in the demand field, and performing cleaning, entity linking and word embedding processing on the question and answer data;

[0049] The model adjustment training module is used to adjust and train the BERT model based on LoRA. In each round of training, forward propagation, loss calculation, back propagation and updating of BERT model parameters are performed;

[0050] The question-answer processing module is used to obtain the question-answer data input by the user, process and extract features of the question-answer data, and input it into the BERT model to generate the best result for intent matching;

[0051] The model update module is used to regularly update the BERT model by collecting question-answering data input by users.

[0052] In a third aspect, the present invention further discloses an intention matching device based on complex network semantics, comprising:

[0053] A memory for storing an intent matching program based on complex network semantics;

[0054] A processor is used to implement the steps of the intention matching method based on complex network semantics as described in any of the above items when executing the intention matching program based on complex network semantics.

[0055] In a fourth aspect, the present invention also discloses a readable storage medium, on which is stored an intention matching program based on complex network semantics. When the intention matching program based on complex network semantics is executed by a processor, the steps of the intention matching method based on complex network semantics as described in any of the above items are implemented.

[0056] Compared with the prior art, the present invention has the following beneficial effects:

[0057] The present invention discloses an intention matching method, system, device and medium based on complex network semantics, which first identifies entities in text and links them to corresponding entries in the knowledge base before data is input into the model, then re-adds the obtained entries to the training set to enrich the training set, and then uses the new training set as model input data for subsequent operations. The present invention can help the model better learn and understand the complex network semantics in a specific field and enhance the model's understanding of the context.

[0058] The present invention adopts LoRA fine-tuning technology, using the pre-processed data in the previous step in the attention layer of the model to redistribute the model weights. This method allows the model to focus on certain key parts of the input sequence, which is particularly useful for understanding user intent.

[0059] The present invention can handle a variety of expressions and contexts, and can accurately understand the user's expression even if it changes, effectively improving the accuracy of identifying the user's true intention. The present invention realizes continuous learning and optimization of the model, and its performance will gradually improve, which can better meet the needs of users.

[0060] The present invention can be used to build a highly accurate, flexible and continuously improving intelligent question-answering system with intent recognition by comprehensively using technologies such as model training, intent recognition, data processing and continuous learning. Such a system can not only improve user satisfaction, but also bring practical value to enterprises, including reducing costs, improving efficiency and enhancing market competitiveness.

[0061] It can be seen that compared with the prior art, the present invention has outstanding substantive features and significant progress, and the beneficial effects of its implementation are also obvious. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0063] Figure 1 It is a method flow chart of a specific implementation mode of the present invention.

[0064] Figure 2 It is a flow chart of step S1 of a specific implementation mode of the present invention.

[0065] Figure 3 It is a flow chart of step S2 of a specific implementation mode of the present invention.

[0066] Figure 4 It is a flow chart of step S3 of a specific implementation mode of the present invention.

[0067] Figure 5 It is a system structure diagram of a specific implementation mode of the present invention.

[0068] In the figure, 1. Data preparation module; 2. Model adjustment and training module; 3. Question and answer processing module; 4. Model update module. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0070] See also Figure 1 As shown, this embodiment provides an intention matching method based on complex network semantics, including the following steps:

[0071] S1: Collect question-answering datasets in the demand domain, clean the question-answering data, perform entity linking and word embedding processing.

[0072] In a specific implementation, this step implements data preparation and preprocessing, including: preprocessing a specific domain question-answering dataset, cleaning the question-answering data, entity linking and word embedding, helping the BERT model to better learn and understand complex network semantics, and enhancing the BERT model's understanding of context.

[0073] S2: Adjust and train the BERT model based on LoRA. In each round of training, forward propagation, loss calculation, backpropagation and update of BERT model parameters are performed.

[0074] In a specific implementation, the BERT model is fine-tuned and trained based on LoRA, and some weights of the model are updated to quickly adapt to new fields, thereby enhancing the model's ability to understand users' specific needs and problem backgrounds in specific fields.

[0075] S3: Obtain the question data entered by the user, process and extract features of the question data, and input it into the BERT model to generate the best result for intent matching.

[0076] In a specific implementation, this step implements user question-answering processing. By processing and extracting features from user input data and inputting them into the model, the best result of intent matching is obtained.

[0077] S4: Regularly update the BERT model by collecting question-answering data input by users.

[0078] In a specific implementation, this step realizes the precipitation of the knowledge base and the regular updating of the model. By collecting the user's input data, the dictionary and phrases are regularly updated.

[0079] As an example, Figure 2 As shown, step S1 discloses a data preparation and preprocessing process, which is as follows:

[0080] First, prepare the data set: collect data sets related to specific fields, which are targeted question-and-answer data in specific fields, as training sets, validation sets, and test sets. The data sources include known question-and-answer business content and external professional knowledge bases.

[0081] Then, entities in the dataset text are identified and linked to corresponding entries in the knowledge base. Contextual information, entity type, attributes, related entities and other attributes are added to the entities in the training set to enrich the corpus of the training set.

[0082] At this point, the data is cleaned, including unifying the format, removing stop words, text segmentation, sentence splitting, word segmentation, etc.

[0083] Finally, word embedding technology is used to vectorize the data so that it can capture the semantic relationship and similarity between texts.

[0084] As an example, Figure 3 As shown, step S2 specifically includes the following sub-steps:

[0085] S201: Import the BERT model, freeze all layers of the BERT model, and set the parameters of the LoRA layer to be trainable.

[0086] Specifically, first import the BERT model, loop through each parameter of the BERT model, and set the requires_grad attribute of each parameter to False to freeze all layers of the BERT model. The specific commands are as follows:

[0087] param.requires_grad=False;

[0088] Among them, param is the model parameter, requires_grad indicates whether the tensor needs to calculate the gradient. When it is set to False, all operations of the tensor will not be tracked and recorded, so its gradient will not be calculated, which is used to freeze the layer.

[0089] Then, the parameters of the LoRA layer of the BERT model are made trainable by setting the requires_grad attribute of the LoRA layer parameters to True.

[0090] The following commands can be used:

[0091] if'lora_'in name:

[0092] param.requires_grad = True

[0093] Among them, name is the model name, param is the model parameter, and requires_grad indicates whether the tensor needs to calculate the gradient. When set to True, the operations of the tensor will be recorded in the calculation graph for training the model.

[0094] S202: Input the question-answering data of the training set into the attention layer of the BERT model, and optimize the model weights by introducing a low-rank matrix into the attention layer of the BERT model.

[0095] Specifically, the preprocessed data is input into the model attention layer, and a low-rank matrix is ​​added to modify the model weights to obtain the final attention output. This method can highlight those more relevant vectors and ignore irrelevant parts, so that the model can focus on specific key parts in the input sequence and better understand user intentions.

[0096] First, define a weight matrix W of size m×n and decompose it into two low-rank matrices U and V, where the size of U is m×r and the size of V is r×n, where r is a positive integer less than m and n, such that: W≈UV.

[0097] Then, the original attention score matrix S is transformed based on the low-rank matrix ij Updated to S i ′ j .

[0098] The original attention score matrix S ij The calculation formula is:

[0099]

[0100] Among them, Q is the query vector in the question-answering data, K is the key vector, and d k is the dimension of the K-bond vector;

[0101] Updated attention score matrix S i ′ j The formula is:

[0102]

[0103] Among them, Q ′ =QU,K ′ =KV.

[0104] At this time, based on the updated attention score matrix S i ′ j Use the softmax function to generate the attention weight matrix A ij .

[0105] Among them, the softmax function is a nonlinear activation function, which is often used to convert a set of real values ​​into a probability distribution. Given a vector z, the softmax function can be defined as follows:

[0106]

[0107] By this formula, the softmax function converts each element into a value between 0 and 1, and the sum of all elements is 1.

[0108] Therefore, for the score matrix S ij Applying the softmax function, we can get the attention weight matrix A ij :

[0109]

[0110] Finally, the weighted sum is calculated using the following formula to get the final attention output:

[0111] Attention(Q,K,J)=A·J;

[0112] Among them, J is the value vector and A is the attention weight matrix A ij .

[0113] S203: Execute multiple rounds of BERT model training. In each round of training, forward propagation, loss calculation, back propagation and model parameter update are performed. After each round of training, the model is saved for the next round of training.

[0114] Specifically, multiple rounds of model training are performed, and forward propagation, loss calculation, back propagation and model parameter update are performed in each round of training. The model is saved after each training and used for the next round of training. Finally, U and V can roughly approximate the original weight matrix W.

[0115] The model training process is as follows:

[0116] 1. Forward propagation:

[0117] Forward propagation is performed by executing the command W_hat = torch.matmul(U,V); where torch is the PyTorch library and the matmul method is used to calculate the matrix product of U and V.

[0118] 2. Calculate the loss:

[0119] Calculate the loss by executing the command loss = torch.norm(W-W_hat,p = 'fro'); the norm method is used to calculate the difference between W and W_hat, and 'fro' uses the Frobenius norm.

[0120] 3. Backpropagation:

[0121] First, clear the accumulated gradients in the optimizer through the command optimizer.zero_grad() to prepare for a new back propagation. Then use the command loss.backward() to execute the back propagation algorithm and calculate the gradient of the loss function relative to all variables that require gradients.

[0122] 4. Use the Adam optimizer to update the model parameters U and V, with a learning rate of 0.001, exponential decay factors for calculating the first-order and second-order moments of the gradient as beta1=0.9 and beta2=0.999, and eps (a small constant added to the denominator to prevent division by zero errors) as the default value 1*e -8 , the weight decays to the default value of 0, and the optimizer is defined as follows:

[0123] optimizer=optim.Adam(model.parameters(),lr=0.001,betas

[0124] =(0.9,0.999),eps=1*e -8 ,weight_decay=0)

[0125] The command to update the model parameters is as follows:

[0126] optimizer.step()

[0127] S204: Use the validation set to evaluate the performance of the adjusted BERT model.

[0128] Specifically, verify performance and evaluation, and evaluate the performance of the fine-tuned model on the validation set to ensure that the model has generalization capabilities.

[0129] S205: After training is completed, save the BERT model and configuration file, and deploy the BERT model.

[0130] Specifically, after training is completed, the model and configuration files are saved and the fine-tuned model is deployed. The fine-tuned model is used as a prediction for downstream tasks.

[0131] As an example, Figure 4 As shown, the specific execution process of step S3 is as follows:

[0132] First, the question data input by the user is obtained, and the question data is preprocessed to generate text data, including sentence segmentation and risk detection of the input content.

[0133] Then, the text data is segmented into multiple words and phrases, each word is converted into a word embedding vector, and the data is converted into a sequence.

[0134] At this point, load the model trained in the previous step, feed the input data into the model, and obtain a probability distribution of the intent category.

[0135] Finally, the category with the highest probability is taken as the output result.

[0136] The present invention provides an intention matching method based on complex network semantics. Before the data is input into the model, the entities in the text are first identified and linked to the corresponding entries in the knowledge base. Then, the obtained entries are added back to the training set to enrich the training set, and the new training set is used as the model input data for subsequent operations. This method can help the model better learn and understand the complex network semantics in a specific field and enhance the model's understanding of the context.

[0137] In addition, this method can be used to build a highly accurate, flexible and continuously improving intelligent question-answering system with intent recognition. Such a system can not only improve user satisfaction, but also bring practical value to enterprises, including reducing costs, improving efficiency and enhancing market competitiveness.

[0138] See also Figure 5 As shown, the present invention also discloses an intention matching system based on complex network semantics, including: a data preparation module 1, a model adjustment training module 2, a question and answer processing module 3 and a model update module 4.

[0139] Data preparation module 1 is used to collect question-answering datasets in the required field, and to clean, entity link, and embed word data.

[0140] Model adjustment training module 2 is used to adjust and train the BERT model based on LoRA. In each round of training, forward propagation, loss calculation, back propagation and updating of BERT model parameters are performed.

[0141] The question and answer processing module 3 is used to obtain the question and answer data input by the user, process and extract features of the question and answer data, input it into the BERT model, and generate the best result for intent matching.

[0142] The model updating module 4 is used to regularly update the BERT model by collecting question and answer data input by users.

[0143] The specific implementation of the intention matching system based on complex network semantics in this embodiment is basically consistent with the specific implementation of the intention matching method based on complex network semantics mentioned above, and will not be repeated here.

[0144] The present invention also discloses an intention matching device based on complex network semantics, comprising a processor and a memory; wherein, when the processor executes the intention matching program based on complex network semantics stored in the memory, it implements the steps of the intention matching method based on complex network semantics as described in any one of the above items.

[0145] Furthermore, the intention matching device based on complex network semantics in this embodiment may also include:

[0146] The input interface is used to obtain the intention matching program based on complex network semantics imported from the outside, and save the obtained intention matching program based on complex network semantics to the memory. It can also be used to obtain various instructions and parameters transmitted by external terminal devices, and transmit them to the processor so that the processor can use the above various instructions and parameters to carry out corresponding processing. In this embodiment, the input interface can specifically include but is not limited to a USB interface, a serial interface, a voice input interface, a fingerprint input interface, a hard disk reading interface, etc.

[0147] The output interface is used to output various data generated by the processor to the terminal device connected thereto, so that other terminal devices connected to the output interface can obtain various data generated by the processor. In this embodiment, the output interface may specifically include but is not limited to a USB interface, a serial interface, etc.

[0148] A communication unit is used to establish a remote communication connection between the intention matching device based on complex network semantics and an external server, so that the intention matching device based on complex network semantics can mount the image file to the external server. In this embodiment, the communication unit may specifically include but is not limited to a remote communication unit based on wireless communication technology or wired communication technology.

[0149] The keyboard is used to obtain various parameter data or instructions input by the user by tapping the keycaps in real time.

[0150] A display is used to display relevant information of the intent matching process based on complex network semantics in real time.

[0151] The mouse can be used to assist users in inputting data and simplify user operations.

[0152] The present invention also discloses a readable storage medium, wherein the readable storage medium includes random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, register, hard disk, removable hard disk, CD-ROM or any other form of storage medium known in the technical field. The readable storage medium stores an intention matching program based on complex network semantics, and when the intention matching program based on complex network semantics is executed by a processor, the steps of the intention matching method based on complex network semantics as described in any of the above items are implemented.

[0153] In summary, the present invention uses LoRA technology to fine-tune the BERT model and allows the model to be trained in a specific field, which can effectively enhance the model's ability to recognize semantic intent in a specific field.

[0154] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. As for the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0155] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0156] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, systems and methods can be implemented in other ways. For example, the system embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of systems or units, which can be electrical, mechanical or other forms.

[0157] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0158] In addition, each functional module in each embodiment of the present invention may be integrated into one processing unit, or each module may exist physically separately, or two or more modules may be integrated into one unit.

[0159] Similarly, each processing unit in each embodiment of the present invention may be integrated into one functional module, or each processing unit may exist physically, or two or more processing units may be integrated into one functional module.

[0160] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0161] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0162] The above is a detailed introduction to the intention matching method, system, device and readable storage medium based on complex network semantics provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An intention matching method based on complex network semantics, characterized in that: The steps include: S1: Collect question-answering datasets in the demand domain, clean the question-answering data, perform entity linking and word embedding processing; S2: Adjust and train the BERT model based on LoRA. In each round of training, forward propagation, loss calculation, backpropagation and update of BERT model parameters are performed; S3: Obtain the question data entered by the user, process and extract features from the question data, and input it into the BERT model to generate the best result for intent matching; S4: Regularly update the BERT model by collecting question-answering data input by users.

2. The intention matching method based on complex network semantics according to claim 1 is characterized in that: The step S1 comprises: Collect question-answering data sets based on known question-answering business content in the required field and external professional knowledge bases, and divide the question-answering data sets into training sets, validation sets, and test sets; Identify entities in the text of the question-answering dataset and link the entities to corresponding entries in the professional knowledge base, and add corpus attributes to the corresponding entities in the training set, wherein the corpus attributes include context information, entity type, attributes, and related entities; Perform data cleaning on the question and answer data of the question and answer dataset; Use word embedding technology to vectorize question and answer data.

3. The intention matching method based on complex network semantics according to claim 1 is characterized in that: The step S2 comprises: Import the BERT model, freeze all layers of the BERT model, and set the parameters of the LoRA layer to be trainable; Input the question-answering data of the training set into the attention layer of the BERT model, and optimize the model weights by introducing a low-rank matrix into the attention layer of the BERT model; Perform multiple rounds of BERT model training, perform forward propagation, calculate loss, back propagate, and update model parameters in each round of training, and save the model after each round of training for the next round of training; Use the validation set to evaluate the performance of the adjusted BERT model; After training is complete, save the BERT model and configuration file, and deploy the BERT model.

4. The intention matching method based on complex network semantics according to claim 3 is characterized in that: The method freezes all layers of the BERT model and sets the parameters of the LoRA layer to be trainable, including: Loop through each parameter of the BERT model and set the requires_grad attribute of each parameter to False to freeze all layers of the BERT model; Make the parameters of the LoRA layer of the BERT model trainable by setting the requires_grad attribute of the LoRA layer parameters to True.

5. The intention matching method based on complex network semantics according to claim 4 is characterized in that: The method of optimizing the model weights by introducing a low-rank matrix into the attention layer of the BERT model includes: Define a weight matrix W of size m×n and decompose it into two low-rank matrices U and V, where the size of U is m×r and the size of V is r×n, where r is a positive integer less than m and n, such that: W≈UV; Based on the low-rank matrix, the original attention score matrix S ij Updated to S i ′ j ; The original attention score matrix S ij The calculation formula is: Among them, Q is the query vector in the question-answering data, K is the key vector, and d k is the dimension of the K-bond vector; The updated attention score matrix S i ′ j The formula is: Among them, Q ′ =QU,K ′ =KV; Based on the updated attention score matrix S i ′ j Use the softmax function to generate the attention weight matrix A ij ; The attention weight matrix A ij The calculation formula is: The final attention output calculation formula is set as: Attention(Q,K,J)=A·J; Among them, J is the value vector and A is the attention weight matrix A ij .

6. The intention matching method based on complex network semantics according to claim 5 is characterized in that: The forward propagation, loss calculation, back propagation and model parameter update include: Forward propagation is performed by executing the command W_hat = torch.matmul(U,V); where torch is the PyTorch library and the matmul method is used to calculate the matrix product of U and V; Calculate the loss by executing the command loss = torch.norm(W-W_hat, p = 'fro'); where the norm method is used to calculate the difference between W and W_hat, and 'fro' uses the Frobenius norm; Clear the accumulated gradient in the optimizer by executing the optimizer.zero_grad() command, and perform backpropagation by calling the loss.backward() command to calculate the gradient of the loss function with respect to the model parameters and use it to update the model parameters; Update the BERT model parameters using the Adam optimizer.

7. The intention matching method based on complex network semantics according to claim 5 is characterized in that: The step S3 comprises: Get the question data entered by the user; Perform sentence segmentation on the problem data, generate text data, and perform risk detection; Split the text data into multiple words and phrases, convert each word into a word embedding vector, and perform sequence conversion to generate input data; Load the trained BERT model, input the input data into the BERT model, and obtain the probability distribution of intent categories; the intent category with the highest probability is taken as the output result.

8. An intention matching system based on complex network semantics, characterized in that: include: The data preparation module is used to collect question-answering datasets in the required fields, and to clean, entity link, and embed the question-answering data. The model adjustment training module is used to adjust and train the BERT model based on LoRA. In each round of training, forward propagation, loss calculation, back propagation and updating of BERT model parameters are performed; The question-answer processing module is used to obtain the question-answer data input by the user, process and extract features of the question-answer data, and input it into the BERT model to generate the best result for intent matching; The model update module is used to regularly update the BERT model by collecting question-answering data input by users.

9. An intention matching device based on complex network semantics, characterized in that: include: A memory for storing an intent matching program based on complex network semantics; A processor, used to implement the steps of the intention matching method based on complex network semantics as described in any one of claims 1 to 7 when executing the intention matching program based on complex network semantics.

10. A readable storage medium, characterized in that: The readable storage medium stores an intention matching program based on complex network semantics, and when the intention matching program based on complex network semantics is executed by a processor, the steps of the intention matching method based on complex network semantics as described in any one of claims 1 to 7 are implemented.

Images