Method for checking necessary questions for asking questions based on lora fine tuning model

Abstract

The invention discloses a method for checking and asking necessary questions based on a lora fine tuning model, which comprises the following steps of: firstly, extracting a question data set marked as unnecessary questions from a database, and then respectively carrying out semantic analysis on questions asked by law enforcement officers and each unnecessary question in the question data set by adopting the lora fine tuning model; according to the method and the device, the question asking accuracy and the work efficiency of the law enforcement officers can be effectively improved, unnecessary repeated labor is reduced, and the efficiency and the quality of law enforcement and case handling are improved.

Description

Technical Field

[0001] The present application relates to the field of intelligent verification, and more specifically, to a method for verifying and asking necessary questions based on a LoRa fine-tuning model. Background Art

[0002] With the development of information technology and the increasing demand for social management, intelligent technology has been widely used in the field of public security management. Especially in the daily work of law enforcement officers, how to improve work efficiency and reduce unnecessary repetitive work has become one of the urgent problems to be solved. In the process of case investigation, law enforcement officers often need to communicate with various groups of people and ask a series of questions to obtain key information. However, in actual operations, due to the high intensity of work and heavy tasks of law enforcement officers, sometimes some questions that are not absolutely necessary may be asked, which not only wastes precious time resources, but also may affect the efficiency and quality of case handling.

[0003] Traditional verification methods usually rely on manual experience and judgment. Although this method is intuitive, it is highly subjective and difficult to ensure consistency. Especially when faced with large amounts of data, the efficiency and accuracy of manual verification will be affected.

[0004] With the continuous advancement of artificial intelligence technology in recent years, it has become possible to use machine learning and natural language processing technology to automatically check the questions asked by law enforcement officers. Therefore, a method based on the LoRa fine-tuning model to check the necessary questions is needed to achieve automatic and intelligent checking of the questions asked by law enforcement officers. Summary of the invention

[0005] In order to solve the above technical problems, this application is proposed. The embodiment of this application provides a method for checking necessary questions based on the Lora fine-tuning model. First, a question data set marked as non-essential questions is extracted from the database, and then the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement officers and each non-essential question in the question data set, and then multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement officers are necessary questions, which can effectively improve the accuracy and work efficiency of law enforcement officers' questions, reduce unnecessary repetitive work, and thus improve the efficiency and quality of case handling.

[0006] According to one aspect of the present application, a method for checking and asking necessary questions based on a LoRa fine-tuning model is provided, which includes:

[0007] Extracting a question data set marked as non-essential questions from the database, and obtaining questions to be verified;

[0008] Input the problem to be checked and each problem data marked as non-essential problem in the problem data set marked as non-essential problem into the semantic encoder based on the lora fine-tuning model to obtain a semantic encoding feature vector of the problem to be checked and a set of semantic encoding feature vectors of non-essential problems;

[0009] Performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain a semantic query matching implicit representation vector of the question to be verified;

[0010] Determining whether the question to be checked is a non-essential question based on the semantic query matching implicit representation vector of the question to be checked;

[0011] Among them, performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified includes:

[0012] Performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors;

[0013] The semantic encoding feature vector of the question to be verified is subjected to attention matching query and multi-granularity query feature fusion with the set of semantic encoding feature vectors of the non-essential questions and the set of local neighborhood semantic association feature vectors of the non-essential questions to obtain the semantic query matching implicit representation vector of the question to be verified.

[0014] The present application has at least the following technical effects: Compared with the prior art, the present application provides a method for checking whether necessary questions are asked based on the Lora fine-tuning model. First, a question data set marked as non-essential questions is extracted from the database, and then the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement officers and each non-essential question in the question data set, and then multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement officers are necessary questions. The present application can effectively improve the accuracy and work efficiency of law enforcement officers' questions, reduce unnecessary repetitive work, and thus improve the efficiency and quality of case handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] By describing the embodiments of the present application in more detail in conjunction with the accompanying drawings, the above and other purposes, features and advantages of the present application will become more apparent. The accompanying drawings are used to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the accompanying drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 A flowchart of a method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application;

[0017] Figure 2 A data flow diagram of a method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application;

[0018] Figure 3 This is a flowchart of sub-step S3 of the method for asking necessary questions based on the lora fine-tuning model verification according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0020] As shown in this application and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0021] Although the present application makes various references to certain modules in the system according to the embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are only illustrative, and different aspects of the system and method can use different modules.

[0022] Flowcharts are used in the present application to illustrate the operations performed by the system according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed accurately in order. On the contrary, various steps may be processed in reverse order or simultaneously as required. Meanwhile, other operations may also be added to these processes, or a certain step or several steps of operations may be removed from these processes.

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described here.

[0024] Traditional verification methods usually rely on manual experience and judgment. Although this method is intuitive, it is highly subjective and difficult to ensure consistency. Especially when faced with a large amount of data, the efficiency and accuracy of manual verification will be affected. In recent years, with the continuous advancement of artificial intelligence technology, it has become possible to use machine learning and natural language processing technology to automatically verify the questions asked by law enforcement officers. Therefore, we expect a method based on the Lora fine-tuning model to verify the necessary questions, which can realize the automated and intelligent verification of the questions asked by law enforcement officers.

[0025] In the technical solution of the present application, a method for checking and asking necessary questions based on the lora fine-tuning model is proposed. Figure 1 The present invention is a flowchart of a method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application. Figure 2 The data flow diagram of the method for asking necessary questions based on the Lora fine-tuning model verification according to an embodiment of the present application is as follows. Figure 1 and Figure 2 As shown, according to an embodiment of the present application, a method for checking and asking necessary questions based on a LoRa fine-tuning model includes the following steps: S1, extracting a question data set marked as non-essential questions from a database, and obtaining a question to be verified; S2, respectively inputting the question to be verified and each question data marked as non-essential questions in the question data set marked as non-essential questions into a semantic encoder based on a LoRa fine-tuning model to obtain a set of semantic encoding feature vectors of the question to be verified and semantic encoding feature vectors of non-essential questions; S3, performing a multi-granularity attention matching query on the set of semantic encoding feature vectors of the question to be verified and the semantic encoding feature vectors of the non-essential questions to obtain an implicit representation vector of the semantic query matching of the question to be verified; S4, determining whether the question to be verified is a non-essential question based on the implicit representation vector of the semantic query matching of the question to be verified.

[0026] In particular, S1 extracts a data set of questions marked as non-essential questions from the database, and obtains questions to be verified. The questions to be verified refer to questions raised by law enforcement personnel during the case investigation process. In order to determine whether they meet the prescribed necessity standards, they need to be checked for necessity. This application uses a data set of questions that have been manually marked as non-essential questions as a reference standard to perform a matching analysis on the questions to be verified to determine whether they are non-essential questions. In this way, the deviation of subjective judgment can be effectively avoided, and the objectivity and accuracy of the verification results can be improved.

[0027] In particular, in S2, the problem data of each marked as a non-essential question in the problem data set marked as a non-essential question and the problem data set marked as a non-essential question are respectively input into the semantic encoder based on the lora fine-tuning model to obtain the semantic coding feature vector of the problem to be verified and the set of semantic coding feature vectors of the non-essential question. First, it is necessary to perform semantic analysis on the problem to be verified and the problem data of each marked as a non-essential question to extract their respective content semantic features. In the specific scheme of the present application, a semantic encoder based on the lora fine-tuning model is used to semantically encode the problem data of each marked as a non-essential question in the problem data set marked as a non-essential question to obtain the semantic coding feature vector of the problem to be verified and the set of semantic coding feature vectors of the non-essential question. It should be understood that LoRA (Low-Rank Adaptation) is a method for fine-tuning large pre-trained models. It inserts a pair of additional low-rank matrices between each layer of the pre-trained language model so that these low-rank matrices are learned and updated during the fine-tuning process, while the original pre-trained weights remain unchanged, so that only a small part of the model weights can be updated to reduce computing costs and storage requirements, and improve the performance of the model on specific tasks. In the technical solution of the present application, the LoRA fine-tuning model is used to semantically encode the problem to be verified and the problem data of each problem marked as non-essential problem, which can effectively extract the contextual semantic features of the problem, map each problem data to the same semantic feature space for comparison and matching, and reduce computing costs and storage requirements.

[0028] In particular, the step S3 performs a multi-granular attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified. Figure 3 As shown, S3 includes: S31, performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors; S32, performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the question to be verified, the set of non-essential question semantic encoding feature vectors, and the set of non-essential question local neighborhood semantic association feature vectors to obtain the semantic query matching implicit representation vector of the question to be verified.

[0029] Specifically, the S31 performs local neighborhood implicit optimization on the set of semantic encoding feature vectors of the non-essential questions to obtain a set of local neighborhood semantic association feature vectors of non-essential questions. In a specific example of the present application, the set of semantic encoding feature vectors of the non-essential questions is subjected to one-dimensional convolutional coding to obtain a set of local neighborhood semantic association feature vectors of the non-essential questions, wherein the scale of the one-dimensional convolutional coding is equal to an integer multiple of the length of the semantic encoding feature vector of the non-essential question. Considering that in the problem data set marked as non-essential questions, there may be semantic correlations between each non-essential question. Therefore, in order to further improve the accuracy of query matching between the questions to be verified and the non-essential questions and the accuracy of the verification results, the present application adopts a multi-granularity attention query matching method, which performs local neighborhood implicit optimization on the set of semantic encoding feature vectors of the non-essential questions to enhance the semantic understanding of non-essential questions, thereby achieving more refined semantic matching. Specifically, the set of non-essential question semantic coding feature vectors is first subjected to one-dimensional convolution coding to capture the local neighborhood information in the set and generate a set of non-essential question local neighborhood semantic association feature vectors. The size of the convolution kernel is set to an integer multiple of the length of the non-essential question semantic coding feature vector to ensure that contextual information within a specific range can be captured, thereby better reflecting the semantic association of the local neighborhood. One-dimensional convolution coding is a technology that encodes one-dimensional time series data (such as text, speech, biological signals, etc.) into a fixed-length vector. It uses a one-dimensional convolutional neural network (1D CNN) to extract important features from time series data.

[0030] Specifically, in S32, the semantic encoding feature vector of the question to be verified is subjected to attention matching query and multi-granularity query feature fusion with the set of semantic encoding feature vectors of the non-essential questions and the set of local neighborhood semantic association feature vectors of the non-essential questions to obtain the semantic query matching implicit representation vector of the question to be verified. Specifically, first, the semantic encoding feature vector of the question to be verified is matched with the set of semantic encoding feature vectors of the non-essential questions and the set of semantic association feature vectors of the local neighborhood of the non-essential questions respectively based on the multi-head self-attention mechanism to obtain a set of semantic query implicit vectors of the question to be verified and a set of semantic query implicit vectors of the local neighborhood granularity of the question to be verified; that is, the multi-head attention mechanism of the converter structure is used to perform query matching between the question to be verified and the non-essential question, which constructs the query vector and the value vector based on the semantic encoding feature vector of the question to be verified, and uses the set of semantic encoding feature vectors of the non-essential question and the set of semantic association feature vectors of the local neighborhood of the non-essential question as the set of key vectors respectively, learns the semantic association matching relationship between the question to be verified and the non-essential question from different perspectives, and pays attention to the semantic characteristics of the non-essential question itself and the association characteristics of the non-essential question and its neighborhood, so as to comprehensively consider local and global information and improve the accuracy of query matching. Furthermore, the set of implicit vectors of the semantic query of the problem to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity of the problem to be verified are fused to obtain the implicit representation vector of the semantic query matching of the problem to be verified. Among them, the query matching information of different granularity levels is summarized by calculating the position mean, and the semantic query matching feature vectors of the problem to be verified and the semantic query matching feature vectors of the local neighborhood granularity of the problem to be verified at different granularity levels are generated, and the two are interactively fused in a multi-dimensional semantic way to integrate the information of different granularity levels and generate the semantic query matching implicit representation vector of the problem to be verified, so as to fully reflect the multiple semantic associations between the problem to be verified and the non-essential problems and improve the accuracy of the verification results.

[0031] Among them, the process of performing query matching based on the multi-head self-attention mechanism on the semantic coding feature vector of the question to be verified, the set of semantic coding feature vectors of the non-essential questions, and the set of local neighborhood semantic association feature vectors of the non-essential questions to obtain a set of semantic query implicit vectors of the question to be verified and a set of local neighborhood granularity semantic query implicit vectors of the question to be verified includes: using a value embedding coding matrix to process the semantic coding feature vector of the question to be verified to obtain a value feature vector; using the semantic coding feature vector of the question to be verified as a query vector, the value feature vector as a value vector and the semantic coding feature vector of the non-essential question as a query vector; A set of quantities is used as a set of key vectors, and the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the semantic query of the question to be verified; the semantic encoding feature vector of the question to be verified is used as the query vector, the value feature vector is used as the value vector and the set of local neighborhood semantic association feature vectors of the non-essential question is used as a set of key vectors, and the query vector, the value vector and the set of key vectors are input into a second multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the local neighborhood granularity semantic query of the question to be verified.

[0032] More specifically, the process of taking the semantic encoding feature vector of the question to be verified as the query vector, the value feature vector as the value vector and the set of the semantic encoding feature vectors of the non-essential question as the set of key vectors, and inputting the query vector, the value vector and the set of key vectors into the first multi-head attention query module based on the converter structure to obtain the set of implicit vectors of the semantic query of the question to be verified includes: multiplying the query vector by the transposed vector of the key vector and dividing it by the square root of the scale of the key vector to obtain an attention score matrix; and multiplying the attention score matrix by the value vector after passing it through a softmax function to obtain the implicit vector of the semantic query of the question to be verified.

[0033] More specifically, the process of fusing the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity to be verified to obtain the implicit representation vector of the semantic query matching of the question to be verified includes: respectively calculating the positional mean vector of the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity to be verified to obtain the semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified; fusing the semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified to obtain the semantic query matching implicit representation vector of the question to be verified. Among them, fusing the semantic query matching feature vector of the problem to be verified and the local neighborhood granularity semantic query matching feature vector of the problem to be verified to obtain the semantic query matching implicit representation vector of the problem to be verified, including: calculating the position point addition, position point subtraction and position point multiplication between the semantic query matching feature vector of the problem to be verified and the local neighborhood granularity semantic query matching feature vector of the problem to be verified to obtain the first semantic fusion result, the second semantic fusion result and the third semantic fusion result; after cascading the first semantic fusion result, the second semantic fusion result and the third semantic fusion result into a multi-scale semantic fusion feature vector, performing one-dimensional convolution processing and maximum pooling processing on them to obtain the semantic query matching implicit representation vector of the problem to be verified.

[0034] In summary, in the above embodiment, the semantic encoding feature vector of the question to be verified is subjected to attention matching query and multi-granularity query feature fusion with the set of semantic encoding feature vectors of the non-essential question and the set of local neighborhood semantic association feature vectors of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified, including: processing the semantic encoding feature vector of the question to be verified and the set of semantic encoding feature vectors of the non-essential question with the following query matching formula to obtain the semantic query matching implicit representation vector of the question to be verified, wherein the query matching formula is:

[0035] K={k1,k2,...k i ,...k n}

[0036] K′={k1′;k2′;...k i ′;...k m ′}

[0037] K′=Conv 1×l ({k1, k2, ..., k n})

[0038] v v =v q Wq +b q

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] Where K represents the set of semantic encoding feature vectors of the non-essential question, k1, k2, k i and k n are respectively the first, second, i-th and n-th non-essential question semantic encoding feature vectors in the set of non-essential question semantic encoding feature vectors,

[0045] The value of n is the number of the semantic encoding feature vectors of the non-essential question, Conv 1×l (·) represents one-dimensional convolution processing, l represents the length of the one-dimensional convolution kernel, K′ represents the set of semantic association feature vectors of the local neighborhood of the non-essential question, k1, k2, k i ′ and k m are the first, second, lth and mth non-essential question local neighborhood semantic association feature vectors in the set of the non-essential question local neighborhood semantic association feature vectors, respectively, the value of m is the number of the non-essential question local neighborhood semantic association feature vectors, v q represents the semantic encoding feature vector of the question to be verified, W q is the value embedding encoding matrix, b q is the bias term, v v is the value vector, softmax is the normalized exponential function, the value of d is the scale of the key vector, (·) T represents the transpose of a vector, v ki represents the implicit vector of the semantic query of the i-th question to be verified, v ni represents the implicit vector of the local neighborhood granularity semantic query of the lth question to be verified, v k represents the semantic query matching feature vector of the question to be verified, v n represents the semantic query matching feature vector of the local neighborhood granularity of the problem to be verified, ⊙ represents the dot product, Indicates point addition, represents point subtraction, [·,·,·] represents a cascade operation, MaxPool(·) represents a maximum pooling operation, conv1D(·) represents a one-dimensional convolution operation, and V represents the implicit representation vector of the semantic query matching of the question to be verified.

[0046] In other specific examples of the present application, a multi-granularity attention matching query can be performed on the set of semantic encoding feature vectors of the question to be verified and the set of semantic encoding feature vectors of the non-essential question in other ways to obtain an implicit representation vector of the semantic query matching of the question to be verified, for example: using natural language processing (NLP) technology to extract semantic features from the question to be verified and the non-essential question; encoding the semantic features into semantic vectors to represent the meaning and semantic relationship of the question; defining multiple attention heads, each head focusing on a different granularity or aspect of the semantic vector, for example, one attention head may focus on word-level similarity, while another attention head may focus on sentence-level similarity; for each attention head, calculating the attention weight between the semantic vector of the question to be verified and the set of semantic vectors of the non-essential question; using these attention weights to perform weighted summation on the semantic vectors of the non-essential question to obtain an implicit representation vector of the semantic query matching of the question to be verified.

[0047] Specifically, the S4 determines whether the question to be verified is a non-essential question based on the implicit representation vector of the semantic query matching of the question to be verified. In a specific example of the present application, the implicit representation vector of the semantic query matching of the question to be verified is input into a classifier-based question verification module to obtain a verification result, and the verification result is used to indicate whether the question to be verified is a non-essential question. Specifically, first, the implicit representation vector of the semantic query matching of the question to be verified is fully connected encoded using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and then the encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the verification result.

[0048] Here, in the preferred example, since the set of the semantic encoding feature vectors of the question to be verified and the set of the semantic encoding feature vectors of the non-essential question respectively represent the encoded semantic features of the question to be verified and each question data marked as non-essential question in the question data set marked as non-essential question, when they are input into the attention gated query network based on 1D convolution local neighborhood optimization, the difference in semantic feature representation will have different cross-domain attention query weights based on the difference in local neighborhood optimization of the imitation key matrix, so that the implicit representation vector of the semantic query matching of the question to be verified will also have the distribution diversity of the aggregated features across the semantic space. Therefore, when the implicit representation vector of the semantic query matching of the question to be verified is subjected to probabilistic regression through the classifier-based problem verification module, it will affect the accuracy of the regression result.

[0049] That is, considering that the weight matrix during probability regression acts on the implicit representation vector of the semantic query matching the question to be verified, the uncertainty of the weight stimulus parameters of the weight matrix caused by the distribution diversity of the implicit representation vector of the semantic query matching the question to be verified makes the posterior probability density inference of the implicit representation vector of the semantic query matching the question to be verified through the weight matrix missing, which will affect the accuracy of the probability regression results.

[0050] Preferably, inputting the semantic query matching implicit representation vector of the question to be verified into a classifier-based question verification module to obtain a verification result comprises: determining a feature mean and a feature variance of the semantic query matching implicit representation vector of the question to be verified, and dividing the feature mean by the feature variance to obtain a semantic query matching implicit probability statistical value of the question to be verified:

[0051]

[0052] Among them, μ and σ 2 They respectively represent the feature mean and feature variance of the implicit representation vector of the semantic query matching of the question to be verified, and ρ represents the implicit probability statistic value of the semantic query matching of the question to be verified.

[0053] The implicit representation vector of the semantic query matching of the question to be verified is multiplied by the inverse of the maximum eigenvalue in the implicit representation vector of the semantic query matching of the question to be verified to obtain the implicit probability constraint vector of the semantic query matching of the question to be verified:

[0054] V1=V⊙v max -1

[0055] Where V represents the implicit representation vector of the semantic query matching of the question to be verified, v max Indicates that the semantic query of the question to be verified matches the maximum eigenvalue in the implicit representation vector,

[0056] ⊙ represents the point product by position, and V1 represents the implicit probability constraint vector of the semantic query matching of the question to be verified.

[0057] After dot-adding the implicit probability constraint vector of the semantic query matching of the question to be verified and the implicit probability statistical value of the semantic query matching of the question to be verified, the logarithmic value with base 2 is calculated to obtain the implicit information interaction vector of the semantic query matching of the question to be verified:

[0058]

[0059] Among them, V1 represents the implicit probability constraint vector of the semantic query matching of the question to be verified, ρ represents the implicit probability statistical value of the semantic query matching of the question to be verified, Indicates adding by position point.

[0060] V2 represents the implicit information interaction vector of the semantic query matching of the question to be verified.

[0061] After performing point subtraction on the implicit probability constraint vector of the semantic query matching of the question to be verified by using the implicit probability statistical value of the semantic query matching of the question to be verified, the inverse of each eigenvalue is calculated to obtain the implicit sequence constraint vector of the semantic query matching of the question to be verified:

[0062]

[0063] Among them, V1 represents the implicit probability constraint vector of the semantic query matching of the question to be verified, ρ represents the implicit probability statistical value of the semantic query matching of the question to be verified, It represents point-by-point subtraction, and V3 represents the implicit sequence constraint vector matching the semantic query of the question to be verified.

[0064] The semantic query matching implicit information interaction vector of the question to be verified is interpolated with the semantic query matching implicit sequence constraint vector of the question to be verified to obtain an optimized semantic query matching implicit representation vector of the question to be verified;

[0065] The optimized semantic query matching implicit representation vector of the question to be verified is input into a classifier-based question verification module to obtain a verification result.

[0066] Therefore, the mesoscale interaction structure under probability constraints is simulated between the eigenvalue scale and the eigenvector scale of the semantic query matching implicit representation vector of the question to be verified through the probability statistical characteristics of the semantic query matching implicit representation vector of the question to be verified, so as to construct a bidirectional latent variable motif based on the mesoscale short sequence relative to the probability statistical value to perform mesoscale bidirectional migration, and to perform posterior recovery based on the sequence constraints on the interactive information, thereby improving the convergence effect in the probability density domain and improving the accuracy of the verification result obtained by inputting the semantic query matching implicit representation vector of the question to be verified into the problem verification module based on the classifier.

[0067] In summary, according to the embodiment of the present application, the method for checking and asking necessary questions based on the Lora fine-tuning model is explained. First, a question data set marked as non-essential questions is extracted from the database, and then the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement officers and each non-essential question in the question data set, and then the multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement officers are necessary questions. In this way, the accuracy and work efficiency of law enforcement officers' questions can be effectively improved, unnecessary repetitive work can be reduced, and the efficiency and quality of case handling can be improved.

[0068] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for checking and asking necessary questions based on the Lora fine-tuning model, characterized in that: include: Extracting a question data set marked as non-essential questions from the database, and obtaining questions to be verified; Input the problem to be checked and each problem data marked as non-essential problem in the problem data set marked as non-essential problem into the semantic encoder based on the lora fine-tuning model to obtain a semantic encoding feature vector of the problem to be checked and a set of semantic encoding feature vectors of non-essential problems; Performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain a semantic query matching implicit representation vector of the question to be verified; Determining whether the question to be checked is a non-essential question based on the semantic query matching implicit representation vector of the question to be checked; Among them, performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified includes: Performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors; The semantic encoding feature vector of the question to be verified is subjected to attention matching query and multi-granularity query feature fusion with the set of semantic encoding feature vectors of the non-essential questions and the set of local neighborhood semantic association feature vectors of the non-essential questions to obtain the semantic query matching implicit representation vector of the question to be verified.

2. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 1 is characterized in that: Performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors, including: One-dimensional convolution coding is performed on the set of non-essential question semantic encoding feature vectors to obtain the set of non-essential question local neighborhood semantic association feature vectors, wherein the scale of the one-dimensional convolution coding is equal to an integer multiple of the length of the non-essential question semantic encoding feature vector.

3. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 2 is characterized in that: Performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the question to be checked, the set of semantic encoding feature vectors of the non-essential question, and the set of local neighborhood semantic association feature vectors of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be checked, including: Performing query matching based on a multi-head self-attention mechanism on the semantic encoding feature vector of the question to be checked, the set of semantic encoding feature vectors of the non-essential question, and the set of local neighborhood semantic association feature vectors of the non-essential question, respectively, to obtain a set of semantic query implicit vectors of the question to be checked and a set of semantic query implicit vectors of local neighborhood granularity of the question to be checked; The set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the question to be verified are fused to obtain the semantic query matching implicit representation vector of the question to be verified.

4. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 3 is characterized in that: The semantic encoding feature vector of the question to be verified is matched with the set of semantic encoding feature vectors of the non-essential question and the set of local neighborhood semantic association feature vectors of the non-essential question respectively to obtain a set of semantic query implicit vectors of the question to be verified and a set of semantic query implicit vectors of local neighborhood granularity of the question to be verified, including: Using a value embedding coding matrix, the semantic coding feature vector of the question to be checked is processed to obtain a value feature vector; Taking the semantic encoding feature vector of the question to be checked as the query vector, the value feature vector as the value vector and the set of the semantic encoding feature vectors of the non-essential question as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain the set of implicit vectors of the semantic query of the question to be checked; Taking the semantic encoding feature vector of the question to be verified as the query vector, the value feature vector as the value vector and the set of local neighborhood semantic association feature vectors of the non-essential question as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a second multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the local neighborhood granularity semantic query of the question to be verified.

5. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 4 is characterized in that: Taking the semantic encoding feature vector of the question to be checked as the query vector, the value feature vector as the value vector and the set of the semantic encoding feature vectors of the non-essential question as the set of key vectors, the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain the set of implicit vectors of the semantic query of the question to be checked, including: Multiplying the query vector by the transposed vector of the key vector and dividing by the square root of the scale of the key vector to obtain an attention score matrix; The attention score matrix is ​​passed through a softmax function and then multiplied by the value vector to obtain the implicit vector of the semantic query of the question to be verified.

6. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 5 is characterized in that: The set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity of the question to be verified are merged to obtain the semantic query matching implicit representation vector of the question to be verified, including: Respectively calculating the position mean vectors of the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity of the question to be verified to obtain the semantic query matching feature vector of the question to be verified and the semantic query matching feature vector of the local neighborhood granularity of the question to be verified; The semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified are fused to obtain the semantic query matching implicit representation vector of the question to be verified.

7. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 6 is characterized in that: The semantic query matching feature vector of the question to be verified and the semantic query matching feature vector of the local neighborhood granularity of the question to be verified are integrated to obtain the semantic query matching implicit representation vector of the question to be verified, including: Calculate the position point addition, position point subtraction and position point multiplication between the semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified to obtain a first semantic fusion result, a second semantic fusion result and a third semantic fusion result; After the first semantic fusion result, the second semantic fusion result and the third semantic fusion result are cascaded into a multi-scale semantic fusion feature vector, one-dimensional convolution processing and maximum pooling processing are performed on them to obtain the implicit representation vector of the semantic query matching of the question to be verified.

8. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 7 is characterized in that: Determining whether the question to be checked is a non-essential question based on the semantic query matching implicit representation vector of the question to be checked includes: The semantic query matching implicit representation vector of the question to be verified is input into a classifier-based question verification module to obtain a verification result, and the verification result is used to indicate whether the question to be verified is a non-essential question.

9. The method for checking and asking necessary questions based on the lora fine-tuning model according to claim 8 is characterized in that: Inputting the semantic query matching implicit representation vector of the question to be verified into a classifier-based question verification module to obtain a verification result, wherein the verification result is used to indicate whether the question to be verified is a non-essential question, including: Using multiple fully connected layers of the classifier, the semantic query matching implicit representation vector of the question to be verified is fully connected to obtain an encoded classification feature vector; The encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the verification result.

Images