Method for checking and asking necessary questions based on Lora fine-tuning model

Abstract

This application discloses a method for checking whether questions are necessary based on the Lora fine-tuning model. First, a question data set marked as non-essential questions is extracted from a database. Then, the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement personnel and the various non-essential questions in the question data set. Then, multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement personnel are necessary questions. This application can effectively improve the accuracy and work efficiency of law enforcement personnel's questions, reduce unnecessary repetitive work, and thus improve the efficiency and quality of law enforcement and case handling.

Description

Technical Field

[0001] The present application relates to the field of intelligent verification, and more specifically, to a method for asking necessary questions based on a LoRa fine-tuning model. Background Art

[0002] With the development of information technology and the increasing demands of social management, intelligent technologies have been widely applied in public security management. Improving efficiency and reducing unnecessary repetitive work have become pressing issues for law enforcement personnel, particularly in their daily work. During case investigations, law enforcement officers often need to communicate with a variety of people and ask a series of questions to obtain key information. However, in practice, due to the high workload and heavy workload, law enforcement officers may sometimes ask questions that are not absolutely necessary. This not only wastes valuable time and resources but also can affect the efficiency and quality of case handling.

[0003] Traditional verification methods usually rely on manual experience and judgment. Although this method is intuitive, it is highly subjective and difficult to ensure consistency. Especially when faced with large amounts of data, the efficiency and accuracy of manual verification will be affected.

[0004] With the continuous advancement of artificial intelligence technology in recent years, it has become possible to use machine learning and natural language processing technologies to automatically verify the questions asked by law enforcement officers. Therefore, a method based on the LoRa fine-tuning model to verify the necessary questions is needed to achieve automated and intelligent verification of law enforcement officers' questions. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides a method for checking the necessary questions asked based on the Lora fine-tuning model. First, a question data set marked as non-essential questions is extracted from the database. Then, the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement officers and each non-essential question in the question data set. Then, multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement officers are necessary questions. This can effectively improve the accuracy and work efficiency of law enforcement officers' questions, reduce unnecessary repetitive work, and thus improve the efficiency and quality of case handling.

[0006] According to one aspect of the present application, a method for asking necessary questions based on the LoRa fine-tuning model is provided, which includes:

[0007] Extracting a dataset of questions marked as non-essential questions from the database and obtaining questions to be verified;

[0008] Input the question to be verified and each question data marked as non-essential question in the question data set marked as non-essential question into the semantic encoder based on the lora fine-tuning model to obtain a semantic encoding feature vector of the question to be verified and a set of semantic encoding feature vectors of non-essential questions;

[0009] Performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain an implicit representation vector matching the semantic query of the question to be verified;

[0010] Determining whether the question to be verified is a non-essential question based on the semantic query matching implicit representation vector of the question to be verified;

[0011] The step of performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain an implicit representation vector matching the semantic query of the question to be verified includes:

[0012] Performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors;

[0013] The semantic encoding feature vector of the question to be verified is subjected to attention matching query and multi-granularity query feature fusion with the set of semantic encoding feature vectors of non-essential questions and the set of local neighborhood semantic association feature vectors of non-essential questions to obtain the semantic query matching implicit representation vector of the question to be verified.

[0014] This application has at least the following technical effects: Compared with the existing technology, this application provides a method for checking whether necessary questions are asked based on the Lora fine-tuning model. First, a question data set marked as non-essential questions is extracted from the database, and then the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement personnel and each non-essential question in the question data set, and then multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement personnel are necessary questions. This application can effectively improve the accuracy and work efficiency of law enforcement personnel's questions, reduce unnecessary repetitive work, and thus improve the efficiency and quality of case handling. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0016] Figure 1 A flowchart of a method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application;

[0017] Figure 2 A data flow diagram of a method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application;

[0018] Figure 3 This is a flowchart of sub-step S3 of the method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application. DETAILED DESCRIPTION

[0019] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0020] As used in this application and the claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not intended to refer to the singular but may include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0021] Although the present application makes various references to certain modules in the system according to embodiments of the present application, any number of different modules can be used and run on the user terminal and / or server. The modules are illustrative only, and different aspects of the system and method can use different modules.

[0022] Flowcharts are used in this application to illustrate the operations performed by the systems according to the embodiments of the present application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, the various steps may be processed in reverse order or simultaneously, as needed. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0023] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0024] Traditional verification methods typically rely on manual judgment. While intuitive, this approach is highly subjective and difficult to ensure consistency. This is especially true when dealing with large amounts of data, where both efficiency and accuracy are compromised. Recent advances in artificial intelligence (AI) have made it possible to automate the verification of law enforcement officer questions using machine learning and natural language processing. Therefore, a method based on a LoRa-based fine-tuned model to verify necessary questions is desired, enabling automated and intelligent verification of law enforcement officer questions.

[0025] In the technical solution of this application, a method for asking necessary questions based on the Lora fine-tuning model is proposed. Figure 1 This is a flowchart of a method for asking necessary questions based on LoRa fine-tuning model verification according to an embodiment of the present application. Figure 2 This is a data flow diagram of the method for asking necessary questions based on the Lora fine-tuning model verification according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the method for checking and asking necessary questions based on the Lora fine-tuning model according to an embodiment of the present application includes the steps of: S1, extracting a question data set marked as non-essential questions from a database, and obtaining the question to be verified; S2, inputting the question to be verified and each question data marked as non-essential question in the question data set marked as non-essential questions into a semantic encoder based on the Lora fine-tuning model respectively to obtain a semantic encoding feature vector of the question to be verified and a set of semantic encoding feature vectors of non-essential questions; S3, performing a multi-granularity attention matching query on the semantic encoding feature vector of the question to be verified and the set of semantic encoding feature vectors of the non-essential questions to obtain an implicit representation vector of the semantic query matching of the question to be verified; S4, determining whether the question to be verified is a non-essential question based on the implicit representation vector of the semantic query matching of the question to be verified.

[0026] In particular, the S1 extracts a dataset of questions marked as non-essential questions from the database and obtains questions to be verified. The questions to be verified refer to questions raised by law enforcement personnel during the case investigation process. In order to determine whether they meet the prescribed necessity standards, they need to be checked for necessity. This application uses a dataset of questions that have been manually marked as non-essential questions as a reference standard to perform a matching analysis on the questions to be verified to determine whether they are non-essential questions. In this way, the deviation of subjective judgment can be effectively avoided and the objectivity and accuracy of the verification results can be improved.

[0027] In particular, the S2 inputs the problem data to be verified and each problem data marked as non-essential problem in the problem data set marked as non-essential problem into the semantic encoder based on the Lora fine-tuning model to obtain the semantic coding feature vector of the problem to be verified and the set of semantic coding feature vectors of non-essential problem. First, it is necessary to perform semantic analysis on the problem to be verified and each problem data marked as non-essential problem to extract their respective content semantic features. In the specific scheme of the present application, a semantic encoder based on the Lora fine-tuning model is used to semantically encode the problem data to be verified and each problem data marked as non-essential problem in the problem data set marked as non-essential problem to obtain the semantic coding feature vector of the problem to be verified and the set of semantic coding feature vectors of non-essential problem. It should be understood that LoRA (Low-Rank Adaptation) is a method for fine-tuning large pre-trained models. It inserts a pair of additional low-rank matrices between each layer of the pre-trained language model so that these low-rank matrices are learned and updated during the fine-tuning process, while the original pre-trained weights remain unchanged. This allows only a small part of the model's weights to be updated to reduce computational costs and storage requirements, thereby improving the model's performance on specific tasks. In the technical solution of the present application, the LoRA fine-tuning model is used to semantically encode the problem to be verified and the problem data marked as non-essential problems, which can effectively extract the contextual semantic features of the problem, map each problem data to the same semantic feature space for comparison and matching, and reduce computational costs and storage requirements.

[0028] In particular, the S3 performs a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified. In a specific example of this application, Figure 3 As shown, the S3 includes: S31, performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors; S32, performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the question to be verified, the set of non-essential question semantic encoding feature vectors, and the set of non-essential question local neighborhood semantic association feature vectors to obtain the implicit representation vector of the semantic query matching of the question to be verified.

[0029] Specifically, the S31 performs local neighborhood implicit optimization on the set of semantic encoding feature vectors of the non-essential questions to obtain a set of local neighborhood semantic association feature vectors of non-essential questions. In a specific example of the present application, one-dimensional convolution coding is performed on the set of semantic encoding feature vectors of the non-essential questions to obtain a set of local neighborhood semantic association feature vectors of the non-essential questions, wherein the scale of the one-dimensional convolution coding is equal to an integer multiple of the length of the semantic encoding feature vector of the non-essential question. Taking into account that in the problem data set marked as non-essential questions, there may be semantic correlations between the various non-essential questions. Therefore, in order to further improve the accuracy of query matching between the questions to be verified and the non-essential questions and the accuracy of the verification results, the present application adopts a multi-granularity attention query matching method, which performs local neighborhood implicit optimization on the set of semantic encoding feature vectors of the non-essential questions to enhance the semantic understanding of non-essential questions, thereby achieving more refined semantic matching. Specifically, the set of non-essential question semantic coding feature vectors is first subjected to one-dimensional convolution coding to capture the local neighborhood information in the set and generate a set of non-essential question local neighborhood semantic association feature vectors. The size of the convolution kernel is set to an integer multiple of the length of the non-essential question semantic coding feature vector in order to ensure that the contextual information within a specific range can be captured, thereby better reflecting the semantic association of the local neighborhood. One-dimensional convolution coding is a technology that encodes one-dimensional time series data (such as text, speech, biological signals, etc.) into a fixed-length vector. It uses a one-dimensional convolutional neural network (1D CNN) to extract important features from time series data.

[0030] Specifically, the S32 performs attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the question to be verified, the set of semantic encoding feature vectors of non-essential questions, and the set of local neighborhood semantic association feature vectors of non-essential questions to obtain the implicit representation vector of the semantic query matching of the question to be verified. Specifically, first, the semantic encoding feature vector of the question to be verified is matched with the set of semantic encoding feature vectors of the non-essential question and the set of semantic association feature vectors of the local neighborhood of the non-essential question respectively based on the multi-head self-attention mechanism to obtain a set of semantic query implicit vectors of the question to be verified and a set of semantic query implicit vectors of the local neighborhood granularity semantic query of the question to be verified; that is, the multi-head attention mechanism of the converter structure is used to perform query matching between the question to be verified and the non-essential question, which constructs the query vector and the value vector based on the semantic encoding feature vector of the question to be verified, and uses the set of semantic encoding feature vectors of the non-essential question and the set of semantic association feature vectors of the local neighborhood of the non-essential question as the set of key vectors respectively, and learns the semantic association matching relationship between the question to be verified and the non-essential question from different perspectives, while focusing on the semantic characteristics of the non-essential question itself and the association characteristics of the non-essential question and its neighborhood, thereby comprehensively considering local and global information to improve the accuracy of query matching. Furthermore, the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity of the question to be verified are fused to obtain the implicit representation vector of the semantic query matching of the question to be verified. Here, query matching information at different granularity levels is aggregated by positional mean calculation to generate semantic query matching feature vectors of the question to be verified and semantic query matching feature vectors of the local neighborhood granularity of the question to be verified at different granularity levels. These two vectors are then subjected to multi-dimensional semantic interaction fusion to integrate information at different granularity levels and generate the semantic query matching implicit representation vector of the question to be verified. This fully reflects the multiple semantic associations between the question to be verified and non-essential questions, thereby improving the accuracy of the verification results.

[0031] Among them, the process of performing query matching based on the multi-head self-attention mechanism on the semantic coding feature vector of the question to be verified, the set of semantic coding feature vectors of the non-essential question, and the set of semantic association feature vectors of the non-essential question local neighborhood to obtain the set of semantic query implicit vectors of the question to be verified and the set of semantic query implicit vectors of the local neighborhood granularity semantic query of the question to be verified includes: using the value embedding coding matrix to process the semantic coding feature vector of the question to be verified to obtain a value feature vector; using the semantic coding feature vector of the question to be verified as the query vector, the value feature vector as the value vector and the semantic coding feature vector of the non-essential question as the query vector The set of quantities is used as a set of key vectors, and the query vector, the value vector and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the semantic query of the question to be verified; the semantic encoding feature vector of the question to be verified is used as the query vector, the value feature vector is used as the value vector and the set of local neighborhood semantic association feature vectors of the non-essential question is used as a set of key vectors, and the query vector, the value vector and the set of key vectors are input into a second multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the local neighborhood granularity semantic query of the question to be verified.

[0032] More specifically, the process of taking the semantic encoding feature vector of the question to be verified as the query vector, the value feature vector as the value vector and the set of the semantic encoding feature vectors of the non-essential question as the set of key vectors, and inputting the query vector, the value vector and the set of key vectors into the first multi-head attention query module based on the converter structure to obtain the set of implicit vectors of the semantic query of the question to be verified includes: multiplying the query vector by the transposed vector of the key vector and dividing it by the square root of the scale of the key vector to obtain an attention score matrix; passing the attention score matrix through a softmax function and multiplying it with the value vector to obtain the implicit vector of the semantic query of the question to be verified.

[0033] More specifically, the process of fusing the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the local neighborhood granularity semantic query of the question to be verified to obtain the implicit representation vector of the semantic query matching of the question to be verified includes: respectively calculating the positional mean vector of the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the local neighborhood granularity semantic query of the question to be verified to obtain the semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified; fusing the semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified to obtain the implicit representation vector of the semantic query matching of the question to be verified. Among them, fusing the semantic query matching feature vector of the problem to be verified and the local neighborhood granularity semantic query matching feature vector of the problem to be verified to obtain the implicit representation vector of the semantic query matching of the problem to be verified, including: calculating the position point addition, position point subtraction and position point multiplication between the semantic query matching feature vector of the problem to be verified and the local neighborhood granularity semantic query matching feature vector of the problem to be verified to obtain the first semantic fusion result, the second semantic fusion result and the third semantic fusion result; after concatenating the first semantic fusion result, the second semantic fusion result and the third semantic fusion result into a multi-scale semantic fusion feature vector, performing one-dimensional convolution processing and maximum pooling processing on them to obtain the implicit representation vector of the semantic query matching of the problem to be verified.

[0034] In summary, in the above embodiment, the semantic encoding feature vector of the question to be verified is subjected to attention matching query and multi-granularity query feature fusion with the set of semantic encoding feature vectors of the non-essential question and the set of local neighborhood semantic association feature vectors of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified, including: processing the semantic encoding feature vector of the question to be verified and the set of semantic encoding feature vectors of the non-essential question with the following query matching formula to obtain the semantic query matching implicit representation vector of the question to be verified, wherein the query matching formula is:

[0035] K={k1,k2,...k i ,...k n}

[0036] K′={k1′;k2′;...k i ';...k m ′}

[0037] K′=Conv 1×l ({k1, k2, ..., k n})

[0038] v v =v q Wq +b q

[0039]

[0040]

[0041]

[0042]

[0043]

[0044] Where K represents the set of semantic encoding feature vectors of the non-essential question, k1, k2, k i and k n are respectively the first, second, i-th and n-th non-essential question semantic encoding feature vectors in the set of non-essential question semantic encoding feature vectors,

[0045] The value of n is the number of semantic encoding feature vectors of the non-essential question, Conv 1×l (·) represents one-dimensional convolution processing, l represents the length of the one-dimensional convolution kernel, K′ represents the set of semantic association feature vectors of the non-essential problem local neighborhood, k1, k2, k i ′ and k m are the first, second, lth and mth non-essential question local neighborhood semantic association feature vectors in the set of non-essential question local neighborhood semantic association feature vectors, respectively, the value of m is the number of the non-essential question local neighborhood semantic association feature vectors, v q Represents the semantic encoding feature vector of the question to be verified, W q is the value embedding encoding matrix, b q is the bias term, v v is the value vector, softmax is the normalized exponential function, the value of d is the scale of the key vector, (·) T Represents the transpose of a vector, v ki represents the implicit vector of the semantic query of the i-th question to be verified, v ni represents the implicit vector of the local neighborhood granularity semantic query of the lth question to be verified, v k Represents the semantic query matching feature vector of the question to be verified, v n represents the semantic query matching feature vector of the local neighborhood granularity of the problem to be verified, ⊙ represents the dot product, Indicates point addition, represents point subtraction, [·,·,·] represents a cascade operation, MaxPool(·) represents a maximum pooling operation, conv1D(·) represents a one-dimensional convolution operation, and V represents the implicit representation vector of the semantic query matching of the question to be verified.

[0046] In other specific examples of the present application, a multi-granularity attention matching query can be performed on the set of semantic encoding feature vectors of the question to be verified and the set of semantic encoding feature vectors of the non-essential question in other ways to obtain an implicit representation vector of the semantic query matching of the question to be verified, for example: using natural language processing (NLP) technology to extract semantic features from the question to be verified and the non-essential question; encoding the semantic features into semantic vectors to represent the meaning and semantic relationship of the question; defining multiple attention heads, each head focusing on a different granularity or aspect of the semantic vector, for example, one attention head may focus on word-level similarity, while another attention head may focus on sentence-level similarity; for each attention head, calculating the attention weight between the semantic vector of the question to be verified and the set of semantic vectors of the non-essential question; using these attention weights to perform weighted summation on the semantic vectors of the non-essential question to obtain an implicit representation vector of the semantic query matching of the question to be verified.

[0047] Specifically, the S4 determines whether the question to be verified is a non-essential question based on the implicit representation vector of the semantic query matching of the question to be verified. In a specific example of the present application, the implicit representation vector of the semantic query matching of the question to be verified is input into a question verification module based on a classifier to obtain a verification result, and the verification result is used to indicate whether the question to be verified is a non-essential question. Specifically, first, the implicit representation vector of the semantic query matching of the question to be verified is fully connected encoded using multiple fully connected layers of the classifier to obtain an encoded classification feature vector; and then the encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the verification result.

[0048] Here, in the preferred example, since the set of the semantic encoding feature vector of the question to be verified and the set of the semantic encoding feature vector of the non-essential question respectively represent the encoded semantic features of the question to be verified and each question data marked as non-essential question in the question data set marked as non-essential question, when they are input into the attention gated query network based on 1D convolution local neighborhood optimization, the semantic feature representation difference will have different cross-domain attention query weights based on the local neighborhood optimization difference of the imitation key matrix, so that the implicit representation vector of the semantic query matching of the question to be verified will also have the distribution diversity of the aggregated features across the semantic space. Therefore, when the implicit representation vector of the semantic query matching of the question to be verified is subjected to probabilistic regression through the classifier-based problem verification module, it will affect the accuracy of the regression result.

[0049] That is, considering that the weight matrix during probability regression acts on the implicit representation vector of the semantic query matching of the question to be verified, the uncertainty of the weight stimulus parameters of the weight matrix caused by the distribution diversity of the implicit representation vector of the semantic query matching of the question to be verified makes the posterior probability density inference of the implicit representation vector of the semantic query matching of the question to be verified through the action of the weight matrix missing, which will affect the accuracy of the probability regression results.

[0050] Preferably, inputting the semantic query matching implicit representation vector of the question to be verified into a classifier-based question verification module to obtain a verification result includes: determining a feature mean and a feature variance of the semantic query matching implicit representation vector of the question to be verified, and dividing the feature mean by the feature variance to obtain an implicit probability statistical value of the semantic query matching of the question to be verified:

[0051]

[0052] Among them, μ and σ 2 They respectively represent the feature mean and feature variance of the implicit representation vector of the semantic query matching of the question to be verified, and ρ represents the implicit probability statistical value of the semantic query matching of the question to be verified.

[0053] The implicit representation vector of the semantic query matching of the question to be verified is multiplied by the inverse of the maximum eigenvalue in the implicit representation vector of the semantic query matching of the question to be verified to obtain the implicit probability constraint vector of the semantic query matching of the question to be verified:

[0054] V1=V⊙v max -1

[0055] Where V represents the implicit representation vector of the semantic query matching of the question to be verified, v max Indicates that the semantic query of the question to be verified matches the maximum eigenvalue in the implicit representation vector,

[0056] ⊙ represents the point product by position, and V1 represents the implicit probability constraint vector of the semantic query matching of the question to be verified.

[0057] After performing dot-wise addition of the implicit probability constraint vector of the semantic query matching of the question to be verified and the implicit probability statistical value of the semantic query matching of the question to be verified, the logarithmic value with base 2 is calculated to obtain the implicit information interaction vector of the semantic query matching of the question to be verified:

[0058]

[0059] Among them, V1 represents the implicit probability constraint vector of the semantic query matching of the question to be verified, ρ represents the implicit probability statistical value of the semantic query matching of the question to be verified, Indicates adding by position point,

[0060] V2 represents the implicit information interaction vector of the semantic query matching of the question to be verified.

[0061] After performing point subtraction on the implicit probability constraint vector of the semantic query matching of the question to be verified by using the implicit probability statistics of the semantic query matching of the question to be verified, the inverse of each eigenvalue is calculated to obtain the implicit sequence constraint vector of the semantic query matching of the question to be verified:

[0062]

[0063] Among them, V1 represents the implicit probability constraint vector of the semantic query matching of the question to be verified, ρ represents the implicit probability statistical value of the semantic query matching of the question to be verified, It represents point-wise subtraction, and V3 represents the implicit sequence constraint vector of the semantic query matching of the question to be verified.

[0064] The semantic query matching implicit information interaction vector of the question to be verified is interpolated with the semantic query matching implicit sequence constraint vector of the question to be verified to obtain an optimized semantic query matching implicit representation vector of the question to be verified;

[0065] The optimized semantic query matching implicit representation vector of the question to be verified is input into a classifier-based question verification module to obtain a verification result.

[0066] Therefore, the probability statistical characteristics of the implicit representation vector of the semantic query matching the question to be verified are used to simulate the mesoscale interaction structure under probability constraints between the eigenvalue scale and the eigenvector scale of the implicit representation vector of the semantic query matching the question to be verified, so as to construct a bidirectional latent variable motif based on the mesoscale short sequence relative to the probability statistical value to perform mesoscale bidirectional migration, and to perform posterior recovery based on the sequence constraints on the interactive information, thereby improving the convergence effect in the probability density domain and improving the accuracy of the verification result obtained by inputting the implicit representation vector of the semantic query matching the question to be verified into the problem verification module based on the classifier.

[0067] In summary, according to the embodiment of the present application, the method for checking the necessary questions based on the Lora fine-tuning model is explained. First, a question data set marked as non-essential questions is extracted from the database. Then, the Lora fine-tuning model is used to perform semantic analysis on the questions asked by law enforcement officers and each non-essential question in the question data set. Then, multi-granularity semantic query matching is performed on the two to intelligently check whether the questions asked by law enforcement officers are necessary questions. In this way, the accuracy and work efficiency of law enforcement officers' questions can be effectively improved, unnecessary duplication of work can be reduced, and the efficiency and quality of case handling can be improved.

[0068] While various embodiments of the present disclosure have been described above, the above descriptions are illustrative, non-exhaustive, and not intended to be limiting of the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to existing technologies, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for checking and asking necessary questions based on the Lora fine-tuning model, characterized by: include: Extracting a dataset of questions marked as non-essential questions from the database and obtaining questions to be verified; Input the question to be verified and each question data marked as non-essential question in the question data set marked as non-essential question into the semantic encoder based on the lora fine-tuning model to obtain a semantic encoding feature vector of the question to be verified and a set of semantic encoding feature vectors of non-essential questions; Performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain an implicit representation vector matching the semantic query of the question to be verified; Determining whether the question to be verified is a non-essential question based on the semantic query matching implicit representation vector of the question to be verified; The step of performing a multi-granularity attention matching query on the set of the semantic encoding feature vector of the question to be verified and the semantic encoding feature vector of the non-essential question to obtain an implicit representation vector matching the semantic query of the question to be verified includes: Performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors; Performing attention matching query and multi-granularity query feature fusion on the semantic encoding feature vector of the question to be verified, the set of semantic encoding feature vectors of the non-essential question, and the set of local neighborhood semantic association feature vectors of the non-essential question to obtain the semantic query matching implicit representation vector of the question to be verified; Among them, the semantic encoding feature vector of the question to be verified is subjected to query matching based on the multi-head self-attention mechanism with the set of semantic encoding feature vectors of the non-essential question and the set of semantic association feature vectors of the local neighborhood of the non-essential question to obtain a set of semantic query implicit vectors of the question to be verified and a set of semantic query implicit vectors of the local neighborhood granularity of the question to be verified; that is, the multi-head attention mechanism of the converter structure is used to perform query matching between the question to be verified and the non-essential question, and the query vector and value vector are constructed based on the semantic encoding feature vector of the question to be verified, and the set of semantic encoding feature vectors of the non-essential question and the set of semantic association feature vectors of the local neighborhood of the non-essential question are used as the set of key vectors respectively, and the semantic association matching relationship between the question to be verified and the non-essential question is learned from different perspectives, while focusing on the semantic characteristics of the non-essential question itself and the association characteristics of the non-essential question and its neighborhood; The set of implicit vectors of the semantic query of the problem to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity to be verified are fused to obtain the implicit representation vector of the semantic query matching of the problem to be verified, wherein the query matching information of different granularity levels is summarized by calculating the position mean to generate semantic query matching feature vectors of the problem to be verified and local neighborhood granularity semantic query matching feature vectors of different granularity levels, and the two are subjected to multi-dimensional semantic interactive fusion to integrate information at different granularity levels to generate the semantic query matching implicit representation vector of the problem to be verified, so as to fully reflect the multiple semantic association relationships between the problem to be verified and non-essential problems.

2. The method for checking and asking necessary questions based on the Lora fine-tuning model according to claim 1 is characterized in that: Performing local neighborhood implicit optimization on the set of non-essential question semantic encoding feature vectors to obtain a set of non-essential question local neighborhood semantic association feature vectors, including: One-dimensional convolution coding is performed on the set of non-essential question semantic encoding feature vectors to obtain the set of non-essential question local neighborhood semantic association feature vectors, wherein the scale of the one-dimensional convolution coding is equal to an integer multiple of the length of the non-essential question semantic encoding feature vector.

3. The method for checking and asking necessary questions based on the LoRa fine-tuning model according to claim 2 is characterized in that: The semantic encoding feature vector of the question to be verified is matched with the set of semantic encoding feature vectors of the non-essential question and the set of semantic association feature vectors of the non-essential question local neighborhood to obtain a set of implicit vectors of semantic queries of the question to be verified and a set of implicit vectors of semantic queries of the local neighborhood granularity of the question to be verified, including: Processing the semantic encoding feature vector of the question to be verified using a value embedding encoding matrix to obtain a value feature vector; Using the semantic encoding feature vector of the question to be verified as a query vector, the value feature vector as a value vector, and the set of the semantic encoding feature vectors of the non-essential question as a set of key vectors, the query vector, the value vector, and the set of key vectors are input into a first multi-head attention query module based on a converter structure to obtain a set of implicit vectors of the semantic query of the question to be verified; Taking the semantic encoding feature vector of the question to be verified as the query vector, the value feature vector as the value vector and the set of the non-essential question local neighborhood semantic association feature vectors as the set of key vectors, the query vector, the value vector and the set of key vectors are input into the second multi-head attention query module based on the converter structure to obtain the set of implicit vectors of the local neighborhood granularity semantic query of the question to be verified.

4. The method for checking and asking necessary questions based on the LoRa fine-tuning model according to claim 3 is characterized in that: The method uses the semantic encoding feature vector of the question to be verified as a query vector, the value feature vector as a value vector, and the set of the semantic encoding feature vectors of the non-essential question as a set of key vectors, and inputs the query vector, the value vector, and the set of key vectors into a first multi-head attention query module based on a transformer structure to obtain a set of implicit vectors of the semantic query of the question to be verified, including: Multiplying the query vector by the transposed vector of the key vector and dividing by the square root of the scale of the key vector to obtain an attention score matrix; The attention score matrix is ​​passed through a softmax function and then multiplied by the value vector to obtain the implicit vector of the semantic query of the question to be verified.

5. The method for checking and asking necessary questions based on the LoRa fine-tuning model according to claim 4 is characterized in that: The set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the question to be verified at the local neighborhood granularity level are integrated to obtain the semantic query matching implicit representation vector of the question to be verified, including: Calculating the positional mean vectors of the set of implicit vectors of the semantic query of the question to be verified and the set of implicit vectors of the semantic query of the local neighborhood granularity of the question to be verified respectively to obtain a matching feature vector of the semantic query of the question to be verified and a matching feature vector of the semantic query of the local neighborhood granularity of the question to be verified; The semantic query matching feature vector of the question to be verified and the local neighborhood granularity semantic query matching feature vector of the question to be verified are fused to obtain the semantic query matching implicit representation vector of the question to be verified.

6. The method for checking and asking necessary questions based on the Lora fine-tuning model according to claim 5 is characterized in that: The semantic query matching feature vector of the question to be verified and the semantic query matching feature vector of the local neighborhood granularity of the question to be verified are integrated to obtain the semantic query matching implicit representation vector of the question to be verified, including: Calculating the position point addition, position point subtraction, and position point multiplication between the semantic query matching feature vector of the question to be verified and the semantic query matching feature vector of the local neighborhood granularity of the question to be verified to obtain a first semantic fusion result, a second semantic fusion result, and a third semantic fusion result; After the first semantic fusion result, the second semantic fusion result and the third semantic fusion result are cascaded into a multi-scale semantic fusion feature vector, one-dimensional convolution processing and maximum pooling processing are performed on them to obtain the implicit representation vector of the semantic query matching of the question to be verified.

7. The method for checking and asking necessary questions based on the LoRa fine-tuning model according to claim 6 is characterized in that: Determining whether the question to be verified is a non-essential question based on the semantic query matching implicit representation vector of the question to be verified includes: The semantic query matching implicit representation vector of the question to be verified is input into a classifier-based question verification module to obtain a verification result, and the verification result is used to indicate whether the question to be verified is a non-essential question.

8. The method for checking and asking necessary questions based on the LoRa fine-tuning model according to claim 7 is characterized in that: Inputting the semantic query matching implicit representation vector of the question to be verified into a classifier-based question verification module to obtain a verification result, wherein the verification result is used to indicate whether the question to be verified is a non-essential question, including: Using multiple fully connected layers of the classifier to perform fully connected encoding on the semantic query matching implicit representation vector of the question to be verified to obtain an encoded classification feature vector; The encoded classification feature vector is passed through the Softmax classification function of the classifier to obtain the verification result.

Images