Large language model sensitive information identification method and device, equipment and storage medium

Abstract

The invention discloses a large language model sensitive information identification method and device, equipment and a storage medium, and the method comprises the steps: obtaining a target dialogue information set of a current dialogue object, carrying out the processing of the target dialogue information set through a preset sensitivity calculation sub-model, generating a sensitive information identification result corresponding to the target dialogue information set, and storing the sensitive information identification result. And performing sensitivity verification based on the first sensitive information table and the second sensitive information table to generate a verification result. According to the scheme, the sensitive information identification result corresponding to the target dialogue information set can be accurately calculated through the preset sensitivity calculation sub-model; besides, sensitivity verification is carried out based on the first sensitive information table and the second sensitive information table, so that the problem that the sensitive information has the same or similar types or has accidental false alarms is solved, and the rationality and accuracy of sensitive information identification are further improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a method, device, equipment and storage medium for identifying sensitive information using a large language model. Background Art

[0002] With the upgrade of Large Language Model (LLM) chat tools, more and more people use LLM to talk and communicate, which has led to many problems of sensitive information leakage and malicious exploitation.

[0003] Traditionally, rule matching technology is usually used when identifying sensitive information in large language models. However, the limitations of rule matching are very obvious. For example, unregistered words cannot be matched, polysemous words have a high rate of false positives, and malicious references cannot be effectively detected. In addition, rule matching is highly dependent on the rule base, and the recognition effect depends to a large extent on the rule base.

[0004] Therefore, the above-mentioned large language model sensitive information identification method has the problem of low accuracy. Summary of the invention

[0005] The present application aims to at least solve the technical problems existing in the prior art. To this end, the first aspect of the present application proposes a method for identifying sensitive information of a large language model, the method comprising:

[0006] Obtaining a target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information;

[0007] The target dialogue information set is processed by a preset sensitivity calculation submodel to generate a sensitive information identification result corresponding to the target dialogue information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, the first sensitive information table includes a first total sensitivity value corresponding to the target question set, a sensitive information category, and a first sensitivity value corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, a sensitive information category, and a second sensitivity value corresponding to each sensitive information category;

[0008] Sensitivity verification is performed based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

[0009] In a possible implementation, sensitivity verification is performed based on the first sensitive information table and the second sensitive information table to generate a verification result, including:

[0010] Obtaining a first weight corresponding to a target question set in a target dialogue information set, and obtaining a second weight corresponding to a target answer set in the target dialogue information set;

[0011] Calculating a total sensitivity difference value based on the first weight, the first total sensitivity value, the second weight, and the second total sensitivity value;

[0012] Calculate the sensitivity difference value of each type corresponding to each sensitive information category based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight, and the second sensitivity value corresponding to each sensitive information category;

[0013] The total sensitivity difference value is compared with a preset total sensitivity difference threshold value to generate a first comparison result, and the sensitivity difference value of each type is compared with a preset sensitivity difference threshold value of each type to generate a second comparison result;

[0014] A verification result is generated based on the first comparison result and the second comparison result.

[0015] In a possible implementation, the target question set includes current question information and a historical question set, the historical question set includes a plurality of historical question information, and obtaining a first weight corresponding to the target question set in the target dialogue information set includes:

[0016] Obtain the first sub-weight corresponding to the current question information and the second sub-weight corresponding to the historical question set;

[0017] Calculate the first time value corresponding to each historical question information based on the preset time decay rate and the sequence number information corresponding to each historical question information;

[0018] Normalize each first time value to generate a first value weight corresponding to each historical question information;

[0019] Calculate the weight corresponding to each historical question information based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each historical question information;

[0020] Based on the first sub-weight corresponding to the current question information and the weights corresponding to each historical question information, a first weight corresponding to the target question set is obtained.

[0021] In a possible implementation, the target answer set includes current answer information and a historical answer set, the historical answer set includes a plurality of historical answer information, and obtaining a second weight corresponding to the target answer set in the target dialog information set includes:

[0022] Obtain the third sub-weight corresponding to the current response information and the fourth sub-weight corresponding to the historical response set;

[0023] Calculate the second time value corresponding to each historical response information based on the preset time decay rate and the sequence number information corresponding to each historical response information;

[0024] Normalizing each second time value to generate a second value weight corresponding to each historical response information;

[0025] Calculate the weight corresponding to each historical response information based on the fourth sub-weight corresponding to the historical response set and the second value weight corresponding to each historical response information;

[0026] Based on the third sub-weight corresponding to the current response information and the weights corresponding to each piece of historical response information, a second weight corresponding to the target response set is obtained.

[0027] In a possible implementation, the target conversation information set is processed by a preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target conversation information set, including:

[0028] The rule matching degree and semantic matching degree corresponding to the target dialogue information set are calculated by using a preset sensitivity calculation submodel; wherein the rule matching degree is used to characterize the matching degree between the target dialogue information set and the feature information of each sensitive information category in the preset sensitive rule set, and the semantic matching degree is used to characterize the matching degree between the target dialogue information set and the semantic information of each sensitive information category in the preset sensitive semantic set;

[0029] The rule matching degree and semantic matching degree are fused to generate sensitive information recognition results corresponding to the target dialogue information set.

[0030] In a possible implementation, calculating the rule matching degree and semantic matching degree corresponding to the target dialogue information set by using a preset sensitivity calculation sub-model includes:

[0031] Obtaining a rule weight set corresponding to a preset sensitive rule set and a semantic weight set corresponding to a preset sensitive semantic set;

[0032] Using a preset matching algorithm to perform keyword matching between the target conversation information set and the preset sensitive rule set, generating a first matching value set, and calculating the rule matching degree based on the first matching value set and the rule weight set;

[0033] The target dialogue information set and the encoding results of the preset sensitive semantic set are similarly calculated to generate multiple similarity values, and the semantic matching degree is calculated based on the multiple similarity values ​​and the semantic weight set.

[0034] In a possible implementation, obtaining a target dialogue information set of a current dialogue object includes:

[0035] Get the current conversation information and historical conversation information of the current conversation object;

[0036] Perform data preprocessing on current conversation information and historical conversation information to generate preprocessing results;

[0037] The preprocessing results are split based on the question information and the answer information, and the split data are sorted in chronological order to generate the target dialogue information set.

[0038] The second aspect of the present application provides a large language model sensitive information identification device, the device comprising:

[0039] An acquisition module is used to acquire a target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information;

[0040] A processing module, used to process the target dialogue information set through a preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target dialogue information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, the first sensitive information table includes a first total sensitivity value corresponding to the target question set, a sensitive information category, and a first sensitivity value corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, a sensitive information category, and a second sensitivity value corresponding to each sensitive information category;

[0041] A verification module is used to perform sensitivity verification based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

[0042] In a possible implementation, the verification module is specifically used for:

[0043] Obtaining a first weight corresponding to a target question set in a target dialogue information set, and obtaining a second weight corresponding to a target answer set in the target dialogue information set;

[0044] Calculating a total sensitivity difference value based on the first weight, the first total sensitivity value, the second weight, and the second total sensitivity value;

[0045] Calculate the sensitivity difference value of each type corresponding to each sensitive information category based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight, and the second sensitivity value corresponding to each sensitive information category;

[0046] The total sensitivity difference value is compared with a preset total sensitivity difference threshold value to generate a first comparison result, and the sensitivity difference value of each type is compared with a preset sensitivity difference threshold value of each type to generate a second comparison result;

[0047] A verification result is generated based on the first comparison result and the second comparison result.

[0048] In a possible implementation, the target question set includes current question information and a historical question set, the historical question set includes a plurality of historical question information, and the verification module is further used to:

[0049] Obtain the first sub-weight corresponding to the current question information and the second sub-weight corresponding to the historical question set;

[0050] Calculate the first time value corresponding to each historical question information based on the preset time decay rate and the sequence number information corresponding to each historical question information;

[0051] Normalize each first time value to generate a first value weight corresponding to each historical question information;

[0052] Calculate the weight corresponding to each historical question information based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each historical question information;

[0053] Based on the first sub-weight corresponding to the current question information and the weights corresponding to each historical question information, a first weight corresponding to the target question set is obtained.

[0054] In a possible implementation, the target response set includes current response information and a historical response set, the historical response set includes a plurality of historical response information, and the verification module is further used to:

[0055] Obtain the third sub-weight corresponding to the current response information and the fourth sub-weight corresponding to the historical response set;

[0056] Calculate the second time value corresponding to each historical response information based on the preset time decay rate and the sequence number information corresponding to each historical response information;

[0057] Normalizing each second time value to generate a second value weight corresponding to each historical response information;

[0058] Calculate the weight corresponding to each historical response information based on the fourth sub-weight corresponding to the historical response set and the second value weight corresponding to each historical response information;

[0059] Based on the third sub-weight corresponding to the current response information and the weights corresponding to each piece of historical response information, a second weight corresponding to the target response set is obtained.

[0060] In a possible implementation manner, the processing module is specifically used to:

[0061] The rule matching degree and semantic matching degree corresponding to the target dialogue information set are calculated by using a preset sensitivity calculation submodel; wherein the rule matching degree is used to characterize the matching degree between the target dialogue information set and the feature information of each sensitive information category in the preset sensitive rule set, and the semantic matching degree is used to characterize the matching degree between the target dialogue information set and the semantic information of each sensitive information category in the preset sensitive semantic set;

[0062] The rule matching degree and semantic matching degree are fused to generate sensitive information recognition results corresponding to the target dialogue information set.

[0063] In a possible implementation manner, the processing module is further used to:

[0064] Obtaining a rule weight set corresponding to a preset sensitive rule set and a semantic weight set corresponding to a preset sensitive semantic set;

[0065] Using a preset matching algorithm to perform keyword matching between the target conversation information set and the preset sensitive rule set, generating a first matching value set, and calculating the rule matching degree based on the first matching value set and the rule weight set;

[0066] The target dialogue information set and the encoding results of the preset sensitive semantic set are similarly calculated to generate multiple similarity values, and the semantic matching degree is calculated based on the multiple similarity values ​​and the semantic weight set.

[0067] In a possible implementation manner, the acquisition module is specifically used to:

[0068] Get the current conversation information and historical conversation information of the current conversation object;

[0069] Perform data preprocessing on current conversation information and historical conversation information to generate preprocessing results;

[0070] The preprocessing results are split based on the question information and the answer information, and the split data are sorted in chronological order to generate the target dialogue information set.

[0071] The third aspect of the present application proposes an electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the large language model sensitive information identification method as described in the first aspect.

[0072] In a fourth aspect, the present application proposes a computer-readable storage medium, in which at least one instruction, at least one program, a code set or an instruction set is stored. The at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by a processor to implement the large language model sensitive information identification method as described in the first aspect.

[0073] The embodiments of the present application have the following beneficial effects:

[0074] The embodiment of the present application provides a method for identifying sensitive information with a large language model, the method comprising: obtaining a target dialogue information set of the current dialogue object, processing the target dialogue information set through a preset sensitivity calculation submodel, generating a sensitive information identification result corresponding to the target dialogue information set, and performing sensitivity verification based on a first sensitive information table and a second sensitive information table to generate a verification result. This solution can accurately calculate the sensitive information identification result corresponding to the target dialogue information set through a preset sensitivity calculation submodel; in addition, by performing sensitivity verification based on the first sensitive information table and the second sensitive information table, the problem of having the same or similar sensitive information categories or occasional false positives is solved, further improving the rationality and accuracy of sensitive information identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 A block diagram of a computer device provided in an embodiment of the present application;

[0076] Figure 2 A flowchart of a method for identifying sensitive information using a large language model provided in an embodiment of the present application;

[0077] Figure 3 A flowchart of the steps for obtaining a target conversation information set provided in an embodiment of the present application;

[0078] Figure 4 A flowchart of the steps for generating sensitive information identification results provided in an embodiment of the present application;

[0079] Figure 5 A flowchart of the steps for generating rule matching degree and semantic matching degree provided in an embodiment of the present application;

[0080] Figure 6 A flowchart of the steps for generating a verification result provided in an embodiment of the present application;

[0081] Figure 7 A flowchart of the steps for obtaining a first weight provided in an embodiment of the present application;

[0082] Figure 8 A flowchart of the steps of obtaining a second weight provided in an embodiment of the present application;

[0083] Fig. 9 A structural block diagram of a large language model sensitive information identification device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0084] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0085] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present disclosure, unless otherwise specified, "multiple" means two or more. In addition, the use of "based on" or "according to" means openness and inclusiveness, because the process, steps, calculations or other actions "based on" or "according to" one or more of the conditions or values ​​may be based on additional conditions or values ​​beyond the described values ​​in practice.

[0086] The large language model sensitive information identification method provided in the present application can be applied to a computer device (electronic device), and the computer device can be a server or a terminal, wherein the server can be a single server or a server cluster composed of multiple servers. The embodiments of the present application do not make specific limitations on this. The terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices.

[0087] Take the computer device as a server as an example. Figure 1 A block diagram of a server is shown, such as Figure 1 As shown, the server may include a processor and a memory connected via a system bus. The processor of the server is used to provide computing and control capabilities. The memory of the server includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. When the computer program is executed by the processor, a large language model sensitive information identification method is implemented.

[0088] Those skilled in the art will understand that Figure 1The structure shown in the figure is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the server to which the solution of the present application is applied. Optionally, the server may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0089] It should be noted that the execution subject of the embodiments of the present application can be a computer device or a large language model sensitive information identification device. The following method embodiments will be described using a computer device as the execution subject.

[0090] Figure 2 A flowchart of a method for identifying sensitive information using a large language model is provided in an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0091] Step 202: Obtain the target dialogue information set of the current dialogue object.

[0092] Among them, Large Language Model (LLM) refers to a deep learning model trained with a large amount of text data, which can generate natural language text or understand the meaning of language text. Large language model can handle a variety of natural language tasks, such as text classification, question answering, dialogue, etc., and is an important path to artificial intelligence.

[0093] In order to solve the problem that the accuracy of sensitive information recognition of large language models is not high, when performing sensitive information recognition of large language models, the target dialogue information set of the current dialogue object can be obtained first. The target dialogue information set can include a target question set and a target answer set. The target question set can include multiple question information, and the target answer set can include multiple answer information.

[0094] Optionally, the current dialogue object may include a large language model dialogue object, and may also include objects such as an intelligent agent, a person, and a virtual subject. Exemplarily, the target dialogue information set here may be dialogue information between a person and objects such as a large language model dialogue object, an intelligent agent, and a virtual subject. Exemplarily, there may also be no human participation, and the target dialogue information set may be dialogue information between objects such as a large language model dialogue object, an intelligent agent, and a virtual subject, or dialogue information between two large language model dialogue objects, or dialogue information between two intelligent agents, or dialogue information between two virtual subjects, which is not specifically limited in the embodiments of the present application.

[0095] The question information may include current question information and historical question information, and the answer information may include current answer information and answer question information. Optionally, if the sensitive information identification process is post-analysis, there is no need to pay attention to the current question information and the current answer information.

[0096] In some optional embodiments, Figure 3 As shown, Figure 3 A flowchart of steps for obtaining a target conversation information set provided in an embodiment of the present application includes:

[0097] Step 302: Obtain the current conversation information and historical conversation information of the current conversation object.

[0098] Step 304: perform data preprocessing on the current conversation information and the historical conversation information to generate preprocessing results.

[0099] Step 306: split the preprocessing result based on the question information and the answer information, and sort the split data in chronological order to generate a target dialogue information set.

[0100] After obtaining the current conversation information and historical conversation information of the current conversation object, data preprocessing can be performed on the current conversation information and historical conversation information. Optionally, data preprocessing can include distinguishing question information from answer information, processing error data or error data explicitly indicated by the user and abnormal data, data standardization, data normalization, reference consistency and other processing processes.

[0101] Next, the preprocessing result can be split into a question set and an answer set, and sorted in reverse chronological order, that is, the newer the data, the earlier it is, so that the target dialogue information set can be generated.

[0102] Step 204: Process the target conversation information set using a preset sensitivity calculation sub-model to generate a sensitive information recognition result corresponding to the target conversation information set.

[0103] Among them, after obtaining the target conversation information set, the target conversation information set can be input into a preset sensitivity calculation sub-model, and the target conversation information set is processed by the preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target conversation information set.

[0104] The sensitive information identification results may include a first sensitive information table and a second sensitive information table. The first sensitive information table is denoted as wqh_rst, and the second sensitive information table is denoted as wah_rst. The first sensitive information table may include a first total sensitivity value corresponding to the target question set, sensitive information categories, and the first sensitivity value corresponding to each sensitive information category. The second sensitive information table may include a second total sensitivity value corresponding to the target answer set, sensitive information categories, and the second sensitivity value corresponding to each sensitive information category.

[0105] In some optional embodiments, Figure 4 As shown, Figure 4A flowchart of steps for generating sensitive information identification results provided in an embodiment of the present application includes:

[0106] Step 402: Calculate the rule matching degree and semantic matching degree corresponding to the target dialogue information set through a preset sensitivity calculation sub-model.

[0107] Step 404: The rule matching degree and the semantic matching degree are integrated to generate a sensitive information recognition result corresponding to the target conversation information set.

[0108] Among them, the rule matching degree is used to characterize the matching degree between the target dialogue information set and the feature information of each sensitive information category in the preset sensitive rule set, and the semantic matching degree is used to characterize the matching degree between the target dialogue information set and the semantic information of each sensitive information category in the preset sensitive semantic set.

[0109] In some optional embodiments, Figure 5 As shown, Figure 5 A flowchart of steps for generating rule matching and semantic matching provided in an embodiment of the present application includes:

[0110] Step 502: Obtain a rule weight set corresponding to a preset sensitive rule set and a semantic weight set corresponding to a preset sensitive semantic set.

[0111] Step 504: Use a preset matching algorithm to perform keyword matching between the target conversation information set and the preset sensitive rule set to generate a first matching value set, and calculate the rule matching degree based on the first matching value set and the rule weight set.

[0112] Step 506: perform similarity calculation on the encoding results of the target dialogue information set and the preset sensitive semantic set to generate multiple similarity values, and calculate the semantic matching degree based on the multiple similarity values ​​and the semantic weight set.

[0113] Among them, the above-mentioned preset sensitivity calculation sub-model needs to be pre-built. Specifically, as much sensitive data as possible can be collected for each sensitive information category. Sensitive information categories may include but are not limited to sensitive information stipulated by laws and regulations, sensitive information of various industries, sensitive information with strong suggestive or referential nature, and sensitive information at the forefront of the network, and then they are classified. Exemplarily, it can be divided into categories such as personal, financial, health, medical, education, national security, trade secrets, technical information, communication content, biometrics, and location.

[0114] Next, for each sensitive information category, relevant feature information can be collected to form a preset sensitive rule set. For example, for biometrics, it is necessary to construct rules for relevant feature information such as fingerprints, irises, and faces, which can be used for rule matching later.

[0115] In addition, it is necessary to collect enough natural semantic information for each sensitive information category to form a preset sensitive semantic set. For example, for biometrics, it is necessary to collect natural semantic information such as the characteristics, identification steps, identification types, and identification results of relevant identification methods such as fingerprints, irises, and faces, which can be used for semantic retrieval later.

[0116] It is also necessary to set a rule weight set, denoted as wsr. Optionally, when setting the rule weight set, different weights can be set for different sensitive information categories, denoted as wsr_type. For example, the weight of regulatory information should be larger than the weight of location information. Then, different weights can be set for each rule in the preset sensitive rule set, denoted as wsr_reg(i,j), where i and j represent the rule number and the rule type number to which it belongs, respectively. Finally, according to the formula Calculate the rule weight of the jth sensitive information category in the rule weight set, where k is a number from 1 to i, that is, the rule weight of each sensitive information category is the product of the weight corresponding to its sensitive information category and the harmonic mean of the rule weights in all rules of this sensitive information category.

[0117] At the same time, it is also necessary to set a semantic weight set, denoted as wst. Optionally, the process of setting the semantic weight set is similar to the process of setting the rule weight set mentioned above, which will not be repeated here.

[0118] Therefore, based on the above information, when constructing the preset sensitivity calculation sub-model, the importance ratio of rules and semantics can be pre-set to a1:a2 according to the relative importance of rules and semantics. The importance is a hyperparameter. During the use phase of the preset sensitivity calculation sub-model, it can be dynamically debugged according to the actual business scenario, and the initial value is given by the business expert. Then, the rule matching method and the semantic matching method can be constructed, and the rule matching degree and the semantic matching degree can be calculated according to the importance ratio a1:a2. The above process is encapsulated and processed, so that the preset sensitivity calculation sub-model can be constructed.

[0119] In some optional embodiments, when calculating the rule matching degree and the semantic matching degree, the rule weight set corresponding to the preset sensitive rule set and the semantic weight set corresponding to the preset sensitive semantic set can be first obtained. Then, a preset matching algorithm such as rule matching can be used to match the current target in the target dialogue information set with the preset sensitive rule set by keywords, thereby generating a first matching value set, denoted as s_words_r. The first matching value set includes data such as matched sensitive information and sensitive information categories. The rule matching degree is then calculated using the first matching value set and the rule weight set, denoted as s_words_rr. The calculation method is: s_words_rr(i,j)=s_words_r(i)*wsr(j), where i is the serial number of the current target, and j is the serial number of the sensitive information category in the preset sensitive rule set, that is, the category to which the sensitive information matched by i belongs.

[0120] When calculating the semantic matching degree, the preset sensitive semantic set may be first encoded using a preset encoding algorithm, and then similarity calculation may be performed on the encoding results of the target dialogue information set and the preset sensitive semantic set to generate multiple similarity values. Optionally, the similarity calculation method may be a cosine similarity algorithm, so that the semantic matching degree may be calculated based on multiple similarity values ​​and a semantic weight set. The preset encoding algorithm may be an Embedding algorithm.

[0121] In some optional embodiments, hyperparameters such as the similarity threshold s_f_limit and the recall number TopN may be preset. These hyperparameters are used to regulate the final result of the semantic matching and may be given by business experts based on actual scenarios and initialized to default values ​​s_f_limit0 and TopN0.

[0122] Therefore, after calculating multiple similarity values, the largest TopN or semantic values ​​greater than the similarity threshold s_f_limit can be selected as the second matching value set of the current target, recorded as s_features_r(x). The second matching value set includes matched sensitive information, sensitive information categories, similarity values ​​and other data.

[0123] Finally, the semantic matching degree can be calculated based on the second matching value set and the semantic weight set, which is recorded as s_features_rr. The calculation method is: s_features_rr(i,j) = s_features_r(i)*wst(j), where i is the serial number of the current target, and j is the serial number of the sensitive information category in the preset sensitive semantic set, that is, the category to which the sensitive information matched by i belongs.

[0124] Finally, the rule matching degree and the semantic matching degree can be fused to generate a sensitive information recognition result corresponding to the target dialogue information set, which is recorded as s_rst. Optionally, the rule matching degree and the semantic matching degree can be fused based on the above importance ratio a1:a2, and the calculation method is: s_rst = s_words_rr*a1+s_features_rr*a2.

[0125] Step 206: Perform sensitivity verification based on the first sensitive information table and the second sensitive information table to generate a verification result.

[0126] In order to further enhance the robustness of the device itself and the rationality of the sensitive results, this method also sets up a sensitivity verification mechanism to verify whether the questions and responses are coordinated, have the same or similar sensitive information categories, or have accidental false positives. Therefore, sensitivity verification can be performed based on the first sensitive information table and the second sensitive information table to generate verification results, which are used to characterize the accuracy of sensitive information identification results.

[0127] In some optional embodiments, Figure 6 As shown, Figure 6 A flowchart of steps for generating a verification result provided in an embodiment of the present application includes:

[0128] Step 602: Obtain a first weight corresponding to a target question set in a target dialogue information set, and obtain a second weight corresponding to a target answer set in a target dialogue information set.

[0129] Step 604: Calculate a total sensitivity difference value based on the first weight, the first total sensitivity value, the second weight, and the second total sensitivity value.

[0130] Step 606: Calculate the sensitivity difference value of each type corresponding to each sensitive information category based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight and the second sensitivity value corresponding to each sensitive information category.

[0131] Step 608: compare the total sensitivity difference value with a preset total sensitivity difference threshold to generate a first comparison result, and compare the sensitivity difference value of each type with a preset sensitivity difference threshold of each type to generate a second comparison result.

[0132] Step 610: Generate a verification result based on the first comparison result and the second comparison result.

[0133] In some optional embodiments, the target question set includes current question information and a historical question set, and the historical question set includes a plurality of historical question information, so when obtaining the first weight corresponding to the target question set in the target dialogue information set, Figure 7 As shown, Figure 7A flowchart of steps for obtaining a first weight provided in an embodiment of the present application includes:

[0134] Step 702: Obtain a first sub-weight corresponding to the current question information and a second sub-weight corresponding to the historical question set.

[0135] Step 704: Calculate the first time value corresponding to each piece of historical question information based on the preset time decay rate and the sequence number information corresponding to each piece of historical question information.

[0136] Step 706: normalize each first time value to generate a first value weight corresponding to each historical question information.

[0137] Step 708: Calculate the weight corresponding to each historical question information based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each historical question information.

[0138] Step 710: Obtain a first weight corresponding to the target question set based on the first sub-weight corresponding to the current question information and the weights corresponding to each historical question information.

[0139] The first sub-weight corresponding to the current question information is recorded as wqc. If it is only for post-analysis, wqc can be set to 0. The second sub-weight corresponding to the historical question set is recorded as wqh. If it is only for post-analysis, wqh can be set to 1.

[0140] Next, the weight of each historical question information in the historical question set can be set as wqh(i), where i represents the serial number of a historical question information in the target question set. Specifically, the first time value corresponding to each historical question information can be calculated based on the preset time decay rate and the serial number information corresponding to each historical question information, denoted as f_time_v(i). According to the time decay characteristics, new data has a higher value than old data, and the old data has a higher time decay value. Optionally, different decay functions can be used to calculate the time value, including but not limited to exponential decay, polynomial decay, forward decay, super exponential decay, etc. For example, if an exponential decay algorithm is used, f_time_v(i) = e -γi , where γ represents the preset time decay rate, which can be given by experts based on experience, and i represents the sequence number of the i-th historical question information.

[0141] Then, each first time value is normalized to generate the first value weight corresponding to each historical question information, which is recorded as w_time_v(i), where i represents the sequence number of the i-th historical question information. Optionally, a normalization function or method can be used for normalization, for example, a softmax function can be used for normalization. And the normalized first value weights of each target question set are sorted according to the order of the sorted data set to form a value weight matrix, that is, a question value weight set.

[0142] Then, based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each historical question information, the weight corresponding to each historical question information is calculated, recorded as wqh_total, and the calculation method is: wqh_total(i) = wqh*w_time_v(i), i represents the sequence number of the i-th historical question information. Finally, based on the first sub-weight corresponding to the current question information and the weight corresponding to each historical question information, the first weight corresponding to the target question set can be obtained, recorded as wqh_total.

[0143] In some optional embodiments, the target answer set includes current answer information and a historical answer set, and the historical answer set includes a plurality of historical answer information, so that when obtaining the second weight corresponding to the target answer set in the target conversation information set, such as Figure 8 As shown, Figure 8 A flowchart of steps for obtaining a second weight is provided in an embodiment of the present application, including:

[0144] Step 802: Obtain the third sub-weight corresponding to the current response information and the fourth sub-weight corresponding to the historical response set.

[0145] Step 804: Calculate the second time value corresponding to each piece of historical response information based on the preset time decay rate and the sequence number information corresponding to each piece of historical response information.

[0146] Step 806: normalize each second time value to generate a second value weight corresponding to each historical response information.

[0147] Step 808: Calculate the weight corresponding to each historical response information based on the fourth sub-weight corresponding to the historical response set and the second value weight corresponding to each historical response information.

[0148] Step 810: Based on the third sub-weight corresponding to the current response information and the weights corresponding to each historical response information, obtain the second weight corresponding to the target response set.

[0149] The specific process of generating the second weight is the same as the specific process of generating the first weight, which will not be repeated here. The second weight corresponding to the target response set is recorded as wah_total.

[0150] After obtaining the first weight corresponding to the target question set in the target dialogue information set and the second weight corresponding to the target answer set in the target dialogue information set, sensitivity verification can be performed. Optionally, a preset total sensitivity difference threshold verify0 can be set first. simlar And preset the sensitivity difference threshold of each type verify_type0 simlar Then, based on the first weight, the first total sensitivity value, the second weight and the second total sensitivity value, the total sensitivity difference value verify is calculated. simlar , calculated as: Among them, wqh_rst (first total sensitivity value) is the first total sensitivity value in the first sensitive information table wqh_rst, and wah_rst (second total sensitivity value) is the second total sensitivity value in the second sensitive information table wah_rst.

[0151] Similarly, based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight and the second sensitivity value corresponding to each sensitive information category, the sensitivity difference value of each type corresponding to each sensitive information category can be calculated. That is, the calculation method of the type sensitivity difference value corresponding to the j-th sensitive information category is: Among them, wqh_rst(j) is the first sensitivity value corresponding to the j-th sensitive information category in the first sensitive information table wqh_rst, and wah_rst(j) is the second sensitivity value corresponding to the j-th sensitive information category in the second sensitive information table wah_rst.

[0152] Next, the total sensitivity difference value can be compared with the preset total sensitivity difference threshold to generate a first comparison result, and different other logical treatments can be taken according to the first comparison result. For example, if the total sensitivity difference value is greater than the preset total sensitivity difference threshold, the total sensitivity verification fails, and at this time, the model hyperparameters can be adjusted, the logic can be verified, and re-verification can be performed.

[0153] The sensitivity difference value of each type is compared with the preset sensitivity difference threshold of each type to generate a second comparison result, and different other processing logic is adopted according to the second comparison result. For example, if the type sensitivity difference value of the j-th sensitive information category is greater than the preset sensitivity difference threshold of each type, the sensitivity verification of the j-th sensitive information category fails. At this time, the preset sensitivity difference threshold of each type can be adjusted, the j-th sensitive information category can be discarded, and other processes can be performed. By setting the preset total sensitivity difference threshold and the preset sensitivity difference threshold of each type, the accuracy of sensitive information identification is further corrected and improved.

[0154] The present application provides a method for identifying sensitive information of a large language model, the method comprising: obtaining a target dialogue information set of a current dialogue object, processing the target dialogue information set through a preset sensitivity calculation submodel, generating a sensitive information identification result corresponding to the target dialogue information set, and performing sensitivity verification based on a first sensitive information table and a second sensitive information table to generate a verification result. This solution can accurately calculate the sensitive information identification result corresponding to the target dialogue information set through a preset sensitivity calculation submodel; in addition, by performing sensitivity verification based on the first sensitive information table and the second sensitive information table, the problem of having the same or similar sensitive information categories or occasional false positives is solved, further improving the rationality and accuracy of sensitive information identification.

[0155] Fig. 9 A structural block diagram of a large language model sensitive information identification device provided in an embodiment of the present application.

[0156] like Fig. 9 As shown, the large language model sensitive information identification device 900 includes:

[0157] The acquisition module 902 is used to acquire the target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information.

[0158] Processing module 904 is used to process the target conversation information set through a preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target conversation information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, the first sensitive information table includes a first total sensitivity value corresponding to the target question set, sensitive information categories and first sensitivity values ​​corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, sensitive information categories and second sensitivity values ​​corresponding to each sensitive information category.

[0159] The verification module 906 is used to perform sensitivity verification based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

[0160] Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment of the method, and will not be elaborated here. Each module in the above-mentioned large language model sensitive information identification device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations of the above modules.

[0161] In one embodiment of the present application, a computer device is provided, the computer device comprising a memory and a processor, the memory storing a computer program, and the processor implementing the following steps when executing the computer program:

[0162] Obtaining a target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information;

[0163] The target dialogue information set is processed by a preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target dialogue information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, the first sensitive information table includes a first total sensitivity value corresponding to the target question set, a sensitive information category, and a first sensitivity value corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, a sensitive information category, and a second sensitivity value corresponding to each sensitive information category;

[0164] Sensitivity verification is performed based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

[0165] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0166] Obtaining a first weight corresponding to a target question set in a target dialogue information set, and obtaining a second weight corresponding to a target answer set in the target dialogue information set;

[0167] Calculating a total sensitivity difference value based on the first weight, the first total sensitivity value, the second weight, and the second total sensitivity value;

[0168] Calculate the sensitivity difference value of each type corresponding to each sensitive information category based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight, and the second sensitivity value corresponding to each sensitive information category;

[0169] The total sensitivity difference value is compared with a preset total sensitivity difference threshold value to generate a first comparison result, and the sensitivity difference value of each type is compared with a preset sensitivity difference threshold value of each type to generate a second comparison result;

[0170] A verification result is generated based on the first comparison result and the second comparison result.

[0171] In one embodiment of the present application, the target question set includes current question information and a historical question set, the historical question set includes a plurality of historical question information, and the processor further implements the following steps when executing the computer program:

[0172] Obtain the first sub-weight corresponding to the current question information and the second sub-weight corresponding to the historical question set;

[0173] Calculate the first time value corresponding to each historical question information based on the preset time decay rate and the sequence number information corresponding to each historical question information;

[0174] Normalize each first time value to generate a first value weight corresponding to each historical question information;

[0175] Calculate the weight corresponding to each historical question information based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each historical question information;

[0176] Based on the first sub-weight corresponding to the current question information and the weights corresponding to each historical question information, a first weight corresponding to the target question set is obtained.

[0177] In one embodiment of the present application, the target response set includes current response information and a historical response set, the historical response set includes a plurality of historical response information, and the processor further implements the following steps when executing the computer program:

[0178] Obtain the third sub-weight corresponding to the current response information and the fourth sub-weight corresponding to the historical response set;

[0179] Calculate the second time value corresponding to each historical response information based on the preset time decay rate and the sequence number information corresponding to each historical response information;

[0180] Normalizing each second time value to generate a second value weight corresponding to each historical response information;

[0181] Calculate the weight corresponding to each historical response information based on the fourth sub-weight corresponding to the historical response set and the second value weight corresponding to each historical response information;

[0182] Based on the third sub-weight corresponding to the current response information and the weights corresponding to each piece of historical response information, a second weight corresponding to the target response set is obtained.

[0183] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0184] The rule matching degree and semantic matching degree corresponding to the target dialogue information set are calculated by using a preset sensitivity calculation submodel; wherein the rule matching degree is used to characterize the matching degree between the target dialogue information set and the feature information of each sensitive information category in the preset sensitive rule set, and the semantic matching degree is used to characterize the matching degree between the target dialogue information set and the semantic information of each sensitive information category in the preset sensitive semantic set;

[0185] The rule matching degree and semantic matching degree are fused to generate sensitive information recognition results corresponding to the target dialogue information set.

[0186] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0187] Obtaining a rule weight set corresponding to a preset sensitive rule set and a semantic weight set corresponding to a preset sensitive semantic set;

[0188] Using a preset matching algorithm to perform keyword matching between the target conversation information set and the preset sensitive rule set, generating a first matching value set, and calculating the rule matching degree based on the first matching value set and the rule weight set;

[0189] The target dialogue information set and the encoding results of the preset sensitive semantic set are similarly calculated to generate multiple similarity values, and the semantic matching degree is calculated based on the multiple similarity values ​​and the semantic weight set.

[0190] In one embodiment of the present application, when the processor executes the computer program, the processor further implements the following steps:

[0191] Get the current conversation information and historical conversation information of the current conversation object;

[0192] Perform data preprocessing on current conversation information and historical conversation information to generate preprocessing results;

[0193] The preprocessing results are split based on the question information and the answer information, and the split data are sorted in chronological order to generate the target dialogue information set.

[0194] The computer device provided in the embodiment of the present application has similar implementation principles and technical effects to those of the above-mentioned method embodiment, and will not be described in detail here.

[0195] In one embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0196] Obtaining a target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information;

[0197] The target dialogue information set is processed by a preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target dialogue information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, the first sensitive information table includes a first total sensitivity value corresponding to the target question set, a sensitive information category, and a first sensitivity value corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, a sensitive information category, and a second sensitivity value corresponding to each sensitive information category;

[0198] Sensitivity verification is performed based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

[0199] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0200] Obtaining a first weight corresponding to a target question set in a target dialogue information set, and obtaining a second weight corresponding to a target answer set in the target dialogue information set;

[0201] Calculating a total sensitivity difference value based on the first weight, the first total sensitivity value, the second weight, and the second total sensitivity value;

[0202] Calculate the sensitivity difference value of each type corresponding to each sensitive information category based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight, and the second sensitivity value corresponding to each sensitive information category;

[0203] Compare the total sensitivity difference value with a preset total sensitivity difference threshold to generate a first comparison result, and compare the sensitivity difference value of each type with a preset sensitivity difference threshold of each type to generate a second comparison result;

[0204] A verification result is generated based on the first comparison result and the second comparison result.

[0205] In one embodiment of the present application, the target question set includes current question information and a historical question set, the historical question set includes a plurality of historical question information, and when the computer program is executed by the processor, the following steps are also implemented:

[0206] Obtain the first sub-weight corresponding to the current question information and the second sub-weight corresponding to the historical question set;

[0207] Calculate the first time value corresponding to each historical question information based on the preset time decay rate and the sequence number information corresponding to each historical question information;

[0208] Normalize each first time value to generate a first value weight corresponding to each historical question information;

[0209] Calculate the weight corresponding to each historical question information based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each historical question information;

[0210] Based on the first sub-weight corresponding to the current question information and the weights corresponding to each historical question information, a first weight corresponding to the target question set is obtained.

[0211] In one embodiment of the present application, the target response set includes current response information and a historical response set, the historical response set includes a plurality of historical response information, and when the computer program is executed by the processor, the following steps are further implemented:

[0212] Obtain the third sub-weight corresponding to the current response information and the fourth sub-weight corresponding to the historical response set;

[0213] Calculate the second time value corresponding to each historical response information based on the preset time decay rate and the sequence number information corresponding to each historical response information;

[0214] Normalizing each second time value to generate a second value weight corresponding to each historical response information;

[0215] Calculate the weight corresponding to each historical response information based on the fourth sub-weight corresponding to the historical response set and the second value weight corresponding to each historical response information;

[0216] Based on the third sub-weight corresponding to the current response information and the weights corresponding to each piece of historical response information, a second weight corresponding to the target response set is obtained.

[0217] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0218] The rule matching degree and semantic matching degree corresponding to the target dialogue information set are calculated by using a preset sensitivity calculation submodel; wherein the rule matching degree is used to characterize the matching degree between the target dialogue information set and the feature information of each sensitive information category in the preset sensitive rule set, and the semantic matching degree is used to characterize the matching degree between the target dialogue information set and the semantic information of each sensitive information category in the preset sensitive semantic set;

[0219] The rule matching degree and semantic matching degree are fused to generate sensitive information recognition results corresponding to the target dialogue information set.

[0220] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0221] Obtaining a rule weight set corresponding to a preset sensitive rule set and a semantic weight set corresponding to a preset sensitive semantic set;

[0222] Using a preset matching algorithm to perform keyword matching between the target conversation information set and the preset sensitive rule set, generating a first matching value set, and calculating the rule matching degree based on the first matching value set and the rule weight set;

[0223] The target dialogue information set and the encoding results of the preset sensitive semantic set are similarly calculated to generate multiple similarity values, and the semantic matching degree is calculated based on the multiple similarity values ​​and the semantic weight set.

[0224] In one embodiment of the present application, when the computer program is executed by a processor, the following steps are further implemented:

[0225] Get the current conversation information and historical conversation information of the current conversation object;

[0226] Perform data preprocessing on current conversation information and historical conversation information to generate preprocessing results;

[0227] The preprocessing results are split based on the question information and the answer information, and the split data are sorted in chronological order to generate the target dialogue information set.

[0228] The computer-readable storage medium provided in this embodiment has similar implementation principles and technical effects to those of the above method embodiments, and will not be described in detail here.

[0229] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0230] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0231] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for identifying sensitive information using a large language model, characterized in that: The method comprises: Acquire a target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information; The target dialogue information set is processed by a preset sensitivity calculation submodel to generate a sensitive information identification result corresponding to the target dialogue information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, wherein the first sensitive information table includes a first total sensitivity value corresponding to the target question set, a sensitive information category, and a first sensitivity value corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, a sensitive information category, and a second sensitivity value corresponding to each sensitive information category; Sensitivity verification is performed based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

2. The method according to claim 1, characterized in that The performing sensitivity verification based on the first sensitive information table and the second sensitive information table to generate a verification result includes: Obtaining a first weight corresponding to a target question set in the target dialogue information set, and obtaining a second weight corresponding to a target answer set in the target dialogue information set; Calculating a total sensitivity difference value based on the first weight, the first total sensitivity value, the second weight, and the second total sensitivity value; Calculate the sensitivity difference value of each type corresponding to each sensitive information category based on the first weight, the first sensitivity value corresponding to each sensitive information category, the second weight and the second sensitivity value corresponding to each sensitive information category; Comparing the total sensitivity difference value with a preset total sensitivity difference threshold to generate a first comparison result, and comparing the sensitivity difference value of each type with a preset sensitivity difference threshold of each type to generate a second comparison result; The verification result is generated based on the first comparison result and the second comparison result.

3. The method according to claim 2, characterized in that The target question set includes current question information and a historical question set, wherein the historical question set includes a plurality of historical question information, and obtaining a first weight corresponding to the target question set in the target dialogue information set includes: Obtaining a first sub-weight corresponding to the current question information and a second sub-weight corresponding to the historical question set; Calculate the first time value corresponding to each piece of historical question information based on a preset time decay rate and the sequence number information corresponding to each piece of historical question information; Normalizing each of the first time values ​​to generate a first value weight corresponding to each of the historical question information; Calculate the weight corresponding to each piece of historical question information based on the second sub-weight corresponding to the historical question set and the first value weight corresponding to each piece of historical question information; Based on the first sub-weight corresponding to the current question information and the weights corresponding to each of the historical question information, a first weight corresponding to the target question set is obtained.

4. The method according to claim 2, characterized in that: The target answer set includes current answer information and a historical answer set, wherein the historical answer set includes a plurality of historical answer information, and obtaining a second weight corresponding to the target answer set in the target dialogue information set includes: Obtaining a third sub-weight corresponding to the current response information and a fourth sub-weight corresponding to the historical response set; Calculate the second time value corresponding to each piece of historical response information based on a preset time decay rate and the sequence number information corresponding to each piece of historical response information; Normalizing each of the second time values ​​to generate a second value weight corresponding to each of the historical response information; Calculate the weight corresponding to each piece of historical response information based on the fourth sub-weight corresponding to the historical response set and the second value weight corresponding to each piece of historical response information; Based on the third sub-weight corresponding to the current response information and the weights corresponding to each of the historical response information, a second weight corresponding to the target response set is obtained.

5. The method according to any one of claims 1 to 4, characterized in that: The processing of the target dialogue information set by using a preset sensitivity calculation sub-model to generate a sensitive information identification result corresponding to the target dialogue information set includes: The rule matching degree and semantic matching degree corresponding to the target dialogue information set are calculated by a preset sensitivity calculation submodel; wherein the rule matching degree is used to characterize the matching degree between the target dialogue information set and the feature information of each sensitive information category in the preset sensitive rule set, and the semantic matching degree is used to characterize the matching degree between the target dialogue information set and the semantic information of each sensitive information category in the preset sensitive semantic set; The rule matching degree and the semantic matching degree are fused to generate a sensitive information recognition result corresponding to the target dialogue information set.

6. The method according to claim 5, characterized in that The calculating the rule matching degree and semantic matching degree corresponding to the target dialogue information set by using the preset sensitivity calculation sub-model includes: Obtaining a rule weight set corresponding to a preset sensitive rule set and a semantic weight set corresponding to a preset sensitive semantic set; Using a preset matching algorithm to perform keyword matching between the target conversation information set and the preset sensitive rule set, generating a first matching value set, and calculating the rule matching degree based on the first matching value set and the rule weight set; A similarity calculation is performed on the encoding results of the target dialogue information set and the preset sensitive semantic set to generate a plurality of similarity values, and the rule matching degree is calculated based on the plurality of similarity values ​​and the semantic weight set.

7. The method according to claim 1 or 2, characterized in that: The step of obtaining the target dialogue information set of the current dialogue object includes: Get the current conversation information and historical conversation information of the current conversation object; Performing data preprocessing on the current conversation information and the historical conversation information to generate a preprocessing result; The preprocessing result is split based on the question information and the answer information, and the split data is sorted in chronological order to generate the target dialogue information set.

8. A large language model sensitive information identification device, characterized in that: The device comprises: An acquisition module, used to acquire a target dialogue information set of the current dialogue object; wherein the target dialogue information set includes a target question set and a target answer set, wherein the target question set includes a plurality of question information, and the target answer set includes a plurality of answer information; a processing module, configured to process the target dialogue information set through a preset sensitivity calculation submodel to generate a sensitive information identification result corresponding to the target dialogue information set; wherein the sensitive information identification result includes a first sensitive information table and a second sensitive information table, wherein the first sensitive information table includes a first total sensitivity value corresponding to the target question set, a sensitive information category, and a first sensitivity value corresponding to each sensitive information category, and the second sensitive information table includes a second total sensitivity value corresponding to the target answer set, a sensitive information category, and a second sensitivity value corresponding to each sensitive information category; A verification module is used to perform sensitivity verification based on the first sensitive information table and the second sensitive information table to generate a verification result; wherein the verification result is used to characterize the accuracy of the sensitive information identification result.

9. An electronic device, characterized in that: The electronic device includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the large language model sensitive information identification method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, the at least one program, the code set or the instruction set is loaded and executed by the processor to implement the large language model sensitive information identification method as described in any one of claims 1-7.

Images