A text security detection method and system

Abstract

The application relates to the field of information security, in particular to a text security detection method and system, the method comprising: generating a to-be-detected data set according to the original text data in combination with a preset extraction rule and a target large language model; and determining whether the original text data belongs to member data according to the original text data, the to-be-detected data set and a preset detector; wherein the preset detector comprises known member data and non-member data. The text security detection method provided by the application judges whether text data is leaked by taking a multi-dimensional feature vector as input data of the preset detector, which can not only reduce the misjudgment rate caused by a single feature, but also make the detection process highly automated by using the preset detector, thereby improving the detection speed and ensuring high accuracy and reliability.

Description

Technical Field

[0001] This invention relates to the field of information security, and in particular to a text security detection method and system. Background Technology

[0002] Driven by the rapid development of artificial intelligence, large language models are advancing at an unprecedented pace, making significant strides in cutting-edge fields such as multimodal learning, scientific problem-solving, and reasoning. Their applications are becoming increasingly widespread in sectors like medicine, manufacturing, and finance. However, the development and improvement of large model performance rely heavily on training with massive amounts of data. Since large model training data is diverse and originates from multiple websites, it inevitably contains sensitive information, such as copyrighted data, corporate secrets, and personal privacy data. Due to their memory mechanisms, large models may output copyrighted private data, potentially infringing on the intellectual property or privacy rights of users and businesses. In short, large models currently face the security risk of sensitive data being used for training without permission, leading to inappropriate output. Therefore, it is necessary to research technologies for detecting sensitive information in datasets to address the risks associated with the collection, use, and use of data, as well as the presence of inappropriate content in training data. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in related technologies. To this end, the first aspect disclosed in this application provides a text security detection method, comprising: Based on the original text data, combined with preset extraction rules and the target large language model, a dataset to be detected is generated. Based on the original text data and the dataset to be detected, a multidimensional feature vector is generated; Based on the original text data, the dataset to be detected, and a preset detector, it is determined whether the original text data belongs to member data; wherein the preset detector includes: known member data and non-member data.

[0004] Optionally, generating the dataset to be detected based on the original text data, combined with preset extraction rules and the target large language model, includes: Based on the original text data and the preset extraction rules, the first original text data is generated. Based on the first original text data and the target large language model, generate the original text dataset.

[0005] Optionally, generating the first original text data based on the original text data and a preset extraction rule includes: The original text data is extracted according to the preset threshold to generate the first original text data.

[0006] Optionally, determining whether the original text data belongs to member data based on the original text data, the dataset to be detected, and a preset detector includes: Generate semantic similarity of the original text data based on the original text data and the dataset to be detected; Generate the n-gram comprehensive matching degree of the original text data based on the original text data and the dataset to be detected; Generate the diversity entropy value of the original text data based on the dataset to be detected; The compression ratio of the original text data generated based on the dataset to be detected; A four-dimensional feature vector is generated based on the semantic similarity, n-gram comprehensive matching degree, diversity entropy value, and compression ratio. The feature vector is input into a preset detector to determine whether the original text data belongs to member data.

[0007] Optionally, generating the semantic similarity of the original text data based on the original text data and the dataset to be detected includes: The original text data and the data in the dataset to be detected are respectively converted into a first embedding vector and a second embedding vector; Based on the first embedding vector and the second embedding vector, generate a cosine similarity. The semantic similarity is generated based on the cosine similarity.

[0008] Optionally, the n-gram comprehensive matching degree of the original text data generated based on the majority of original text data and the dataset to be detected includes: Based on the original text data and the dataset to be detected, and combined with the n-gram algorithm, first matching data and second matching data are generated; wherein, the first matching data is the matching data generated by processing the original text data and the dataset to be detected using the n-gram algorithm, and the second matching data is the data generated by processing the original text data using the n-gram algorithm. A first matching degree is generated based on the ratio of the first matching data to the second matching data; A comprehensive matching score is generated based on the first matching score, the text data to be detected, the dataset to be detected, and a compression algorithm.

[0009] Optionally, generating the diversity entropy value of the original text data based on the dataset to be detected includes: Based on the dataset to be detected, determine the probability distribution of each data point; Based on the distribution probability of each data point, a diversity entropy value for the original text data is generated.

[0010] Optionally, the compression ratio of the original text data generated based on the dataset to be detected includes: Based on the dataset to be detected, determine the length of each data point; The compression algorithm is used to compress each data length to generate the compressed data length; The compression ratio of the original text data is generated based on the length of each data point, the length of the compressed data, and the number of data points in the dataset to be tested.

[0011] Optionally, the method further includes: The detector is trained based on the original text data and the loss function, and the loss function is calculated. The detector is trained based on the text data in the training set and the loss function, and a preset detector is generated.

[0012] Optionally, in another aspect of this application, a text security detection system is provided, the system comprising: The first detection module is used to generate a dataset to be detected based on the original text data, combined with preset extraction rules and a target large language model. The second detection module is used to generate a multi-dimensional feature vector based on the original text data and the dataset to be detected; The third detection module is used to determine whether the original text data belongs to member data based on the multidimensional feature vector and a preset detector; wherein the preset detector includes: member data, and the member data belongs to the original text data.

[0013] This application ensures a certain correlation between the generated detection dataset and the original text data through preset extraction rules, providing a high-quality data foundation for subsequent judgment. Subsequently, the data extracted from the original text data is used as input to the target large language model to generate the detection dataset. The text security detection method proposed in this application does not require direct access to the internal parameters of the target large language model, the pre-trained dataset, or the conditional probabilities of word segmentation accessible by the open-source model. It relies solely on analyzing the final text output results, providing strong technical support for protecting privacy and copyright when users cannot obtain internal model information. Furthermore, by comparing the correlation and characteristics between the original text data and the data in the detection dataset, a multi-dimensional feature vector is calculated. This multi-dimensional feature vector is input into a preset detector to determine whether the data in the detection dataset belongs to the member data. If the result indicates that the original text data has been leaked, the preset detector achieves a high degree of automation throughout the detection process. From feature extraction to final membership inference, everything can be completed in a short time, significantly shortening the detection time and ensuring high accuracy and reliability while improving processing speed. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a text security detection method according to this application; Figure 2 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 3 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 4 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 5 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 6 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 7 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 8 This is a flowchart illustrating another embodiment of a text security detection method according to this application; Figure 9 This is a schematic diagram of the structure of a text security detection system according to this application. Detailed Implementation

[0015] To better understand this application, various aspects of this application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are merely illustrative of exemplary embodiments of this application and are not intended to limit the scope of this application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0016] Current methods for text detection using large language models require allowing the detector to access the probability distribution of word segments generated by the large model. However, in real-world scenarios, user interaction with the large model only yields the final text output. To make the application scenarios of text detection methods trained by large language models closer to actual situations, this application uses the original text data and the dataset to be detected obtained through training a large language target model to train a binary classification random forest model. This makes the trained detector more suitable for this application. In addition, this application generates multi-dimensional feature vectors based on the original text data and the dataset to be detected as input data for the detector. This solves the problem of high false positive rates caused by single-dimensional features and is more cost-effective and automated than manual detection methods.

[0017] It should be noted that LLM stands for Large Language Model, n-gram stands for n-gram model, and zlib stands for DEFLATE algorithm, which is a lossless compression algorithm.

[0018] It should be noted that, where there is no conflict, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] like Figure 1 As shown, this application proposes a text security detection method, the method comprising: S101, Based on the original text data, combined with preset extraction rules and the target large language model, generate the dataset to be detected.

[0020] S102, Generate a multidimensional feature vector based on the original text data and the dataset to be detected.

[0021] S103, Based on the multidimensional feature vector and a preset detector, determine whether the original text data belongs to member data; wherein the preset detector includes: a known member database and a non-member database, the member data belongs to the original text data, and the non-member data does not belong to the original text data.

[0022] For example, suppose nouns or adjectives from the original text data are extracted as key prompts, and then the extracted key prompts are input into the target large language model. The target large language model generates multiple text data containing the key prompts based on the key prompts. The generated text data are summarized to form the dataset to be detected. Then, a multi-dimensional input vector is generated based on the original text data and the dataset to be detected. This multi-dimensional vector is input into a preset detector to determine whether the data in the dataset to be detected belongs to the member data.

[0023] First, this application ensures that the generated detection dataset has a certain correlation with the original text data through preset extraction rules, providing a high-quality data foundation for subsequent judgment. Furthermore, this application uses the data extracted from the original text data as input to the target large language model to generate the detection dataset. Then, by comparing the data in the original text data and the detection dataset, a multi-dimensional feature vector is calculated between the two. This multi-dimensional feature vector improves the expressive power of the features and effectively reduces the misjudgment rate caused by a single feature. The multi-dimensional feature vector is then used as input to a preset detector to determine whether the data in the detection dataset belongs to the member data. If the result indicates that the original text data has been leaked, it means that the original text data has been leaked. The text security detection method proposed in this application does not require direct access to the internal parameters of the large model, the pre-trained dataset, or the conditional probabilities of word segmentation accessible by the open-source model; it relies solely on analyzing the final text output. This characteristic greatly broadens the application scope of this invention, enabling it to function in various real-world environments, especially when users cannot obtain the internal information of the model, providing strong technical support for protecting privacy and copyright. In addition, this application also trained an efficient binary classification random forest model as a detector. Based on the trained detector, the entire detection process is highly automated. From feature extraction to the final membership inference, it can be completed in a very short time, which greatly shortens the detection time and ensures high accuracy and reliability while improving processing speed.

[0024] In one embodiment that can be implemented in this application, such as Figure 2 As shown, the step of generating the dataset to be detected based on the original text data, combined with preset extraction rules and the target large language model, includes: S201, Generate the first original text data based on the original text data and the preset extraction rules.

[0025] S202, Generate the dataset to be detected based on the first original text data and the target large language model.

[0026] For example, suppose the original text is "How to ensure battery life in low-temperature environments". The preset extraction rule is to extract nouns from the original text data. According to this preset extraction rule, the current first original text data is "environment" and "battery". These two nouns are input into the target large language model. At this time, the text output by the target large language model is: "Explosion-proof test of battery in extreme environment", "Safety performance test of battery charging in high temperature environment", "Energy storage time of battery in low temperature environment" and "How to ensure battery life in low temperature environment". The text generated by the target large language model is summarized to generate the dataset to be tested.

[0027] This application uses this step to ensure that the generated dataset to be detected has a certain correlation with the original text data through preset extraction rules, providing a high-quality data foundation for subsequent judgment.

[0028] In one embodiment of this application, generating the first original text data based on the original text data and a preset extraction rule includes: The original text data is extracted according to the preset threshold to generate the first original text data.

[0029] For example, the preset extraction rule of this application is to extract the first 50% of the original text data to generate the first original text data. Specifically, if the original text is "How to ensure battery life in a low-temperature environment," the first original text data generated according to the preset extraction rule would be "How in a low-temperature environment." It should be noted that the preset threshold of 50% proposed in this application is merely an example and is not intended to be limiting.

[0030] In one embodiment that can be implemented in this application, such as Figure 3 As shown, generating a multidimensional feature vector based on the original text data and the dataset to be detected includes: S301, Generate semantic similarity of the original text data based on the original text data and the dataset to be detected.

[0031] S302, Generate the n-gram comprehensive matching degree of the original text data based on the original text data and the dataset to be detected.

[0032] S303, Generate the diversity entropy value of the original text data based on the dataset to be detected.

[0033] S304, Generate the compression ratio of the original text data based on the dataset to be detected.

[0034] S305, Based on the semantic similarity, n-gram comprehensive matching degree, diversity entropy value and compression ratio, generate a multidimensional feature vector.

[0035] It should be noted that a relatively simple feature dimension will make it difficult to improve the detection success rate. This application constructs four-dimensional feature values ​​as input data for the preset detector, which can more comprehensively capture the differences between member data and non-member data. In addition, the text similarity mentioned in this application includes semantic similarity and comprehensive matching degree values.

[0036] It should be noted that, theoretically, if the original text data is leaked, the target large language model can generate complete original text data based on the prefixes of the original text data. However, due to the diversity and uncertainty of the content generated by LLM, the content generated according to the preset extraction rules will have some differences from the original text data. But if the first original text data belongs to the training set, the content generated according to the preset extraction rules will have a high degree of similarity to the original text data.

[0037] This application combines semantic similarity, comprehensive matching degree, diversity entropy, and compression ratio to form a multi-dimensional feature vector, providing comprehensive feature support for text security detection. It achieves full coverage of multi-dimensional features and effectively reduces the false positive rate of text security detection.

[0038] Compared to existing technologies that utilize single-dimensional features such as probability distribution and perplexity in member reasoning, the detection method proposed in this application, through text similarity, the stability of the generated results (i.e., diversity entropy), and the compression rate of the generated text, can more comprehensively capture the differences between member and non-member data. By constructing multi-dimensional feature vectors, the false positive rate caused by single features is effectively reduced. Especially for data containing a large number of common words, traditional methods are prone to incorrect judgments. The detection method in this application, by introducing multi-dimensional feature analysis, can effectively reduce the false positive rate.

[0039] In one embodiment that can be implemented in this application, such as Figure 4 As shown, generating semantic similarity of the original text data based on the original text data and the dataset to be detected includes: S401, the original text data and the data in the dataset to be detected are converted into a first embedding vector and a second embedding vector, respectively.

[0040] S402, generate cosine similarity based on the first embedding vector and the second embedding vector.

[0041] S403, Generate the semantic similarity based on the cosine similarity.

[0042] It should be noted that the embedded vectors in this application must adhere to the principle that vectors with similar semantics must maintain consistent directions. The semantic similarity is the average of the cosine similarities.

[0043] Specifically, the formula for calculating cosine similarity is: ,in, The original text data, This is the dataset to be tested.

[0044] For example, suppose the original text data To enable artificial intelligence to assist in medical diagnosis, the dataset to be tested contains the following: For artificial intelligence-assisted medical diagnosis, To enable artificial intelligence to change lives, the first embedding vector is transformed from the original text data. Data set to be detected The transformed second embedding vector , The direction of the embedding vector and The directions are almost the same. The transformed second embedding vector , The direction of the embedding vector and The directions differed significantly, so calculations were performed subsequently. Length of the module Based on the above data, the cosine similarity of each dataset to be detected is calculated. and The cosine similarity calculation method is as follows: First, calculate the first embedding vector. Second embedding vector dot product Then calculate Length of the module Single cosine similarity ,according to The result was that and They are highly similar in meaning. and The cosine similarity calculation method is as follows: First, calculate the first embedding vector. Second embedding vector dot product Then calculate Length of the module Single cosine similarity ,according to The result was that and Their semantics are almost dissimilar. According to and To calculate the final semantic similarity, the number of datasets to be tested is n=2. Substitute these values ​​into the formula. In this example, the semantic similarity between the original text data and the dataset to be detected is 0.598.

[0045] It should be noted that when the semantic similarity is close to 1, the original text data and the dataset to be detected are highly consistent in meaning; when the semantic similarity is close to 0, the original text data and the dataset to be detected are highly consistent in meaning; and when the semantic similarity is close to -1, the original text data and the dataset to be detected have opposite meanings.

[0046] This application reduces the impact of individual outlier samples on semantic similarity by averaging the cosine similarity of all datasets to be detected. (Example) The semantically unrelated outlier similarity is only 0.196, but due to averaging, the final semantic similarity can still reflect the overall association between the dataset and the original text, avoiding the distortion of results due to individual outlier samples.

[0047] It should be noted that this application introduces two similarity metrics: cosine similarity, which represents the semantic similarity of the text, and n-gram matching, which is used to measure the matching degree between the dataset to be detected and the original text data. If only cosine similarity, representing semantics, is used, the high similarity between the dataset to be detected and the original text data is due to the strong contextual understanding and generation capabilities of the LLM model itself, but does not necessarily mean that the LLM target model uses the original text content. Using only text similarity may cause some non-member data to be misclassified as member data. Furthermore, the text generated by LLM has uncertainty; therefore, using only n-gram matching as the similarity metric for discrimination will result in a large amount of training data being misclassified as non-member data. Based on the above-mentioned current state of similarity discrimination, this application combines cosine similarity and n-gram matching to improve the accuracy of text data detection with the synergistic assistance of multiple similarity discrimination methods.

[0048] In one embodiment that can be implemented in this application, such as Figure 5 As shown, the n-gram comprehensive matching degree of the original text data generated based on the majority of original text data and the dataset to be detected includes: S501, based on the original text data and the dataset to be detected, and combined with the n-gram algorithm, generate first matching data and second matching data; wherein, the first matching data is the matching data generated by processing the original text data and the dataset to be detected using the n-gram algorithm, and the second matching data is the data generated by processing the original text data using the n-gram algorithm.

[0049] S502, a first matching degree is generated based on the ratio of the first matching data and the second matching data.

[0050] S503, Based on the first matching degree, the text data to be detected, the dataset to be detected, and the compression algorithm, a comprehensive matching degree is generated.

[0051] Specifically, the formula for calculating the overall matching degree of n-grams is as follows: ， ,in, The first match degree, Represents n-grammar, This represents the number of n-gram matches in the first original text data and the original text data. This represents the total number of n-grams in the original text data. For overall matching accuracy, This represents the result of processing the j-th original text data using the zlib compression algorithm.

[0052] For example, suppose the original text data To optimize medical diagnosis through deep learning using artificial intelligence, the generated dataset contains three data points: To optimize medical diagnosis using deep learning-driven artificial intelligence, its relationship with The number of matches is 7. For the application of artificial intelligence in the medical field, algorithm models, and The number of matches is 3. Cats enjoy grooming their fur in the sun, which is related to... The number of matches is 0, and the total number of matches in the original text is 8. Specifically, the words in the total matched data are: depth, learning, artificial intelligence, smart technology, optimization, medical, diagnosis, and no match. Therefore... of , of , of Then calculate separately , , The compression ratio, assuming , , The compression ratios are 2, 1.524, and 1.2, respectively. Substituting these into the formula for calculating the overall matching degree, .

[0053] This application calculates the first matching degree between the original text data and the data in the dataset to be detected using the n-gram algorithm, then integrates zlib compressed features, and generates a comprehensive matching degree through weighted averaging, which not only reflects semantic and syntactic associations, but also takes into account the inherent structural features of the text data.

[0054] It should be noted that a high cosine similarity indicates that the LLM model itself has strong contextual understanding capabilities, but does not mean that the LLM model includes the original text data. Therefore, judging text similarity solely by calculating cosine similarity is not comprehensive. Based on this background, this application introduces n-gram similarity, which represents the degree of token matching, as an auxiliary. The combination of the two improves the detection accuracy with multi-view assistance.

[0055] In one embodiment that can be implemented in this application, such as Figure 6As shown, the process of generating diversity entropy values ​​for the original text data based on the dataset to be detected includes: S601, Determine the distribution probability of each data point based on the dataset to be detected.

[0056] S602, Generate the diversity entropy value of the original text data based on the distribution probability of each data point.

[0057] It should be noted that this application uses the same first original text data as prompt words to input into the target large language model, which can generate multiple detection datasets with different texts. For member data, LLM remembers the complete content and grammatical structure of the original text data, and the content distribution will be relatively stable with low diversity. However, for non-member data, even if the same first original text data is used as prompt words, the generated results will have significant differences. Therefore, this application introduces the concept of diversity entropy to measure the stability of the generated results.

[0058] Specifically, the formula for calculating diversity entropy is: ,in, This represents the probability distribution of each word in the dataset to be detected.

[0059] It should be noted that the higher the diversity entropy value, the more evenly the words are distributed, and the stronger the diversity of the dataset to be detected; conversely, the lower the value, the weaker the diversity of the dataset to be detected.

[0060] For example, suppose we have two datasets, dataset 1 and dataset 2. In dataset 1, there are a total of 10 words, and 4 of them are unique. Word A appears 6 times, with a probability distribution of 6 / 10 = 0.6. Word B appears 2 times, with a probability distribution of 2 / 10 = 0.2. Word C appears 1 time, with a probability distribution of 1 / 10 = 0.1. Word D appears 1 time, with a probability distribution of 1 / 10 = 0.1. In dataset 2, there are a total of 9 words, with 9 unique words. Word A appears once, with a probability distribution of 1 / 9 ≈ 0.111; word B appears once, with a probability distribution of 1 / 9 ≈ 0.111; word C appears once, with a probability distribution of 1 / 9 ≈ 0.111; word D appears once, with a probability distribution of 1 / 9 ≈ 0.111; word E appears once, with a probability distribution of 1 / 9 ≈ 0.111; word F appears once, with a probability distribution of 1 / 9 ≈ 0.111; word G appears once, with a probability distribution of 1 / 9 ≈ 0.111; word H appears once, with a probability distribution of 1 / 9 ≈ 0.111; and word I appears once, with a probability distribution of 1 / 9 ≈ 0.111. This application uses log2 as the logarithm base. The calculated diversity entropy value for dataset 1 is... The calculated diversity entropy value for dataset 2 is as follows: The entropy value of dataset 2 is 3.17, which is much higher than that of dataset 1 (1.57). This corresponds to the fact that the word distribution in dataset 2 is more even, there is no dominant word, and the diversity is stronger, while dataset 1 is dominated by word A and has weaker diversity.

[0061] This application calculates specific diversity entropy values ​​and directly compares the diversity of different datasets horizontally, thereby making the generated multidimensional feature vector data more comprehensive.

[0062] In one embodiment that can be implemented in this application, such as Figure 7 As shown, the compression ratio of the original text data generated based on the dataset to be detected includes: S701, determine the length of each data point based on the dataset to be detected.

[0063] S702 uses a compression algorithm to compress each data length to generate a compressed data length.

[0064] S703, based on the length of each data point, the length of the compressed data, and the number of data points in the dataset to be tested, generate the compression ratio of the original text data.

[0065] Specifically, the formula for calculating the compression ratio is as follows: ,in, , representing the compression ratio index, the The length of the j-th original text data is represented by the following: This represents the result of compressing the j-th original text data using the zlib compression algorithm. This represents the length of the compressed data after the j-th original text object is compressed.

[0066] It should be noted that the length of the original text data is usually expressed in bytes or bits, and this application does not impose any restrictions on this. Furthermore, the compression ratio is used to measure the compression performance of a compression method in a specific scenario; the closer the compression ratio is to 1, the worse the compression effect; the higher the compression ratio, the better the compression effect.

[0067] It should be noted that zlib is based on the DEFLATE algorithm, which is the industry standard for lossless compression. It is especially suitable for text data. The core characteristics of text data are repeated words and fixed structures. zlib has extremely high compression efficiency for repeated sequences and can accurately capture the inherent repetition of text. In addition, the lossless compression feature ensures that the original text can be completely restored after compression, and semantic and structural information will not be lost due to compression.

[0068] For example, suppose the dataset to be detected includes three data points, namely... , , ,in, It is aaaaa aaaaa. Machine learning enables smart office solutions. The Apple Banana Cherry Date is used in this example with UTF-8 encoding. This application does not impose any restrictions on the comparison; the result is calculated using this encoding method. The text data is 11 characters long. The text data is 10 Chinese characters long. The text data is 22 characters long. After calculating the length of each text data item, zlib is used to compress the text data. Let the compressed lengths be 1, 2, and 3 respectively. It is 5. It is 15. Given a value of 19, calculate the compression ratio of a single text data point, where... The compression ratio is , The compression ratio is , The compression ratio is The average value is calculated based on the compression ratio of all text data. The compression ratio is 1.564. According to the principle that the higher the compression ratio, the better the compression effect, the compression effect of this example is moderate.

[0069] This application uses the mean as the final compression ratio, which can avoid misleading results caused by a single abnormal text data. In addition, averaging can balance the characteristics of different samples, accurately reflect the overall compression capability of the dataset to be tested, and provide a reliable data basis for whether it belongs to the member data.

[0070] In one embodiment that can be implemented in this application, such as Figure 8 As shown, the method further includes: S801, calculate the loss function based on the training set data and the probability values ​​generated by the binary classification random forest model using the training set data; S802, Train the binary classification random forest model based on the training set data and the loss function to generate a preset detector.

[0071] It should be noted that the detector preset in this application is a binary classification random forest model trained on the training set. In addition, this application uses binary cross-entropy as the loss function for training, and the training set data is multidimensional feature vector data.

[0072] For example, the formula for calculating the loss function is: ,in Representing binary cross-entropy, the This indicates the number of samples in the test set. This represents the true membership degree of the i-th sample, specifically represented by 0 or 1, where 0 represents non-member data and 1 represents member data. This represents the probability that the i-th sample in the binary classification random forest model belongs to the training set.

[0073] It should be noted that the smaller the loss value, the higher the accuracy of the final prediction, and vice versa.

[0074] For example, suppose the training set contains four training samples (T1, y1), (T2, y2), (T3, y3), and (T4, y4), where y1, y3, and y4 are all 1, and y2 is 0. The following steps involve using a binary classification random forest model to determine the membership of these four samples. Four four-dimensional feature vectors are obtained by calculating the feature vectors of samples T1 to T4. These four feature vectors are then input into the binary classification random forest model. Assuming the probabilities of predicting each sample as 1 are p1=0.75, p2=0.30, p3=0.83, and p4=0.72, respectively, the binary cross-entropy is calculated by substituting these probabilities into the formula. = [1 log(0.75)+1 log(0.83)+1 log(0.72)+1 log(0.70)] / 4 0.290.

[0075] In the process of training a binary classification random forest model, this application utilizes the ensemble property of multiple decision trees in the random forest model. Through steps such as random sampling of the training set, random selection of features, and optimization of splitting tree by tree according to the loss function, the multiple decision trees are ultimately trained to collaboratively minimize the overall loss, thereby training the detector pre-defined in this application.

[0076] In the second aspect of this application, as Figure 9 The present invention proposes a text security detection system, the system comprising: The first detection module 901 is used to generate a dataset to be detected based on the original text data, combined with preset extraction rules and a target large language model.

[0077] The second detection module 902 is used to generate a multi-dimensional feature vector based on the original text data and the dataset to be detected.

[0078] The third detection module 903 is used to determine whether the original text data belongs to member data based on the multidimensional feature vector and a preset detector; wherein the preset detector includes: known member data and non-member data, the member data belongs to the original text data, and the non-member data does not belong to the original text data.

[0079] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A text security detection method, characterized in that, The method includes: Based on the original text data, combined with preset extraction rules and the target large language model, a dataset to be detected is generated. Based on the original text data and the dataset to be detected, a multidimensional feature vector is generated; Based on the multidimensional feature vector and a preset detector, it is determined whether the original text data belongs to member data; wherein the preset detector includes: member data, and the member data belongs to the original text data; The process of generating the dataset to be detected based on the original text data, combined with preset extraction rules and the target large language model, includes: Based on the original text data and the preset extraction rules, the first original text data is generated. Based on the first original text data and the target large language model, a dataset to be detected is generated; The step of generating a multidimensional feature vector based on the original text data and the dataset to be detected includes: Generate semantic similarity of the original text data based on the original text data and the dataset to be detected; Generate the n-gram comprehensive matching degree of the original text data based on the original text data and the dataset to be detected; Generate the diversity entropy value of the original text data based on the dataset to be detected; The compression ratio of the original text data generated based on the dataset to be detected; Based on the semantic similarity, n-gram comprehensive matching degree, diversity entropy value and compression ratio, a multidimensional feature vector is generated; The method further includes: The loss function is calculated based on the training set data and the probability values ​​generated by the binary classification random forest model from the training set data. The binary classification random forest model is trained based on the training set data and the loss function to generate a preset detector.

2. The text security detection method according to claim 1, characterized in that, The step of generating the first original text data based on the original text data and a preset extraction rule includes: The original text data is extracted according to the preset threshold to generate the first original text data.

3. The text security detection method according to claim 1, characterized in that, The step of generating semantic similarity of the original text data based on the original text data and the dataset to be detected includes: The original text data and the data in the dataset to be detected are respectively converted into a first embedding vector and a second embedding vector; Based on the first embedding vector and the second embedding vector, generate a cosine similarity. The semantic similarity is generated based on the cosine similarity.

4. The text security detection method according to claim 1, characterized in that, The n-gram comprehensive matching degree of the original text data generated based on the original text data and the dataset to be detected includes: Based on the original text data and the dataset to be detected, and combined with the n-gram algorithm, first matching data and second matching data are generated; wherein, the first matching data is the matching data generated by processing the original text data and the dataset to be detected using the n-gram algorithm, and the second matching data is the data generated by processing the original text data using the n-gram algorithm. A first matching degree is generated based on the ratio of the first matching data to the second matching data; A comprehensive matching score is generated based on the first matching score, the text data to be detected, the dataset to be detected, and a compression algorithm.

5. The text security detection method according to claim 1, characterized in that, The process of generating the diversity entropy value of the original text data based on the dataset to be detected includes: Based on the dataset to be detected, determine the probability distribution of each data point; Based on the distribution probability of each data point, a diversity entropy value for the original text data is generated.

6. The text security detection method according to claim 1, characterized in that, The compression ratio of the original text data generated based on the dataset to be detected includes: Based on the dataset to be detected, determine the length of each data point; The compression algorithm is used to compress each data length to generate the compressed data length; The compression ratio of the original text data is generated based on the length of each data point, the length of the compressed data, and the number of data points in the dataset to be tested.

7. A text security detection system, characterized in that, The system includes: The first detection module is used to generate a dataset to be detected based on the original text data, combined with preset extraction rules and the target large language model; The second detection module is used to generate a multi-dimensional feature vector based on the original text data and the dataset to be detected; The third detection module is used to determine whether the original text data belongs to member data based on the multidimensional feature vector and a preset detector; wherein the preset detector includes: member data, and the member data belongs to the original text data; The process of generating the dataset to be detected based on the original text data, combined with preset extraction rules and the target large language model, includes: Based on the original text data and the preset extraction rules, the first original text data is generated. Based on the first original text data and the target large language model, a dataset to be detected is generated; The step of generating a multidimensional feature vector based on the original text data and the dataset to be detected includes: Generate semantic similarity of the original text data based on the original text data and the dataset to be detected; Generate the n-gram comprehensive matching degree of the original text data based on the original text data and the dataset to be detected; Generate the diversity entropy value of the original text data based on the dataset to be detected; The compression ratio of the original text data generated based on the dataset to be detected; Based on the semantic similarity, n-gram comprehensive matching degree, diversity entropy value and compression ratio, a multidimensional feature vector is generated; The method executed by the system further includes: The loss function is calculated based on the training set data and the probability values ​​generated by the binary classification random forest model from the training set data. The binary classification random forest model is trained based on the training set data and the loss function to generate a preset detector.

Images