Method, device, medium and equipment for promoting legality of large model dialogue

Abstract

The application relates to a method, device, medium and equipment for promoting legalization of a large model dialogue. The method comprises the following steps: acquiring sensitive word data of multiple data sources in real time; performing data cleaning on the sensitive word data to generate or update a sensitive word library; performing vectorization on the sensitive words in the sensitive word library to generate sensitive word vectors; performing clustering on the sensitive word vectors to calculate a center vector of each semantic cluster after clustering; using an interception model to respectively extract keywords and embedding vectors from input and output, determining a risk score of the input or output based on a matching result of the keywords and the sensitive words, a similarity of the embedding vectors and the center vectors of the semantic clusters, and intercepting the dialogue content when the risk score of the input or output is greater than or equal to a risk threshold. The sensitive word library can be dynamically updated to adapt to dynamic changes of laws of different countries in real time; the differences of laws, sensitive word 1 and sensitive word 2 of different countries or regions are accurately targeted to realize localized interception and improve dialogue security.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, apparatus, medium, and device for enhancing the legitimacy of large-scale model dialogues. Background Technology

[0002] Large Language Models (LLMs) (such as GPT and Claude) are widely used in various applications. Improving the legality and compliance of large model dialogues is a very real problem. Current technologies use static sensitive word libraries, extracting keywords from dialogue content and matching them to the library to identify and block risks. However, static sensitive word libraries require manual maintenance and cannot adapt to dynamic changes in laws across countries in real time, resulting in weak and delayed updates. Furthermore, the review of dialogue content has flaws, only blocking user input and ignoring potentially illegal information contained in the large model's generated results. There is a lack of localized blocking capabilities tailored to the differences in laws, sensitive words 1, and sensitive words 2 across different countries or regions, and insufficient cross-cultural and cross-sensitive word 1 adaptability. Therefore, a more flexible and reliable large model blocking method and system that adapts to different cultures and different sensitive words 1 is needed. Summary of the Invention

[0003] To overcome the problems existing in related technologies, this application provides a method, apparatus, medium and device for improving the legitimacy of large model dialogues.

[0004] According to a first aspect of this application, a method for improving the legitimacy of large-scale model dialogues is provided, comprising:

[0005] Real-time acquisition of sensitive word data from multiple data sources;

[0006] The sensitive word data is cleaned to generate or update the sensitive word lexicon.

[0007] The sensitive words in the aforementioned sensitive word lexicon are vectorized to generate sensitive word vectors;

[0008] Cluster the sensitive word vectors and calculate the center vector of each semantic cluster after clustering;

[0009] The interception model is used to extract keywords and embedding vectors from the input and output respectively. Based on the matching results of the keywords and the sensitive words, the similarity between the embedding vector and the center vector of each semantic cluster, the risk score of the input or output is determined.

[0010] When the risk score of the input or output is greater than or equal to the risk threshold, the dialogue content is blocked.

[0011] In some embodiments of this application, based on the foregoing scheme, the multiple data sources include:

[0012] Government legal documents, user-reported prohibited words, and prohibited words that are trending online.

[0013] In some embodiments of this application, the method for improving the legality of large model dialogue based on the foregoing scheme further includes: training the interception model with an updated sensitive word lexicon in real time or periodically.

[0014] In some embodiments of this application, based on the aforementioned scheme, the interception model includes a first interception model and a second interception model. When the first interception model is deployed online, the second interception model is trained in real time. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined period of time. After the comparison results are consistent, the first interception model is taken offline.

[0015] In some embodiments of this application, based on the foregoing scheme, the input risk score is determined by the matching results of the keywords and the sensitive words, and the similarity between the embedding vector and the center vector of each semantic cluster, including:

[0016] Extract keywords from the input text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a first preset threshold, use the exact match score as the input risk score.

[0017] When the exact match score is less than the first preset threshold, the embedding vector of the input text is extracted, and the similarity between the embedding vector and the center vector is calculated. When the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score.

[0018] When the similarity is less than the second preset threshold, the dialogue history is obtained, the context matching score is determined, and the exact matching score, similarity, and context matching score are multiplied by their respective weight coefficients and summed to determine the input risk score.

[0019] In some embodiments of this application, based on the foregoing scheme, and based on the matching results of the keywords and the sensitive words, and the similarity between the embedding vector and the center vector of each semantic cluster, the output risk score is determined as follows:

[0020] Extract keywords from the output text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a third preset threshold, use the exact match score as the output risk score.

[0021] When the exact match score is less than the third preset threshold, the embedding vector of the output text is extracted, the similarity between the embedding vector and the center vector is calculated, and the similarity is used as the output risk score.

[0022] According to another aspect of this application, an apparatus for enhancing the legitimacy of large-scale dialogue models is provided, comprising:

[0023] The sensitive word data acquisition module is used to acquire sensitive word data from multiple data sources in real time.

[0024] The sensitive word processing module is used to clean the sensitive word data and generate or update the sensitive word lexicon.

[0025] The vectorization module is used to vectorize the sensitive words in the sensitive word lexicon to generate sensitive word vectors;

[0026] The clustering module is used to cluster the sensitive word vectors and calculate the center vector of each semantic cluster after clustering.

[0027] The risk assessment module is used to extract keywords and embedding vectors from the input and output using an interception model, and to determine the risk score of the input or output based on the matching results of the keywords and the sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster;

[0028] The interception module is used to intercept dialogue content when the risk score of the input or output is greater than or equal to the risk threshold.

[0029] In some embodiments of this application, the apparatus for improving the legitimacy of large-scale dialogue models, based on the foregoing scheme, further includes:

[0030] The training module is used to train the interception model in real time or periodically with an updated sensitive word lexicon.

[0031] In some embodiments of this application, based on the aforementioned scheme, the interception model includes a first interception model and a second interception model. When the first interception model is deployed online, the second interception model is trained in real time. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined period of time. After the comparison results are consistent, the first interception model is taken offline.

[0032] In some embodiments of this application, based on the foregoing scheme, the risk assessment module determines the input risk score as follows:

[0033] Extract keywords from the input text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a first preset threshold, use the exact match score as the input risk score.

[0034] When the exact match score is less than the first preset threshold, the embedding vector of the input text is extracted, and the similarity between the embedding vector and the center vector is calculated. When the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score.

[0035] When the similarity is less than the second preset threshold, the dialogue history is obtained, the context matching score is determined, and the exact matching score, similarity, and context matching score are multiplied by their respective weight coefficients and summed to determine the input risk score.

[0036] In some embodiments of this application, based on the foregoing scheme, the risk assessment module determines the output risk score as follows:

[0037] Extract keywords from the output text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a third preset threshold, use the exact match score as the output risk score.

[0038] When the exact match score is less than the third preset threshold, the embedding vector of the output text is extracted, the similarity between the embedding vector and the center vector is calculated, and the similarity is used as the output risk score.

[0039] According to another aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, implements the steps of a method for enhancing the legitimacy of large model dialogues.

[0040] According to another aspect of this application, a computer device is provided, including a processor, a memory, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of a method for enhancing the legitimacy of large model dialogues.

[0041] This application acquires sensitive word data from multiple data sources in real time; cleans the sensitive word data to generate or update a sensitive word lexicon; vectorizes the sensitive words in the lexicon to generate sensitive word vectors; clusters the sensitive word vectors and calculates the center vector of each semantic cluster; uses an interception model to extract keywords and embedding vectors from the input and output respectively; and determines the risk score of the input or output based on the matching results of keywords and sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster; when the risk score of the input or output is greater than or equal to a preset threshold, the dialogue content is intercepted. This allows for real-time updates to the sensitive word lexicon, improving interception accuracy. Simultaneously, vector clustering of sensitive words generates semantic clusters related to laws, sensitive word 1, and sensitive word 2 in different countries or regions, enabling accurate localized interception based on differences in laws, sensitive word 1, and sensitive word 2 across different countries or regions. The interception model can simultaneously assess the risk of both input and output dialogue content, promptly blocking prohibited content and improving dialogue security.

[0042] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0043] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0044] Figure 1 This is a flowchart illustrating a method for enhancing the legitimacy of large model dialogues according to an exemplary embodiment.

[0045] Figure 2 This is a block diagram illustrating an apparatus for enhancing the legitimacy of large model dialogues according to an exemplary embodiment.

[0046] Figure 3 This is a block diagram illustrating an apparatus for enhancing the legitimacy of large model dialogues according to an exemplary embodiment.

[0047] Figure 4 This is a block diagram illustrating a computer device according to an exemplary embodiment. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be arbitrarily combined with each other.

[0049] To improve the legitimacy of large model dialogues, this application provides a method for improving the legitimacy of large model dialogues. Figure 1 This is a flowchart illustrating a method for enhancing the legitimacy of large model dialogues according to an exemplary embodiment. (See reference) Figure 1 The method to enhance the legitimacy of large model dialogue includes at least steps S11-S16.

[0050] Step S11: Obtain sensitive word data from multiple data sources in real time.

[0051] The method for enhancing the legitimacy of large-scale model dialogues provided in this application acquires sensitive word data from multiple data sources in real time. For example, it acquires government legal documents from various countries in real time, extracts legal texts from these documents, and then extracts data related to sensitive words based on the legal texts. Alternatively, it acquires prohibited information reported by users based on user reports and extracts sensitive words from it. By acquiring sensitive word data from multiple data sources in real time, the sensitive word database can be updated promptly, prohibited risks can be detected in a timely manner, and timely interception can be achieved, thereby improving network security.

[0052] Step S12: Clean the sensitive word data and generate or update the sensitive word thesaurus.

[0053] Sensitive word data extracted from different data sources may have different expressions for the same semantic meaning. Data cleaning is necessary. Multiple semantically identical sensitive words can be deduplicated. For example, "sensitive word 3" and "sensitive word 4" have the same meaning; multiple semantically identical words can be merged, such as merging "sensitive word 4" into "sensitive word 3," reducing the size of the sensitive word database. Normalization can also be performed on sensitive words. Furthermore, sensitive words can be tagged, with tag types including "political," "sensitive word 1," "sensitive word 2," etc. For example, "sensitive word 3" can be tagged as "political," and "sensitive word B" as "sensitive word 1." The acquired sensitive words are added to a designated database to generate a sensitive word database for use by the interception program. Additionally, as the system runs, when new sensitive words are acquired, they are simply added to the sensitive word database to update it.

[0054] Step S13: Vectorize the sensitive words in the sensitive word lexicon to generate sensitive word vectors.

[0055] The sensitive words in the sensitive word lexicon are vectorized to generate sensitive word vectors. For example, the BERT multilingual model can be used to vectorize the sensitive words in the sensitive word lexicon.

[0056] Step S14: Cluster the sensitive word vectors and calculate the center vector of each semantic cluster after clustering.

[0057] After processing sensitive words into vectors, these vectors are clustered. For example, the DBSCAN algorithm is used to cluster the embedded vectors, generating prohibited semantic clusters. For instance, "sensitive word 3" and "sensitive word 5" are clustered into the "circumventing regulation" cluster, and "sensitive word B" and "sensitive word 6" are clustered into the "sensitive word 1 prohibition" cluster. The center vector of each semantic cluster is then calculated. The center vector represents the core semantic features of a class of sensitive words.

[0058] Step S15: Use the interception model to extract keywords and embedding vectors from the input and output respectively. Based on the matching results of keywords and sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster, determine the risk score of the input or output.

[0059] The interception model takes text and context as input and outputs risk probability. It is pre-trained with training data including positive samples (labeled prohibited content) and negative samples (safe content), and a loss function is set. Once trained, the model can be deployed to the network.

[0060] The interception model can be the T5-Small model, which is a compressed version of the T5 model. It aims to provide a smaller but still efficient text generation solution through optimization and simplification.

[0061] In applications, interception models can be used to extract keywords and embedding vectors from the input and output, respectively. If the keywords match sensitive words, the dialogue is considered risky. If the keywords and sensitive words do not match, the embedding vectors of the input or output can be further extracted, and the similarity between the embedding vectors and the center vectors of each semantic cluster can be calculated. If the embedding vector is highly similar to a certain semantic cluster, it indicates that the semantics of the input or output content are highly similar to the semantics represented by the center vector of the semantic cluster, and this also indicates a risk. Based on the matching results of keywords and sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster, the existence of a risk in the dialogue can be determined, and the magnitude of the risk can be further determined and scored.

[0062] Step S16: When the input or output risk score is greater than or equal to the risk threshold, the dialogue content is intercepted.

[0063] When the risk score of the input or output is greater than or equal to the preset risk threshold, it indicates that the probability of the dialogue content being prohibited is very high. The interception model will intercept it in time to prevent the input or output of illegal content and improve the legality of the dialogue.

[0064] In this embodiment, sensitive word data is acquired in real time from multiple data sources, and a sensitive word database is generated or updated. This ensures that the sensitive word database is always dynamically updated, requiring no manual maintenance. It can adapt to the dynamic changes in laws of various countries in real time, promptly review online content, and prevent the spread of prohibited content. Simultaneously, an interception model performs bidirectional interception of input and output, preventing the output of illegal content and avoiding the large model being blocked or reported. Furthermore, by vectorizing and clustering sensitive words, the laws of various countries or regions, as well as sensitive words related to each sensitive word 1, can be clustered into a semantic cluster, achieving comprehensive adaptation across countries and across sensitive words 1.

[0065] In an exemplary embodiment, in step S11, the multiple data sources include:

[0066] Government legal documents, user-reported prohibited words, and prohibited words that are trending online.

[0067] In practical applications, APIs can be used to connect to local government websites in real time to obtain sensitive words as defined by local laws. For example, a "Sensitive Word List 2024" can be retrieved from a government website. If the local government updates its sensitive word list, the sensitive word database can be updated promptly, ensuring its timeliness and effectiveness.

[0068] Users can also be provided with a "Report Inappropriate Content" button. When users discover potentially illegal or inappropriate content, they can click this button to report it and submit modifications. Upon receiving a user report, if the reported content is confirmed to be true, the sensitive word database will be updated promptly.

[0069] It can also monitor trending online content in real time. For example, it can monitor search engine hot keywords to determine if there is any illegal content among the top N search terms. For instance, it can monitor the hourly search volume of hot keywords in a certain region, and add "sensitive word 5" to the sensitive word database when it appears in the top ten search results for that keyword. Alternatively, it can monitor trending online topics, promptly obtaining trending topics within 30 minutes. If a trending topic violates public order and good morals and may lead to negative public opinion, the relevant keywords can be added to the sensitive word database.

[0070] By acquiring sensitive word data from multiple data sources in real time, we ensure that the sensitive word database is updated at any time, improve the accuracy of interception, prevent the output of illegal and prohibited content, and enhance session security.

[0071] In one exemplary embodiment, the method for enhancing the legitimacy of large model dialogues further includes:

[0072] The interception model is trained in real time or periodically with an updated sensitive word lexicon.

[0073] After updating the sensitive word database, the interception model is trained using negative samples containing new sensitive words or with the same semantics as the new sensitive words to improve the interception accuracy of the interception model.

[0074] The interception model can be trained in real time each time new sensitive words are acquired, ensuring that the new sensitive word database takes effect promptly and that illegal content is blocked in a timely manner. Alternatively, the interception model can be trained periodically, for example, by triggering incremental training every hour, reducing the number of training sessions required for the interception model.

[0075] In an exemplary embodiment, the interception model includes a first interception model and a second interception model. When the first interception model is deployed online, the second interception model is trained in real time or periodically. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined period of time. Once the comparison results are consistent, the first interception model is taken offline.

[0076] In this embodiment, two identical interception models can be set up: one for online interception and the other for real-time training based on the updated sensitive word database. The first interception model refers to the online interception model, and the second interception model refers to the offline interception model. When the first interception model is running online, it uses the current prohibited word database for interception. If new sensitive words are acquired at this time, the offline second interception model can be trained using the new sensitive word database. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined time, for example, one minute. After the results are consistent, the first interception model is taken offline. Model training and model deployment do not conflict with each other, achieving uninterrupted content review and interception.

[0077] In an exemplary embodiment, step S15, determining the input risk score based on the matching results of keywords and sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster, includes:

[0078] Extract keywords from the input text. When a keyword matches a sensitive word, determine the exact match score. When the exact match score is greater than or equal to the first preset threshold, use the exact match score as the input risk score.

[0079] When the exact match score is less than the first preset threshold, the input text embedding vector is extracted, and the similarity between the embedding vector and the center vector is calculated. When the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score.

[0080] When the similarity is less than the second preset threshold, the dialogue history is obtained, the context matching score is determined, and the exact matching score, similarity, and context matching score are multiplied by their respective weight coefficients and summed to determine the input risk score.

[0081] The interception model extracts keywords from the input text. If a keyword matches a sensitive word, an exact match score is determined. Weights can be assigned to sensitive words in the sensitive word dictionary; different sensitive words can have the same or different weights. The weights of sensitive words can be set and adjusted according to actual needs.

[0082] For example, all sensitive words have the same weight score of 0.4. The exact match score is determined based on the number of sensitive words matched. If one keyword matches a sensitive word, the exact match score is 0.4; if two keywords match a sensitive word, the exact match score is 0.8.

[0083] For example, the weight of the sensitive word "bypass" is set to 0.4, the weight of "censorship" is set to 0.4, and the weight of "sensitive word 3" is set to 0.7. Given the input text "How to bypass censorship?", the keywords "bypass" and "censorship" can be extracted, resulting in an exact match score of 0.4 + 0.4 = 0.8. Similarly, given the input text "How to detect sensitive word 3?", the keyword "sensitive word 3" can be extracted, resulting in an exact match score of 0.7.

[0084] If the exact match score is greater than or equal to the first preset threshold, the exact match score can be used as the input risk score.

[0085] If the exact match score is less than the first preset threshold, the embedding vector of the input text is further extracted, and the similarity between the embedding vector and the center vector of each semantic cluster is calculated. If the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score. For example, the embedding vector extracted from the user's input text has a similarity of 0.92 with the center vector of the "circumventing regulation" cluster and a similarity of 0.25 with the center vector of the "sensitive word 1 taboo" cluster. The maximum similarity value of 0.92 is taken. If it is greater than or equal to the second preset threshold, the similarity of 0.92 is used as the input risk score.

[0086] If the exact match score is less than the first preset threshold, and the similarity between the embedding vector and the center vector of each semantic cluster is also less than the second preset threshold, then the dialogue history is retrieved to determine the context match score. The context match score, exact match score, and similarity are multiplied by their respective weight coefficients and summed to determine the risk score.

[0087] The context matching score can be calculated as the sum of the similarity between the embedding vectors of the current input text and the dialogue history, and the weighted value of the sensitive word density. It can be expressed as: C = cos(E) input E context )+λ×Keyword_count. Where C is the context matching score; cos(E input E context E represents the similarity between the embedding vectors of the current input text and the dialogue history. input For the currently input text, E context Let C be the dialogue history; λ×Keyword_Count is the sensitive word density weighting value, where λ is the sensitive word weight coefficient and Keyword_Count is the number of sensitive words. For example, if the user's dialogue history contains the sensitive words "sensitive word 5" and "sensitive word 3", λ = 0.1, and the similarity between the current input text and the embedding vector of the dialogue history is 0.5, then C = 0.5 + 0.1 × 2 = 0.7.

[0088] The exact match score, similarity score, and context match score are multiplied by their respective weight coefficients and then summed to obtain the total score, which is determined as the input risk score. The total score can be expressed as: S = αK + βSemantic + γC, where S is the total score, K is the exact match score, α is the weight coefficient of the exact match score, Semantic is the similarity between the embedding vector and the center vector, β is the similarity weight coefficient, C is the context match score, and γ is the weight coefficient of the context match score.

[0089] The weighting coefficients for exact match, similarity, and context matching scores can be optimized based on labeled datasets through web search or gradient descent to maximize interception accuracy; they can also be set based on domain knowledge or past experience. For example, since semantic matching is more flexible than keyword matching and can cover variant expressions, the similarity between the embedding vector and the center vector is assigned the highest weight of 0.5; keyword matching can achieve fast interception, but it is easy to bypass the interception model's review through word splitting, homophones, etc., so its weight is secondary, and the exact match score is assigned a weight of 0.4; the context matching score can prevent progressive attacks, but it is prone to false interception, so the context matching score is assigned a weight of 0.1.

[0090] The first and second preset thresholds can be set according to actual conditions. They can be the same or different, but both should be higher than the risk threshold. That is, if the exact match score is greater than or equal to the first preset threshold, using the exact match score as the input risk score will inevitably result in a score greater than the risk threshold. In this case, similarity calculation and total score calculation are unnecessary, and the input can be directly blocked. If the similarity is greater than or equal to the second preset threshold, using the similarity as the input risk score will also inevitably result in a score greater than the risk threshold. Therefore, total score calculation is unnecessary, and the input can be directly blocked. Only when the exact match score is less than the first preset threshold and the similarity is less than the second preset threshold is total score calculation required. If the total score is greater than or equal to the risk threshold, the input will be blocked.

[0091] For example, in one exemplary embodiment, the risk threshold is 0.85, the first preset threshold is 0.95, and the second preset threshold is 0.92.

[0092] User 1 entered "How to use sensitive word 5 to achieve sensitive word 3?", and the keywords "sensitive word 5" and "sensitive word 3" were matched. "Sensitive word 5" had a weight of 0.4 and "sensitive word 3" had a weight of 0.7. The exact match score was 1.1, which is greater than the first preset threshold. The conversation was directly blocked and the interface displayed "Content contains prohibited information. Please modify and resubmit".

[0093] User 2 inputs "How to bypass censorship?", which matches the sensitive keywords "bypass" and "censorship", each with a weight score of 0.4. The exact match score is 0.8, which is less than the first preset threshold of 0.95. Therefore, the embedding vector of the input content is further extracted, and the similarity between the embedding vector and the center vector of each cluster is calculated. Suppose that the similarity between the embedding vector of "How to bypass censorship?" and "circumvent regulation" is 0.92, which is greater than or equal to the preset threshold of 0.92. The user input is intercepted, and the interface displays "Content contains prohibited information. Please modify and resubmit".

[0094] User 3 typed "How to access the internet freely?"

[0095] The keywords "science", "internet access", and "scientific internet access" were extracted from the input text. "Scientific internet access" matched the sensitive word list, with an exact match score of 0.6, which is less than the first preset threshold of 0.95. Further extraction of the embedding vector of the input content showed that the embedding vector of "how to access the internet scientifically?" had the highest similarity to "circumvent regulation", at 0.9, which is less than the second preset threshold of 0.92. The user 3's conversation history was retrieved, and the user's conversation history contained the sensitive word "sensitive word 5". According to the formula C = cos(E... input E context The similarity between the input text and the embedding vectors of the dialogue history is 0.5, λ = 0.1, the number of sensitive words is 1, and C = 0.5 + 0.1 × 1 = 0.51. The total score is calculated as: S = αK + βSemantic + γC = 0.4 × 0.6 + 0.5 × 0.9 + 0.1 × 0.51 = 0.741. The total score is less than the risk threshold, so the input dialogue is allowed.

[0096] User 4 typed "How to access the internet freely?"

[0097] The keywords "science," "internet access," and "scientific internet access" were extracted from the input text. "Scientific internet access" matched the sensitive word list, with an exact match score of 0.6, which is less than the first preset threshold of 0.95. Further extraction of the embedding vector of the input text showed that the embedding vector of "How to access the internet scientifically?" had the highest similarity to "circumvent regulation," at 0.9, which is less than the second preset threshold of 0.92. The user 4's conversation history was retrieved. This history contained multiple sensitive words such as "sensitive word 5," "sensitive word 3," "ladder," "censorship," and "bypass," assuming a total of 10. According to the formula C = cos(E... input E contextThe similarity between the input text and the embedding vectors of the dialogue history is 0.8, λ = 0.1, the number of sensitive words is 10, and C = 0.8 + 0.1 × 10 = 1.8. The total score is calculated as: S = αK + βSemantic + γC = 0.4 × 0.6 + 0.5 × 0.9 + 0.1 × 1.8 = 0.87. If the total score is greater than the risk threshold, the input dialogue is blocked.

[0098] As can be seen, the method for enhancing the legality of large-scale model dialogues provided in this application can quickly intercept user input text when the keywords are sensitive words and the exact matching score is greater than or equal to the first preset threshold. If the user uses other words to replace the sensitive words in an attempt to evade the interception model's review, the interception model can further determine the similarity between the embedding vector of the input text and the center vectors of each cluster, and perform semantic review of the input text. If the similarity is greater than or equal to the second preset threshold, the user input can be intercepted, effectively identifying and intercepting various variant expressions. When the peak of the exact matching of the user input is less than the first preset threshold, and the similarity of the semantic review is also less than the first preset threshold, the user's dialogue history can be obtained, and a total score can be calculated. Once the total score is greater than or equal to the risk threshold, the user's input will be intercepted. If a user frequently violates the rules, the total score will increase, increasing the sensitivity of interception for that user and effectively preventing progressive attacks.

[0099] In one exemplary embodiment, the output risk score is determined based on the matching results of keywords and sensitive words, and the similarity between the embedding vector and the center vector of each semantic cluster.

[0100] Extract keywords from the output text. When a keyword matches a sensitive word, determine the exact match score. When the exact match score is greater than or equal to the third preset threshold, use the exact match score as the output risk score.

[0101] When the exact match score is less than the third preset threshold, the output text embedding vector is extracted, the similarity between the embedding vector and the center vector is calculated, and the similarity is used as the output risk score.

[0102] When the large model outputs content based on the user's input text, the interception model performs a second review of the output content. First, keywords are extracted from the output text. When a keyword matches a sensitive word, a precise match score is determined. If the precise match score is greater than or equal to a third preset threshold, it can be used as the risk score. If the precise match score is less than the third preset threshold, the output text embedding vector is extracted, and the similarity between the embedding vector and the center vector is calculated. This similarity score is used as the output risk score. The third preset threshold can be the same as the first preset threshold, meaning that the same sensitive word library is used for both input and output review.

[0103] For example, in an exemplary embodiment, the risk threshold is 0.85, and the third preset threshold is 0.95. The interception model located in country A retrieves the legal document "Sensitive Word List 2024" from the local government website. "Sensitive word B" is listed as a sensitive word by the local government and is considered a high-risk term. "Sensitive word B" is added to the sensitive word database, and its weight is set to 0.98. The large model's output text is detected to contain the keyword "sensitive word B," with an exact match score of 0.98, which is greater than the third preset threshold of 0.95. The output risk score is 0.98, which is greater than the risk threshold of 0.85. Therefore, the output content is intercepted, and the user is shown "According to local regulations, this content is not displayed."

[0104] If the large model generates the text "In some regions, the use of the sensitive word 5 may not be allowed, but there are still ways to access restricted content," the keywords of the output text are extracted. Among them, "sensitive word 5" and "access restricted content" match the sensitive word library, with each having a weight of 0.4. The exact match score is 0.8, which is less than the third preset threshold of 0.95. The embedding vector of the output text is further extracted, and the similarity between the embedding vector and the center vector of each cluster is calculated. The similarity with the center vector of the cluster corresponding to Article 5 of the Cybersecurity Law of Country A (prohibiting the promotion of the sensitive word 5) is 0.9. Then the output risk score is 0.9, which is greater than the risk threshold of 0.85. The output content is blocked, and the user is shown "According to local regulations, this content is not allowed to be displayed."

[0105] Through the above embodiments, the method for improving the legality of large-scale model dialogues provided in this application can achieve real-time updates of the sensitive word lexicon, thereby improving the interception accuracy. Simultaneously, it performs vector clustering on sensitive words to generate semantic clusters related to laws, sensitive word 1, and sensitive word 2 in different countries or regions, thus accurately targeting localized interception based on the differences in laws, sensitive word 1, and sensitive word 2 across different countries or regions. The interception model can simultaneously perform risk assessment on both input and output dialogue content, promptly blocking prohibited content and improving dialogue security.

[0106] Figure 2 This is a block diagram illustrating an apparatus for enhancing the legitimacy of large model dialogues according to an exemplary embodiment. (Reference) Figure 2 The device for enhancing the legitimacy of large-scale model dialogue includes: a sensitive word data acquisition module 201, a vectorization module 202, a clustering module 203, a risk assessment module 204, and an interception module 205.

[0107] The sensitive word data acquisition module 201 is configured to acquire sensitive word data from multiple data sources in real time.

[0108] The vectorization module 202 is configured to vectorize sensitive words in the sensitive word lexicon to generate sensitive word vectors.

[0109] The clustering module 203 is configured to cluster sensitive word vectors and calculate the center vector of each semantic cluster after clustering.

[0110] The risk assessment module 204 is configured to use an interception model to extract keywords and embedding vectors from the input and output respectively, and to determine the risk score of the input or output based on the matching results of the keywords and the sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster.

[0111] The interception module 205 is configured to intercept dialogue content when the risk score of the input or output is greater than or equal to the risk threshold.

[0112] Figure 3 This is a block diagram illustrating an apparatus for enhancing the legitimacy of large model dialogues according to an exemplary embodiment. (Reference) Figure 3 The devices for enhancing the legitimacy of large-scale model dialogues also include: training module 301

[0113] The training module 301 is configured to train the interception model with an updated sensitive word lexicon in real time or periodically.

[0114] In an exemplary embodiment, the interception model includes a first interception model and a second interception model. When the first interception model is deployed online, the second interception model is trained in real time. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined period of time. Once the comparison results are consistent, the first interception model is taken offline.

[0115] In one exemplary embodiment, the risk assessment module determines the input risk score by including:

[0116] Extract keywords from the input text. When a keyword matches a sensitive word, determine the exact match score. When the exact match score is greater than or equal to the first preset threshold, use the exact match score as the input risk score.

[0117] When the exact match score is less than the first preset threshold, the input text embedding vector is extracted, and the similarity between the embedding vector and the center vector is calculated. When the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score.

[0118] When the similarity is less than the second preset threshold, the dialogue history is obtained, the context matching score is determined, and the exact matching score, similarity, and context matching score are multiplied by their respective weight coefficients and summed to determine the input risk score.

[0119] In one exemplary embodiment, the risk assessment module determines the output risk score by including:

[0120] Extract keywords from the output text. When a keyword matches a sensitive word, determine the exact match score. When the exact match score is greater than or equal to the third preset threshold, use the exact match score as the output risk score.

[0121] When the exact match score is less than the third preset threshold, the embedding vector of the output text is extracted, the similarity between the embedding vector and the center vector is calculated, and the similarity is used as the output risk score.

[0122] Figure 4 This is a block diagram illustrating a computer device 400 for enhancing the legitimacy of large model dialogues, according to an exemplary embodiment. For example, the computer device 400 may be provided as a server. (Refer to...) Figure 4 The computer device 400 includes a processor 401, the number of which can be set to one or more as needed. The computer device 400 also includes a memory 402 for storing instructions executable by the processor 401, such as application programs. The number of memories can be set to one or more as needed. The stored application programs can be one or more. The processor 401 is configured to execute instructions to perform the aforementioned method for enhancing the legitimacy of large-scale dialogue models.

[0123] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program code. Computer storage media include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data), including but not limited to RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible by a computer. Furthermore, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and can include any information delivery medium.

[0124] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (devices), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] In this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that an article or device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such an article or device. Without further limitation, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the article or device that includes said element.

[0128] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0129] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations fall within the scope of the claims of this application and their equivalents, the intent of this application also includes these modifications and variations.

Claims

1. A method for improving the legitimacy of large-scale model dialogues, characterized in that, include: Real-time acquisition of sensitive word data from multiple data sources; The sensitive word data is cleaned to generate or update the sensitive word lexicon. The sensitive words in the aforementioned sensitive word lexicon are vectorized to generate sensitive word vectors; Cluster the sensitive word vectors and calculate the center vector of each semantic cluster after clustering; The interception model is used to extract keywords and embedding vectors from the input and output respectively. Based on the matching results of the keywords and the sensitive words, the similarity between the embedding vector and the center vector of each semantic cluster, the risk score of the input or output is determined. When the risk score of the input or output is greater than or equal to the risk threshold, the dialogue content is blocked.

2. The method for improving the legitimacy of large-scale model dialogues as described in claim 1, characterized in that, The various data sources include: Government legal documents, user-reported prohibited words, and prohibited words that are trending online.

3. The method for improving the legitimacy of large-scale model dialogues as described in claim 1, characterized in that, Also includes: The interception model is trained in real time or periodically with an updated sensitive word lexicon.

4. The method for improving the legitimacy of large-scale model dialogues as described in claim 3, characterized in that, The interception model includes a first interception model and a second interception model. When the first interception model is deployed online, the second interception model is trained in real time. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined period of time. Once the comparison results are consistent, the first interception model is taken offline.

5. The method for improving the legitimacy of large-scale model dialogues as described in claim 1, characterized in that, Based on the matching results of the keywords and the sensitive words, and the similarity between the embedding vector and the center vector of each semantic cluster, the input risk score is determined as follows: Extract keywords from the input text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a first preset threshold, use the exact match score as the input risk score. When the exact match score is less than the first preset threshold, the embedding vector of the input text is extracted, and the similarity between the embedding vector and the center vector is calculated. When the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score. When the similarity is less than the second preset threshold, the dialogue history is obtained, the context matching score is determined, and the exact matching score, similarity, and context matching score are multiplied by their respective weight coefficients and summed to determine the input risk score.

6. The method for improving the legitimacy of large-scale model dialogues as described in claim 1, characterized in that, Based on the matching results of the keywords and the sensitive words, and the similarity between the embedding vector and the center vector of each semantic cluster, the output risk score is determined to include: Extract keywords from the output text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a third preset threshold, use the exact match score as the output risk score. When the exact match score is less than the third preset threshold, the embedding vector of the output text is extracted, the similarity between the embedding vector and the center vector is calculated, and the similarity is used as the output risk score.

7. An apparatus for enhancing the legitimacy of large-scale model dialogues, characterized in that, include: The sensitive word data acquisition module is used to acquire sensitive word data from multiple data sources in real time. The sensitive word processing module is used to clean the sensitive word data and generate or update the sensitive word lexicon. The vectorization module is used to vectorize the sensitive words in the sensitive word lexicon to generate sensitive word vectors; The clustering module is used to cluster the sensitive word vectors and calculate the center vector of each semantic cluster after clustering. The risk assessment module is used to extract keywords and embedding vectors from the input and output using an interception model, and to determine the risk score of the input or output based on the matching results of the keywords and the sensitive words, the similarity between the embedding vectors and the center vectors of each semantic cluster; The interception module is used to intercept dialogue content when the risk score of the input or output is greater than or equal to the risk threshold.

8. The apparatus for enhancing the legitimacy of large-scale dialogues as described in claim 7, characterized in that, Also includes: The training module is used to train the interception model in real time or periodically with an updated sensitive word lexicon.

9. The apparatus for enhancing the legitimacy of large-scale dialogues as described in claim 8, characterized in that, The interception model includes a first interception model and a second interception model. When the first interception model is deployed online, the second interception model is trained in real time. The trained second interception model is then deployed online and run in parallel with the first interception model for a predetermined period of time. Once the comparison results are consistent, the first interception model is taken offline.

10. The apparatus for enhancing the legitimacy of large-scale dialogues as described in claim 7, characterized in that, The risk assessment module determines the input risk score, including: Extract keywords from the input text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a first preset threshold, use the exact match score as the input risk score. When the exact match score is less than the first preset threshold, the embedding vector of the input text is extracted, and the similarity between the embedding vector and the center vector is calculated. When the similarity is greater than or equal to the second preset threshold, the similarity is used as the input risk score. When the similarity is less than the second preset threshold, the dialogue history is obtained, the context matching score is determined, and the exact matching score, similarity, and context matching score are multiplied by their respective weight coefficients and summed to determine the input risk score.

11. The apparatus for enhancing the legitimacy of large-scale dialogues as described in claim 7, characterized in that, The risk assessment module determines the output risk score, which includes: Extract keywords from the output text. When the keywords match the sensitive words, determine the exact match score. When the exact match score is greater than or equal to a third preset threshold, use the exact match score as the output risk score. When the exact match score is less than the third preset threshold, the embedding vector of the output text is extracted, the similarity between the embedding vector and the center vector is calculated, and the similarity is used as the output risk score.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1-6.

13. A computer device comprising a processor, a memory, and a computer program stored in the memory, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1-6.

Images