Large model sensitive information processing method, computer equipment and storage medium
By verifying sensitive words on the answer text generated by the big model, the problem that the big model may generate sensitive information when responding to user problems is solved, the security and standardization of the output content are achieved, and the overall output quality is improved.
Patent Information
- Application Number
- CN202410946684.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-15
- Publication Date
- 2024-11-08
AI Technical Summary
Large models may generate sensitive information that violates existing laws in response to user problems, resulting in poor social impact and reduced user experience.
By obtaining the question text and inputting it into the big model for reasoning, obtaining the answer text in real time and performing sensitive word verification, including regular checks and segmented verification. If sensitive words are found, output preset answer text to avoid the publication of unpleasant content. .
Effectively detect and process sensitive information generated by the big model, enhance the security and standardization of the output content, and thus improve the overall output quality.
Smart Images

Figure CN118916455A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a large-model sensitive information processing method, computer equipment and storage medium. Background Art
[0002] A big model is a deep learning model trained with massive amounts of data. It has the ability to generate natural language text and deeply understand the meaning of text. It can handle a variety of natural language tasks such as text summarization, question answering, and translation. However, as the application of big models in the field of reasoning question answering becomes more and more widespread, some problems have also emerged. Due to the limitations of factors such as the amount of training data and algorithm optimization, big models sometimes find it difficult to fully follow the current value orientation when responding to user questions and intentions, and may even generate information that violates current laws. The release of such sensitive information will not only cause adverse social impacts, but also reduce the user experience.
[0003] Accordingly, the art needs a new large-model sensitive information processing solution to solve the above problems. Summary of the invention
[0004] In order to overcome the above-mentioned defects, the present application is proposed to solve or at least partially solve the technical problem that sensitive information may be involved when a large model answers user questions.
[0005] In a first aspect, a sensitive information processing method of a large model is provided, the method comprising: obtaining a question text; inputting the question text into a large model for reasoning, obtaining a streaming answer text generated by the large model in real time, and caching the answer text; obtaining word count information of the answer text; based on the word count information of the answer text, performing a sensitive word check on the answer text, the sensitive word check comprising a conventional check or a segmented check; and outputting the answer text based on the sensitive word check result of the answer text.
[0006] In a technical solution of the sensitive information processing method of the above-mentioned large model, before the question text is input into the large model for reasoning, the method also includes: calling a sensitive word service to verify the question text to determine whether the question text contains a combination of sensitive words; if the question text does not contain a combination of sensitive words, inputting the question text into the large model for reasoning; or, if the question text contains a combination of sensitive words, outputting a preset answer text.
[0007] In a technical solution of the sensitive information processing method of the above-mentioned large model, the answer text is routinely checked based on the word count information of the answer text, including: in response to the word count information of the answer text being equal to a first threshold, calling a sensitive word service to check the answer text to determine whether the answer text contains a combination of sensitive words.
[0008] In a technical solution of the sensitive information processing method of the above-mentioned large model, the answer text is output based on the sensitive word verification result of the answer text, including: when the answer text does not contain a combination of sensitive words, the sensitive word verification result of the answer text is passed, and the answer text is output; or, when the answer text contains a combination of sensitive words, the sensitive word verification result of the answer text is failed, and a preset answer text is output.
[0009] In a technical solution of the sensitive information processing method of the above-mentioned large model, the answer text is segmented and verified based on the word count information of the answer text, including: in response to the word count information of the answer text being greater than a first threshold, a preset sliding window is used to slide and intercept the answer text based on a preset step size to obtain multiple text segments; and a sensitive word service is called to verify each of the text segments to determine whether the text segment contains a combined sensitive word.
[0010] In a technical solution of the sensitive information processing method of the above-mentioned large model, the answer text is output based on the sensitive word verification result of the answer text, and also includes: when the text fragment does not contain a combined sensitive word, the sensitive word verification result of the text fragment is passed, and the text fragment that has passed the verification is integrated and output with the output answer text.
[0011] In a technical solution of the sensitive information processing method of the above-mentioned large model, the method also includes: when the text fragment contains a combination of sensitive words, the sensitive word verification result of the text fragment is verification failure, a withdrawal operation is performed on the output answer text, and a preset answer text is output.
[0012] In a technical solution of the sensitive information processing method of the above-mentioned large model, the method also includes: obtaining a sensitive word set, the sensitive word set including multiple sensitive words; combining the sensitive words in the sensitive word set to obtain multiple combined sensitive words; constructing a sensitive word service based on the multiple combined sensitive words and a deterministic finite automaton algorithm, and the sensitive word service is used to verify whether the text contains the combined sensitive words.
[0013] In a second aspect, a computer device is provided, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the method described in any one of the technical solutions of the above-mentioned large model sensitive information processing method is implemented.
[0014] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored therein, wherein the program codes are suitable for being loaded and run by a processor to execute the method described in any one of the technical solutions of the sensitive information processing method of the above-mentioned large model.
[0015] The above one or more technical solutions of the present application have at least one or more of the following beneficial effects:
[0016] The sensitive information processing method of the large model provided in the present application includes: obtaining a question text; inputting the question text into the large model for reasoning, obtaining the streaming answer text generated by the large model in real time, and caching the answer text; obtaining the word count information of the answer text; based on the word count information of the answer text, performing sensitive word verification on the answer text, the sensitive word verification includes conventional verification or segmented verification; based on the sensitive word verification result of the answer text, outputting the answer text. The present application obtains the streaming answer text generated by the large model based on the question text reasoning, caches the answer text, and performs conventional verification or segmented verification on the answer text based on the word count information of the answer text, and then outputs the answer text based on the verification result of the answer text, which can effectively detect and process the sensitive information involved in the answer generated by the large model, enhance the security and standardization of the output content, and further improve the overall output quality of the large model. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The disclosure of the present application will become easier to understand with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. Among them:
[0018] Figure 1 It is a flowchart of the main steps of a sensitive information processing method of a large model according to an embodiment of the present application;
[0019] Figure 2 This is a flowchart of the main steps of constructing a sensitive word service according to an embodiment of the present application;
[0020] Figure 3 is a schematic diagram of a combination of sensitive words according to an embodiment of the present application;
[0021] Figure 4 It is a detailed step flow chart of a sensitive information processing method of a large model according to an embodiment of the present application;
[0022] Figure 5 It is a main structural diagram of a computer device according to an embodiment of the present application.
[0023] Reference numerals:
[0024] 11: processor; 12: memory. DETAILED DESCRIPTION
[0025] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.
[0026] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.
[0027] This application attaches great importance to the security of user personal information and has taken reasonable and feasible security protection measures that meet industry standards to protect user information and prevent personal information from being accessed, disclosed, used, modified, damaged or lost without authorization.
[0028] As the application of big models in the field of reasoning question answering becomes more and more widespread, some problems have also emerged. Due to the limitations of factors such as the amount of training data and algorithm optimization, big models sometimes find it difficult to fully follow the current value orientation when responding to user questions and intentions, and may even generate information that violates current laws. The release of such sensitive information will not only cause adverse social impacts, but also reduce the user experience.
[0029] To this end, the present application provides a sensitive information processing method for a large model, including: obtaining a question text; inputting the question text into a large model for reasoning, obtaining a streaming answer text generated by the large model in real time, and caching the answer text; obtaining the word count information of the answer text; based on the word count information of the answer text, performing a sensitive word check on the answer text, the sensitive word check including a conventional check or a segmented check; based on the sensitive word check result of the answer text, outputting the answer text. The present application obtains a streaming answer text generated by a large model based on question text reasoning, caches the answer text, and based on the word count information of the answer text, performs a conventional check or a segmented check on the answer text, and then outputs the answer text based on the check result of the answer text, which can effectively detect sensitive information in the answer generated by the large model, and perform corresponding processing on the answer text containing sensitive information based on the check result, thereby enhancing the security and standardization of the output answer, thereby further improving the overall output quality of the large model.
[0030] See attached Figure 1 , Figure 1 FIG. 1 is a flow chart of the main steps of a sensitive information processing method of a large model according to an embodiment of the present application. Figure 1 As shown, the sensitive information processing method of the large model in the embodiment of the present application mainly includes the following steps S101 to S105.
[0031] Step S101: Obtain question text.
[0032] In this embodiment, the question text is the question or intention information input by the user in the macro model.
[0033] Step S102: input the question text into the large model for reasoning, obtain the streaming answer text generated by the large model in real time, and cache the answer text.
[0034] In this embodiment, the question text is first input into the large model for reasoning. Since the large model has the characteristic of word-by-word output, it only outputs a part of the complete text each time to form a streaming answer text. To ensure the real-time and coherence of the answer text verification, these streaming output answer texts are cached.
[0035] Step S103: Obtain the word count information of the answer text.
[0036] Step S104: Based on the word count information of the answer text, the answer text is checked for sensitive words, wherein the sensitive word check includes a conventional check or a segmented check.
[0037] In this embodiment, a conventional check or a segmented check is performed on the answer text according to the word count information of the cached answer text.
[0038] Step S105: outputting the answer text based on the sensitive word verification result of the answer text.
[0039] Based on the method described in steps S101 to S105 above, the present application obtains the streaming answer text generated by the large model based on question text reasoning, caches the answer text, and performs regular verification or segmented verification on the answer text based on the word count information of the answer text, and then outputs the answer text based on the verification result of the answer text. This can effectively detect sensitive information involved in the answer generated by the large model, and perform corresponding processing on the answer text containing sensitive information based on the verification result, thereby enhancing the security and standardization of the output answer, thereby further improving the overall output quality of the large model.
[0040] This application pre-builds a sensitive word service before executing step S101.
[0041] In one embodiment, the method further includes: obtaining a sensitive word set, the sensitive word set including multiple sensitive words; combining the sensitive words in the sensitive word set to obtain multiple combined sensitive words; constructing a sensitive word service based on the multiple combined sensitive words and a deterministic finite automaton algorithm, the sensitive word service being used to verify whether the text contains the combined sensitive words.
[0042] See attached Figure 2 , Figure 2 This is a flowchart of the main steps of building a sensitive word service according to an embodiment of the present application. Specifically, the construction of the sensitive word service includes the following steps:
[0043] Step S201: Start.
[0044] Step S202: Acquire a sensitive word set, where the sensitive word set includes a plurality of sensitive words.
[0045] Specifically, the sensitive word set is a set of multiple sensitive words that are pre-collected and organized based on current mainstream values. Sensitive words usually refer to words or phrases that may cause adverse social impacts, violate regulations, or involve sensitive topics. These words or phrases may be politically sensitive, violent, unhealthy, illegal, or involve certain specific people, place names, or names of objects.
[0046] Step S203: combining the sensitive words in the sensitive word set to obtain a plurality of combined sensitive words.
[0047] Specifically, due to the diversity of text expressions, some words are sensitive words only when they appear in combination, but not when they appear independently. In order to avoid verification errors, the collected sensitive words are cleansed and combined to obtain a batch of combined sensitive words. When verifying sensitive words, the words in the combination must appear at the same time for the match to be successful.
[0048] Step S204: constructing a sensitive word service based on the multiple combined sensitive words and determining a finite automaton algorithm, wherein the sensitive word service is used to verify whether the text contains the combined sensitive word.
[0049] Deterministic Finite Automaton (DFA) is an algorithm for string matching. The core of DFA is to build a finite state automaton, which consists of a finite number of states and transitions between states. It consists of five elements: a finite set of states, a finite set of input symbols, a transition function, an initial state, and a set of accepting states. Its principle is that given an input, starting from the initial state, according to the rules of the input symbol and the state transition function, it automatically moves forward to the next state until it reaches the accepting state or cannot continue to transfer.
[0050] Specifically, the obtained multiple combined sensitive words are organized into a tree structure, a combined sensitive word dictionary tree is established, and the sensitive word service is determined based on the algorithm strategy of the finite automaton algorithm and the combined sensitive word dictionary tree. The sensitive word service has an http interface for easy calling.
[0051] Each node in the combined sensitive word dictionary tree is used to identify a sensitive word, and the phrase consisting of the complete path from the root node to the child node is considered a combined sensitive word. Figure 3 As shown, assuming that A, B, and C are all sensitive words, respectively, on three nodes, the phrase ABC consisting of the complete path from the root node to the child node is a combined sensitive word.
[0052] When using the sensitive word service for verification, the characters of the text to be verified are matched one by one with the nodes of the combined sensitive word dictionary tree. When the i-th character of the text to be verified matches the A node, the i+1-th character is matched with the child node associated with the A node. If the i+1-th character successfully matches the B node, the i+2-th character is matched with the child node C associated with the B node. If the i+2-th character successfully matches the C node, it is determined that the text to be verified contains the combined sensitive word, and the verification result is verification failure.
[0053] Step S205: End
[0054] Steps S101 to S105 are further described below.
[0055] With respect to step S101, the question text is obtained.
[0056] Specifically, the question or intention information input by the user in the large model is obtained as the question text.
[0057] In one embodiment, before the question text is input into the big model for reasoning, the method also includes: calling a sensitive word service to verify the question text to determine whether the question text contains a combination of sensitive words; if the question text does not contain a combination of sensitive words, inputting the question text into the big model for reasoning; or, if the question text contains a combination of sensitive words, outputting a preset answer text.
[0058] Specifically, when the question text input by the user into the big model involves sensitive information, such sensitive information may affect the reasoning of the big model. Therefore, before reasoning, the sensitive word service is called to verify the question text input by the user, and the question text is input into the sensitive word service, and matched with the combined sensitive words in the sensitive word service to determine whether the question text contains the combined sensitive words. If the question text contains the combined sensitive words, a preset answer text is output to the user. The preset answer text can be "Let's change the topic" to prompt the user to stop the question and answer process involving sensitive information. If the question text does not contain the combined sensitive words, the question text is input into the big model for reasoning.
[0059] With respect to step S102, the question text is input into the large model for reasoning, the streaming answer text generated by the large model is obtained in real time, and the answer text is cached.
[0060] Specifically, the question text is first input into the large model for reasoning. Since the large model has the characteristic of word-by-word output, it only outputs a part of the complete text each time to form a streaming answer text. In order to ensure the real-time and consistency of the answer text verification, these streaming output answer texts are cached.
[0061] With respect to step S103, the word count information of the answer text is obtained.
[0062] Specifically, after obtaining the streaming answer text generated by the large model in real time and caching it, a conventional word count tool can be used to detect the word count information of the answer text. The word count information only counts the number of Chinese characters, English characters, etc., and punctuation marks, spaces, etc. are not included in the word count.
[0063] Regarding step S104, in one embodiment, the routine verification of the answer text based on the word count information of the answer text includes: in response to the word count information of the answer text being equal to the first threshold, invoking a sensitive word service to verify the answer text and determine whether the answer text contains combined sensitive words.
[0064] Specifically, the word count of the answer text output by the large model increases gradually. After the word count information of the cached answer text reaches the first threshold, a sensitive word service is invoked to perform routine verification on the answer text. Routine verification means inputting the current answer text into the sensitive word service for the first verification to determine whether the current answer text contains combined sensitive words. The first threshold can be set to a relatively small value, such as 10 words, 15 words, or 20 words.
[0065] In a specific embodiment, assume that "A", "B", and "C" are all defined as sensitive words, and they form a combined sensitive word "ABC". When the cached answer text is "Adolescents ABC once addicted", the word count information of the answer text is 10, which is equal to the first threshold (10 words). The sensitive word service is used to perform a character-by-character verification on the answer text. Starting from the root node of the combined sensitive word trie, traverse the tree according to the character order in the text.
[0066] In this embodiment, the answer text is "Adolescents ABC once addicted". Starting from the first character "青 (qīng)", it is not on the root node of the tree, so continue to check the next characters "少 (shǎo)" and "年 (nián)". Similarly, "少 (shǎo)" and "年 (nián)" are not on the root node, so continue to check.
[0067] When checking the character "A", a corresponding branch is found on the root node of the tree, which means that "A" is the starting character of a combined sensitive word. At this time, the state switches, and the next search range becomes the child nodes of "A".
[0068] Among the child nodes associated with "A", the next character "B" of the answer text "A" matches successfully with the child node "B" of "A". The state switches again, and the next search range becomes the child nodes of "B". The next character "C" of the answer text "B" is matched with the child nodes associated with the "B" node. The character "C" of the answer text matches successfully with the child node "C", and the child node "C" is the leaf node of the root node "A", which indicates that a complete combined sensitive word "ABC" has been matched, and it is determined that the answer text contains combined sensitive words.
[0069] With respect to step S105, in one embodiment, the answer text is output based on the sensitive word verification result of the answer text, including: when the answer text does not contain a combination of sensitive words, the sensitive word verification result of the answer text is passed, and the answer text is output; or, when the answer text contains a combination of sensitive words, the sensitive word verification result of the answer text is failed, and a preset answer text is output.
[0070] Specifically, when it is determined after the first verification that the current answer text does not contain a combination of sensitive words, the current answer text is immediately output to the user to achieve the effect of timely responding to the user and ensure that the output content complies with regulations and requirements.
[0071] After the first verification, if it is determined that the current answer text contains a combination of sensitive words, in order to avoid directly outputting content that may cause problems or discomfort, a preset answer text is output. The preset answer text can be "Let's change the topic". In this way, it can ensure that the user experience will not be affected by the appearance of sensitive words, while maintaining the security and standardization of the output content.
[0072] Further, with respect to step S104, in one embodiment, the answer text is segmented and verified based on the word count information of the answer text, including: in response to the word count information of the answer text being greater than a first threshold, a preset sliding window is used to perform sliding interception on the answer text based on a preset step size to obtain multiple text segments; and a sensitive word service is called to verify each of the text segments to determine whether the text segment contains a combined sensitive word.
[0073] Specifically, the answer texts streamed out of the cached large model will gradually increase. When the number of words in the cached answer text exceeds a certain threshold, the detection efficiency may be affected by the length of the detection text. Therefore, after the word count information of the answer text exceeds the first threshold, a preset sliding window is used to slide and intercept the answer text from the starting position with a preset interception length and a preset step length. The preset interception length can be pre-set, or it can be dynamically determined based on the word count information of the answer text. For example, the more words in the answer text, the larger the interception length. After the sliding window intercepts the answer text, multiple text fragments are obtained. The text fragments are input into the sensitive word service one by one for verification to determine whether the text fragment contains a combination of sensitive words.
[0074] In a specific embodiment, assuming that the currently cached answer text is "Once a teenager becomes addicted to ABC", the word count information of the answer text is 11, and the word count exceeds the first threshold (10 words), the answer text is intercepted using a preset sliding window. In this embodiment, the preset step size is 1, and the preset interception length is 10. The answer text is slidingly intercepted using a sliding window, and multiple text segments can be obtained, such as "Once a teenager becomes addicted to ABC" and "Once a teenager becomes addicted to ABC", and the text segments are input into the sensitive word service one by one for verification to determine whether the text segment contains a combination of sensitive words.
[0075] With respect to step S105, in one embodiment, the answer text is output based on the sensitive word verification result of the answer text, and further includes: when the text segment does not contain a combined sensitive word, the sensitive word verification result of the text segment is passed, and the text segment that has passed the verification is integrated and output with the output answer text.
[0076] Specifically, after verifying the text fragments one by one, if the text fragment does not contain the combined sensitive words, the verified text fragment will be integrated with the answer text that has been output and output. The integration can be to delete the repeated content in the text fragment and the answer text that has been output, and output the content that has not been output in the text fragment as a supplementary answer text, until the final answer text generated by the large model is output.
[0077] In a specific embodiment, assuming that the answer text that has been output is "The physical and mental development of adolescents is not yet complete", and the large model continues to output "mature", the cached answer text becomes "The physical and mental development of adolescents is not yet mature", and the number of words in the answer text exceeds the first threshold. The answer text is intercepted by a sliding window to obtain text fragments: "The physical and mental development of adolescents is not yet complete", "The physical and mental development of teenagers is not yet mature", and the sensitive word service verifies the two text fragments. If the verification passes, the "mature" newly output by the large model is used as the supplementary answer text and is merged with the output answer text.
[0078] In one embodiment, the method further includes: when the text segment contains a combined sensitive word, the sensitive word verification result of the text segment is verification failure, performing a withdrawal operation on the output answer text, and outputting a preset answer text.
[0079] Specifically, when verifying the text fragments one by one, if a text fragment contains a combination of sensitive words, the answer text that has been output is withdrawn, and a preset answer text is output to the user. The preset answer text may be "Let's change the topic."
[0080] In a specific embodiment, suppose the cached answer text is "The physical and mental development of adolescents is not yet mature once AB", at this time, the large model continues to infer and generates "C", then the currently cached answer text becomes "The physical and mental development of adolescents is not yet mature once ABC", and the answer text is intercepted by a sliding window to obtain 7 text fragments, namely "The physical and mental development of adolescents is not yet mature", "The physical and mental development of teenagers is not yet mature", "The physical and mental development of young people is not yet mature", "The physical and mental development of young people is not yet mature", "The physical and mental development of young people is not yet mature", "The physical and mental development is not yet mature once", "The mental development is not yet mature once A", "The development is not yet mature once AB" and "The development is not yet mature once ABC", the sensitive word service verifies the text fragments one by one, outputs the text fragments that pass the verification, and after verifying the last text fragment, because the last text fragment contains the combined sensitive word "ABC", the verification fails, and the "The physical and mental development of adolescents is not yet mature once AB" that has been output by the large model is withdrawn, and the preset answer text is output.
[0081] See attached Figure 4 , Figure 4 FIG. 1 is a schematic diagram of a detailed step flow of a sensitive information processing method of a large model according to an embodiment of the present application. Figure 4 As shown, the sensitive information processing method of the large model of the present application includes the following steps:
[0082] Step S301: Start
[0083] Step S302: Obtain the question text input by the user.
[0084] Step S303: calling the sensitive word service to check the question text and determine whether the question text contains a combination of sensitive words. If so, executing step S315; if not, executing step S304.
[0085] Step S304: Input the question text into the large model for reasoning.
[0086] Step S305: Obtain the streaming answer text generated by the large model in real time and cache the answer text.
[0087] Step S306: Obtain the word count information of the answer text.
[0088] Step S307: Determine whether the word count information of the answer text is equal to the first threshold, if so, execute step S308; if not, execute step S311.
[0089] Step S308: Call the sensitive word service to perform routine verification on the answer text to determine whether the answer text contains a combination of sensitive words. If so, execute step S310; if not, execute step S309.
[0090] Step S309: Output the answer text.
[0091] Step S310: Output a preset answer text.
[0092] Step S311: Determine whether the word count information of the answer text is greater than a first threshold, if so, execute step S312; if not, return to step S305.
[0093] Step S312: Use a preset sliding window to perform sliding interception on the answer text based on a preset step size to obtain multiple text segments.
[0094] Step S313: calling the sensitive word service to check each text segment to determine whether the text segment contains a combined sensitive word. If so, executing step S315; if not, executing step S314.
[0095] Step S314: Integrate and output the text fragments with the output answer text until all the text fragments that have passed the verification are output.
[0096] Step S315: Output the preset answer text.
[0097] Step S316: End.
[0098] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These adjusted schemes are equivalent to the technical schemes described in this application, and therefore will also fall within the scope of protection of this application.
[0099] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.
[0100] Another aspect of the present application provides a computer device.
[0101] See attached Figure 5 , Figure 5It is exemplarily shown that the processor 11 and the memory 12 are connected through bus communication. In a computer device embodiment according to the present application, the computer device includes at least one processor 11 and at least one memory 12. The memory 12 can be configured to store a program for executing the sensitive information processing method of the large model of the above-mentioned method embodiment, and the processor 11 can be configured to execute the program in the memory, which includes but is not limited to the program for executing the sensitive information processing method of the large model of the above-mentioned method embodiment. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer device can be a computer device formed by various electronic devices.
[0102] Another aspect of the present application also provides a computer-readable storage medium.
[0103] In an embodiment of a computer-readable storage medium according to the present application, the computer-readable storage medium may be configured to store a program for executing the sensitive information processing method of the large model of the above-mentioned method embodiment, and the program may be loaded and run by the processor to implement the sensitive information processing method of the above-mentioned large model. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium may be a storage device formed by various electronic devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.
[0104] So far, the technical solution of the present application has been described in conjunction with an embodiment shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.
Claims
1. A sensitive information processing method for a large model, characterized in that: The method comprises: Get the question text; Input the question text into the big model for reasoning, obtain the streaming answer text generated by the big model in real time, and cache the answer text; Obtain word count information of the answer text; Based on the word count information of the answer text, performing sensitive word check on the answer text, wherein the sensitive word check includes conventional check or segmented check; Based on the sensitive word verification result of the answer text, the answer text is output.
2. The sensitive information processing method of a large model according to claim 1 is characterized in that: Before inputting the question text into the large model for reasoning, the method further includes: Calling a sensitive word service to verify the question text to determine whether the question text contains a combination of sensitive words; In the case where the question text does not contain the combined sensitive words, the question text is input into the large model for reasoning; or, When the question text contains a combination of sensitive words, a preset answer text is output.
3. The sensitive information processing method of a large model according to claim 1 is characterized in that: The step of performing a routine check on the answer text based on the word count information of the answer text comprises: In response to the word count information of the answer text being equal to the first threshold, a sensitive word service is called to verify the answer text to determine whether the answer text contains a combination of sensitive words.
4. The sensitive information processing method of a large model according to claim 3 is characterized in that: The step of outputting the answer text based on the sensitive word verification result of the answer text comprises: In the case where the answer text does not contain the combined sensitive words, the sensitive word verification result of the answer text is passed, and the answer text is output; or, In the case where the answer text contains a combination of sensitive words, the sensitive word verification result of the answer text is verification failure, and a preset answer text is output.
5. The sensitive information processing method of a large model according to claim 4 is characterized in that: The step of performing segmented verification on the answer text based on the word count information of the answer text comprises: In response to the word count information of the answer text being greater than a first threshold, using a preset sliding window to perform sliding interception on the answer text based on a preset step length to obtain a plurality of text segments; The sensitive word service is called to verify each of the text segments to determine whether the text segment contains a combined sensitive word.
6. The sensitive information processing method of a large model according to claim 5 is characterized in that: The step of outputting the answer text based on the sensitive word verification result of the answer text further includes: In the case that the text segment does not contain the combined sensitive word, the sensitive word verification result of the text segment is passed, and the text segment that has passed the verification is integrated and output with the output answer text.
7. The sensitive information processing method of a large model according to claim 5 is characterized in that: The method further comprises: In the case that the text segment contains a combination of sensitive words, the sensitive word verification result of the text segment is verification failure, a withdrawal operation is performed on the output answer text, and a preset answer text is output.
8. The sensitive information processing method of a large model according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtain a sensitive word set, where the sensitive word set includes multiple sensitive words; Combining the sensitive words in the sensitive word set to obtain a plurality of combined sensitive words; A sensitive word service is constructed based on the multiple combined sensitive words and a determined finite automaton algorithm, wherein the sensitive word service is used to verify whether a text contains the combined sensitive words.
9. A computer device comprising at least one processor and at least one memory, wherein the memory is suitable for storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by the processor to execute the sensitive information processing method of the large model according to any one of claims 1 to 8.
10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the sensitive information processing method of the large model according to any one of claims 1 to 8.
Citation Information
Cited By
Model risk assessment method and device, electronic equipment and storage medium
CN120744415A
