Sensitive word review method, computing device and computer readable storage medium

Abstract

The invention discloses a sensitive word review method, computing equipment and a computer readable storage medium, and the sensitive word review method comprises the steps that a preset sensitive word review mechanism is acquired, and the sensitive word review mechanism comprises a sensitive word list and a violation processing strategy; performing sensitive word review on input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list; and when it is detected that the input content and / or the pre-output content contain the sensitive words in the sensitive word list, executing a violation processing strategy. According to the sensitive word review method provided by the invention, sensitive word review can be performed on the input content and the pre-output content of the generative artificial intelligence platform according to the preset sensitive word review mechanism, so that the flexibility and expansibility of sensitive word review are enhanced.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a sensitive word review method, a computing device, and a computer-readable storage medium. Background Art

[0002] Generative AI (Artificial Intelligence Generated Content, AIGC) refers to the technology based on artificial intelligence technical methods such as generative adversarial networks and large pre-trained models, which generates relevant content with appropriate generalization capabilities through learning and recognition of existing data. The core idea of ​​AIGC technology is to use artificial intelligence algorithms to generate content with certain creativity and quality. Through training models and learning from a large amount of data, AIGC can generate relevant content based on input conditions or guidance. The Large Language Model (LLM) is one of the technical means to achieve AIGC, especially in the generation of text content. With its excellent semantic understanding and text generation capabilities, LLM plays a key role in many fields such as question-answering systems, intelligent customer service, online education, and information retrieval. However, the widespread deployment of these models has also triggered a series of new challenges, especially for the supervision of content security and compliance.

[0003] Since the AIGC platform creates content through LLM obtained through training based on massive amounts of Internet text, it is inevitable that some content containing sensitive information or not suitable for public discussion will be learned during the training process. When users ask questions in the question-and-answer scenario, the AIGC platform may return answers containing sensitive words due to its powerful association and generation capabilities. This may not only lead to a decline in user experience, but may also violate relevant laws and regulations and cause social ethical disputes. In addition, during the training process, if the company's internal business data is used for training, when the AIGC platform outputs content, it will inevitably involve the company's internal data and privacy information. Directly using a general sensitive word review tool may not help the company filter out these words involving confidential information of the company, posing a greater threat to the security of the company's own information. Summary of the invention

[0004] The purpose of this application is to provide a sensitive word review method, a computing device, and a computer-readable storage medium, which can perform sensitive word review on the input content and pre-output content of a generative artificial intelligence platform according to a preset sensitive word review mechanism, thereby enhancing the flexibility and scalability of sensitive word review.

[0005] To achieve the above objectives:

[0006] In a first aspect, an embodiment of the present application provides a sensitive word review method, the method comprising:

[0007] Obtaining a preset sensitive word review mechanism, wherein the sensitive word review mechanism includes a sensitive word list and a violation handling strategy;

[0008] Performing sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list;

[0009] When it is detected that the input content and / or pre-output content contains a sensitive word in the sensitive word list, the violation handling strategy is executed.

[0010] In one embodiment, obtaining a preset sensitive word review mechanism includes: establishing an application programming interface for the sensitive word review mechanism according to business needs, including a customized sensitive word list application programming interface and a customized violation handling policy application programming interface; adding an endpoint path and key of the application programming interface to call the sensitive word review mechanism when performing sensitive word review.

[0011] In one implementation, the application programming interface for establishing a sensitive word review mechanism according to business needs further includes: a customized third-party sensitive word detection service interface.

[0012] In one embodiment, the sensitive word detection on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list includes: calling the third-party sensitive word detection service interface to perform sensitive word detection on the input content and / or pre-output content of the generative artificial intelligence platform.

[0013] In one embodiment, the sensitive word review method further includes: responding to a selection button set based on the application programming interface of the sensitive word review mechanism, wherein the selection button is used to select to enable or disable the customized sensitive word list application programming interface, the customized violation handling policy application programming interface and the customized third-party sensitive word detection service interface.

[0014] In one embodiment, when the output of the generative artificial intelligence platform is a streaming output, the sensitive word review method further includes:

[0015] Counting the character length of the pre-output content;

[0016] When the character length of the pre-output content reaches a preset segment character length, the pre-output content satisfying the segment character length is determined as a review field;

[0017] The review field is reviewed for sensitive words.

[0018] In one embodiment, the violation handling strategy includes at least one of the following: controlling the generative artificial intelligence platform to output preset reply content, or preventing the input content or the pre-output content from being published, or replacing detected sensitive words.

[0019] In one embodiment, performing sensitive word review on input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list includes:

[0020] In response to the operation of the review switch for input content and / or the review switch for output content, the input content and / or pre-output content of the generative artificial intelligence platform is reviewed for sensitive words based on the sensitive word list.

[0021] In a second aspect, an embodiment of the present application provides a computing device, comprising: a processor and a memory storing a computer program, and when the processor runs the computer program, the above-mentioned sensitive word review method is implemented.

[0022] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the above-mentioned sensitive word review method is implemented.

[0023] The sensitive word review method, computing device and computer-readable storage medium provided in the embodiments of the present application perform sensitive word review on the input content and pre-output content of the generative artificial intelligence platform based on the sensitive word list in the preset sensitive word review mechanism. When it is detected that the input content or pre-output content contains a sensitive word in the sensitive word list, the violation handling strategy in the sensitive word review mechanism is executed. By performing sensitive word review on the input content and pre-output content of the generative artificial intelligence platform through the sensitive word review mechanism preset according to enterprise needs, the flexibility and scalability of sensitive word review can be enhanced, the risk of enterprise data leakage in the question and answer process can be reduced, the controllability of business question and answer can be improved, and the user experience can be enhanced. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A flowchart of a sensitive word review method provided in an embodiment of the present invention.

[0025] Figure 2 A flowchart of a sensitive word review method during streaming output provided by an embodiment of the present invention.

[0026] Figure 3 A schematic diagram of the structure of a computing device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0028] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element. In addition, components, features, and elements with the same name in different embodiments of the present application may have the same meaning or different meanings, and their specific meanings need to be determined by their explanation in the specific embodiment or further combined with the context of the specific embodiment.

[0029] It should be understood that, although the terms first, second, third, etc. may be used to describe various information in this article, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this article, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein can be interpreted as "at the time of..." or "when..." or "in response to determination". Furthermore, as used in this article, the singular forms "one", "one" and "the" are intended to also include plural forms, unless there is an opposite indication in the context. It should be further understood that the terms "comprising", "including" indicate that there are described features, steps, operations, elements, components, projects, kinds, and / or groups, but do not exclude the existence, occurrence or addition of one or more other features, steps, operations, elements, components, projects, kinds, and / or groups. The terms "or" and "and / or" used herein are interpreted as inclusive, or mean any one or any combination. Thus, “A, B, or C” or “A, B and / or C” means “any of the following: A; B; C; A and B; A and C; B and C; A, B, and C.” An exception to this definition will occur only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0030] It should be understood that, although the various steps in the flowchart in the embodiment of the present application are displayed in sequence according to the indication of the arrows, these steps are not necessarily performed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and it can be performed in other orders. Moreover, at least a portion of the steps in the figure may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and their execution order is not necessarily performed in sequence, but can be performed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0031] It should be noted that, in this article, step codes such as S101, S102, etc. are used for the purpose of expressing the corresponding content more clearly and concisely, and do not constitute a substantial limitation on the sequence. When implementing the step, those skilled in the art may execute S102 first and then S101, etc., but these should all be within the scope of protection of this application.

[0032] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0033] In the subsequent description, the suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application, and have no specific meanings. Therefore, "module", "component" or "unit" can be used in a mixed manner.

[0034] The following is an explanation of the terms that may be involved in this application, as follows:

[0035] Sensitive word review: refers to the process of screening and evaluating generated content to ensure that it does not contain sensitive information that may cause controversy, violate laws and regulations, damage social morality, or infringe on personal privacy.

[0036] See also Figure 1 , is a sensitive word review method provided in an embodiment of the present application. The sensitive word review method can be executed by a sensitive word review module provided in an embodiment of the present application. The sensitive word review module can be implemented in software and / or hardware. In this embodiment, the execution subject of the sensitive word review method is a computing device such as a computer or a server. The sensitive word review method provided in this embodiment includes:

[0037] Step S101: Obtain a preset sensitive word review mechanism, wherein the sensitive word review mechanism includes a sensitive word list and a violation handling strategy.

[0038] In one embodiment, obtaining a preset sensitive word review mechanism specifically includes: establishing an application programming interface (API) for the sensitive word review mechanism according to business needs, including a customized sensitive word list application programming interface and a customized violation handling policy application programming interface; adding an endpoint path and key of the application programming interface to be used for calling the sensitive word review mechanism when performing sensitive word review.

[0039] Specifically, users can collect sensitive words that they think need to be blocked according to their business needs. These can be words that are closely related to corporate confidential information, or words that violate relevant laws and regulations or may cause social ethical disputes. Based on the collected sensitive words, users can create a sensitive word list API that can add sensitive words, delete sensitive words, and retrieve sensitive words, so that they can call a customized sensitive word list when conducting sensitive word review. Users can also customize the handling strategy for illegal content according to business needs, including controlling the generative artificial intelligence platform to output preset reply content, or blocking the input content or pre-output content from being published, or replacing detected sensitive words, in the form of a violation handling strategy API, to implement the call of a customized violation handling strategy during sensitive word review.

[0040] After the user establishes a customized sensitive word list API or violation handling strategy API, the user also needs to integrate the customized API into the AIGC platform. When the AIGC platform publishes input content or pre-output content, the sensitive word review module is triggered to call the customized API for sensitive word review. In a specific embodiment, the user can directly add a customized API by editing the information of the extended API in the API extension interface, which can specifically include setting a name identifier for the extended API, describing the functions of the extended API so that users can understand the functions that the API can achieve, setting the endpoint path of the API so that the method in the API can be called according to the endpoint path during the sensitive word review, entering the API authentication key, and opening the API usage rights for the caller. After the customized API is integrated into the AIGC platform, the added API can be displayed in the API extension list directory.

[0041] Step S102: Perform sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list.

[0042] Specifically, the AIGC platform can use artificial intelligence technology to automatically generate answers or assist users in finding information, helping users deal with various types of problems, such as factual inquiries, request for advice, data analysis, etc. Users can submit the text content they want to enter to the AIGC platform through a chat interface, form submission, or other means. The AIGC platform can understand the user's intention through natural language processing (NLP) technology and return the answer content that matches the input question to the user through a large language model. Before the AIGC platform processes the user's input content, and after the AIGC platform generates the answer content but before it is displayed to the user, the sensitive word review module is required to review the input or pre-output content for sensitive words to prevent the AIGC platform and users from being affected by inappropriate content, ensure the compliance of the content, and maintain a healthy network environment.

[0043] When conducting sensitive word review, you can directly use the sensitive word review rules that come with the AIGC platform to detect sensitive words, or you can call a third-party sensitive word detection service interface to perform sensitive word detection on the input content and pre-output content of the generative artificial intelligence platform. Among them, the third-party sensitive word detection service interface can be a sensitive word detection API customized by enterprise developers based on the internal review mechanism of the enterprise. In a specific embodiment, enterprise developers can use open source tools, such as the sensitive-word tool, to implement a customized sensitive word filtering algorithm by determining the finite automaton (DFA) algorithm and combining a customized sensitive word tree structure. It only takes one scan of the text to be detected to detect all sensitive words, which reduces the number of retrieval times and improves efficiency. In some other embodiments, the third-party sensitive word detection service interface can also be other general external review service interfaces customized by enterprise developers, such as the OpenAI Moderation API.

[0044] In one embodiment, the application programming interface of the sensitive word review mechanism is provided with a selection button, and in response to the user's operation of the selection button, the customized sensitive word list application programming interface, the customized violation handling strategy application programming interface, and the customized third-party sensitive word detection service interface are enabled or disabled. Specifically, when the enterprise manager wants to use the user of the AIGC platform of the enterprise, in the question-and-answer scenario based on the large language model, to use the customized sensitive word list, violation handling strategy, or customized third-party sensitive word detection service interface of the enterprise to conduct sensitive word review, the selection button of the corresponding API can be set to the enabled state in the background management interface of the AIGC platform, and the corresponding API interface can be called during the sensitive word review. If the enterprise manager believes that the default sensitive word review mechanism of the AIGC platform can also meet the current needs of the enterprise for data security and content compliance, the selection button of the API of these customized sensitive word review mechanisms can also be set to the disabled state, and the sensitive words can be reviewed using the default sensitive word review mechanism of the AIGC platform.

[0045] In one embodiment, performing sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list also includes: in response to the operation of the review switch for the input content and / or the review switch for the output content, performing sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list.

[0046] Specifically, the sensitive word review module in this implementation is also provided with a review switch for input content and a review switch for output content. Enterprise managers can control the AIGC platform to enable or disable the sensitive word review function for the user's input content or the platform's pre-output content by adjusting the switch state of the review switch for input content or output content, so that users can flexibly configure the sensitive word review mechanism and achieve a high degree of customization of sensitive word review. After the review switch for input content and / or output content is turned on, the AIGC platform can perform sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on a customized sensitive word list.

[0047] Step S103: When it is detected that the input content and / or the pre-output content contains a sensitive word in the sensitive word list, a violation handling strategy is executed.

[0048] In one embodiment, when the input content review switch is turned on, before the AIGC platform processes the user's input content, if the sensitive word review module detects that the user's input content contains sensitive words, the AIGC platform can process the input content according to a preset violation handling strategy, such as outputting a preset reply content to the user, prompting the user that the content input does not comply with the platform regulations. When the output content review switch is turned on, if the pre-output content generated by the AIGC platform contains sensitive words, the AIGC platform can process the pre-output content according to a preset violation handling strategy, such as preventing the pre-output content from being published to the user, or replacing sensitive words before publishing.

[0049] The sensitive word review method provided in the above embodiment performs sensitive word review on the input content and pre-output content of the generative artificial intelligence platform based on the customized sensitive word list in the preset sensitive word review mechanism. When it is detected that the input content or pre-output content contains a sensitive word in the sensitive word list, the customized violation handling strategy in the sensitive word review mechanism is executed, allowing the enterprise's managers or developers to design and add sensitive word review mechanisms according to internal enterprise needs, such as customized sensitive word lists and customized violation handling strategies, thereby enhancing the flexibility and scalability of sensitive word review, reducing the risk of enterprise data leakage during the question and answer process, improving the controllability of business questions and answers, and enhancing the user experience.

[0050] See also Figure 2 In one embodiment, when the output of the generative artificial intelligence platform is a streaming output, the sensitive word review method in the embodiment of the present application further includes:

[0051] Step S201: counting the character length of the pre-output content;

[0052] Step S202: when the character length of the pre-output content reaches a preset segment character length, the pre-output content that meets the segment character length is determined as a review field;

[0053] Step S203: Conduct sensitive word review on the review field.

[0054] Specifically, streaming output is a data processing technology that allows data to be transmitted gradually while it is generated, without waiting for the entire process to be completed. When the AIGC platform is streaming output, the character length of the pre-output content generated by the platform can be counted in real time. When the character length of the generated pre-output content reaches the preset segmented character length, such as 100 characters, the 100-character string is used as the review field and passed to the sensitive word review module for sensitive word review. By segmenting the content output by the platform for sensitive word review, the problem of untimely review when the output content is long can be avoided.

[0055] In one embodiment, the sensitive word review method in the embodiment of the present application also includes: creating a review log to record the process and results of the sensitive word review. Specifically, creating a review log includes determining the type of information that needs to be recorded, such as a timestamp for recording the time when the review operation occurred, a user identifier for recording the identity of the user or system that initiated the review, and the ID of the reviewed content, original content, sensitive words, review results, processing actions, etc. By recording the process and results of the sensitive word review in a log, it is convenient for subsequent review and evaluation of the review mechanism and further optimize the review strategy.

[0056] Based on the same inventive concept as the above embodiments, the above embodiments are described in detail below through a specific application scenario.

[0057] In the first application scenario of the present application, enterprise developers can add and enable custom APIs through the API extension interface, such as a custom sensitive word list API or a custom violation handling strategy API. After adding the sensitive word list API, enterprise developers can also add new sensitive words directly to the sensitive word list through the keyword page in the sensitive word review module according to their needs. For example, if "death" needs to be filtered as a sensitive word, "death" can be added to the sensitive word list directly through the Add button on the keyword page, or the original sensitive words in the sensitive word list can be deleted through the Delete button.

[0058] In an application scenario of the present application, enterprise developers can also directly turn on or off the sensitive word review function through the review switch of the input content and the review switch of the output content. After the review switch of the input content is turned on, the enterprise developer can also customize the reply content output by the platform to the user when the input content violates the rules in the reply content editing area. For example, when the user inputs "the definition of death" on the AIGC platform, the AIGC platform will trigger the sensitive word review and return the preset reply content "Your content violates our usage policy.", prompting the user's input does not comply with the platform's regulations. After the review switch of the output content is turned on, when the pre-output content of the AIGC platform contains sensitive words, such as when the pre-output content of the AIGC platform is "I will kill you.", the word "kill" is a preset sensitive word, and the AIGC platform will replace "kill" before outputting it. For example, the content output to the user by the AIGC platform will be replaced with "I will***you."

[0059] In summary, the sensitive word review method provided in the embodiment of the present application can add an application programming interface of a customized sensitive word review mechanism according to the internal review mechanism of the enterprise. For example, based on the current needs of the enterprise for data security and content compliance, the sensitive words that need to be reviewed and the violation handling strategy for sensitive words are defined, and integrated into the AGGC platform in the form of an API, so that the customized sensitive word list and violation handling strategy can be called during the sensitive word review; and through the API selection button, input content and output content review switch, the enterprise can flexibly configure the sensitive word review mechanism according to internal needs, realize high customization and privacy protection of the sensitive word review process, enhance the flexibility and scalability of the sensitive word review, and when streaming output, the content output by the platform is segmented for sensitive word review, so as to avoid the problem of untimely review when the output content is long, reduce the user's waiting time, and improve the user experience.

[0060] Based on the same inventive concept as the above-mentioned embodiment, an embodiment of the present invention provides a sensitive word review module, which includes: a sensitive word review extension unit, a sensitive word review execution unit and a violation handling unit. Among them, the sensitive word review extension unit is used to obtain a preset sensitive word review mechanism, which includes a sensitive word list and a violation handling strategy; the sensitive word review execution unit is used to perform sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list; the violation handling unit is used to execute the violation handling strategy when it is detected that the input content and / or pre-output content contains a sensitive word in the sensitive word list.

[0061] It should be noted that the detailed functional implementation of each of the above units can refer to the introduction in the above method embodiment, and will not be repeated here. The sensitive word review method described above is realized through the cooperation between the above sensitive word review module and the various units set inside it, and has exactly the same beneficial effects.

[0062] Based on the same inventive concept as the above embodiments, an embodiment of the present invention provides a computing device, such as Figure 3 As shown, the device includes: a processor 310 and a memory 311 storing a computer program; wherein, Figure 3 The processor 310 shown in the figure is not used to indicate that the number of the processor 310 is one, but is only used to indicate the positional relationship of the processor 310 relative to other devices. In actual applications, the number of the processor 310 may be one or more; similarly, Figure 3 The memory 311 shown in the figure has the same meaning, that is, it is only used to refer to the position relationship of the memory 311 relative to other devices. In practical applications, the number of memories 311 can be one or more. When the processor 310 runs the computer program, the sensitive word review method applied to the above computing device is implemented.

[0063] The device may also include: at least one network interface 312. The various components in the device are coupled together via a bus system 313. It is understood that the bus system 313 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 313 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 3 Various buses are labeled as bus system 313.

[0064] The memory 311 may be a volatile memory or a nonvolatile memory, or may include both volatile and nonvolatile memories. The nonvolatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disk, or a compact disc read-only memory (CD-ROM); the magnetic surface memory may be a disk memory or a tape memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM, SyncLink Dynamic Random Access Memory), and direct RAM bus random access memory (DRRAM, Direct Rambus Random Access Memory).The memory 311 described in the embodiments of the present invention is intended to include but is not limited to these and any other suitable types of memories.

[0065] The memory 311 in the embodiment of the present invention is used to store various types of data to support the operation of the device. Examples of these data include: any computer program used to operate on the device, such as an operating system and an application; contact data; phone book data; messages; pictures; videos, etc. Among them, the operating system includes various system programs, such as a framework layer, a core library layer, a driver layer, etc., which are used to implement various basic services and process hardware-based tasks. The application program may include various applications, such as a media player (Media Player), a browser (Browser), etc., which are used to implement various application services. Here, the program that implements the method of the embodiment of the present invention may be included in the application program.

[0066] Based on the same inventive concept as the above-mentioned embodiment, this embodiment further provides a computer-readable storage medium, in which a computer program is stored. The computer-readable storage medium may be a ferromagnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); it may also be various devices including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, etc. When the computer program stored in the computer-readable storage medium is executed by the processor, the above-mentioned sensitive word review method is implemented. For the specific steps implemented when the computer program is executed by the processor, please refer to Figure 1 The description of the illustrated embodiment will not be repeated here.

[0067] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] In this document, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion of elements other than those listed and may also include additional elements not expressly listed.

[0069] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A sensitive word review method, characterized in that: include: Obtaining a preset sensitive word review mechanism, wherein the sensitive word review mechanism includes a sensitive word list and a violation handling strategy; Performing sensitive word review on the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list; When it is detected that the input content and / or pre-output content contains a sensitive word in the sensitive word list, the violation handling strategy is executed.

2. The sensitive word review method according to claim 1, characterized in that: The method of obtaining a preset sensitive word review mechanism includes: According to business needs, establish the API of the sensitive word review mechanism, including the customized sensitive word list API and the customized violation handling strategy API; Add the endpoint path and key of the application programming interface to call the sensitive word review mechanism when performing sensitive word review.

3. The sensitive word review method according to claim 2, characterized in that: The application programming interface for establishing a sensitive word review mechanism based on business needs also includes: a customized third-party sensitive word detection service interface.

4. The sensitive word review method according to claim 3, characterized in that: The sensitive word review of the input content and / or pre-output content of the generative artificial intelligence platform based on the sensitive word list includes: calling the third-party sensitive word detection service interface to perform sensitive word detection on the input content and / or pre-output content of the generative artificial intelligence platform.

5. The sensitive word review method according to claim 3, characterized in that: The method also includes: responding to a selection button set based on the application programming interface of the sensitive word review mechanism, wherein the selection button is used to select to enable or disable the customized sensitive word list application programming interface, the customized violation handling policy application programming interface and the customized third-party sensitive word detection service interface.

6. The sensitive word review method according to claim 1, characterized in that: When the output of the generative artificial intelligence platform is a streaming output, the method further includes: Counting the character length of the pre-output content; When the character length of the pre-output content reaches a preset segment character length, the pre-output content satisfying the segment character length is determined as a review field; The review field is reviewed for sensitive words.

7. The sensitive word review method according to claim 1, characterized in that: The violation handling strategy includes at least one of the following: controlling the generative artificial intelligence platform to output preset reply content, or preventing the input content or the pre-output content from being published, or replacing detected sensitive words.

8. The sensitive word review method according to claim 1, characterized in that: Based on the sensitive word list, the input content and / or pre-output content of the generative artificial intelligence platform is reviewed for sensitive words, including: In response to the operation of the review switch for input content and / or the review switch for output content, the input content and / or pre-output content of the generative artificial intelligence platform is reviewed for sensitive words based on the sensitive word list.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, which implements the sensitive word review method described in any one of claims 1 to 8 when the processor runs the computer program.

10. A computer-readable storage medium, characterized in that: A computer program is stored, and when the computer program is executed by a processor, the sensitive word review method described in any one of claims 1 to 8 is implemented.

Images