Virtual human behavior safety management system and method based on AI monitoring and prediction
By adopting an AI-based monitoring and prediction system in the virtual world, the conversation content of virtual characters is analyzed in real time, and the problem of insufficient interaction security in the virtual world is solved, and a wider and more effective detection and prevention of malicious behavior is achieved.
Patent Information
- Application Number
- CN202411557496.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2025-03-11
AI Technical Summary
The prior art cannot effectively address the problem of insufficient security of interactions in the virtual world, especially in detecting and preventing malicious harassment, fraud and inappropriate language.
The AI-based monitoring and prediction system is adopted to convert the conversational voice of virtual characters into text information through voice recognition technology and voice-to-text technology. The conversational prediction information is generated in combination with the search and strengthening generation technology, and input it into the pre-trained security protection language model to predict whether the virtual characters have improper behavior characteristics in real time. If there is, users are prohibited from manipulating it immediately.
It improves the interactive security of the virtual world, can effectively detect and prevent malicious behavior, has a wider scope of application, and can deal with avoidance methods such as alternative texts or homophones.
Smart Images
Figure CN119598278A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a safety management system and method thereof, in particular to a virtual human behavior safety management system and method thereof based on AI monitoring and prediction. Background Art
[0002] In recent years, with the popularization and vigorous development of various real-world technologies, various virtual character applications have sprung up like mushrooms after rain. However, in the virtual world, how to improve the safety of interaction has always been one of the problems that manufacturers are eager to solve.
[0003] Generally speaking, traditional virtual characters are usually controlled by users, for example, they move and talk in the virtual world, etc. However, since users control virtual characters in the virtual world, there is a lack of constraints in reality, which often leads to malicious harassment, fraud, inappropriate language, etc. Therefore, there is a problem of insufficient interactive security in the virtual world.
[0004] In view of this, some manufacturers have proposed a technical means of judging and banning based on keywords. It detects whether there are pre-set keywords in the conversation content, such as: swear words. If there are keywords, the virtual character will be banned. However, this method can only target pre-set keywords. If there are no pre-set keywords, it cannot effectively detect and prevent. In other words, the scope of application of this method is very limited, and it cannot detect the situation of avoiding keywords by replacing words, such as: using English, numbers or homophones to insult. Therefore, the existing technology still cannot effectively solve the problem of insufficient interactive security in the virtual world.
[0005] In summary, it can be seen that the existing technology has long had the problem of insufficient interactive security in the virtual world, so it is necessary to propose improved technical means to solve this problem. Summary of the invention
[0006] The present invention discloses a virtual human behavior safety management system based on AI monitoring and prediction and a method thereof.
[0007] First, the present invention discloses a virtual human behavior safety management system based on AI monitoring prediction, and the system includes: a non-transitory computer-readable storage medium and a hardware processor. The non-transitory computer-readable storage medium is used to store a plurality of computer-readable instructions and a pre-trained safety protection language model; and the hardware processor is electrically connected to the non-transitory computer-readable storage medium to execute the plurality of computer-readable instructions, so that the hardware processor is configured to: when a user operates a virtual character to have a conversation, obtain the conversation of the virtual character to generate a conversation voice, and convert the conversation voice into a plurality of conversation information through human voice recognition technology and speech-to-text technology; retrieve a plurality of knowledge information related to the conversation information from an external knowledge base through a retrieval-augmented generation (RAG) retriever, and generate accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing (Natural Language Processing, NLP) technology by the retrieval-augmented generation generator; and input the conversation prediction information into the safety protection language model to predict whether the virtual character meets the characteristics of improper behavior, and if so, immediately prohibit the user from controlling the virtual character.
[0008] In addition, the present invention also discloses a virtual human behavior safety management method based on AI monitoring and prediction, which is executed by a hardware processor, and its steps include: the hardware processor loads a pre-trained safety protection language model; when the user operates the virtual character to have a conversation, the hardware processor obtains the virtual character's conversation to generate a conversation voice, and converts the conversation voice into multiple conversation information through human voice recognition technology and speech-to-text technology; the hardware processor retrieves multiple knowledge information related to the conversation information from an external knowledge base through the RAG retriever, and the RAG generator generates accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing technology; and the hardware processor inputs the conversation prediction information into the safety protection language model to predict whether the virtual character meets the characteristics of improper behavior. If so, the user is immediately prohibited from controlling the virtual character.
[0009] The system and method disclosed in the present invention are as described above. The difference from the prior art is that the present invention monitors the conversations between the virtual character controlled by the user and other virtual characters in real time, and converts the voice conversation content into text information by combining human voice recognition technology and speech-to-text technology, and then generates conversation prediction information through retrieval enhancement generation technology, and inputs it into a pre-trained safety protection language model to predict whether the virtual character meets the characteristics of inappropriate behavior. If so, the user's control over the virtual character is immediately restricted.
[0010] Through the above-mentioned technical means, the present invention can achieve the technical effect of improving the interactive security of the virtual world. BRIEF DESCRIPTION OF THE DRAWINGS
[0011] Figure 1 This is a system block diagram of the virtual human behavior safety management system based on AI monitoring and prediction of the present invention.
[0012] Figure 2 This is a method flow chart of the virtual human behavior safety management method based on AI monitoring and prediction of the present invention.
[0013] Figure 3 A schematic diagram of applying the present invention to prohibit inappropriate behavior of a virtual character.
[0014] Figure 4 A schematic diagram of adjusting the time for prohibiting manipulation for applying the present invention.
[0015] Description of reference numerals:
[0016] 100: Computer devices
[0017] 110: Non-transitory computer readable storage medium
[0018] 120: Hardware processor
[0019] 130: External Knowledge Base
[0020] 310, 320: Virtual Characters
[0021] 311, 312: Dialogue Block
[0022] 410: Mute block
[0023] 420: Mute time
[0024] Step 210: The hardware processor loads the pre-trained security protection language model
[0025] Step 220: When the user operates the virtual character to have a conversation, the hardware processor obtains the conversation of the virtual character to generate a conversation voice, and converts the conversation voice into multiple conversation messages through human voice recognition technology and speech-to-text technology.
[0026] Step 230: The hardware processor retrieves a plurality of knowledge information related to the conversation information from an external knowledge base through a retrieval-augmented generation (RAG) retriever, and the retrieval-augmented generation generator generates accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing (NLP) technology.
[0027] Step 240: The hardware processor inputs the conversation prediction information into the safety protection language model to predict whether the virtual character meets the inappropriate behavior characteristics. If so, the user is immediately prohibited from controlling the virtual character. DETAILED DESCRIPTION
[0028] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.
[0029] First, please refer to the Figure 1 ”, “ Figure 1” is a system block diagram of the virtual human behavior safety management system based on AI monitoring and prediction of the present invention. The system includes: a non-transitory computer-readable storage medium 110 and a hardware processor 120, which can be set in a computer device 100. Among them, the non-transitory computer-readable storage medium 110 is used to store a plurality of computer-readable instructions, and store a pre-trained safety protection language model. In actual implementation, the non-transitory computer-readable storage medium 110 may include: a hard disk, an optical disk, a flash memory or the like. In addition, the computer-readable instructions are executed by a computer device, and the computer-readable instructions may be assembly language instructions, instruction set architecture instructions, machine instructions, machine-related instructions, microinstructions, firmware instructions, or source code or object code (Object Code) written in any combination of one or more programming languages, and the programming language includes an object-oriented programming language, such as: Common Lisp, Python, C++, Objective-C, Smalltalk, Delphi, Java, Swift, C#, Perl, Ruby and PHP, as well as conventional procedural programming languages, such as C or similar programming languages. In addition, the security protection language model may include bidirectional encoder representations from transformers (BERT), generative pre-trained transformers (GPT), Transformer, GPT) and other AI models, and pre-use text, images, sounds and combinations thereof containing at least one of inappropriate language, fraudulent intent, abnormal behavior and potential security threats as training data. It should be particularly noted that the security protection language model can also allow the weights to be dynamically adjusted according to the user's conversation topic and interpersonal relationship to increase the accuracy of the prediction when predicting whether the virtual character meets the characteristics of inappropriate behavior. For example, assuming that the topic of the conversation is joking and the interpersonal relationship is close friends, then the security protection language model can adjust the predicted weights to loosely predict whether the virtual character meets the characteristics of inappropriate behavior. Conversely, if the interpersonal relationship is a stranger, the predicted weights are adjusted to strictly predict whether the virtual character meets the characteristics of inappropriate behavior. The way to adjust the weights can be achieved through fine-tuning, post-training optimization and similar technologies.
[0030] Next, in the part of the hardware processor 120, it is electrically connected to the non-transitory computer-readable storage medium 110 to execute the multiple computer-readable instructions, so that the hardware processor 120 is configured as follows: when the user operates the virtual character to have a conversation, the conversation of the virtual character is obtained to generate a conversation voice, and the conversation voice is converted into multiple conversation information through human voice recognition technology and speech-to-text technology; through the RAG retriever, multiple knowledge information related to the conversation information is retrieved from the external knowledge base 130, and the RAG generator generates accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing technology; and the conversation prediction information is input into the safety protection language model to predict whether the virtual character meets the characteristics of improper behavior. If so, the user's control of the virtual character is immediately prohibited. In actual implementation, the inappropriate behavior characteristics may include: inappropriate language, fraudulent intent, abnormal behavior, and conversations with potential security threats. When the security protection language model predicts that the inappropriate behavior characteristics are met, warning points can be accumulated, and the time for prohibiting the user from controlling the virtual character can be dynamically adjusted according to the warning points. Among them, the warning points are negatively correlated with the control time, that is, the more warning points are accumulated, the less time the virtual character can be controlled. In addition, the external knowledge base 130 includes: news, pictures, sounds, images and combinations thereof related to fraud, bullying, emotional blackmail and emotional control, and embeds corresponding vectors to allow the searcher to perform similarity search.
[0031] It should be particularly noted that, in actual implementation, the present invention may be partially or completely implemented based on hardware. For example, the hardware processor 120 in the system may be implemented by an integrated circuit chip, a system on chip (SoC), a complex programmable logic device (CPLD), or a field programmable gate array (FPGA). The non-transitory computer-readable storage medium 110 described in the present invention carries computer-readable instructions (or computer program instructions) for enabling the hardware processor 120 to implement various aspects of the present invention. The non-transitory computer-readable storage medium 110 may be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of the non-transitory computer-readable storage medium 110 include: a hard disk, a random access memory, a read-only memory, a flash memory, an optical disk, a floppy disk, and any suitable combination thereof. As used herein, the non-transitory computer-readable storage medium 110 is not to be construed as a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical signals through fiber-optic cables), or electrical signals transmitted through wires. In addition, the computer-readable instructions described herein may be downloaded from the non-transitory computer-readable storage medium 110 to each computing / processing device, or downloaded to an external computer device or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber-optic transmission, wireless transmission, routers, firewalls, switches, hubs, and / or gateways. The network card or network interface in each computing / processing device receives the computer-readable instructions from the network and forwards the computer-readable instructions for storage in the non-transitory computer-readable storage medium 110 in each computing / processing device.
[0032] Please refer to " Figure 2 ”, “ Figure 2” is a method flow chart of the virtual human behavior safety management method based on AI monitoring prediction of the present invention, which is executed by a hardware processor, and its steps include: the hardware processor 120 loads the pre-trained safety protection language model (step 210); when the user operates the virtual character to have a conversation, the hardware processor 120 obtains the conversation of the virtual character to generate a conversation voice, and converts the conversation voice into multiple conversation information through human voice recognition technology and speech-to-text technology (step 220); the hardware processor 120 retrieves multiple knowledge information related to the conversation information from the external knowledge base 130 through the RAG retriever, and the RAG generator generates an accurate and highly The hardware processor 120 inputs the conversation prediction information into the safety protection language model to predict whether the virtual character meets the inappropriate behavior characteristics. If so, the user is immediately prohibited from controlling the virtual character (step 240). Through the above steps, the conversation between the virtual character controlled by the user and other virtual characters can be monitored in real time, and the voice conversation content can be converted into text information by combining human voice recognition technology and speech-to-text technology. The conversation prediction information is then generated through the retrieval enhancement generation technology and input into the pre-trained safety protection language model to predict whether the virtual character meets the inappropriate behavior characteristics. If so, the user's control of the virtual character is immediately restricted.
[0033] The following matches " Figure 3 "and" Figure 4 ” is described as follows by way of example, such as “ Figure 3 " Figure 3 ” is a schematic diagram of applying the present invention to prohibit inappropriate behavior of a virtual character. In actual implementation, when the user operates the virtual character 310 to have a conversation, the hardware processor 120 obtains the conversation between the virtual character 310 and other virtual characters 320 to “ Figure 3" is taken as an example, and the conversation blocks (311, 312) are used to respectively indicate the conversation voices of the virtual character 310 and other virtual characters 320. The hardware processor 120 obtains the two voices to generate the conversation voice, and converts the conversation voice into multiple conversation information through voice recognition technology and speech-to-text technology, namely: "A: Who are you?" and "B: Guess who I am?". Then, the hardware processor retrieves multiple knowledge information related to the conversation information from the external knowledge base 130 through the RAG retriever, and the RAG generator generates accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing technology, for example: "OO? By the way, I haven't contacted you for a long time, and my phone number has changed... Can you please buy points for me? ”, where “OO” represents any name or nickname. Next, the hardware processor 120 inputs the conversation prediction information into the safety protection language model to predict whether the virtual character 320 meets the inappropriate behavior characteristics. In this example, the conversation prediction information meets the inappropriate behavior characteristics, so the user is immediately prohibited from manipulating the virtual character 320, for example, prohibiting movement, prohibiting speech, etc. In this way, the possible inappropriate behavior can be stopped immediately before the virtual character 320 actually performs inappropriate behavior. In actual implementation, in order to avoid misjudgment, the safety protection language model predicts the virtual character (310, 320) meets the characteristics of improper behavior, the weight can be dynamically adjusted according to the conversation topic and interpersonal relationship between the two to increase the accuracy of the prediction. For example, if the conversation topic between the two is playful and the interpersonal relationship is friends, the weight can be dynamically adjusted to relax the judgment standard. On the contrary, if the two are strangers with no intersection or just acquaintances, the weight can be dynamically adjusted to set the judgment standard to be strict. In fact, the method of judging whether they are strangers or even acquaintances can be judged by the contact history of virtual characters 310 and virtual characters 320, such as whether they have talked before, the length of the conversation, etc.
[0034] like" Figure 4 " Figure 4 ” is a schematic diagram of the application of the present invention to adjust the time for prohibiting manipulation. In actual implementation, if the security protection language model predicts that there may be inappropriate language, fraudulent intent, abnormal behavior and conversations with potential security threats (i.e., meeting the inappropriate behavior characteristics), the hardware processor 120 can also accumulate warning points. For example, each time the inappropriate behavior characteristics occur, the warning points are added with a value of 1 (the initial value of the warning points is zero), and the time for prohibiting the user from manipulating the virtual character 320 is dynamically adjusted according to the warning points, wherein the warning points are negatively correlated with the manipulation time. For example, assuming that the inappropriate behavior characteristics are met, it can be as follows “ Figure 4” As shown, a mute block 410 is displayed, and a mute time 420 is displayed above it, such as 3 minutes. If after 3 minutes, it is predicted that the inappropriate behavior characteristics are met again, the warning point will be 2, and the original mute time 420 will be extended from the original 3 minutes to a longer time, such as 4 minutes. It should be particularly noted that, in addition to prohibiting speech, the present invention does not limit the method of prohibiting the user from controlling the virtual character 320, that is, any method of prohibiting the control of the virtual character 320 does not deviate from the application scope of the present invention. For example, the method of prohibiting the user from controlling the virtual character 320 may also include prohibiting movement, prohibiting login, and prohibiting logout.
[0035] In summary, it can be seen that the difference between the present invention and the prior art lies in that the conversations between the virtual character controlled by the user and other virtual characters are monitored in real time, and the voice conversation content is converted into text information by combining voice recognition technology and speech-to-text technology, and then the conversation prediction information is generated by retrieval enhancement generation technology, and input into the pre-trained security protection language model to predict whether the virtual character meets the characteristics of inappropriate behavior. If so, the user's control over the virtual character is immediately restricted. This technical means can solve the problems existing in the prior art, thereby achieving the technical effect of improving the interactive security of the virtual world.
[0036] Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of patent protection of the present invention shall be subject to the scope defined by the claims attached to this specification.
Claims
1. A virtual human behavior safety management system based on AI monitoring and prediction, the system comprising: A non-transitory computer-readable storage medium for storing a plurality of computer-readable instructions and a pre-trained security protection language model; and A hardware processor is electrically connected to the non-transitory computer-readable storage medium and is used to execute the plurality of computer-readable instructions, so that the hardware processor is configured to: When the user operates the virtual character to have a conversation, the conversation of the virtual character is acquired to generate a conversation voice, and the conversation voice is converted into a plurality of conversation information through human voice recognition technology and speech-to-text technology; Retrieving a plurality of pieces of knowledge information related to the conversation information from an external knowledge base through a searcher generated by search enhancement, and generating accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing technology through a generator generated by search enhancement; as well as The conversation prediction information is input into the safety protection language model to predict whether the virtual character meets the inappropriate behavior characteristics. If so, the user is immediately prohibited from controlling the virtual character.
2. The virtual human behavior safety management system based on AI monitoring and prediction as claimed in claim 1, wherein: The inappropriate behavior characteristics include inappropriate language, fraudulent intent, abnormal behavior and conversations with potential security threats. When the security protection language model predicts that the inappropriate behavior characteristics are met, warning points are accumulated, and the time for prohibiting the user from controlling the virtual character is dynamically adjusted based on the warning points, wherein the warning points are negatively correlated with the control time.
3. The virtual human behavior safety management system based on AI monitoring and prediction as claimed in claim 1, wherein: The security protection language model includes a transformer-based bidirectional encoder representation technology, a generative pre-trained converter, and is pre-trained with text, images, sounds and combinations thereof containing at least one of inappropriate language, fraudulent intent, abnormal behavior and potential security threats.
4. The virtual human behavior safety management system based on AI monitoring and prediction as claimed in claim 1, wherein: The external knowledge base contains news, pictures, sounds, images and combinations thereof related to fraud, bullying, emotional blackmail and emotion control, and is embedded with corresponding vectors to allow the retriever to perform similarity retrieval.
5. The virtual human behavior safety management system based on AI monitoring and prediction as claimed in claim 1, wherein: When predicting whether the virtual character meets the inappropriate behavior characteristics, the safety protection language model allows the weight to be dynamically adjusted according to the user's conversation topic and interpersonal relationship to increase the accuracy of the prediction.
6. A virtual human behavior safety management method based on AI monitoring and prediction, executed by a hardware processor, comprising the following steps: The hardware processor loads a pre-trained security protection language model; When the user operates the virtual character to have a conversation, the hardware processor acquires the conversation of the virtual character to generate a conversation voice, and converts the conversation voice into a plurality of conversation messages through voice recognition technology and speech-to-text technology; The hardware processor retrieves a plurality of knowledge information related to the conversation information from an external knowledge base through a retrieval-augmented generation (RAG) retriever, and the RAG generator generates accurate and highly relevant conversation prediction information based on the knowledge information and natural language processing technology; and The hardware processor inputs the conversation prediction information into the safety protection language model to predict whether the virtual character meets the inappropriate behavior characteristics. If so, the user is immediately prohibited from controlling the virtual character.
7. The virtual human behavior safety management method based on AI monitoring and prediction as claimed in claim 6, wherein: The inappropriate behavior characteristics include inappropriate language, fraudulent intent, abnormal behavior and conversations with potential security threats. When the security protection language model predicts that the inappropriate behavior characteristics are met, warning points are accumulated, and the time for prohibiting the user from controlling the virtual character is dynamically adjusted based on the warning points, wherein the warning points are negatively correlated with the control time.
8. The method for virtual human behavior safety management based on AI monitoring and prediction as claimed in claim 6, wherein: The security protection language model includes a transformer-based bidirectional encoder representation technology, a generative pre-trained converter, and is pre-trained with text, images, sounds and combinations thereof containing at least one of inappropriate language, fraudulent intent, abnormal behavior and potential security threats.
9. The method for virtual human behavior safety management based on AI monitoring and prediction as claimed in claim 6, wherein: The external knowledge base contains news, pictures, sounds, images and combinations thereof related to fraud, bullying, emotional blackmail and emotion control, and is embedded with corresponding vectors to allow the retriever to perform similarity retrieval.
10. The method for virtual human behavior safety management based on AI monitoring and prediction as claimed in claim 6, wherein: When predicting whether the virtual character meets the inappropriate behavior characteristics, the safety protection language model allows the weight to be dynamically adjusted according to the user's conversation topic and interpersonal relationship to increase the accuracy of the prediction.