Virtual human evolution system and method based on AI learning observation

Abstract

According to the virtual human evolution system and method based on AI learning observation, when a virtual human triggers a conversation event, voice is continuously recorded to generate complete conversation voice, and the complete voice is converted into complete conversation information through the voice-to-text technology, so that the conversation efficiency is improved. Inputting the complete conversation information as new training data into a pre-trained language model so as to re-train the language model, inputting the complete conversation information into a personality analysis model so as to obtain real person personality data, and dynamically adjusting virtual personality data of a virtual character according to the real person personality data so as to obtain the real person personality data. Therefore, the technical effect of improving the interaction conformity of the virtual character is achieved.

Description

Technical Field

[0001] The present invention relates to an evolution system and method thereof, in particular to a virtual human evolution system and method thereof based on AI learning observation. Background Art

[0002] In recent years, with the popularization and vigorous development of virtual technology, various related applications have sprung up, among which virtual characters are the most common.

[0003] Generally speaking, traditional virtual characters usually interact with users in a fixed behavior mode. For example, the conversation information of the virtual character is pre-set. When the user interacts with the virtual character, the virtual character talks to the user according to the pre-set conversation information, such as the non-player character (NPC) in the game. However, although this method can interact with the user, the interaction method is often rigid and cannot provide the user with a good interactive experience. Therefore, traditional virtual characters have the problem of insufficient interactivity.

[0004] In view of this, some manufacturers have proposed combining natural language processing technology to understand and generate natural language so that the lines of virtual characters are no longer rigid. However, although this method can make the lines of virtual characters no longer monotonous, it may not necessarily resonate with the user. For example, if the user is introverted, but the virtual character's speech is very extroverted, it is easy to make the user feel uneasy or uncomfortable. Conversely, if the user is extroverted, but the virtual character's speech and behavior are very introverted, it will also make the user feel uncomfortable. Therefore, this method is likely to reduce the willingness to interact or communicate due to the personality differences between the user and the virtual character. In other words, this method has the problem of poor interaction with the virtual character.

[0005] In summary, it can be seen that the existing technology has long had the problem of poor interaction compatibility with virtual characters, so it is necessary to propose improved technical means to solve this problem. Summary of the invention

[0006] The present invention discloses a virtual human evolution system and method based on AI learning observation.

[0007] First, the present invention discloses a virtual human evolution system based on AI learning observation, which includes: a non-transitory computer-readable storage medium and a hardware processor. The non-transitory computer-readable storage medium is used to store multiple computer-readable instructions, a personality analysis model, and virtual personality data and a language model corresponding to the virtual character, wherein the personality analysis model and the language model are both pre-trained machine learning models. The hardware processor is electrically connected to the non-transitory computer-readable storage medium to execute the multiple computer-readable instructions, so that the hardware processor is configured to: load the personality analysis model at the beginning, and load the virtual personality data and the language model corresponding to the virtual character; when the virtual character triggers a conversation event, continuously record the voice of the virtual character to generate a complete conversation voice, and convert the complete conversation voice into a complete conversation information through speech-to-text technology; use the complete conversation information as new training data, and input this new training data into the pre-trained language model to retrain the language model; and input the complete conversation information into the personality analysis model to output real person personality data, and dynamically adjust the virtual personality data of the virtual character according to this real person personality data.

[0008] In addition, the present invention also discloses a virtual human evolution method based on AI learning observation, which is executed by a hardware processor, and the steps include: at the initial stage, the hardware processor loads a personality analysis model, and loads virtual personality data and a language model corresponding to the virtual character, wherein the personality analysis model and the language model are both pre-trained machine learning models; when the hardware processor detects that the virtual character triggers a conversation event, the virtual character's voice is continuously recorded to generate a complete conversation voice, and the complete conversation voice is converted into complete conversation information through speech-to-text technology; the hardware processor uses the complete conversation information as new training data, and inputs the new training data into the pre-trained language model to retrain the language model; and the hardware processor inputs the complete conversation information into the personality analysis model to output real person personality data, and dynamically adjusts the virtual personality data of the virtual character according to the real person personality data.

[0009] The system and method disclosed in the present invention are as described above. The difference from the prior art is that the present invention continuously records voice to generate a complete conversation voice when a virtual character triggers a conversation event, and converts the complete voice into complete conversation information through voice-to-text technology, and then inputs the complete conversation information as new training data into a pre-trained language model to retrain the language model, and inputs the complete conversation information into a personality analysis model to obtain real person personality data, and dynamically adjusts the virtual personality data of the virtual character according to the real person personality data.

[0010] Through the above-mentioned technical means, the present invention can achieve the technical effect of improving the interactive compatibility of virtual characters. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 This is a system block diagram of the virtual human evolution system based on AI learning observation of the present invention.

[0012] Figure 2 This is a method flow chart of the virtual human evolution method based on AI learning observation of the present invention.

[0013] Figure 3 A schematic diagram of adjusting the interactive personality of a virtual character by applying the present invention.

[0014] Figure 4 A schematic diagram of adjusting virtual personality data using the present invention.

[0015] Description of reference numerals:

[0016] 100: Computer host

[0017] 110: Non-transitory computer readable storage medium

[0018] 120: Hardware processor

[0019] 310: User

[0020] 311, 312: Dialogue Box

[0021] 320: Virtual Characters

[0022] 410: Original virtual personality data

[0023] 420: Adjusted virtual personality data

[0024] Step 210: Initially, the hardware processor loads the personality analysis model, as well as the virtual personality data and language model corresponding to the virtual character, wherein the personality analysis model and the language model are both pre-trained machine learning models.

[0025] Step 220: When the hardware processor detects that the virtual character triggers a conversation event, the virtual character's voice is continuously recorded to generate a complete conversation voice, and the complete conversation voice is converted into a complete conversation message through a speech-to-text technology.

[0026] Step 230: The hardware processor uses the complete conversation information as new training data, and inputs the new training data into the pre-trained language model to retrain the language model.

[0027] Step 240: The hardware processor inputs the complete conversation information into the personality analysis model to output real person personality data, and dynamically adjusts the virtual personality data of the virtual character according to the real person personality data. DETAILED DESCRIPTION

[0028] The following will describe the implementation methods of the present invention in detail with reference to the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0029] First, please refer to the Figure 1 ”, “ Figure 1 ” is a system block diagram of a virtual human evolution system based on AI learning observation of the present invention, and the system includes: a non-transitory computer-readable storage medium 110 and a hardware processor 120, and can be set in a computer host 100. Among them, the non-transitory computer-readable storage medium 110 is used to store a plurality of computer-readable instructions, a personality analysis model, and virtual personality data and a language model corresponding to the virtual character, wherein the personality analysis model and the language model are both pre-trained machine learning models. In actual implementation, the non-transitory computer-readable storage medium 110 may include: a hard disk, an optical disk, a flash memory or the like. In addition, the computer-readable instructions are executed by the hardware processor 120, and the computer-readable instructions may be assembly language instructions, instruction set architecture instructions, machine instructions, machine-related instructions, microinstructions, firmware instructions, or source code or object code (Object Code) written in any combination of one or more programming languages, and the programming language includes an object-oriented programming language, such as: Common Lisp, Python, C++, Objective-C, Smalltalk, Delphi, Java, Swift, C#, Perl, Ruby and PHP, as well as conventional procedural programming languages, such as C or similar programming languages. It should be particularly noted that the virtual personality data can be stored in a structured file format (such as JSON, XML, etc.) and used as the basis for the personality expression of the virtual character.

[0030] The hardware processor 120 is electrically connected to the non-transitory computer-readable storage medium 110 to execute the plurality of computer-readable instructions, so that the hardware processor 120 is configured to: initially, load the personality analysis model, and load the virtual personality data and language model corresponding to the virtual character; when a conversation event is detected to be triggered by the virtual character, continuously record the voice of the virtual character to generate a complete conversation voice, and convert the complete conversation voice into a complete conversation message through a speech-to-text technology; use the complete conversation message as new training data, and input the new training data into a pre-trained language model to retrain the language model; and input the complete conversation information into the personality analysis model to output real person personality data, and dynamically adjust the virtual personality data of the virtual character according to the real person personality data. In actual implementation, the retraining of the language model can be performed by fine-tuning technology, and after pre-processing the new training data (for example, noise removal, standardization, regularization, etc.), multiple fine-tuning hyperparameters are set for retraining, and the hyperparameters at least include a learning rate and a batch size. In addition, the language model can be retrained through incremental learning technology to adjust the language model based on new training data and gradient descent algorithm, and memory augmented neural networks can be used to store historical learning content during training to prevent the language model from forgetting past learning content. In addition, the personality analysis model includes support vector machines, decision trees and deep learning models to extract features and classification, which are used to infer personality characteristics based on complete conversation information to output real person personality data, and allow feedback (such as: through questions, questionnaires, etc.) to adjust the weight of the personality analysis model as a basis for optimization. It should be noted that both real person personality data and virtual personality data may include multiple characteristic parameters consisting of personality traits (such as extroversion, introversion, etc.), conversation styles (such as humor, indifference, etc.), decision-making modes (such as cooperation, adventure, etc.) and interaction methods (such as initiative, listening, etc.), each characteristic parameter corresponds to a characteristic weight, and when the virtual personality data of the virtual character is dynamically adjusted according to the real person personality data, the adjustment is performed by selecting one of the combinations of linear weighted adjustment, adaptive adjustment, weighted average and direct replacement. Similarly, the real person personality data may also be stored in a structured file format.

[0031] It should be particularly noted that in actual implementation, the present invention can be partially or completely implemented based on hardware. For example, the hardware processor 120 in the system can be implemented by an integrated circuit chip, a system on chip (System on Chip, SoC), a complex programmable logic device (Complex Programmable Logic Device, CPLD), and a field programmable gate array (Field Programmable Gate Array, FPGA). The non-transitory computer-readable storage medium 110 described in the present invention carries computer-readable instructions (or computer program instructions) for enabling the hardware processor 120 to implement various aspects of the present invention. The non-transitory computer-readable storage medium 110 can be, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of the non-transitory computer-readable storage medium 110 include: a hard disk, a random access memory, a read-only memory, a flash memory, an optical disk, a floppy disk, and any suitable combination of the above. As used herein, the non-transitory computer-readable storage medium 110 is not to be construed as a transient signal per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., optical signals through fiber-optic cables), or electrical signals transmitted through wires. In addition, the computer-readable instructions described herein may be downloaded from the non-transitory computer-readable storage medium 110 to each computing / processing device, or downloaded to an external computer device or external storage device through a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, fiber-optic transmission, wireless transmission, routers, firewalls, switches, hubs, and / or gateways. The network card or network interface in each computing / processing device receives the computer-readable instructions from the network and forwards the computer-readable instructions for storage in the non-transitory computer-readable storage medium 110 in each computing / processing device.

[0032] Please refer to " Figure 2 ”, “ Figure 2” is a method flow chart of the virtual human evolution method based on AI learning observation of the present invention, which is executed by the hardware processor 120, and the steps include: at the initial stage, the hardware processor 120 loads the personality analysis model, and loads the virtual personality data and language model corresponding to the virtual character, wherein the personality analysis model and the language model are both pre-trained machine learning models (step 210); when the hardware processor 120 detects that the virtual character triggers a conversation event, the virtual character's voice is continuously recorded to generate a complete conversation voice, and the complete conversation voice is converted into a complete conversation information through speech-to-text technology (step 220); the hardware processor 120 uses the complete conversation information as new training data, and inputs the new training data into the pre-trained language model , to retrain the language model (step 230); and the hardware processor 120 inputs the complete conversation information into the personality analysis model to output the real person personality data, and dynamically adjusts the virtual personality data of the virtual character according to the real person personality data (step 240). Through the above steps, when the virtual character triggers a conversation event, the voice is continuously recorded to generate a complete conversation voice, and the complete voice is converted into complete conversation information through speech-to-text technology, and the complete conversation information is used as new training data to input into the pre-trained language model, so as to retrain the language model, and the complete conversation information is input into the personality analysis model to obtain the real person personality data, and the virtual personality data of the virtual character is dynamically adjusted according to the real person personality data.

[0033] The following matches " Figure 3 "and" Figure 4 ” is described as follows by way of example, such as “ Figure 3 " Figure 3” is a schematic diagram of adjusting the interactive personality of a virtual character by applying the present invention. When the user 310 and the virtual character 320 are talking, the hardware processor 120 detects that the virtual character 320 triggers a conversation event. At this time, the hardware processor 120 continues to record the voice of the virtual character 320 and the user 310 to generate a complete conversation voice. Then, the hardware processor 120 converts the complete conversation voice into a complete conversation information in text form through the voice-to-text technology. Afterwards, the hardware processor 120 first uses the complete conversation information as new training data, and then inputs the new training data into the pre-trained language. In addition, the hardware processor 120 will input the complete conversation information into the personality analysis model to output the real person personality data, and dynamically adjust the virtual personality data of the virtual character 320 according to the real person personality data. At this point, the language model has been retrained, and the virtual personality data has also been dynamically adjusted. In this way, the sentences used by the virtual character 320 during the conversation will be consistent with the user because of the retrained language model. At the same time, the personality displayed by the virtual character 320 will also be consistent with the user because of the adjusted virtual personality data. For example, " Figure 3 ” As shown in FIG. 3 , the conversation content of the user 310 is indicated by a dialogue box 311, and the conversation content of the virtual character 320 is indicated by a dialogue box 312. Assuming that the conversation voice of the user 310 is extroverted, initially, the virtual character 320 will have a conversation with the original sentences and virtual personality data. When the language model is retrained and the virtual personality data is readjusted, the virtual character 320 will change from the original conversation voice to the extroverted conversation voice. At this time, since the words and personality expressions of the virtual character 320 are highly similar to those of the user 310, the user 310 can feel that they are very compatible with each other, effectively improving the interaction compatibility.

[0034] like" Figure 4 " Figure 4” is a schematic diagram of the application of the present invention to adjust virtual personality data. In actual implementation, the personality expression of the virtual character is controlled by the virtual personality data. Assuming that the personality characteristic of the original virtual personality data 410 is introversion, the corresponding characteristic weight is a value of 50; the conversation style is cold, and the corresponding characteristic weight is a value of 30; the decision-making mode is inclined to cooperate, and the corresponding characteristic weight is a value of 60; the interaction method is listening, and the corresponding characteristic weight is a value of 40. After the extroverted user has a conversation with the virtual character, the complete conversation information generated by the user is not only used as new training data for retraining the language model, but also input into the personality analysis model to output real person personality data, and then adjust the virtual personality data of the virtual character according to the real person personality data. Among them, the simplest adjustment method is to make the data of the two consistent by direct replacement, but the disadvantage is the lack of difference, which may instead produce a feeling of repulsion between the same sex. Therefore, in actual implementation, linear weighted adjustment, adaptive adjustment, weighted average and other methods can also be selected to adjust the weights, and the weights of the two can be made similar but not completely the same through different feature weight calculation methods. With " Figure 3 "For example, the adjusted virtual personality data 420 will have personality traits that change from introversion to extroversion, conversation style to humor, and interaction mode to initiative. The characteristic weight of personality traits can be set to be greater than 50 to represent extroversion, and less than or equal to 50 to represent introversion; the characteristic weight of conversation style can be set to be greater than 50 to represent humor, and less than or equal to 50 to represent indifference; the characteristic weight of decision-making mode can be set to be greater than 50 to represent cooperation, and less than or equal to 50 to represent adventure; the characteristic weight of interaction mode can be set to be greater than 50 to represent initiative, and less than or equal to 50 to represent listening. It should be particularly noted that although the present invention uses the above examples to illustrate the characteristic weights, it does not limit the present invention to this, and any numerical representation method that can be used to distinguish different personalities does not depart from the application scope of the present invention.

[0035] In summary, it can be seen that the difference between the present invention and the prior art lies in that when a virtual character triggers a conversation event, voice is continuously recorded to generate a complete conversation voice, and the complete voice is converted into complete conversation information through speech-to-text technology, and the complete conversation information is input into a pre-trained language model as new training data to retrain the language model, and the complete conversation information is input into a personality analysis model to obtain real person personality data, and the virtual personality data of the virtual character is dynamically adjusted according to the real person personality data. This technical means can solve the problems existing in the previous technology, thereby achieving the technical effect of improving the interactive compatibility of the virtual character.

[0036] Although the present invention is disclosed as above with the aforementioned embodiments, it is not intended to limit the present invention. Any person skilled in the art may make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of patent protection of the present invention shall be subject to the scope defined by the claims attached to this specification.

Claims

1. A virtual human evolution system based on AI learning observation, the system comprising: A non-transitory computer-readable storage medium is used to store a plurality of computer-readable instructions, a personality analysis model, and virtual personality data and a language model corresponding to a virtual character, wherein: The personality analysis model and the language model are both pre-trained machine learning models; as well as A hardware processor is electrically connected to the non-transitory computer-readable storage medium and is used to execute the plurality of computer-readable instructions, so that the hardware processor is configured to: Initially, the personality analysis model is loaded, and the virtual personality data and the language model corresponding to the virtual character are loaded; When it is detected that the virtual character triggers a conversation event, the voice of the virtual character is continuously recorded to generate a complete conversation voice, and the complete conversation voice is converted into a complete conversation message through a speech-to-text technology; Using the complete conversation information as new training data, and inputting the new training data into the pre-trained language model to retrain the language model; as well as The complete conversation information is input into the personality analysis model to output real person personality data, and the virtual personality data of the virtual character is dynamically adjusted according to the real person personality data.

2. The virtual human evolution system based on AI learning and observation as claimed in claim 1, wherein: The language model is retrained by fine-tuning technology, and after pre-processing the new training data, multiple fine-tuning hyperparameters are set for retraining, and the hyperparameters at least include a learning rate and a batch size.

3. The virtual human evolution system based on AI learning and observation as claimed in claim 1, wherein: The language model is retrained through incremental learning technology to adjust the language model based on the new training data and gradient descent algorithm, and is combined with a memory enhancement network to store historical learning content during the training process to prevent the language model from forgetting past learning content.

4. The virtual human evolution system based on AI learning and observation as claimed in claim 1, wherein: The personality analysis model includes a support vector machine, a decision tree and a deep learning model to extract features and classification, so as to infer personality characteristics based on the complete conversation information to output as the real person personality data, and allows receiving at least one feedback opinion to adjust the weight of the personality analysis model as a basis for optimization.

5. The virtual human evolution system based on AI learning and observation as claimed in claim 1, wherein: The real person personality data and the virtual personality data both include a plurality of characteristic parameters consisting of personality traits, conversation style, decision-making mode and interaction method. Each characteristic parameter corresponds to a characteristic weight. When the virtual personality data of the virtual character is dynamically adjusted according to the real person personality data, the adjustment is performed by selecting one of the combinations of linear weighted adjustment, adaptive adjustment, weighted average and direct replacement.

6. A method for virtual human evolution based on AI learning observation, executed by a hardware processor, comprising the following steps: Initially, the hardware processor loads a personality analysis model, as well as virtual personality data and a language model corresponding to the virtual character, wherein the personality analysis model and the language model are both pre-trained machine learning models; When the hardware processor detects that the virtual character triggers a conversation event, the hardware processor continuously records the voice of the virtual character to generate a complete conversation voice, and converts the complete conversation voice into a complete conversation message through a speech-to-text technology; The hardware processor uses the complete conversation information as new training data, and inputs the new training data into the pre-trained language model to retrain the language model; and The hardware processor inputs the complete conversation information into the personality analysis model to output real person personality data, and dynamically adjusts the virtual personality data of the virtual character according to the real person personality data.

7. The method for virtual human evolution based on AI learning observation as claimed in claim 6, wherein: The language model is retrained by fine-tuning technology, and after pre-processing the new training data, multiple fine-tuning hyperparameters are set for retraining, and the hyperparameters at least include a learning rate and a batch size.

8. The method for virtual human evolution based on AI learning observation as claimed in claim 6, wherein: The language model is retrained through incremental learning technology to adjust the language model based on the new training data and gradient descent algorithm, and is combined with a memory enhancement network to store historical learning content during the training process to prevent the language model from forgetting past learning content.

9. The method for virtual human evolution based on AI learning observation as claimed in claim 6, wherein: The personality analysis model includes a support vector machine, a decision tree and a deep learning model to extract features and classification, so as to infer personality characteristics based on the complete conversation information to output as the real person personality data, and allows receiving at least one feedback opinion to adjust the weight of the personality analysis model as a basis for optimization.

10. The method for virtual human evolution based on AI learning observation as claimed in claim 6, wherein: The real person personality data and the virtual personality data both include a plurality of characteristic parameters consisting of personality traits, conversation style, decision-making mode and interaction method. Each characteristic parameter corresponds to a characteristic weight. When the virtual personality data of the virtual character is dynamically adjusted according to the real person personality data, the adjustment is performed by selecting one of the combinations of linear weighted adjustment, adaptive adjustment, weighted average and direct replacement.

Images