System and method based on interaction between virtual human model and real person

Abstract

The present invention relates to a method and system based on the interaction between a virtual human model and a real person, the method comprising the following steps: performing video acquisition, image acquisition and audio acquisition on a person to be recorded to obtain corresponding video data, image data and audio data; performing three-dimensional modeling on the person to be recorded based on the video data and image data to obtain a virtual human model; using audio data to establish a corresponding speech synthesis model; after being triggered, using holographic technology to project the virtual human model, and using AI technology to realize the interaction between the virtual human model and the real person, and in the process of interaction, using the speech synthesis model to synthesize and imitate the voice of the person to be recorded for speech interaction. The present invention can project a three-dimensional virtual human model through holographic technology, retaining the voice, appearance and smile of the person, and can combine AI technology to realize interaction with real people, realize face-to-face communication and interaction with real people, and can bring greater emotional comfort to people.

Description

Technical Field

[0001] The present invention relates to the field of information engineering technology, and in particular to a system and method based on interaction between a virtual human model and a real person. Background Art

[0002] In a person's life, there will be many special times, which may be when you are a child, when you are young, when you are old, or before you die. In different times and different environments, there are always some insights, some scenes, some knowledge or wealth, or some other things that you want to preserve.

[0003] At present, photos and videos will become our main way of recording, but the emotional comfort that photos and videos can bring to people is limited. People who watch them can only simply see the recorded content and cannot interact with it further. Summary of the invention

[0004] The purpose of the present invention is to overcome the defects of the prior art and provide a system and method based on the interaction between a virtual human model and a real person, so as to solve the problem that the existing photo and video recording methods can only bring limited emotional comfort to people and cannot further interact with people.

[0005] The technical solution to achieve the above purpose is:

[0006] The present invention provides a method for interaction between a virtual human model and a real person, comprising the following steps:

[0007] Perform video capture, image capture and audio capture on the person to be recorded to obtain corresponding video data, image data and audio data;

[0008] Performing three-dimensional modeling of the person to be recorded based on the video data and the image data to obtain a virtual human model;

[0009] Using the audio data to establish a corresponding speech synthesis model; and

[0010] After being triggered, the virtual human model is projected using holographic technology, and the interaction between the virtual human model and a real person is achieved using AI technology. During the interaction, the voice of the person to be recorded is synthesized and imitated by the voice synthesis model for voice interaction.

[0011] The present invention provides a method for interaction between a virtual human model and a real person, which records the person through video, image and audio. However, when restoring the person, a three-dimensional virtual human model can be projected through holographic technology, retaining the person's voice, appearance and smile. In addition, the method can be combined with AI technology to realize interaction with real people, thereby realizing face-to-face communication and interaction with real people, which can bring greater emotional comfort to people.

[0012] A further improvement of the method for interaction between a virtual human model and a real person according to the present invention is that it further comprises:

[0013] Establishing an action behavior library of the person to be recorded based on the video data;

[0014] When interacting with a real person, the virtual person projected by the holographic technology is controlled to perform corresponding actions according to the action feedback parameters formed by the AI ​​technology and the action behavior library.

[0015] A further improvement of the method based on the interaction between a virtual human model and a real person of the present invention is that it further comprises: collecting data of a specific scene for the person to be recorded;

[0016] Modeling the environment in the collected specific scene data to obtain a virtual environment model;

[0017] After a specific scene is triggered, the virtual human model and the virtual environment model are projected using holographic technology to present the corresponding specific scene.

[0018] A further improvement of the method based on the interaction between a virtual human model and a real person of the present invention is that the action behavior of the person to be recorded and the corresponding action conditions are extracted from the video data to form action intention data;

[0019] Extracting the conversation language of the characters to be recorded and the corresponding conversation conditions from the audio data to form conversation intention data;

[0020] Using AI technology to perform model training on the action intention data and the dialogue intention data to obtain an intention model;

[0021] When interacting with a real person, the intention model is used to output feedback information that conforms to the character habits to be recorded to complete the interaction with the real person.

[0022] A further improvement of the method for interaction between a virtual human model and a real person according to the present invention is that it further comprises:

[0023] Establish a cloud brain system;

[0024] When interacting with a real person, video of the real person is collected to obtain interactive video data;

[0025] The interactive video data is uploaded to the cloud brain system, and the cloud brain system is used to identify the interactive video data to form interactive feedback data, thereby completing the interaction with real people.

[0026] The present invention also provides a system based on interaction between a virtual human model and a real person, comprising:

[0027] A collection unit, used for collecting video, image and audio of the person to be recorded to obtain corresponding video data, image data and audio data;

[0028] A character modeling unit, connected to the acquisition unit, for performing three-dimensional modeling on the character to be recorded based on the video data and the image data to obtain a corresponding virtual human model;

[0029] a speech modeling unit, connected to the acquisition unit, and configured to establish a corresponding speech synthesis model using the audio data; and

[0030] The processing unit is connected to the acquisition unit, the character modeling unit and the voice modeling unit, and is used to project the virtual human model using holographic technology after receiving a trigger signal, and to use AI technology to realize the interaction between the virtual human model and a real person. During the interaction process, it is also used to synthesize and imitate the voice of the character to be recorded through a voice synthesis model for voice interaction.

[0031] A further improvement of the system based on the interaction between a virtual human model and a real person of the present invention is that it further comprises an action parameter extraction unit connected to the acquisition unit and the processing unit, and used to establish an action behavior library of the person to be recorded based on the video data;

[0032] The processing unit is also used to control the virtual person projected by the holographic technology to perform corresponding actions according to the action feedback parameters formed by the AI ​​technology and the action behavior library when interacting with a real person.

[0033] A further improvement of the system based on the interaction between a virtual human model and a real person of the present invention is that it further comprises a scene recording unit connected to the acquisition unit and the processing unit, and the data of a specific scene of the person to be recorded is collected through the acquisition unit;

[0034] The scene recording unit is used to model the environment in the collected specific scene data to obtain a virtual environment model;

[0035] The processing unit is used to project the virtual human model and the virtual environment model using holographic technology to present the corresponding specific scene after receiving a triggering specific scene.

[0036] A further improvement of the system based on the interaction between a virtual human model and a real person of the present invention is that it further includes an intention modeling unit connected to the acquisition unit and the processing unit, and is used to extract the action behavior of the person to be recorded and the corresponding action conditions from the video data to form action intention data;

[0037] Used to extract the conversation language of the characters to be recorded and the corresponding conversation conditions from the audio data to form conversation intention data;

[0038] It is also used to use AI technology to perform model training on the action intention data and the dialogue intention data to obtain an intention model;

[0039] When interacting with a real person, the processing unit uses the intention model to output feedback information that conforms to the character habits to be recorded to complete the interaction with the real person.

[0040] A further improvement of the system based on the interaction between a virtual human model and a real person of the present invention is that it also includes a cloud brain system;

[0041] The processing unit is connected to the cloud brain system. When interacting with a real person, it captures video of the real person to obtain interactive video data, transmits the interactive video data to the cloud brain system, and receives interactive feedback data from the cloud brain system to complete the interaction with the real person. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 The figure is a flow chart of the method for interaction between a virtual human model and a real person according to the present invention.

[0043] Figure 2 It is a system diagram of the system based on the interaction between a virtual human model and a real person according to the present invention. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0045] See also Figure 1 The present invention provides a system and method based on the interaction between a virtual human model and a real person, which is used to solve the problem that the existing recording method of photos and videos is relatively simple, and can only be viewed, but cannot communicate and interact with them, and the emotional comfort brought to people is limited. The system and method of the present invention combine holographic technology and AI technology to create a lifelike virtual person who truly imitates the recorded person, can communicate and interact with real people, realize the three-dimensional restoration of the person, retain the person's voice, appearance and smile, can provide warm and familiar communication and interaction, and can bring greater emotional comfort to people. The system and method based on the interaction between a virtual human model and a real person of the present invention are explained below in conjunction with the accompanying drawings.

[0046] See also Figure 2 , showing a system diagram of the system based on the interaction between the virtual human model and the real person of the present invention. Figure 2 , the system based on the interaction between the virtual human model and the real person of the present invention is explained.

[0047] like Figure 2As shown, the system based on the interaction between a virtual human model and a real person of the present invention comprises a collection unit 21, a character modeling unit 22, a voice modeling unit 23 and a processing unit 24, wherein the character modeling unit 22 is connected to the collection unit 21, the voice modeling unit 23 is connected to the collection unit 21, and the processing unit 24 is connected to the collection unit 21, the character modeling unit 22 and the voice modeling unit 23; the collection unit 21 is used to perform video collection, image collection and audio collection on the person to be recorded to obtain corresponding video data, image data and audio data; the character modeling unit 22 is used to perform three-dimensional modeling of the person to be recorded based on the video data and the image data to obtain a corresponding virtual human model; the voice modeling unit 23 is used to establish a corresponding voice synthesis model using audio data; the processing unit 24 is used to project the virtual human model using holographic technology after receiving a trigger signal, and to realize the interaction between the virtual human model and the real person using AI technology, and in the process of interaction, it is also used to synthesize and imitate the voice of the person to be recorded through the voice synthesis model to perform voice interaction.

[0048] The system of the present invention can establish a lifelike virtual person based on the video, image and audio of the person to be recorded, and the virtual person can communicate and interact with the real person in the voice of the recorded person, which can bring greater emotional comfort to people.

[0049] In a specific embodiment of the present invention, the acquisition unit 21 is connected to a camera, and the camera is used to shoot the person to be recorded to form corresponding video data. When shooting the person to be recorded, the acquisition can be divided into scenes, including scenes such as sleeping, reading, daydreaming, and eating. The acquisition unit 21 is also connected to a microphone, and the microphone is used to record the person to be recorded to form corresponding audio data. The recording content can be a conversation between the person and others, a reading of the person to a book, or a perception of the person. The acquisition unit 21 is also connected to a camera, and the camera is used to capture images of the person to be recorded to form corresponding image data. Preferably, the amount of video data, audio data, and image data collected by the acquisition unit 21 should be large, so that the virtual person can be more like the recorded person.

[0050] In another specific embodiment of the present invention, the acquisition unit 21 can also receive existing video data, image data and audio data uploaded by the user, that is, the user can generate video data, image data and audio data through other devices and upload these data to the acquisition unit 21.

[0051] In a specific implementation of the present invention, the system of the present invention further includes a storage unit, the acquisition unit 21 is connected to the storage unit, and the storage unit is used to store the video data, image data and audio data obtained by the acquisition unit.

[0052] In a specific embodiment of the present invention, the character modeling unit 22 is used to extract images recording various angles of the character from the video data, and also find images of various angles from the image data, and then extract point cloud data recording the entire body of the character from the image data, and then use the modeling software to establish a three-dimensional model based on the point cloud data to form a virtual human model.

[0053] In another specific embodiment of the present invention, the character modeling unit 22 can also receive an uploaded virtual human model, that is, the recorded character is hand-drawn based on the image and video of the recorded character offline to form a virtual human model, and then the virtual human model is uploaded to the character modeling unit 22.

[0054] In a specific embodiment of the present invention, the speech modeling unit 23 uses AI technology to cluster and analyze speech data, that is, to analyze the speaking habits and colloquialisms of the recorded person, and then customizes the sound features such as timbre and pitch in combination with the operation of TTS voice personalization, thereby obtaining a speech synthesis model that conforms to the speech of the recorded person. The speech synthesis model can be used to convert text into the voice of the recorded person, so that when interacting with a real person, it feels like a conversation with the recorded person, which can bring better emotional comfort to people.

[0055] In a specific embodiment of the present invention, the system of the present invention further comprises an action parameter extraction unit, which is connected to the acquisition unit 21 and the processing unit 24 and is used to establish an action behavior library of the person to be recorded based on the video data;

[0056] The processing unit 24 is also used to control the virtual person projected by the holographic technology to perform corresponding actions according to the action feedback parameters and action behavior library formed by the AI ​​technology when interacting with a real person.

[0057] Based on this, the system of the present invention not only realizes the voice interaction of the virtual person, but also realizes the action interaction of the virtual person. Through the combination of voice and action, the virtual person is made more lifelike, and the simulation restoration of the recorded person is realized.

[0058] Specifically, human actions are formed through the movement of limbs. The motion parameters of various parts and joints of the human body can be analyzed through video data, including the motion parameters of the head, neck, hands, arms, legs, feet and various joints, and then these motion parameters are stored to form a motion behavior library. When interacting with real people, corresponding action feedback parameters are formed according to the content of the interaction at that time, such as waving, nodding and other actions. According to the action feedback parameters, the corresponding action motion parameters are searched from the action behavior library, and then the virtual human model is controlled by the action motion parameters to perform corresponding movements, thus forming action interaction.

[0059] The actions stored in the action behavior library of the present invention are from the actual actions of the recorded characters, so when performing action interaction, the actions of the virtual person can be made very similar to the recorded characters, and the simulation effect is good.

[0060] In a specific embodiment of the present invention, the system of the present invention further comprises a scene recording unit connected to the acquisition unit 21 and the processing unit 24, and performs data acquisition of a specific scene on a person to be recorded through the acquisition unit 21;

[0061] The scene recording unit is used to model the environment in the collected specific scene data to obtain a virtual environment model;

[0062] The processing unit 24 is used to project the virtual human model and the virtual environment model using holographic technology to present the corresponding specific scene after receiving the trigger of the specific scene.

[0063] By setting up a scene recording unit, you can record the scenes you particularly want to save and restore them in three dimensions, such as life philosophy, life insights, meaningful conversations, etc.

[0064] Preferably, an option key associated with a specific scene may be set on the operation interface, and the user may trigger the reproduction of the specific scene by clicking and selecting the option key.

[0065] In a specific embodiment of the present invention, the system of the present invention further comprises an intention modeling unit connected to the acquisition unit 21 and the processing unit 24, for extracting the action behavior of the person to be recorded and the corresponding action conditions from the video data to form action intention data;

[0066] Used to extract the conversation language of the characters to be recorded and the corresponding conversation conditions from the audio data to form conversation intention data;

[0067] It is also used to use AI technology to train the action intention data and dialogue intention data to obtain the intention model;

[0068] When interacting with a real person, the processing unit 24 uses the intention model to output feedback information that conforms to the character habits to be recorded to complete the interaction with the real person.

[0069] The intention model can reflect the will of the person being recorded and give feedback that is consistent with the habits of the person being recorded. The intention model also includes semantic intention, sound intention, image intention, and behavioral intention, and can give feedback that is consistent with the recorder based on the input information. For example, when facing a frustrated interlocutor, the recorder will make tea for the other party and say some comforting words. The feedback information given by the intention module can be consistent with the habits of the recorder, making the interlocutor feel as if the real recorder is in front of him.

[0070] Furthermore, when establishing the intention model, a knowledge graph related to the recorded person is established based on all data related to the recorded person, forming a network knowledge structure, so that feedback that conforms to the habits of the recorded person can be given based on the input information.

[0071] In a specific embodiment of the present invention, the system of the present invention further includes a cloud brain system;

[0072] The processing unit is connected to the cloud brain system. When interacting with a real person, it captures video of the real person to obtain interactive video data, transmits the interactive video data to the cloud brain system, and receives interactive feedback data from the cloud brain system to complete the interaction with the real person.

[0073] Preferably, the cloud brain system includes a speech recognition subsystem, a semantic understanding subsystem, an image recognition subsystem and an encyclopedia knowledge graph. Through the speech recognition subsystem, the semantic understanding subsystem and the image recognition subsystem, the intention of the real person (i.e., the interlocutor) in the interactive video data can be known, and corresponding feedback can be given based on the understanding of the interlocutor's intention, so that a conversation can be carried out with the interlocutor to achieve companionship.

[0074] Furthermore, the intention model of the present invention is also stored in the cloud. After receiving the interactive video data, the cloud brain system obtains the speaker's intention, and inputs the speaker's intention into the intention model to obtain feedback data that matches the recorded person. After obtaining the feedback data that matches the recorded person, it is sent to the processing unit for execution. If the feedback data that matches the recorded person is not obtained, the corresponding feedback is obtained according to the encyclopedia knowledge graph in the cloud, and then sent to the processing unit for execution.

[0075] In a specific embodiment of the present invention, the system of the present invention further comprises a device for realizing holographic projection. The trigger signal received by the processing unit of the present invention may be a trigger signal formed by starting the system, or may be a trigger signal formed by collecting surrounding environment information and judging the occurrence of certain specific situations.

[0076] The system of the present invention can record infants, children, adults and the elderly, and establish models that conform to the habits of the recorders to achieve communication and interaction. The present invention uses holographic technology to restore the characters in three dimensions, integrates the behavior, posture and other characteristics of the characters, and combines AI technology to create a 100% restored virtual character. Combining TTS, ASR, NLP and OCR technologies in the AI ​​field, as well as knowledge graphs, it can restore real people's voices, movements and thinking habits, and intelligently interact with questions and answers to create a lifelike character.

[0077] The following describes the workflow of the system of the present invention by taking recording infants and children as an example.

[0078] Step 1: Record the child’s behavior through video and images, including the child’s response to external stimuli (including language, movements, behaviors, etc.).

[0079] Step 2: Classify the behaviors reported by the child and organize them into intents, and use AI technology to train the model. The intent includes but is not limited to semantic intent, sound intent, image intent, or behavior-related intent. For example: "When the child hears his mother's voice, he will open his mouth and smile", "When the child sees a toy, he will reach out to get the toy", "When the child hears loud talking, he will cry and make a fuss".

[0080] Step 3: Train the knowledge graph and construct the above content into a network knowledge structure.

[0081] Step 4: Use holographic projection technology to project the child’s image.

[0082] Step 5: We can interact with the projection through language and actions. Equipped with microphones, cameras, and sound receiving devices, AI technology can be used to identify the language of behavior and action, and the projection can use the knowledge graph for feedback.

[0083] The final effect is that the child is projected in front of us through holographic projection. We can see the child’s image and interact with the child.

[0084] The following describes the workflow of the system of the present invention by taking recording of adults and the elderly as an example.

[0085] Step 1: Record the behavior of the characters in video and images, which can be collected in different scenes, such as sleeping, reading, daydreaming, eating, etc. Record and train the behavior action models in various situations to facilitate the use of holographic technology for three-dimensional projection.

[0086] Step 2: Collect the speech of real people, use AI technology to cluster and analyze the language habits and colloquialisms of speaking, and then use TTS voice personalization to customize the sound characteristics such as timbre and tone.

[0087] Step 3: You can record the language that needs to be preserved, such as life philosophy, life insights, or words spoken to other people, and add it to the specific scene described in step 1.

[0088] Step 4: You can use long-term video recording to record how people deal with certain things, what they might say, and what actions they might take, and organize them into intents, and use AI technology to train models. The intent includes but is not limited to semantic intent, sound intent, image intent, or behavior-related intent. For example, "When facing a frustrated child, he will make a pot of tea for the child and say something to comfort him," "When his wife is sad, how will he comfort his wife?" "When he is irrational, how will he convince himself to stay calm."

[0089] Step 5: Train the knowledge graph and construct the above content into a network knowledge structure.

[0090] Step 6: Use holographic technology to project the characters. You can either actively turn on the holographic projection technology or automatically trigger the holographic projection device when certain situations are triggered. The content projected may be comfort, teaching, or entrustment, etc.

[0091] Step 7: Some encyclopedia knowledge graphs can be loaded into the knowledge graph for conversation and companionship.

[0092] The final effect is that when you are not calm or miss your deceased relatives, you can use the device to holographically project the image you want to see, communicate and conversation, and alleviate the negative feelings in your heart.

[0093] The present invention also provides a method based on the interaction between a virtual human model and a real person, which is described below.

[0094] like Figure 1 As shown, the method of the present invention comprises the following steps:

[0095] Execute step S11, perform video capture, image capture and audio capture on the person to be recorded to obtain corresponding video data, image data and audio data; then execute step S12;

[0096] Execute step S12, perform three-dimensional modeling of the person to be recorded based on the video data and the image data to obtain a virtual human model; then execute step S13;

[0097] Execute step S13 to establish a corresponding speech synthesis model using the audio data; then execute step S14;

[0098] Execute step S14. After being triggered, the virtual human model is projected using holographic technology, and the interaction between the virtual human model and the real person is realized using AI technology. During the interaction, the voice of the person to be recorded is synthesized and imitated by the voice synthesis model for voice interaction.

[0099] In a specific embodiment of the present invention, it also includes:

[0100] Establishing the action and behavior library of the person to be recorded based on the video data;

[0101] When interacting with real people, the virtual person projected by holographic technology is controlled to perform corresponding actions based on the action feedback parameters and action behavior library formed by AI technology.

[0102] In a specific implementation of the present invention, it also includes: collecting data of a specific scene for a person to be recorded;

[0103] Modeling the environment in the collected specific scene data to obtain a virtual environment model;

[0104] After a specific scene is triggered, the virtual human model and the virtual environment model are projected using holographic technology to present the corresponding specific scene.

[0105] In a specific embodiment of the present invention, the action behavior of the person to be recorded and the corresponding action conditions are extracted from the video data to form action intention data;

[0106] Extracting the conversation language of the characters to be recorded and the corresponding conversation conditions from the audio data to form conversation intention data;

[0107] Use AI technology to train the action intention data and conversation intention data to obtain the intention model;

[0108] When interacting with real people, the intention model is used to output feedback information that conforms to the character habits that need to be recorded to complete the interaction with the real person.

[0109] In a specific embodiment of the present invention, it also includes:

[0110] Establish a cloud brain system;

[0111] When interacting with a real person, video of the real person is collected to obtain interactive video data;

[0112] The interactive video data is uploaded to the cloud brain system, and the cloud brain system is used to identify the interactive video data to form interactive feedback data, thereby completing the interaction with real people.

[0113] In life, when faced with beautiful memories, warm scenes, loved ones, or instructions to future generations, confessions to lovers, or remembrance of deceased relatives, it is very necessary to preserve them effectively and then restore and reproduce them, which can bring great emotional comfort to people.

[0114] The present invention is described in detail above in conjunction with the embodiments of the accompanying drawings. A person skilled in the art can make various variations of the present invention according to the above description. Therefore, certain details in the embodiments should not constitute a limitation of the present invention, and the scope of protection of the present invention shall be defined by the scope of the attached claims.

Claims

1. A method for interaction between a virtual human model and a real person, characterized in that: The steps include: Perform video capture, image capture and audio capture on the person to be recorded to obtain corresponding video data, image data and audio data; Based on the video data and the image data, three-dimensional modeling is performed on the person to be recorded to obtain a virtual human model; images of various angles of the recorded person are extracted from the video data, and images of various angles are also found from the image data, and then point cloud data of the whole body of the recorded person is extracted from the image, and a three-dimensional model is established according to the point cloud data through modeling software to form a virtual human model; or hand-drawn modeling of the recorded person is performed based on the image and video of the recorded person to form a virtual human model; The audio data is used to establish a corresponding speech synthesis model; the speech data is clustered and analyzed using AI technology to analyze the speaking habits and spoken language of the recorded person, and then the timbre and tone are customized in combination with the TTS voice personalization customization operation, so as to obtain a speech synthesis model that conforms to the recorded person's speaking voice; After being triggered, the virtual human model is projected using holographic technology, and the interaction between the virtual human model and the real person is realized using AI technology. During the interaction, the voice of the person to be recorded is synthesized and imitated by the voice synthesis model for voice interaction; Also includes: Based on the video data, a motion behavior library of the person to be recorded is established; the human motion is formed by the movement of the limbs, and the motion parameters of various parts and joints of the human body are obtained by analyzing the video data, including the motion parameters of the head, the neck, the hands, the arms, the legs, the feet and the motion parameters of various joints, and then these motion parameters are stored to form a motion behavior library; When interacting with a real person, the virtual person projected by the holographic technology is controlled to perform corresponding actions according to the action feedback parameters formed by the AI ​​technology and the action behavior library; the corresponding action feedback parameters are formed according to the content of the interaction at that time, and the corresponding action motion parameters are searched from the action behavior library according to the action feedback parameters, and then the virtual person model is controlled to perform corresponding movements through the action motion parameters, thus forming action interaction; It also includes: collecting data on specific scenes of people to be recorded; Modeling the environment in the collected specific scene data to obtain a virtual environment model; After a specific scene is triggered, the virtual human model and the virtual environment model are projected using holographic technology to present the corresponding specific scene; Extracting the action behaviors of the person to be recorded and the corresponding action conditions from the video data to form action intention data; Extracting the conversation language of the characters to be recorded and the corresponding conversation conditions from the audio data to form conversation intention data; Using AI technology to perform model training on the action intention data and the dialogue intention data to obtain an intention model; When interacting with a real person, the intention model is used to output feedback information that conforms to the character habits to be recorded to complete the interaction with the real person.

2. The method for interaction between a virtual human model and a real person according to claim 1, characterized in that: Also includes: Establish a cloud brain system; When interacting with a real person, video of the real person is collected to obtain interactive video data; The interactive video data is uploaded to the cloud brain system, and the cloud brain system is used to identify the interactive video data to form interactive feedback data, thereby completing the interaction with real people.

3. A system based on interaction between a virtual human model and a real person, characterized in that: include: A collection unit, used for collecting video, image and audio of the person to be recorded to obtain corresponding video data, image data and audio data; A character modeling unit is connected to the acquisition unit and is used to perform three-dimensional modeling of the character to be recorded based on the video data and the image data to obtain a corresponding virtual human model; the character modeling unit is used to extract images of various angles of the recorded character from the video data, and also find images of various angles from the image data, and then extract point cloud data of the whole body of the recorded character from the image data, and then establish a three-dimensional model according to the point cloud data through modeling software to form a virtual human model; or, the character modeling unit receives the uploaded virtual human model, performs hand-drawn modeling of the recorded character based on the image and video of the recorded character to form a virtual human model, and then uploads the virtual human model to the character modeling unit; A speech modeling unit is connected to the acquisition unit and is used to establish a corresponding speech synthesis model using the audio data; the speech modeling unit uses AI technology to cluster and analyze the speech data, analyze and record the speaking habits and spoken language of the person, and then customizes the timbre and tone in combination with the operation of TTS voice personalization, so as to obtain a speech synthesis model that conforms to the speech of the person being recorded; A processing unit connected to the acquisition unit, the character modeling unit and the voice modeling unit, and configured to project the virtual human model using holographic technology after receiving a trigger signal, and to realize the interaction between the virtual human model and a real person using AI technology, and to perform voice interaction by synthesizing and imitating the voice of the character to be recorded through a voice synthesis model during the interaction; It also includes an action parameter extraction unit, which is connected to the acquisition unit and the processing unit and is used to establish an action behavior library of the person to be recorded based on the video data; The processing unit is also used to control the virtual person projected by the holographic technology to perform corresponding actions according to the action feedback parameters formed by the AI ​​technology and the action behavior library when interacting with a real person; the human action is formed by the movement of the limbs, and the motion parameters of various parts and joints of the human body are obtained by video data analysis, including the motion parameters of the head, the neck, the hands, the arms, the legs, the feet and the motion parameters of various joints, and then these motion parameters are stored to form an action behavior library; when interacting with a real person, corresponding action feedback parameters are formed according to the content of the interaction at that time, and the corresponding action motion parameters are searched from the action behavior library according to the action feedback parameters, and then the virtual human model is controlled to perform corresponding movements through the action motion parameters, thereby forming action interaction; It also includes a scene recording unit, which is connected to the acquisition unit and the processing unit, and collects data of a specific scene for a person to be recorded through the acquisition unit; The scene recording unit is used to model the environment in the collected specific scene data to obtain a virtual environment model; The processing unit is used to project the virtual human model and the virtual environment model using holographic technology to present the corresponding specific scene after receiving the triggering specific scene; It also includes an intention modeling unit connected to the acquisition unit and the processing unit, and is used to extract the action behavior of the person to be recorded and the corresponding action conditions from the video data to form action intention data; Used to extract the conversation language of the characters to be recorded and the corresponding conversation conditions from the audio data to form conversation intention data; It is also used to use AI technology to perform model training on the action intention data and the dialogue intention data to obtain an intention model; When interacting with a real person, the processing unit uses the intention model to output feedback information that conforms to the character habits to be recorded to complete the interaction with the real person.

4. The system based on interaction between a virtual human model and a real person as claimed in claim 3, characterized in that: It also includes the cloud brain system; The processing unit is connected to the cloud brain system. When interacting with a real person, it captures video of the real person to obtain interactive video data, transmits the interactive video data to the cloud brain system, and receives interactive feedback data from the cloud brain system to complete the interaction with the real person.

Images