Digital human platform interaction system and method based on artificial intelligence

Abstract

The invention discloses a digital human platform interaction system and method based on artificial intelligence, and relates to the technical field of man-machine interaction, and the method comprises the steps: combining facial features and preferences of a user under different emotions through a digital twinborn technology, and constructing a virtual user image; constructing a digital person independently belonging to each user according to the generated digital person image and voice of each user, and storing the digital person of each user to construct a digital person warehouse; when the user interacts with the digital person, face data of the user is collected in real time to be matched with the virtual user image, and after matching, the digital person independently belonging to the real-time user is selected according to the virtual user image and is displayed; the digital person collects answers in the network according to the interaction question and classifies all the collected answers; selecting an interaction question answer satisfied by the user; and the digital person modifies the answer of the interaction question according to the user emotion and outputs the modified answer to the real-time user.

Description

Technical Field

[0001] The present invention relates to the field of human-computer interaction technology, and in particular to an artificial intelligence-based digital human platform interaction system and method. Background Art

[0002] Digital humans first appeared in the film and game fields, and were used to create simple virtual characters or character animations. These early digital human images were relatively crude, mainly achieved through manual modeling and keyframe animation, and their interactivity was also very limited. They could usually only perform simple actions and dialogues according to preset scripts. With the initial development of computer graphics and artificial intelligence technology, the appearance and action performance of digital humans have improved to a certain extent, and they have begun to have some simple interactive functions, such as limited interaction with users through mouse clicks or keyboard input. At the same time, some virtual assistant-type digital human applications began to appear, but the technology in speech recognition and natural language understanding was not mature enough, and the interactive experience was relatively stiff. Deep learning technology in the field of artificial intelligence has made a major breakthrough, bringing a qualitative leap in digital human interaction. Digital humans have begun to be widely used in many fields, such as intelligent customer service, virtual anchors, education, etc. Voice interaction technology has gradually matured, and digital humans can more accurately recognize and understand users' voice commands and respond with natural and fluent voice. At the same time, the generation of facial expressions and body movements is also more natural and realistic, which can better match the voice content, enhancing the realism and fun of the interaction. However, when users interact with digital humans today, they are served by one image for all users. Users feel alienated and disconnected when interacting with digital humans. Moreover, when digital humans generate answers to user questions, they are unable to change the form of answers based on the user's real-time status. This has a significant impact on the user's experience and senses when interacting with digital humans. Summary of the invention

[0003] The purpose of the present invention is to provide a digital human platform interaction system and method based on artificial intelligence to solve the problems raised in the prior art.

[0004] To achieve the above object, the present invention provides the following technical solutions: A digital human platform interaction method based on artificial intelligence, the method comprising the following steps: S100, collect statistics on all users who interact with the digital human, collect different emotions and corresponding facial data of each user when interacting with the digital human in the historical records, extract the facial features of the user when they are in different emotions, collect the feedback data of the user on the interaction results when each user interacts with the digital human in the historical records, and use the feedback data to obtain the user's preference for the interaction results; use the digital twin technology to combine the facial features and preferences of the user when they are in different emotions to construct a virtual user image; Furthermore, the specific steps of using digital twin technology to combine the facial features and preferences of users with different emotions to construct a virtual user image are as follows: S101. Count all users who interact with the digital human to obtain a user number of n, and collect the emotion types of each user when interacting with the digital human in the historical records. , It represents the 1st, 2nd, 3rd, ..., nth emotions of each user in the collected historical records when interacting with the digital human, where n is a positive integer; extract the facial data of the user when interacting with the digital human when the user has different emotions, and use the Dlib library to find the key points in the facial data: , Indicates the 1st, 2nd, 3rd, ..., mth key points in the facial data to be searched, where m is a positive integer; S102: Establish a coordinate system in the user's facial data and extract the coordinate points of each key point. , Represent the coordinates of the 1st, 2nd, 3rd, ..., mth key points in the facial data, and calculate the geometric distance between every two key points. The formula is: ; In the formula, Represents the geometric distance between the i-th and j-th key points among the m key points, and represents the horizontal and vertical coordinates of the i-th key point, and represents the horizontal and vertical coordinates of the jth key point; i and j are from 1 to m; the geometric distance of each key point under different user emotions is calculated as follows: , Indicates the 1st, 2nd, 3rd, ..., For the geometric distance of key points, the geometric distance set A of key points of the user with different emotions is used as the facial feature of each emotion; S103, collecting feedback data of each user on the interaction results when interacting with the digital human in the historical records, and correspondingly extracting the result features of the digital human when the user interacted with the digital human in the history; matching the collected user feedback data with the corresponding result features of the digital human to obtain a composition element combination as , represents the 1st, 2nd, 3rd, ..., kth element combination obtained after matching, where F represents the feedback data, J represents the result feature of the digital person; k is a positive integer; the number of times for each element combination is collected and the average value is calculated, and the element combination with a number greater than the average value is defined as a frequent group; the confidence of all frequent groups is calculated, and the formula is: ; In the formula, represents the confidence of the calculated t-th frequent group, represents the number of times the tth frequent group is collected, Represents the number of feedback data collected in the tth frequent group; calculates the confidence of all frequent groups, and after judgment, selects the result characteristics of the digital human in the frequent group with the largest confidence as the user's preference, and repeats the calculation to obtain the preferences of all users; uses digital twin technology to combine the facial features and preferences of users in different emotions to construct a virtual user image.

[0005] By using the facial data and feedback data of the user's interaction with the digital human in the historical records, the facial features of different emotions and user preferences are obtained to build a virtual user image; through the analysis of the virtual image, the user's appearance characteristics, expression habits, action preferences and other personalized information can be deeply understood. Based on these analysis results, the digital human can provide users with interactive content and service methods that are more in line with their personal preferences and needs, making the interaction process more natural, smooth and comfortable, and enhancing the user's sense of participation and immersion.

[0006] Virtual images can be used as an external manifestation of users' subconscious needs. Analyzing them can help tap into users' potential needs that are not clearly expressed. Enterprises or service providers can plan and optimize products or services in advance to better meet users' future needs and improve user satisfaction and loyalty.

[0007] By continuously analyzing the virtual image, the implementation effect of the strategy can be monitored and evaluated in real time. According to the analysis results, problems and deviations in the strategy execution process can be discovered in time, and corresponding adjustments and optimizations can be made to ensure that the strategy can always move towards the expected goal and achieve effective management and continuous optimization of the business.

[0008] S200, for each user's virtual user image constructed, collect the virtual images browsed by each user on the network, extract the appearance features of the browsed virtual images to construct a digital human image unique to each user; collect the audio data listened to by each user on the network, extract the voice features of the audio data, and construct a digital human voice unique to each user; construct a digital human unique to each user with the generated digital human image and voice of each user, and save the digital human of each user to construct a digital human warehouse; Furthermore, the specific steps for saving each user's digital human and building a digital human warehouse are as follows: S201, for each user's virtual user image constructed, collect the virtual images browsed by each user on the network, use the face detection algorithm to extract the facial key points of the collected virtual images, standardize the data value of each facial key point, unify them into the same data format, and calculate the performance value of each facial key point, the formula is: ; In the formula, B represents the performance value of each key point, R represents the total number of differences between every two data values ​​in each key point, and H and H' represent the different data values ​​in each key point. After calculating the performance value of each key point, the image segmentation technology is used to segment each key point in the facial data to form a separate appearance feature. The number of views of each segmented appearance feature by the user in the historical records is extracted respectively, and a probability model is constructed, which is expressed as: ; In the model represents the probability of the vth appearance feature, represents the number of views of the vth appearance feature, Indicates the total number of times the user has viewed all appearance features; uses a deep learning model to train the probabilities of all appearance features, and selects all appearance features with a probability greater than one-half as the user's favorite appearance; finally, uses image generation technology to combine all the user's favorite appearance features to generate the user's favorite digital human image; S202, collecting audio data listened to by each user on the network, extracting timbre features from the collected audio data using an audio processing algorithm, calculating the probability of each timbre feature using the same probability model as in S201, and selecting the timbre feature with the highest probability as the user's favorite audio feature; repeating the operation to calculate the favorite audio feature of each user; S203. After calculating the digital human image and audio features that each user likes, use an adversarial network algorithm to build a digital human that belongs to each user according to the digital human image and audio features that each user likes, and save the digital human of each user to build a digital human warehouse.

[0009] Different users have different preferences and needs for the appearance and voice of digital people. Some users may like a youthful and energetic appearance and a crisp and pleasant voice, while others may prefer a mature and stable appearance and a deep and mellow voice. Generating a digital person that is unique to each user can accurately meet these diverse needs and provide users with an exclusive digital partner that fits their personal taste. When users interact with digital humans, familiar appearance and voice help users understand the information conveyed by digital humans more quickly. For example, in a voice interaction scenario, the user's own voice style allows them to more instinctively capture the key points and emotions in the digital human's speech, reducing the understanding barriers that may arise due to unfamiliar voices, thereby making communication smoother and improving the efficiency of information transmission.

[0010] S300, when the user interacts with the digital human, the user's facial data is collected in real time and matched with the virtual user image, and after matching, a digital human unique to the real-time user is selected according to the virtual user image and displayed; Furthermore, after matching, the specific steps of selecting a digital person unique to the real-time user according to the virtual user image and displaying it are as follows: S301, when the user interacts with the digital human, collect the user's facial data in real time, extract the key points in the real-time user's facial data, collect the data value of each key point accordingly, and extract the data value of each key point in the virtual user image; calculate the facial similarity of the real-time user and all virtual user images, the formula is: ; In the formula, represents the facial similarity between the real-time user and each virtual user image, E represents the number of key points in the extracted real-time user facial data, Represents the data value of the pth key point of the real-time user, Represents the data value of the pth key point in the virtual user image; S302, after calculating the facial similarity between the real-time user and each virtual user image, select the virtual user image with the facial similarity closest to 1 and match it with the real-time user to obtain the mapped virtual image of the real-time user; and select a digital human unique to the real-time user in the digital human warehouse for display according to the mapped virtual image of the real-time user.

[0011] S400: After selecting the digital human of the real-time user, voice data of the user interacting with the digital human is collected, and the user voice data is analyzed using a natural language processing algorithm to generate interactive questions that the real-time user asks the digital human; the digital human collects answers on the network according to the interactive questions and classifies all the collected answers; Furthermore, the specific steps for the digital human to collect answers on the Internet according to the interactive questions and classify all the collected answers are as follows: S401, after selecting the digital human of the real-time user, collecting the voice data of the user interacting with the digital human, analyzing the user voice data using a natural language processing algorithm, and generating interactive questions that the real-time user asks the digital human; S402: The digital human collects answers on the network according to the interactive questions and then classifies all the collected answers according to the interactive result features in the historical records of S100. , Represents the 1st, 2nd, 3rd, ..., bth answers obtained after classification, where b is a positive integer.

[0012] S500, extracting preferences from the user's virtual image to judge each of the classified answers, and selecting an answer to the interactive question that satisfies the user; Furthermore, the specific steps for selecting the answers to the interactive questions that satisfy the user are as follows: S501, extracting the preferences of the user's virtual image to judge each answer after classification, and selecting the answer with the same result characteristics as the user's preference as the optimal answer.

[0013] S600: After the answer to the interactive question is selected, the real-time facial data of the user and the digital human when interacting is collected, and the emotions of the real-time user are judged in the user's virtual image. The digital human modifies the answer to the interactive question according to the user's emotions and outputs it to the real-time user.

[0014] Furthermore, the specific steps for the digital human to modify the answer to the interactive question according to the user's emotions and then output it to the real-time user are: S601, after selecting the answer to the interactive question, collect the real-time facial data of the user and the digital human when interacting, extract the coordinates of each key point in the real-time user facial data, and calculate the geometric distance between every two key points in the real-time user facial data. , Indicates the 1st, 2nd, 3rd, ..., The geometric distance between the key points is compared with the key point geometric distance A' of the real-time user and the key point geometric distance set A in the mapped virtual image of the real-time user; S602, when A=A', extract the user emotions corresponding to the A set as the interactive emotions of the real-time user, add the real-time user interactive emotion modifier to the selected best answer using the natural language processing algorithm, and finally output it to the real-time user.

[0015] Analyzing the expressions, postures and other elements of the virtual image can help the digital human better understand the user's emotional state. In this way, the digital human can make corresponding emotional responses and establish a deeper emotional connection with the user, allowing the user to feel more understanding and resonance in the interaction, thus improving the quality and depth of the interaction.

[0016] A digital human platform interaction system based on artificial intelligence, the digital human platform interaction system includes a data collection module, a virtual user image construction module, a digital human construction module, a real-time interaction module, and an output module; The data collection module is used to collect data on the user's interaction with the digital human in the historical records; The virtual user image construction module is used to extract the facial features of the user when in different emotions based on the facial data of the user during interaction in the collected historical records, obtain the user's preference for the interaction results using the feedback data, and use the digital twin technology to combine the facial features and preferences of the user when in different emotions to construct a virtual user image; The digital human construction module is used to extract the appearance features of the virtual images browsed in the collected historical records to construct a digital human image unique to each user, extract the voice features of the audio data, construct a digital human voice unique to each user, and construct a digital human unique to each user; The real-time interaction module is used to collect the real-time user's facial data to find and map the virtual image, then select the digital human answer according to the user's preference, and judge the real-time user's emotions according to the real-time user's facial data; The output module is used to add modifiers to the selected best answer according to the real-time user's emotions and then output it to the real-time user.

[0017] The virtual user image building module includes emotion unit and preference unit; The emotion unit is used to extract the facial features of the user when they are in different emotions based on the facial data of the user when they interact in the collected historical records; The preference unit is used to calculate the preference of each user according to the collected user feedback data and result characteristics.

[0018] The real-time interaction module includes a mapping avatar unit, an answer selection unit, and an emotion judgment unit; The mapping virtual image unit is used to collect facial data of the real-time user, calculate the similarity between the real-time user and the virtual user image, and select the mapping virtual image of the real-time user; The answer selection unit is used to collect real-time user facial data, search for mapped virtual images, and then select digital human answers according to user preferences; The emotion judgment unit is used to judge the real-time user emotion according to the real-time user's facial data.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention uses the facial data and feedback data of the user's interaction with the digital human in the historical records to obtain the facial features of different emotions and user preferences, and construct a virtual user image; through the analysis of the virtual image, the user's appearance characteristics, expression habits, action preferences and other personalized information can be deeply understood. Based on these analysis results, the digital human can provide users with interactive content and service methods that are more in line with their personal preferences and needs, making the interaction process more natural, smooth and comfortable, and enhancing the user's sense of participation and immersion.

[0020] 2. After the present invention builds a digital human that is unique to each user, the familiar appearance and voice of the user when interacting with the digital human will help the user understand the information conveyed by the digital human more quickly. For example, in a voice interaction scenario, the user's own voice style can allow him to more instinctively capture the key points and emotions in the digital human's speech, reducing the understanding barriers that may arise due to unfamiliar voice, thereby making communication smoother and improving the efficiency of information transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a module distribution diagram of a digital human platform interaction system based on artificial intelligence according to the present invention; Figure 2 The present invention is a schematic diagram of the steps of a digital human platform interaction method based on artificial intelligence. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] Example: Figure 1-Figure 2 As shown, the present invention provides a technical solution. A digital human platform interaction method based on artificial intelligence, the method comprising the following steps: S100, collect statistics on all users who interact with the digital human, collect different emotions and corresponding facial data of each user when interacting with the digital human in the historical records, extract the facial features of the user when they are in different emotions, collect the feedback data of the user on the interaction results when each user interacts with the digital human in the historical records, and use the feedback data to obtain the user's preference for the interaction results; use the digital twin technology to combine the facial features and preferences of the user when they are in different emotions to construct a virtual user image; The specific steps of using digital twin technology to combine the facial features and preferences of users with different emotions to build a virtual user image are: S101. Count all users who interact with the digital human to obtain a user number of n, and collect the emotion types of each user when interacting with the digital human in the historical records. , It represents the 1st, 2nd, 3rd, ..., nth emotions of each user in the collected historical records when interacting with the digital human, where n is a positive integer; extract the facial data of the user when interacting with the digital human when the user has different emotions, and use the Dlib library to find the key points in the facial data: , Indicates the 1st, 2nd, 3rd, ..., mth key points in the facial data to be searched, where m is a positive integer; S102: Establish a coordinate system in the user's facial data and extract the coordinate points of each key point. , Represent the coordinates of the 1st, 2nd, 3rd, ..., mth key points in the facial data, and calculate the geometric distance between every two key points. The formula is: ; In the formula, Represents the geometric distance between the i-th and j-th key points among the m key points, and represents the horizontal and vertical coordinates of the i-th key point, and represents the horizontal and vertical coordinates of the jth key point; i and j are from 1 to m; the geometric distance of each key point under different user emotions is calculated as follows: , Indicates the 1st, 2nd, 3rd, ..., For the geometric distance of key points, the geometric distance set A of key points of the user with different emotions is used as the facial feature of each emotion; S103, collecting feedback data of each user on the interaction results when interacting with the digital human in the historical records, and correspondingly extracting the result features of the digital human when the user interacted with the digital human in the history; matching the collected user feedback data with the corresponding result features of the digital human to obtain a composition element combination as , represents the 1st, 2nd, 3rd, ..., kth element combination obtained after matching, where F represents the feedback data, J represents the result feature of the digital person; k is a positive integer; the number of times for each element combination is collected and the average value is calculated, and the element combination with a number greater than the average value is defined as a frequent group; the confidence of all frequent groups is calculated, and the formula is: ; In the formula, represents the confidence of the calculated t-th frequent group, represents the number of times the tth frequent group is collected, Represents the number of feedback data collected in the tth frequent group; calculates the confidence of all frequent groups, and after judgment, selects the result characteristics of the digital human in the frequent group with the largest confidence as the user's preference, and repeats the calculation to obtain the preferences of all users; uses digital twin technology to combine the facial features and preferences of users in different emotions to construct a virtual user image.

[0024] By using the facial data and feedback data of the user's interaction with the digital human in the historical records, the facial features of different emotions and user preferences are obtained to build a virtual user image; through the analysis of the virtual image, the user's appearance characteristics, expression habits, action preferences and other personalized information can be deeply understood. Based on these analysis results, the digital human can provide users with interactive content and service methods that are more in line with their personal preferences and needs, making the interaction process more natural, smooth and comfortable, and enhancing the user's sense of participation and immersion.

[0025] Virtual images can be used as an external manifestation of users' subconscious needs. Analyzing them can help tap into users' potential needs that are not clearly expressed. Enterprises or service providers can plan and optimize products or services in advance to better meet users' future needs and improve user satisfaction and loyalty.

[0026] By continuously analyzing the virtual image, the implementation effect of the strategy can be monitored and evaluated in real time. According to the analysis results, problems and deviations in the strategy execution process can be discovered in time, and corresponding adjustments and optimizations can be made to ensure that the strategy can always move towards the expected goal and achieve effective management and continuous optimization of the business.

[0027] S200, for each user's virtual user image constructed, collect the virtual images browsed by each user on the network, extract the appearance features of the browsed virtual images to construct a digital human image unique to each user; collect the audio data listened to by each user on the network, extract the voice features of the audio data, and construct a digital human voice unique to each user; construct a digital human unique to each user with the generated digital human image and voice of each user, and save the digital human of each user to construct a digital human warehouse; The specific steps to save each user's digital human and build a digital human warehouse are: S201, for each user's virtual user image constructed, collect the virtual images browsed by each user on the network, use the face detection algorithm to extract the facial key points of the collected virtual images, standardize the data value of each facial key point, unify them into the same data format, and calculate the performance value of each facial key point, the formula is: ; In the formula, B represents the performance value of each key point, R represents the total number of differences between every two data values ​​in each key point, and H and H' represent the different data values ​​in each key point. After calculating the performance value of each key point, the image segmentation technology is used to segment each key point in the facial data to form a separate appearance feature. The number of views of each segmented appearance feature by the user in the historical records is extracted respectively, and a probability model is constructed, which is expressed as: ; In the model represents the probability of the vth appearance feature, represents the number of views of the vth appearance feature, Indicates the total number of times the user has viewed all appearance features; uses a deep learning model to train the probabilities of all appearance features, and selects all appearance features with a probability greater than one-half as the user's favorite appearance; finally, uses image generation technology to combine all the user's favorite appearance features to generate the user's favorite digital human image; S202, collecting audio data listened to by each user on the network, extracting timbre features from the collected audio data using an audio processing algorithm, calculating the probability of each timbre feature using the same probability model as in S201, and selecting the timbre feature with the highest probability as the user's favorite audio feature; repeating the operation to calculate the favorite audio feature of each user; S203. After calculating the digital human image and audio features that each user likes, use an adversarial network algorithm to build a digital human that belongs to each user according to the digital human image and audio features that each user likes, and save the digital human of each user to build a digital human warehouse.

[0028] Different users have different preferences and needs for the appearance and voice of digital people. Some users may like a youthful and energetic appearance and a crisp and pleasant voice, while others may prefer a mature and stable appearance and a deep and mellow voice. Generating a digital person unique to each user can accurately meet these diverse needs and provide users with an exclusive digital partner that fits their personal taste.

[0029] When users interact with digital humans, familiar appearance and voice help users understand the information conveyed by digital humans more quickly. For example, in a voice interaction scenario, the user's own voice style allows them to more instinctively capture the key points and emotions in the digital human's speech, reducing the understanding barriers that may arise due to unfamiliar voices, thereby making communication smoother and improving the efficiency of information transmission.

[0030] S300, when the user interacts with the digital human, the user's facial data is collected in real time and matched with the virtual user image, and after matching, a digital human unique to the real-time user is selected according to the virtual user image and displayed; After matching, the specific steps of selecting a digital person unique to the real-time user according to the virtual user image and displaying it are as follows: S301, when the user interacts with the digital human, collect the user's facial data in real time, extract the key points in the real-time user's facial data, collect the data value of each key point accordingly, and extract the data value of each key point in the virtual user image; calculate the facial similarity of the real-time user and all virtual user images, the formula is: ; In the formula, represents the facial similarity between the real-time user and each virtual user image, E represents the number of key points in the extracted real-time user facial data, Represents the data value of the pth key point of the real-time user, Represents the data value of the pth key point in the virtual user image; S302, after calculating the facial similarity between the real-time user and each virtual user image, select the virtual user image with the facial similarity closest to 1 and match it with the real-time user to obtain the mapped virtual image of the real-time user; and select a digital human unique to the real-time user in the digital human warehouse for display according to the mapped virtual image of the real-time user.

[0031] S400: After selecting the digital human of the real-time user, voice data of the user interacting with the digital human is collected, and the user voice data is analyzed using a natural language processing algorithm to generate interactive questions that the real-time user asks the digital human; the digital human collects answers on the network according to the interactive questions and classifies all the collected answers; The specific steps for the digital human to collect answers on the Internet based on interactive questions and classify all the collected answers are: S401, after selecting the digital human of the real-time user, collecting the voice data of the user interacting with the digital human, analyzing the user voice data using a natural language processing algorithm, and generating interactive questions that the real-time user asks the digital human; S402: The digital human collects answers on the network according to the interactive questions and then classifies all the collected answers according to the interactive result features in the historical records of S100. , Represents the 1st, 2nd, 3rd, ..., bth answers obtained after classification, where b is a positive integer.

[0032] S500, extracting preferences from the user's virtual image to judge each of the classified answers, and selecting an answer to the interactive question that satisfies the user; The specific steps to select the answers to interactive questions that satisfy users are as follows: S501, extracting the preferences of the user's virtual image to judge each answer after classification, and selecting the answer with the same result characteristics as the user's preference as the optimal answer.

[0033] S600: After the answer to the interactive question is selected, the real-time facial data of the user and the digital human when interacting is collected, and the emotions of the real-time user are judged in the user's virtual image. The digital human modifies the answer to the interactive question according to the user's emotions and outputs it to the real-time user.

[0034] The specific steps for the digital human to modify the answer to the interactive question based on the user's emotions and then output it to the real-time user are: S601, after selecting the answer to the interactive question, collect the real-time facial data of the user and the digital human when interacting, extract the coordinates of each key point in the real-time user facial data, and calculate the geometric distance between every two key points in the real-time user facial data. , Indicates the 1st, 2nd, 3rd, ..., The geometric distance between the key points is compared with the key point geometric distance A' of the real-time user and the key point geometric distance set A in the mapped virtual image of the real-time user; S602, when A=A', extract the user emotions corresponding to the A set as the interactive emotions of the real-time user, add the real-time user interactive emotion modifier to the selected best answer using the natural language processing algorithm, and finally output it to the real-time user.

[0035] Analyzing the expressions, postures and other elements of the virtual image can help the digital human better understand the user's emotional state. In this way, the digital human can make corresponding emotional responses and establish a deeper emotional connection with the user, allowing the user to feel more understanding and resonance in the interaction, thus improving the quality and depth of the interaction.

[0036] A digital human platform interaction system based on artificial intelligence, the digital human platform interaction system includes a data collection module, a virtual user image construction module, a digital human construction module, a real-time interaction module, and an output module; The data collection module is used to collect data on the user's interaction with the digital human in the historical records; The virtual user image construction module is used to extract the facial features of the user when in different emotions based on the facial data of the user during interaction in the collected historical records, obtain the user's preference for the interaction results using the feedback data, and use the digital twin technology to combine the facial features and preferences of the user when in different emotions to construct a virtual user image; The digital human construction module is used to extract the appearance features of the virtual images browsed in the collected historical records to construct a digital human image unique to each user, extract the voice features of the audio data, construct a digital human voice unique to each user, and construct a digital human unique to each user; The real-time interaction module is used to collect the real-time user's facial data to find and map the virtual image, then select the digital human answer according to the user's preference, and judge the real-time user's emotions according to the real-time user's facial data; The output module is used to add modifiers to the selected best answer according to the real-time user's emotions and then output it to the real-time user.

[0037] The virtual user image building module includes emotion unit and preference unit; The emotion unit is used to extract the facial features of the user when they are in different emotions based on the facial data of the user when they interact in the collected historical records; The preference unit is used to calculate the preference of each user according to the collected user feedback data and result characteristics.

[0038] The real-time interaction module includes a mapping avatar unit, an answer selection unit, and an emotion judgment unit; The mapping virtual image unit is used to collect facial data of the real-time user, calculate the similarity between the real-time user and the virtual user image, and select the mapping virtual image of the real-time user; The answer selection unit is used to collect real-time user facial data, search for mapped virtual images, and then select digital human answers according to user preferences; The emotion judgment unit is used to judge the real-time user emotion according to the real-time user's facial data.

[0039] Embodiment: Now a digital human platform is constructed, and three users are counted, and a virtual user image is constructed based on the historical records of the three users' interactions; Assume that there are three users, namely user 1, user 2, and user 3; facial features include the corners of the eyes, the corners of the mouth, and the tail of the eyebrows; and the types of emotions include joy and depression; The facial features of different emotions in the virtual user images of the three users are obtained as follows: User 1 - joy (10, 5, 13), depression (8, 5, 9); the virtual user images of User 2 and User 3 are obtained in turn; According to the characteristics of the interaction results in the history, it is found that user 1 prefers simplicity, user 2 prefers details, and user 3 prefers gorgeousness; Based on the browsing images and audio data of each user in history, digital characters unique to three users are constructed: User 1 (child, immature), User 2 (animal, stable), User 3 (cartoon girl, cheerful); The real-time user's facial data is collected to calculate the similarity, and the real-time user is judged to be user 2. The digital person is selected as an animal and the voice is steady. According to the real-time user's interactive question "Introduce Dream of Red Mansions", the answers are classified into concise type, detailed introduction type and ornate type. Select the detailed introduction category according to the mapped virtual image; collect the real-time user's facial data, obtain the real-time emotion as joy, and use the natural language processing algorithm to add the modifier of joy to the answer, for example: "It's a pleasure to serve you, etc.", and finally output it to the user.

[0040] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.

Claims

1. A digital human platform interaction method based on artificial intelligence, characterized by: The method comprises the following steps: S100, collect statistics on all users who interact with the digital human, collect different emotions and corresponding facial data of each user when interacting with the digital human in the historical records, extract the facial features of the user when they are in different emotions, collect the feedback data of the user on the interaction results when each user interacts with the digital human in the historical records, and use the feedback data to obtain the user's preference for the interaction results; use the digital twin technology to combine the facial features and preferences of the user when they are in different emotions to construct a virtual user image; S200, for each user's virtual user image constructed, collect the virtual images browsed by each user on the network, extract the appearance features of the browsed virtual images to construct a digital human image unique to each user; collect the audio data listened to by each user on the network, extract the voice features of the audio data, and construct a digital human voice unique to each user; construct a digital human unique to each user with the generated digital human image and voice of each user, and save the digital human of each user to construct a digital human warehouse; S300, when the user interacts with the digital human, the user's facial data is collected in real time and matched with the virtual user image, and after matching, a digital human unique to the real-time user is selected according to the virtual user image and displayed; S400: After selecting the digital human of the real-time user, voice data of the user interacting with the digital human is collected, and the user voice data is analyzed using a natural language processing algorithm to generate interactive questions that the real-time user asks the digital human; the digital human collects answers on the network according to the interactive questions and classifies all the collected answers; S500, extracting preferences from the user's virtual image to judge each of the classified answers, and selecting an answer to the interactive question that satisfies the user; S600: After the answer to the interactive question is selected, the real-time facial data of the user and the digital human when interacting is collected, and the emotions of the real-time user are judged in the user's virtual image. The digital human modifies the answer to the interactive question according to the user's emotions and outputs it to the real-time user.

2. The method for digital human platform interaction based on artificial intelligence according to claim 1, characterized in that: The specific steps of using the digital twin technology in S100 to combine the facial features and preferences of the user in different emotions to construct a virtual user image are: S101. Count all users who interact with the digital human to obtain a user number of n, and collect the emotion types of each user when interacting with the digital human in the historical records. , It represents the 1st, 2nd, 3rd, ..., nth emotions of each user in the collected historical records when interacting with the digital human, where n is a positive integer; extract the facial data of the user when interacting with the digital human when the user has different emotions, and use the Dlib library to find the key points in the facial data: , Indicates the 1st, 2nd, 3rd, ..., mth key points in the facial data to be searched, where m is a positive integer; S102: Establish a coordinate system in the user's facial data and extract the coordinate points of each key point. , Represent the coordinates of the 1st, 2nd, 3rd, ..., mth key points in the facial data, and calculate the geometric distance between every two key points. The formula is: ； In the formula, Represents the geometric distance between the i-th and j-th key points among the m key points, and represents the horizontal and vertical coordinates of the i-th key point, and Represents the horizontal and vertical coordinates of the jth key point; i and j are from 1 to m; the geometric distance of each key point under different user emotions is calculated as follows: , Indicates the 1st, 2nd, 3rd, ..., For the geometric distance of key points, the geometric distance set A of key points of the user with different emotions is used as the facial feature of each emotion; S103, collecting the feedback data of each user on the interaction result when the user interacts with the digital human in the historical records, and correspondingly extracting the result features of the digital human when the user interacts with the digital human in the historical records; The collected user feedback data is matched with the corresponding digital human result features to obtain the constituent elements combined as , represents the 1st, 2nd, 3rd, ..., kth element combination obtained after matching, where F represents the feedback data, J represents the result feature of the digital person; k is a positive integer; the number of times for each element combination is collected and the average value is calculated, and the element combination with a number greater than the average value is defined as a frequent group; the confidence of all frequent groups is calculated, and the formula is: ； In the formula, represents the confidence of the calculated t-th frequent group, represents the number of times the tth frequent group is collected, Indicates the number of feedback data collected in the tth frequent group; The confidence of all frequent groups is calculated, and the result feature of the digital human in the frequent group with the largest confidence is selected as the user's preference after judgment, and the calculation is repeated to obtain the preferences of all users; Digital twin technology is used to combine the user's facial features and preferences when in different emotions to construct a virtual user image.

3. The method for digital human platform interaction based on artificial intelligence according to claim 2, characterized in that: The specific steps of saving each user's digital human and building a digital human warehouse in S200 are: S201, for each user's virtual user image constructed, collect the virtual images browsed by each user on the network, use the face detection algorithm to extract the facial key points of the collected virtual images, standardize the data value of each facial key point, unify them into the same data format, and calculate the performance value of each facial key point, the formula is: ； In the formula, B represents the performance value of each key point, R represents the total number of differences between every two data values ​​in each key point, and H and H' represent the different data values ​​in each key point. After calculating the performance value of each key point, the image segmentation technology is used to segment each key point in the facial data to form a separate appearance feature. The number of views of each segmented appearance feature by the user in the historical records is extracted respectively, and a probability model is constructed, which is expressed as: ; In the model represents the probability of the vth appearance feature, represents the number of views of the vth appearance feature, Indicates the total number of views of all appearance features by users; The deep learning model is used to train the probabilities of all appearance features, and all appearance features with a probability greater than one-half are selected as the user's favorite appearance; finally, the image generation technology is used to combine all the user's favorite appearance features to generate the user's favorite digital human image; S202, collecting audio data listened to by each user on the network, extracting timbre features from the collected audio data using an audio processing algorithm, calculating the probability of each timbre feature using the same probability model as in S201, and selecting the timbre feature with the highest probability as the user's favorite audio feature; repeating the operation to calculate the favorite audio feature of each user; S203. After calculating the digital human image and audio features that each user likes, use an adversarial network algorithm to build a digital human that belongs to each user according to the digital human image and audio features that each user likes, and save the digital human of each user to build a digital human warehouse.

4. The method for digital human platform interaction based on artificial intelligence according to claim 3, characterized in that: The specific steps of selecting and displaying a digital person unique to the real-time user according to the virtual user image after matching in S300 are: S301, when the user interacts with the digital human, collect the user's facial data in real time, extract the key points in the real-time user's facial data, collect the data value of each key point accordingly, and extract the data value of each key point in the virtual user image; calculate the facial similarity of the real-time user and all virtual user images, the formula is: ； In the formula, represents the facial similarity between the real-time user and each virtual user image, E represents the number of key points in the extracted real-time user facial data, Represents the data value of the pth key point of the real-time user, Represents the data value of the pth key point in the virtual user image; S302, after calculating the facial similarity between the real-time user and each virtual user image, select the virtual user image with the facial similarity closest to 1 and match it with the real-time user to obtain the mapped virtual image of the real-time user; and select a digital human unique to the real-time user in the digital human warehouse for display according to the mapped virtual image of the real-time user.

5. The method for digital human platform interaction based on artificial intelligence according to claim 4, characterized in that: The specific steps of the digital human collecting answers on the network according to the interactive questions and classifying all the collected answers in S400 are: S401, after selecting the digital human of the real-time user, collecting the voice data of the user interacting with the digital human, analyzing the user voice data using a natural language processing algorithm, and generating interactive questions that the real-time user asks the digital human; S402: The digital human collects answers on the network according to the interactive questions and then classifies all the collected answers according to the interactive result features in the historical records of S100. , Represents the 1st, 2nd, 3rd, ..., bth answers obtained after classification, where b is a positive integer.

6. The method for digital human platform interaction based on artificial intelligence according to claim 5, characterized in that: The specific steps of selecting the answer to the interactive question that satisfies the user in S500 are: S501, extracting the preferences of the user's virtual image to judge each answer after classification, and selecting the answer with the same result characteristics as the user's preference as the optimal answer.

7. The method for digital human platform interaction based on artificial intelligence according to claim 6, characterized in that: The specific steps of the digital human modifying the answer to the interactive question according to the user's emotions and then outputting it to the real-time user in S600 are: S601, after selecting the answer to the interactive question, collect the real-time facial data of the user and the digital human when interacting, extract the coordinates of each key point in the real-time user facial data, and calculate the geometric distance between every two key points in the real-time user facial data. , Indicates the 1st, 2nd, 3rd, ..., The geometric distance between the key points is compared with the key point geometric distance A' of the real-time user and the key point geometric distance set A in the mapped virtual image of the real-time user; S602, when A=A', extract the user emotions corresponding to the A set as the interactive emotions of the real-time user, add the real-time user interactive emotion modifier to the selected best answer using the natural language processing algorithm, and finally output it to the real-time user.

8. A digital human platform interaction system based on artificial intelligence, characterized by: The digital human platform interaction system includes a data collection module, a virtual user image construction module, a digital human construction module, a real-time interaction module, and an output module; The data collection module is used to collect data on the user's interaction with the digital human in the historical records; The virtual user image construction module is used to extract the facial features of the user when in different emotions based on the facial data of the user during interaction in the collected historical records, obtain the user's preference for the interaction results using the feedback data, and use the digital twin technology to combine the facial features and preferences of the user when in different emotions to construct a virtual user image; The digital human construction module is used to extract the appearance features of the virtual images browsed in the collected historical records to construct a digital human image unique to each user, extract the voice features of the audio data, construct a digital human voice unique to each user, and construct a digital human unique to each user; The real-time interaction module is used to collect the real-time user's facial data to find and map the virtual image, then select the digital human answer according to the user's preference, and judge the real-time user's emotions according to the real-time user's facial data; The output module is used to add modifiers to the selected best answer according to the real-time user's emotions and then output it to the real-time user.

9. The digital human platform interaction system based on artificial intelligence according to claim 8, characterized in that: The virtual user image building module includes an emotion unit and a preference unit; The emotion unit is used to extract the facial features of the user when they are in different emotions based on the facial data of the user when they interact in the collected historical records; The preference unit is used to calculate the preference of each user according to the collected user feedback data and result characteristics.

10. The digital human platform interaction system based on artificial intelligence according to claim 8, characterized in that: The real-time interaction module includes a mapping virtual image unit, an answer selection unit and an emotion judgment unit; The mapping virtual image unit is used to collect facial data of the real-time user, calculate the similarity between the real-time user and the virtual user image, and select the mapping virtual image of the real-time user; The answer selection unit is used to collect real-time user facial data, search for mapped virtual images, and then select digital human answers according to user preferences; The emotion judgment unit is used to judge the real-time user emotion according to the real-time user's facial data.

Images