Audience intelligent interactive experience system based on AI

Abstract

The invention relates to the technical field of intelligent interaction, and particularly discloses an audience intelligent interaction experience system based on AI, and the system comprises a collection module which is used for collecting the image information of an audience in a current time interval in real time, carrying out the preprocessing of the image information, and obtaining a key frame; the AI identification interface is used for identifying basic information and expression information of audiences according to the key frames; the attitude analysis end is used for performing attitude analysis according to the basic information of the audience and the key frame to obtain the attitude accuracy of the audience; the analysis processing center is used for obtaining a corresponding judgment model according to the basic information of the audience, evaluating the audience based on the judgment model, the expression information and the posture accuracy, and obtaining an audience evaluation value; and the content output adjusting end is used for adjusting the interactive content according to the audience evaluation value. According to the invention, the feedback timeliness of the AI to the video content can be improved, and the accuracy of interactive content interaction can be improved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent interactive technologies, and specifically to an intelligent interactive experience system for audiences based on AI. Background Art

[0002] With the rapid development of technologies such as artificial intelligence, natural language processing, and computer vision, it is possible to complete natural, efficient, and intelligent information exchange and operations between humans and machines through intelligent devices, enabling machines to understand human intentions and provide feedback in a way familiar to humans (such as voice, gestures, expressions), thereby enhancing the user experience and interaction efficiency.

[0003] In existing intelligent interactive systems, by obtaining the image information of users in real time, and then analyzing the image information based on artificial intelligence technology, and then judging the state of users, and feeding back corresponding content according to the judgment results, thereby realizing the interaction process between users and machines. Taking practicing dance and aerobics as examples, by collecting and analyzing the audience data, and then comparing the practice state of the audience with the standard state, it is convenient to adjust the focus and progress of teaching in real time to adapt to different users.

[0004] In the specific operation of existing intelligent interactive systems, it mainly performs real-time recognition of the voices of users and feeds back corresponding content according to the voices. For the process of video processing and analysis, since the speed of AI in processing videos is relatively slow, there is a high lag in the feedback process, and it is difficult to achieve the effect of real-time interaction. At the same time, although various tasks can be completed through the AI method, it is difficult to judge the practice state of the audience under a unified standard. Therefore, the AI is only used as an auxiliary judgment basis. Therefore, how to improve the timeliness of the feedback of AI on video content and improve the accuracy of interactive content is the fundamental problem to be solved by the present invention. Summary of the Invention

[0005] The purpose of the present invention is to provide an intelligent interactive experience system for audiences based on AI, and solve the following technical problems:

[0006] How to improve the timeliness of the feedback of AI on video content and improve the accuracy of interactive content.

[0007] The purpose of the present invention can be achieved through the following technical solutions:

[0008] An intelligent interactive experience system for audiences based on AI, the system includes:

[0009] A collection module, used to collect the image information of the audience in the current time interval in real time, preprocess the image information, and obtain key frames;

[0010] AI recognition interface, which is used to identify the basic information and expression information of the audience according to the key frames;

[0011] Posture analysis terminal, which is used to perform posture analysis based on the basic information of the audience and the key frames to obtain the posture accuracy of the audience;

[0012] Analysis and processing center, which is used to obtain the corresponding judgment model according to the basic information of the audience, and evaluate the audience based on the judgment model, expression information and posture accuracy to obtain the audience evaluation value;

[0013] Content output adjustment terminal, which is used to adjust the interactive content according to the audience evaluation value.

[0014] Through the above technical solution, the method of using key frames for AI image recognition processing can greatly reduce the time required for AI to process videos, thereby reducing the lag of the interactive experience and improving the user experience. Through the collaborative process of AI technology and the local model, the local model can ensure a unified standard when judging the state of the audience, while AI can reduce the judgment error compared with the local model when judging the user's expression information, and thus the combination of the two improves the accuracy of the interaction.

[0015] Furthermore, obtain the basic body shape framework of the audience in the basic information, calculate the deviation coefficient between the basic body shape framework and the preset model in the system, and select the preset model with the smallest deviation coefficient as the comparison model;

[0016] Identify the posture framework of the audience in the key frames, compare the posture framework of the audience with the posture of the comparison model at the corresponding time point of the key frames, and determine the posture accuracy according to the result of the coincidence comparison and the deviation coefficient.

[0017] Through the above technical solution, the deviation state between the motion posture of the audience and the standard motion posture can be accurately obtained, and at the same time, it is evaluated through the posture accuracy. Furthermore, the subsequent content output can be adjusted according to the posture accuracy, improving the accuracy of the interaction.

[0018] Furthermore, the process of calculating the deviation coefficient includes:

[0019] Through the formula Calculate to obtain the deviation coefficient v;

[0020] Where n is the number of limbs of the basic body shape framework, i = 1, 2,..., n; x i is the first influence coefficient of the i-th limb, l i is the length value of the i-th limb, lt i is the length value of the i-th limb corresponding to the preset model, f i is the error reference quantity function of the i-th limb.

[0021] Through the above technical solution, it is possible to judge the deviation between the basic body shape framework of the audience and the preset model in the system by obtaining the deviation coefficient v. On the one hand, the subsequent comparison model determination process can be realized according to the magnitude of the deviation coefficient v, and at the same time, the influence caused by the deviation between the basic body shape framework of the audience and the comparison model can be reduced, and the deviation state between the movement posture of the audience and the standard movement posture can be accurately obtained, improving the accuracy of interaction.

[0022] Furthermore, the process of obtaining the posture accuracy includes:

[0023] By the formula calculate to obtain the posture deviation degree Op;

[0024] By the formula R = (1 + Op - v * k) -1 calculate to obtain the posture accuracy R;

[0025] Wherein, is the vector of the i-th limb, is the vector of the comparison model of the i-th limb, is and the included angle of, θt i is the reference quantity of the error included angle of the i-th limb, is the vector of the upper limb of the i-th limb, is the vector of the comparison model of the upper limb of the i-th limb, is and the included angle of, when the i-th limb has no upper limb, θut i is the reference quantity of the error included angle of the upper limb of the i-th limb, γ is the correction coefficient, y i is the second influence coefficient of the i-th limb, k is the adjustment coefficient.

[0026] Through the calculation process of the posture accuracy R in the above technical solution, the accuracy of the user's posture can be accurately judged, and then the subsequent content output can be adjusted according to the posture accuracy, improving the accuracy of interaction.

[0027] Furthermore, the process of the preprocessing includes:

[0028] Perform scene editing detection on the image information to obtain the first key frame;

[0029] Obtain the second key frame at a preset time interval;

[0030] Combine the first key frame and the second key frame to obtain the key frame.

[0031] Through the above technical solution, it is possible to ensure that the content of the video can be comprehensively extracted while avoiding the problem of important scenes being omitted, and thus it is possible to ensure that the key frames accurately reflect the situation of the audience in the entire video, greatly reducing the time required for AI to process the video, reducing the lag of the interactive experience, and improving the user experience.

[0032] Furthermore, the process of obtaining the audience evaluation value includes:

[0033] Assign values to the preset expressions according to the emotional tendencies corresponding to the expressions;

[0034] Establish an audience emotional tendency curve based on the values assigned to the expression information corresponding to the key frames;

[0035] Evaluate the audience according to the audience emotional tendency curve and the posture accuracy to obtain the audience evaluation value.

[0036] Through the above technical solution, it is possible to evaluate the audience according to the audience emotional tendency curve and the posture accuracy to obtain the audience evaluation value, and then to make a comprehensive judgment based on the current psychological state of the audience (audience emotional tendency curve) and the current learning state of the audience (posture accuracy), obtain the audience evaluation value, realize the adjustment of the subsequent content output, and improve the accuracy of the interaction.

[0037] Furthermore, the process of obtaining the audience evaluation value also includes:

[0038] Obtain the audience evaluation value P through the formula

[0039] where m is the number of key frames, j = 1, 2,..., m; R j is the posture accuracy of the j-th key frame, e(t) is the audience emotional tendency curve, t0 is the starting time point of the current time interval, t1 is the ending time point of the current time interval, β is the correction coefficient, and U is the judgment model corresponding to the current audience.

[0040] Through the calculation process of the evaluation value in the above technical solution, the adjustment of the subsequent content output is realized, and the accuracy of the interaction is improved.

[0041] Furthermore, the process of the content output adjustment end adjusting the interaction content according to the audience evaluation value includes:

[0042] Compare the audience evaluation value with the preset evaluation interval, and use the content corresponding to the preset evaluation interval where the audience evaluation value is located as the adjusted interaction content.

[0043] Through the above technical solution, it can ensure that the audience quickly and scientifically obtains the correct interactive feedback content.

[0044] ​Advantages of the present invention:

[0045] (1) By using the method of AI image recognition processing through key frames, the present invention can greatly reduce the time required for AI to process videos, thereby reducing the lag of the interactive experience and improving the user experience. Through the collaborative process of AI technology and the localization model, the localization model can ensure a unified standard when judging the state of the audience, while AI can reduce the judgment error when judging the user's expression information. Therefore, the combination of the two improves the accuracy of the interaction.

[0046] (2) The present invention can accurately obtain the deviation state between the movement posture of the audience and the standard movement posture, and at the same time evaluate through the posture accuracy, so as to adjust the subsequent content output according to the posture accuracy and improve the accuracy of the interaction.

[0047] (3) Through the process of obtaining key frames, the present invention can ensure that the content of the video can be comprehensively extracted while avoiding the problem of important scenes being missed. Therefore, it can ensure that the key frames accurately reflect the situation of the audience in the entire video, greatly reducing the time required for AI to process the video, reducing the lag of the interactive experience, and improving the user experience.

[0048] (4) It can comprehensively judge according to the current psychological state of the audience (audience emotion tendency curve) and the current learning state of the audience (posture accuracy), obtain the audience evaluation value, and realize the adjustment of the subsequent content output, so as to improve the accuracy of the interaction. Description of the Drawings

[0049] The present invention will be further described below with reference to the accompanying drawings.

[0050] Figure 1 is the logic block diagram of the intelligent interactive experience system for the audience based on AI of the present invention. Detailed Embodiments

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0052] Please refer to Figure 1As shown, in one embodiment, an AI-based intelligent interactive experience system for audiences is provided. The system includes a collection module, an AI recognition interface, a posture analysis terminal, an analysis and processing center, and a content output adjustment terminal. Among them, the collection module realizes its functions through a camera and a processor. The camera is used to collect the image information of the audience in the current time interval in real time. During operation, the full-body image of the audience needs to be collected. The processor is used to preprocess the image information to obtain key frames, and the key frames reflect the overall state of the audience in the current time interval. Therefore, by using the method of AI image recognition processing based on key frames, the time required for AI to process videos can be greatly reduced, thereby reducing the lag of the interactive experience and improving the user experience. It should be noted here that although this embodiment still analyzes a previous time interval, the time interval is short and can be completed during the interval of teaching. In addition, the AI recognition interface is used to identify the basic information and expression information of the audience based on the key frames. Among them, the basic information is the age, gender, and posture framework of the user, and different AI models are used to identify the basic information and expression information, which are not limited here. The posture analysis terminal is used to perform posture analysis based on the basic information of the audience and the key frames to obtain the posture accuracy of the audience. This process is realized through a localized model, so the running speed is controllable. Finally, the analysis and processing center obtains the corresponding judgment model according to the basic information of the audience. The judgment model is applicable to audiences of different age intervals and genders, so it can adjust the corresponding standards according to the different adaptabilities of the audience. Based on the judgment model, expression information, and posture accuracy, the audience is evaluated to obtain the audience evaluation value. Finally, the content output adjustment terminal adjusts the interactive content according to the audience evaluation value. During this process, through the collaborative process of AI technology and the localized model, the duration of the video analysis and processing process can be greatly reduced, thereby improving the timeliness of AI's feedback on video content. At the same time, the localized model can ensure a unified standard when judging the state of the audience, and AI can reduce the judgment error compared with the localized model when judging the user's expression information. Therefore, the combination of the two improves the accuracy of the interaction.

[0053] In one embodiment, a process for a pose analysis terminal to perform pose analysis on key frames is provided, including: obtaining the basic body shape framework of the audience in the basic information, which is implemented through an image recognition model. It should be noted that the training of the image recognition model is prior art and will not be elaborated here; then obtaining the basic body shape framework of the audience in the basic information, calculating the deviation coefficient between the basic body shape framework and the preset model in the system, and selecting the preset model with the smallest deviation coefficient as the comparison model; identifying the pose framework of the audience in the key frame, comparing the pose framework of the audience with the pose of the comparison model at the corresponding time point of the key frame, and determining the pose accuracy according to the coincidence comparison result and the deviation coefficient. In the above process, the system will preset multiple models according to the body shape status of the general population, and select a comparison model similar to the basic body shape framework of the audience for comparison, which can better judge the movement state of the audience. The acquisition of the deviation coefficient can reduce the influence caused by the deviation between the basic body shape framework of the audience and the comparison model in the subsequent comparison process; therefore, through the above process, the deviation state between the movement pose of the audience and the standard movement pose can be accurately obtained, and at the same time, it can be evaluated through the pose accuracy, and then the subsequent content output can be adjusted according to the pose accuracy to improve the accuracy of interaction.

[0054] In one embodiment, a process for calculating the deviation coefficient is given, including: calculating the deviation coefficient v through the formula where n is the number of limbs of the basic body shape framework, and i = 1, 2,..., n; usually in the simplified basic body shape framework, n is selected as 10, including 6 positions of the torso, thighs, calves, upper arms, forearms, and head. x i is the first influence coefficient of the i-th limb, which is adaptively set according to the type and requirements of the interaction content. For example, when higher requirements are placed on the arms, the coefficients corresponding to the upper arms and forearms can be increased, and there is no limit here. l i is the length value of the i-th limb, lt i is the length value of the i-th limb corresponding to the preset model, f i is the error reference quantity function of the i-th limb, and this error reference quantity function is a look-up table, which determines the corresponding reference error value according to multiple test data for different limb types. Therefore, the corresponding reference error value is determined according to the interval where the length value of the i-th limb corresponding to the comparison model is located; through the above calculation process, the deviation situation between the basic body shape framework of the audience and the preset model in the system can be judged through the obtained deviation coefficient v. On the one hand, the subsequent determination process of the comparison model can be realized according to the size of the deviation coefficient v, and at the same time, the influence caused by the deviation between the basic body shape framework of the audience and the comparison model can be reduced, and the deviation state between the movement pose of the audience and the standard movement pose can be accurately obtained, improving the accuracy of interaction.

[0055] In one embodiment, a process for obtaining attitude accuracy is provided, including: through the formula calculate to obtain the attitude deviation Op; then through the formula R = (1 + Op - v * k) -1 calculate to obtain the attitude accuracy R; where is the vector of the i-th limb, is the vector of the comparison model of the i-th limb, is and the included angle between, θt i is the reference quantity of the error included angle of the i-th limb, and this parameter is obtained by fitting and selecting according to test data, is the vector of the upper limb of the i-th limb, is the vector of the comparison model of the upper limb of the i-th limb, is and the included angle between, θut i is the reference quantity of the error included angle of the upper limb of the i-th limb. It should be noted that there are only two types of upper limbs, namely the thigh and the upper arm. That is, only the calf and the forearm will consider the upper limb. When the i-th limb has no upper limb, such as the torso, thigh and upper arm, then γ is a correction coefficient, which is used to adjust the influence degree of the upper limb on the lower limb. It is obtained by fitting according to test data, y i is the second influence coefficient of the i-th limb. This coefficient is also adaptively set according to the type and requirements of the interaction content, and is used to adjust the influence degree of different limbs on the attitude accuracy. k is an adjustment coefficient. This parameter is used to adjust the influence degree of the deviation between the basic body shape frame of the audience and the preset model in the system on the overall attitude accuracy. This parameter is obtained by fitting according to the comparison results of the attitude accuracy obtained under different deviation coefficients and the manual judgment results in multiple groups of test data. Therefore, through the above calculation process, the user's attitude accuracy can be accurately judged, and then the subsequent content output can be adjusted according to the attitude accuracy, improving the accuracy of the interaction.

[0056] In one embodiment, the preprocessing process includes: performing scene editing detection on the image information to obtain a first key frame. It should be noted that the scene editing detection is implemented based on existing video processing technologies, which will mark when the scene of the video changes, and use the image corresponding to the marked time point as the key frame. Then, a second key frame is obtained at a preset time interval; the first key frame and the second key frame are combined to obtain the key frame. Through the above process, it is possible to ensure that the content of the video can be comprehensively extracted while avoiding the problem of important scenes being missed, and thus ensure that the key frame can accurately reflect the situation of the audience in the entire video, greatly reducing the time required for AI to process the video, reducing the lag of the interactive experience, and improving the user experience.

[0057] In one embodiment, the process of obtaining the audience evaluation value includes: assigning values to the preset expressions according to the emotional tendencies corresponding to the expressions. Among the emotional tendencies in the above solution, the positive types include happy, relaxed, etc.; the neutral type includes calm; the negative types include pain, anxiety, anger, etc. The neutral type is set to zero, the positive types are set to positive values of different magnitudes according to the expression types, and the negative types are set to negative values of different magnitudes according to the expression types. Therefore, it is possible to establish an audience emotional tendency curve based on the values assigned to the expression information corresponding to the key frames; evaluate the audience according to the audience emotional tendency curve and the pose accuracy to obtain the audience evaluation value, and then be able to comprehensively judge according to the current mental state of the audience (audience emotional tendency curve) and the current learning state of the audience (pose accuracy) to obtain the audience evaluation value and realize the adjustment of the subsequent content output, improving the accuracy of the interaction.

[0058] In one embodiment, the process of obtaining the audience evaluation value further includes: through the formula calculate to obtain the audience evaluation value P; where m is the number of key frames, j = 1, 2,..., m; R j is the pose accuracy of the j-th key frame, e(t) is the audience emotional tendency curve, t0 is the starting time point of the current time interval, t1 is the ending time point of the current time interval, β is a correction coefficient, which is obtained by fitting according to the test data and is used to adjust the influence weight of the audience emotional tendency curve, U is the judgment model corresponding to the current audience, which is obtained according to the intervals of the user's age and gender, and is obtained for the age group and corresponding gender of the audience is adjusted to obtain the final evaluation value, and the subsequent content output is adjusted according to the evaluation value, improving the accuracy of the interaction.

[0059] It should be noted that the preset judgment model determines the corresponding influence weights in advance according to the empirical data of different age intervals and genders. Therefore, Input it into the corresponding judgment model to obtain the audience evaluation value.

[0060] In addition, the process of the content output adjustment end adjusting the interactive content according to the audience evaluation value includes: comparing the audience evaluation value with a preset evaluation range, and using the content corresponding to the preset evaluation range where the audience evaluation value is located as the adjusted interactive content. The content corresponding to different evaluation ranges is obtained according to the teaching content and the setting of professionals. Therefore, through the process of obtaining the evaluation value and the comparison process, it can be ensured that the audience can quickly and scientifically obtain the correct interactive feedback content.

[0061] The above has described an embodiment of the present invention in detail, but the described content is only a preferred embodiment of the present invention and cannot be considered as limiting the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the application of the present invention should still fall within the scope covered by the patent of the present invention.

Claims

1. An AI-based intelligent interactive experience system for audiences, characterized in that, The system includes: A collection module, which is used to collect the image information of the audience in the current time interval in real time, preprocess the image information, and obtain key frames; An AI recognition interface, which is used to recognize the basic information and expression information of the audience according to the key frames; A posture analysis terminal, which is used to perform posture analysis according to the basic information of the audience and the key frames to obtain the posture accuracy of the audience; An analysis and processing center, which is used to obtain the corresponding judgment model according to the basic information of the audience, and evaluate the audience based on the judgment model, expression information and posture accuracy to obtain the audience evaluation value; A content output adjustment terminal, which is used to adjust the interactive content according to the audience evaluation value.

2. An AI-based intelligent interactive experience system for audiences according to claim 1, wherein The process of the posture analysis terminal performing posture analysis on the key frames includes: Obtain the basic body shape framework of the audience in the basic information, calculate the deviation coefficient between the basic body shape framework and the preset model in the system, and select the preset model with the smallest deviation coefficient as the comparison model; Identify the posture framework of the audience in the key frame, compare the posture framework of the audience with the posture of the comparison model at the corresponding time point of the key frame for coincidence comparison, and determine the posture accuracy according to the coincidence comparison result and the deviation coefficient.

3. An AI-based intelligent interactive experience system for audiences according to claim 2, characterized in that, The process of calculating the deviation coefficient includes: Obtained through the formula The deviation coefficient v is calculated; where n is the number of limbs of the basic body shape framework, and i = 1, 2, …, n; x i is the first influence coefficient of the i-th limb, l i is the length value of the i-th limb, lt i is the length value of the i-th limb corresponding to the preset model, f i is the error reference quantity function of the i-th limb.

4. An AI-based intelligent interactive experience system for audiences according to claim 3, characterized in that, The process of obtaining the posture accuracy includes: The attitude deviation Op is obtained by calculating through the formula ​ The attitude accuracy R is obtained by calculating with the formula R = (1 + Op - v * k) -1 where the attitude accuracy R is calculated Among them, is the vector of the i-th limb, is the vector of the i-th limb comparison model, is the included angle between and i is the reference quantity of the error included angle of the i-th limb, is the vector of the upper limb of the i-th limb, is the vector of the upper limb comparison model of the i-th limb, is the included angle between and θut i is the reference quantity of the error included angle of the upper limb of the i-th limb, γ is the correction coefficient, y i is the second influence coefficient of the i-th limb, k is the adjustment coefficient.

5. An AI-based intelligent interactive experience system for audiences according to claim 4, characterized in that, The process of the preprocessing includes: Perform scene editing detection on the image information to obtain the first key frame; Obtain the second key frame at a preset time interval; Combine the first key frame and the second key frame to obtain the key frame.

6. An AI-based intelligent interactive experience system for audiences according to claim 5, characterized in that, The process of obtaining the audience evaluation value includes: Assign values to the preset expressions according to the emotional tendencies corresponding to the expressions; Establish an audience emotional tendency curve according to the values assigned to the expression information corresponding to the key frames; Evaluate the audience according to the audience emotional tendency curve and the posture accuracy to obtain the audience evaluation value.

7. An AI-based intelligent interactive experience system for audiences according to claim 6, characterized in that, The process of obtaining the audience evaluation value further includes: Obtained through the formula Calculate the audience evaluation value P; where m is the number of key frames, j = 1, 2, …, m; R j is the pose accuracy of the j-th key frame, e(t) is the audience emotion tendency curve, t0 is the start time point of the current time interval, t1 is the end time point of the current time interval, β is the correction coefficient, and U is the judgment model corresponding to the current audience.

8. An AI-based intelligent interactive experience system for audiences according to claim 7, characterized in that, The process of the content output adjustment terminal adjusting the interactive content according to the audience evaluation value includes: Compare the audience evaluation value with the preset evaluation interval, and use the content corresponding to the preset evaluation interval where the audience evaluation value is located as the adjusted interactive content.

Images