Satisfaction survey method and system based on video recognition

Through the time marking and behavioral characteristics analysis of video information, automated and real-time evaluation of student satisfaction in video teaching is achieved, which solves the problem that teachers find it difficult to judge students' satisfaction, and improves teaching quality control.

CN118485945BActive Publication Date: 2024-11-08FERMAT (SHENZHEN) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410677672.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-29
Publication Date
2024-11-08
Estimated Expiration
2044-05-29

AI Technical Summary

Technical Problem

In video teaching, it is difficult for teachers to observe students' reactions, thus unable to accurately judge students' satisfaction with the teaching content, and thus difficult to control the teaching quality.

Method used

By obtaining video information of the subject user and the user to be identified for time stamping, analyzing the display content of the subject user and the behavior characteristics of the user to be identified, generating a display content sequence and a behavior evaluation sequence, and evaluating it with time weights to achieve user satisfaction Automated, real-time evaluation.

Benefits of technology

It improves the accuracy and timeliness of the evaluation results, provides more objective and reliable satisfaction feedback, solves the problem of difficulty in controlling students' satisfaction in video teaching, and improves data acquisition and processing efficiency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118485945B_ABST
    Figure CN118485945B_ABST
Patent Text Reader

Abstract

The present invention relates to the technical field of video recognition, and discloses a satisfaction survey method and system based on video recognition. The present invention obtains video information of a main user and a user to be recognized and performs time marking, analyzes the video of the main user to generate a display content sequence and a behavior evaluation sequence, analyzes the video of the user to be recognized to generate a behavior feature sequence, substitutes the behavior feature sequence into the behavior evaluation sequence, evaluates the user behavior features to obtain a basic evaluation group, assigns a time weight to the behavior feature sequence according to a time marking axis, and performs parameter correction on the basic evaluation group to obtain a corrected evaluation group. The method realizes automatic real-time evaluation of user satisfaction through video analysis, uses time marking and weight correction to ensure the accuracy and timeliness of evaluation results, improves the efficiency of data acquisition and processing, provides more objective and reliable satisfaction feedback, and solves the problem in the prior art that it is difficult to control students' satisfaction with teaching quality in video teaching.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of video recognition, and in particular to a satisfaction survey method and system based on video recognition. Background Art

[0002] With the development of network information technology, video calls have been maturely applied to various fields, such as online teaching in the field of education.

[0003] In traditional classrooms, teachers can see at a glance how each student reacts in class, and thus control the quality of teaching. However, in video teaching, due to the size of the screen, teachers cannot observe students’ reactions, making it difficult to judge students’ satisfaction with the teaching content, and thus difficult to control the quality of teaching. Summary of the invention

[0004] The purpose of the present invention is to provide a satisfaction survey method and system based on video recognition, aiming to solve the problem in the prior art that it is difficult to control students' satisfaction with teaching quality in video teaching.

[0005] The present invention is implemented in this way. In a first aspect, the present invention provides a satisfaction survey method based on video recognition, comprising:

[0006] Acquire video information of a subject user and a plurality of users to be identified; wherein the video information has a time mark axis, and the time mark axis is used to time mark the video information;

[0007] Performing analysis and processing of user display content on the video information of the subject user to obtain a display content sequence of the subject user, and generating a corresponding behavior evaluation sequence according to the display content sequence; wherein the display content sequence includes a plurality of user display contents corresponding to the time mark axis, the user display contents are used to describe the video content displayed by the subject user to each of the to-be-identified users through video, and the behavior evaluation sequence includes a plurality of behavior evaluation frameworks corresponding to the time mark axis, the behavior evaluation frameworks are used to evaluate and process the user behavior characteristics of the to-be-identified users;

[0008] Performing analysis and processing of user behavior characteristics on the video information of each of the users to be identified, respectively, to obtain a behavior characteristic sequence of each of the users to be identified; wherein the behavior characteristic sequence includes a plurality of user behavior characteristics corresponding to the time mark axis, and the user behavior characteristics are used to describe the behavior characteristics of the users to be identified fed back by the video information;

[0009] Substituting each of the behavior feature sequences into the behavior evaluation sequence, respectively, and making each of the behavior evaluation frameworks of the behavior evaluation sequence perform behavior evaluation processing on each of the user behavior features of each of the behavior feature sequences, so as to obtain a basic evaluation group of each of the behavior feature sequences; wherein the basic evaluation group includes a number of basic evaluation parameters corresponding to each of the user behavior features of the behavior feature sequence, and the basic evaluation parameters are used to perform an initial satisfaction evaluation on the user behavior features with respect to the user displayed content;

[0010] The behavior feature sequence is assigned time weights according to the time mark axis to obtain a time weight sequence of the behavior feature sequence, and the basic evaluation group is subjected to parameter correction processing according to the time weight sequence to obtain a corrected evaluation group; wherein the time weight sequence includes a number of time weights corresponding to each of the user behavior features of the behavior feature sequence, and the corrected evaluation group includes a number of corrected evaluation parameters, and the corrected evaluation parameters are used to perform a final satisfaction evaluation on the user behavior features with respect to the user's displayed content.

[0011] In a second aspect, the present invention provides a satisfaction survey system based on video recognition, which is used to implement a satisfaction survey method based on video recognition as described in any one of the first aspects.

[0012] The present invention provides a satisfaction survey method based on video recognition, which has the following beneficial effects:

[0013] The present invention obtains video information of a main user and a user to be identified, and performs time marking, analyzes the video of the main user, generates a display content sequence and a behavior evaluation sequence, analyzes the video of the user to be identified, generates a behavior feature sequence, substitutes the behavior feature sequence into the behavior evaluation sequence, evaluates the user's behavior features, obtains a basic evaluation group, assigns a time weight to the behavior feature sequence according to a time marking axis, performs parameter correction on the basic evaluation group, and obtains a corrected evaluation group. The method realizes automatic and real-time evaluation of user satisfaction through video analysis, utilizes time marking and weight correction, ensures the accuracy and timeliness of the evaluation results, improves the efficiency of data acquisition and processing, provides more objective and reliable satisfaction feedback, and solves the problem in the prior art that it is difficult to control students' satisfaction with teaching quality in video teaching. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is a schematic diagram of the steps of a satisfaction survey method based on video recognition provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0015] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0016] The implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0017] Reference Figure 1 As shown, a preferred embodiment of the present invention is provided.

[0018] In a first aspect, the present invention provides a satisfaction survey method based on video recognition, comprising:

[0019] S1: Obtain video information of a subject user and a plurality of users to be identified; wherein the video information has a time mark axis, and the time mark axis is used to time mark the video information;

[0020] S2: Analyze and process the user display content of the video information of the subject user to obtain the display content sequence of the subject user, and generate a corresponding behavior evaluation sequence according to the display content sequence; wherein the display content sequence includes a plurality of user display contents corresponding to the time mark axis, and the user display contents are used to describe the video content displayed by the subject user to each of the to-be-identified users through the video, and the behavior evaluation sequence includes a plurality of behavior evaluation frameworks corresponding to the time mark axis, and the behavior evaluation frameworks are used to evaluate and process the user behavior characteristics of the to-be-identified users;

[0021] S3: Analyze and process the user behavior characteristics of the video information of each of the to-be-identified users respectively, so as to obtain a behavior characteristic sequence of each of the to-be-identified users; wherein the behavior characteristic sequence includes a plurality of user behavior characteristics corresponding to the time mark axis, and the user behavior characteristics are used to describe the behavior characteristics of the to-be-identified users fed back by the video information;

[0022] S4: Substituting each of the behavior feature sequences into the behavior evaluation sequence, respectively, and making each of the behavior evaluation frameworks of the behavior evaluation sequence perform behavior evaluation processing on each of the user behavior features of each of the behavior feature sequences, so as to obtain a basic evaluation group of each of the behavior feature sequences; wherein the basic evaluation group includes a number of basic evaluation parameters corresponding to each of the user behavior features of the behavior feature sequence, and the basic evaluation parameters are used to perform an initial satisfaction evaluation on the user behavior features with respect to the user displayed content;

[0023] S5: Assigning time weights to the behavior feature sequence according to the time mark axis to obtain a time weight sequence of the behavior feature sequence, and performing parameter correction processing on the basic evaluation group according to the time weight sequence to obtain a corrected evaluation group; wherein the time weight sequence includes a number of time weights of each of the user behavior features corresponding to the behavior feature sequence, and the corrected evaluation group includes a number of corrected evaluation parameters, and the corrected evaluation parameters are used to perform a final satisfaction evaluation on the user behavior features with respect to the user's displayed content.

[0024] Specifically, in step S1 of the embodiment provided by the present invention, the main user and the user to be identified are the divisions of different roles in a multi-person video call. Taking online courses as an example, the teacher in charge of teaching is the main user, and the students are the users to be identified. The main user and the users to be identified communicate through real-time online video, wherein the video information displayed by the main user is the content received by each user to be identified, and the video information displayed by each user to be identified is the basis for judging the attitude of the user to be identified towards the content displayed by the main user.

[0025] More specifically, in video communication, video information of both the main user and the user to be identified will be collected and broadcast on a designated video communication platform. In other words, the video information of the main user and the user to be identified can be directly obtained by accessing the video information of the video communication platform.

[0026] More specifically, the time synchronization of the video information of the main user and the user to be identified is ensured, and a unified time-stamp axis is used for marking. Each frame of video can be associated with a specific time point through timestamp technology. During the video acquisition process, key time points are automatically marked. Each frame of video information is associated with a specific time point on the time-stamp axis to form a time-video frame mapping relationship. The video information is structured and stored according to the time-stamp axis to facilitate subsequent analysis and processing. Each video clip can be accurately located to a specific time point.

[0027] It is understandable that the unified time-stamp axis ensures the precise synchronization of the video information of the main user and the user to be identified in time. This synchronization is the basis for subsequent behavior analysis and evaluation. The time-stamp axis allows the video information to be accurately segmented and analyzed according to time points. Fine-grained time tags help capture subtle changes and reactions in user behavior, thereby improving the accuracy of analysis. Through precise time tags, the relationship between user behavior and the content displayed by the main user can be better associated. This correlation helps to deeply understand the user's reaction to the displayed content and improve the accuracy and reliability of behavior evaluation.

[0028] Specifically, in step S2 of the embodiment provided by the present invention, the video of the subject user is divided into several segments according to the time mark axis, each segment corresponds to a specific time period, and key frames are extracted from each video segment. These key frames will be used for subsequent content analysis. The extracted key frames are analyzed using computer vision and natural language processing technology to identify the user display content displayed by the subject user. The user display content is the communication content displayed by the subject user in a specific time period, including image recognition, text recognition (such as OCR), audio, etc.

[0029] More specifically, the user display contents of each time period are arranged in chronological order to form a display content sequence, and each content item is associated with a specific time point on the time mark axis.

[0030] More specifically, a corresponding behavior evaluation framework is generated according to the user's displayed content. It can be understood that the user's displayed content is the video communication content displayed by the main user to the user to be identified. As the recipient of the video communication content, the posture displayed by the user to be identified in his video information can reflect the attitude of the user to be identified towards the user's displayed content. The attitude of the user to be identified towards the user's displayed content is different, and the behavioral posture displayed in his video information will be different. Based on this, a corresponding behavior evaluation framework can be generated according to the user's displayed content. The purpose of the behavior evaluation framework is to judge the behavioral posture displayed by the user to be identified in his video information to evaluate the attitude of the user to be identified towards the user's displayed content, such as satisfaction.

[0031] More specifically, the user behavior analysis results are arranged in chronological order to form a behavior evaluation sequence, and each evaluation item is associated with a specific time point on the time stamp axis.

[0032] It is understandable that through the time-stamp axis and the display content sequence, we can accurately understand the specific content displayed by the main user at each time point, which is helpful for detailed analysis of the display effect and user response. The behavior evaluation sequence associates the user's behavior data when watching different display contents with the specific content, which facilitates in-depth understanding of the user's reaction pattern and interest points. Based on the display content sequence and behavior evaluation sequence, data-driven display content optimization can be performed. By analyzing which content users show higher interest and positive response, the display strategy can be adjusted and optimized to improve user participation and satisfaction.

[0033] Specifically, in step S3 of the embodiment provided by the present invention, video information is collected from each user to be identified. These videos are usually records of the user's reaction when watching the content displayed by the main user. The collected videos are divided into several segments according to the time mark axis. Each segment corresponds to a specific time period. Key frames are extracted from each video segment. These key frames will be used to identify the specific behavior of the user. Computer vision and machine learning technology are used to extract the user's behavior characteristics from the key frames. It mainly includes:

[0034] Facial expression analysis: Identify changes in the user's facial expressions and determine their emotional state, such as happiness, confusion, concentration, etc.

[0035] Eye tracking: Analyze the user's eye movements to determine their level of concentration.

[0036] Body movement recognition: Identify the user's gestures, body posture, etc., and determine their interactive behavior.

[0037] More specifically, the extracted behavioral features are classified and labeled, each feature is associated with a specific time point on the time mark axis, and the labeled behavioral features are arranged in chronological order to form a behavioral feature sequence. Each behavioral feature item corresponds to a time period and describes the user's specific behavioral performance in that time period.

[0038] More specifically, the behavior evaluation framework obtained in the above steps can evaluate the user behavior characteristics to determine the user behavior characteristics of the user to be identified, such as the attention level, emotional response intensity, interaction frequency, etc. of the user to be identified on the user display content, that is, the satisfaction of the user to be identified with the user display content.

[0039] It is understandable that through the time-stamp axis and behavioral feature sequence, the specific behavior and reaction of each user to be identified at each time point can be accurately understood, providing detailed behavioral analysis data. Using a variety of behavioral recognition technologies (such as facial expressions, eye tracking, body movements, etc.), the user's behavioral characteristics can be fully captured, providing multi-dimensional user behavior data, and data-driven optimization can be performed based on the analysis and evaluation results of user behavior characteristics. For example, in the field of education, the teaching content and methods can be adjusted according to the behavioral characteristics of students; in the field of marketing, product display and advertising strategies can be optimized according to user reactions.

[0040] Specifically, in step S4 of the embodiment provided by the present invention, a behavior feature sequence extracted and generated from video information includes user behavior features corresponding to a time-stamp axis, and a predefined behavior evaluation sequence includes multiple behavior evaluation frameworks for evaluating user behavior features in different time periods.

[0041] More specifically, each user behavior feature in the behavior feature sequence is substituted into the corresponding behavior evaluation framework. Each evaluation framework may evaluate the user behavior features of different time periods to ensure the synchronization of the behavior feature sequence and the evaluation sequence on the timeline, so that the behavior features at each time point can be accurately substituted into the corresponding evaluation framework for evaluation.

[0042] More specifically, the behavior evaluation framework analyzes and calculates the user behavior characteristics substituted in, generates corresponding evaluation parameters, stores the calculated evaluation parameters as basic evaluation parameters, organizes and records them according to the timeline and behavior characteristics, and aggregates the evaluation parameters generated by each behavior evaluation framework to form a basic evaluation group. The basic evaluation group includes basic evaluation parameters of user behavior characteristics corresponding to a number of behavior feature sequences.

[0043] It can be understood that by substituting the behavioral feature sequence into multiple evaluation frameworks, a refined satisfaction evaluation of the user's behavioral characteristics can be performed. The basic evaluation group contains evaluation parameters of multiple dimensions and can comprehensively reflect the user's behavioral characteristics.

[0044] Specifically, in step S5 of the embodiment provided by the present invention, a time weight rule is set according to the time position of the user behavior characteristics in the entire behavior characteristic sequence. For example, in video communication, the concentration of the user to be identified will continue to decline. Therefore, the attitude reflected by the behavior posture of the user to be identified in the later stage is different from the attitude reflected by the behavior posture displayed in the early stage. That is, the negative satisfaction of the user to be identified with the user display content reflected by the negative tendency behavior posture displayed in the early stage is higher than the negative satisfaction of the user to be identified with the user display content reflected by the negative tendency behavior posture displayed in the later stage.

[0045] More specifically, according to the pre-set time weight rule, the time weight of each user behavior feature is calculated, the user behavior features in the behavior feature sequence are matched with the time mark axis to ensure that each behavior feature corresponds to the correct time point, and according to the time weight rule, each user behavior feature is assigned a corresponding time weight to generate a time weight sequence, which contains the time weight value of each user behavior feature.

[0046] More specifically, basic evaluation parameters of each user behavior feature are extracted from the basic evaluation group, and weighted correction is performed on the basic evaluation parameters according to the time weight sequence, and the corrected evaluation parameters are summarized to form a corrected evaluation group. The corrected evaluation group includes the corrected evaluation parameters of each user behavior feature.

[0047] It can be understood that by assigning time weights to the behavior feature sequences, the impact of the user's behavior features at different time points on the overall satisfaction can be more accurately reflected.

[0048] The present invention provides a satisfaction survey method based on video recognition, which has the following beneficial effects:

[0049] The present invention obtains video information of a main user and a user to be identified, and performs time marking, analyzes the video of the main user, generates a display content sequence and a behavior evaluation sequence, analyzes the video of the user to be identified, generates a behavior feature sequence, substitutes the behavior feature sequence into the behavior evaluation sequence, evaluates the user's behavior features, obtains a basic evaluation group, assigns a time weight to the behavior feature sequence according to a time marking axis, performs parameter correction on the basic evaluation group, and obtains a corrected evaluation group. The method realizes automatic and real-time evaluation of user satisfaction through video analysis, utilizes time marking and weight correction, ensures the accuracy and timeliness of the evaluation results, improves the efficiency of data acquisition and processing, provides more objective and reliable satisfaction feedback, and solves the problem in the prior art that it is difficult to control students' satisfaction with teaching quality in video teaching.

[0050] Preferably, the step of analyzing and processing the user display content of the video information of the subject user to obtain the display content sequence of the subject user includes:

[0051] S21: Acquire the subject of the display content of the subject user, and retrieve the blackboard analysis model, portrait analysis model and voice analysis model corresponding to the display content theme from a preset database according to the display content theme;

[0052] S22: dividing the video information of the subject user into a blackboard part, a portrait part and a voice part;

[0053] S23: Analyze and process the blackboard writing part of the video information of the subject user according to the blackboard writing analysis model to obtain a blackboard writing parameter sequence of the video information; wherein the blackboard writing parameter sequence includes blackboard writing relevance parameters of each user's displayed content corresponding to the displayed content sequence, and the blackboard writing relevance parameters are used to describe the degree of relevance between the blackboard writing part of the video information and the displayed content theme;

[0054] S24: Analyze and process the portrait part of the video information of the subject user according to the portrait analysis model to obtain a portrait parameter sequence of the video information; wherein the portrait parameter sequence includes portrait relevance parameters of each of the user's displayed contents corresponding to the displayed content sequence, and the portrait relevance parameters are used to describe the degree of relevance between the portrait part of the video information and the theme of the displayed content;

[0055] S25: Analyze and process the speech part of the video information of the subject user according to the speech analysis model to obtain a speech parameter sequence of the video information; wherein the speech parameter sequence includes speech relevance parameters of each user's display content corresponding to the display content sequence, and the speech relevance parameters are used to describe the degree of relevance between the speech part of the video information and the theme of the display content;

[0056] S26: The blackboard parameter sequence, the portrait parameter sequence and the voice parameter sequence of the video information are collectively used as the display content sequence of the main user; wherein each user display content of the display content sequence is generated by a combination of the blackboard association parameters, the portrait association parameters and the voice association parameters corresponding to the time mark.

[0057] Specifically, the theme of the display content is extracted from the video information of the subject user, which can be input by the subject user himself or obtained by identifying the video information.

[0058] More specifically, according to the identified display content theme, the corresponding blackboard analysis model, portrait analysis model and voice analysis model are retrieved from the preset database. These models are pre-trained and are used to analyze the content of the blackboard, portrait and voice parts respectively. It is not difficult to understand that in video communication, such as online classes, the subject user displays different topics of user display content, and the subject user's own behavioral posture has different relevance to the topic of the user display content. That is to say, the behavioral posture displayed by the subject user in the video information, for different topics, the content quality and content quantity of the corresponding user display content are different, and corresponding to different qualities of user display content, The different behavioral postures displayed by the user to be identified are also different. For example, when the main user is displaying important content, if the behavioral posture of the user to be identified shows that his / her reception of the content is poor, then the feedback level of the user to be identified on the low satisfaction with the content displayed by the main user is higher. When the main user is displaying unimportant content, if the behavioral posture of the user to be identified shows that his / her reception of the content is poor, then the feedback level of the user to be identified on the low satisfaction with the content displayed by the main user is lower. For example, it is equivalent to that when the main user is displaying unimportant content, the behavioral posture of the user to be identified shows that he / she is distracted, and this behavior shows that the feedback level of the user to be identified on the low satisfaction with the content displayed by the main user is lower.

[0059] More specifically, the video information of the main user is divided into a blackboard part, a portrait part, and a voice part. This step can be achieved through video segmentation technology and image processing technology, and the blackboard part in the video information is analyzed using the retrieved blackboard analysis model. The result of the analysis generates a blackboard parameter sequence, which includes the blackboard relevance parameters of each user's displayed content. These parameters describe the degree of relevance between the blackboard part and the theme of the displayed content. The retrieved portrait analysis model is used to analyze the portrait part in the video information. The result of the analysis generates a portrait parameter sequence, which includes the portrait relevance parameters of each user's displayed content. These parameters describe the degree of relevance between the portrait part and the theme of the displayed content. The retrieved voice analysis model is used to analyze the voice part in the video information. The result of the analysis generates a voice parameter sequence, which includes the voice relevance parameters of each user's displayed content. These parameters describe the degree of relevance between the voice part and the theme of the displayed content.

[0060] More specifically, the blackboard parameter sequence, portrait parameter sequence and voice parameter sequence of the video information are combined to generate the display content sequence of the main user, ensuring that each user display content in the display content sequence is composed of blackboard relevance parameters, portrait relevance parameters and voice relevance parameters with corresponding time marks.

[0061] It can be understood that by analyzing the blackboard, portrait and voice parts separately, we can fully understand all aspects of the video content and improve the accuracy of the analysis. The generated blackboard relevance parameters, portrait relevance parameters and voice relevance parameters can specifically describe the relevance between each part of the content and the display theme, provide more detailed analysis results, and combine the parameters of different parts to form a complete display content sequence, which can more accurately reflect the overall relevance and display effect of the video content.

[0062] Preferably, the step of generating a corresponding behavior evaluation sequence according to the display content sequence comprises:

[0063] S27: generating a content portrait of the user displayed content according to the blackboard-writing relevance parameter, the portrait relevance parameter, and the voice relevance parameter of the user displayed content, and using the content portrait as a mapping subject; wherein the content portrait is used to describe the display form of the user displayed content;

[0064] S28: constructing a mapping object according to the user to be identified, and taking various possible behavior characteristics of the user to be identified when receiving the user display content of the subject user as the preliminary mapping posture of the mapping object; wherein the possible behavior characteristics are user behavior characteristics that the user to be identified may exhibit when receiving the user display content of the subject user;

[0065] S29: performing mapping analysis processing on the preliminary mapping posture of the mapping object according to the mapping subject to generate a posture adaptability distribution of the preliminary mapping posture of the mapping object; wherein the posture adaptability distribution is used to describe the adaptability of each of the possible behavior features of the preliminary mapping posture when receiving the information mapping of the mapping subject;

[0066] S210: generating a corresponding behavior evaluation framework according to the posture adaptability distribution; wherein the behavior evaluation framework is used to evaluate the user behavior characteristics according to the posture adaptability distribution to obtain the corresponding basic evaluation parameters;

[0067] S211: sequentially arranging the behavior evaluation frames corresponding to the user displayed contents to obtain the behavior evaluation sequence.

[0068] Specifically, the blackboard relevance parameters, portrait relevance parameters and voice relevance parameters of the user's displayed content are extracted, and these parameters are used to generate a content portrait of the user's displayed content. The content portrait is used to describe the display form of the displayed content, including visual, auditory and interactive features.

[0069] More specifically, the user to be identified is the actual audience receiving the content displayed by the main user. The user to be identified is taken as the mapping object, and a preliminary mapping posture of the mapping object is constructed according to the behavioral characteristics that the user to be identified may exhibit when receiving the displayed content. These behavioral characteristics include gaze direction, facial expressions, body postures, hand operations and other aspects of behavioral characteristics. These behavioral characteristics represent the satisfaction of the user to be identified with the user display content of the main user.

[0070] More specifically, the content portrait is used as the mapping subject, and the mapping object's preparatory mapping posture is mapped and analyzed. Through the analysis, the posture adaptability distribution of the mapping object's preparatory mapping posture is generated. The posture adaptability distribution is used to describe the adaptability of different behavioral characteristics when receiving display content, that is, to describe the satisfaction of the user to be identified with the user display content of the subject user as fed back by each possible behavioral characteristic in the preparatory mapping posture of the mapping object when receiving the information mapping of the mapping subject.

[0071] More specifically, based on the posture adaptability distribution, a corresponding behavior evaluation framework is generated. The behavior evaluation framework is used to evaluate the adaptability of user behavior characteristics and generate basic evaluation parameters. These parameters quantify the response and adaptability of user behavior characteristics when receiving display content. The behavior evaluation frameworks corresponding to each user's display content are arranged in sequence to form a behavior evaluation sequence. The behavior evaluation sequence comprehensively reflects the user's behavior characteristics and adaptability at different stages of the entire display content reception process.

[0072] It is understandable that by generating content portraits, the multi-dimensional characteristics of the displayed content can be described in detail. The mapping analysis process associates the displayed content with the user behavior characteristics, providing accurate user behavior analysis. According to different displayed content and user behavior characteristics, a personalized behavior evaluation framework is generated to provide targeted evaluation results. The posture adaptive distribution can quantify the adaptability of user behavior characteristics and provide data support for personalized optimization. The generated basic evaluation parameters provide quantitative user behavior analysis data to support data-driven optimization decisions.

[0073] Preferably, the step of analyzing and processing the user behavior characteristics of the video information of each of the to-be-identified users to obtain the behavior characteristic sequence of each of the to-be-identified users comprises:

[0074] S31: dividing the video information into a plurality of video contents arranged in sequence according to the time mark axis;

[0075] S32: performing feature recognition processing of the gaze direction, facial expression, body posture and hand operation of the user to be identified according to the video content to obtain gaze direction features, facial expression features, body posture features and hand operation features of the user to be identified, and taking the gaze direction features, facial expression features, body posture features and hand operation features of the user to be identified as user behavior features of the user to be identified;

[0076] S33: Arrange the user behavior features in sequence according to the time stamp axis to obtain the behavior feature sequence.

[0077] Specifically, the video content is accurately located through the time-stamp axis, and the video information is divided into a number of continuous and sequentially arranged video content segments according to the time-stamp axis.

[0078] More specifically, computer vision technology or eye tracking technology is used to identify the user's gaze direction, extract the user's gaze direction features in each video clip, record the gaze movement trajectory and gaze point, identify the user's facial area through a facial detection algorithm, analyze the user's facial expression and identify the emotional state (such as happiness, surprise, anger, etc.) through deep learning or a specific expression recognition model, identify the user's body posture through a posture estimation algorithm, capture the body's tilt, rotation and other movements, extract the user's posture features in each video clip, record the posture changes, identify the user's hand position and movement through hand detection technology, analyze the user's hand operations, and identify specific gestures or operation behaviors (such as clicking, sliding, grabbing, etc.).

[0079] More specifically, the gaze direction features, facial expression features, body posture features and hand operation features are integrated to form the user's comprehensive behavioral features, and the comprehensive behavioral features in different video clips are marked according to the time marking axis.

[0080] More specifically, the user behavior features in each video clip are arranged in sequence according to the time stamp axis to generate a complete behavior feature sequence, and the behavior feature sequence of each user to be identified is output as basic data for subsequent analysis and evaluation.

[0081] It is understandable that through advanced computer vision and deep learning technology, it is possible to identify and extract multiple behavioral characteristics of users with high precision, and use the time stamp axis to accurately divide the video content, making the behavioral characteristics capture more detailed and accurate. Feature recognition in four dimensions of gaze direction, facial expression, body posture and hand operation provides a full range of user behavior analysis. By integrating multiple behavioral characteristics, a more comprehensive and accurate evaluation of user behavior can be made.

[0082] Preferably, the steps of respectively substituting each of the behavior feature sequences into the behavior evaluation sequence, and allowing each of the behavior evaluation frameworks of the behavior evaluation sequence to respectively perform behavior evaluation processing on each of the user behavior features of each of the behavior feature sequences, so as to obtain a basic evaluation group of each of the behavior feature sequences include:

[0083] S41: Substituting each of the behavior feature sequences into the behavior evaluation sequence respectively, and according to the time mark axis, making each of the behavior evaluation frameworks of the behavior evaluation sequence perform behavior evaluation processing on the user behavior features with the corresponding time mark in each of the behavior evaluation sequences, so as to obtain basic evaluation parameters of each of the user behavior features of each of the behavior feature sequences;

[0084] S42: The basic evaluation parameters of each of the user behavior characteristics of each of the behavior characteristic sequences are sequentially arranged to obtain the basic evaluation group of each of the behavior characteristic sequences.

[0085] Specifically, the behavior feature sequences of each user are obtained, and these sequences contain user behavior features arranged by time stamps. Each behavior feature sequence is substituted into a predefined behavior evaluation sequence. The behavior evaluation sequence contains multiple behavior evaluation frameworks, and each framework is used to evaluate user behavior features in different time periods.

[0086] More specifically, according to the time stamp axis, the user behavior features in the behavior feature sequence are matched with the time stamps in the behavior evaluation sequence to ensure that the user behavior features at each time point can be accurately mapped to the evaluation framework.

[0087] More specifically, after aligning the time markers, the application behavior evaluation framework evaluates the user behavior characteristics at the corresponding time points. Each behavior evaluation framework analyzes its corresponding user behavior characteristics, extracts basic evaluation parameters, and arranges the basic evaluation parameters of each time point in chronological order to form a complete basic evaluation group.

[0088] Preferably, the step of assigning time weights to the behavior feature sequence according to the time mark axis to obtain a time weight sequence of the behavior feature sequence includes:

[0089] S51: Obtain the subject of the display content of the subject user, and retrieve the theoretical concentration change curve corresponding to the time mark axis in a preset database according to the display content theme; wherein the theoretical concentration change curve is used to describe the change curve of the concentration of the to-be-identified user under the display content theme over time, and the theoretical concentration change curve includes a plurality of theoretical concentrations, and each of the theoretical concentrations corresponds to each of the time marks of the time mark axis one by one;

[0090] S52: Analyze and process the behavior feature feedback degree of the time mark according to the theoretical concentration to obtain the behavior feature feedback distribution of the time mark; wherein the behavior feature feedback distribution includes a positive feedback vector and a negative feedback vector, the positive feedback vector is used to describe the feedback degree of the positive tendency of the user display content fed back by the user behavior feature corresponding to the time mark, and the negative feedback vector is used to describe the feedback degree of the negative tendency of the user display content fed back by the user behavior feature corresponding to the time mark;

[0091] S53: Binding the behavior feature feedback distribution with the time mark to obtain the time weight, and sorting the time weights according to the order of the time marks in the time mark axis to obtain the time weight sequence.

[0092] Specifically, the content theme that the main user is displaying is obtained, and according to the identified content theme, the corresponding theoretical concentration change curve is retrieved from a preset database. This curve describes the change in concentration of the user to be identified under a given theme over time.

[0093] More specifically, a time mark axis is established to mark each time point in the behavior feature sequence, and each theoretical concentration mark of the theoretical concentration change curve is matched one by one with each time point on the time mark axis.

[0094] More specifically, according to the time points on the time-marker axis, the user behavior characteristics in the corresponding time period are analyzed, and the user behavior characteristic feedback degree at each time point is calculated, including positive feedback and negative feedback. The positive feedback vector describes the degree of positive tendency of the user behavior characteristics towards the displayed content, and the negative feedback vector describes the degree of negative tendency of the user behavior characteristics towards the displayed content.

[0095] It should be noted that the basic evaluation parameters have positive and negative tendencies, that is, the basic evaluation parameters can indicate that the satisfaction of the user to be identified with the user display content is positive satisfaction or negative dissatisfaction, and because the concentration of the user to be identified will decrease over time, the feedback value of the positive satisfaction and negative dissatisfaction shown in the early and later stages is different. Specifically, the negative satisfaction in the early stage has more feedback value than the negative satisfaction in the later stage, indicating that the user to be identified is more dissatisfied, and the positive satisfaction in the later stage has more feedback value than the positive satisfaction in the early stage, indicating that the user to be identified is more satisfied.

[0096] More specifically, a complete behavior feature feedback distribution is generated based on the positive and negative feedback vectors at each time point, the behavior feature feedback distribution is bound to the time mark, the feedback distribution at each time point is associated with its corresponding time mark, and the time weight of each time mark is calculated based on the bound behavior feature feedback distribution. The time weight can be calculated based on the proportion and intensity of positive and negative feedback, and the time weights of each time mark are sorted according to the order of the time mark axis to form a complete time weight sequence, and the time weight sequence sorted according to the time mark axis is output. This sequence describes the user's attention and feedback weight distribution in the entire behavior feature sequence.

[0097] It is understandable that the theoretical concentration change curve is used to accurately describe the changes in user concentration at different time points, ensure the scientificity and rationality of time weight distribution, and comprehensively capture user behavioral feature feedback through the evaluation of positive and negative feedback vectors to ensure the accuracy of feedback analysis. Combined with positive and negative feedback vectors, multi-dimensional user behavior analysis is provided to help deeply understand the user's behavioral response to the displayed content theme, and generate time series data through the time mark axis to facilitate dynamic behavior analysis and trend prediction.

[0098] Preferably, the step of performing parameter correction processing on the basic evaluation group according to the time weight sequence to obtain a corrected evaluation group includes:

[0099] S54: determining the basic evaluation parameter corresponding to the time weight in the time weight sequence in the basic evaluation group according to the time mark;

[0100] S55: Analyzing and processing the evaluation tendency of the basic evaluation parameter to determine whether the basic evaluation parameter has a positive tendency or a negative tendency;

[0101] S56: When the basic evaluation parameter has a positive tendency, performing parameter correction processing on the basic evaluation parameter according to the positive feedback vector in the time weight to obtain the corrected evaluation parameter;

[0102] S57: When the basic evaluation parameter has a negative tendency, performing parameter correction processing on the basic evaluation parameter according to the negative feedback vector in the time weight to obtain the corrected evaluation parameter;

[0103] S58: Arrange the various modified evaluation parameters according to the time stamp axis to obtain the modified evaluation group.

[0104] Specifically, according to the time mark axis, basic evaluation parameters of corresponding time points are extracted from the basic evaluation group, and these basic evaluation parameters are initial values ​​used to describe the behavior characteristics of the user at each time point.

[0105] More specifically, an evaluation tendency analysis is performed on each extracted basic evaluation parameter to determine whether it has a positive tendency or a negative tendency. The evaluation tendency can be determined based on the positivity or negativity of the user's behavior.

[0106] More specifically, when the basic evaluation parameter has a positive tendency, the positive feedback vector in the time weight sequence is used to correct the parameter. The correction method may be to give the basic evaluation parameter a correction coefficient for calculation. When the basic evaluation parameter has a negative tendency, the negative feedback vector in the time weight sequence is used to correct the parameter. The correction method may be to give the basic evaluation parameter a correction coefficient for calculation. Through the above correction processing, the corrected evaluation parameter is generated. These parameters reflect the user behavior characteristics adjusted under the action of the time weight and feedback vector.

[0107] More specifically, according to the time stamp axis, all revised evaluation parameters are arranged in chronological order to ensure that the revised evaluation parameters at each time point can be correctly mapped to the corresponding time stamp. After the arrangement is completed, a complete revised evaluation group is obtained. This set of data reflects the user behavior feature sequence under the adjustment of time weight and feedback vector.

[0108] It can be understood that by distinguishing between positive and negative tendencies, more detailed analysis and correction of basic evaluation parameters are carried out, the accuracy of the evaluation is improved, and the correction processing using positive and negative feedback vectors can better reflect the user's actual behavioral tendencies and feedback. During the process of user behavior, parameter correction and adjustment can be carried out in real time, providing more comprehensive user behavior evaluation results, helping to deeply understand the user's behavioral responses at different time points.

[0109] In a second aspect, the present invention provides a satisfaction survey system based on video recognition, which is used to implement a satisfaction survey method based on video recognition as described in any one of the first aspects.

[0110] In this embodiment, for the specific implementation of each module in the above system embodiment, please refer to the above method embodiment, which will not be repeated here.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A satisfaction survey method based on video recognition, It is characterized in that include: Acquire video information of a subject user and a plurality of users to be identified; wherein the video information has a time mark axis, and the time mark axis is used to time mark the video information; Performing analysis and processing of user display content on the video information of the subject user to obtain a display content sequence of the subject user, and generating a corresponding behavior evaluation sequence according to the display content sequence; wherein the display content sequence includes a plurality of user display contents corresponding to the time mark axis, the user display contents are used to describe the video content displayed by the subject user to each of the to-be-identified users through video, and the behavior evaluation sequence includes a plurality of behavior evaluation frameworks corresponding to the time mark axis, the behavior evaluation frameworks are used to evaluate and process the user behavior characteristics of the to-be-identified users; Performing analysis and processing of user behavior characteristics on the video information of each of the users to be identified, respectively, to obtain a behavior characteristic sequence of each of the users to be identified; wherein the behavior characteristic sequence includes a plurality of user behavior characteristics corresponding to the time mark axis, and the user behavior characteristics are used to describe the behavior characteristics of the users to be identified fed back by the video information; Substituting each of the behavior feature sequences into the behavior evaluation sequence, respectively, and making each of the behavior evaluation frameworks of the behavior evaluation sequence perform behavior evaluation processing on each of the user behavior features of each of the behavior feature sequences, so as to obtain a basic evaluation group of each of the behavior feature sequences; wherein the basic evaluation group includes a number of basic evaluation parameters corresponding to each of the user behavior features of the behavior feature sequence, and the basic evaluation parameters are used to perform an initial satisfaction evaluation on the user behavior features with respect to the user displayed content; The behavior feature sequence is assigned a time weight according to the time mark axis to obtain a time weight sequence of the behavior feature sequence, and the basic evaluation group is subjected to parameter correction processing according to the time weight sequence to obtain a corrected evaluation group; wherein the time weight sequence includes a number of time weights of each of the user behavior features corresponding to the behavior feature sequence, and the corrected evaluation group includes a number of corrected evaluation parameters, and the corrected evaluation parameters are used to perform a final satisfaction evaluation of the user behavior features with respect to the user displayed content; The step of analyzing and processing the user display content of the video information of the subject user to obtain the display content sequence of the subject user includes: Acquire the subject of the display content of the subject user, and retrieve the blackboard analysis model, portrait analysis model and voice analysis model corresponding to the display content theme from a preset database according to the display content theme; Dividing the video information of the subject user into a blackboard part, a portrait part and a voice part; The blackboard writing part of the video information of the subject user is analyzed and processed according to the blackboard writing analysis model to obtain a blackboard writing parameter sequence of the video information; wherein the blackboard writing parameter sequence includes blackboard writing relevance parameters of each user's display content corresponding to the display content sequence, and the blackboard writing relevance parameters are used to describe the degree of relevance between the blackboard writing part of the video information and the display content theme; Analyzing and processing the portrait part of the video information of the subject user according to the portrait analysis model to obtain a portrait parameter sequence of the video information; wherein the portrait parameter sequence includes portrait relevance parameters of each of the user's displayed contents corresponding to the displayed content sequence, and the portrait relevance parameters are used to describe the degree of relevance between the portrait part of the video information and the theme of the displayed content; Analyze and process the speech part of the video information of the subject user according to the speech analysis model to obtain a speech parameter sequence of the video information; wherein the speech parameter sequence includes speech relevance parameters of each of the user's displayed contents corresponding to the displayed content sequence, and the speech relevance parameters are used to describe the degree of relevance between the speech part of the video information and the theme of the displayed content; The blackboard parameter sequence, the portrait parameter sequence and the voice parameter sequence of the video information are collectively used as the display content sequence of the main user; wherein each user display content of the display content sequence is generated by a combination of the blackboard association parameters, the portrait association parameters and the voice association parameters corresponding to the time mark.

2. A satisfaction survey method based on video recognition as claimed in claim 1, It is characterized in that The step of generating a corresponding behavior evaluation sequence according to the display content sequence comprises: Generate a content portrait of the user's displayed content according to the blackboard-writing relevance parameter, the portrait relevance parameter, and the voice relevance parameter of the user's displayed content, and use the content portrait as a mapping subject; wherein the content portrait is used to describe the presentation form of the user's displayed content; A mapping object is constructed according to the user to be identified, and various possible behavior characteristics of the user to be identified when receiving the user display content of the subject user are used as preliminary mapping gestures of the mapping object; wherein the possible behavior characteristics are user behavior characteristics that the user to be identified may exhibit when receiving the user display content of the subject user; According to the mapping subject, mapping analysis processing is performed on the preliminary mapping posture of the mapping object to generate a posture adaptability distribution of the preliminary mapping posture of the mapping object; wherein the posture adaptability distribution is used to describe the adaptability of each of the possible behavior characteristics of the preliminary mapping posture when receiving the information mapping of the mapping subject; Generate a corresponding behavior evaluation framework according to the posture adaptability distribution; wherein the behavior evaluation framework is used to evaluate the user behavior characteristics according to the posture adaptability distribution to obtain the corresponding basic evaluation parameters; The behavior evaluation frames corresponding to the user displayed contents are arranged in sequence to obtain the behavior evaluation sequence.

3. A satisfaction survey method based on video recognition as claimed in claim 1, It is characterized in that The steps of analyzing and processing the user behavior characteristics of the video information of each user to be identified to obtain the behavior characteristic sequence of each user to be identified include: Dividing the video information into a plurality of sequentially arranged video contents according to the time mark axis; Performing feature recognition processing on the gaze direction, facial expression, body posture and hand operation of the user to be identified according to the video content to obtain gaze direction features, facial expression features, body posture features and hand operation features of the user to be identified, and using the gaze direction features, facial expression features, body posture features and hand operation features of the user to be identified as user behavior features of the user to be identified; The user behavior features are arranged in sequence according to the time stamp axis to obtain the behavior feature sequence.

4. A satisfaction survey method based on video recognition as claimed in claim 1, It is characterized in that Substituting each of the behavior feature sequences into the behavior evaluation sequence, respectively, and allowing each of the behavior evaluation frameworks of the behavior evaluation sequence to perform behavior evaluation processing on each of the user behavior features of each of the behavior feature sequences, respectively, to obtain a basic evaluation group of each of the behavior feature sequences includes the following steps: Substituting each of the behavior feature sequences into the behavior evaluation sequence respectively, and making each of the behavior evaluation frameworks of the behavior evaluation sequence perform behavior evaluation processing on the user behavior features with the corresponding time mark in each of the behavior evaluation sequences according to the time mark axis, so as to obtain basic evaluation parameters of each of the user behavior features of each of the behavior feature sequences; The basic evaluation parameters of each of the user behavior characteristics of each of the behavior characteristic sequences are arranged in sequence to obtain the basic evaluation group of each of the behavior characteristic sequences.

5. A satisfaction survey method based on video recognition as claimed in claim 1, It is characterized in that The step of assigning time weights to the behavior feature sequence according to the time mark axis to obtain a time weight sequence of the behavior feature sequence includes: Acquire the subject content theme of the subject user, and retrieve the theoretical concentration change curve corresponding to the time mark axis in a preset database according to the subject content theme; wherein the theoretical concentration change curve is used to describe the change curve of the concentration of the to-be-identified user under the subject content theme over time, and the theoretical concentration change curve includes a plurality of theoretical concentrations, and each of the theoretical concentrations corresponds to each of the time marks of the time mark axis one by one; Performing analysis and processing of the behavioral feature feedback degree of the time mark according to the theoretical concentration to obtain the behavioral feature feedback distribution of the time mark; wherein the behavioral feature feedback distribution includes a positive feedback vector and a negative feedback vector, the positive feedback vector is used to describe the feedback degree of the positive tendency of the user display content fed back by the user behavioral feature corresponding to the time mark, and the negative feedback vector is used to describe the feedback degree of the negative tendency of the user display content fed back by the user behavioral feature corresponding to the time mark; The behavior feature feedback distribution is bound to the time mark to obtain the time weight, and the time weights are sorted according to the order of the time marks in the time mark axis to obtain the time weight sequence.

6. A satisfaction survey method based on video recognition as claimed in claim 5, It is characterized in that The step of performing parameter correction processing on the basic evaluation group according to the time weight sequence to obtain a corrected evaluation group includes: Determining, in the basic evaluation group, the basic evaluation parameters corresponding to the time weights in the time weight sequence according to the time stamps; Performing an analysis process on the evaluation tendency of the basic evaluation parameter to determine whether the basic evaluation parameter has a positive tendency or a negative tendency; When the basic evaluation parameter has a positive tendency, performing parameter correction processing on the basic evaluation parameter according to the positive feedback vector in the time weight to obtain the corrected evaluation parameter; When the basic evaluation parameter has a negative tendency, performing parameter correction processing on the basic evaluation parameter according to the negative feedback vector in the time weight to obtain the corrected evaluation parameter; The modified evaluation parameters are arranged according to the time stamp axis to obtain the modified evaluation group.

7. A satisfaction survey system based on video recognition, It is characterized in that Used to implement a satisfaction survey method based on video recognition as described in any one of claims 1-6.