Interaction method and system based on dynamic image recognition, terminal and medium

Abstract

The invention discloses an interaction method and system based on dynamic image recognition, a terminal and a medium, and relates to the technical field of image recognition, and the method comprises the following steps: firstly, a camera or other image capturing equipment is used for collecting dynamic images of a user in real time, and the collected dynamic images are continuous video streams or a series of picture frames; then, the collected dynamic image is preprocessed; according to the invention, by using an advanced dynamic image identification technology, the dynamic image of the user can be captured and analyzed in real time, the dynamic image is accurately segmented into paragraphs representing different actions, and a comprehensive analysis method based on threshold segmentation, model segmentation and deep learning segmentation is combined. The system can more accurately recognize the action and intention of the user, so that the accurate interaction instruction is generated, the accuracy and the real-time response capability of the interaction system are remarkably improved, and smoother and more natural interaction experience is provided for the user.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to an interactive method, system, terminal and medium based on dynamic image recognition. Background Art

[0002] With the rapid development of science and technology, human-computer interaction technology has evolved from traditional keyboard and mouse operations to more natural and intuitive interaction methods. Especially in the fields of smart devices and virtual reality (VR), users expect to interact with the system in a more intuitive way, such as through gestures, expressions or other body movements. However, existing interaction technologies still have some limitations in terms of accuracy and user experience. The accuracy of dynamic image recognition directly affects the generation of interaction instructions. The current technology may cause recognition errors or missed recognition in complex environments, such as lighting changes, background interference and other factors, thereby affecting the accuracy of interaction.

[0003] User experience issues: When using interactive systems, users often need to adapt to specific interaction modes, which may increase learning costs and operational difficulties. In addition, the system lacks an effective user feedback mechanism, making it difficult for users to adjust unsatisfactory interactions.

[0004] In order to solve the above defects, a technical solution is now provided. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the existing interactive technology still has some limitations in terms of accuracy and user experience, and to propose an interactive method, system, terminal and medium based on dynamic image recognition.

[0006] The purpose of the present invention can be achieved by the following technical solutions:

[0007] An interactive method based on dynamic image recognition comprises the following steps:

[0008] S1: First, use a camera or other image capture device to collect the user's dynamic image in real time;

[0009] S2: pre-processing the collected dynamic images;

[0010] S3: then performing action segmentation on the pre-processed dynamic image, so that the dynamic image is segmented into dynamic image segments representing different actions based on the content;

[0011] S4: then intercepting image frames in each segment of the dynamic image that has been cut, and analyzing each intercepted image frame with a representative interactive image in the database to select an image frame with the highest similarity;

[0012] S5: forming one or more corresponding continuous interaction instructions according to the verified image frames and the matching representative interaction images, and executing corresponding interaction commands or operations;

[0013] S6: Finally, obtain user feedback, analyze the user feedback data and modify the interaction instructions. The specific process is as follows:

[0014] The user feedback data obtained includes user operation parameters and historical data parameters, and the operation value and historical parameter value are obtained according to the feedback data. After normalization, the operation value is used as the bottom circle radius and the historical parameter value is used as the height to establish a cone model, and then the center of mass of the cone model is found, and the center of mass is used as the sphere, and a spherical body is established with the preset correction parameter as the radius, and then the spherical body is used as the outline to perform cutting in the established cone model, and the volume of the remaining irregular shape is calculated and recorded as the decision value;

[0015] The obtained decision value is compared with the preset decision threshold. When the decision value exceeds the preset decision threshold, it is determined that the user is not satisfied with the executed interactive command, and there is an error in the interactive command analysis of the dynamic image. Then, the analyzed interactive command is corrected. The specific correction process is as follows:

[0016] First, the different interactive signals identified by the image frame corresponding to each dynamic image segment in the same dynamic image are displayed to the interactive interface through a prompt window, prompting the user whether to execute other interactive instructions in the same dynamic image, and providing the user with the option of whether to execute other displayed interactive instructions. The user can decide whether to modify the interactive instructions independently.

[0017] Furthermore, the specific process in S2 is as follows:

[0018] First, the action segmentation of dynamic images is performed using three segmentation methods: threshold-based segmentation, model-based segmentation, and deep learning-based segmentation. Then, the output results of the above three segmentation methods are comprehensively analyzed to obtain the most accurate segmentation results. The specific steps of the comprehensive analysis are as follows:

[0019] b1: First, the segmentation results obtained by the three segmentation methods are normalized so that the outputs of different methods can be compared and integrated under the same standard. Normalization can be a simple scaling so that all results fall within the same numerical range, or a more complex statistical normalization;

[0020] b2: Define a fusion strategy to comprehensively consider the segmentation results of the three methods. The fusion strategy can be a voting mechanism, weighted average or model fusion, or assign a weight to each method.

[0021] b3: Merge the normalized segmentation results of the three segmentation methods according to the fusion strategy;

[0022] b4: Then use evaluation indicators, including accuracy, recall and F1 score, to evaluate the quality of the fusion result; when the fusion result does not reach the expected accuracy, return to step b2, adjust the fusion strategy or weight, and then repeat steps b3 and b4 for iterative optimization.

[0023] Furthermore, the specific operation steps of performing action segmentation on the dynamic image by using the three segmentation methods based on threshold segmentation, model segmentation and deep learning segmentation in S2 are as follows:

[0024] Based on threshold segmentation: First, motion estimation is performed on the continuous frames in the dynamic image sequence, and the inter-frame motion vector is calculated by the optical flow method Lucas-Kanade method or the block matching method three-frame difference method;

[0025] Then use metrics such as Euclidean distance or normalized mutual information to calculate the inter-frame motion difference;

[0026] Then a difference threshold is set, and when the difference between frames exceeds the difference threshold, it is determined that the action in the dynamic image has changed or converted;

[0027] Finally, according to the threshold judgment, the dynamic image is divided into multiple sections, each section represents a continuous action;

[0028] Model-based segmentation: First, extract features from dynamic image frames that are helpful for action recognition;

[0029] Then use the hidden Markov model or dynamic time warping model to train the feature sequence;

[0030] Apply the trained model to new dynamic image sequences, and the model will recognize the actions in the dynamic images based on the learned patterns;

[0031] Then, based on the recognition results of the model, the action segmentation points of the video sequence are output;

[0032] Based on deep learning segmentation: First, a large number of dynamic interactive image datasets containing different actions are collected and annotated for training deep learning models;

[0033] Then determine the deep learning model, including convolutional neural network for extracting spatial features, recurrent neural network or long short-term memory network for processing time series data;

[0034] The selected deep learning model is trained using the labeled dataset and recognizes the start and end of actions based on the contextual information of the time series;

[0035] Then, the trained model is applied to new dynamic image data, and the model will automatically identify and segment different action segments;

[0036] Finally, some post-processing steps are performed to remove overlapping detection boxes and optimize segmentation boundaries.

[0037] Furthermore, the specific operation steps of S4 are as follows:

[0038] Firstly, for each dynamic image segment, image frames are captured according to a preset time interval or based on a specific action mark;

[0039] A database containing representative interactive images is then established. The interactive images in the database are images of standard interactive actions and can represent different interactive scenes and actions.

[0040] Then, feature extraction is performed on the captured image frames and representative interactive images in the database;

[0041] A similarity measurement method is defined to compare the similarity between a plurality of image frames and the representative interactive image. The similarity measurement methods include Euclidean distance, Hamming distance, structural similarity index and cosine similarity;

[0042] Then, the similarity of several image frames captured in each dynamic image segment is compared with all representative interactive images in the database, and the one with the highest similarity among the several captured image frames is marked;

[0043] Finally, the screened image frames with the highest similarity are verified.

[0044] Furthermore, the method of extracting features from the captured image frames and representative interactive images in the database in S4 includes:

[0045] Appearance features: Use SIFT, SURF and ORB algorithms to extract key points and descriptors;

[0046] Motion features: Calculate optical flow or use optical flow estimation methods to extract motion information;

[0047] Depth features: Use a depth camera or pre-processed depth data to extract features of the depth image;

[0048] Global features: Calculate the histogram of the image including color and gradient or other global descriptors.

[0049] Furthermore, the specific operation steps of obtaining the operation value and the historical parameter value according to the feedback data in S6 are as follows:

[0050] User operation parameters are analyzed and judged by the user's click flow, reaction time, exit rate and expression tracking;

[0051] The click stream is the click commands of the user after executing the corresponding interactive command or operation, and the frequency of the last three clicks is counted and recorded as the click rate value;

[0052] The reaction time is the interval time from the user's first instruction or action after executing the corresponding interactive command or operation, recorded as the response time value;

[0053] The exit rate is the probability of a user exiting after executing the corresponding interactive command;

[0054] Expression tracking uses a camera to collect the user's facial image in real time, uses a face detection algorithm to locate and track the face area, and then uses facial images with emotional expressions to train the expression recognition model. The collected facial images are input into the trained expression recognition model, and the model outputs the expression classification result. The user's expression classification result is then associated with the satisfaction level. The satisfaction level is assigned a score of 1-10, and the obtained score is recorded as the tracking value;

[0055] Then, the obtained point rate value, on-time value, exit probability and table tracking value are calibrated as dl, ys, tg and bz respectively, and after normalization, they are entered into the following formula: To get the operation value YCZ;

[0056] Historical data parameters: collect the number of interactions, error rate, number of retries, and average time to complete tasks in the interactive system;

[0057] The number of interactions is the total number of interactions between the user and the system; the error rate is the ratio of the number of errors encountered when executing interactive commands to the total number of attempts; the number of retries is measured by tracking the number of times the user attempts to re-execute the operation after encountering a problem; the average task completion time is the average time it takes for the system to execute the interactive command after the user passes the interactive command;

[0058] The obtained number of interactions, error rate, number of retries and average time to complete the task are calibrated as jj, cc, ss and pp respectively, and then normalized and entered into the following formula: To obtain the historical parameter value LCC.

[0059] Furthermore, a system applied to the interactive method based on dynamic image recognition comprises:

[0060] An image acquisition module is used to acquire a user's dynamic image in real time through a camera or other image capture device;

[0061] The preprocessing module is used to perform format conversion, resolution adjustment, denoising and color correction preprocessing operations on the collected dynamic images;

[0062] An action segmentation module is used to perform action segmentation on dynamic images based on threshold, model or deep learning methods;

[0063] The fusion analysis module is used to normalize the results of different segmentation methods and apply fusion strategies to obtain the most accurate segmentation results;

[0064] An interactive instruction generation module, used for forming one or continuous interactive instructions according to the segmentation and analysis results;

[0065] The user feedback analysis module is used to collect and analyze user feedback data and make subsequent interactive instruction corrections.

[0066] Furthermore, an interactive terminal includes:

[0067] Image capture devices, including cameras, smartphones, and tablets, are used to capture dynamic images of users;

[0068] Display devices, including computer monitors, TV screens, and mobile device screens, are used to display interactive instructions and prompt information;

[0069] Input devices, including keyboard, mouse, touch screen or voice recognition device, for users to input commands or interact;

[0070] Output devices, including speakers and vibration motors, for providing feedback or audible prompts;

[0071] It also includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the interactive method based on dynamic image recognition by calling the computer program stored in the memory.

[0072] Furthermore, a computer-readable storage medium is provided, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed on a computer, the computer is enabled to execute the interactive method based on dynamic image recognition.

[0073] Compared with the prior art, the present invention has the following beneficial effects:

[0074] (1) The present invention, by using advanced dynamic image recognition technology, can capture and analyze the user's dynamic images in real time, and accurately segment the dynamic images into segments representing different actions. Combining comprehensive analysis methods based on threshold segmentation, model segmentation, and deep learning segmentation, the system can more accurately identify the user's actions and intentions, thereby generating precise interaction instructions, significantly improving the accuracy and real-time response capabilities of the interactive system, and providing users with a more smooth and natural interactive experience;

[0075] (2) The present invention establishes a database containing representative interactive images and performs similarity matching with user action image frames to screen out the interactive instructions that best match the user actions. In addition, the system can also correct and optimize the interactive instructions based on user feedback data, such as click flow, reaction time, exit rate, and expression tracking. This adaptive adjustment mechanism based on user feedback enables the interactive system to better meet the user's personalized needs, thereby significantly improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0076] In order to facilitate understanding by those skilled in the art, the present invention is further described below in conjunction with the accompanying drawings;

[0077] Figure 1 It is a flow chart of the method in the present invention. DETAILED DESCRIPTION

[0078] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0079] It should be understood that the terms "include" and "comprising" used in the specification and claims of the present disclosure indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0080] It should also be understood that the terms used in this disclosure are only for the purpose of describing specific embodiments and are not intended to limit the disclosure. As used in this disclosure and claims, the singular forms of "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise. It should also be further understood that the term "and / or" used in this disclosure and claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations.

[0081] like Figure 1 As shown, an interactive method based on dynamic image recognition includes the following steps:

[0082] First, a camera or other image capture device is used to collect the user's dynamic image in real time. The collected dynamic image is a continuous video stream or a series of picture frames;

[0083] Then pre-process the captured dynamic images, including format conversion, resolution adjustment, denoising and color correction, to facilitate subsequent processing. Pre-processing can improve video quality and reduce errors in subsequent analysis.

[0084] The pre-processed dynamic image is segmented into dynamic image segments representing different actions based on the content. The specific process is as follows:

[0085] Based on threshold segmentation: First, the motion of continuous frames in the dynamic image sequence is estimated, and the inter-frame motion vector is calculated by the optical flow method Lucas-Kanade method or the block matching method three-frame difference method; then the inter-frame motion difference is calculated using measurement methods such as Euclidean distance or normalized mutual information; then a difference threshold is set, and when the inter-frame difference exceeds the difference threshold, it is determined that the action in the dynamic image has changed or converted; finally, according to the threshold judgment, the dynamic image is divided into multiple sections, each section represents a continuous action;

[0086] Model-based segmentation: First, extract features that are helpful for action recognition from dynamic image frames, including HOG, SIFT and 3D-CNN; then use the hidden Markov model (HMM) or dynamic time warping (DTW) model to train the feature sequence, where HMM models the action sequence through state transition probability and emission probability, while DTW aligns and recognizes actions by minimizing the distance between time series; apply the trained model to the new dynamic image sequence, and the model will recognize the action in the dynamic image based on the learned pattern; then output the action segmentation points of the video sequence based on the recognition results of the model, thereby realizing the segmentation of the action;

[0087] Based on deep learning segmentation: First, collect and annotate a large number of dynamic interactive image datasets containing different actions for training deep learning models; then determine the deep learning model, including convolutional neural network (CNN) for extracting spatial features, recurrent neural network (RNN) or long short-term memory network (LSTM) for processing time series data; use the annotated dataset to train the selected deep learning model, the model will learn how to extract features from dynamic image frames, and identify the start and end of the action based on the contextual information of the time series; then apply the trained model to new dynamic image data, the model will automatically identify and segment different action segments; finally, perform some post-processing steps, including non-maximum suppression (NMS) to remove overlapping detection boxes, and use CRF (conditional random field) to optimize the segmentation boundary;

[0088] Then conduct a comprehensive analysis of the above three segmentation methods to obtain the most accurate segmentation results. The specific process is as follows:

[0089] b1: First, the segmentation results obtained by the above three segmentation methods are normalized to ensure that the outputs of different methods can be compared and integrated under the same standard. Normalization can be a simple scaling so that all results fall within the same numerical range, or a more complex statistical normalization, such as Z-score normalization or minimum-maximum normalization;

[0090] b2: Define a fusion strategy that will comprehensively consider the segmentation results of the three methods. The fusion strategy can be a voting mechanism, weighted average, model fusion, or other more complex fusion techniques. A weight can be assigned to each method. These weights can be determined based on the performance of each method on the validation dataset.

[0091] b3: Merge the normalized segmentation results of the three segmentation methods according to the fusion strategy. For example, if weighted average is used, the weighted average position of each segmentation point can be calculated; if a voting mechanism is used, the number of occurrences of each segmentation point in the three methods can be counted, and the position with the most occurrences can be selected as the final segmentation point;

[0092] b4: Then use evaluation indicators, including accuracy, recall and F1 score, to evaluate the quality of the fusion result. Evaluation indicators can help determine the effectiveness of the fusion strategy. When the fusion result does not achieve the expected accuracy, return to b2, adjust the fusion strategy or weight, and then repeat steps b3 and b4 for iterative optimization.

[0093] Then, image frames are captured from each segment of the dynamic image that has been cut, and each captured image frame is analyzed with the representative interactive image in the database to select an image frame with the highest similarity. The specific steps are as follows:

[0094] First, for each dynamic image segment, image frames are captured at preset time intervals or based on specific action markers. These captured image frames will be used as candidate frames to be compared. Then, a database containing representative interactive images is established. The interactive images in the database are high-quality images of standard interactive actions and can represent different interactive scenes and actions. Then, feature extraction is performed on the captured image frames and the representative interactive images in the database using the following method:

[0095] Appearance features: Use SIFT, SURF and ORB algorithms to extract key points and descriptors;

[0096] Motion features: Calculate optical flow or use optical flow estimation methods to extract motion information;

[0097] Depth features: Use a depth camera or pre-processed depth data to extract features of the depth image;

[0098] Global features: Calculate the image's histogram including color and gradient or other global descriptors;

[0099] A similarity measurement method is defined to compare the similarity between several image frames and representative interactive images. The similarity measurement methods include Euclidean distance, Hamming distance, structural similarity index (SSIM) and cosine similarity. Then, several image frames captured in each dynamic image segment are compared with the similarity of all representative interactive images in the database, and the one with the highest similarity among the captured image frames is marked. Finally, the screened image frame with the highest similarity is verified to ensure that it does match the actual interactive scene.

[0100] Then, one or more corresponding continuous interaction instructions are formed according to the verified image frames and the matching representative interaction images, and the corresponding interaction commands or operations are executed.

[0101] After the execution or operation of the interactive command is completed, user feedback is obtained, and the user feedback data is analyzed and the interactive command is corrected, wherein the feedback data includes:

[0102] The user operation parameters are analyzed and judged by the user's click stream, reaction time, exit rate and expression tracking, where the click stream is the user's click command after executing the corresponding interactive command or operation, and the frequency of the last three clicks is counted and recorded as the click rate value; the reaction time is the interval time between the user's first instruction or action after executing the corresponding interactive command or operation, recorded as the response time value; the exit rate is the probability of the user's exit after executing the corresponding interactive command, and a high exit rate indicates that the user is dissatisfied with or confused about the executed interactive command; expression tracking uses a camera to collect the user's facial image in real time, uses a face detection algorithm to locate and track the face area, and then uses facial images with emotional expressions to train the expression recognition model, and the collected facial images are input into the trained expression recognition model. The model outputs the expression classification result, and then the user's expression classification result is associated with the satisfaction level. The satisfaction level is assigned a score of 1-10, and a high score indicates high user satisfaction. The obtained score is recorded as the tracking value, and this tracking value is used as the standard for measuring expression tracking;

[0103] Then, the obtained point rate value, on-time value, exit probability and table tracking value are calibrated as dl, ys, tg and bz respectively, and after normalization, they are entered into the following formula: To obtain the operation value YCZ, and use this operation value as a measure of the user's reaction state after executing the interactive command;

[0104] Historical data parameters: The number of interactions, error rate, number of retries and average time to complete a task of the interactive system are collected. The number of interactions is the total number of interactions between the user and the system. The error rate is the ratio of the number of errors encountered when executing interactive commands to the total number of attempts. A high error rate indicates that the interactive command analysis is inaccurate or the user interface is not intuitive enough. The number of retries is measured by tracking the number of times the user tries to re-execute the operation after encountering a problem. The average time to complete a task is the average time it takes for the system to execute the interactive command after the user passes the interactive command. The obtained number of interactions, error rate, number of retries and average time to complete a task are calibrated as jj, cc, ss and pp respectively, and then normalized and entered into the following formula: To obtain the calendar parameter value LCC;

[0105] After normalizing the calculated operation value and the historical parameter value, a cone model is established with the operation value as the base circle radius and the historical parameter value as the height. The center of mass of the cone model is found, and a spherical body is established with the center of mass as the sphere and the preset correction parameter as the radius. The preset correction parameter value is 1.115. The spherical body is cut out in the established cone model with the spherical body as the outline, and the volume of the remaining irregular body is calculated and recorded as the decision value.

[0106] The obtained decision value is compared with the preset decision threshold. When the decision value exceeds the preset decision threshold, it is determined that the user is not satisfied with the executed interactive command, and there may be errors in the interactive command analysis of the dynamic image. The analyzed interactive instructions are then corrected, and the different interactive signals identified by the image frame corresponding to each dynamic image segment in the same dynamic image are displayed to the interactive interface through a prompt window, prompting the user whether to execute other interactive instructions in the same dynamic image, and providing the user with the option of whether to execute, thereby avoiding the user from repeating actions to input interactive instructions.

[0107] An interactive system based on dynamic image recognition, including an image acquisition module, a preprocessing module, an action segmentation module,

[0108] The image acquisition module is used to collect the user's dynamic images, fusion analysis module, interaction instruction generation module and user feedback analysis module in real time through a camera or other image capture device; the preprocessing module is used to perform format conversion, resolution adjustment, denoising and color correction preprocessing operations on the collected dynamic images; the action segmentation module is used to perform action segmentation on dynamic images based on thresholds, models or deep learning methods; the fusion analysis module is used to normalize the results of different segmentation methods and apply fusion strategies to obtain the most accurate segmentation results; the interaction instruction generation module is used to form one or continuous interaction instructions based on the segmentation and analysis results; the user feedback analysis module is used to collect and analyze user feedback data, and make subsequent interaction instruction corrections.

[0109] An interactive terminal includes: an image capture device, including a camera, a smart phone and a tablet computer, for capturing dynamic images of a user; a display device, including a computer monitor, a TV screen and a mobile device screen, for displaying interactive instructions and prompt information; an input device, including a keyboard, a mouse, a touch screen or a voice recognition device, for a user to input instructions or interact; an output device, including a speaker and a vibration motor, for providing feedback or prompt sounds;

[0110] It also includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the interactive method based on dynamic image recognition by calling the computer program stored in the memory.

[0111] A computer-readable storage medium stores a computer program. When the computer program runs on a computer, the computer executes an interactive method based on dynamic image recognition.

[0112] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An interactive method based on dynamic image recognition, characterized in that: The following steps are involved: S1: First, use a camera or other image capture device to collect the user's dynamic image in real time; S2: pre-processing the collected dynamic images; S3: then performing action segmentation on the pre-processed dynamic image, so that the dynamic image is segmented into dynamic image segments representing different actions based on the content; S4: then intercepting image frames in each segment of the dynamic image that has been cut, and analyzing each intercepted image frame with a representative interactive image in the database to select an image frame with the highest similarity; S5: forming one or more corresponding continuous interaction instructions according to the verified image frames and the matching representative interaction images, and executing corresponding interaction commands or operations; S6: Finally, obtain user feedback, analyze the user feedback data and modify the interaction instructions. The specific process is as follows: The user feedback data obtained includes user operation parameters and historical data parameters, and the operation value and historical parameter value are obtained according to the feedback data. After normalization, the operation value is used as the bottom circle radius and the historical parameter value is used as the height to establish a cone model, and then the center of mass of the cone model is found, and the center of mass is used as the sphere, and a spherical body is established with the preset correction parameter as the radius, and then the spherical body is used as the outline to perform cutting in the established cone model, and the volume of the remaining irregular shape is calculated and recorded as the decision value; The obtained decision value is compared with the preset decision threshold. When the decision value exceeds the preset decision threshold, it is determined that the user is not satisfied with the executed interactive command, and there is an error in the interactive command analysis of the dynamic image. Then, the analyzed interactive command is corrected. The specific correction process is as follows: First, the different interactive signals identified by the image frame corresponding to each dynamic image segment in the same dynamic image are displayed to the interactive interface through a prompt window, prompting the user whether to execute other interactive instructions in the same dynamic image, and providing the user with the option of whether to execute other displayed interactive instructions. The user can decide whether to modify the interactive instructions independently.

2. The interactive method based on dynamic image recognition according to claim 1, characterized in that: The specific process in S2 is as follows: First, the action segmentation of dynamic images is performed using three segmentation methods: threshold-based segmentation, model-based segmentation, and deep learning-based segmentation. Then, the output results of the above three segmentation methods are comprehensively analyzed to obtain the most accurate segmentation results. The specific steps of the comprehensive analysis are as follows: b1: First, the segmentation results obtained by the three segmentation methods are normalized so that the outputs of different methods can be easily compared and integrated under the same standard. Normalization can be a simple scaling so that all results fall within the same numerical range, or a more complex statistical normalization; b2: Define a fusion strategy to comprehensively consider the segmentation results of the three methods. The fusion strategy can be a voting mechanism, weighted average or model fusion, or assign a weight to each method. b3: Merge the normalized segmentation results of the three segmentation methods according to the fusion strategy; b4: Then use evaluation indicators, including accuracy, recall and F1 score, to evaluate the quality of the fusion result; when the fusion result does not reach the expected accuracy, return to step b2, adjust the fusion strategy or weight, and then repeat steps b3 and b4 for iterative optimization.

3. The interactive method based on dynamic image recognition according to claim 2, characterized in that: The specific operation steps of performing action segmentation on the dynamic image by using three segmentation methods, namely, threshold-based segmentation, model-based segmentation and deep learning-based segmentation, in S2 are as follows: Based on threshold segmentation: First, motion estimation is performed on the continuous frames in the dynamic image sequence, and the inter-frame motion vector is calculated by the optical flow method Lucas-Kanade method or the block matching method three-frame difference method; Then use metrics such as Euclidean distance or normalized mutual information to calculate the inter-frame motion difference; Then a difference threshold is set, and when the difference between frames exceeds the difference threshold, it is determined that the action in the dynamic image has changed or converted; Finally, according to the threshold judgment, the dynamic image is divided into multiple sections, each section represents a continuous action; Model-based segmentation: First, features that are helpful for action recognition are extracted from dynamic image frames; Then use the hidden Markov model or dynamic time warping model to train the feature sequence; Apply the trained model to new dynamic image sequences, and the model will recognize the actions in the dynamic images based on the learned patterns; Then, based on the recognition results of the model, the action segmentation points of the video sequence are output; Based on deep learning segmentation: First, a large number of dynamic interactive image datasets containing different actions are collected and annotated for training deep learning models; Then determine the deep learning model, including convolutional neural network for extracting spatial features, recurrent neural network or long short-term memory network for processing time series data; The selected deep learning model is trained using the labeled dataset and recognizes the start and end of actions based on the contextual information of the time series; Then, the trained model is applied to new dynamic image data, and the model will automatically identify and segment different action segments; Finally, some post-processing steps are performed to remove overlapping detection boxes and optimize segmentation boundaries.

4. The interactive method based on dynamic image recognition according to claim 1, characterized in that: The specific operation steps of S4 are as follows: Firstly, for each dynamic image segment, image frames are captured according to a preset time interval or based on a specific action mark; A database containing representative interactive images is then established. The interactive images in the database are images of standard interactive actions and can represent different interactive scenes and actions. Then, feature extraction is performed on the captured image frames and representative interactive images in the database; A similarity measurement method is defined to compare the similarity between a plurality of image frames and the representative interactive image. The similarity measurement methods include Euclidean distance, Hamming distance, structural similarity index and cosine similarity; Then, the similarity of several image frames captured in each dynamic image segment is compared with all representative interactive images in the database, and the one with the highest similarity among the several captured image frames is marked; Finally, the screened image frames with the highest similarity are verified.

5. The interactive method based on dynamic image recognition according to claim 4, characterized in that: The method for extracting features from the captured image frames and representative interactive images in the database in S4 includes: Appearance features: Use SIFT, SURF and ORB algorithms to extract key points and descriptors; Motion features: Calculate optical flow or use optical flow estimation methods to extract motion information; Depth features: Use a depth camera or pre-processed depth data to extract features of the depth image; Global features: Calculate the histogram of the image including color and gradient or other global descriptors.

6. The interactive method based on dynamic image recognition according to claim 1, characterized in that: The specific operation steps of obtaining the operation value and the historical parameter value according to the feedback data in S6 are as follows: User operation parameters are analyzed and judged by user click flow, reaction time, exit rate and expression tracking; The click stream is the click commands of the user after executing the corresponding interactive command or operation, and the frequency of the last three clicks is counted and recorded as the click rate value; The reaction time is the interval time from the user's first instruction or action after executing the corresponding interactive command or operation, recorded as the response time value; The exit rate is the probability of a user exiting after executing the corresponding interactive command; Expression tracking uses a camera to collect the user's facial image in real time, uses a face detection algorithm to locate and track the face area, and then uses facial images with emotional expressions to train the expression recognition model. The collected facial images are input into the trained expression recognition model, and the model outputs the expression classification result. The user's expression classification result is then associated with the satisfaction level. The satisfaction level is assigned a score of 1-10, and the obtained score is recorded as the tracking value; Then, the obtained point rate value, on-time value, exit probability and table tracking value are calibrated as dl, ys, tg and bz respectively, and after normalization, they are entered into the following formula: To get the operation value YCZ; Historical data parameters: collect the number of interactions, error rate, number of retries, and average time to complete tasks in the interactive system; The number of interactions is the total number of interactions between the user and the system, and the error rate is the ratio of the number of errors encountered when executing interactive instructions to the total number of attempts; Retries by tracking the number of times a user tries to re-execute an action after encountering a problem; The average task completion time is the average time it takes for the system to execute the interactive command after the user passes the interactive command; The obtained number of interactions, error rate, number of retries and average time to complete the task are calibrated as jj, cc, ss and pp respectively, and then normalized and entered into the following formula: To obtain the historical parameter value LCC.

7. A system applied to the interactive method based on dynamic image recognition according to any one of claims 1 to 6, characterized in that: include: An image acquisition module is used to acquire a user's dynamic image in real time through a camera or other image capture device; The preprocessing module is used to perform format conversion, resolution adjustment, denoising and color correction preprocessing operations on the collected dynamic images; An action segmentation module is used to perform action segmentation on dynamic images based on threshold, model or deep learning methods; The fusion analysis module is used to normalize the results of different segmentation methods and apply fusion strategies to obtain the most accurate segmentation results; An interactive instruction generation module, used for forming one or continuous interactive instructions according to the segmentation and analysis results; The user feedback analysis module is used to collect and analyze user feedback data and make subsequent interactive instruction corrections.

8. An interactive terminal, characterized in that: include: Image capture devices, including cameras, smartphones, and tablets, are used to capture dynamic images of users; Display devices, including computer monitors, TV screens, and mobile device screens, are used to display interactive instructions and prompt information; Input devices, including keyboards, mice, touch screens, or voice recognition devices, for users to input commands or interact; Output devices, including speakers and vibration motors, for providing feedback or audible prompts; It also includes a processor and a memory, wherein a computer program is stored in the memory, and the processor is used to execute the interactive method based on dynamic image recognition as described in any one of claims 1 to 6 by calling the computer program stored in the memory.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the computer executes the interactive method based on dynamic image recognition according to any one of claims 1 to 6.

Images