Motion analysis method based on image data processing

Abstract

The invention discloses a motion analysis method and system based on image data processing, a medium and equipment, and relates to the technical field of image processing. Comprising the following steps: preprocessing an obtained image sequence of a motion scene to obtain a preprocessed motion image data set; extracting a moving target from the preprocessed moving image data set through background subtraction and inter-frame difference, and segmenting the moving target to obtain a moving image segmentation result; target tracking is carried out on the image motion image segmentation result, and an identifier of each target is obtained; maintaining the consistency of the targets among continuous frames by using a tracking algorithm to obtain a plurality of tracking targets; performing feature extraction on each tracking target to obtain a plurality of target features; performing feature selection, feature re-extraction and pattern recognition on the plurality of target features to obtain the type and behavior recognition result of the moving target; and a deep learning model is adopted to classify the type of the moving target and the behavior recognition result, and a motion analysis result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a motion analysis method based on image data processing.

[0002] Background Art

[0003] Motion analysis is a key technical means. By capturing and analyzing parameters such as the behavior, posture, and speed of a moving object, it provides important data support for fields such as sports science, biomechanics, medical rehabilitation, and ergonomics. In traditional motion analysis, due to technological limitations, the analysis work often relies on the subjective judgment and manual recording of coaches or researchers. This method is not only inefficient but also easily affected by factors such as the experience, perspective, and emotion of the observer, resulting in a large subjective deviation in the analysis results.

[0004] With the progress of technology, especially the rapid development of image processing technology, using image data for motion analysis has become a trend. Image processing technology can non-invasively capture detailed information of a moving object. By analyzing this information, the motion performance can be evaluated more objectively and accurately. However, although the application prospect of image processing technology in motion analysis is broad, in actual operation, it still faces the problem of how to accurately identify and separate the moving object from the background in a complex motion scene, especially in the case of light changes, occlusion, and fast movement. Summary of the Invention

[0005] In order to overcome the above existing drawbacks, the main object of the present invention is to provide a motion analysis method based on image data processing.

[0006] To achieve the above object, the present invention adopts the following technical solutions. A motion analysis method based on image data processing includes the following steps:

[0007] Preprocess the image sequence of the motion scene obtained to obtain a preprocessed motion image data set;

[0008] Extract moving targets from the preprocessed motion image data set through background subtraction and inter-frame difference, and segment the moving targets to obtain the motion image segmentation result;

[0009] Perform target tracking on the motion image segmentation result, assign a unique identifier to each target; use the tracking algorithm to maintain the consistency of the target between consecutive frames to obtain a number of tracked targets;

[0010] Extract features for each of the tracked targets respectively to obtain a number of target features;

[0011] Perform feature selection, feature re - extraction, and pattern recognition on the several target features to obtain the type of the moving target and the behavior recognition result;

[0012] Use a deep - learning model to classify the type and behavior recognition result of the moving target to obtain a motion classification result as the motion analysis result.

[0013] The moving - image segmentation result includes:

[0014] Create a background model based on the Gaussian model;

[0015] Subtract the background model from the current frame and compare the differences between consecutive frames to extract the foreground moving target;

[0016] Convert the extracted foreground into a binary image;

[0017] Use dilation and erosion to remove noise and fill holes in the binary image to obtain a filled image;

[0018] Segment the connected regions of the filled image into independent moving targets through connected - component labeling and region growing.

[0019] The pre - processing includes the following steps:

[0020] Obtain an image sequence of the motion scene from the collected motion video;

[0021] Use median filtering and Gaussian filtering to remove image noise from the image sequence of the motion scene to obtain a denoised image dataset;

[0022] Use histogram equalization to adjust the contrast and brightness of the denoised image dataset to obtain an image - enhanced dataset;

[0023] Adjust the image size of the image - enhanced dataset and correct the geometric distortion of the image to obtain a pre - processed motion - image dataset.

[0024] The obtaining of several tracking targets specifically includes:

[0025] Execute an object - detection algorithm to recognize the moving - image segmentation result, obtain the moving targets in the initial frame of the video sequence, and assign a unique identifier to each detected target;

[0026] Perform object tracking through Kalman filtering, particle filtering, and a deep - learning tracker in sequence to obtain an object - tracking result;

[0027] Use the Hungarian algorithm to match the detected targets with the existing tracking targets in consecutive frames to obtain a data - association result:

[0028] Update the position, velocity, and other attributes of the moving object according to the data association result to obtain a number of tracked objects.

[0029] The object features include: appearance features, motion features, and spatio-temporal features;

[0030] The appearance features include the color, texture, and shape of the object;

[0031] The motion features include the velocity, acceleration, and trajectory of the object;

[0032] The spatio-temporal features include the object in time and space.

[0033] The feature selection includes:

[0034] Select important features from the several object features using the correlation coefficient and information gain to obtain important features;

[0035] The feature re-extraction includes:

[0036] Use PCA and LDA to perform feature re-extraction on the important features to obtain re-extracted features;

[0037] The pattern recognition includes:

[0038] Use machine learning algorithms to perform preliminary classification on the re-extracted features to obtain the type and behavior recognition results of the moving object.

[0039] A motion analysis system based on image data processing, comprising:

[0040] A data acquisition module; used to preprocess the image sequence of the motion scene obtained to obtain a preprocessed motion image data set;

[0041] An image feature acquisition module, used to extract moving objects from the preprocessed motion image data set through background subtraction and inter-frame difference, segment the moving objects to obtain the image motion image segmentation result; perform object tracking on the image motion image segmentation result, assign a unique identifier to each object; use a tracking algorithm to maintain the consistency of the object between consecutive frames to obtain a number of tracked objects;

[0042] A feature analysis module, used to perform feature extraction on each of the tracked objects respectively to obtain a number of object features; perform feature selection, feature re-extraction, and pattern recognition on the number of object features to obtain the type and behavior recognition results of the moving object;

[0043] A motion result analysis module, used to classify the type and behavior recognition results of the moving object using a deep learning model to obtain a motion classification result as the motion analysis result.

[0044] Compared with the prior art, the beneficial effects of the present invention are as follows: Through multi-level processing of motion image data for denoising, enhancement, and geometric correction, the present invention improves the accuracy of motion target extraction and recognition. At the same time, the target tracking algorithm ensures the consistency of target tracking, avoids recognition errors caused by the rapid movement or occlusion of moving targets, and improves the accuracy and robustness of motion analysis. By adopting a variety of target tracking and target recognition technologies, the system can adapt to different motion scenarios and complex backgrounds. This way of multi-algorithm cooperation enhances the stability and scalability of the system in a variety of application scenarios, and enhances the adaptability and scalability of the system. Through target tracking technologies such as Kalman filtering and particle filtering, the target position and attributes can be updated in real time in consecutive frames, effectively reducing the computational complexity, ensuring the efficient operation of the system in applications with high real-time requirements, and improving the computational efficiency and real-time performance. By combining with deep learning models for classification, the types and behaviors of moving targets can be more accurately identified, supporting more complex behavior analysis and applications, and improving the behavior recognition accuracy of moving targets BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of the present application and form a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation of the present application

[0046] Figure 1 is a schematic structural diagram of the process of the present invention

[0047] Figure 2 is a schematic process diagram of obtaining the motion image segmentation result of the present invention

[0048] Figure 3 is a schematic process diagram of obtaining several tracking targets of the present invention DETAILED DESCRIPTION OF THE INVENTION

[0049] With the progress of technology, especially the rapid development of image processing technology, using image data for motion analysis has become a trend. Image processing technology can non-invasively capture detailed information of moving objects. By analyzing this information, the motion performance can be evaluated more objectively and accurately. However, although the application prospect of image processing technology in motion analysis is broad, in actual operation, it still faces the problem of how to accurately identify and separate moving objects from the background in complex motion scenarios, especially in the case of light changes, occlusion, and rapid movement

[0050] To achieve the above object, the present invention adopts the following technical solutions. A motion analysis method based on image data processing, refer to Figure 1 , including the following steps

[0051] Preprocess the obtained image sequence of the motion scene to obtain a preprocessed motion image dataset;

[0052] Successively perform background subtraction and inter-frame difference on the preprocessed motion image dataset, extract moving objects, segment the moving objects, and obtain a motion image segmentation result;

[0053] Perform object tracking on the motion image segmentation result, assign a unique identifier to each object; use a tracking algorithm to maintain the consistency of the object between consecutive frames, and obtain a number of tracked objects;

[0054] Extract features for each of the tracked objects respectively to obtain a number of object features;

[0055] Perform feature selection, feature re-extraction, and pattern recognition on the number of object features to obtain the type and behavior recognition results of the moving objects;

[0056] Use a deep learning model to classify the type and behavior recognition results of the moving objects to obtain a motion classification result, which is the motion analysis result.

[0057] The present invention will be further described below in conjunction with the drawings and embodiments.

[0058] Embodiment:

[0059] In this embodiment, during a football game, the actions of the players are analyzed. Refer to Figures 1-3 , which details each step of the motion analysis method based on image data processing:

[0060] Use a high-definition camera to record the real-time video of the football game to obtain an image sequence; then apply median filtering to remove the noise generated due to light changes or camera jitter, and denoise the image: in image enhancement, improve the contrast and brightness of the image through histogram equalization, making it easier to distinguish the players and the ball. And scale the image to a unified size, and correct the geometric distortion caused by the perspective to achieve image size adjustment and geometric correction.

[0061] Use a Gaussian model to create a background model, ignoring the fixed scene elements. Subtract the background model from the current frame, and compare the differences between consecutive frames through inter-frame difference to extract the moving objects, which are the players and the ball. Convert the extracted foreground into a binary image for subsequent processing. Apply dilation and erosion operations to remove noise and small holes to ensure the integrity of the moving objects. Segment the connected regions in the binary image into independent moving objects through connected component labeling and region growing algorithms.

[0062] Detect moving objects in the initial frame and assign a unique identifier to each object. Then use Kalman filter, particle filter, and deep learning tracker in sequence to track the objects in consecutive frames. Apply the Hungarian algorithm to match the detected objects with the existing tracked objects and update the positions and attributes of the objects.

[0063] Then calculate the color histogram, texture, and shape descriptors of each tracked object for appearance feature extraction, and then proceed with motion feature extraction, including calculating the speed, acceleration, and motion trajectory of the object. And spatio-temporal feature extraction is also required, including analyzing the changes of the object in time and space, such as using the optical flow method.

[0064] Use correlation coefficient and information gain for feature selection to select the features that contribute most to classification. Then apply PCA and LDA to reduce the dimension and re-extract the selected features. Use the machine learning algorithm SVM to perform preliminary classification on the features to identify the action types and behaviors of the players.

[0065] Use the trained CNN model to perform fine classification on the preliminary classification results to obtain more accurate player action recognition results, and output the classification results, such as whether the player is running, passing, shooting, etc., as the results of motion analysis.

[0066] Through precise preprocessing and feature extraction, the accuracy of action recognition is improved. The fast processing speed enables the system to analyze the actions of players in real time during the game. The entire process is highly automated, reducing manual intervention. It provides rich player action data, which helps coaches and analysts conduct tactical analysis and player performance evaluation.

[0067] Embodiment 2

[0068] Motion analysis methods based on image data processing are widely used in fields such as intelligent monitoring, sports analysis, autonomous driving, and security monitoring. Through the processing of image sequences, moving objects can be accurately identified and tracked, providing a basis for further analysis and decision-making. This embodiment demonstrates how to apply this method to an intelligent monitoring system to analyze the behavior patterns of the crowd in real time.

[0069] A set of high-definition cameras are installed at the entrance of a certain shopping mall, and the collected video data will be transmitted to the central processing unit for real-time processing. The goal is to detect and track the dynamics of the crowd in the shopping mall through motion analysis methods, and identify potential abnormal behaviors such as running fast, pushing, staying, etc., so as to provide abnormal warnings. The analysis of crowd behavior in intelligent monitoring specifically includes:

[0070] A video data sequence is obtained from a high-definition camera at the mall entrance, with a resolution of 1920x1080 and a frame rate of 30 frames per second. Assume the video capture lasts for 10 minutes, capturing 18000 frames of image data. Salt noise in the video images is removed by median filtering (window size 3x3), and Gaussian filtering (standard deviation 1.0) is applied to remove high-frequency noise in the images. After this processing, a denoised image dataset is obtained. Histogram equalization is performed on the denoised image dataset to adjust brightness and contrast, enhancing the contrast between people and the background for subsequent processing. The images are resized (uniformly to 640x360) and geometric distortion is corrected to ensure spatial consistency during subsequent motion analysis.

[0071] The Gaussian Mixture Model (GMM) is used to create a background model. Considering the possible slight illumination changes and occasional moving backgrounds such as door switches in the mall, the background model adopts a dynamic update method. Foreground moving objects are extracted through background subtraction and frame difference methods. The foreground objects are quickly separated using the difference between the background model and the current frame. After dilation and erosion operations to remove noise, a relatively clear binary image is obtained. Connected component analysis is applied to the binary image to identify and label moving objects, and the connected foreground regions are segmented to obtain independent segmented images of each moving object.

[0072] The YOLO or SSD object detection algorithm is used to identify crowd objects in each frame and assign a unique identifier (ID) to each object. For the detected objects, the Kalman Filter and Particle Filter are sequentially applied for motion tracking. These algorithms help accurately predict the motion trajectories of the objects and maintain object consistency between consecutive frames. The Hungarian algorithm is used to match the detected objects with the existing tracked objects to precisely track the motion trajectories of each object in each frame.

[0073] For each object, the following features are extracted:

[0074] Appearance features: the clothing colors of the crowd, the details of the clothing, the contour shapes of the objects.

[0075] Motion features: calculate the pixel distance / time the object moves, the rate of change of the object's speed, the object's motion path.

[0076] Spatio-temporal features: such as the coordinates and time information of the object in the image, the object appears within a specific time period.

[0077] Use information gain or mutual information methods to select multiple features and screen out the most discriminative features such as color, speed, trajectory, etc. Use principal component analysis (PCA) or linear discriminant analysis (LDA) to perform dimensionality reduction on the features, mapping the important features to a low-dimensional space for subsequent analysis. Behavior recognition: Use SVM to classify the extracted and dimensionally reduced features to identify the behavior types of the targets. For example, determine whether it is an abnormal behavior such as pushing or running.

[0078] Deep learning model. In this embodiment, a convolutional neural network (CNN) is selected to classify the recognized behaviors, such as "normal walking", "running", "gathering", or "loitering", etc. If abnormal behaviors such as running or pushing are detected, the system will automatically trigger an alarm to remind the staff to intervene. The system will display the categories and behaviors of the moving targets in real time on the screen of the monitoring center, and highlight the abnormal targets with different colors or identifiers. For example, if someone starts running fast, the system will mark the target with a red box and display the words "Abnormal behavior: running".

[0079] The image dataset in this embodiment: 30 frames per second, video duration of 10 minutes, a total of 18,000 frames. The size of each frame of the image is 640x360, and the data volume is 18,000×640×360×3 bytes.

[0080] The target tracking range is in the mall entrance area, and at most 50 target pedestrians can appear in the monitoring screen at the same time. Through the tracking algorithm, the system successfully maintains the movement trajectories of 50 targets and calculates the speed and acceleration of each target.

[0081] In this embodiment, system analysis is carried out, and 3 abnormal behavior events are detected, which are respectively:

[0082] Event 1: 3 people quickly gathered at the entrance of a certain store, and the system detected the gathering behavior.

[0083] Event 2: 1 person suddenly started running from the crowd, and the system detected the running behavior and triggered an alarm.

[0084] Event 3: 1 person loitered in the corner of the mall for more than 5 minutes, and the system detected the loitering behavior and triggered a warning.

[0085] Through this series of steps, the motion analysis method based on image data can effectively monitor and analyze the behaviors of the crowd in real-time scenarios. By combining deep learning with traditional algorithms, the robustness and accuracy of the system in different scenarios are improved. It can not only track targets in real time but also accurately identify abnormal behaviors, providing effective security protection for places such as shopping malls.

[0086] It should be noted that in the present invention, relative terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprising", "including" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device.

[0087] The above embodiments are only illustrative examples of the present invention and do not constitute a limitation on the protection scope of the present invention. Any design identical or similar to the present invention falls within the protection scope of the present invention.

Claims

1. A motion analysis method based on image data processing, characterized in that, It includes the following steps: Preprocess the acquired image sequence of the motion scene to obtain a preprocessed motion image dataset; Successively perform background subtraction and inter-frame difference on the preprocessed motion image dataset, extract moving objects, segment the moving objects, and obtain a motion image segmentation result; Perform object tracking on the motion image segmentation result and assign a unique identifier to each object; Use a tracking algorithm to maintain the consistency of the object between consecutive frames and obtain a number of tracked objects; Extract features for each of the tracked objects respectively to obtain a number of object features; Perform feature selection, feature re-extraction, and pattern recognition on the number of object features to obtain the type and behavior recognition result of the moving object; Use a deep learning model to classify the type and behavior recognition result of the moving object to obtain a motion classification result, which is the motion analysis result.

2. The motion analysis method based on image data processing according to claim 1, characterized in that The obtaining of the motion image segmentation result includes the following steps: Create a background model based on the Gaussian model to obtain a background model; Subtract the background model from the current frame and compare the differences between consecutive frames to extract foreground moving objects; Convert the extracted foreground moving objects into a binary image; Use dilation and erosion to remove noise and fill holes in the binary image to obtain a filled image; Segment the connected regions of the filled image into independent moving objects through connected component labeling and region growing to obtain a motion image segmentation result.

3. The motion analysis method based on image data processing according to claim 1, wherein, The preprocessing includes the following steps: Obtain the image sequence of the motion scene from the collected motion video; Use median filtering and Gaussian filtering to remove image noise from the image sequence of the motion scene to obtain a denoised image dataset; Use histogram equalization to adjust the contrast and brightness of the denoised image dataset to obtain an image enhancement dataset; Adjust the image size of the image enhancement dataset and correct the geometric distortion of the image to obtain a preprocessed motion image dataset.

4. The motion analysis method based on image data processing according to claim 1, characterized in that The obtaining of a number of tracked objects specifically includes: Execute an object detection algorithm to recognize the motion image segmentation result of the figure, obtain the moving objects in the initial frame of the video sequence, and assign a unique identifier to each detected object; Successively perform object tracking on the moving objects through Kalman filtering, particle filtering, and a deep learning tracker to obtain an object tracking result; Use the Hungarian algorithm to match the detected object tracking result with the existing tracked objects in consecutive frames to obtain a data association result; According to the data association result, update the position, speed, and other attributes of the moving object to obtain a number of tracked objects.

5. The motion analysis method based on image data processing according to claim 1, wherein The object features include: appearance features, motion features, and spatio-temporal features; The appearance features include the color, texture, and shape of the object; The motion features include the speed, acceleration, and trajectory of the object; The spatio-temporal features include the object in time and space.

6. The motion analysis method based on image data processing according to claim 1, wherein, The feature selection includes: Use the correlation coefficient and information gain to select important features from the number of object features to obtain important features; The feature re-extraction includes: Use PCA and LDA to perform feature re-extraction on the important features to obtain re-extracted features; The pattern recognition includes: Use a machine learning algorithm to preliminarily classify the re-extracted features and obtain the type and behavior recognition results of the moving target.

7. A motion analysis system based on image data processing, characterized in that, It includes: A data acquisition module; Used to preprocess the image sequence of the motion scene obtained to obtain a preprocessed motion image data set; An image feature acquisition module, used to extract moving targets from the preprocessed motion image data set through background subtraction and inter-frame difference, segment the moving targets to obtain the moving image segmentation result; perform target tracking on the moving image segmentation result and assign a unique identifier to each target; Use a tracking algorithm to maintain the consistency of the target between consecutive frames and obtain a number of tracked targets; A feature analysis module, used to extract features for each of the tracked targets respectively to obtain a number of target features; Perform feature selection, feature re-extraction and pattern recognition on the number of target features to obtain the type and behavior recognition results of the moving target; A motion result analysis module, used to classify the type and behavior recognition results of the moving target by using a deep learning model to obtain a motion classification result as the motion analysis result.

8. A computer-readable storage medium, characterized in that The storage medium stores a computer program, and when the computer program is executed by a processor, it implements the method according to any one of claims 1 to 6 above.

9. A computer device, characterized in that, It includes a memory, a processor and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the method according to any one of claims 1 to 6 above.

Images