Digital runway imaging system
By installing a camera on the runway and analyzing user movements using Kalman filters and face recognition modules, the problem of lack of real-time data analysis on the existing runway is solved, scientific management and personalized suggestions for user movements are achieved, and the applicability of the system is improved.
Patent Information
- Application Number
- CN202510402418.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-01
- Publication Date
- 2025-07-18
AI Technical Summary
The lack of real-time motion data analysis and personalized recommendations on existing runways leads to unscientific and sustainable sports management.
The camera is used to collect user motion pictures, combine Kalman filters and face recognition modules, and obtain motion types and action defects by analyzing the user's motion pictures and face recognition, and provide targeted suggestions.
It realizes scientific and standardized management of user movements and improves the applicability of digital runway imaging systems.
Smart Images
Figure CN120324862A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of sports equipment, and in particular, to a digital runway imaging system. Background Art
[0002] With the popularization of health knowledge, more and more people like sports. Appropriate exercise can not only improve people's physical quality, but also improve their mood. Among various sports, running, as an aerobic exercise, can exercise the body in all aspects and is the choice of many people. In the current running tracks, there are usually only tracks and no other equipment. Therefore, during running, only the runner's own perception and motor ability can be relied on to judge the motion state, and real-time motion data analysis and personalized suggestions cannot be provided, resulting in the lack of scientific and standardized management of existing tracks, lack of sustainability, and easy waste of resources. Summary of the Invention
[0003] To overcome the above disadvantages, the purpose of the present invention is to provide a digital runway imaging system. By using a Kalman filter to analyze the action of the user's motion pictures captured by a camera, the motion that the user is doing and the defects in the action can be obtained. By using a face recognition module to recognize the face on the user's motion pictures captured by the camera, the face can be corresponding in continuous user motion pictures, and then different users can be corresponding to the motions they make, which is beneficial to providing targeted suggestions for different users, improving the scientific and standardized management of the user's actions, and enhancing the applicability of the digital runway imaging system.
[0004] To achieve the above purpose, the technical solution adopted by the present invention is: A digital runway imaging system, comprising:
[0005] A camera for collecting user motion pictures;
[0006] A Kalman filter, in which a motion data set and a standard action model are stored. The Kalman filter is used to record the motion of key points to obtain motion data. The motion data is used to compare with the motion data set to obtain the human body posture. The standard action model is used to compare with the human body posture to obtain action defects;
[0007] A face recognition module connected to the Kalman filter. The face recognition module includes an OpenCV model, and a first function is set in the face recognition module for recognizing the face in the picture;
[0008] A state prediction module connected to the Kalman filter. The state prediction module includes a state prediction equation: X k= A(X k - 1) - w k ; wherein,
[0009] X k is the state matrix;
[0010] A is the state transition matrix;
[0011] w k is the process noise vector.
[0012] In this technical solution, the camera is used to collect pictures of the user's movement, so as to facilitate the analysis of the user's identity and the actions the user is performing based on the pictures of the user's movement; the Kalman filter is used to analyze the human body contour structure obtained from consecutive recognition frames of the user's movement video, and compare it with the movement data set stored internally to obtain the type of movement the user is performing, and compare the user's movement data with the standard action model stored internally to obtain the defects existing in the movement actions made by the user, which is beneficial for correcting the user; the face recognition module conducts face recognition by using the first function, so as to analyze and distinguish the faces captured by the camera, so as to correspond the actions made by different people in the same picture, and then effectively track and give suggestions on the movements made by each person; by using the Kalman filter to analyze the actions in the pictures of the user's movement captured by the camera, the movement the user is doing and the defects in the actions can be obtained. By using the face recognition module to recognize the faces in the pictures of the user's movement captured by the camera, the faces can be corresponded in consecutive pictures of the user's movement, and then different users can be corresponded with the movements they make, which is beneficial for providing targeted suggestions for different users, improving the scientific and standardized management of the user's actions, and enhancing the applicability of the digital runway imaging system; the state prediction equation is used to predict the user's actions, so as to facilitate the analysis of the user's actions.
[0013] In some embodiments, the face recognition module further includes a second function, and the second function is used to create different types of face recognition instances.
[0014] In this technical solution, the second function is used to create different types of face recognition instances, which is beneficial for the face recognition module to perform recognition and analysis on faces of different feature types, and then improve the performance of the face recognition module.
[0015] In some embodiments, the face recognition module further includes a third function, and the third function is used to train the face recognition module by using a public face library and corresponding face feature tags.
[0016] In this technical solution, the third function is used to train the face recognition module, so as to improve the recognition and analysis performance of the face recognition module for faces.
[0017] In some embodiments, the face recognition module further includes a fourth function, and the fourth function is used to perform face recognition on a single picture or an array of pictures and return the predicted similarity.
[0018] In this technical solution, the fourth function is used to identify and match the faces in two adjacent recognition frame pictures, so as to match the actions made by different users.
[0019] In some embodiments, the state prediction module further includes an observation equation: Z k = HX k - v k ; where:
[0020] Z k is the observation matrix;
[0021] H is the measurement matrix;
[0022] v k is the measurement noise.
[0023] In this technical solution, the observation equation is used to correct the errors of the sensor and the camera.
[0024] In some embodiments, the digital runway imaging system further includes a human body detection module. The Kalman filter is connected to the human body detection module. A convolutional neural network model is set in the human body detection module, and the convolutional neural network model is used to perform convolutional processing on the recognition frames of the user's motion video to obtain feature maps.
[0025] In this technical solution, the human body detection module is used to detect the outline of the human body in the picture, which is beneficial to judging the actions made by the user.
[0026] In some embodiments, the digital runway imaging system further includes a key point detection module. The human body detection module is connected to the key point detection module. The key point detection module is provided with an OpenPose model and a multi-level network. The OpenPose model is used to obtain the complete human body outline from the feature map, and the multi-level network is used to generate the distribution of the key points and the key point connection information according to the human body outline.
[0027] In this technical solution, the key points are used to connect with each other to form a human body outline.
[0028] In some embodiments, the multi-level network includes a pose tracking network, and the pose tracking network is connected to the human body detection module.
[0029] In this technical solution, the pose tracking network is used to analyze and obtain the coordinates of the key points of the human body contour in the recognition frame, and transmit the key point coordinates to the human body detection module
[0030] The beneficial effect of the present invention is that by using the Kalman filter to analyze the action of the user motion picture captured by the camera, the motion that the user is doing and the defects in the action can be obtained. By using the face recognition module to recognize the face on the user motion picture captured by the camera, the face can be corresponded in the continuous user motion pictures, and then different users can be corresponded with the motions they do, which is beneficial to providing targeted suggestions for different users, improving the scientific and standardized management of the user's actions, and enhancing the applicability of the digital runway imaging system. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a schematic diagram of the overall structure of the digital runway imaging system according to an embodiment of the present invention; DETAILED DESCRIPTION OF THE EMBODIMENTS
[0032] The following describes in detail the preferred embodiments of the present invention with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.
[0033] A digital runway imaging system includes:
[0034] A camera, which is used to collect user motion pictures, so as to facilitate the analysis of the user's identity and the actions that the user is performing based on the user motion pictures;
[0035] A Kalman filter, in which a motion data set and a standard action model are stored. The Kalman filter is used to record the motion of the key points to obtain motion data. The motion data is used to compare with the motion data set to obtain the human body pose, and the standard action model is used to compare with the human body pose to obtain action defects;
[0036] For example, the Kalman filter identifies the motion data of the user based on the change of the human body contour structure in the continuous recognition frames, and then compares it with the motion data set stored internally to obtain the human body pose of the user, that is, the type of motion that the user is doing. Then, the human body pose of the user is compared with the standard action model of the corresponding motion type stored internally to obtain the defects existing in the user's motion. For example, after obtaining the continuous actions of the user running, the continuous actions of the user are compared with the motion data set stored internally to confirm that the action of the user is "running". Then, the running action of the user is compared with the "running" action in the standard action model stored internally to obtain the defects in the user's running action.
[0037] Control the operation of the Kalman filter using the following code:
[0038] # Define the Kalman filter class
[0039] class KalmanFilter:
[0040] def __init__(self, A, B, H, Q, R, Z1):
[0041] self.A = A # State transition matrix
[0042] self.B = B # Control input matrix
[0043] self.H = H # Observation matrix
[0044] self.Q = 0 # Process noise covariance matrix
[0045] self.R = R # Observation noise covariance matrix
[0046] # Initialize the state estimate and error covariance matrix
[0047] self.X_hat = Z1 # Initial state estimate is the first observation
[0048] self.P = np.eye(3) # Initial error covariance matrix, diagonal values are 1, others are 0
[0049] def predict(self, U_k):
[0050] # State prediction
[0051] self.X_hat_pred = np.dot(self.A, self.X_hat) + np.dot(self.B, U_k) # Error covariance matrix prediction
[0052] self.P_pred = np.dot(np.dot(self.A, self.P), self.A.T) + self.Q
[0053] return self.X_hat_pred, self.P_pred
[0054] def update(self, Zk):
[0055] # Calculate the Kalman gain
[0056] It should be noted that there seems to be a syntax error in the original code snippet you provided, specifically in line 46 where there is a missing closing parenthesis in `self.P_pred = np.dot(np.dot(self.A,self.P),self.A.T+self.Q`. The corrected code in the translation is adjusted accordingly.S = np.dot(self.H, np.dot(self.P_pred, self.H.T)) + self.R
[0057] K = np.dot(np.dot(self.P_pred, self.H.T), np.linalg.inv(S))
[0058] # Update state estimation
[0059] self.X_hat = self.X_hat_pred + np.dot(K, (Z_k - np.dot(self.H, self.X_hat_predy)))。
[0060] Face recognition module, the face recognition module is connected to the Kalman filter. The face recognition module may include a face recognizer, and a face recognizer is used to analyze human faces. The face recognition module includes an OpenCV model. A first function is set in the face recognition module. The first function is: CascadeClassifier.detectMultiScale(image, scaleFactor = 1.1, minNeighbors = 5, flags = 0, minSize = (30, 30), maxSize = ()); the first function is used to recognize human faces in pictures;
[0061] For example, Haar features or LBP features are used to train the face recognition module. The face recognition module uses Haar features or LBP features and the first function to recognize specific objects (such as human faces) in pictures. The first function returns a list of rectangles according to the recognition results, and each rectangle represents the position of the detected specific object.
[0062] State prediction module, the state prediction module is connected to the Kalman filter. The state prediction module includes a state prediction equation: X k = A(X k -1) - w k ; where
[0063] X k is the state matrix, and the state transition of X k (described by matrix A) predicts the user's action trend at the next moment, providing a dynamic data basis for subsequent comparison with the standard action model;
[0064] A is the state transition matrix, that is, the matrix representing the transition of the human body posture from the previous state to the next state. For example, among two adjacent recognition frames, the transition of the human body posture from the previous recognition frame to the next recognition frame;
[0065] w k is the process noise vector, which represents the prediction error.
[0066] In some embodiments, the face recognition module further includes a second function, and the second function is: face.LBPHFaceRecognizer_create(), EigenFaceRecognizer_create(), FisherFaceRecognizer_create(). The second function is used to create face recognition instances of different types.
[0067] For example, use the second function to construct recognizers for local binary pattern histograms, eigenfaces, and Fisher faces. The face recognizer is a functional module in the system that integrates multiple face recognition instances (such as LBPH, EigenFace). By calling different recognizers, diverse face analyses can be achieved.
[0068] In some embodiments, the face recognition module further includes a third function, and the third function is: face_recognizer.train(src, labels). The third function is used to train the face recognition module using a public face database and corresponding face feature labels. First, input the image dataset of the public face database and the face feature label corresponding to each image (such as user ID). By extracting face features (such as Haar features, LBP histograms) and performing supervised learning (such as SVM, deep learning), a mapping relationship from face features to labels is established, so that the trained model can recognize new faces and return the matching label and similarity.
[0069] The third function uses the pictures in the public face database and the corresponding face feature labels in the pictures to sequentially detect the pictures in the public face database by the face recognition module, and combines the feature labels, thereby playing a training role on the face recognition module.
[0070] In some embodiments, the face recognition module further includes a fourth function, and the fourth function is:
[0071] face_recognizer.predict(src). The fourth function is used to perform face recognition on a single picture or an array of pictures and return the predicted similarity.
[0072] By using the predicted similarity, it is possible to determine whether the faces in multiple pictures are of the same person. By setting a threshold, if the similarity exceeds or is equal to the threshold, it is determined that the faces in the multiple pictures are of the same person; if the similarity does not exceed the threshold, it means that the faces in the multiple pictures are of different people. For example, if the threshold is set to 80% and the similarity returned by the face recognition module is 85%, it is determined that the faces recognized by the face recognition module are of the same person.
[0073] In some embodiments, the state prediction module further includes an observation equation: Z k = HX k - v k ; where:
[0074] Z k is the observation matrix, that is, the matrix composed of relevant data is the actual measurement data (such as joint coordinates), which directly comes from the sensor;
[0075] H is the measurement matrix, which maps the state variables (such as position, velocity) to observable physical quantities (such as position). Xk contains three-dimensional joint coordinates, and H can project it onto the two-dimensional image plane;
[0076] v k is the measurement noise, that is, the error of the observation equation.
[0077] In some embodiments, the digital runway imaging system further includes a human body detection module. The Kalman filter is connected to the key point detection module. A convolutional neural network model is set in the human body detection module, and the convolutional neural network model is used to perform convolutional processing on the recognition frames of the user's motion video to obtain feature maps.
[0078] For example, the convolutional neural network model can adopt the YOLOv5 architecture, the input resolution is set to 640×640, and the confidence threshold is set to 0.5". After obtaining the pictures of the user's motion, in order to facilitate the analysis of the user's motion, it is necessary to analyze the user's actions frame by frame. Therefore, convolutional processing is performed on each recognition frame of the user's motion picture to obtain the feature maps corresponding to all recognition frames.
[0079] In some embodiments, the digital runway imaging system further includes a key point detection module. The human body detection module is connected to the key point detection module. The key point detection module is provided with an OpenPose model and a multi-level network. The OpenPose model is used to obtain the complete human body contour from the feature map, and the multi-level network is used to generate the distribution of the key points and the key point connection information according to the human body contour.
[0080] For example, based on the feature map, obtaining the human body position and bounding box of the user in the recognition frame is beneficial for analyzing the distribution of key points and the key point connection information of the user in the recognition frame, so as to obtain the human body contour structure according to the key points and the key point connection information, and then analyze and process the actions that the user is performing according to the human body contour structure. At the same time, the key points and the key point connection information can be further optimized. There are 33 key points, which are: 0 nose, 1 inner left eye, 2 left eye, 3 outer left eye, 4 inner right eye, 5 right eye, 6 outer right eye, 7 left ear, 8 right ear, 9 left part of the mouth, 10 right part of the mouth, 11 left shoulder, 12 right shoulder, 13 left elbow, 14 right elbow, 15 left wrist, 16 right wrist, 17 left little finger, 18 right little finger, 19 left hand, 20 right hand, 21 left thumb, 22 right thumb, 23 left hip joint, 24 right hip joint, 25 left knee, 26 right knee, 27 left ankle, 28 right ankle, 29 left heel, 30 right heel, 31 left foot, 32 right foot. The key point connection information is, for example: 0 nose is connected to 1 inner left eye and 4 inner right eye; after obtaining the key points and the key point connection information, the non-maximum suppression processing step can be performed on the key points and the key point connection information to optimize the key point detection result of the human body contour structure and filter out redundant and incorrect detections.
[0081] In some embodiments, the multi-level network includes a pose tracking network, and the pose tracking network is connected to the human body detection module, and the human body detection module is used to detect whether there is a human body in the recognition frame.
[0082] In summary, the present invention provides a digital runway imaging system. By using a Kalman filter to analyze the actions of the user motion pictures captured by the camera, the motion that the user is performing and the defects in the actions can be obtained. By using the face recognition module to recognize the face on the user motion pictures captured by the camera, the faces can be corresponding in the continuous user motion pictures, and then different users can be corresponding to the motions they perform, which is beneficial to providing targeted suggestions for different users, improving the scientific and standardized management of the user's actions, and improving the applicability of the digital runway imaging system.
[0083] The above embodiments are only used to illustrate the technical concept and features of the present invention. The purpose is to enable those who are familiar with this technology to understand the content of the present invention and implement it, and it cannot be used to limit the protection scope of the present invention. All equivalent changes or modifications made according to the spirit of the present invention should be covered within the protection scope of the present invention.
Claims
1. A digital runway imaging system, characterized in that, Including: A camera for collecting pictures of the user's movement; A Kalman filter in which a motion data set and a standard action model are stored. The Kalman filter is used to record the movement of key points to obtain motion data. The motion data is used to compare with the motion data set to obtain the human body posture, and the standard action model is used to compare with the human body posture to obtain action defects; A face recognition module connected to the Kalman filter. The face recognition module includes an OpenCV model, and a first function is set in the face recognition module for identifying human faces in pictures; A state prediction module connected to the Kalman filter. The state prediction module includes a state prediction equation: Xk = A(Xk-1) - wk; where, Xk is a state matrix; A is a state transition matrix; wk is a process noise vector.
2. The digital runway imaging system according to claim 1, wherein The face recognition module further includes a second function for creating face recognition instances of different types.
3. According to claim 1, characterized in that, The face recognition module further includes a third function for training the face recognition module using a public face library and corresponding face feature labels.
4. The digital runway imaging system according to claim 1, characterized in that, The face recognition module further includes a fourth function for performing face recognition on a single picture or an array of pictures and returning the predicted similarity.
5. The digital runway imaging system according to claim 1, wherein The state prediction module further includes an observation equation: Zk = HXk - vk; where: Zk is an observation matrix; H is a measurement matrix; vk is a measurement noise.
6. The digital runway imaging system according to claim 1, wherein The digital runway imaging system further includes a human body detection module. The Kalman filter is connected to the human body detection module. A convolutional neural network model is set in the human body detection module for performing convolutional processing on the recognition frames of the user's movement video to obtain a feature map.
7. The digital runway imaging system according to claim 6, wherein The digital runway imaging system further includes a key point detection module. The human body detection module is connected to the key point detection module. The key point detection module is provided with an OpenPose model and a multi-level network. The OpenPose model is used to obtain a complete human body contour from the feature map, and the multi-level network is used to generate the distribution of key points and key point connection information according to the human body contour.
8. The digital runway imaging system according to claim 7, wherein, The multi-level network includes a pose tracking network connected to the human body detection module.