Attendance method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose an attendance recording method and device, electronic equipment and storage medium, the method comprising: obtaining a gait sequence graph of an employee from a collected walking video of the employee; obtaining a gait gradient direction histogram feature and a gait sequence gravity center feature of the employee based on the gait sequence graph; performing feature correlation fusion on the gait gradient direction histogram feature and the gait sequence gravity center feature through kernel canonical correlation analysis, to obtain a gait fusion feature vector of the employee; and recording the attendance of the employee based on an identity label of the employee determined based on the gait fusion feature vector. The technical solution of the present application avoids the loss of effective gait features of the employee when performing feature correlation fusion on the gait gradient direction histogram feature and the gait sequence gravity center feature, solves the problem of incorrect attendance recording caused by the loss of effective gait features of the employee, improves the accuracy of attendance recording, and achieves the purpose of simplifying the attendance process and reducing the workload of the attendance personnel.

Description

Technical Field

[0001] The embodiments of the present invention relate to image processing technology, and more particularly to an attendance method, device, electronic device, and storage medium. Background Technology

[0002] Currently, employee attendance can be recorded by training and classifying employees' gait features to obtain employee identification tags. For example, gait features can be extracted manually and classified using a Support Vector Machine (SVM). Alternatively, deep learning methods can be employed to identify employee gait features and obtain employee identification tags; convolutional neural networks (CNNs) are typically used.

[0003] Using the SVM method to classify gait features has limitations. When dealing with manually extracted high-dimensional, non-linear gait features, the feature fusion method employs both serial and parallel fusion at the feature layer. This only changes the combination of gait features without considering the relationships between them, leading to the loss of effective gait features. This results in low accuracy in classifying employee gait features, causing errors in attendance records. While deep learning eliminates the need for manual gait feature extraction, effective gait features are still lost during convolutional and pooling layers. This insufficient gait feature richness leads to low accuracy in identifying employee gait features, again resulting in inaccurate attendance records. Summary of the Invention

[0004] This invention provides an attendance recording method, device, electronic device, and storage medium, which can solve the problem of incorrect attendance records caused by the loss of employees' effective gait characteristics, improve the accuracy of attendance records, automate and simplify the attendance process, thereby reducing the workload of attendance personnel.

[0005] In a first aspect, embodiments of the present invention provide an attendance recording method, the method comprising:

[0006] Collect walking videos of employees during their walking process, and obtain gait sequence diagrams of the employees from the walking videos;

[0007] Based on the gait sequence diagram, the gait gradient direction histogram features and gait sequence centroid features of the employee are obtained;

[0008] The gait gradient direction histogram features and the gait sequence centroid features are fused using kernel canonical correlation analysis to obtain the employee's gait fusion feature vector.

[0009] The employee's identity label is determined based on the gait fusion feature vector;

[0010] The employee's attendance is recorded based on the identity tag.

[0011] Secondly, embodiments of the present invention provide an attendance device, the device comprising:

[0012] The gait sequence map acquisition module is used to collect walking videos of employees during their walking process and to acquire the gait sequence map of the employees from the walking videos.

[0013] The feature acquisition module is used to obtain the gait gradient direction histogram features and gait sequence centroid features of the employee based on the gait sequence map;

[0014] The feature fusion module is used to perform feature association and fusion of the gait gradient direction histogram features and the gait sequence centroid features through the kernel canonical correlation analysis method to obtain the employee's gait fusion feature vector;

[0015] An identity label determination module is used to determine the employee's identity label based on the gait fusion feature vector;

[0016] The attendance recording module is used to record the employee's attendance based on the identity tag.

[0017] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the attendance recording method as described in any of the embodiments of the present invention.

[0018] Fourthly, embodiments of the present invention also provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the attendance recording method as described in any of the embodiments of the present invention.

[0019] The technical solution of this invention can collect walking videos of employees and obtain gait sequence maps from the videos; based on the gait sequence maps, obtain the gait gradient direction histogram features and gait sequence center of gravity features of the employees; use kernel canonical correlation analysis to perform feature association and fusion of the gait gradient direction histogram features and gait sequence center of gravity features to obtain the gait fusion feature vector of the employees; determine the employee's identity label based on the gait fusion feature vector; and record the employee's attendance according to the identity label. The technical solution of this invention, based on the gait sequence map of employees obtained from walking videos, obtains the gait gradient direction histogram features and gait sequence centroid features of employees. Kernel canonical correlation analysis is used to focus on the correlation between the gait gradient direction histogram features and gait sequence centroid features in a common space. The gait gradient direction histogram features and gait sequence centroid features are then fused to obtain a gait fusion feature vector in the common space. This results in a richer gait feature vector after feature fusion, and employee attendance is recorded based on the employee's identity label determined by the gait fusion feature vector. This avoids the loss of effective gait features when fusing gait gradient direction histogram features and gait sequence centroid features, solving the problem of incorrect attendance records due to the loss of effective gait features, improving the accuracy of attendance records, automating and simplifying the attendance process, and thus reducing the workload of attendance personnel. Attached Figure Description

[0020] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 A flowchart illustrating the attendance recording method provided in an embodiment of the present invention;

[0022] Figure 2 A schematic diagram of an attendance method provided in an embodiment of the present invention;

[0023] Figure 3 This is another flowchart illustrating the attendance recording method provided in an embodiment of the present invention;

[0024] Figure 4 A schematic diagram of the gait gradient direction histogram features used to generate the gait energy map in the attendance recording method provided in this embodiment of the invention;

[0025] Figure 5A schematic diagram illustrating the calculation of gait fusion feature vectors in the attendance recording method provided in this embodiment of the invention;

[0026] Figure 6 A schematic diagram illustrating the determination of an employee's identity tag in the attendance recording method provided in an embodiment of the present invention;

[0027] Figure 7 A schematic diagram of the attendance recording device provided in an embodiment of the present invention;

[0028] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0029] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0030] Figure 1 This is a flowchart illustrating an attendance recording method provided in an embodiment of the present invention. The method can be executed by an attendance device provided in this embodiment, which can be implemented using software and / or hardware. In a specific embodiment, the device can be integrated into an electronic device, such as a computer or server. The following embodiments will illustrate this using the integration of the device into an electronic device as an example. Figure 1 The method may specifically include the following steps:

[0031] Step 101: Collect walking videos of employees during their walk and obtain gait sequence diagrams of employees from the walking videos.

[0032] Among them, gait can be understood as the posture of a person walking; gait sequence diagram can be understood as the posture diagram of an employee walking in one cycle of a walking video.

[0033] In one alternative implementation, a video capture device, such as a camera, can be used to capture walking videos of employees from different perspectives, ensuring the identification of employees in the captured videos from different viewpoints. After obtaining the employee's walking video, it can be segmented into multiple frames to obtain multi-frame walking video images. These multi-frame walking video images then undergo image preprocessing.

[0034] Image preprocessing of multi-frame walking video images may include:

[0035] 1) Denoise the multi-frame walking video images to obtain denoised multi-frame walking video images, which can reduce noise interference from video acquisition equipment and the external environment.

[0036] 2) Extract motion target information (posture information of employees during walking) from multi-frame walking video images, remove invalid information from the images, and obtain the first candidate gait map. This can reduce the interference of invalid information when processing and analyzing the denoised multi-frame walking video images in the future, and improve the accuracy of obtaining gait sequence maps.

[0037] 3) Since the first candidate gait map obtained by extracting the target information may contain holes and burrs, it is necessary to repair the missing features of the employee walking process in the first candidate gait map to obtain the second candidate gait map.

[0038] 4) Normalize the second candidate gait map to obtain the third candidate gait map. This will ensure that the position and size of the moving target (employee) in the second candidate gait map of different sizes are consistent in the third candidate gait map. This will make it easier to obtain the gait gradient direction histogram features and gait sequence center of gravity features of the employee more quickly and accurately based on the gait sequence map.

[0039] Finally, the gait sequence of an employee within one cycle can be obtained by calculating the width-to-height ratio of the moving target in the third candidate gait image. The gait sequence within one cycle can be understood as a series of candidate gait sequences between two candidate gait sequences in the third candidate gait image where the width-to-height ratio of the moving target is the same.

[0040] For example, Figure 2 A schematic diagram of the attendance method provided in an embodiment of the present invention, as shown below. Figure 2 As shown, a video capture device can be used to capture walking videos of employees, and then an image processor can be used to preprocess multiple frames of walking video images to obtain a third candidate gait map. Finally, a gait sequence extractor can be used to calculate the ratio of the width to the height of the moving target in the third candidate gait map to obtain the gait sequence map of the employee in one cycle.

[0041] Step 102: Based on the gait sequence diagram, obtain the gait gradient direction histogram features and gait sequence centroid features of the employees.

[0042] Among them, the gait gradient orientation histogram feature can be understood as the feature of the histogram of oriented gradient (HOG) of the gait sequence image; HOG can be understood as a feature extraction algorithm in image processing, where image features can be represented by the directional gradient of each pixel. The gait sequence centroid feature can be understood as the amplitude information of the ordinate change curve of the employee's centroid in the frequency domain in the gait sequence image.

[0043] In one optional implementation, the gait sequence image can be binarized to obtain a binary gait sequence image. The pixel values ​​of each pixel in the binary gait sequence image can be averaged to obtain a Gait Energy Image (GEI). The color space of the GEI image is then normalized. The GEI can then be divided into multiple image blocks, and each image block can be further divided into multiple units. Gradient direction histogram (HOG) statistics are performed on each unit within an image block, and these HOGs are concatenated to obtain the HOG of the corresponding image block. The HOGs of all image blocks are then concatenated to obtain the complete gait gradient direction histogram feature of the GEI. This complete gait gradient direction histogram feature is the gait gradient direction histogram feature vector extracted through the GEI. The GEI can be understood as an image obtained by summing and averaging the pixel values ​​of the binary gait sequence image within one period.

[0044] The ordinate of the employee's center of gravity can be extracted from the gait sequence diagram to obtain the curve of the change of the ordinate of the employee's center of gravity during the walking process in the time domain. Since the change of the ordinate of the employee's center of gravity in the time domain is not obvious, the extracted curve of the change of the ordinate of the employee's center of gravity in the time domain can be transformed into the frequency domain to obtain the curve of the change of the ordinate of the employee's center of gravity in the frequency domain. Finally, the amplitude information in the frequency domain is taken as the feature vector, and the feature vector is determined as the center of gravity feature of the gait sequence.

[0045] For example, such as Figure 2 As shown, the GEI (Gazette Energy Image) can be obtained by averaging the pixel values ​​of each pixel in the gait sequence image using a gait energy map generator. Then, a gradient orientation calculator is used to normalize the color space of the gait energy map. The gradient orientation calculator then divides the GEI into multiple image blocks, and each image block is further divided into multiple units. The gradient orientation calculator performs gradient orientation histogram statistics on each unit within an image block and concatenates them to obtain the HOG (Histogram of Oriented Gradients) for that image block. Finally, the gradient orientation calculator concatenates the HOGs of all image blocks to obtain the gait gradient orientation histogram feature.

[0046] like Figure 2As shown, the ordinate of the employee's center of gravity in the gait sequence diagram can be extracted using an employee center of gravity extractor to obtain the curve of the change of the ordinate of the employee's center of gravity during the employee's walking process in the time domain. Since the change of the ordinate of the employee's center of gravity in the time domain is not obvious, the extracted curve of the change of the ordinate of the employee's center of gravity in the time domain can be converted by a center of gravity frequency domain converter to obtain the curve of the change of the ordinate of the employee's center of gravity in the frequency domain. Finally, the amplitude information in the frequency domain is used as a feature vector, and the feature vector is determined as the center of gravity feature of the gait sequence.

[0047] Step 103: The gait gradient direction histogram features and gait sequence centroid features are fused using the kernel canonical correlation analysis method to obtain the employee's gait fusion feature vector.

[0048] Among them, nuclear canonical correlation analysis can be understood as a method for analyzing and characterizing two sets of features that have a nonlinear relationship.

[0049] In one alternative implementation, a kernel canonical correlation analyzer can be used to calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and the gait sequence centroid features, and then construct a feature association parameter kernel matrix based on the various feature association parameters; then, a feature association fusion kernel matrix can be constructed based on the gait gradient direction histogram features and the gait sequence centroid features, and finally, the feature vectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix are calculated to obtain the gait fusion feature vector of the employee.

[0050] For example, such as Figure 2 As shown, a feature association fusion kernel matrix can be constructed using a feature fusion builder based on the gait gradient direction histogram features and the gait sequence centroid features. Then, the feature fusion builder calculates the feature association parameter kernel matrix and the feature association fusion kernel matrix's eigenvectors to obtain the employee's gait fusion feature vector.

[0051] Step 104: Determine the employee's identity label based on the gait fusion feature vector.

[0052] The identity tag can be understood as the employee's name, such as the employee's name.

[0053] For example, such as Figure 2 As shown, the gait fusion feature vector can be trained and classified using a feature trainer to obtain the employee's identity label, and finally the employee's identity label can be output through a label outputter.

[0054] Step 105: Record employee attendance based on identity tags.

[0055] In one optional implementation, employee identification tags can be matched with preset identification tags in the attendance system. If an employee's identification tag matches a preset identification tag in the attendance system, it can be determined that the employee is present, and the attendance of the employee corresponding to the identification tag can be recorded. Here, preset identification tags can be understood as all employee identification tags stored in the attendance system.

[0056] In this embodiment, based on the gait sequence map of employees obtained from walking videos, the gait gradient direction histogram features and gait sequence centroid features of employees are obtained. Kernel canonical correlation analysis is used to focus on the correlation between the gait gradient direction histogram features and gait sequence centroid features in the common space. The gait gradient direction histogram features and gait sequence centroid features are fused to obtain a gait fusion feature vector of gait gradient direction histogram features and gait sequence centroid features in the common space. This makes the gait fusion feature vector containing richer gait features. Then, based on the employee's identity label determined by the gait fusion feature vector, employee attendance is recorded. This avoids the situation where the employee's effective gait features are lost when the gait gradient direction histogram features and gait sequence centroid features are fused, thus solving the problem of incorrect attendance records due to the loss of effective gait features of employees, improving the accuracy of attendance records, realizing the automation of the attendance process and simplifying the attendance process, thereby reducing the workload of attendance personnel.

[0057] The attendance recording method provided in the embodiments of the present invention is further described below, such as... Figure 3 As shown, Figure 3 Another flowchart illustrating the attendance recording method provided in this embodiment of the invention may specifically include the following steps:

[0058] Step 201: Collect walking videos of employees during their walking process and obtain gait sequence diagrams of employees from the walking videos.

[0059] Step 202: Generate a gait energy map based on the gait sequence map, and divide the gait energy map into multiple image blocks. The gait energy map includes the pixel values ​​of each point in each frame of video image.

[0060] In one alternative implementation, GEI can be generated according to the following formula:

[0061]

[0062] Where x and y represent pixel information, N can represent the total number of frames in the gait sequence map within one period, and B i (x,y) can represent the information of the pixels in the gait sequence map of the i-th frame.

[0063] Step 203: Calculate the gradient and gradient direction of each image block based on the pixel values.

[0064] For example, Figure 4 This is a schematic diagram of the gait gradient direction histogram features used to generate the gait energy map in the attendance recording method provided in this embodiment of the invention, as shown below. Figure 4 As shown, after standardizing the color space of the gait energy map, the gradient and gradient direction of each unit can be calculated based on the pixel value of each unit in each image block. Then, gradient direction histogram statistics can be performed based on the gradient direction of each unit. Finally, the gradients of each unit can be concatenated based on the gradient direction of each unit to obtain the gradient and gradient direction of each image block.

[0065] Step 204: Segment the gradients of each image block according to the gradient direction to obtain the gait gradient direction histogram feature of the gait energy map.

[0066] For example, such as Figure 4 As shown, gradient direction histogram statistics can be performed based on the gradient direction of each image block, and the gradients of each image block can be concatenated based on the gradient direction of each image block to obtain the complete gradient direction histogram features of GEI, which is the HOG feature vector extracted by GEI.

[0067] Step 205: Determine the gait gradient direction histogram features as the gait gradient direction histogram features of the employees.

[0068] In an alternative implementation, the complete HOG features of the obtained GEI can be determined as the gait gradient orientation histogram features of the employee.

[0069] Step 206: Extract the ordinate of the employee during the walking process from the gait sequence to obtain the ordinate change curve in the time domain.

[0070] Step 207: Perform frequency domain transformation on the ordinate change curve to obtain the ordinate change curve in the frequency domain.

[0071] In one alternative implementation, a Fast Fourier Transform (FFT) can be used to convert the ordinate change curve of the employee's centroid in the time domain into a ordinate change curve in the frequency domain.

[0072] Step 208: Determine the amplitude information in the vertical axis change curve in the frequency domain as the center of gravity feature of the employee's gait sequence.

[0073] Step 209: Using the kernel canonical correlation analysis calculator, calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and gait sequence centroid features, and construct the feature association parameter kernel matrix based on the various feature association parameters.

[0074] Step 210: Construct a feature association fusion kernel matrix based on the gait gradient direction histogram features and the gait sequence centroid features.

[0075] In one optional implementation, it can be determined whether the label corresponding to the feature category of the gait gradient orientation histogram feature is a first label, and whether the label corresponding to the feature category of the gait sequence centroid feature is a second label. When the label corresponding to the feature category of the gait gradient orientation histogram feature is the first label and the label corresponding to the feature category of the gait sequence centroid feature is the second label, a feature association fusion kernel matrix is ​​constructed based on the gait gradient orientation histogram feature and the gait sequence centroid feature. This avoids the label corresponding to the feature category of the gait gradient orientation histogram feature not matching the first label, and avoids the label corresponding to the feature category of the gait sequence centroid feature not matching the second label, which would lead to incorrect employee identity labels obtained when training and classifying the gait fusion feature vector, thus improving the accuracy of determining employee identity labels based on the gait fusion feature vector. Here, the first label can be understood as the preset association label of the feature category of the gait gradient orientation histogram feature, and the second label can be understood as the preset association label of the feature category of the gait sequence centroid feature.

[0076] For example, such as Figure 2 As shown, a feature validator can determine whether the label corresponding to the feature category of the gait gradient direction histogram feature is the first label, and whether the label corresponding to the feature category of the gait sequence centroid feature is the second label. When the label corresponding to the feature category of the gait gradient direction histogram feature is the first label and the label corresponding to the feature category of the gait sequence centroid feature is the second label, as shown... Figure 5 As shown, the gait gradient direction histogram features and gait sequence centroid features are mapped to a high-dimensional space sample matrix to obtain the feature association fusion kernel matrix.

[0077] Step 211: The Lagrange multiplier method is used to calculate the eigenvectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix to obtain the gait fusion feature vector of the employee.

[0078] For example, such as Figure 2 As shown, the kernel canonical correlation analysis calculator is used to calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and gait sequence centroid features. The feature association parameter kernel matrix constructed based on these parameters is shown in the following formula. σ and ε are the correlation parameters of each feature obtained by the kernel canonical correlation analysis calculator, and T represents the transpose of the kernel matrix of each feature correlation parameter.

[0079] Based on the gait gradient direction histogram features and the gait sequence centroid features, a feature association fusion kernel matrix is ​​constructed as follows: (The formula is missing from the provided text.) Among them, K a K is the kernel matrix of the gait gradient orientation histogram features. b This is the kernel matrix of the centroid features of the gait sequence.

[0080]

[0081] like Figure 5 As shown, the Lagrange multiplier method can be used to calculate the eigenvectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix according to the above formula, so as to obtain the gait fusion feature vector c of the employee.

[0082] In this embodiment, a kernel canonical correlation analysis calculator is used to calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and the gait sequence centroid features. A feature association parameter kernel matrix is ​​then constructed based on these parameters, and a feature association fusion kernel matrix is ​​constructed based on the gait gradient direction histogram features and the gait sequence centroid features. The Lagrange multiplier method is used to calculate the eigenvectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix, resulting in the employee's gait fusion feature vector. This maximizes the correlation between the gait gradient direction histogram features and the gait sequence centroid features in the gait fusion feature vector, focusing on the correlation between these two features in the common space. This results in a richer gait feature vector after feature association, and employee attendance is recorded based on the employee's identity label determined by the gait fusion feature vector. This avoids the loss of effective gait features when performing feature association fusion of the gait gradient direction histogram features and the gait sequence centroid features, thus improving the accuracy of attendance records.

[0083] Step 212: Determine the employee's identity label based on the gait fusion feature vector.

[0084] Among them, the label of the gait gradient direction histogram feature is the first label, and the label of the gait sequence centroid feature is the second label.

[0085] In one alternative implementation, Figure 6 This is a schematic diagram illustrating the method for determining an employee's identity tag in the attendance recording method provided in this embodiment of the invention, as shown below. Figure 6 As shown, the gait fusion feature vector can be input into the feature trainer for feature training and classification (random allocation of hidden layer parameters, calculation of hidden layer output matrix, and determination of output weights) to obtain and output the training label of the gait fusion feature vector; when the training label is the same as the first label and the training label is the same as the second label, the training label is determined as the employee's identity label. Figure 2 As shown, after determining the employee's identity label through the feature trainer, the employee's identity label can be output through the label outputter.

[0086] Here, the training labels can be understood as the employee identity labels obtained after training and classification of gait fusion feature vectors. The feature trainer can be an Extreme Learning Machine (ELM). ELM can be understood as a classification algorithm based on a single hidden layer feedforward neural network. It randomly generates connection weights between the input layer and the hidden layer, and trains the network using the least squares method. Only the number of hidden layer nodes needs to be adjusted to obtain a unique optimal solution. A single hidden layer feedforward network can be represented as:

[0087]

[0088] Among them, g(W i ·X j +b i ) is the activation function, L is the number of hidden layer nodes, and W is the activation function. i For the input weights, β i For the output weights, b i This represents the bias of the i-th hidden layer unit. X represents the input vector.

[0089] In this embodiment, an over-limit learning machine is used to train and classify the gait fusion feature vectors. This can shorten the feature training time, improve the recognition efficiency of the training labels of the gait fusion feature vectors, and also improve the classification accuracy of the gait fusion feature vectors, thereby improving the accuracy of attendance records.

[0090] Step 213: Record employee attendance based on identity tags.

[0091] For example, if Figure 2 The identity tag output by the tag outputter is Zhang San, which is achieved through... Figure 2 The attendance statistics tool in the system counts the identity tags; then, it matches the identity tag (Zhang San) counted by the attendance statistics tool with the preset identity tag (Zhang San) in the attendance system. If the identity tag matches the preset identity tag in the attendance system successfully, it can be determined that the employee Zhang San is at work, and the attendance of the employee Zhang San corresponding to the identity tag can be recorded. This can simplify the attendance process, reduce the workload of attendance personnel, and improve the efficiency of attendance recording.

[0092] In this embodiment, based on the gait sequence map of employees obtained from walking videos, the gait gradient direction histogram features and gait sequence centroid features of employees are obtained. Kernel canonical correlation analysis is used to focus on the correlation between the gait gradient direction histogram features and gait sequence centroid features in the common space. The gait gradient direction histogram features and gait sequence centroid features are fused to obtain a gait fusion feature vector of gait gradient direction histogram features and gait sequence centroid features in the common space. This makes the gait fusion feature vector containing richer gait features. Then, based on the employee's identity label determined by the gait fusion feature vector, employee attendance is recorded. This avoids the situation where the employee's effective gait features are lost when the gait gradient direction histogram features and gait sequence centroid features are fused, thus solving the problem of incorrect attendance records due to the loss of effective gait features of employees, improving the accuracy of attendance records, realizing the automation of the attendance process and simplifying the attendance process, thereby reducing the workload of attendance personnel.

[0093] Figure 7 This is a schematic diagram of an attendance recording device provided in an embodiment of the present invention. This device is suitable for executing the attendance recording method provided in an embodiment of the present invention. Figure 7 As shown, the device may specifically include:

[0094] The gait sequence acquisition module 301 is used to collect walking videos of employees during their walking process and to acquire the gait sequence map of the employees from the walking videos.

[0095] The feature acquisition module 302 is used to obtain the gait gradient direction histogram features and gait sequence centroid features of the employee based on the gait sequence map;

[0096] The feature fusion module 303 is used to perform feature association fusion of the gait gradient direction histogram features and the gait sequence centroid features through the kernel canonical correlation analysis method to obtain the employee's gait fusion feature vector;

[0097] The identity label determination module 304 is used to determine the employee's identity label based on the gait fusion feature vector;

[0098] The attendance recording module 305 is used to record the employee's attendance based on the identity tag.

[0099] Optionally, the feature acquisition module 302 is specifically used for:

[0100] A gait energy map is generated based on the gait sequence map, and the gait energy map is divided into multiple image blocks, wherein the gait energy map includes the pixel values ​​of each point in each frame of video image;

[0101] Calculate the gradient and gradient direction of each image block based on the pixel values;

[0102] By stitching together the gradients of each image block according to the gradient direction, the gait gradient direction histogram feature of the gait energy map is obtained;

[0103] The gait gradient direction histogram features are determined as the gait gradient direction histogram features of the employee.

[0104] Optionally, the feature acquisition module 302 is specifically used for:

[0105] Extract the ordinate of the employee during the walking process from the gait sequence to obtain the ordinate change curve in the time domain;

[0106] The ordinate change curve is transformed in the frequency domain to obtain the ordinate change curve in the frequency domain;

[0107] The amplitude information in the ordinate change curve in the frequency domain is determined as the center of gravity feature of the employee's gait sequence.

[0108] Optionally, the feature fusion module 303 is specifically used for:

[0109] The kernel canonical correlation analysis calculator is used to calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and the gait sequence centroid features, and a feature association parameter kernel matrix is ​​constructed based on the various feature association parameters;

[0110] Construct a feature association fusion kernel matrix based on the gait gradient direction histogram features and the gait sequence centroid features;

[0111] The eigenvectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix are calculated using the Lagrange multiplier method to obtain the gait fusion feature vector of the employee.

[0112] Optionally, the feature fusion module 303 constructs a feature association fusion kernel matrix based on the gait gradient direction histogram features and the gait sequence centroid features, including:

[0113] The gait gradient direction histogram features and the gait sequence centroid features are mapped to a high-dimensional space sample matrix to obtain the feature association fusion kernel matrix.

[0114] Optionally, the label of the gait gradient direction histogram feature is a first label, and the label of the gait sequence centroid feature is a second label. The feature fusion module 303 is specifically used for:

[0115] Determine whether the label corresponding to the feature category of the gait gradient direction histogram feature is the first label, and determine whether the label corresponding to the feature category of the gait sequence centroid feature is the second label;

[0116] When the label corresponding to the feature category of the gait gradient direction histogram feature is the first label and the label corresponding to the feature category of the gait sequence centroid feature is the second label, the step of constructing a feature association fusion kernel matrix based on the gait gradient direction histogram feature and the gait sequence centroid feature is triggered.

[0117] Optionally, the identity tag determination module 304 is specifically used for:

[0118] The gait fusion feature vector is input into the feature trainer for feature training to obtain the training label of the gait fusion feature vector;

[0119] When the training label is the same as the first label and the training label is the same as the second label, the training label is determined as the employee's identity label.

[0120] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional modules is merely an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. The specific working process of the functional modules described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0121] The device in this embodiment obtains the gait sequence map of employees acquired from walking videos, and then obtains the gait gradient direction histogram features and gait sequence centroid features of the employees. Kernel canonical correlation analysis is used to focus on the correlation between the gait gradient direction histogram features and gait sequence centroid features in a common space. The gait gradient direction histogram features and gait sequence centroid features are then fused to obtain a gait fusion feature vector in the common space. This results in a richer gait feature vector after feature fusion, and employee attendance is recorded based on the employee's identity tag determined by the gait fusion feature vector. This avoids the loss of valid gait features when fusing gait gradient direction histogram features and gait sequence centroid features, solving the problem of incorrect attendance records due to the loss of valid gait features, improving the accuracy of attendance records, automating and simplifying the attendance process, and thus reducing the workload of attendance personnel.

[0122] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the attendance recording method provided in any of the above embodiments.

[0123] This invention also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processor, implements the attendance recording method provided in any of the above embodiments.

[0124] The following is for reference. Figure 8 It shows a schematic diagram of the structure of a computer system 500 suitable for implementing an electronic device according to embodiments of the present invention. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0125] like Figure 8 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) 502 or programs loaded from storage section 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the computer system 500. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0126] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to I / O interface 505 as needed. A removable medium 511, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 510 as needed so that computer programs read from it can be installed into storage section 508 as needed.

[0127] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 509, and / or installed from removable medium 511. When the computer program is executed by central processing unit (CPU) 501, it performs the functions defined above in the system of this invention.

[0128] It should be noted that the computer-readable medium shown in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this invention, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this invention, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0129] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The modules and / or units described in the embodiments of the present invention can be implemented in software or hardware. The described modules and / or units can also be housed in a processor; for example, a processor can be described as including a gait sequence map acquisition module, a feature acquisition module, a feature fusion module, an identity tag determination module, and an attendance record module. The names of these modules do not necessarily constitute a limitation on the module itself.

[0131] In another aspect, the present invention also provides a computer-readable medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable medium carries one or more programs, which, when executed by the device, cause the device to include: acquiring walking videos of employees during their walking process and obtaining gait sequence maps of the employees from the walking videos; obtaining gait gradient direction histogram features and gait sequence centroid features of the employees based on the gait sequence maps; performing feature association fusion of the gait gradient direction histogram features and gait sequence centroid features using a kernel canonical correlation analysis method to obtain a gait fusion feature vector of the employees; determining the employee's identity label based on the gait fusion feature vector; and recording the employee's attendance according to the identity label.

[0132] According to the technical solution of the present invention, based on the gait sequence map of employees obtained from walking videos, the gait gradient direction histogram features and gait sequence centroid features of employees are obtained. Kernel canonical correlation analysis is used to focus on the correlation between the gait gradient direction histogram features and gait sequence centroid features in a common space. The gait gradient direction histogram features and gait sequence centroid features are then fused to obtain a gait fusion feature vector in the common space. This results in a richer gait feature vector after feature fusion, and employee attendance is recorded based on the employee's identity label determined by the gait fusion feature vector. This avoids the loss of effective gait features when fusing gait gradient direction histogram features and gait sequence centroid features, solving the problem of incorrect attendance records due to the loss of effective gait features, improving the accuracy of attendance records, automating and simplifying the attendance process, and thus reducing the workload of attendance personnel.

[0133] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An attendance method, characterized in that, The method includes: Collect walking videos of employees during their walking process, and obtain gait sequence diagrams of the employees from the walking videos; Based on the gait sequence diagram, the gait gradient direction histogram features and gait sequence centroid features of the employee are obtained; The gait gradient direction histogram features and the gait sequence centroid features are fused using kernel canonical correlation analysis to obtain the employee's gait fusion feature vector. The employee's identity label is determined based on the gait fusion feature vector; The employee's attendance is recorded based on the identity tag; the gait gradient direction histogram features and gait sequence centroid features are fused using kernel canonical correlation analysis to obtain the employee's gait fusion feature vector, including: The kernel canonical correlation analysis calculator is used to calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and the gait sequence centroid features, and a feature association parameter kernel matrix is ​​constructed based on the various feature association parameters; Construct a feature association fusion kernel matrix based on the gait gradient direction histogram features and the gait sequence centroid features; The eigenvectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix are calculated using the Lagrange multiplier method according to the following formula, thereby obtaining the gait fusion feature vector of the employee; ； Where c is the gait fusion feature vector of the employee. and These are the feature correlation parameters obtained through a kernel canonical correlation analysis calculator. This represents the transpose of the kernel matrix of the correlation parameters of each feature. The kernel matrix is ​​the feature of the gait gradient direction histogram. The kernel matrix represents the centroid features of the gait sequence. The label of the gait gradient orientation histogram feature is the first label, and the label of the gait sequence centroid feature is the second label. Before constructing the feature association fusion kernel matrix based on the gait gradient orientation histogram feature and the gait sequence centroid feature, the method further includes: Determine whether the label corresponding to the feature category of the gait gradient direction histogram feature is the first label, and determine whether the label corresponding to the feature category of the gait sequence centroid feature is the second label; When the label corresponding to the feature category of the gait gradient direction histogram feature is the first label and the label corresponding to the feature category of the gait sequence centroid feature is the second label, the step of constructing a feature association fusion kernel matrix based on the gait gradient direction histogram feature and the gait sequence centroid feature is triggered.

2. The method according to claim 1, characterized in that, The step of obtaining the gait gradient orientation histogram features of the employee based on the gait sequence map includes: A gait energy map is generated based on the gait sequence map, and the gait energy map is divided into multiple image blocks, wherein the gait energy map includes the pixel values ​​of each point in each frame of video image; Calculate the gradient and gradient direction of each image block based on the pixel values; By stitching together the gradients of each image block according to the gradient direction, the gait gradient direction histogram feature of the gait energy map is obtained; The gait gradient direction histogram features are determined as the gait gradient direction histogram features of the employee.

3. The method according to claim 1, characterized in that, The process of obtaining the employee's gait sequence center of gravity features based on the gait sequence includes: Extract the ordinate of the employee during the walking process from the gait sequence to obtain the ordinate change curve in the time domain; The ordinate change curve is transformed in the frequency domain to obtain the ordinate change curve in the frequency domain; The amplitude information in the ordinate change curve in the frequency domain is determined as the center of gravity feature of the employee's gait sequence.

4. The method according to claim 1, characterized in that, The step of constructing a feature association fusion kernel matrix based on the gait gradient orientation histogram features and the gait sequence centroid features includes: The gait gradient direction histogram features and the gait sequence centroid features are mapped to a high-dimensional space sample matrix to obtain the feature association fusion kernel matrix.

5. The method according to claim 1, characterized in that, The process of determining the employee's identity label based on the gait fusion feature vector includes: The gait fusion feature vector is input into the feature trainer for feature training to obtain the training label of the gait fusion feature vector; When the training label is the same as the first label and the training label is the same as the second label, the training label is determined as the employee's identity label.

6. An attendance device, characterized in that, The device includes: The gait sequence map acquisition module is used to collect walking videos of employees during their walking process and to acquire the gait sequence map of the employees from the walking videos. The feature acquisition module is used to obtain the gait gradient direction histogram features and gait sequence centroid features of the employee based on the gait sequence map; The feature fusion module is used to perform feature association and fusion of the gait gradient direction histogram features and the gait sequence centroid features through the kernel canonical correlation analysis method to obtain the employee's gait fusion feature vector; An identity label determination module is used to determine the employee's identity label based on the gait fusion feature vector; The attendance recording module is used to record the employee's attendance based on the identity tag; The feature fusion module is specifically used for: The kernel canonical correlation analysis calculator is used to calculate the various feature association parameters required for feature association fusion based on the gait gradient direction histogram features and the gait sequence centroid features, and a feature association parameter kernel matrix is ​​constructed based on the various feature association parameters; Construct a feature association fusion kernel matrix based on the gait gradient direction histogram features and the gait sequence centroid features; The eigenvectors of the feature association parameter kernel matrix and the feature association fusion kernel matrix are calculated using the Lagrange multiplier method according to the following formula, thereby obtaining the gait fusion feature vector of the employee; ； Where c is the gait fusion feature vector of the employee. and These are the feature correlation parameters obtained through a kernel canonical correlation analysis calculator. This represents the transpose of the kernel matrix of the correlation parameters of each feature. The kernel matrix is ​​the feature of the gait gradient direction histogram. The kernel matrix represents the centroid features of the gait sequence. The label of the gait gradient orientation histogram feature is the first label, and the label of the gait sequence centroid feature is the second label. Before constructing the feature association fusion kernel matrix based on the gait gradient orientation histogram feature and the gait sequence centroid feature, the method further includes: Determine whether the label corresponding to the feature category of the gait gradient direction histogram feature is the first label, and determine whether the label corresponding to the feature category of the gait sequence centroid feature is the second label; When the label corresponding to the feature category of the gait gradient direction histogram feature is the first label and the label corresponding to the feature category of the gait sequence centroid feature is the second label, the step of constructing a feature association fusion kernel matrix based on the gait gradient direction histogram feature and the gait sequence centroid feature is triggered.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the attendance method as described in any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the attendance method as described in any one of claims 1 to 5.

Images