Gait Feature Extraction Method and Device

Abstract

The present application discloses a gait feature extraction method and apparatus. Among them, the gait feature extraction method includes: obtaining an image sequence collected by a target object; determining a gait contour map sequence, a sequence of skeletal key points, and a body part division map sequence of the target object based on the image sequence; and extracting gait features of the target object based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object. Through the gait feature extraction method of the present application, relatively excellent gait features with rich motion details can be extracted.

Description

Technical Field

[0001] This application relates to the technical field of feature extraction, and particularly to a gait feature extraction method and device. Background Art

[0002] With the development of technology, gait features have become one of the important biometric features concerned in the field of recognition. Therefore, the extraction of gait features plays a crucial role in the field of recognition.

[0003] Currently, the gait feature information extracted by the existing gait feature extraction methods is not very accurate and is easily affected by external factors such as clothing. Summary of the Invention

[0004] This application provides a gait feature extraction method and device, so that the gait feature extraction method of this application can extract relatively excellent gait features with rich motion details, thereby improving the recognition accuracy of gait recognition using the gait features extracted by this application.

[0005] To achieve the above object, this application provides a gait feature extraction method, which includes:

[0006] Obtain an image sequence collected by a target object;

[0007] Based on the image sequence, determine a gait contour map sequence, a bone key point sequence, and a body part division map sequence of the target object;

[0008] Based on the gait contour map sequence, the bone key point sequence, and the body part division map sequence of the target object, extract the gait features of the target object.

[0009] Among them, the step of extracting the gait features of the target object based on the gait contour map sequence, the bone key point sequence, and the body part division map sequence of the target object includes:

[0010] Based on the gait contour map sequence, the bone key point sequence, and the body part division map sequence of the target object, extract the gait spatial features of the target object;

[0011] Extract the information in the time dimension of the gait spatial features of the target object to obtain the gait features of the target object.

[0012] Among them, the step of extracting the gait spatial features of the target object based on the gait contour map sequence, the bone key point sequence, and the body part division map sequence of the target object includes:

[0013] Extract the gait spatial features of the gait contour map sequence, the bone key point sequence, and the body part division map sequence respectively;

[0014] Fuse the gait space features of the gait contour map sequence, the gait space features of the bone key point sequence, and the gait space features of the body part division map sequence to obtain the gait space features of the target object.

[0015] Among them, the steps of extracting the gait features of the target object based on the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object, and then include:

[0016] Divide the gait features into multiple sub-blocks;

[0017] Map each sub-block through a mapping function corresponding to each sub-block separately;

[0018] Stitch the mapping results of the multiple sub-blocks to obtain the final gait features of the target object.

[0019] Among them, the step of dividing the gait features into multiple sub-blocks includes:

[0020] Horizontally divide the gait features into the multiple sub-blocks.

[0021] Among them, the step of mapping each sub-block through a mapping function corresponding to each sub-block separately includes:

[0022] Input the multiple sub-blocks one by one into multiple fully connected neural networks so that each fully connected neural network maps the received sub-block;

[0023] The step of stitching the mapping results of the multiple sub-blocks includes:

[0024] Stitch the outputs of the multiple fully connected neural networks to obtain the final gait features of the target object.

[0025] Among them, the steps of extracting the gait features of the target object, and then include:

[0026] Based on the gait features, perform identity recognition on the target object.

[0027] Among them, the step of using the gait features to be processed to perform identity recognition on the target object includes:

[0028] Determine the similarity between each candidate gait feature and the gait feature; among them, the candidate gait feature is determined based on the gait feature of the corresponding historical object;

[0029] Based on the determined similarities, perform identity recognition on the target object.

[0030] Among them, performing identity recognition on the target object based on the determined similarities includes:

[0031] In response to the existence of a first similarity among the similarities, determining the identity information of the historical object corresponding to the first similarity as the identity information of the target object; the first similarity is the maximum value among the similarities greater than the first similarity threshold.

[0032] Among them, the historical object includes an object with target permissions;

[0033] Performing identity recognition on the target object based on the determined similarities includes:

[0034] In response to the existence of a similarity greater than the second similarity threshold among the similarities, determining that the target object has the target permissions.

[0035] Among them, the method further includes:

[0036] If it is determined that the target object does not have the target permissions, sending out an alarm signal.

[0037] Among them, the method further includes:

[0038] In response to the gait feature creation operation triggered by the historical object, determining the image sequence collected for the historical object;

[0039] Determining the gait features extracted from the image sequence collected for the historical object as the candidate gait features of the historical object.

[0040] Among them, the candidate gait features are stored in a preset storage space, and the method further includes:

[0041] In response to the feature deletion instruction triggered by the historical object, deleting the candidate gait features indicated by the feature deletion instruction from the preset storage space.

[0042] To achieve the above object, the present application further provides an electronic device, which includes a processor; the processor is used to execute instructions to implement the above method.

[0043] To achieve the above object, the present application further provides a computer-readable storage medium, which is used to store instructions / program data, and the instructions / program data can be executed to implement the above method.

[0044] In the gait feature extraction method of this application, an image sequence of the target object is obtained, and a gait contour map sequence, a sequence of skeletal key points, and a body part division map sequence of the target object are obtained based on the image sequence of the target object; then, the gait features of the target object are extracted based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object; thus, the gait features of the target object are extracted using three feature maps of the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object. The influence brought by clothing and backpacks can be offset through the sequence of skeletal key points, and the motion state of components can be extracted using the body part division map sequence, thereby extracting the motion details of the gait. As a result, relatively excellent gait features with rich motion details can be extracted through the gait feature extraction method of this application, so as to improve the recognition accuracy of gait recognition using the gait features extracted by this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings described herein are used to provide a further understanding of this application, form a part of this application, and the illustrative embodiments of this application and their descriptions are used to explain this application and do not constitute an improper limitation to this application. In the drawings:

[0046] Figure 1 is a schematic flowchart of an embodiment of the gait feature extraction method of this application;

[0047] Figure 2 is a schematic structural diagram of an embodiment of the gait spatial feature extraction model in the gait feature extraction method of this application;

[0048] Figure 3 is a schematic structural diagram of an embodiment of the gait time feature extraction module in the gait feature extraction method of this application;

[0049] Figure 4 is a schematic flowchart of another embodiment of the gait feature extraction method of this application;

[0050] Figure 5 is a schematic flowchart of the gait feature block mapping processing method in the gait feature extraction method of this application;

[0051] Figure 6 is a schematic structural diagram of an embodiment of the gait feature extraction model in the gait feature extraction method of this application;

[0052] Figure 7 is a schematic structural diagram of an alarm system applying the gait feature extraction method of this application;

[0053] Figure 8 is a schematic structural diagram of an embodiment of an electronic device of this application;

[0054] Figure 9It is a structural schematic diagram of an implementation method of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0055] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without making creative work are within the scope of protection of the present application. In addition, unless otherwise specified (for example, "or in addition" or "or in an alternative"), the term "or" as used herein refers to a non-exclusive "or" (that is, "and / or"). Moreover, the various embodiments described herein are not necessarily mutually exclusive, because some embodiments can be combined with one or more other embodiments to form new embodiments.

[0056] Specifically, Figure 1 As shown, the gait feature extraction method of the present application includes the following steps. It is understandable that the execution subject of the gait feature extraction method of the present application is not limited, for example, it can be a server or a terminal. It should be noted that the following step numbers are only used to simplify the description and are not intended to limit the execution order of the steps. The execution order of each step of this embodiment can be changed arbitrarily without violating the technical idea of ​​the present application.

[0057] S101: Acquire a sequence of images collected from a target object.

[0058] An image sequence of the target object can be obtained so as to subsequently determine the target object's gait contour map sequence, skeletal key point sequence and body part division map sequence based on the target object's image sequence; and then extract the target object's gait features based on the target object's gait contour map sequence, skeletal key point sequence and body part division map sequence.

[0059] The type of the target object is not limited, for example, it can be a person, a dog, a pig or a cat.

[0060] Optionally, a walking image sequence or a running image sequence of the target object may be obtained, wherein the legs of the target object in the image sequence of the target object should be unobstructed and relatively clear and complete, so that clear and effective gait features can be extracted.

[0061] The image sequence of the target object can be collected by the camera device. Specifically, the walking or running video of the target object is captured by the camera device, the target object is preprocessed such as detection and tracking, and a video segment of the target object's relatively clear and complete legs without obstructions is intercepted in the video to obtain the image sequence of the target object.

[0062] S102: Based on the image sequence, determine the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object.

[0063] The image sequence can be processed to obtain the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object; so as to subsequently extract the gait features of the target object based on the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object.

[0064] Optionally, semantic segmentation technology can be used to separate the foreground (i.e., the target object) from the background in the images of the image sequence and perform binary processing to obtain the gait contour map sequence of the target object. In other alternative embodiments, a gait contour extraction model can be used to process the images in the image sequence to obtain the gait contour map of each image in the image sequence, thereby obtaining the gait contour map sequence of the target object.

[0065] Optionally, target object bone key point detection technology can be used to extract the bone key points of the target object in the images of the image sequence to obtain the bone key point sequence of the target object. In other alternative embodiments, a human pose estimation algorithm can be used to extract the bone key points of the target object in the images of the image sequence to obtain the bone key point sequence of the target object.

[0066] In addition, body part recognition technology can be used to extract each body part of the target object in the images of the image sequence to obtain the body part division map sequence of the target object.

[0067] After obtaining the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object through the above methods, these three different sequences can be subjected to size normalization processing so as to subsequently extract the gait features of the target object based on these three different sequences.

[0068] S103: Based on the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object, extract the gait features of the target object.

[0069] After obtaining the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object based on the above steps, the gait features of the target object can be extracted based on the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object.

[0070] Optionally, extract the gait space features of the target object based on the gait contour map sequence, bone key point sequence, and body part division map sequence of the target object; then perform information extraction in the time dimension on the gait space features of the target object to obtain the gait features of the target object.

[0071] In one implementation, the gait spatial features of the gait contour map sequence, the bone key point sequence, and the body part division map sequence of the target object can be extracted respectively; then, information extraction in the time dimension is performed on the spatial features of the gait contour map sequence to obtain the gait features corresponding to the gait contour map sequence; and information extraction in the time dimension is performed on the spatial features of the bone key point sequence to obtain the gait features of the bone key point sequence; and information extraction in the time dimension is performed on the spatial features of the body part division map to obtain the gait features of the body part division map; then, the gait features corresponding to the gait contour map sequence, the gait features of the bone key point sequence, and the gait features of the body part division map are fused to obtain the gait features of the target object.

[0072] In another implementation, the gait spatial features of the gait contour map sequence, the bone key point sequence, and the body part division map sequence are extracted respectively; the gait spatial features of the gait contour map sequence, the gait spatial features of the bone key point sequence, and the gait spatial features of the body part division map sequence are fused to obtain the gait spatial features of the target object; information extraction in the time dimension is performed on the gait spatial features of the target object to obtain the gait features of the target object; performing feature fusion before information extraction in the time dimension can reduce the number of parameters and improve the training efficiency, and extracting in the time dimension the fusion result of the spatial features of the three sequences, the extracted spatial information is richer, and the gait features of the target object obtained by time extraction are better.

[0073] Specifically, in this implementation, the gait contour map sequence, the bone key point sequence, and the body part division map sequence can be respectively input into Figure 2 the three branches of the gait spatial feature extraction model shown in the figure, so that the three branches of the gait spatial feature extraction model respectively extract the gait spatial features of the gait contour map sequence, the bone key point sequence, and the body part division map sequence, and the gait spatial feature extraction model fuses the gait spatial features extracted by the three branches to obtain the gait spatial features of the target object, so that the finally extracted spatial features contain richer information, and thus the gait features extracted based on the gait spatial features are better.

[0074] Among them, the structures of the three branches of the gait spatial feature extraction model can be specifically set according to the actual situation and are not limited here. For example, the three branches of the gait spatial feature extraction model can be composed of a multi-layer two-dimensional convolutional neural network; among them, the shallow neural network mainly extracts shallow features such as edge information and texture information of the input multi-dimensional and multi-feature gait sequence; while the deep neural network can abstractly represent the overall information and extract the deep features of the input multi-dimensional and multi-feature gait sequence, implicitly containing various information such as the body shape, posture, and the associations between various parts of the body. In addition, a residual structure can be provided in the branches of the gait spatial feature extraction model, and the residual structure is used to connect the shallow features and the deep features to achieve the combination of multi-level features.

[0075] In addition, the three branches of the gait spatial feature extraction model can have the same or different structures.

[0076] After obtaining the gait spatial features of the target object, the gait spatial features of the target object can be input into the gait temporal feature extraction module to use the gait temporal feature extraction module to extract information in the time dimension from the gait spatial features of the target object, thereby obtaining the gait features of the target object.

[0077] Among them, the structure of the gait temporal feature extraction module can be specifically set according to the actual situation and is not limited here. For example, as Figure 3 shown, the gait temporal feature extraction module can be composed of two layers of two-dimensional and one layer of three-dimensional convolutional neural networks. The convolutional kernel size of the two-dimensional convolutional neural network is 1×1, so that more non-linear operations can be introduced without changing the feature size and channel dimension, thereby enhancing the representation ability of the model; the three-dimensional convolutional neural network adds the processing of the time dimension to capture the change characteristics of each part of the body in the time dimension. Similarly, in order to further improve the effectiveness of feature extraction, a residual structure and a feature fusion mechanism are used in this gait temporal feature extraction module to achieve the extraction of the time information of the gait of the target object.

[0078] In this embodiment, an image sequence of a target object is obtained, and a gait contour map sequence, a sequence of skeletal key points, and a body part division map sequence of the target object are obtained based on the image sequence of the target object; then, gait features of the target object are extracted based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object; thus, gait features of the target object are extracted by using three feature maps of the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object. The influence brought by clothes and backpacks can be offset by the sequence of skeletal key points, and the motion state of components can be extracted by using the body part division map sequence, so as to extract the motion details of the gait, so that relatively excellent gait features with rich motion details can be extracted by the gait feature extraction method of the present application, so as to improve the recognition accuracy of gait recognition using the gait features extracted by the present application.

[0079] Further, considering that when the target object is moving, the degrees of position and shape change of each part of the body are different. For example, the head moves slightly up and down at a certain height, the arms swing back and forth on the sides of the body or have other movements, the legs move back and forth, and so on. The gait features of the target object can be divided into multiple sub-blocks, and each sub-block is mapped through its corresponding mapping function, and the mapping results of the multiple sub-blocks are fused to obtain the final gait features, so that the gait features of each body part in the final gait features are relatively obvious, so that the motion details of some components with small motion amplitudes will not be ignored, thereby improving the accuracy of identity recognition using the gait features of the target object. Specifically, as Figure 4 shown, the present application provides a gait feature extraction method applying the technical features of the above-mentioned gait feature block processing and fusion. The gait feature extraction method includes the following steps.

[0080] S201: Obtain an image sequence of a target object.

[0081] S202: Based on the image sequence, obtain a gait contour map sequence, a sequence of skeletal key points, and a body part division map sequence of the target object.

[0082] S203: Extract gait features of the target object based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object.

[0083] S204: Divide the gait features into multiple sub-blocks.

[0084] After obtaining the gait features of the target object based on the above steps, the gait features can be divided into multiple sub-blocks, so that in the subsequent process, each sub-block can be mapped using its separately corresponding mapping function, and the mapping results of the multiple sub-blocks can be fused to obtain the final gait features, making the gait features of each body part in the final gait features relatively obvious. In this way, the movement details of some body parts with small movement amplitudes will not be ignored, and the movement characteristics of different sub-blocks can be finely adapted, thereby improving the accuracy of identity recognition using the gait features of the target object.

[0085] Among them, the way of dividing the gait features can be specifically set according to the actual situation and is not limited here. For example, the gait features can be horizontally divided into multiple sub-blocks. Another example is that the gait features can be divided into multiple sub-blocks according to body parts.

[0086] Considering that during the human movement process, the movement states of body parts in the same horizontal direction are similar, while the movement states of body parts in other directions (such as the same vertical direction) are quite different, and the movement details information contained in areas with similar movement states tends to be consistent. Thus, preferably, the gait features are horizontally divided into multiple sub-blocks.

[0087] S205: Map each sub-block using its separately corresponding mapping function.

[0088] After dividing the gait features into multiple sub-blocks based on the above steps, each sub-block can be mapped through the mapping function separately corresponding to it.

[0089] Optionally, in step S205, each sub-block is mapped through the mapping function separately corresponding to it, that is, the sub-block and the mapping function are in one-to-one correspondence, and the mapping functions corresponding to different sub-blocks are independent of each other and their parameters do not affect each other. Only in this way can the movement details information contained in each sub-block be better retained. For example, if the gait features are horizontally divided into n sub-blocks, then correspondingly, n mapping functions (such as fully connected neural networks) are required to map the n sub-blocks respectively to obtain the mapping results of these n sub-blocks.

[0090] Optionally, the mapping function can be a fully connected neural network, so as to reduce the influence of the feature position on discrimination by performing weighted summation on different positions. In other alternative embodiments, the mapping function can be a convolutional neural network (for example, a convolutional neural network with a convolution kernel size of 1×1), so that more non-linear expressions can be introduced.

[0091] S206: Concatenate the mapping results of multiple sub-blocks to obtain the final gait features of the target object.

[0092] After obtaining the mapping results of multiple sub - blocks based on the above steps, the mapping results of the multiple sub - blocks can be spliced to obtain the final gait feature of the target object.

[0093] Optionally, the mapping results of multiple sub - blocks can be spliced according to the positions of the multiple sub - blocks in the gait feature to obtain the final gait feature of the target object. For example, as Figure 5 shown, the gait feature is horizontally divided into four sub - blocks A, B, C, and D. Then, the mapping functions of A, B, C, and D are used to map the sub - blocks A, B, C, and D respectively to obtain the mapping results A', B', C', and D' of the sub - blocks A, B, C, and D. According to the positions of the sub - blocks A, B, C, and D in the gait feature, A', B', C', and D' are spliced.

[0094] Generally speaking, steps S204, S205, and S206 are to divide the gait feature map into N sub - blocks by adopting a block - based strategy, use independent mapping functions with non - interfering parameters to map each sub - block to the discriminant space respectively, and then splice the mapped feature vectors in the discriminant space according to the positions before block - division to obtain the final gait feature.

[0095] S207: Determine whether the target object has the target permission based on the final gait feature.

[0096] After obtaining the final gait feature of the target object based on the above steps, it can be determined whether the target object has the target permission based on the final gait feature of the target object. It can be understood that in other alternative embodiments, it can also be determined whether the target object has the target permission based on the gait feature of the target object (i.e., the output of step S203). When the target object has the target permission, maintain the normal state or respond to the instructions of the target object.

[0097] Optionally, the types of target permissions are not restricted and can be set according to the actual situation. For example, the target permission can be door - opening permission, device - control permission, account - login permission, and payment permission.

[0098] In step S207, the final gait feature of the target object can be matched with the final gait features of users with target permissions; if the final gait feature of the target object successfully matches the final gait feature of a user with target permissions, the target object has the target permission; if the final gait feature of the target object does not match the final gait features of all users with target permissions, the target object does not have the target permission.

[0099] Among them, the specific process of the step of matching the final gait feature of the target object with the final gait feature of the user with the target permission may be as follows: calculate the cosine distance between the final gait feature of the target object and the final gait feature of the user with the target permission; if the cosine distance is greater than the threshold, it is determined that the final gait feature of the target object does not match the final gait feature of the user with the target permission; if the cosine distance is less than the threshold, it is determined that the final gait feature of the target object matches the final gait feature of the user with the target permission. Of course, in other alternative embodiments, the similarity or Euclidean distance between the final gait feature of the target object and the final gait feature of the user with the target permission may also be used to confirm whether the final gait feature of the target object matches the final gait feature of the user with the target permission.

[0100] In addition, if it is necessary to confirm the identity of the target object through the final gait feature of the target object, the identity of the target object may be classified as the identity of the sample with the smallest distance (cosine distance or Euclidean distance) between the final gait feature of the target object and the samples in the bottom database when the smallest distance is less than the threshold.

[0101] Optionally, the present application may specifically implement the above-mentioned gait feature extraction method through a gait feature extraction network as Figure 6 shown to obtain the final gait feature of the target object.

[0102] Specifically, the gait contour map extraction module may be used to process the images in the image sequence of the target object to obtain the gait contour map of each image in the image sequence, so as to obtain the gait contour map sequence of the target object. The contour map spatial feature extraction branch is used to extract the gait spatial features of the gait contour map sequence.

[0103] The bone key point extraction module may be used to extract the bone key points of the target object in the images in the image sequence to obtain the bone key point sequence of the target object. The key point spatial feature extraction branch is used to extract the gait spatial features of the bone key point sequence.

[0104] The body part division map extraction module may be used to extract each body part of the target object in the images in the image sequence to obtain the body part division map sequence of the target object. The body part spatial feature extraction branch is used to extract the gait spatial features of the body part division map sequence.

[0105] The fusion module is used to fuse the gait spatial features of the gait contour map sequence, the gait spatial features of the bone key point sequence, and the gait spatial features of the body part division map sequence to obtain the gait spatial feature of the target object.

[0106] The time feature extraction module is used to extract information in the time dimension from the gait spatial features of the target object, and the gait features of the target object are obtained.

[0107] The gait features are divided into multiple sub-blocks by using the block division module.

[0108] The sub-block individual mapping module is used to map each sub-block by using its individually corresponding mapping function, and the mapping result of each sub-block is obtained.

[0109] The splicing module is used to fuse the mapping results of multiple sub-blocks to obtain the final gait features of the target object.

[0110] Before using the gait feature extraction network to extract the gait features from the image sequence of the target object, at least some modules of the gait feature extraction network can be trained by using the training data. For example, the contour map spatial feature extraction branch, the key point spatial feature extraction branch, the body part spatial feature extraction branch, the fusion module, the time feature extraction module, the block division module, the sub-block individual mapping module, and the splicing module in the gait feature extraction network can be trained by using the training data.

[0111] Specifically, at least some modules of the gait feature extraction network can be trained by using loss functions such as the triplet loss function and / or the cross-entropy loss function.

[0112] In addition, the above gait feature extraction method can be applied to the identity recognition method, so that the identity of the target object can be recognized based on the gait features of the target object determined by the above gait feature extraction method.

[0113] Specifically, the identity information of the target object can be determined by comparing the similarity between each candidate gait feature and the gait feature of the target object.

[0114] Among them, the candidate gait feature is the gait feature of the corresponding historical object extracted based on the gait feature extraction method of the present application.

[0115] The acquisition process of the candidate gait feature of the historical object can be as follows: in response to the gait feature creation operation triggered by the historical object, the image sequence collected for the historical object is determined; the gait feature extracted from the image sequence collected for the historical object based on the gait feature extraction method of the present application is determined as the candidate gait feature of the historical object.

[0116] In addition, all candidate gait features determined based on the above method can be combined into a gait database and stored in a preset storage space (such as a server or a terminal), so that when determining the identity of the target object based on the gait feature extraction method of the present application, the candidate gait features in the preset storage space can be retrieved for comparison.

[0117] Optionally, in response to a user operation instruction, a deletion or addition operation may be performed on candidate gait features stored in a preset storage space. For example, in response to a feature deletion instruction triggered by a historical object, the candidate gait features indicated by the feature deletion instruction may be deleted from the preset storage space.

[0118] Optionally, the above identity recognition may be understood as determining the identity of the target object (such as a person's name). More preferably, in response to the existence of a first similarity among the similarities, the identity information of the historical object corresponding to the first similarity is determined as the identity information of the target object; the first similarity is the maximum value among the similarities greater than the first similarity threshold. In this way, relatively accurate identity information can be determined based on the gait features of the target object, and through the first similarity threshold, it can be avoided that in the case where the target object does not belong to the historical objects corresponding to all candidate gait features, the identity of the historical object corresponding to the maximum similarity is misattributed to the identity of the target object. Among them, the first similarity threshold can be set according to the actual situation and is not limited here. For example, it can be 90% or 80%.

[0119] In addition, the above identity recognition is not limited to being understood as determining the identity of the target object (such as a person's name), but can also be understood as determining whether the target object has target permissions. Thus, the above historical objects may include objects with target permissions; in the implementation process of the step of determining the identity information of the target object by comparing the similarities between each candidate gait feature and the gait feature of the target object, it may be: determining the similarities between each candidate gait feature and the gait feature of the target object; in response to the existence of similarities greater than the second similarity threshold among the similarities, determining that the target object has target permissions. Additionally, if it is confirmed that the target object has target permissions, the normal state may be maintained or the target object instruction may be responded to. If it is determined that the target object does not have target permissions, an alarm message may be sent to protect the user with target permissions. Among them, the second similarity threshold can be set according to the actual situation and is not limited here. For example, it can be 70% or 80%.

[0120] Optionally, the above target permissions may include login permissions, information query permissions, access permissions, call permissions, etc., and are not limited here.

[0121] Specifically, the above gait feature extraction method may be applied to, for example, Figure 7 the alarm system shown, to avoid the trouble of multiple detectors installed at the entrance of the house. In this way, only one camera device needs to be installed in the area to be monitored to achieve the security monitoring of the house, and by using the characteristics of gait information such as long distance, non-contact, and not easy to disguise, the identity of the people inside the house can be recognized, which can handle problems such as unclear face shooting and unrecognizable faces blocked by intruders in the traditional face monitoring scenario.

[0122] Among them, asFigure 7 As shown, the alarm system may include a camera device (which may be an intelligent camera) for collecting an image sequence of a target object. The camera device may be configured with an electric pan-tilt system, which can achieve multi-angle monitoring of the indoor environment, resulting in a larger monitoring range and a higher safety factor.

[0123] The alarm system further includes a server, which can be used to obtain the image sequence of the target object collected by the camera device and apply the above gait feature extraction method to confirm whether the target object has the target permission. In this way, it is possible to directly analyze the walking posture at a long distance without the cooperation of the target object, and regardless of whether the face of the target object is blocked, it is possible to determine whether the target object has the target permission through the gait features of the target object.

[0124] The server can also send an alarm signal when it is confirmed based on the above gait feature extraction method that the target object does not have the target permission, so as to protect the user with the target permission.

[0125] For example, when the target object does not have the target permission, the server can send an alarm signal to the alarm module so that the alarm module emits a high-decibel alarm sound or lights up a warning light.

[0126] For another example, when the target object does not have the target permission, the server can send an alarm signal to the user or management terminal with the target permission.

[0127] Furthermore, when sending the alarm signal, the server can store the video of the target object and send the path of the target object to the user or management terminal with the target permission, so that the user or manager can know who the intruder is and retain the evidence of the intruder's intrusion.

[0128] Before the above alarm system is actually applied, the base library can be edited first to input the gait image sequence of the user with the target permission into the base library, so as to subsequently confirm whether the target object has the target permission based on the base library. The base library can be stored in the server, so that the server can subsequently confirm whether the target object has the user permission based on the collected image sequence of the target object and the base library.

[0129] When editing the base library, the user can perform operations of adding and deleting the gait base library to cope with the situation of friends, relatives, nannies, etc. coming to visit, stay for a short or long time and leave, making the use experience more intelligent and flexible.

[0130] Please refer to Figure 8 , Figure 8It is a schematic structural diagram of an embodiment of the electronic device 20 of the present application. The electronic device 20 of the present application includes a processor 22, and the processor 22 is used to execute instructions to implement the methods provided by any one of the above embodiments of the present application and any non-conflicting combinations.

[0131] The electronic device 20 can be a device such as a camera device or a server, etc., which is not limited here.

[0132] The processor 22 can also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 22 may be an integrated circuit chip with signal processing capabilities. The processor 22 can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor, or the processor 22 can also be any conventional processor, etc.

[0133] The electronic device 20 may further include a memory 21 for storing instructions and data required for the operation of the processor 22.

[0134] Please refer to Figure 9 , Figure 9 It is a schematic structural diagram of the computer-readable storage medium in the embodiment of the present application. The computer-readable storage medium 30 of the embodiment of the present application stores instruction / program data 31, and when the instruction / program data 31 is executed, it implements the methods provided by any one of the above methods of the present application and any non-conflicting combinations. Among them, the instruction / program data 31 can form a program file and be stored in the above storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor can execute all or part of the steps of the methods of various embodiments of the present application. And the foregoing storage medium 30 includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks or optical discs, or devices such as computers, servers, mobile phones, and tablets.

[0135] In several embodiments provided by this application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of devices or units can be in electrical, mechanical, or other forms.

[0136] In addition, each functional unit in various embodiments of this application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

[0137] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, commodity or device. Without further limitation, the element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, commodity or device including the element.

[0138] The above is only the implementation mode of this application, and does not limit the patent scope of this application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be included in the patent protection scope of this application by the same token.

Claims

1. A gait feature extraction method, characterized in that, The method includes: Obtaining an image sequence collected by a target object; Based on the image sequence, determining a gait contour map sequence, a sequence of skeletal key points, and a body part division map sequence of the target object; Based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object, extracting gait features of the target object; The step of extracting the gait features of the target object based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object includes: Performing spatial feature extraction on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence respectively to obtain spatial features of the gait contour map sequence, spatial features of the sequence of skeletal key points, and respective spatial features of the body part division map sequence; Performing feature fusion on the spatial features of the gait contour map sequence, the spatial features of the sequence of skeletal key points, and the spatial features of the body part division map sequence first and then extracting temporal features to obtain the gait features; or, extracting temporal features from the spatial features of the gait contour map sequence, the spatial features of the sequence of skeletal key points, and the spatial features of the body part division map sequence first and then performing fusion to obtain the gait features.

2. The method according to claim 1, characterized in that, The step of performing feature fusion on the spatial features of the gait contour map sequence, the spatial features of the sequence of skeletal key points, and the spatial features of the body part division map sequence first and then extracting temporal features to obtain the gait features includes: Fusing the gait spatial features of the gait contour map sequence, the gait spatial features of the sequence of skeletal key points, and the gait spatial features of the body part division map sequence to obtain the gait spatial features of the target object; Performing information extraction in the time dimension on the gait spatial features of the target object to obtain the gait features of the target object.

3. The method according to claim 1, wherein After the step of extracting the gait features of the target object based on the gait contour map sequence, the sequence of skeletal key points, and the body part division map sequence of the target object, it includes: Dividing the gait features into multiple sub-blocks; Mapping each sub-block through a mapping function corresponding to each sub-block alone; Stitching the mapping results of the multiple sub-blocks to obtain the final gait features of the target object.

4. The method according to claim 3, characterized in that, The step of mapping each sub-block through a mapping function corresponding to each sub-block alone includes: Inputting the multiple sub-blocks one by one into multiple fully connected neural networks so that each fully connected neural network maps the received sub-block; The step of stitching the mapping results of the multiple sub-blocks includes: Stitching the outputs of the multiple fully connected neural networks to obtain the final gait features of the target object.

5. The method according to any one of claims 1-4, characterized in that After the step of extracting the gait features of the target object, it includes: Performing identity recognition on the target object based on the gait features.

6. The method according to claim 5, wherein The step of performing identity recognition on the target object based on the gait features includes: Determining the similarity between each candidate gait feature and the gait feature; wherein, the candidate gait feature is determined based on the gait features of the corresponding historical object; Performing identity recognition on the target object based on the determined similarities.

7. The method according to claim 6, wherein Based on the determined similarities, perform identity recognition on the target object, including: In response to the existence of a first similarity among the similarities, determine the identity information of the historical object corresponding to the first similarity as the identity information of the target object; the first similarity is the maximum value among the similarities greater than the first similarity threshold.

8. The method according to claim 6, wherein The historical object includes an object with target permissions; Based on the determined similarities, perform identity recognition on the target object, including: In response to the existence of a similarity greater than the second similarity threshold among the similarities, determine that the target object has the target permissions.

9. The method according to claim 8, characterized in that, The method further includes: If it is determined that the target object does not have target permissions, send an alarm signal.

10. The method according to claim 9, wherein The method further includes: In response to the gait feature creation operation triggered by the historical object, determine the image sequence collected for the historical object; Determine the gait feature extracted from the image sequence collected for the historical object as the candidate gait feature of the historical object.

11. The method according to claim 10, characterized in that, The candidate gait feature is stored in a preset storage space, and the method further includes: In response to the feature deletion instruction triggered by the historical object, delete the candidate gait feature indicated by the feature deletion instruction from the preset storage space.

12. An electronic device, characterized in that, The electronic device includes a processor, and the processor is configured to execute instructions to implement the steps of the method according to any one of claims 1-11.

13. A computer-readable storage medium having a program and / or instructions stored thereon, characterized in that, When the program and / or instructions are executed, the steps of the method according to any one of claims 1-11 are implemented.

Images