Identity recognition method and device based on user gaits, equipment and storage medium

Abstract

The invention relates to an identity recognition method and device based on user gaits, equipment and a storage medium, and the method comprises the steps: obtaining at least two discrimination frames from a plurality of image frames, and extracting the coordinate information of preset joint points corresponding to limbs from the discrimination frames; each limb corresponds to at least three preset joint points; according to the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame, determining the change information of the limbs; the change information represents posture change conditions of limbs of the user; determining gait feature information of the user according to the change information of the limbs; the gait feature information is used for representing the characteristics of the walking posture of the user; and performing identity recognition on the user according to the gait feature information of the user. According to the method, the dynamic change information of the limbs of the user during walking is determined from the multiple key discrimination frames, the gait features of the user are obtained according to the dynamic change information, then identity recognition is carried out on the user, and the accuracy of identity recognition is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a user gait-based identity recognition method and device, equipment and a storage medium. BACKGROUND

[0002] With the continuous development of intelligent vehicles, the scene and demand of identity recognition function applied in the vehicle end are also increasing. By performing identity recognition of a user, the risk of vehicle theft and misoperation can be greatly reduced. For example, in terms of driving right control, identity recognition can ensure that the driver is an authorized user, thereby improving the safety of vehicle driving.

[0003] The commonly used identity recognition method is susceptible to factors such as light, shielding and noise, and how to improve the accuracy of identity recognition is a technical problem to be solved. SUMMARY

[0004] The present application aims to provide a user gait-based identity recognition method, device, equipment and storage medium to improve the accuracy of identity recognition.

[0005] In a first aspect, the present application provides a user gait-based identity recognition method, comprising:

[0006] From the collected multiple image frames, at least two frames of discrimination frames of a user are obtained, and the coordinate information of the preset joint nodes corresponding to the limbs is extracted from the discrimination frames; wherein the discrimination frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint nodes;

[0007] According to the coordinate information of the preset joint nodes corresponding to the limbs in each discrimination frame, the change information of the limbs is determined; wherein the change information represents the posture change of the user's limbs;

[0008] According to the change information of the limbs, the gait feature information of the user is determined; wherein the gait feature information is used to represent the characteristics of the user's walking posture;

[0009] According to the gait feature information of the user, the identity of the user is recognized.

[0010] In a second aspect, the present application provides a user gait-based identity recognition device, comprising:

[0011] A coordinate extraction unit is configured to obtain at least two frames of discrimination frames of a user from the collected multiple image frames, and extract the coordinate information of the preset joint nodes corresponding to the limbs from the discrimination frames; wherein the discrimination frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint nodes;

[0012] The change determination unit is configured to determine change information of the limb according to coordinate information of preset joint nodes corresponding to the limb in each determination frame; wherein the change information represents a posture change of the limb of the user;

[0013] The feature determination unit is configured to determine gait feature information of the user according to the change information of the limb; wherein the gait feature information represents characteristics of the walking posture of the user.

[0014] The identity recognition unit is configured to recognize the identity of the user according to the gait feature information of the user.

[0015] In a third aspect, the present application provides an electronic device, comprising a processor and a memory connected to the processor in communication;

[0016] The memory stores computer execution instructions;

[0017] The processor executes the computer execution instructions stored in the memory to realize the method of the first aspect.

[0018] In a fourth aspect, the present application provides a computer readable storage medium, wherein the computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by a processor to realize the method of the first aspect.

[0019] In a fifth aspect, the present application provides a computer program product, comprising a computer program, wherein the computer program is executed by a processor to realize the method of the first aspect.

[0020] In a sixth aspect, the present application provides a vehicle for realizing the method of the first aspect.

[0021] The present application provides a user gait-based identity recognition method, device, equipment and storage medium, by collecting multiple image frames, the key image frames can be determined as determination frames, so as to recognize the gait features of the user according to the determination frames, which can avoid missing gait features and reduce the calculation amount of image recognition, improve the efficiency and comprehensiveness of identity recognition. The coordinate information of each preset joint node corresponding to the limb is extracted from the determination frame, and the change information of the limb is determined according to the coordinate information of each preset joint node corresponding to the limb in each determination frame, that is, the posture change of the user's limb during walking is obtained. According to the change of the user's limb, the gait feature information of the user is extracted, that is, the characteristics of the user's walking posture are obtained. According to the characteristics of the user's walking posture, the identity of the user is recognized. By combining multiple determination frames to extract gait features, the noise or occlusion in the image can be reduced to interfere with identity recognition, and the gait features are dynamic features that are not easy to be imitated by photos or 3D masks, which improves the accuracy of identity recognition and enhances the user experience. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] Figure 1 A flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of preset joint points provided in an embodiment of the present invention;

[0025] Figure 3 A flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a limb triangle provided in an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the gait characteristic curve of the right upper limb provided in an embodiment of the present invention;

[0028] Figure 6 A flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of gait cycles provided in an embodiment of the present invention;

[0030] Figure 8 This is a structural block diagram of a user gait-based identity recognition device provided in an embodiment of the present invention;

[0031] Figure 9 A structural block diagram of an electronic device provided in an embodiment of the present invention;

[0032] Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0033] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0034] The present application will be described in more detail by the following embodiments with reference to the attached drawings; however, the present application can be variously embodied and implemented by one of ordinary skill in the art. The following embodiments are provided to fully convey the scope of the present application to the skilled in the art. The present application is defined by the appended claims.

[0035] It should be noted that the drawings provided in the following embodiments are only schematic and are intended to provide the basic understanding of the application. In reality, the shape, size, and number of components shown in the drawings can be varied to meet specific application requirements. The layout of components in the drawings can also be more complex.

[0036] In the description of the present application, it should be understood that the terms "first", "second", "third", etc. are only used to distinguish similar objects, and do not necessarily describe a specific order or sequence, nor can they be understood as indicating or implying relative importance. For those of ordinary skill in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances. In addition, in the description of the present application, "a plurality of" means two or more, unless otherwise specified. "And / or", which describes the relationship between the associated objects, means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects.

[0037] It should be noted that due to the limited length of the specification, not all alternative embodiments are listed in this specification. Those skilled in the art should be able to think of any combination of technical features as long as the technical features do not contradict each other, which can constitute alternative embodiments. Each embodiment will be described in detail below.

[0038] With the continuous development of intelligent vehicles, the scene and demand of identity recognition function applied in the vehicle end are also increasing. For example, in terms of theft prevention and anti-hijacking, identity recognition has uniqueness, which can greatly reduce the risk of vehicle theft compared to the shortcomings of traditional keys that can be easily copied; in terms of driving right control, identity recognition can ensure that the driver is an authorized user, thereby facilitating management in car rental, shared car, and other scenarios, and preventing unauthorized personnel from starting the vehicle; in terms of personalized configuration, identity recognition can quickly switch the driving style, seat and rearview mirror angle, welcome function height adjustment, etc. of different members among common drivers without manual adjustment by the user.

[0039] The commonly used identity recognition methods mainly include face recognition, fingerprint recognition, voiceprint recognition, DNA (DeoxyriboNucleic Acid) recognition, iris recognition and other biological authentication technologies. However, face recognition is easily affected by factors such as light and shielding, and can be deceived by photos or 3D masks; fingerprint recognition is easily affected by the cleanliness of the contact surface; voiceprint recognition is easily disturbed by background noise and can be imitated through recording; DNA / iris recognition is accurate but has a high collection threshold, and the above methods are basically close-range or contact-type recognition, and the accuracy and efficiency of identity recognition are low.

[0040] The present application provides a user gait-based identity recognition method, device, equipment and storage medium, aiming at solving the above technical problems of the prior art.

[0041] The technical solutions of the present application and how the technical solutions of the present application solve the above technical problems will be described in detail below with specific examples. The following specific examples can be combined with each other, and the same or similar concepts or processes may not be described again in some examples. The embodiments of the present application will be described below with reference to the drawings.

[0042] Figure 1 is a flowchart of a user gait-based identity recognition method according to an embodiment of the present application, which can be executed by a user gait-based identity recognition device. As shown in Figure 1 The method comprises the following steps:

[0043] S101, from the collected multiple image frames, obtain at least two frames of user discrimination frames, and extract the coordinate information of the preset joint nodes corresponding to the limbs from the discrimination frames; wherein the discrimination frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint nodes.

[0044] Illustratively, the method of the present embodiment can be applied to vehicles, door locks, smart home devices and the like. When a user approaches the device, the device can collect multiple image frames containing the user. For example, a camera can be installed on the device, and when a person appears within a preset range around the device, the camera starts to continuously collect image frames. For another example, when a user holding a Bluetooth car key approaches a vehicle, the vehicle senses the Bluetooth signal and starts the camera to collect image frames. That is, the vehicle and other devices can collect image frames after sensing the approach of the user, thereby ensuring that the user exists in the image frames.

[0045] That is, the user gradually approaches the device, and image frames are collected in the process that the user walks to the device, so as to obtain the process that the user walks. The identity recognition is performed in the process that the user approaches the device, so as to realize the non-contact recognition. After the plurality of collected image frames are obtained, the plurality of image frames can be selected from the plurality of image frames, and the selected image frames are used as the discrimination frames. Each discrimination frame contains different walking postures of the user. The plurality of image frames can be selected as the discrimination frames, and the number of the discrimination frames is less than the number of the collected image frames. For example, one discrimination frame can be determined every 5 image frames. The discrimination frames can be used for subsequent identification of the gait of the user. Through the selection of the discrimination frames, the subsequent calculation amount can be reduced, and the recognition efficiency can be improved.

[0046] For each discrimination frame, image recognition processing can be performed on the discrimination frame, and a preset joint node of the user is recognized from the discrimination frame, so as to obtain coordinate information of the preset joint node in the discrimination frame. The preset joint node can be a joint node on the limb of the user. The limb of the user can include an upper limb and a lower limb. The upper limb can include a left arm and a right arm. The lower limb can include a left leg and a right leg. Each limb can correspond to at least three preset joint nodes. For example, the preset joint nodes can include shoulder joints, elbow joints, and wrist joints in the upper limb, and hip joints, knee joints, and ankle joints in the lower limb. Specifically, the preset joint nodes can include a right shoulder joint, a right elbow joint, a right wrist, a left shoulder joint, a left elbow joint, a left wrist, a right hip joint, a right knee, a right ankle, a left hip joint, a left knee, and a left ankle, that is, 12 key skeleton points. Figure 2 A schematic diagram of the preset joint nodes.

[0047] After the discriminant frame of the user's gait is obtained, specific skeleton points, i.e., preset joint points, of the discriminant frame can be extracted by Open Pose (an open pose recognition algorithm) as key points for recognizing the gait. Depth information of the preset joint points can be obtained by a depth estimation method based on optical flow, so as to obtain coordinate information of the preset joint points. For example, the depth information of the preset joint points can be obtained by analyzing the motion of the pixel points in the discriminant frame, i.e., optical flow, and using a preset depth estimation method to infer the three-dimensional structure of the scene. In this embodiment, the depth estimation method based on optical flow is not specifically limited. For example, a method of optical flow based on SfM (Structure from Motion) can be used. First, the pixel points of interest, i.e., the preset joint points, are detected in the first frame image. The positions of the preset joint points are tracked in subsequent frames by using the method of optical flow, so as to obtain the movement trajectories of the preset joint points. The 2D motion of the tracked preset joint points is combined with the intrinsic parameters of the camera, so as to estimate the motion of the camera between two frames. Once the accurate motion of the camera is known, the position information of each tracked preset joint point in the three-dimensional space can be calculated by the principle of triangulation, and the position information contains the depth information. The depth information refers to the spatial distance between the preset joint point and the camera. That is, the coordinate information can be three-dimensional coordinates, which can represent the position of the preset joint point in the image plane or the distance between the preset joint point and the device in space. Each preset joint point corresponding to each limb of the human body can be extracted, or each preset joint point corresponding to a specified limb can be extracted. For example, only the coordinate information of the preset joint points of the lower limbs can be determined.

[0048] In S102, the change information of the limb is determined according to the coordinate information of the preset joint points corresponding to the limb in each discriminant frame. The change information represents the change in the posture of the limb of the user.

[0049] Exemplarily, for each limb or a specified limb of the human body, the coordinate information of each preset joint point on the limb is determined from each frame of the discriminant frame. The change in the posture of the limb is determined by combining the coordinate information of all the preset joint points on the limb in each discriminant frame, i.e., the change information of the limb is obtained. That is, by combining multiple frames of the discriminant frame, the swinging of the limb during the walking of the user can be obtained. The change information of the limb can include the change information of a certain limb, or the change information of multiple limbs.

[0050] For example, for a certain limb, the movement trend of a preset joint on the limb in the walking process of the user can be obtained according to the coordinate information of the preset joint in each discrimination frame. According to the movement trend of the preset joint, the change information of the limb can be obtained. For example, when the user is advancing, the positions of the limb of the user at different times can be determined, and according to the positions at different times, it is determined that the limb of the user presents a trend of deviating to the right side of the user.

[0051] S103, determining gait feature information of the user according to the change information of the limb; wherein the gait feature information is used to represent the characteristics of the walking posture of the user.

[0052] Exemplarily, after obtaining the change information of the limb of the user, according to the change information of the limb, the characteristics of the walking of the user can be obtained, that is, the gait feature information can be obtained. The change information can be subjected to feature extraction processing, and the gait feature information can be extracted. For example, according to the change information, the frequency and amplitude of the swing of the limb of the user in the walking process and the like can be determined, and the frequency and amplitude of the swing of the limb and the like are taken as the gait feature information of the user. The feature points to be extracted from the change information can be preset, and according to the preset feature points, the corresponding gait feature information can be extracted from the change information.

[0053] The neural network model for feature extraction can be trained in advance, the change information is input into the neural network model, and the output data is the gait feature information. In this embodiment, the model structure of the neural network model is not specifically limited. For example, the neural network model can contain multiple convolution layers for feature extraction processing with different convolution kernel sizes, and the convolution layers can be connected with pooling layers and fully connected layers and the like. The neural network model is trained in advance, for example, the change information and information labels used for training can be collected in advance, and the information labels are the correct gait feature information. The change information used for training is input into the model to be trained, and the output data of the model is compared with the information labels. If they are consistent, it means that the model training is completed, if they are not consistent, the preset loss function and the back propagation algorithm are used for iterative training of the model until the model training is completed.

[0054] S104, identifying the identity of the user according to the gait feature information of the user.

[0055] Exemplarily, each limb can correspond to its own gait feature information, and the identity of the user can be identified in combination with the gait feature information of one or more limbs. Different users have different walking postures, i.e., different users correspond to different gait feature information. According to the gait feature information, the identity of the user can be identified. For example, a user white list can be pre-stored, and the user white list stores correct gait feature information of users allowed to operate the device. The determined gait feature information is compared with the gait feature information in the white list. If the determined gait feature information exists in the white list, it is determined that the identity recognition of the user is passed; if it does not exist in the white list, it is determined that the identity recognition of the user is not passed. If the identity recognition of the user is passed, the user is allowed to operate the device, for example, the user can perform operations such as opening the door, closing the window, starting, etc.; if the identity recognition of the user is not passed, the user is not allowed to operate the device, and the user can also be prompted to use other verification methods, or an alarm information can be sent.

[0056] In this embodiment, the identity of the user is identified according to the gait feature information of the user, including: obtaining preset standard feature information; wherein the preset standard feature information is pre-stored correct gait feature information; determining similarity information between the gait feature information of the user and the preset standard feature information; wherein the similarity information represents the closeness between the gait feature information of the user and the preset standard feature information; and identifying the identity of the user according to the similarity information.

[0057] Specifically, the standard feature information is pre-stored, and the preset standard feature information is pre-stored correct gait feature information of the user. The determined gait feature information of the user is compared with the preset standard feature information, and the similarity information between the gait feature information and the standard feature information is calculated. The similarity information represents the closeness between the gait feature information of the user and the preset standard feature information. In this embodiment, the calculation method of the similarity information is not specifically limited. For example, the Euclidean distance formula can be used to calculate the similarity information, and the Euclidean distance can represent the shortest path in a multi-dimensional space from one point to another point, and can be used to calculate the similarity or difference between two vectors or points. By adjusting the parameters in the Euclidean formula, the sensitivity of the Euclidean distance to the similarity can be adjusted, so that the similarity of the two closest vectors is as close to 100% as possible, and the similarity of the two farthest vectors is as close to 0% as possible.

[0058] A similarity threshold is pre-set. After obtaining the similarity information, the similarity information can be compared with the similarity threshold. If the similarity information is equal to or greater than the similarity threshold, it is determined that the identity recognition of the user is passed; if the similarity information is less than the similarity threshold, it is determined that the identity recognition of the user is not passed.

[0059] The change information can correspond to each limb, that is, the gait feature information corresponds to each limb. The standard feature information of each limb is stored in advance, so that the similarity information corresponding to each limb is obtained. The similarity information of each limb is weighted and summed to obtain the total similarity of the user. The total similarity is compared with the similarity threshold value. If the total similarity is equal to or greater than the similarity threshold value, it is determined that the user's identity recognition is passed. If the total similarity is less than the similarity threshold value, it is determined that the user's identity recognition is not passed.

[0060] For example, the total similarity can be calculated by the following formula: S=S1×W1+S2×W2. Wherein, S represents the total similarity, S1 represents the similarity information of the upper limbs, S2 represents the similarity information of the lower limbs, W1 represents the weight of the upper limbs, and W2 represents the weight of the lower limbs. The similarity information of the upper limbs can be equal to the sum of the similarity information of the left arm and the similarity information of the right arm, and the similarity information of the lower limbs can be equal to the sum of the similarity information of the left leg and the similarity information of the right leg.

[0061] The beneficial effect of such setting is that by calculating the similarity, the identity of the user can be effectively identified, it is determined whether the user has the operation permission, and the security of the device use is improved.

[0062] In the embodiment, if it is determined that the user passes the identity recognition, the operation instruction issued by the user to the vehicle is responded, and the action corresponding to the operation instruction is performed.

[0063] Specifically, the device to be operated by the user can be a vehicle. The gait of the user is identified during the process of the user approaching the vehicle. If it is determined that the user passes the identity recognition, the user can directly issue an operation instruction such as opening the door or starting the vehicle when the user reaches the vehicle position. The vehicle responds to the operation instruction of the user and performs the corresponding action.

[0064] That is, the embodiment can be applied to a vehicle terminal. The vehicle can be installed with a camera, which can be a surround view camera, an ADAS (Advanced Driver Assistance System) camera, a blind spot monitoring camera, etc. The vehicle can perform gait feature extraction on the image collected by the camera in real time, obtain gait feature information, and compare the similarity.

[0065] When the vehicle is dormant, only the low-frequency signal detector on the vehicle can be continuously operated for energy saving purposes. When the low-frequency signal detector detects the smart key, it indicates that the driver has appeared near the vehicle, at which time the vehicle central controller can be awakened to start the identity recognition process. After the vehicle central controller is awakened, the camera is further awakened to start collecting and recording videos or images around the vehicle, thereby performing the subsequent identity recognition process. If the user is determined to be an authorized user, the vehicle is controlled to be unlocked.

[0066] In this embodiment, if it is determined that the user passes the identity recognition, the identity information of the user is obtained, and based on a preset association relationship, component adjustment information corresponding to the identity information is determined; wherein the preset association relationship represents the association relationship between the identity information and the component adjustment information, and the component adjustment information is used to adjust the components in the vehicle.

[0067] Specifically, the identity information can represent the unique identifier of the user, and the vehicle pre-stores the association relationship between the identity information and the standard feature information. After determining that the user passes the identity authentication according to the standard feature information, the identity information corresponding to the standard feature information can be found. The component adjustment information can be used to adjust the components in the vehicle, for example, the components can include the suspension, the seat, the brake pedal, the driving pedal, the steering wheel, etc., and correspondingly, the component adjustment information can include the suspension height, the seat position, the pedal sensitivity, the steering wheel damping, etc. The central controller of the vehicle can individually adjust the height of the active suspension to the welcome height previously preset by the user according to the specific identity information of the user, thereby realizing the individual welcome function. The central controller can also individually adjust the throttle pedal sensitivity and pedal curve, the brake characteristics of the brake actuator, the damping and power characteristics of the steering wheel, etc. according to the previous preset configuration of the user, thereby realizing the driving style pre-adjustment function.

[0068] The beneficial effects of such a setting are that the user does not need to be identified again after reaching the vehicle, reducing the waiting time of the user, improving the operation efficiency of the vehicle, and improving the user experience.

[0069] In this embodiment, it can also be applied to the scene of driver action recognition, such as intelligent gesture control, air control, etc. That is, the driver can complete the identity recognition without touching the vehicle, thereby realizing the control of the vehicle through the body action.

[0070] The embodiment of the present application provides a user gait-based identity recognition method, through collecting multiple image frames, key image frames can be determined therefrom as judgment frames, so that the gait features of the user are recognized according to the judgment frames, missing of the gait features can be avoided, the calculation amount of image recognition can be reduced, and the efficiency and comprehensiveness of identity recognition are improved. The coordinate information of each preset joint corresponding to the limbs is extracted from the judgment frames, the change information of the limbs is determined according to the coordinate information of each preset joint corresponding to the limbs in each judgment frame, that is, the posture change of the limbs of the user in the walking process is obtained. According to the change of the limbs of the user, the gait feature information of the user is extracted, that is, the characteristics of the walking posture of the user are obtained. According to the characteristics of the walking posture of the user, the identity of the user is recognized. Through the combination of multiple judgment frames, the interference caused by the noise or shielding in the image to the identity recognition can be reduced, the gait features are dynamic features and are not easy to be imitated by photos or 3D masks, the accuracy of the identity recognition is improved, and the user experience is improved.

[0071] Figure 3 A flowchart of a user gait-based identity recognition method is provided in the embodiment of the present application, and the embodiment is an optional embodiment based on the above-mentioned embodiment.

[0072] In the embodiment, the change information of the limbs is determined according to the coordinate information of the preset joints corresponding to the limbs in each judgment frame, including: for each judgment frame, the area information of the limb triangle corresponding to the limbs is determined according to the coordinate information of the preset joints corresponding to the limbs in the judgment frame; wherein the limb triangle represents a triangle surrounded by the preset joints corresponding to the limbs; and the change information of the limbs is determined according to the area information of the limb triangle corresponding to the limbs in each judgment frame.

[0073] As shown in the method shown in the figure, the method comprises the following steps: Figure 3

[0074] S301, at least two judgment frames of the user are obtained from the collected multiple image frames, and the coordinate information of the preset joints corresponding to the limbs is extracted from the judgment frames; wherein the judgment frame represents an image frame used to determine the gait of the user, and each limb corresponds to at least three preset joints.

[0075] Exemplarily, the present step can refer to the above-mentioned step S101, and will not be repeated here.

[0076] S302, for each judgment frame, the area information of the limb triangle corresponding to the limbs is determined according to the coordinate information of the preset joints corresponding to the limbs in the judgment frame; wherein the limb triangle represents a triangle surrounded by the preset joints corresponding to the limbs.

[0077] ​Exemplarily, for each limb in each discrimination frame, coordinate information of a plurality of preset joint nodes corresponding to the limb in the discrimination frame is determined. According to the coordinate information of the plurality of preset joint nodes corresponding to the limb in the discrimination frame, a triangle corresponding to the limb in the discrimination frame is constructed, that is, a limb triangle is obtained. The limb triangle represents a triangle surrounded by the preset joint nodes corresponding to the limb.

[0078] In a discrimination frame, each limb can correspond to a limb triangle, that is, there can be four limb triangles in a discrimination frame. Each limb triangle corresponds to the coordinates of three vertices, that is, the coordinate information of three preset joint nodes. According to the coordinates of the three vertices of the limb triangle, the area of the triangle can be obtained, that is, the area information of the limb triangle is obtained. For example, the area information can be calculated based on the Heron formula, and the specific process can be that first, the length between each two preset joint nodes in the limb is determined according to the coordinate information of the two preset joint nodes, that is, the three side lengths of the triangle are obtained, and then the semi-perimeter of the triangle is calculated by using the Heron formula according to the three side lengths of the triangle, and finally the area of the triangle is calculated according to the semi-perimeter of the triangle.

[0079] In this embodiment, the area information of the limb triangle corresponding to the limb is determined according to the coordinate information of the preset joint nodes corresponding to the limb in the discrimination frame, comprising: performing coordinate conversion processing on the coordinate information of the preset joint nodes in the discrimination frame according to a preset coordinate system to obtain target coordinates of the preset joint nodes; wherein the target coordinates represent the coordinates of the preset joint nodes in the preset coordinate system; and determining the area information of the limb triangle corresponding to the limb according to the target coordinates of the preset joint nodes corresponding to the limb in the discrimination frame.

[0080] Specifically, in addition to the limbs of the user, the image can also contain other information, which can interfere with the extraction of gait features. In order to highlight the local features in the image, that is, to highlight the limbs of the user, and facilitate subsequent feature extraction and comparison, the coordinate information of the preset joint nodes can be uniformly transformed into a local coordinate system. A coordinate system can be preset as a local coordinate system, and the coordinates of the preset joint nodes are converted in the local coordinate system to obtain the coordinates of the preset joint nodes in the preset coordinate system as target coordinates.

[0081] Each limb can correspond to a preset coordinate system, that is, the preset joint nodes corresponding to each limb can be subjected to coordinate conversion in the corresponding preset coordinate system. That is, for different limbs, the target coordinates of different preset joint nodes can be the same. The construction of the limb triangle of each limb to the limb triangle of the other limb does not cause any influence, and therefore, for different limbs, there can be preset joint nodes with the same target coordinates.

[0082] For each limb, after obtaining the target coordinates of the preset joint, the area information of the limb triangle corresponding to the limb is determined according to the target coordinates of the preset joint.

[0083] The beneficial effect of such an arrangement is that the coordinates of the preset joint are converted, and the area of the triangle is calculated according to the new coordinates. By performing coordinate conversion, the local features in the image can be highlighted, and the efficiency and accuracy of identity recognition can be improved.

[0084] In this embodiment, it also includes: determining the target joint from the preset joint corresponding to the limb in the judgment frame; determining the target joint as the origin to construct the preset coordinate system.

[0085] Specifically, in order to focus on the limbs in the image, for each limb, a certain preset joint in the limb can be taken as the origin in the preset coordinate system corresponding to the limb. That is, a joint is determined from the preset joints of the limb as the target joint. The target joint is taken as the origin in the preset coordinate system of the limb, so as to construct the preset coordinate system.

[0086] Taking the right upper limb as an example, the coordinates of the right shoulder joint in the judgment frame are P1(x1, y1, z1), the coordinates of the right elbow joint are P2(x2, y2, z2), and the coordinates of the right wrist joint are P3(x3, y3, z3). Taking the right shoulder joint as the origin, the target coordinates of the right shoulder joint are (0, 0, 0), the coordinates of the right elbow joint are P2-P1, and the coordinates of the right wrist joint are P3-P1.

[0087] Similarly, for the left upper limb, a preset coordinate system is constructed with the left shoulder joint as the coordinate origin; for the right lower limb, a preset coordinate system is constructed with the right hip joint as the coordinate origin; for the left lower limb, a preset coordinate system is constructed with the left hip joint as the coordinate origin.

[0088] The beneficial effect of such an arrangement is that each limb can construct its own preset coordinate system using the target joint, which facilitates the construction of the limb triangle for each limb and improves the determination accuracy of the limb triangle.

[0089] In this embodiment, each limb includes two limb segments, and a limb segment is composed of two adjacent preset joints in the limb; the method further includes: determining the length information of the limb segment corresponding to the limb according to the coordinate information of the preset joint corresponding to the limb in the judgment frame; adjusting the coordinate information of the preset joint corresponding to the limb in the judgment frame according to the length information of the limb segment corresponding to the limb in each judgment frame, obtaining the adjusted coordinate information, and performing coordinate conversion processing on the adjusted coordinate information of the preset joint in the judgment frame according to the preset coordinate system, to obtain the target coordinates of the preset joint.

[0090] Specifically, each limb of the human body includes two limb segments, for the upper limbs, the limb segments include the upper arm and the lower arm; for the lower limbs, the limb segments include the upper leg and the lower leg. Each limb segment is composed of two adjacent preset joints, for example, for the upper arm of the right upper limb, composed of the right shoulder joint and the right elbow joint; for the lower leg of the left lower limb, composed of the left knee joint and the left ankle joint.

[0091] The length of the limb segment of the human body does not change with walking, but for the same limb segment, the length in different image frames can be different, therefore, for each limb segment, the length information of the limb segment in the determination frame can be uniformly normalized to eliminate the change error of the skeleton size caused by the distance.

[0092] For each limb in the determination frame, according to the coordinate information of the preset joint in the limb, the length information of each limb segment corresponding to the limb is determined, that is, the length information of two limb segments can be determined. For any one limb segment, the length of the limb segment should be fixed, but the length information of the limb segment is different in different determination frames, therefore, the length information of the limb segment can be normalized by a preset normalization function, and the length information of the limb segment in each determination frame is adjusted to a value. According to the adjusted length information, the coordinate information of the preset joint corresponding to the limb segment is adjusted, so that the length information of the limb segment calculated by the adjusted coordinate information in each determination frame is the same. For example, there are 10 determination frames, for each limb segment, 10 length information can be calculated, and the 10 length information is normalized. According to the processed length information, the coordinate information of the preset joint corresponding to the limb segment is adjusted to obtain the adjusted coordinate information. For example, the coordinate information of a preset joint in the limb segment can be fixed first, and the position of the other preset joint in the limb segment is adjusted according to the normalized length information, so that the length between the two preset joints in the 10 determination frames is uniform.

[0093] After obtaining the adjusted coordinate information, the adjusted coordinate information is further converted according to the preset coordinate system, and the subsequent steps are continued.

[0094] The beneficial effect of such setting is that by performing normalization and coordinate position adjustment, the error caused by different depths of field can be eliminated, the determination accuracy of the area information is improved, and the accuracy of the identity recognition is further improved.

[0095] S303、According to the area information of the limb triangle corresponding to the limb in each determination frame, the change information of the limb is determined.

[0096] Exemplarily, in the process that the user walks, the limbs can swing, and the area information of the limb triangle corresponding to the limbs will change. That is, for the same limb, the area information of the limb triangle formed by the limb can be different in different determination frames. According to the change of the area information of the limb triangle in all determination frames, the posture change of the limb in the process that the user walks can be obtained, that is, the change information of the limb is obtained. For example, as the area information increases, it can be determined that the swing amplitude of the user is getting larger and larger.

[0097] Figure 4 The schematic diagram of the limb triangle is shown. Figure 4 Taking the right arm as an example, the limb triangle of the right arm in two determination frames is shown. As shown in FIG. 6, the limb triangle of the right arm changes from the left form to the right form, and the area information of the limb triangle changes. Figure 4

[0098] In this embodiment, if it is detected that the user lacks at least one preset joint node, the missing preset joint node can be determined as a missing joint node. For the limb or limb segment where the missing joint node is located, the coordinate transformation processing can not be performed, that is, the area of the triangle corresponding to the limb does not need to be calculated, or a default triangle area can be used for the limb.

[0099] In this embodiment, the change information of the limb is determined according to the area information of the limb triangle corresponding to the limb in each determination frame, including: performing curve fitting processing on the area information of the limb triangle corresponding to the limb in each determination frame to obtain a gait feature curve of the limb; wherein the abscissa of the gait feature curve represents the determination frame, and the ordinate represents the area information; and the gait feature curve of the limb is determined as the change information of the limb.

[0100] Specifically, the swing amplitudes, motion speeds, motion accelerations, motion phases and other motion characteristics of different individuals at different times are different, that is, the gait feature information is different. The differences in these characteristics directly affect the relative positions of different joint nodes on the limb at different times, thereby affecting the shape of the figure formed by the preset joint nodes on the limb.

[0101] In this embodiment, 12 preset joint nodes are selected, and each limb includes 3 preset joint nodes. For a specific limb, the figure formed by the preset joint nodes is a triangle, and the change rule of the triangle shape is a mapping of the motion characteristics of the preset joint nodes on the limb. Therefore, according to the change of the area information of the limb triangle, the change information of the limb can be obtained.

[0102] ​For each limb, the area information of the limb triangle of the limb in each frame of the discrimination frame can be obtained. The area information of each limb triangle corresponding to the limb is subjected to curve fitting processing, the change of the limb triangle is mapped to a curve, and the fitted curve is the gait feature curve of the limb. The abscissa of the gait feature curve represents the discrimination frame, that is, the time, and the ordinate represents the area information. The gait feature curve of the limb is determined as the change information of the limb. That is, the change information of the limb can be expressed in the form of a curve.

[0103] The fitting processing can be performed based on a Bezier curve, that is, the area information is fitted by using a Bezier curve, and the motion characteristics of the user walking are mapped to the Bezier curve. The reason for selecting the Bezier curve is that it has a smooth and continuous derivative characteristic, which can better avoid the overfitting problem that may be caused by a high-order polynomial curve, and effectively avoid boundary bias. In addition, the Bezier curve has strong global regulation ability, and is especially suitable for global adjustment application scenarios. In this embodiment, an n-order Bezier curve can be used according to the number of discrimination frames, and n is equal to the number of discrimination frames minus one. For example, if there are 10 discrimination frames, n can be equal to 9. That is, a 9-order Bezier curve is used for fitting, wherein the first discrimination frame is used as the starting point of the curve, the tenth discrimination frame is used as the ending point of the curve, and the points of the remaining discrimination frames are control points.

[0104] Figure 5 FIG. 3 is a schematic diagram of a gait feature curve of a right upper limb. The abscissa represents each frame of the discrimination frame, that is, there are 10 discrimination frames, and the ordinate represents the area information of the limb triangle of the right upper limb. The dashed line represents the curve before fitting, and the solid line represents the gait feature curve after fitting.

[0105] The beneficial effect of such a setting is that the change information of the limb can be more clearly and intuitively represented by performing curve fitting, which facilitates subsequent analysis of the characteristics of the walking posture of the user according to the curve, and improves the efficiency and accuracy of user identity recognition.

[0106] S304, determining gait feature information of the user according to the change information of the limb; wherein the gait feature information is used to represent the characteristics of the walking posture of the user.

[0107] For example, the change information is global information representing the walking posture of the user, and after obtaining the change information, the key feature points representing the gait of the user can be extracted from the change information, that is, the gait feature information is obtained, so that the user is identified according to the gait feature information, effectively reducing the calculation amount of identity recognition and improving the recognition efficiency.

[0108] In this embodiment, the change information is represented by a gait feature curve; according to the change information of the limb, the gait feature information of the user is determined, including: determining the curve feature information of the gait feature curve of the limb in at least one target dimension; wherein the target dimension is a dimension representing the curve feature pre-set, and the curve feature information is information representing the feature of the curve in the target dimension; the curve feature information in each target dimension is determined as the gait feature information of the user.

[0109] Specifically, the gait feature information of the user is determined according to the change information of the limb, that is, the gait feature information is determined according to the gait feature curve. That is, the gait feature information is extracted from the gait feature curve. For a curve, the features of the curve can be represented from multiple aspects, that is, the features of the curve can be represented from multiple dimensions. A plurality of dimensions can be pre-set as target dimensions. The features of the gait feature curve in the target dimensions are determined as the curve feature information. That is, the curve feature information is information representing the features of the curve in the target dimensions. For example, the target dimensions can include the ratio of the minimum slope to the maximum slope absolute value, the time period ratio of the maximum slope, the time period ratio of the minimum slope, the time period ratio of the maximum value, the ratio of the minimum value to the maximum value, etc., that is, the data corresponding to the gait feature curve in these target dimensions can be obtained. The gait feature information of the user is obtained by combining the curve feature information of the gait feature curve in each target dimension. For example, the set of curve feature information in each target dimension can be used as the gait feature information.

[0110] A plurality of candidate dimensions can be pre-set, and the target dimensions that best represent the curve features can be pre-selected from the candidate dimensions. Through experiments, it can be found that as the walking speed decreases, the step frequency slows down, the gait cycle lengthens, the amplitude of the gait curve decreases, and the curve shape tends to be flat, which indicates that slower walking speed is accompanied by smaller limb movement amplitude and flatter gait features; as the walking speed increases, the step frequency increases, the gait cycle shortens, the amplitude of the gait curve increases, and the curve shape tends to be steep, which indicates that faster walking speed is accompanied by larger limb movement amplitude and more intense gait features.

[0111] In order to eliminate the influence of different time periods in gait comparison, when selecting the target dimension, for the time-related candidate dimension, the embodiment can compare the time corresponding to the candidate dimension with the time elapsed by all the discriminant frames, for example, the time elapsed by all the discriminant frames is a time period, that is, the time proportion of the time corresponding to the candidate dimension relative to the entire time period is determined, and the time proportion is referred to as the time period ratio of the candidate dimension. In addition, in view of the different signs of the minimum slope and the maximum slope, in order to simplify the subsequent calculation process, the ratio of the minimum slope and the maximum slope can be converted into the ratio of their absolute values. There are 24 preset candidate dimensions, labeled 1 to 24, which are the maximum slope, the minimum slope, the ratio of the absolute values of the minimum slope and the maximum slope, the time period ratio of the maximum slope, the time period ratio of the minimum slope, the curve inflection point, the curve standard deviation, the curve midpoint, the maximum value, the minimum value, the time period ratio of the maximum value, the time period ratio of the minimum value, the ratio of the minimum value and the maximum value, the time period ratio of the zero-crossing point, the quartile, the curve length, the energy, the peak interval, the root mean square, the curve period, the skewness, the local variance, the mean absolute error, and the curve mean.

[0112] Among the above-mentioned candidate dimensions, the optimal candidate dimension is selected as the target dimension to construct the gait feature information. The more the number of target dimensions, the higher the correct rate of identity recognition. Sometimes, too many target dimensions can lead to a decrease in the correct rate. Therefore, reasonable selection of target dimensions is crucial to recognition performance. The embodiment can use a decision tree classifier based on a stepwise forward selection feature selection algorithm to gradually construct a set of target dimensions through a greedy strategy and incremental feature fusion. For example, all candidate dimensions can be traversed to determine the correct rate of each candidate dimension for identity recognition. The candidate dimension with the highest correct rate is selected as the first target dimension and put into the target dimension set. Each candidate dimension is combined with the current target dimension to obtain the correct rate of the combination for identity recognition, and the combination with the highest correct rate is selected as the second target dimension. New candidate dimensions are continuously added until the newly added candidate dimensions no longer bring an improvement in the correct rate, thereby obtaining all the final target dimensions.

[0113] The beneficial effects of such settings are that selecting the preset target dimension to extract the curve feature can reduce the amount of calculation and improve the accuracy of identity recognition and user experience.

[0114] S305, identity recognition is performed on the user according to the gait feature information of the user.

[0115] By way of example, this step can refer to step S104 described above, and will not be described again.

[0116] The embodiment of the present application provides a user gait-based identity recognition method, through collecting multiple image frames, key image frames can be determined as judgment frames, so that the gait features of the user are recognized according to the judgment frames, the gait features can be avoided to be missed, the calculation amount of image recognition can be reduced, and the efficiency and comprehensiveness of identity recognition are improved. The coordinate information of each preset joint point corresponding to the limb is extracted from the judgment frame, the change information of the limb is determined according to the coordinate information of each preset joint point corresponding to the limb in each judgment frame, that is, the posture change of the user's limb in the walking process is obtained. According to the change of the user's limb, the gait feature information of the user is extracted, that is, the characteristics of the user's walking posture are obtained. According to the characteristics of the user's walking posture, the identity of the user is recognized. Through the combination of multiple judgment frames, the noise or shielding in the image can be reduced to interfere with the identity recognition, the gait features are dynamic features and are not easy to be imitated by photos or 3D masks, the accuracy of identity recognition is improved, and the user experience is improved.

[0117] Figure 6 A flowchart of a user gait-based identity recognition method is provided in the embodiment of the present application, and the embodiment is an optional embodiment based on the above-mentioned embodiment.

[0118] In the embodiment,.

[0119] As Figure 6 shown, the method comprises the following steps:

[0120] S601, according to a preset acquisition period, multiple image frames of a user are acquired.

[0121] Exemplarily, the acquisition period of the image is set in advance, and the camera can acquire the image frame according to the preset acquisition period. For example, when it is detected that a user approaches, an image frame can be acquired once per second, so that multiple image frames are obtained. That is, multiple image frames of the user can be acquired according to the preset acquisition period.

[0122] S602, according to the multiple image frames, a gait cycle of the user is determined, and an image frame in the gait cycle is acquired from the multiple image frames; wherein the gait cycle represents a time period in which the same preset action appears when the user walks.

[0123] Exemplarily, when the user walks, the walking posture changes in a certain cycle. According to the acquired multiple image frames, the gait cycle of the user can be determined. The gait cycle represents a time period from when a preset action appears to when the same preset action appears when the user walks. Figure 7 A schematic diagram of the gait cycle. Figure 7In the middle, the time from the moment when the right foot is in front and the heel is on the ground to the moment when the right foot is in front and the heel is on the ground again is taken as a gait cycle.

[0124] That is, whether the preset action exists can be identified from each image frame according to the preset image recognition algorithm, and the time between the two image frames closest to the moment when the preset action appears is taken as the gait cycle of the user. In this embodiment, the preset image recognition algorithm is not specifically limited.

[0125] S603, at least two frames of judgment frames are extracted from the image frames in the gait cycle, and the coordinate information of the preset joint corresponding to the limbs is extracted from the judgment frames.

[0126] Exemplarily, the image frames in a gait cycle are obtained from all the collected image frames. A plurality of image frames are determined from the image frames in the gait cycle as key image frames for identifying the identity of the user, that is, judgment frames, and the coordinate information of each preset joint is extracted from each judgment frame. The image frames arranged in a preset order in the gait cycle can be determined as judgment frames, for example, the first frame, the middle frame and the last frame in the gait cycle are taken as judgment frames.

[0127] In this embodiment, at least two frames of judgment frames are extracted from the image frames in the gait cycle, including: dividing the image frames in the gait cycle equally to obtain a preset number of judgment groups; wherein the image frames in the preset number of judgment groups constitute the image frames in the gait cycle; and one frame of image frame is extracted from each judgment group as a judgment frame.

[0128] Specifically, for a gait cycle, the image frames in the gait cycle are divided equally. The number of equal division can be set in advance, and the image frames of the entire gait cycle are divided into a preset number of judgment groups. That is, the image frames in the preset number of judgment groups constitute a complete gait cycle. The gait cycle can be equally divided in time, or the image frames in the gait cycle can be equally divided in number. One frame of image frame is extracted from each judgment group as a judgment frame. For example, the first frame in each judgment group can be taken as a judgment frame, or the last frame in each judgment group can be taken as a judgment frame.

[0129] In this embodiment, the extracted gait cycle can be equally divided into 10 parts on the time axis. That is, 10 key image frames can be extracted from a gait cycle as judgment frames for identity recognition.

[0130] The beneficial effect of the arrangement is that by grouping the image frames in a gait cycle, the discrimination frames can be distributed in each stage of the gait cycle, which is beneficial to extract accurate gait feature information and improve the recognition accuracy of the user identity.

[0131] S604, determine the change information of the limb according to the coordinate information of the preset joint point corresponding to the limb in each discrimination frame; wherein the change information represents the posture change of the limb of the user.

[0132] By way of example, this step can refer to the above step S102, and will not be repeated.

[0133] S605, determine the gait feature information of the user according to the change information of the limb; wherein the gait feature information is used to represent the characteristics of the walking posture of the user.

[0134] By way of example, this step can refer to the above step S103, and will not be repeated.

[0135] S606, identify the identity of the user according to the gait feature information of the user.

[0136] By way of example, this step can refer to the above step S104, and will not be repeated.

[0137] The embodiment of the present application provides a user gait-based identity recognition method, by collecting multiple image frames, the key image frames can be determined as discrimination frames, so as to identify the gait features of the user according to the discrimination frames, which can avoid missing gait features and reduce the calculation amount of image recognition, and improve the efficiency and comprehensiveness of identity recognition. The coordinate information of each preset joint point corresponding to the limb is extracted from the discrimination frames, and the change information of the limb is determined according to the coordinate information of each preset joint point corresponding to the limb in each discrimination frame, that is, the posture change of the user's limb during walking is obtained. According to the change of the user's limb, the gait feature information of the user is extracted, that is, the characteristics of the user's walking posture are obtained. According to the characteristics of the user's walking posture, the identity of the user is identified. By combining multiple discrimination frames to extract gait features, the noise or occlusion in the image can be reduced to interfere with identity recognition, and the gait features are dynamic features that are not easy to be imitated by photos or 3D masks, which improves the accuracy of identity recognition and enhances the user experience.

[0138] Figure 8 A structure block diagram of a user gait-based identity recognition device provided by the embodiment of the present application is provided. For ease of illustration, only the part related to the embodiment of the present disclosure is shown. For details, refer to Figure 8The identity recognition device 800 based on the user gait includes a coordinate extraction unit 801, a change determination unit 802, a feature determination unit 803, and an identity recognition unit 804.

[0139] The coordinate extraction unit 801 is configured to acquire at least two discrimination frames of the user from the acquired multiple image frames, and extract coordinate information of preset joint nodes corresponding to the limbs from the discrimination frames; wherein the discrimination frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint nodes.

[0140] The change determination unit 802 is configured to determine change information of the limbs according to the coordinate information of the preset joint nodes corresponding to the limbs in each discrimination frame; wherein the change information represents the posture change of the limbs of the user.

[0141] The feature determination unit 803 is configured to determine gait feature information of the user according to the change information of the limbs; wherein the gait feature information is used to represent the characteristics of the walking posture of the user.

[0142] The identity recognition unit 804 is configured to perform identity recognition on the user according to the gait feature information of the user.

[0143] In one example, the change determination unit 802 includes:

[0144] The area determination module is configured to determine, for each discrimination frame, area information of a limb triangle corresponding to the limbs according to the coordinate information of the preset joint nodes corresponding to the limbs in the discrimination frame; wherein the limb triangle represents a triangle formed by the preset joint nodes corresponding to the limbs.

[0145] The change determination module is configured to determine the change information of the limbs according to the area information of the limb triangle corresponding to the limbs in each discrimination frame.

[0146] In one example, the area determination module is specifically configured to:

[0147] According to the preset coordinate system, the coordinate information of the preset joint nodes in the discrimination frame is subjected to coordinate conversion processing to obtain target coordinates of the preset joint nodes; wherein the target coordinates represent the coordinates of the preset joint nodes in the preset coordinate system.

[0148] The area information of the limb triangle corresponding to the limbs is determined according to the target coordinates of the preset joint nodes corresponding to the limbs in the discrimination frame.

[0149] In one example, further comprising:

[0150] The joint node determination unit is configured to determine a target joint node from the preset joint nodes corresponding to the limbs in the discrimination frame.

[0151] The coordinate system construction unit is configured to determine a target joint as an origin and construct a preset coordinate system.

[0152] In one example, each limb includes two limb segments, and each limb segment is composed of two adjacent preset joints in the limb; the device further includes:

[0153] The length determination unit is configured to determine length information of the limb segment corresponding to the limb according to coordinate information of the preset joint corresponding to the limb in the determination frame.

[0154] The coordinate conversion unit is configured to adjust the coordinate information of the preset joint corresponding to the limb in the determination frame according to the length information of the limb segment corresponding to the limb in each determination frame, to obtain adjusted coordinate information, and to perform coordinate conversion processing on the adjusted coordinate information of the preset joint in the determination frame according to the preset coordinate system, to obtain target coordinates of the preset joint.

[0155] In one example, the change determination module is specifically configured to:

[0156] The area information of the limb triangle corresponding to the limb in each determination frame is subjected to curve fitting processing to obtain a gait feature curve of the limb; wherein the horizontal coordinate of the gait feature curve represents the determination frame, and the vertical coordinate represents the area information.

[0157] The gait feature curve of the limb is determined as the change information of the limb.

[0158] In one example, the change information is represented by a gait feature curve; the feature determination unit 803 is specifically configured to:

[0159] Determine curve feature information of the gait feature curve of the limb in at least one target dimension; wherein the target dimension is a dimension representing a feature of the curve that is set in advance, and the curve feature information is information representing the feature of the curve in the target dimension.

[0160] The curve feature information in each target dimension is determined as the gait feature information of the user.

[0161] In one example, the coordinate extraction unit 801 includes:

[0162] The image acquisition module is configured to acquire multiple image frames of the user according to a preset acquisition period.

[0163] The period determination module is configured to determine a gait period of the user according to the multiple image frames, and to acquire image frames within the gait period from the multiple image frames; wherein the gait period represents a time period in which the same preset action occurs when the user is walking.

[0164] The determination frame extraction module is configured to extract at least two determination frames from the image frames within the gait period.

[0165] In one example, the discrimination frame extraction module is specifically configured to:

[0166] The image frames in the gait cycle are evenly divided to obtain a preset number of discrimination groups; wherein the image frames in the preset number of discrimination groups constitute the image frames in the gait cycle;

[0167] One image frame is extracted from each discrimination group as a discrimination frame.

[0168] In one example, the identity recognition unit 804 is specifically configured to:

[0169] Obtain preset standard feature information; wherein the preset standard feature information is correct gait feature information stored in advance;

[0170] Determine similarity information between the gait feature information of the user and the preset standard feature information; wherein the similarity information represents the closeness between the gait feature information of the user and the preset standard feature information;

[0171] According to the similarity information, identity recognition is performed on the user.

[0172] In one example, it further includes:

[0173] The vehicle control unit is configured to, if it is determined that the user passes the identity recognition, perform an action corresponding to an operation instruction issued by the user to the vehicle in response to the operation instruction.

[0174] Figure 9 A structural block diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 1, which includes a memory 91 and a processor 92; the memory 91 is configured to store a memory of executable instructions of the processor 92. Figure 9

[0175] The processor 92 is configured to execute the method provided by the above-mentioned embodiments.

[0176] The electronic device further includes a receiver 93 and a transmitter 94; the receiver 93 is configured to receive instructions and data sent by other devices, and the transmitter 94 is configured to send instructions and data to external devices.

[0177] An embodiment of the present application provides a vehicle, which is installed with a camera, and can be used to perform:

[0178] From the collected multiple image frames, at least two discrimination frames of the user are obtained, and coordinate information of preset joint points corresponding to the limbs is extracted from the discrimination frames; wherein the discrimination frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint points;

[0179] ​According to the coordinate information of the preset joint points corresponding to the limbs in each of the discrimination frames, change information of the limbs is determined; wherein, the change information represents posture change of the limbs of the user;

[0180] According to the change information of the limbs, gait feature information of the user is determined; wherein, the gait feature information is used to represent characteristics of the walking posture of the user;

[0181] According to the gait feature information of the user, identity of the user is identified.

[0182] Figure 10 is a block diagram of an electronic device according to an example embodiment. The device can be a mobile phone, computer, digital broadcast terminal, messaging device, game console, tablet device, personal digital assistant, vehicle, etc.

[0183] The apparatus 1000 can include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0184] The processing component 1002 usually controls overall operations of the apparatus 1000, such as operations associated with displaying, making phone calls, data communications, camera operations and recording operations. The processing component 1002 can include one or more processors 1020 to execute instructions to complete all or part of steps of the methods described above. In addition, the processing component 1002 can include one or more modules to facilitate interaction between the processing component 1002 and other components. For example, the processing component 1002 can include a multimedia module to facilitate the interaction between the multimedia component 1008 and the processing component 1002.

[0185] The apparatus 1000 can include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0186] The memory 1004 is configured to store various types of data to support operations of the device 1000. Examples of these data include instructions for any application or method operating on the device 1000, contact data, phonebook data, messages, pictures, videos, and the like. The memory 1004 can be implemented by any type of volatile or nonvolatile memory, or a combination thereof such as static random access memory (SRAM), electrically erasable programmable read only memory (EEPROM), erasable programmable read only memory (EPROM), programmable read only memory (PROM), read only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0187] The power supply component 1006 supplies electrical power for the various components of the device 1000. The power supply component 1006 can include a power supply management system, one or more power supplies, and other components associated with generating, managing and distributing electrical power for the device 1000.

[0188] The multimedia component 1008 includes a screen providing an output interface between the device 1000 and a user. In some embodiments, the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, swiping and gestures on the touch panel. The touch sensors can not only sense a boundary of a touch or swiping action, but also detect duration and pressure associated with the touch or swiping action. In some embodiments, the multimedia component 1008 includes a front camera and / or a rear camera. The front and / or rear camera can receive external multimedia data when the device 1000 is in an operating mode, such as a shooting mode or a video mode. Each of the front and rear camera can be a fixed optical lens system or have a focal length and optical zoom capability.

[0189] The audio component 1010 is configured to output and / or input audio signals. For example, the audio component 1010 includes a microphone (MIC) configured to receive external audio signals when the device 1000 is in an operating mode, such as a call mode, a recording mode and a voice recognition mode. The received audio signals can be further stored in the memory 1004 or transmitted via the communication component 1016. In some embodiments, the audio component 1010 also includes a speaker for outputting audio signals.

[0190] The I / O interface 1012 provides an interface between the processing component 1002 and peripheral interface modules, which can be a keyboard, a click wheel, a button, and the like. These buttons can include, but are not limited to, a home button, a volume button, a start button and a lock button.

[0191] The sensor component 1014 includes one or more sensors for providing status assessments for various aspects of the device 1000. For example, the sensor component 1014 can detect an open / closed position of the device 1000, relative positioning of components of the device 1000, such as a display and keypad of the device 1000, a change in position of the device 1000 or a component of the device 1000, presence or absence of user contact with the device 1000, orientation or acceleration / deceleration of the device 1000, and temperature changes of the device 1000. The sensor component 1014 can include proximity sensor(s) configured to detect presence of nearby objects without any physical contact. The sensor component 1014 can further include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 1014 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0192] The communication component 1016 is configured to facilitate wired or wireless communication between the device 1000 and another device. The device 1000 can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1016 receives a broadcast signal or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1016 further includes a Near Field Communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on Radio Frequency Identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technology.

[0193] In an exemplary embodiment, the device 1000 can be implemented using one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, or other electronic units to perform the above-described methods.

[0194] In an exemplary embodiment, a non-transitory computer readable storage medium, such as the memory 1004 including instructions, is also provided, which can be executed by the processor 1020 of the device 1000 to complete the above-described methods. For example, the non-transitory computer readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disc, and an optical data storage device, etc.

[0195] A non-transitory computer readable storage medium, when instructions in the storage medium are executed by a processor of an electronic device, enables the electronic device to perform the above-mentioned user gait-based identity recognition method.

[0196] Other embodiments of the application will be apparent to those skilled in the art from consideration of the specification and practice of the application disclosed herein. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the application being indicated by the following claims.

[0197] The above embodiments are only preferred examples of the present application, and the protection scope of the present application is not limited thereto. Any equivalent replacement or transformation made by those skilled in the art based on the present application is within the protection scope of the present application.

Claims

1. A user gait-based identity recognition method, characterized in that, include: From the acquired multiple image frames, at least two discrimination frames of the user are obtained, and the coordinate information of the preset joint points corresponding to the limbs is extracted from the discrimination frames; wherein, the discrimination frames represent image frames used to determine the user's gait, and each limb corresponds to at least three preset joint points; Based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame, the change information of the limbs is determined; wherein, the change information represents the changes in the posture of the user's limbs; Based on the changes in the limbs, the user's gait feature information is determined; wherein, the gait feature information is used to characterize the user's walking posture. The user is identified based on their gait characteristics. Based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame, the change information of the limbs is determined, including: For each discrimination frame, the area information of the limb triangle corresponding to the limb is determined based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame; wherein, the limb triangle represents the triangle enclosed by the preset joint points corresponding to the limb. Based on the area information of the limb triangle corresponding to the limb in each discrimination frame, the change information of the limb is determined.

2. The method according to claim 1, characterized in that, Based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame, the area information of the limb triangle corresponding to the limb is determined, including: According to a preset coordinate system, the coordinate information of a preset joint point in the discrimination frame is subjected to coordinate transformation processing to obtain the target coordinates of the preset joint point; wherein, the target coordinates represent the coordinates of the preset joint point in the preset coordinate system; Based on the target coordinates of the preset joint points corresponding to the limbs in the discrimination frame, the area information of the limb triangle corresponding to the limbs is determined.

3. The method according to claim 2, characterized in that, Also includes: The target joint point is determined from the preset joint points corresponding to the limb in the discrimination frame; The target joint is determined as the origin, and the preset coordinate system is constructed.

4. The method according to claim 2, characterized in that, The method further includes: Based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame, the length information of the limb segment corresponding to the limb is determined; wherein, the limb segment is composed of two adjacent preset joint points in the limb; Based on the length information of the limb segment corresponding to the limb in each discrimination frame, the coordinate information of the preset joint point corresponding to the limb in the discrimination frame is adjusted to obtain the adjusted coordinate information. Then, according to the preset coordinate system, the adjusted coordinate information of the preset joint point in the discrimination frame is transformed to obtain the target coordinates of the preset joint point.

5. The method according to claim 1, characterized in that, Based on the area information of the limb triangle corresponding to the limb in each discrimination frame, the change information of the limb is determined, including: Curve fitting is performed on the area information of the limb triangle corresponding to the limb in each discrimination frame to obtain the gait feature curve of the limb. The gait characteristic curve of the limb is determined as the change information of the limb.

6. The method according to claim 5, characterized in that, The change information is represented by gait feature curves; based on the limb change information, the user's gait feature information is determined, including: Determine the curve feature information of the gait feature curve of the limb in at least one target dimension; wherein, the target dimension is a pre-set dimension that characterizes the curve characteristics, and the curve feature information is information that characterizes the curve characteristics in the target dimension; The curve feature information under each target dimension is determined as the user's gait feature information.

7. The method according to claim 1, characterized in that, From the acquired multi-frame image data, at least two discrimination frames are obtained from the user, including: According to the preset acquisition cycle, acquire multiple image frames from the user; Based on the multiple image frames, the user's gait cycle is determined, and image frames within the gait cycle are obtained from the multiple image frames; wherein, the gait cycle represents the time period during which the user performs the same preset action while walking; Extract at least two discrimination frames from the image frames within the gait cycle.

8. The method according to claim 7, characterized in that, Extract at least two discrimination frames from the image frames within the gait period, including: The image frames within the gait cycle are evenly divided to obtain a preset number of discrimination groups; wherein the image frames in the preset number of discrimination groups constitute the image frames within the gait cycle; One image frame is extracted from each discrimination group to form the discrimination frame.

9. The method according to claim 1, characterized in that, Based on the user's gait feature information, the user is identified, including: Obtain preset standard feature information; wherein, the preset standard feature information is pre-stored correct gait feature information; Determine the similarity information between the user's gait feature information and the preset standard feature information; wherein, the similarity information represents the degree of closeness between the user's gait feature information and the preset standard feature information; The user is identified based on the similarity information.

10. The method according to any one of claims 1-9, characterized in that, Also includes: If it is determined that the user has passed the identity verification, then in response to the operation command issued by the user to the vehicle, the action corresponding to the operation command is executed.

11. The method according to any one of claims 1-9, characterized in that, Also includes: If it is determined that the user has passed the identity recognition, then the user's identity information is obtained, and based on a preset association relationship, component adjustment information corresponding to the identity information is determined; wherein, the preset association relationship represents the association relationship between the identity information and the component adjustment information, and the component adjustment information is used to adjust the components in the vehicle.

12. A user gait-based identity recognition device, characterized in that, include: The coordinate extraction unit is used to obtain at least two discrimination frames of the user from the acquired multi-frame image frames, and extract the coordinate information of the preset joint points corresponding to the limbs from the discrimination frames; wherein, the discrimination frames represent image frames used to determine the user's gait, and each limb corresponds to at least three preset joint points. The change determination unit is used to determine the change information of the limb based on the coordinate information of the preset joint points corresponding to the limb in each discrimination frame; wherein the change information represents the posture change of the user's limb. A feature determination unit is used to determine the user's gait feature information based on the limb change information; wherein the gait feature information is used to characterize the user's walking posture. An identity recognition unit is used to identify the user based on the user's gait feature information; The change determination unit includes: The area determination module is used to determine the area information of the limb triangle corresponding to the limb for each discrimination frame based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame; wherein, the limb triangle represents the triangle enclosed by the preset joint points corresponding to the limb. The change determination module is used to determine the change information of the limbs based on the area information of the limb triangles corresponding to the limbs in each discrimination frame.

13. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-11.

15. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-11.

16. A vehicle, characterized in that, The vehicle is used to implement the method as described in any one of claims 1-11.

Images