User gait-based identity recognition method, device and equipment and storage medium

Abstract

The present application relates to a kind of identity recognition method, device and equipment based on user gait, equipment and storage medium, comprising: at least two frames of discrimination frame are obtained from multiple image frames, the coordinate information of the preset joint corresponding to the body is extracted from discrimination frame;Each body has at least three preset joints;According to the coordinate information of the preset joint corresponding to the body in each discrimination frame, the change information of body is determined;Change information represents the posture change of the body of user;According to the change information of body, the gait feature information of user is determined;Gait feature information is used to represent the characteristics of the posture of user walking;According to the gait feature information of user, the identity of user is identified.From multiple key discrimination frames, the dynamic change information of body when user walks is determined, according to the dynamic change information, the gait feature of user is obtained, and then the identity of user is identified, and the precision of identity recognition is improved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method, apparatus, device, and storage medium for user gait-based identity recognition. Background Technology

[0002] With the continuous development of intelligent vehicles, the application scenarios and demands for identity recognition functions in vehicles are also constantly increasing. By identifying users, the risks of vehicle theft and accidental operation can be greatly reduced. For example, in terms of driving control, identity recognition can ensure that the driver is an authorized user, thereby improving vehicle safety.

[0003] Currently used identity recognition methods are easily affected by factors such as lighting, occlusion, and noise. How to improve the accuracy of identity recognition is a technical problem that urgently needs to be solved. Summary of the Invention

[0004] The purpose of this invention is to provide a method, apparatus, device, and storage medium for identity recognition based on user gait, so as to improve the accuracy of identity recognition.

[0005] In a first aspect, the present invention provides a user gait-based identity recognition method, comprising:

[0006] From the acquired multiple image frames, at least two discrimination frames of the user are obtained, and the coordinate information of the preset joint points corresponding to the limbs is extracted from the discrimination frames; wherein, the discrimination frames represent image frames used to determine the user's gait, and each limb corresponds to at least three preset joint points;

[0007] Based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame, the change information of the limbs is determined; wherein, the change information represents the changes in the posture of the user's limbs;

[0008] Based on the changes in the limbs, the user's gait feature information is determined; wherein, the gait feature information is used to characterize the user's walking posture.

[0009] The user is identified based on their gait characteristics.

[0010] Secondly, the present invention provides an identity recognition device based on user gait, comprising:

[0011] The coordinate extraction unit is used to obtain at least two discrimination frames of the user from the acquired multi-frame image frames, and extract the coordinate information of the preset joint points corresponding to the limbs from the discrimination frames; wherein, the discrimination frames represent image frames used to determine the user's gait, and each limb corresponds to at least three preset joint points.

[0012] The change determination unit is used to determine the change information of the limb based on the coordinate information of the preset joint points corresponding to the limb in each discrimination frame; wherein the change information represents the posture change of the user's limb.

[0013] A feature determination unit is used to determine the user's gait feature information based on the limb change information; wherein the gait feature information is used to characterize the user's walking posture.

[0014] The identity recognition unit is used to identify the user based on the user's gait feature information.

[0015] Thirdly, the present invention provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executed instructions;

[0017] The processor executes computer execution instructions stored in the memory to implement the method as described in the first aspect.

[0018] Fourthly, the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0019] Fifthly, the present invention provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0020] In a sixth aspect, the present invention provides a vehicle for implementing the method described in the first aspect.

[0021] This invention provides a method, apparatus, device, and storage medium for user gait-based identity recognition. By acquiring multiple image frames, key image frames can be identified as discrimination frames. Gait features of the user are then recognized based on these discrimination frames, avoiding omissions and reducing the computational load of image recognition, thus improving the efficiency and comprehensiveness of identity recognition. The coordinate information of each preset joint point corresponding to a limb is extracted from the discrimination frames. Based on the coordinate information of each preset joint point corresponding to a limb in each discrimination frame, the change information of that limb is determined, i.e., the posture changes of the user's limbs during walking are obtained. Based on the changes in the user's limbs, the user's gait feature information is extracted, i.e., the characteristics of the user's walking posture are obtained. Based on the characteristics of the user's walking posture, user identity is recognized. By combining multiple discrimination frames for gait feature extraction, interference from noise or occlusion in the image can be reduced. Furthermore, gait features are dynamic and not easily imitated by photos or 3D masks, improving the accuracy of identity recognition and enhancing the user experience. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] Figure 1 A flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention;

[0024] Figure 2 This is a schematic diagram of preset joint points provided in an embodiment of the present invention;

[0025] Figure 3 A flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a limb triangle provided in an embodiment of the present invention;

[0027] Figure 5 A schematic diagram of the gait characteristic curve of the right upper limb provided in an embodiment of the present invention;

[0028] Figure 6 A flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention;

[0029] Figure 7 A schematic diagram of gait cycles provided in an embodiment of the present invention;

[0030] Figure 8 This is a structural block diagram of a user gait-based identity recognition device provided in an embodiment of the present invention;

[0031] Figure 9 A structural block diagram of an electronic device provided in an embodiment of the present invention;

[0032] Figure 10 This is a structural block diagram of an electronic device provided in an embodiment of the present invention.

[0033] The accompanying drawings have illustrated specific embodiments of the invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the invention in any way, but rather to illustrate the concept of the invention to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0034] The embodiments of the present invention will be described below with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for illustrating the present invention and not for limiting the scope of protection of the present invention.

[0035] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. Therefore, the drawings only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0036] In the description of this invention, it should be understood that the terms "first," "second," "third," etc., are used only to distinguish similar objects and are not necessarily used to describe a specific order or sequence, nor should they be construed as indicating or implying relative importance. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances. Furthermore, in the description of this invention, unless otherwise stated, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0037] It should be noted that, due to space limitations, this specification does not exhaustively list all possible implementation methods. Those skilled in the art, after reading this specification, should be able to deduce that any combination of technical features can constitute an optional implementation method, provided that the technical features do not contradict each other. The following provides a detailed description of each embodiment.

[0038] With the continuous development of intelligent vehicles, the application scenarios and demands for identity recognition functions in vehicles are also constantly increasing. For example, in terms of anti-theft and anti-hijacking, identity recognition is unique, and compared to the disadvantages of traditional keys that are easily copied, identity recognition can greatly reduce the risk of vehicle theft; in terms of driving control, identity recognition can ensure that the driver is an authorized user, thereby facilitating management in scenarios such as car rental and car sharing, and preventing unauthorized personnel from starting the vehicle; in terms of personalized configuration, identity recognition can quickly switch between different members' driving styles, seat and rearview mirror angles, welcome function height adjustment, etc., without requiring manual adjustments by the user.

[0039] Currently, commonly used identity verification methods include biometric authentication technologies such as facial recognition, fingerprint recognition, voiceprint recognition, DNA (Deoxyribonucleic acid) recognition, and iris recognition. However, facial recognition is easily affected by factors such as lighting and occlusion, and can be fooled by photos or 3D masks; fingerprint recognition is easily affected by the cleanliness of the contact surface; voiceprint recognition is easily interfered with by background noise and can also be imitated by recordings; while DNA / iris recognition is accurate, the data collection threshold is high, and the above methods are basically close-range or contact-based identification, resulting in relatively low accuracy and efficiency in identity verification.

[0040] The present invention provides a method, apparatus, device and storage medium for user gait-based identity recognition, which aims to solve the above-mentioned technical problems of the prior art.

[0041] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0042] Figure 1 This is a flowchart illustrating a user gait-based identity recognition method according to an embodiment of the present invention. This method can be executed by a user gait-based identity recognition device. Figure 1 As shown, the method includes the following steps:

[0043] S101. From the acquired multi-frame image, obtain at least two discrimination frames of the user, and extract the coordinate information of the preset joint points corresponding to the limbs from the discrimination frames; wherein, the discrimination frame represents the image frame used to determine the user's gait, and each limb corresponds to at least three preset joint points.

[0044] For example, the method of this embodiment can be applied to devices such as vehicles, door locks, and smart home devices. When a user approaches the device, the device can capture multiple image frames containing the user. For instance, a camera can be installed on the device, and when someone appears within a preset range around the device, the camera starts continuously capturing image frames. As another example, when a user approaches a vehicle holding a Bluetooth car key, the vehicle detects the Bluetooth signal and activates its camera to capture image frames. That is, devices such as vehicles can capture image frames only after sensing the user's approach, thereby ensuring that the user is present in the image frames.

[0045] In other words, as the user gradually approaches the device, image frames are captured during this walking process, thus documenting the user's gait. Identification is then performed as the user approaches the device, achieving contactless recognition. After acquiring multiple image frames, a selection is made from them to serve as a discrimination frame. Each discrimination frame contains different walking postures of the user. Multiple discrimination frames can be selected, with the number of discrimination frames being less than the number of acquired image frames. For example, a discrimination frame can be determined every five image frames. These discrimination frames can be used to subsequently identify the user's gait. By selecting discrimination frames, the amount of subsequent computation can be reduced, improving recognition efficiency.

[0046] For each discrimination frame, image recognition processing can be performed to identify the user's preset joint points and obtain their coordinate information within the frame. These preset joint points can be joint points on the user's limbs, which can include the upper and lower limbs. The upper limbs can include the left and right arms, and the lower limbs can include the left and right legs. Each limb can correspond to at least three preset joint points. For example, preset joint points can include the shoulder, elbow, and wrist joints in the upper limbs, and the hip, knee, and ankle joints in the lower limbs. Specifically, they can include the right shoulder, right elbow, right wrist, left shoulder, left elbow, left wrist, right hip, right knee, right ankle, left hip, left knee, and left ankle—a total of 12 key skeletal points. Figure 2 This is a schematic diagram of the preset joint points.

[0047] After acquiring the user's gait discrimination frames, specific skeletal points, or preset joint points, can be extracted from the discrimination frames using the Open Pose algorithm as key points for gait recognition. Then, depth information of the preset joint points is obtained through an optical flow-based depth estimation method, thereby yielding their coordinate information. For example, by analyzing the motion of pixels in the discrimination frames (i.e., optical flow), the 3D structure of the scene can be inferred using a preset depth estimation method, thus obtaining the depth information of the preset joint points. In this embodiment, the depth estimation method based on optical flow is not specifically limited. For example, an optical flow method based on SfM (Structure from Motion) can be used. First, the pixels of interest, i.e., preset joints, are detected in the first frame. Then, the optical flow method is used to track the positions of the preset joints in subsequent frames to obtain their movement trajectories. By tracking the 2D motion of multiple preset joints and combining it with the camera's intrinsic parameters, the camera's motion between two frames can be estimated. Once the precise motion of the camera is known, the position information of each tracked preset joint in three-dimensional space can be calculated using triangulation principles. This position information includes depth information. Depth information refers to the spatial distance between the preset joint and the camera. That is, the coordinate information can be three-dimensional coordinates, representing both the position of the preset joint in the image plane and the distance between the preset joint and the device in space. Preset joints corresponding to each limb in the human body can be extracted, or preset joints corresponding to a specific limb can be extracted. For example, only the coordinate information of the preset joints of the lower limb can be determined.

[0048] S102. Determine the limb change information based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame; wherein, the change information represents the change in the user's limb posture.

[0049] For example, for each limb or a specified limb of the human body, the coordinate information of each preset joint point on that limb is determined from each discrimination frame. By combining the coordinate information of all preset joint points on that limb in each discrimination frame, the posture changes of that limb are determined, thus obtaining the limb change information. In other words, by combining multiple discrimination frames, the limb swinging situation during the user's walking process can be obtained. The limb change information may include the change information of a single limb or the change information of multiple limbs.

[0050] For example, for a specific limb, the movement trend of a preset joint on that limb during the user's walking process can be obtained based on the coordinate information of that preset joint in each discrimination frame. Based on this movement trend, the limb's change information can be obtained. For instance, as the user moves forward, the position of the user's limb at different times can be determined, and based on the position at different times, it can be determined that the limb tends to lean towards the user's right side.

[0051] S103. Determine the user's gait feature information based on limb change information; wherein, gait feature information is used to characterize the user's walking posture.

[0052] For example, after obtaining information about a user's limb changes, the characteristics of the user's walking can be determined based on this information, i.e., gait feature information. Feature extraction processing can be performed on the change information to extract the gait feature information. For instance, based on the change information, features such as the frequency and amplitude of the user's limb swings during walking can be determined, and these features can be used as the user's gait feature information. Feature points to be extracted from the change information can be preset, and the corresponding gait feature information can be extracted from the change information based on these preset feature points.

[0053] A neural network model for feature extraction can be pre-trained. Change information is input into the neural network model, and the output data is the gait feature information. In this embodiment, the model structure of the neural network is not specifically limited. For example, the neural network model can contain multiple convolutional layers for feature extraction with different kernel sizes, followed by pooling layers and fully connected layers. Pre-training the neural network model involves, for example, pre-collecting change information and information identifiers for training; the information identifiers represent the correct gait feature information. The change information used for training is input into the model to be trained. The model's output data is compared with the information identifiers. If they match, the model training is complete; if they do not match, a preset loss function and backpropagation algorithm are used for iterative training until the model training is complete.

[0054] S104. Based on the user's gait characteristics, identify the user.

[0055] For example, each limb can have its own gait feature information. The gait feature information of one or more limbs can be combined to identify the user. Different users have different walking postures; that is, different users have different gait feature information. Based on the gait feature information, the user's identity can be identified. For example, a user whitelist can be pre-stored, containing the correct gait feature information of users allowed to operate the device. The determined gait feature information is compared with the gait feature information in the whitelist. If the determined gait feature information exists in the whitelist, the user's identity is confirmed; if it does not exist in the whitelist, the user's identity is confirmed to have failed. If the user's identity is confirmed, the user is allowed to operate the device, such as opening doors, closing windows, or starting the device. If the user's identity is confirmed to have failed, the user is not allowed to operate the device, and the user may be prompted to use other verification methods, or an alarm message may be issued.

[0056] In this embodiment, user identification is performed based on the user's gait feature information, including: obtaining preset standard feature information; wherein the preset standard feature information is pre-stored correct gait feature information; determining similarity information between the user's gait feature information and the preset standard feature information; wherein the similarity information characterizes the degree of closeness between the user's gait feature information and the preset standard feature information; and identifying the user based on the similarity information.

[0057] Specifically, standard feature information is pre-stored, which consists of pre-stored correct gait feature information of the user. The determined gait feature information of the user is compared with the pre-stored standard feature information to calculate the similarity information between the gait feature information and the standard feature information. The similarity information represents the degree of closeness between the user's gait feature information and the pre-stored standard feature information. In this embodiment, the calculation method of the similarity information is not specifically limited. For example, the Euclidean distance formula can be used to calculate the similarity information. Euclidean distance can represent the shortest path from one point to another in multidimensional space and can be used to calculate the similarity or difference between two vectors or points. By adjusting the parameters in the Euclidean formula, the sensitivity of the Euclidean distance to similarity can be adjusted so that the similarity between the two closest vectors is as close as possible to 100%, while the similarity between the two farthest vectors is as close as possible to 0%.

[0058] A similarity threshold is preset. After obtaining similarity information, the similarity information can be compared with the similarity threshold. If the similarity information is equal to or greater than the similarity threshold, the user's identity verification is confirmed to be successful; if the similarity information is less than the similarity threshold, the user's identity verification is confirmed to be unsuccessful.

[0059] Each limb can be associated with a specific change in gait characteristic. Standard feature information for each limb is pre-stored, thus obtaining similarity information for each limb. The similarity information of each limb is weighted and summed to obtain the user's total similarity. The total similarity is compared to a similarity threshold. If the total similarity is equal to or greater than the threshold, the user's identity verification is successful; if the total similarity is less than the threshold, the user's identity verification fails.

[0060] For example, the total similarity can be calculated using the following formula: S = S1 × W1 + S2 × W2. Where S represents the total similarity, S1 represents the similarity information of the upper limbs, S2 represents the similarity information of the lower limbs, W1 represents the weight of the upper limbs, and W2 represents the weight of the lower limbs. The similarity information of the upper limbs can be equal to the sum of the similarity information of the left arm and the right arm, and the similarity information of the lower limbs can be equal to the sum of the similarity information of the left leg and the right leg.

[0061] The advantage of this setup is that by calculating similarity, the user's identity can be effectively identified, determining whether the user has the necessary operating permissions, thus improving the security of device use.

[0062] In this embodiment, it further includes: if it is determined that the user has passed the identity verification, then in response to the operation command issued by the user to the vehicle, the action corresponding to the operation command is executed.

[0063] Specifically, the device the user wants to operate can be a vehicle. As the user approaches the vehicle, their gait is recognized. If the user's identity is verified, the user can directly issue operation commands such as opening the door or starting the vehicle once they reach the vehicle's location. The vehicle responds to the user's operation commands and executes the corresponding actions.

[0064] In other words, this embodiment can be applied to vehicle terminals, where cameras can be installed. These cameras can be surround-view cameras, ADAS (Advanced Driver Assistance System) cameras, blind spot monitoring cameras, etc. The vehicle can extract gait features from the images captured by the cameras in real time, obtain gait feature information, and compare similarity.

[0065] When the vehicle is in sleep mode, for energy conservation, only the low-frequency signal detector on the vehicle can remain operational. When the low-frequency signal detector detects the smart key, it indicates that the driver is near the vehicle. At this point, the vehicle's central controller can be activated to begin identity verification. After the central controller is activated, it further activates the camera to begin capturing and recording video or images of the surrounding area for subsequent identity verification. If the user is determined to be authorized, the vehicle will unlock.

[0066] In this embodiment, if it is determined that the user has passed the identity recognition, the user's identity information is obtained, and the component adjustment information corresponding to the identity information is determined based on the preset association relationship; wherein, the preset association relationship represents the association relationship between the identity information and the component adjustment information, and the component adjustment information is used to adjust the components in the vehicle.

[0067] Specifically, identity information can represent a user's unique identifier, and the vehicle pre-stores the association between identity information and standard feature information. After verifying the user's identity based on the standard feature information, the identity information corresponding to that standard feature information can be retrieved. Component adjustment information can be used to adjust components in the vehicle. For example, components may include the suspension, seats, brake pedal, drive pedal, steering wheel, etc. Correspondingly, component adjustment information may include suspension height, seat position, pedal sensitivity, steering wheel damping, etc. The vehicle's central controller can personalize the active suspension height to the user's previously preset welcome height based on the user's specific identity information, thereby realizing a personalized welcome function. The central controller can also personalize the accelerator pedal sensitivity and pedal curve, the braking characteristics of the brake actuator, the damping and power assist characteristics of the steering wheel, etc., according to the user's previously preset configuration, thereby realizing a driving style pre-adjustment function.

[0068] The advantage of this setup is that once a user arrives at the vehicle, there is no need for further identification, reducing the user's waiting time, improving the efficiency of vehicle operation, and enhancing the user experience.

[0069] This embodiment can also be applied to scenarios involving driver action recognition, such as intelligent gesture control and air control. That is, the driver can complete identity verification without touching the vehicle, thereby controlling the vehicle through body movements.

[0070] This invention provides a user gait-based identity recognition method. By acquiring multiple image frames, key frames can be identified as discrimination frames. Gait features are then recognized based on these discrimination frames, avoiding omissions and reducing computational load, thus improving efficiency and comprehensiveness. Coordinate information of preset joints corresponding to limbs is extracted from the discrimination frames. Based on this coordinate information, the changes in the limbs are determined, i.e., the posture changes of the user's limbs during walking. Gait feature information is extracted based on these changes, revealing the characteristics of the user's walking posture. User identity is then recognized based on these posture characteristics. By combining multiple discrimination frames for gait feature extraction, interference from noise or occlusion in the image can be reduced. Furthermore, gait features are dynamic and difficult to imitate with photographs or 3D masks, improving recognition accuracy and enhancing user experience.

[0071] Figure 3 This is a flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention. This embodiment is an optional embodiment based on the above embodiment.

[0072] In this embodiment, the change information of the limb is determined based on the coordinate information of the preset joint points corresponding to the limb in each discrimination frame, including: for each discrimination frame, the area information of the limb triangle corresponding to the limb is determined based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame; wherein, the limb triangle represents the triangle enclosed by the preset joint points corresponding to the limb; and the change information of the limb is determined based on the area information of the limb triangle corresponding to the limb in each discrimination frame.

[0073] like Figure 3 As shown, the method includes the following steps:

[0074] S301. From the acquired multi-frame image, obtain at least two discrimination frames of the user, and extract the coordinate information of the preset joint points corresponding to the limbs from the discrimination frames; wherein, the discrimination frame represents the image frame used to determine the user's gait, and each limb corresponds to at least three preset joint points.

[0075] For example, this step can refer to step S101 above, and will not be repeated here.

[0076] S302. For each discrimination frame, determine the area information of the limb triangle corresponding to the limb based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame; wherein, the limb triangle represents the triangle enclosed by the preset joint points corresponding to the limb.

[0077] For example, for each limb in each discrimination frame, the coordinate information of multiple preset joint points corresponding to that limb in the discrimination frame is determined. Based on the coordinate information of the multiple preset joint points corresponding to that limb in the discrimination frame, a triangle corresponding to that limb in the discrimination frame is constructed, thus obtaining the limb triangle. The limb triangle represents the triangle enclosed by the preset joint points corresponding to the limb.

[0078] In a single discrimination frame, each limb can correspond to a limb triangle, meaning there can be four limb triangles in a single discrimination frame. Each limb triangle has the coordinates of three vertices, which are the coordinates of three preset joint points. Based on the coordinates of the three vertices of the limb triangle, the area of ​​the triangle can be obtained. For example, the area can be calculated using Heron's formula. Specifically, first, based on the coordinates of every two preset joint points in the limb, determine the length between these two preset joint points, thus obtaining the three side lengths of the triangle. Then, based on these three side lengths, use Heron's formula to calculate the semi-perimeter of the triangle. Finally, calculate the area of ​​the triangle based on its semi-perimeter.

[0079] In this embodiment, the area information of the limb triangle corresponding to the limb is determined based on the coordinate information of the preset joint point corresponding to the limb in the discrimination frame. This includes: performing coordinate transformation processing on the coordinate information of the preset joint point in the discrimination frame according to the preset coordinate system to obtain the target coordinates of the preset joint point; wherein, the target coordinates represent the coordinates of the preset joint point in the preset coordinate system; and determining the area information of the limb triangle corresponding to the limb based on the target coordinates of the preset joint point corresponding to the limb in the discrimination frame.

[0080] Specifically, in addition to the user's limbs, images may contain other information that can interfere with gait feature extraction. To highlight local features in the image, specifically the user's limbs, and facilitate subsequent feature extraction and comparison, the coordinate information of preset joint points can be uniformly transformed into a local coordinate system. A preset coordinate system can be used as the local coordinate system, and the coordinates of the preset joint points can be transformed within this local coordinate system to obtain their coordinates, which serve as the target coordinates.

[0081] Each limb can correspond to a preset coordinate system, meaning that the preset joints of each limb can undergo coordinate transformation within that system. In other words, different limbs may have different preset joints with the same target coordinates. Each limb does not affect the construction of the limb triangles of the others; therefore, different limbs can have preset joints with the same target coordinates.

[0082] For each limb, after obtaining the target coordinates of the preset joints, the area information of the limb triangle corresponding to that limb is determined based on the target coordinates of the preset joints.

[0083] The advantage of this setup is that it transforms the coordinates of preset key points and calculates the area of ​​triangles based on the new coordinates. By performing coordinate transformation, local features in the image can be highlighted, improving the efficiency and accuracy of identity recognition.

[0084] In this embodiment, the method further includes: determining a target joint point from preset joint points corresponding to limbs in the discrimination frame; and determining the target joint point as the origin to construct a preset coordinate system.

[0085] Specifically, to focus on the limbs in the image, for each limb, a predetermined joint point within that limb can be used as the origin of the corresponding predetermined coordinate system. That is, a joint point is determined from the predetermined joint points of the limb and designated as the target joint point. This target joint point is then used as the origin of the predetermined coordinate system for that limb, thus constructing the predetermined coordinate system.

[0086] Taking the right upper limb as an example, the coordinates of the right shoulder joint in the discrimination frame are P1(x1, y1, z1), the coordinates of the right elbow joint are P2(x2, y2, z2), and the coordinates of the right wrist joint are P3(x3, y3, z3). Taking the right shoulder joint as the origin, the target coordinates of the right shoulder joint are (0, 0, 0), the coordinates of the right elbow joint are P2-P1, and the coordinates of the right wrist joint are P3-P1.

[0087] Similarly, for the left upper limb, a preset coordinate system is constructed with the left shoulder joint as the origin; for the right lower limb, a preset coordinate system is constructed with the right hip joint as the origin; and for the left lower limb, a preset coordinate system is constructed with the left hip joint as the origin.

[0088] The advantage of this setup is that each limb can construct its own preset coordinate system using the target joint points, which makes it easier to construct limb triangles for each limb and improves the accuracy of limb triangle determination.

[0089] In this embodiment, each limb includes two segments, each segment consisting of two adjacent preset joints. The method further includes: determining the length information of the segment corresponding to the limb based on the coordinate information of the preset joints corresponding to the limb in the discrimination frame; adjusting the coordinate information of the preset joints corresponding to the limb in the discrimination frame based on the length information of the segment corresponding to the limb in each discrimination frame to obtain the adjusted coordinate information; and performing coordinate transformation processing on the adjusted coordinate information of the preset joints in the discrimination frame according to the preset coordinate system to obtain the target coordinates of the preset joints.

[0090] Specifically, each limb of the human body consists of two segments. For the upper limb, the segments include the upper arm and the forearm; for the lower limb, the segments include the thigh and the lower leg. Each segment is composed of two adjacent pre-defined joints. For example, for the upper arm of the right upper limb, it is composed of the right shoulder joint and the right elbow joint; for the lower leg of the left lower limb, it is composed of the left knee joint and the left ankle joint.

[0091] The length of human limbs does not change with walking. However, the length of the same limb may be different in different image frames. Therefore, for each limb, the length information of the limb in the discrimination frame can be uniformly normalized to eliminate the error of skeleton size variation caused by distance.

[0092] For each limb in the discrimination frame, the length information of each segment corresponding to that limb is determined based on the coordinate information of the preset joint points within that limb; that is, the length information of two limb segments can be determined. For any given limb segment, the length should be fixed, but the length information of that limb segment differs in different discrimination frames. Therefore, a preset normalization function can be used to normalize the length information of each limb segment, adjusting the length information of that limb segment in each discrimination frame to a single value. Based on the adjusted length information, the coordinate information of the preset joint points corresponding to that limb segment is adjusted so that the length information of the limb segment calculated from the adjusted coordinate information is the same in each discrimination frame. For example, if there are 10 discrimination frames, 10 length information values ​​can be calculated for each limb segment, and these 10 length information values ​​are normalized. Based on the processed length information, the coordinate information of the preset joint points corresponding to that limb segment is adjusted to obtain the adjusted coordinate information. For example, the coordinate information of a certain preset joint point in a limb segment can be fixed first, and the position of another preset joint point in the same limb segment can be adjusted according to the normalized length information, so that the lengths of the two preset joint points are consistent across 10 discrimination frames.

[0093] After obtaining the adjusted coordinate information, the adjusted coordinate information is then transformed according to the preset coordinate system before proceeding to the next step.

[0094] The advantage of this setup is that by performing normalization and adjusting the coordinate position, errors caused by different depths of field can be eliminated, improving the accuracy of area information determination and thus improving the accuracy of identity recognition.

[0095] S303. Determine the limb change information based on the area information of the limb triangle corresponding to the limb in each discrimination frame.

[0096] For example, during a user's walking process, limbs can swing, and the area information of the corresponding limb triangle will change accordingly. That is, for the same limb, the area information of the limb triangle formed by that limb can be different in different decision frames. Based on the changes in the area information of the limb triangle in all decision frames, the changes in the user's limb posture during walking can be obtained, i.e., the limb change information can be obtained. For example, as the area information increases, it can be determined that the amplitude of the user's swing is getting larger and larger.

[0097] Figure 4 This is a schematic diagram of a limb triangle. Figure 4 Taking the right arm as an example, the diagram shows the limb triangle of the right arm in two discrimination frames. Figure 4 As shown, the limb triangle of the right arm changes from its shape on the left to its shape on the right, and the area information of this limb triangle changes.

[0098] In this embodiment, if it is detected that the user is missing at least one preset joint point, the missing preset joint point can be identified as a missing joint point. For the limb or segment where the missing joint point is located, coordinate transformation processing can be omitted, that is, it is not necessary to calculate the area of ​​the triangle corresponding to the limb, or the default triangle area can be used for the limb.

[0099] In this embodiment, the change information of the limb is determined based on the area information of the limb triangle corresponding to the limb in each discrimination frame, including: performing curve fitting processing on the area information of the limb triangle corresponding to the limb in each discrimination frame to obtain the gait feature curve of the limb; wherein, the horizontal axis of the gait feature curve represents the discrimination frame, and the vertical axis represents the area information; the gait feature curve of the limb is determined as the change information of the limb.

[0100] Specifically, the swing amplitude, speed, acceleration, and phase of movement vary from individual to individual at different times, resulting in different gait characteristics. These differences directly affect the relative positions of different joints on the limb at different times, thereby influencing the shape of the pattern formed by the pre-defined joints on the limb.

[0101] This embodiment selects 12 preset joints, with each limb including 3 preset joints. For a specific limb, the shape formed by its preset joints is a triangle. The changing pattern of the triangle shape is a mapping of the motion characteristics of the preset joints on that limb. Therefore, by observing the changes in the area of ​​the limb triangle, the change information of the limb can be obtained.

[0102] For each limb, the area information of the limb triangle in each discrimination frame can be obtained. Curve fitting is then performed on the area information of each limb triangle, mapping the changes in the limb triangle onto a curve. The fitted curve is the gait feature curve of that limb. The horizontal axis of the gait feature curve represents the discrimination frame, i.e., time, and the vertical axis represents the area information. The gait feature curve of a limb is thus defined as the limb's change information. That is, the change information of a limb can be represented in the form of a curve.

[0103] Fitting can be performed based on Bézier curves. Specifically, Bézier curves are used to fit this area information, mapping the user's walking motion characteristics onto the curve. Bézier curves are chosen because their smooth and continuous derivative properties effectively avoid overfitting problems that may arise from higher-order polynomial curves and mitigate boundary bias. Furthermore, Bézier curves have strong global adjustment capabilities, making them particularly suitable for applications requiring global adjustment. In this embodiment, an n-order Bézier curve can be used based on the number of discrimination frames, where n equals the number of discrimination frames minus one. For example, if there are 10 discrimination frames, then n can be equal to 9. That is, a 9th-order Bézier curve is used for fitting, where the first discrimination frame is the starting point of the curve, the 10th discrimination frame is the ending point, and the points in the remaining discrimination frames are control points.

[0104] Figure 5 This is a schematic diagram of the gait characteristic curve of the right upper limb. The horizontal axis represents each discrimination frame, i.e., there are 10 discrimination frames, and the vertical axis represents the area information of the limb triangle of the right upper limb. The dashed line represents the curve before fitting, and the solid line represents the curve after fitting, i.e., the gait characteristic curve.

[0105] The advantage of this setup is that by performing curve fitting, the changes in limbs can be represented more clearly and intuitively, making it easier to analyze the characteristics of the user's walking posture based on the curve, thereby improving the efficiency and accuracy of user identification.

[0106] S304. Determine the user's gait feature information based on limb change information; wherein, gait feature information is used to characterize the user's walking posture.

[0107] For example, the change information is global information representing the user's walking posture. After obtaining the change information, key feature points that can characterize the user's gait can be extracted from the change information, that is, gait feature information can be obtained. In this way, the user can be identified based on the gait feature information, which effectively reduces the amount of computation for identification and improves the identification efficiency.

[0108] In this embodiment, the change information is represented by gait feature curves; based on the limb change information, the user's gait feature information is determined, including: determining the curve feature information of the limb's gait feature curve in at least one target dimension; wherein, the target dimension is a pre-set dimension that represents the characteristics of the curve, and the curve feature information is information that represents the characteristics of the curve in the target dimension; the curve feature information in each target dimension is determined as the user's gait feature information.

[0109] Specifically, gait characteristic information is determined based on limb changes, which means determining gait characteristic information based on gait characteristic curves. In other words, gait characteristic information is extracted from gait characteristic curves. For a curve, its characteristics can be characterized from multiple aspects, i.e., from multiple dimensions. Multiple dimensions can be pre-defined as target dimensions. The characteristics of the gait characteristic curve under the target dimensions are determined as curve characteristic information. That is, curve characteristic information is information that characterizes the curve's characteristics under the target dimensions. For example, target dimensions may include the ratio of the absolute value of the minimum slope to the absolute value of the maximum slope, the time period ratio of the maximum slope, the time period ratio of the minimum slope, the time period ratio of the maximum value, the ratio of the minimum value to the maximum value, etc., thus obtaining the data corresponding to the gait characteristic curve under these target dimensions. Combining the curve characteristic information of the gait characteristic curve under each target dimension yields the user's gait characteristic information. For example, the set of curve characteristic information under each target dimension can be used as gait characteristic information.

[0110] Multiple candidate dimensions can be pre-set, and the target dimension that best represents the characteristics of the curve can be pre-selected from the candidate dimensions. Experiments show that as walking speed decreases, step frequency decreases accordingly, gait cycle lengthens, the amplitude of the gait curve decreases, and the curve shape tends to be flatter. This indicates that slower walking speed is accompanied by smaller limb movement amplitude and gentler gait characteristics. As walking speed increases, step frequency increases accordingly, gait cycle shortens, the amplitude of the gait curve increases, and the curve shape tends to be steeper. This indicates that faster walking speed is accompanied by larger limb movement amplitude and more vigorous gait characteristics.

[0111] To eliminate the influence of different time periods during gait comparison, when selecting the target dimension, for time-related candidate dimensions, this embodiment compares the time corresponding to the candidate dimension with the elapsed time of all discrimination frames. For example, the elapsed time of all discrimination frames constitutes one time period. That is, the proportion of the time corresponding to the candidate dimension relative to the entire time period is determined, and this proportion is called the time period ratio of the candidate dimension. Furthermore, given that the minimum slope and the maximum slope have different signs, to simplify the subsequent calculation process, the ratio of the minimum slope to the maximum slope can be converted into the ratio of their absolute values. There are 24 pre-defined candidate dimensions, numbered from 1 to 24. These 24 candidate dimensions are: maximum slope, minimum slope, ratio of the absolute value of the minimum slope to the maximum slope, time period ratio of the maximum slope, time period ratio of the minimum slope, curve inflection point, curve standard deviation, curve midpoint, maximum value, minimum value, time period ratio of the maximum value to the minimum value, ratio of the minimum value to the maximum value, time period ratio of the zero-crossing point, quartiles, curve length, energy, peak interval, root mean square, curve period, skewness, local variance, mean absolute difference, and curve mean.

[0112] Among the candidate dimensions mentioned above, the superior candidate dimension is selected as the target dimension to construct gait feature information. A higher number of target dimensions does not necessarily lead to higher accuracy in identity recognition; sometimes, too many target dimensions can actually decrease accuracy. Therefore, the appropriate selection of target dimensions is crucial for recognition performance. This embodiment uses a decision tree classifier, based on a stepwise forward selection feature selection algorithm, to progressively construct a set of target dimensions through a greedy strategy and incremental feature fusion. For example, all candidate dimensions can be traversed, and the accuracy of each candidate dimension for identity recognition can be determined sequentially. The candidate dimension with the highest accuracy is then added to the target dimension set as the first target dimension. Each candidate dimension is then combined with the current target dimension to obtain the accuracy of that combination for identity recognition. The combination with the highest accuracy is selected, and the candidate dimensions in that combination are used as the second target dimension. New candidate dimensions are then added until the newly added candidate dimensions no longer improve the accuracy, thus obtaining the final set of all target dimensions.

[0113] The advantage of this setting is that by selecting a preset target dimension to extract curve features, the amount of computation can be reduced, the accuracy of identity recognition can be improved, and the user experience can be enhanced.

[0114] S305. Identify the user based on their gait characteristics.

[0115] For example, this step can refer to step S104 above, and will not be repeated here.

[0116] This invention provides a user gait-based identity recognition method. By acquiring multiple image frames, key frames can be identified as discrimination frames. Gait features are then recognized based on these discrimination frames, avoiding omissions and reducing computational load, thus improving efficiency and comprehensiveness. Coordinate information of preset joints corresponding to limbs is extracted from the discrimination frames. Based on this coordinate information, the changes in the limbs are determined, i.e., the posture changes of the user's limbs during walking. Gait feature information is extracted based on these changes, revealing the characteristics of the user's walking posture. User identity is then recognized based on these posture characteristics. By combining multiple discrimination frames for gait feature extraction, interference from noise or occlusion in the image can be reduced. Furthermore, gait features are dynamic and difficult to imitate with photographs or 3D masks, improving recognition accuracy and enhancing user experience.

[0117] Figure 6 This is a flowchart illustrating a user gait-based identity recognition method provided in an embodiment of the present invention. This embodiment is an optional embodiment based on the above embodiment.

[0118] In this embodiment, ...

[0119] like Figure 6 As shown, the method includes the following steps:

[0120] S601. Acquire multiple image frames from the user according to the preset acquisition cycle.

[0121] For example, an image acquisition cycle is preset, and the camera can acquire image frames according to the preset acquisition cycle. For instance, when a user is detected approaching, an image frame can be acquired once per second, thus obtaining multiple image frames. That is, multiple image frames of the user can be obtained according to the preset acquisition cycle.

[0122] S602. Based on multiple image frames, determine the user's gait cycle and obtain image frames within the gait cycle from the multiple image frames; wherein, the gait cycle represents the time period during which the user performs the same preset action while walking.

[0123] For example, when a user walks, their gait changes at a certain period. Based on multiple captured image frames, the user's gait period can be determined. The gait period represents the time interval between when a user performs a preset action and when the same preset action is performed again while walking. Figure 7 This is a schematic diagram of the gait cycle. Figure 7In gait analysis, the time elapsed from the moment the right foot is forward and the heel touches the ground to the moment the right foot is forward and the heel touches the ground again is considered a gait cycle.

[0124] In other words, a preset image recognition algorithm can be used to identify whether a preset action exists in each image frame, and the time between the two image frames where the preset action occurs is the user's gait cycle. In this embodiment, the preset image recognition algorithm is not specifically limited.

[0125] S603. Extract at least two discrimination frames from the image frames within the gait cycle, and extract the coordinate information of the preset joint points corresponding to the limbs from the discrimination frames.

[0126] For example, image frames within a gait cycle are obtained from all acquired image frames. Multiple image frames are then identified from these gait cycle frames as key image frames for user identification, i.e., discrimination frames. The coordinate information of each preset joint point is then extracted from each discrimination frame. Image frames arranged in a preset order within the gait cycle can be identified as discrimination frames; for example, the first, middle, and last frames within the gait cycle can be used as discrimination frames.

[0127] In this embodiment, at least two discrimination frames are extracted from the image frames within the gait cycle, including: dividing the image frames within the gait cycle equally to obtain a preset number of discrimination groups; wherein the image frames in the preset number of discrimination groups constitute the image frames within the gait cycle; and extracting one image frame from each discrimination group as a discrimination frame.

[0128] Specifically, for a gait cycle, the image frames within that gait cycle are evenly divided. A pre-set number of divisions can be used to divide the image frames of the entire gait cycle into a predetermined number of discrimination groups. That is, the image frames in the predetermined number of discrimination groups constitute a complete gait cycle. The gait cycle can be evenly divided temporally, or the image frames within the gait cycle can be evenly divided by the number of image frames. One image frame is extracted from each discrimination group as the discrimination frame. For example, the first frame in each discrimination group can be used as the discrimination frame, or the last frame in each discrimination group can be used as the discrimination frame.

[0129] In this embodiment, the extracted gait cycle can be divided into 10 equal parts on the time axis. That is, one gait cycle can extract 10 key image frames as discrimination frames for identity recognition.

[0130] The advantage of this setup is that by grouping image frames within the gait cycle, it ensures that the discrimination frames are distributed throughout all stages of the gait cycle, which facilitates the extraction of accurate gait feature information and improves the accuracy of user identification. Furthermore, extracting discrimination frames reduces subsequent computational load and improves the efficiency of identity recognition.

[0131] S604. Determine the limb change information based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame; wherein, the change information represents the change in the user's limb posture.

[0132] For example, this step can refer to step S102 above, and will not be repeated here.

[0133] S605. Determine the user's gait feature information based on limb change information; wherein, the gait feature information is used to characterize the user's walking posture.

[0134] For example, this step can refer to step S103 above, and will not be repeated here.

[0135] S606. Based on the user's gait characteristics, identify the user.

[0136] For example, this step can refer to step S104 above, and will not be repeated here.

[0137] This invention provides a user gait-based identity recognition method. By acquiring multiple image frames, key frames can be identified as discrimination frames. Gait features are then recognized based on these discrimination frames, avoiding omissions and reducing computational load, thus improving efficiency and comprehensiveness. Coordinate information of preset joints corresponding to limbs is extracted from the discrimination frames. Based on this coordinate information, the changes in the limbs are determined, i.e., the posture changes of the user's limbs during walking. Gait feature information is extracted based on these changes, revealing the characteristics of the user's walking posture. User identity is then recognized based on these posture characteristics. By combining multiple discrimination frames for gait feature extraction, interference from noise or occlusion in the image can be reduced. Furthermore, gait features are dynamic and difficult to imitate with photographs or 3D masks, improving recognition accuracy and enhancing user experience.

[0138] Figure 8 This is a structural block diagram of a user gait-based identity recognition device provided in an embodiment of the present invention. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 8The user gait-based identity recognition device 800 includes: a coordinate extraction unit 801, a change determination unit 802, a feature determination unit 803, and an identity recognition unit 804.

[0139] The coordinate extraction unit 801 is used to obtain at least two discrimination frames of the user from the acquired multi-frame image frames, and extract the coordinate information of the preset joint points corresponding to the limbs from the discrimination frames; wherein, the discrimination frame represents the image frame used to determine the user's gait, and each limb corresponds to at least three preset joint points.

[0140] The change determination unit 802 is used to determine the change information of the limbs based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame; wherein, the change information represents the change in the posture of the user's limbs;

[0141] The feature determination unit 803 is used to determine the user's gait feature information based on limb change information; wherein, the gait feature information is used to characterize the user's walking posture.

[0142] The identity recognition unit 804 is used to identify the user based on the user's gait feature information.

[0143] In one example, the change determination unit 802 includes:

[0144] The area determination module is used to determine the area information of the limb triangle corresponding to the limb for each discrimination frame based on the coordinate information of the preset joint points corresponding to the limb in the discrimination frame; wherein, the limb triangle represents the triangle enclosed by the preset joint points corresponding to the limb.

[0145] The change determination module is used to determine the change information of the limbs based on the area information of the limb triangles corresponding to the limbs in each discrimination frame.

[0146] In one example, the area determination module is specifically used for:

[0147] Based on the preset coordinate system, the coordinate information of the preset joint points in the discrimination frame is transformed to obtain the target coordinates of the preset joint points; where the target coordinates represent the coordinates of the preset joint points in the preset coordinate system.

[0148] Based on the target coordinates of the preset joint points corresponding to the limbs in the discrimination frame, the area information of the limb triangle corresponding to the limb is determined.

[0149] In one example, it also includes:

[0150] The joint point determination unit is used to determine the target joint point from the preset joint points corresponding to the limbs in the discrimination frame;

[0151] The coordinate system construction unit is used to determine the target joint point as the origin and construct a preset coordinate system.

[0152] In one example, each limb includes two segments, each segment consisting of two adjacent pre-defined joints within the limb; the device also includes:

[0153] The length determination unit is used to determine the length information of the limb segment corresponding to the limb based on the coordinate information of the preset joint point corresponding to the limb in the discrimination frame.

[0154] The coordinate transformation unit is used to adjust the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame according to the length information of the limb segments corresponding to the limbs in each discrimination frame, to obtain the adjusted coordinate information, and to perform coordinate transformation processing on the adjusted coordinate information of the preset joint points in the discrimination frame according to the preset coordinate system, so as to obtain the target coordinates of the preset joint points.

[0155] In one example, the change determination module is specifically used for:

[0156] Curve fitting is performed on the area information of the limb triangles corresponding to the limbs in each discrimination frame to obtain the gait feature curve of the limb; where the horizontal axis of the gait feature curve represents the discrimination frame and the vertical axis represents the area information.

[0157] The gait characteristic curve of the limb is used to identify the changes in the limb.

[0158] In one example, the change information is represented by gait feature curves; the feature determination unit 803 is specifically used for:

[0159] Determine the curve feature information of the gait feature curve of the limb in at least one target dimension; wherein, the target dimension is a pre-set dimension that characterizes the curve characteristics, and the curve feature information is information that characterizes the curve characteristics in the target dimension;

[0160] The curve feature information under each target dimension is determined as the user's gait feature information.

[0161] In one example, coordinate extraction unit 801 includes:

[0162] The image acquisition module is used to acquire multiple image frames from the user according to a preset acquisition period;

[0163] The cycle determination module is used to determine the user's gait cycle based on multiple image frames and to obtain image frames within the gait cycle from the multiple image frames; wherein, the gait cycle represents the time period during which the user performs the same preset action while walking;

[0164] The discrimination frame extraction module is used to extract at least two discrimination frames from image frames within the gait cycle.

[0165] In one example, the discriminant frame extraction module is specifically used for:

[0166] The image frames within the gait cycle are divided equally to obtain a preset number of discrimination groups; the image frames in the preset number of discrimination groups constitute the image frames within the gait cycle.

[0167] One image frame is extracted from each discrimination group to form the discrimination frame.

[0168] In one example, the identity recognition unit 804 is specifically used for:

[0169] Obtain preset standard feature information; wherein, the preset standard feature information is pre-stored correct gait feature information;

[0170] Determine the similarity information between the user's gait feature information and the preset standard feature information; whereby the similarity information represents the degree of closeness between the user's gait feature information and the preset standard feature information;

[0171] User identity is identified based on similarity information.

[0172] In one example, it also includes:

[0173] The vehicle control unit is used to execute the corresponding action in response to the user's operation command issued to the vehicle if it is determined that the user has passed the identity verification.

[0174] Figure 9 A structural block diagram of an electronic device provided in an embodiment of this application, such as... Figure 9 As shown, the electronic device includes: a memory 91 and a processor 92; the memory 91 is a memory used to store instructions executable by the processor 92.

[0175] The processor 92 is configured to perform the methods provided in the embodiments described above.

[0176] The electronic device also includes a receiver 93 and a transmitter 94. The receiver 93 is used to receive instructions and data sent by other devices, and the transmitter 94 is used to send instructions and data to external devices.

[0177] This application provides a vehicle equipped with a camera, which can be used to perform the following:

[0178] From the acquired multiple image frames, at least two discrimination frames of the user are obtained, and the coordinate information of the preset joint points corresponding to the limbs is extracted from the discrimination frames; wherein, the discrimination frame represents the image frame used to determine the user's gait, and each limb corresponds to at least three preset joint points.

[0179] Based on the coordinate information of the preset joint points corresponding to the limbs in each discrimination frame, the change information of the limbs is determined; whereby the change information represents the changes in the user's limb posture.

[0180] Based on limb changes, the user's gait characteristics are determined; these characteristics are used to characterize the user's walking posture.

[0181] User identification is performed based on the user's gait characteristics.

[0182] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. The device may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, personal digital assistant, vehicle, or other similar device.

[0183] The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0184] Processing component 1002 typically controls the overall operation of device 1000, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1002 may include one or more processors 1020 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1002 may include one or more modules to facilitate interaction between processing component 1002 and other components. For example, processing component 1002 may include a multimedia module to facilitate interaction between multimedia component 1008 and processing component 1002.

[0185] The device 1000 may include one or more of the following components: a processing component 1002, a memory 1004, a power supply component 1006, a multimedia component 1008, an audio component 1010, an input / output (I / O) interface 1012, a sensor component 1014, and a communication component 1016.

[0186] Memory 1004 is configured to store various types of data to support the operation of device 1000. Examples of such data include instructions for any application or method operating on device 1000, contact data, phonebook data, messages, pictures, videos, etc. Memory 1004 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0187] Power supply component 1006 provides power to various components of device 1000. Power supply component 1006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to device 1000.

[0188] The multimedia component 1008 includes a screen that provides an output interface between the device 1000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 1008 includes a front-facing camera and / or a rear-facing camera. When the device 1000 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0189] Audio component 1010 is configured to output and / or input audio signals. For example, audio component 1010 includes a microphone (MIC) configured to receive external audio signals when device 1000 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1004 or transmitted via communication component 1016. In some embodiments, audio component 1010 also includes a speaker for outputting audio signals.

[0190] I / O interface 1012 provides an interface between processing component 1002 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0191] Sensor assembly 1014 includes one or more sensors for providing state assessments of various aspects of device 1000. For example, sensor assembly 1014 may detect the on / off state of device 1000, the relative positioning of components such as the display and keypad of device 1000, changes in position of device 1000 or a component of device 1000, the presence or absence of user contact with device 1000, orientation or acceleration / deceleration of device 1000, and temperature changes of device 1000. Sensor assembly 1014 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1014 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1014 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0192] Communication component 1016 is configured to facilitate wired or wireless communication between device 1000 and other devices. Device 1000 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1016 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1016 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0193] In an exemplary embodiment, the apparatus 1000 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0194] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1004 including instructions, which can be executed by a processor 1020 of the device 1000 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0195] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the aforementioned user gait-based identity recognition method.

[0196] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0197] The above embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention.

Claims

1. A user gait-based identity recognition method, characterized in that, The method comprises the following steps: obtaining at least two frames of discriminant frames of a user from a plurality of collected image frames, and extracting coordinate information of preset joint points corresponding to a limb from the discriminant frames; wherein the discriminant frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint points; determining change information of the limb according to the coordinate information of the preset joint points corresponding to the limb in each discriminant frame; wherein the change information represents the posture change of the user's limb; determining gait feature information of the user according to the change information of the limb; wherein the gait feature information is used to represent the characteristics of the user's walking posture; performing identity recognition on the user according to the gait feature information of the user; determining the change information of the limb according to the coordinate information of the preset joint points corresponding to the limb in each discriminant frame, comprising: for each discriminant frame, determining the area information of the limb triangle corresponding to the limb according to the coordinate information of the preset joint points corresponding to the limb in the discriminant frame; wherein the limb triangle represents a triangle formed by the preset joint points corresponding to the limb; the preset joint points corresponding to the limb triangle of the upper limb include the shoulder joint, the elbow joint and the wrist joint, and the preset joint points corresponding to the limb triangle of the lower limb include the hip joint, the knee joint and the ankle joint; performing curve fitting processing on the area information of the limb triangle corresponding to the limb in each discriminant frame based on a Bezier curve to obtain a gait feature curve of the limb; determining the gait feature curve of the limb as the change information of the limb.

2. The method of claim 1, wherein, determining the area information of the limb triangle corresponding to the limb according to the coordinate information of the preset joint points corresponding to the limb in the discriminant frame, comprising: performing coordinate conversion processing on the coordinate information of the preset joint points in the discriminant frame based on a preset coordinate system to obtain target coordinates of the preset joint points; wherein the target coordinates represent the coordinates of the preset joint points in the preset coordinate system; determining the area information of the limb triangle corresponding to the limb according to the target coordinates of the preset joint points corresponding to the limb in the discriminant frame.

3. The method of claim 2, wherein, Further comprising: determining a target joint point from the preset joint points corresponding to the limb in the discriminant frame; determining the target joint point as the origin to construct the preset coordinate system.

4. The method of claim 2, wherein, The method further comprises: determining length information of a limb segment corresponding to the limb according to the coordinate information of the preset joint points corresponding to the limb in the discriminant frame; wherein the limb segment is composed of two adjacent preset joint points in the limb; adjusting the coordinate information of the preset joint points corresponding to the limb in the discriminant frame according to the length information of the limb segment corresponding to the limb in each discriminant frame to obtain adjusted coordinate information, and performing coordinate conversion processing on the adjusted coordinate information of the preset joint points in the discriminant frame based on a preset coordinate system to obtain target coordinates of the preset joint points.

5. The method of claim 1, wherein, The change information is represented by a gait feature curve; determining the gait feature information of the user according to the change information of the limb, comprising: Determine the gait feature curve of the limb in at least one target dimension; wherein the target dimension is a pre-set dimension representing the characteristics of the curve, and the curve characteristic information is information representing the characteristics of the curve in the target dimension; Determine the gait feature information of the user in each target dimension.

6. The method of claim 1, wherein, From the collected multiple image frames, at least two frames of discriminant frames of the user are obtained, including: According to the preset acquisition period, multiple image frames of the user are obtained; According to the multiple image frames, the gait cycle of the user is determined, and the image frames in the gait cycle are obtained from the multiple image frames; wherein the gait cycle represents the time period in which the user appears the same preset action when walking; At least two frames of discriminant frames are extracted from the image frames in the gait cycle.

7. The method of claim 6, wherein, At least two frames of discriminant frames are extracted from the image frames in the gait cycle, including: The image frames in the gait cycle are equally divided to obtain a preset number of discriminant groups; wherein the image frames in the preset number of discriminant groups constitute the image frames in the gait cycle; One frame of image frame is extracted from each discriminant group as the discriminant frame.

8. The method of claim 1, wherein, According to the gait feature information of the user, identity recognition of the user is performed, including: Obtain the preset standard feature information; wherein the preset standard feature information is the correct gait feature information stored in advance; Determine the similarity information between the gait feature information of the user and the preset standard feature information; wherein the similarity information represents the closeness between the gait feature information of the user and the preset standard feature information; According to the similarity information, identity recognition of the user is performed.

9. The method according to any one of claims 1-8, characterized in that, Further comprising: If it is determined that the user passes the identity recognition, the operation instruction issued by the user to the vehicle is responded to, and the action corresponding to the operation instruction is executed.

10. The method according to any one of claims 1-8, characterized in that, Further comprising: If it is determined that the user passes the identity recognition, the identity information of the user is obtained, the component adjustment information corresponding to the identity information is determined based on the preset association relationship; wherein the preset association relationship represents the association relationship between the identity information and the component adjustment information, and the component adjustment information is used to adjust the components in the vehicle.

11. A user gait-based identity recognition device, comprising: Including: A coordinate extraction unit is configured to obtain at least two frames of discriminant frames of the user from the collected multiple image frames, and extract coordinate information of preset joint nodes corresponding to the limbs from the discriminant frames; wherein the discriminant frames represent image frames used to determine the gait of the user, and each limb corresponds to at least three preset joint nodes; A change determination unit is configured to determine the change information of the limb according to the coordinate information of the preset joint nodes corresponding to the limb in each discriminant frame; wherein the change information represents the posture change of the limb of the user; A feature determination unit is configured to determine the gait feature information of the user according to the change information of the limb; wherein the gait feature information is used to represent the characteristics of the walking posture of the user; An identity recognition unit is configured to perform identity recognition of the user according to the gait feature information of the user; The change determination unit comprises: The area determination module is configured to determine, for each of the determination frames, area information of a limb triangle corresponding to the limb according to coordinate information of preset joint nodes corresponding to the limb in the determination frame; the limb triangle represents a triangle surrounded by the preset joint nodes corresponding to the limb; the preset joint nodes corresponding to the limb triangle of the upper limb include a shoulder joint, an elbow joint, and a wrist joint; the preset joint nodes corresponding to the limb triangle of the lower limb include a hip joint, a knee joint, and an ankle joint; The change determination module is configured to perform curve fitting processing on the area information of the limb triangle corresponding to the limb in each of the determination frames based on a Bezier curve, to obtain a gait feature curve of the limb; and determine the gait feature curve of the limb as change information of the limb.

12. An electronic device, comprising: The method comprises: a processor, and a memory connected to the processor in communication; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by the processor to implement the method according to any one of claims 1-10.

14. A computer program product, characterised in that, The computer program is executed by the processor to implement the method according to any one of claims 1-10.

15. A vehicle characterized by comprising: The vehicle is used to implement the method according to any one of claims 1-10.

Images