Abnormal behavior recognition method and device based on gait detection, electronic equipment and medium

Abstract

The invention provides an abnormal behavior recognition method and device based on gait detection, electronic equipment and a medium, and relates to the field of artificial intelligence. The method comprises the steps that a pre-constructed gait database is obtained, and multiple pieces of reference face information and reference gait data corresponding to each piece of reference face information are stored in the gait database; responding to the detected real-time face information and real-time gait data of any target object in the detected area, and matching the real-time face information and the real-time gait data with a gait database; and according to a matching result, determining whether the target object has an abnormal behavior.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and particularly to an abnormal behavior recognition method, device, electronic device, storage medium and program product based on gait detection. Background Art

[0002] With the development of computer science and technology and the progress of human society, people's awareness of safety prevention in aspects such as life, work, and travel has gradually increased, and how to ensure social security has become the most serious problem faced by people today. Nowadays, as an effective way to protect people's lives and property, biometric recognition technologies such as face recognition, voice recognition, and iris recognition are increasingly widely used.

[0003] Gait recognition is a relatively new biometric authentication technology that has been increasingly concerned by more and more researchers in recent years. Gait refers to the way people walk, which is a complex behavioral characteristic. Especially when a person's mentality changes, especially when preparing to perform abnormal behaviors, there are subtle differences in his muscle strength, center of gravity, and walking "style" compared with normal situations.

[0004] However, the existing gait recognition takes a long time and cannot comprehensively, accurately and timely detect abnormal behaviors and abnormal emotions of personnel. Summary of the Invention

[0005] In view of the above problems, the present disclosure provides an abnormal behavior recognition method, device, electronic device, storage medium and program product based on gait detection.

[0006] According to a first aspect of the present disclosure, there is provided an abnormal behavior recognition method based on gait detection, including: obtaining a pre-constructed gait database, where the gait database stores a plurality of reference facial information and reference gait data corresponding to each reference facial information; in response to detecting real-time facial information and real-time gait data of any target object in the detected area, respectively matching the real-time facial information and real-time gait data with the gait database; and determining whether the target object has an abnormal behavior according to the matching result.

[0007] According to an embodiment of the present disclosure, the gait database is pre-constructed in the following manner: within a first preset time period, detecting first facial information in the detected area; storing the detected multiple first facial information in a facial information library; within a second preset time period later than the first preset time period, detecting second facial information in the detected area, and matching the second facial information with the facial information library; in response to the second facial information matching any first facial information in the facial information library, determining the second facial information as reference facial information, and capturing reference gait data corresponding to the reference facial information; and storing the multiple reference facial information and the reference gait data corresponding to each reference facial information in the gait database.

[0008] According to an embodiment of the present disclosure, after matching the second facial information with the facial information database, it further includes: in response to the second facial information not matching any of the first facial information in the facial information database, returning an operation of detecting the first facial information in the measured area within a first preset period.

[0009] According to an embodiment of the present disclosure, the reference gait data includes step spacing, step frequency, and the relationship between multiple specified parts of the legs during movement; capturing the reference gait data corresponding to the reference facial information includes: collecting a video stream of the reference facial information within a third preset period, where the third preset period is later than the second preset period; splitting the video stream into multiple frame images, analyzing the reference gait data of each frame image in the multiple frame images; and determining the reference gait data corresponding to the reference facial information according to the multiple reference gait data of the multiple frame images.

[0010] According to an embodiment of the present disclosure, determining whether an abnormal behavior occurs in the target object according to the matching result includes: determining the attention level of the target object according to the matching result, where the attention levels are different under different matching results; and determining that the target object has an abnormal behavior when the attention level of the target object is a specified attention level.

[0011] According to an embodiment of the present disclosure, determining the attention level of the target object according to the matching result includes: matching the real-time facial information with any of the reference facial information in the gait database, and calculating the similarity between the real-time gait data and the reference gait data of the corresponding real-time gait data in the gait database when the matching is successful; determining that the attention level of the target object is the first attention level when the similarity is higher than the first similarity threshold; determining that the attention level of the target object is the second attention level when the similarity is between the second similarity threshold and the first similarity threshold, where the second similarity threshold is lower than the first similarity threshold; and determining that the attention level of the target object is the third attention level when the similarity is lower than the second similarity threshold, where the first attention level, the second attention level, and the third attention level deepen step by step.

[0012] According to an embodiment of the present disclosure, the method further includes: issuing a warning message in response to determining that the target object has an abnormal behavior.

[0013] The second aspect of the present disclosure provides an abnormal behavior recognition device based on gait detection, including: a gait data acquisition module for acquiring a pre-constructed gait database, where the gait database stores multiple reference facial information and reference gait data corresponding to each reference facial information; a gait data matching module for, in response to detecting real-time facial information and real-time gait data of any target object within the measured area, respectively matching the real-time facial information and real-time gait data with the gait database; and an abnormal behavior recognition module for determining whether the target object has an abnormal behavior according to the matching result.

[0014] The third aspect of the present disclosure provides an electronic device, including: one or more processors; a memory for storing one or more computer programs, where the above one or more processors execute the above one or more computer programs to implement the steps of the above method.

[0015] The fourth aspect of the present disclosure further provides a computer-readable storage medium having stored thereon a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented.

[0016] The fifth aspect of the present disclosure further provides a computer program product, including a computer program or instruction, and when the computer program or instruction is executed by a processor, the steps of the above method are implemented. Description of the Drawings

[0017] Through the following description of the embodiments of the present disclosure with reference to the drawings, the above content and other objects, features, and advantages of the present disclosure will become clearer. In the drawings:

[0018] Figure 1 Schematically shows an application scenario diagram of an abnormal behavior recognition method, device, equipment, medium, and program product based on gait detection according to an embodiment of the present disclosure;

[0019] Figure 2 Schematically shows a flowchart of an abnormal behavior recognition method based on gait detection according to an embodiment of the present disclosure;

[0020] Figure 3 Schematically shows a flowchart of constructing a gait database according to an embodiment of the present disclosure;

[0021] Figure 4 Schematically shows a flowchart of determining whether a target object has an abnormal behavior according to an embodiment of the present disclosure;

[0022] Figure 5 Schematically shows a block diagram of an abnormal behavior recognition device based on gait detection according to an embodiment of the present disclosure;

[0023] Figure 6A block diagram of an electronic device suitable for implementing an abnormal behavior recognition method based on gait detection according to an embodiment of the present disclosure is schematically shown. Detailed implementation manners

[0024] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the following detailed description, for the sake of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, obviously, one or more embodiments can also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present disclosure.

[0025] The terms used herein are merely for describing specific embodiments and are not intended to limit the present disclosure. The terms "including", "comprising", etc. used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0027] In the case of using expressions such as "at least one of A, B, and C", generally, it should be interpreted according to the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include, but is not limited to, a system having only A, only B, only C, having A and B, having A and C, having B and C, and / or having A, B, and C).

[0028] In the technical solution of the present disclosure, the user information involved (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties. Moreover, the processing of relevant data, such as collection, storage, use, processing, transmission, provision, disclosure, and application, all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] In the scenario of making automated decisions using personal information, the methods, devices, and systems provided by the embodiments of the present disclosure all provide corresponding operation entrances for users to choose to consent to or reject the results of automated decisions; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision" here refers to the activity of automatically analyzing and evaluating an individual's behavior habits, interests, or economic, health, credit status, etc. through a computer program and making decisions. The expression "expert decision" here refers to the activity of making decisions by personnel who are engaged in work in a specific field, have specialized experience, knowledge, and skills, and have reached a certain professional level.

[0030] An embodiment of the present disclosure provides a method for recognizing abnormal behaviors based on gait detection, including: obtaining a pre-constructed gait database, where the gait database stores multiple reference facial information and the reference gait data corresponding to each reference facial information; in response to detecting the real-time facial information and real-time gait data of any target object in the measured area, respectively matching the real-time facial information and real-time gait data with the gait database; and determining whether the target object has an abnormal behavior according to the matching result.

[0031] Figure 1 Schematically shows an application scenario diagram of a method, device, equipment, medium, and program product for recognizing abnormal behaviors based on gait detection according to an embodiment of the present disclosure.

[0032] As Figure 1 shown, the application scenario 100 according to this embodiment may include a gait detection device 101 and a server 102. The network is a medium for providing a communication link between the gait detection device 101 and the server 102, and this network may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0033] The gait detection device 101 is an electronic device that can provide functions for detecting facial information and gait data, such as surveillance cameras, webcams, wearable devices, image sensors, pressure sensors, inertial sensors, etc. The server 102 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.

[0034] The gait detection device 101 can detect the real-time facial information and real-time gait data of any target object in the measured area, and send the real-time facial information and real-time gait data to the server 102 via the network.

[0035] The server 102 can, based on the obtained gait database, analyze and process the real-time facial information and real-time gait data sent by the gait detection device 101 to determine whether the target object has an abnormal behavior.

[0036] For example, the server 102 can obtain a pre-constructed gait database, where the gait database stores multiple reference facial information and reference gait data corresponding to each reference facial information; when the server 102 receives the real-time facial information and real-time gait data sent by the gait detection device 101, it matches the real-time facial information and real-time gait data with the gait database respectively, and then determines whether the target object has abnormal behavior according to the matching result.

[0037] It should be noted that the abnormal behavior recognition method based on gait detection provided by the embodiments of the present disclosure can generally be executed by the server 102. Correspondingly, the abnormal behavior recognition device based on gait detection provided by the embodiments of the present disclosure can generally be set in the server 102. The abnormal behavior recognition method based on gait detection provided by the embodiments of the present disclosure can also be executed by a server or a server cluster different from the server 102 and capable of communicating with the gait detection device 101 and / or the server 102. Correspondingly, the abnormal behavior recognition device based on gait detection provided by the embodiments of the present disclosure can also be set in a server or a server cluster different from the server 102 and capable of communicating with the gait detection device 101 and / or the server 102.

[0038] It should be understood that Figure 1 the numbers of the gait detection devices, networks, and servers in

[0039] are merely illustrative. According to the implementation requirements, there can be any number of gait detection devices, networks, and servers. Figure 1 are merely illustrative. According to the implementation requirements, there can be any number of gait detection devices, networks, and servers. Figures 2 to 4 The following will be based on

[0040] Figure 2 The flowchart of the abnormal behavior recognition method based on gait detection according to the embodiments of the present disclosure is schematically shown.

[0041] As Figure 2 shown, the abnormal behavior recognition method based on gait detection in this embodiment includes operation S210 to operation S230, and this method can be executed by the server.

[0042] In operation S210, obtain a pre-constructed gait database, where the gait database stores multiple reference facial information and reference gait data corresponding to each reference facial information.

[0043] The gait database is constructed before real-time detection of gait data, and its construction method will be described in detail later.

[0044] It can be understood that the gait database stores multiple reference facial information and corresponding reference gait data for each piece of reference facial information. Among them, the multiple reference facial information can be the facial information of the same target object at multiple different times, or the different facial information of different target objects. Each piece of reference facial information has a corresponding reference gait data, and the mutually corresponding reference facial information and reference gait data belong to the same target object.

[0045] In operation S220, in response to detecting the real-time facial information and real-time gait data of any target object within the measured area, the real-time facial information and real-time gait data are respectively matched with the gait database.

[0046] In the embodiments of the present disclosure, before detecting the real-time facial information and real-time gait data of any target object within the measured area, consent or authorization from the target object, such as a user, can be obtained. For example, before operation S220, a request to obtain the real-time facial information and real-time gait data of the user within the measured area can be sent to the user. When the user consents or authorizes the acquisition, operation S220 is executed.

[0047] For example, after constructing the gait database, the real-time facial information and real-time gait data of any target object within the measured area can be detected through a gait detection device, and then the real-time facial information and real-time gait data are sent to the server via the network. When the server receives the real-time facial information and real-time gait data sent by the gait detection device, it can call the pre-constructed gait database and match it with the received real-time facial information and real-time gait data.

[0048] The measured area can be an area with high security requirements, such as a public square, bank, railway station, bus station, airport, community, industrial park, office area, etc. The facial information can be human face information.

[0049] For example, the measured area can be an office area, and the target object can be a person entering and leaving the office area. When the real-time facial information and real-time gait data of a certain person entering and leaving the office area are detected, the server can respectively match the real-time facial information and real-time gait data with the obtained gait database.

[0050] In operation S230, according to the matching result, it is determined whether the target object has abnormal behavior.

[0051] Based on the matching results of the real-time facial information, real-time gait data and the gait database respectively, it is determined whether the currently detected target object has abnormal behavior.

[0052] Through the embodiments of the present disclosure, when the real-time facial information and real-time gait data of any target object entering or exiting the measured area are detected, the real-time facial information and real-time gait data can be respectively matched with a pre-constructed gait database, and then according to the matching results, it can be determined whether the target object has abnormal behavior. In this way, based on gait detection, the recognition accuracy and recognition speed of personnel abnormal behavior can be improved, and it has very broad application scenarios and application spaces.

[0053] For example, for some occasions with high security requirements, based on the detected real-time gait data, this method can comprehensively, accurately and timely detect personnel abnormal behavior and abnormal emotions. In addition, for some specific occasions, such as when close-range, contact-based biometric recognition technologies are not applicable, this method can also be well applied.

[0054] Figure 3 Schematically shows a flowchart of constructing a gait database according to an embodiment of the present disclosure.

[0055] As Figure 3 shown, in some embodiments, the gait database in the above operation S210 is pre-constructed through the following operations S301 to S305.

[0056] In operation S301, within a first preset time period, the first facial information within the measured area is detected.

[0057] In the embodiments of the present disclosure, before detecting the first facial information of the user within the measured area, the consent or authorization of the user can be obtained. For example, before operation S301, a request to obtain the first facial information of the user can be sent to the user. When the user consents or authorizes to obtain it, operation S301 is executed.

[0058] For example, within a first preset time period, the first facial information of any first target object entering or exiting the measured area can be detected by a gait detection device such as a surveillance camera. The first target object can be any person in the measured area such as an office area, and the first facial information of this person can be detected by the surveillance camera.

[0059] In operation S302, the detected multiple first facial information is stored in the facial information library.

[0060] For example, the first facial information of multiple first target objects detected within the first preset time period can be stored in the facial information library. Thus, the facial information library stores multiple first facial information. Also for example, the first facial information of a first target object at multiple different times detected within the first preset time period can be stored in the facial information library.

[0061] In operation S303, within a second preset period that is later than the first preset period, second facial information within the area to be measured is detected, and the second facial information is matched with the facial information database.

[0062] In an embodiment of the present disclosure, before detecting the second facial information of the user within the area to be measured, consent or authorization of the user may be obtained. For example, before operation S303, a request to obtain the user's second facial information may be sent to the user. When the user consents or authorizes the acquisition, operation S303 is executed.

[0063] For example, the first preset period may be the first month, and the second preset period may be the second month. Based on the detected multiple first facial information in the first month, a facial information database may be constructed; then, in the second month, the second facial information within the area to be measured is detected, and the second facial information is matched with the previously constructed facial information database.

[0064] In operation S304, in response to the second facial information matching any one of the first facial information in the facial information database, the second facial information is determined as the reference facial information, and the reference gait data corresponding to the reference facial information is captured.

[0065] It can be understood that the first detection of the first facial information within the first preset period is to construct the facial information database. If the second facial information matches any one of the first facial information in the facial information database, that is, the same facial information is detected again within the second preset period, it means that the second facial information has been included in the facial information database.

[0066] To distinguish from the second facial information that does not match any one of the first facial information in the facial information database, the second facial information that has been included in the facial information database may be marked as the reference facial information, and at this time, the reference gait data corresponding to the reference facial information may be captured by the monitoring camera.

[0067] In an embodiment of the present disclosure, before capturing the reference gait data of the user, consent or authorization of the user may be obtained. For example, before operation S304, a request to obtain the user's reference gait data may be sent to the user. When the user consents or authorizes the acquisition, operation S304 is executed.

[0068] In this embodiment, the reference gait data includes the step interval, the step frequency, and the relationship between multiple specified parts of the legs during the movement process.

[0069] In operation S305, multiple reference facial information and the reference gait data of each reference facial information are stored in the gait database.

[0070] In this way, a gait database can be constructed to comprehensively and fully detect the facial information and gait data of the people who frequently enter and exit the area to be measured, for subsequent real-time gait detection and abnormal behavior recognition.

[0071] It should be noted that the first preset time period is earlier than the second preset time period in terms of time. The durations of the first preset time period and the second preset time period can be set according to actual needs to comprehensively and fully detect the facial information and gait data of the people who frequently enter and exit the area to be measured.

[0072] Such as Figure 3 As shown, in some embodiments, after the above operation S303 matches the second facial information with the facial information database, it further includes: in response to the second facial information not matching any of the first facial information in the facial information database, returning to the operation of detecting the first facial information in the area to be measured within the first preset time period.

[0073] It can be understood that if the second facial information does not match any of the first facial information in the facial information database, that is, the second facial information detected within the second preset time period is not included in the facial information database. At this time, the above operation S301 can be returned to for supplementary recording of facial information to construct the facial information database.

[0074] For example, when returning to the above operation S301 for supplementary recording of facial information, the relevant employees can pass through the straight channel alone for supplementary recording. During the supplementary recording process, the gait data under this facial information can be enriched, and the gait data of the target object in multiple mental states such as excitement, depression, and sadness can be detected.

[0075] In some embodiments, the above operation S304 of capturing the reference gait data corresponding to the reference facial information includes:

[0076] Collecting the video stream of the reference facial information within the third preset time period, where the third preset time period is later than the second preset time period;

[0077] Splitting the video stream into multiple frames of images and analyzing the reference gait data of each frame of image in the multiple frames of images;

[0078] Determining the reference gait data corresponding to the reference facial information according to the multiple reference gait data of the multiple frames of images.

[0079] For example, the monitoring camera can be placed on the side of a straight channel facing the area to be measured to intuitively capture the straight movement dynamics of the target object that frequently enters and exits the area to be measured. The video stream of the target object in the normal straight state within the third preset time period is extracted, and the extracted video stream is split into multiple frames of images.

[0080] For example, the third preset time period can be 3s to 5s, and the video stream within 3s to 5s of normal straight walking can be split into 20 frame images.

[0081] Then, based on the reference gait data when each frame of the multi-frame images is static, the reference gait data corresponding to the reference facial information of the target object is formed. For example, through the multi-frame images, reference gait information such as step distance and step frequency can be detected; it is also possible to screen out 5 pictures with the maximum distance between the two legs from the fixed-frame images, and the step frequency or step distance and other reference gait information can be determined through the maximum distance between the two legs.

[0082] Next, after the gait database is constructed, real-time detection of gait data can be performed. The real-time facial information and real-time gait data of the detected target object are respectively matched with the pre-constructed gait database. According to the matching results, it can be determined whether the target object has abnormal behavior.

[0083] Figure 4 Schematically shows a flowchart of determining whether a target object has abnormal behavior according to an embodiment of the present disclosure.

[0084] As Figure 4 shown, in some embodiments, the above operation S230 determines whether the target object has abnormal behavior according to the matching results, and may include operations S401 to S402.

[0085] In operation S401, according to the matching results, the attention level for the target object is determined, where the attention levels under different matching results are different.

[0086] Since the real-time facial information and real-time gait data of the detected target object need to be respectively matched with the pre-constructed gait database, there are various possible matching results for these two aspects of matching, and the attention levels for the target object under different matching results are different.

[0087] In operation S402, when the attention level for the target object is the specified attention level, it is determined that the target object has abnormal behavior.

[0088] A specified attention level can be determined from multiple attention levels under different matching results. If the attention level for the target object is determined to be the specified attention level according to the matching results, it can be determined that the target object has abnormal behavior.

[0089] In some embodiments, the above operation S401 determines the attention level for the target object according to the matching results, and may further include:

[0090] Match the real-time facial information with any reference facial information in the gait database. In the case of successful matching, calculate the similarity between the real-time gait data and the reference gait data corresponding to the real-time gait data in the gait database;

[0091] When the similarity is higher than the first similarity threshold, determine that the attention level for the target object is the first attention level;

[0092] When the similarity is between the second similarity threshold and the first similarity threshold, determine that the attention level for the target object is the second attention level, where the second similarity threshold is lower than the first similarity threshold;

[0093] When the similarity is lower than the second similarity threshold, determine that the attention level for the target object is the third attention level, where the first attention level, the second attention level, and the third attention level increase gradually.

[0094] In this way, during the matching process, first match the real-time facial information with the gait database. Only when the real-time facial information has been included in the gait database, then match the real-time gait data with the gait database.

[0095] If the real-time facial information fails to match with any reference facial information in the gait database, it indicates that the real-time facial information has not been included in the gait database. At this time, the target object is regarded as an abnormal object, such as an outsider, and will not enter the subsequent gait data matching process. At this time, it can be determined that the attention level for the target object is the third attention level.

[0096] For example, the first similarity threshold can be 60%, and the second similarity threshold can be 20%. During the process of matching the real-time gait data with the gait database, according to the preset second similarity threshold and the first similarity threshold, three similarity intervals can be determined, including the similarity being higher than the first similarity threshold, the similarity being between the second similarity threshold and the first similarity threshold, and the similarity being lower than the second similarity threshold. Based on these three different similarity intervals, different attention levels for the target object can be determined.

[0097] For example, the first attention level can be the normal level, the second attention level can be the special attention level, and the third attention level can be the alarm level. The third attention level can be set as the specified attention level.

[0098] In some embodiments, during the process of matching real-time gait data with reference gait data in a gait database, it can also be directly achieved by comparing the number of photos of the maximum distance between the two legs during walking. This solution has simple data processing and does not require a large amount of calculation and comparison, but the accuracy is relatively low. In some embodiments, the real-time gait data can also be directly compared with each frame image of the reference gait data. This solution requires a large amount of calculation, but the comparison result is relatively accurate.

[0099] In this way, based on the detected real-time gait data, abnormal behaviors of personnel can be detected comprehensively, accurately and in a timely manner.

[0100] In some embodiments, the method further includes: in response to determining that an abnormal behavior occurs to a target object, sending out a warning message.

[0101] For example, the warning message can be pushed on the display module of the server to prompt the management personnel in the measured area to pay special attention to the target object or conduct an artificial intervention conversation. The server can also be connected to an alarm module such as a buzzer so that the management personnel in the measured area can timely learn the warning message and take corresponding measures or means.

[0102] It can be understood that since the psychological state, work pressure, abnormal behaviors, etc. of personnel will all affect the fixed gait actions of personnel, the management personnel can, according to the warning message, timely conduct heart-to-heart talks with the relevant personnel to understand the psychological state of the employees, or transfer them from key positions, etc., to reduce the probability of occurrence of risk events.

[0103] The above is only an exemplary illustration, and the embodiments of the present disclosure are not limited thereto. For example, in some embodiments, the real-time gait data may not be matched with the gait database. In the case where the real-time gait data indicates that a person looks around or hesitates in steps and paces back and forth during normal walking, the person can be directly included in the alarm level.

[0104] Based on the above abnormal behavior recognition method based on gait detection, the present disclosure also provides an abnormal behavior recognition device based on gait detection. The following will be combined with Figure 5 to describe this device in detail.

[0105] Figure 5 Schematically shows a block diagram of an abnormal behavior recognition device based on gait detection according to an embodiment of the present disclosure.

[0106] As Figure 5 shown, the abnormal behavior recognition device 500 based on gait detection in this embodiment includes a gait data acquisition module 510, a gait data matching module 520 and an abnormal behavior recognition module 530.

[0107] The gait data acquisition module 510 is configured to acquire a pre-constructed gait database, where the gait database stores multiple reference facial information and the reference gait data corresponding to each reference facial information. In one embodiment, the gait data acquisition module 510 may be configured to perform the operation S210 described above, which will not be elaborated herein.

[0108] The gait data matching module 520 is configured to, in response to detecting the real-time facial information and real-time gait data of any target object within the measured area, match the real-time facial information and real-time gait data with the gait database respectively. In one embodiment, the gait data matching module 520 may be configured to perform the operation S220 described above, which will not be elaborated herein.

[0109] The abnormal behavior recognition module 530 is configured to determine whether the target object has an abnormal behavior according to the matching result. In one embodiment, the abnormal behavior recognition module 530 may be configured to perform the operation S230 described above, which will not be elaborated herein.

[0110] It should be noted that the embodiments of the apparatus part are similar to the embodiments of the method part, and the achieved technical effects are also similar. For specific details, please refer to the method embodiment part above, which will not be elaborated herein.

[0111] According to an embodiment of the present disclosure, any multiple of the gait data acquisition module 510, the gait data matching module 520, and the abnormal behavior recognition module 530 may be combined and implemented in one module, or any one of them may be split into multiple modules. Alternatively, at least part of the functions of one or more of these modules may be combined with at least part of the functions of other modules and implemented in one module. According to an embodiment of the present disclosure, at least one of the gait data acquisition module 510, the gait data matching module 520, and the abnormal behavior recognition module 530 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable way of integrating or packaging circuits, etc., implemented by hardware or firmware, or implemented in any one of the three implementation manners of software, hardware, and firmware, or in any suitable combination of several of them. Alternatively, at least one of the gait data acquisition module 510, the gait data matching module 520, and the abnormal behavior recognition module 530 may be at least partially implemented as a computer program module, and when the computer program module is run, it can perform the corresponding functions.

[0112] Figure 6 A block diagram of an electronic device suitable for implementing an abnormal behavior recognition method based on gait detection according to an embodiment of the present disclosure is schematically shown.

[0113] As Figure 6 shown, the electronic device 600 according to an embodiment of the present disclosure includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage section 608 into a random access memory (RAM) 603. The processor 601 can include, for example, a general-purpose microprocessor (e.g., CPU), an instruction set processor, and / or a related chipset, and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 can also include on-board memory for caching purposes. The processor 601 can include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0114] In the RAM 603, various programs and data required for the operation of the electronic device 600 are stored. The processor 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to an embodiment of the present disclosure by executing the program in the ROM 602 and / or the RAM 603. It should be noted that the program can also be stored in one or more memories other than the ROM 602 and the RAM 603. The processor 601 can also perform various operations of the method flow according to an embodiment of the present disclosure by executing the program stored in the one or more memories.

[0115] According to an embodiment of the present disclosure, the electronic device 600 can further include an input / output (I / O) interface 605, and the input / output (I / O) interface 605 is also connected to the bus 604. The electronic device 600 can further include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, a mouse, etc.; an output section 607 including, for example, a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 606 including a network interface card such as a LAN card, a modem, etc. The communication section 606 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed so that a computer program read from it can be installed into the storage section 608 as needed.

[0116] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or may exist alone without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the methods according to the embodiments of the present disclosure are implemented.

[0117] According to an embodiment of the present disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program, and this program can be used by or combined with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the above-described ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603.

[0118] Embodiments of the present disclosure also include a computer program product, which includes a computer program that contains program code for executing the method shown in the flowchart. When the computer program product runs in a computer system, the program code is used to enable the computer system to implement the abnormal behavior recognition method based on gait detection provided by the embodiments of the present disclosure.

[0119] When the computer program is executed by the processor 601, the above functions defined in the system / apparatus of the embodiments of the present disclosure are executed. According to an embodiment of the present disclosure, the above-described systems, apparatuses, modules, units, etc. may be implemented by computer program modules.

[0120] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices and magnetic storage devices. In another embodiment, the computer program may also be transmitted and distributed in the form of a signal on a network medium, and is downloaded and installed through the communication part 606, and / or installed from the removable medium 611. The program code included in the computer program can be transmitted by any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0121] In such an embodiment, the computer program can be downloaded and installed from a network through the communication section 606, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiments of the present disclosure are performed. According to the embodiments of the present disclosure, the above-described systems, devices, apparatuses, modules, units, etc. can be implemented by computer program modules.

[0122] According to the embodiments of the present disclosure, the program code for executing the computer program provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level procedures and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages include, but are not limited to, such as Java, C++, python, the "C" language, or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, by connecting through the Internet using an Internet service provider).

[0123] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0124] Those skilled in the art can understand that the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present disclosure. In particular, without departing from the spirit and teachings of the present disclosure, the features described in the various embodiments of the present disclosure can be combined and / or combined in various ways. All such combinations and / or combinations fall within the scope of the present disclosure.

[0125] The embodiments of the present disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although the embodiments have been described separately above, this does not mean that the measures in each embodiment cannot be used advantageously in combination. Without departing from the scope of the present disclosure, those skilled in the art can make various substitutions and modifications, and all such substitutions and modifications should fall within the scope of the present disclosure.

Claims

1. A method for identifying abnormal behavior based on gait detection, characterized in that: The method comprises: Acquire a pre-built gait database, wherein the gait database stores a plurality of reference facial information and reference gait data corresponding to each reference facial information; In response to detecting real-time facial information and real-time gait data of any target object in the detected area, matching the real-time facial information and real-time gait data with the gait database respectively; According to the matching result, it is determined whether the target object has abnormal behavior.

2. The method according to claim 1, characterized in that The gait database is pre-constructed in the following way: Detecting first facial information within the detected area within a first preset time period; storing the detected plurality of first facial information into a facial information database; In a second preset time period later than the first preset time period, detecting second facial information in the detected area, and matching the second facial information with the facial information database; In response to the second facial information matching any first facial information in the facial information database, determining the second facial information as reference facial information, and capturing reference gait data corresponding to the reference facial information; The plurality of reference facial information and the reference gait data of each reference facial information are stored in a gait database.

3. The method according to claim 2, characterized in that After matching the second facial information with the facial information database, the method further includes: In response to the second facial information not matching any first facial information in the facial information library, returning to the operation of detecting the first facial information in the detected area within a first preset time period.

4. The method according to claim 2, characterized in that: The reference gait data includes step distance, step frequency, and the relationship between multiple designated parts of the leg during movement; The capturing of reference gait data corresponding to the reference facial information comprises: Collecting a video stream of the reference facial information within a third preset time period, wherein the third preset time period is later than the second preset time period; Splitting the video stream into multiple frames of images, and analyzing the reference gait data of each frame of the multiple frames of images; The reference gait data corresponding to the reference facial information is determined according to the plurality of reference gait data of the plurality of frames of images.

5. The method according to claim 1, characterized in that Determining whether the target object has abnormal behavior according to the matching result includes: Determining, according to the matching results, a level of attention to the target object, wherein the level of attention is different for different matching results; When the attention level to the target object is a designated attention level, it is determined that abnormal behavior occurs to the target object.

6. The method according to claim 5, characterized in that Determining the attention level of the target object according to the matching result includes: Matching the real-time facial information with any reference facial information in the gait database, and if the match is successful, calculating the similarity between the real-time gait data and the reference gait data in the gait database corresponding to the real-time gait data; When the similarity is higher than a first similarity threshold, determining the attention level to the target object as a first attention level; When the similarity is between a second similarity threshold and the first similarity threshold, determining that the attention level to the target object is a second attention level, wherein the second similarity threshold is lower than the first similarity threshold; When the similarity is lower than the second similarity threshold, the attention level to the target object is determined to be a third attention level, wherein the first attention level, the second attention level and the third attention level are gradually deepened.

7. The method according to claim 1 or 5, characterized in that: The method further comprises: In response to determining that the target object has abnormal behavior, an early warning message is issued.

8. An abnormal behavior recognition device based on gait detection, characterized in that: The device comprises: A gait data acquisition module, used to acquire a pre-built gait database, wherein the gait database stores a plurality of reference facial information and reference gait data corresponding to each reference facial information; a gait data matching module, configured to, in response to detecting real-time facial information and real-time gait data of any target object in the detected area, match the real-time facial information and real-time gait data with the gait database respectively; The abnormal behavior identification module is used to determine whether the target object has abnormal behavior based on the matching result.

9. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images