Artificial intelligence-based behavior detection method, device and storage medium

Abstract

The application discloses a behavior detection method and device based on artificial intelligence and a storage medium. The application obtains image frames corresponding to a to-be-processed video, then performs image segmentation on the image frames to obtain passenger information in the image frames, further obtains key point information of each part of a human body according to the passenger information, and transmits the key point information of each part of the human body to a cloud server, so that the cloud server performs abnormal behavior early warning according to the key point information of each part of the human body. The application can accurately identify the image frames and obtain the key point information of each part of the human body by performing image segmentation on the image frames and obtaining the key point information of each part of the human body according to the passenger information, so that the cloud server performs abnormal behavior early warning. Compared with the existing real-time monitoring of monitoring videos by humans, the application can accurately identify passengers with abnormal behaviors according to the to-be-processed video and perform abnormal behavior early warning.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a behavior detection method, device and storage medium based on artificial intelligence. Background Technology

[0002] In recent years, with the increase in high-rise buildings, people have become increasingly reliant on elevators. Cases of elderly people falling alone in elevators and not being detected or treated promptly are becoming increasingly common. While most elevator cars are equipped with surveillance cameras, traditional video monitoring systems require real-time human monitoring, which is prone to misjudgments and missed detections. Therefore, accurately identifying passengers exhibiting abnormal behavior has become a pressing problem.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a behavior detection method, device, and storage medium based on artificial intelligence, which aims to solve the technical problem that existing technologies cannot accurately identify passengers with abnormal behavior.

[0005] To achieve the above objectives, the present invention provides an artificial intelligence-based behavior detection method applied to a terminal, the artificial intelligence-based behavior detection method comprising:

[0006] Obtain the image frames corresponding to the video to be processed;

[0007] The image frame is segmented to obtain passenger information in the image frame;

[0008] Based on the passenger information, key point information of various parts of the human body is obtained and transmitted to the cloud server so that the cloud server can issue abnormal behavior warnings based on the key point information of various parts of the human body.

[0009] Optionally, the step of obtaining key point information of various parts of the human body based on the passenger information and transmitting the key point information of various parts of the human body to a cloud server, so that the cloud server can issue an abnormal behavior warning based on the key point information of various parts of the human body, specifically includes:

[0010] Obtain passenger identification information and passenger body contour information from the passenger information;

[0011] The posture of the passenger's human body contour information is estimated to obtain key point information of various parts of the human body;

[0012] The passenger identification information and key point information of various parts of the human body are transmitted to the cloud server so that the cloud server can determine abnormal behavior information based on the passenger identification information and key point information of various parts of the human body, and issue abnormal behavior warnings based on the abnormal behavior information.

[0013] Optionally, the step of performing image segmentation on the image frame to obtain passenger information in the image frame specifically includes:

[0014] Add identifiers to passengers in the image frame to obtain passenger identifier information;

[0015] The image frame is segmented to obtain the segmented regions, and the human region of interest is extracted from the segmented regions.

[0016] Edge extraction is performed on the region of interest of the human body using a preset neural network to obtain the passenger's human body contour information in the image frame.

[0017] To achieve the above objectives, the present invention provides an artificial intelligence-based behavior detection method applied to a cloud server, the artificial intelligence-based behavior detection method comprising:

[0018] Obtain passenger characteristic information of the current passengers in the elevator to be monitored;

[0019] Passenger behavior fusion features are determined based on the passenger feature information and the preset spatiotemporal fusion model;

[0020] Behavioral analysis is performed on the fused passenger behavior features to obtain passenger behavior analysis results, and abnormal behavior warnings are issued based on the passenger behavior analysis results.

[0021] Optionally, the passenger characteristic information includes passenger identification information and key point information of various parts of the human body;

[0022] The step of determining passenger behavior fusion features based on the passenger feature information and the preset spatiotemporal fusion model specifically includes:

[0023] Based on the key point information of each part of the human body and the passenger identification information, determine the key point information of each part of the human body of the same passenger in consecutive image frames and the key point information of each part of the human body of different passengers in the same image frame;

[0024] The key point information of various parts of the human body of the same passenger in consecutive image frames and the key point information of various parts of the human body of different passengers in the same image frame are input into a preset spatiotemporal fusion model to obtain passenger behavior fusion features.

[0025] Optionally, the preset spatiotemporal fusion model includes: a preset long short-term memory network, a preset convolutional neural network, and a preset feature fusion neural network;

[0026] The step of inputting the key point information of various human body parts of the same passenger in consecutive image frames and the key point information of various human body parts of different passengers in the same image frame into a preset spatiotemporal fusion model to obtain passenger behavior fusion features specifically includes:

[0027] The key point information of various parts of the human body of the same passenger in consecutive image frames is input into the preset long short-term memory network to obtain the behavioral characteristics of the first passenger;

[0028] The key point information of different human body parts of the different passengers in the same image frame is input into the preset convolutional neural network to obtain the second passenger behavior features;

[0029] The first passenger behavior feature and the second passenger behavior feature are input into the preset feature fusion neural network to obtain passenger behavior fusion features.

[0030] Optionally, the step of performing behavioral analysis on the fused passenger behavior features to obtain passenger behavior analysis results, and issuing abnormal behavior warnings based on the passenger behavior analysis results, specifically includes:

[0031] The passenger behavior fusion features are classified to obtain different types of passenger behavior fusion features;

[0032] The different types of passenger behavior fusion features are matched with the abnormal types of passenger behavior fusion features;

[0033] Based on the matching results, abnormal passenger behavior fusion features are determined, and abnormal behavior warnings are issued based on the abnormal passenger behavior fusion features.

[0034] Optionally, the step of determining abnormal passenger behavior fusion features based on the matching results and issuing abnormal behavior warnings based on the abnormal passenger behavior fusion features specifically includes:

[0035] The abnormal passenger behavior fusion features are determined based on the matching results, and the abnormal behavior is determined based on the abnormal passenger behavior fusion features;

[0036] The number of times the abnormal behavior occurs within a preset time period is obtained, and when the number of occurrences exceeds the preset number, the abnormal passenger corresponding to the abnormal behavior is identified.

[0037] An alert for abnormal behavior is issued based on the aforementioned abnormal passengers.

[0038] Furthermore, to achieve the above objectives, the present invention also proposes an artificial intelligence-based behavior detection device, which includes: a memory, a processor, and an artificial intelligence-based behavior detection program stored in the memory and executable on the processor. The artificial intelligence-based behavior detection program is configured to implement the steps of the artificial intelligence-based behavior detection method described above.

[0039] Furthermore, to achieve the above objectives, the present invention also proposes a storage medium storing an artificial intelligence-based behavior detection program, which, when executed by a processor, implements the steps of the artificial intelligence-based behavior detection method described above.

[0040] This invention acquires image frames corresponding to the video to be processed, then performs image segmentation on the image frames to obtain passenger information within the image frames. Based on the passenger information, it obtains key point information of various parts of the human body and transmits this key point information to a cloud server. The cloud server then uses this key point information to issue abnormal behavior warnings. This invention, by segmenting image frames to obtain passenger information and then obtaining key point information of various parts of the human body based on the passenger information, enables accurate identification of image frames and the acquisition of key point information of various parts of the human body. Compared to existing methods that require real-time human monitoring of surveillance videos, this invention can accurately identify passengers exhibiting abnormal behavior based on the video to be processed and issue abnormal behavior warnings. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the structure of an AI-based behavior detection device in the hardware operating environment involved in the embodiments of the present invention;

[0042] Figure 2 This is a flowchart illustrating the first embodiment of the behavior detection method based on artificial intelligence of the present invention;

[0043] Figure 3 This is a schematic diagram of the structure of the AI-based behavior detection system of the present invention;

[0044] Figure 4 This is a flowchart illustrating the second embodiment of the behavior detection method based on artificial intelligence of the present invention;

[0045] Figure 5 This is a flowchart illustrating the third embodiment of the behavior detection method based on artificial intelligence of the present invention;

[0046] Figure 6 This is a flowchart illustrating the fourth embodiment of the behavior detection method based on artificial intelligence of the present invention.

[0047] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0048] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0049] Reference Figure 1 , Figure 1 This is a schematic diagram of the structure of an AI-based behavior detection device in the hardware operating environment of an embodiment of the present invention.

[0050] like Figure 1 As shown, the AI-based behavior detection device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wireless-Fidelity (Wi-Fi) interface). The memory 1005 may be high-speed random access memory (RAM) or stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0051] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on AI-based behavior detection devices and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0052] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and an artificial intelligence-based behavior detection program.

[0053] exist Figure 1In the AI-based behavior detection device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and memory 1005 in the AI-based behavior detection device of the present invention can be set in the AI-based behavior detection device, and the AI-based behavior detection device calls the AI-based behavior detection program stored in the memory 1005 through the processor 1001 and executes the AI-based behavior detection method provided in the embodiment of the present invention.

[0054] This invention provides an artificial intelligence-based behavior detection method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the behavior detection method based on artificial intelligence of the present invention.

[0055] In this embodiment, the AI-based behavior detection method includes the following steps:

[0056] Step S10: Obtain the image frames corresponding to the video to be processed;

[0057] It should be noted that the execution subject in this embodiment can be a terminal, such as an edge computing device, which can process images, such as image segmentation and image denoising.

[0058] Furthermore, this embodiment also provides an artificial intelligence-based behavior detection system, which can implement the artificial intelligence-based behavior detection method in this embodiment. (Refer to...) Figure 3 , Figure 3 This is a schematic diagram of the structure of the AI-based behavior detection system of the present invention.

[0059] like Figure 3 As shown, the AI-based behavior detection system consists of 1 (cloud server), 2 (cloud server database), 3 and 7 (general-purpose edge computing devices), 4 and 8 (local database), 5 and 6 (cameras), and 9 (property management terminal computer). Cloud server 1 is connected to cloud database 2, edge computing devices 3 and 7 respectively; edge computing device 3 is connected to local database 4, elevator camera 5, and elevator camera 6 respectively; and edge computing device 7 is connected to local database 8 and property management terminal computer 9 respectively.

[0060] Understandably, when a user remotely accesses the cloud server to view the status of camera 5 via the network of the property terminal computer 9, the cloud server 1 broadcasts an access command to the edge computing device 3 and the edge computing device 7. The edge computing device 3 and the edge computing device 7 then match the access command. After receiving the access command, the peripheral interface or network module of the edge computing device 3 collects data through the corresponding camera 5 and camera 6.

[0061] In a practical implementation, the video to be processed in the elevator can be captured by a camera and uploaded frame by frame to a local database for storage via the network. Then, the image frames corresponding to the video to be processed in the local database can be obtained through an edge computing device.

[0062] Step S20: Perform image segmentation on the image frame to obtain passenger information in the image frame;

[0063] It should be noted that passenger information refers to the information corresponding to the passenger in the image frame, which may include passenger identification information, passenger behavior information, etc. This embodiment does not impose specific limitations on this.

[0064] As we can understand it, image segmentation refers to the technique and process of dividing an image frame into several specific regions with unique properties and extracting targets of interest. Specific image segmentation methods can include threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories, etc.

[0065] Step S30: Obtain key point information of various parts of the human body based on the passenger information, and transmit the key point information of various parts of the human body to the cloud server so that the cloud server can issue an abnormal behavior warning based on the key point information of various parts of the human body.

[0066] It should be noted that the key point information of each part of the human body refers to the location information of key points of each part of the passenger's body, which may include hand information, head information, leg information, etc. This embodiment does not impose specific limitations on this.

[0067] In practice, after the cloud server receives key information about various parts of the human body transmitted by the edge computing device, it can accurately identify abnormal behavior and issue warnings for abnormal behavior.

[0068] This embodiment acquires image frames corresponding to the video to be processed, then performs image segmentation on the image frames to obtain passenger information within the image frames. Based on the passenger information, it obtains key point information for various parts of the human body and transmits this key point information to a cloud server. The cloud server then uses this key point information to issue abnormal behavior warnings. This embodiment, by segmenting image frames to obtain passenger information and then obtaining key point information for various parts of the human body based on that information, can accurately identify image frames and obtain key point information for various parts of the human body. This allows the cloud server to issue abnormal behavior warnings based on this key point information. Compared to existing methods that require real-time human monitoring of surveillance videos, this embodiment can accurately identify passengers exhibiting abnormal behavior based on the video to be processed and issue abnormal behavior warnings.

[0069] refer to Figure 4 , Figure 4 This is a flowchart illustrating the second embodiment of the behavior detection method based on artificial intelligence of the present invention.

[0070] Based on the first embodiment described above, in this embodiment, step S30 includes:

[0071] Step S301: Obtain passenger identification information and passenger body contour information from the passenger information;

[0072] It should be noted that passenger identification information refers to information used to identify passengers, allowing identification of specific passengers. Passenger body contour information refers to information about the outline of a passenger's body, which can be used to determine the passenger's actions and behaviors.

[0073] Step S302: Perform pose estimation on the passenger's human body contour information to obtain key point information of various parts of the human body;

[0074] Understandably, pose estimation can determine the orientation of a three-dimensional target object. In this embodiment, pose estimation can be performed on the passenger's human body contour information to obtain key point information of each part of the human body, that is, the three-dimensional coordinate information of the key points of each part. Specifically, the pose estimation of the passenger's human body contour information can be performed through Regional Multi-person Pose Estimation (RMPE), or other methods can be used for pose estimation. This embodiment does not impose any specific restrictions on this.

[0075] Step S303: Transmit the passenger identification information and the key point information of each part of the human body to the cloud server, so that the cloud server can determine abnormal behavior information based on the passenger identification information and the key point information of each part of the human body, and issue an abnormal behavior warning based on the abnormal behavior information.

[0076] In practice, after the cloud server receives key information about various parts of the human body and passenger identification information transmitted by the edge computing device, it can accurately identify abnormal behavior and determine the passenger corresponding to the abnormal behavior, and issue an abnormal behavior warning based on the abnormal behavior and the corresponding passenger.

[0077] Further, in this embodiment, step S20 includes:

[0078] Step S201: Add an identifier to the passenger in the image frame to obtain passenger identifier information;

[0079] Understandably, adding an identifier to a passenger in an image frame provides passenger identification information, and the passenger identification information corresponding to the same passenger is the same.

[0080] Step S202: Perform image segmentation on the image frame to obtain the segmented regions, and extract the human region of interest from the segmented regions;

[0081] It should be noted that the region of interest in human image processing refers to the area that needs to be processed in the image being processed, outlined using shapes such as rectangles, circles, ellipses, and irregular polygons.

[0082] In practice, after image segmentation of the image frame, multiple segmented regions can be obtained, and then the region of interest for the human body can be extracted from the segmented regions.

[0083] Step S203: Extract the edges of the human body region of interest using a preset neural network to obtain the passenger human body contour information in the image frame.

[0084] Understandably, edge extraction refers to the processing of image contours in digital image processing. The edge is defined as the place where the gray-level change rate of the image is the largest, that is, the place where the gray-level value of the image changes most drastically.

[0085] In practice, edge extraction of the human body's region of interest can be performed using a pre-set neural network to obtain the passenger's human body contour information.

[0086] This embodiment acquires passenger identification information and passenger body contour information from passenger information, then performs posture estimation on the passenger body contour information to obtain key point information of various parts of the body. The passenger identification information and key point information of various parts of the body are then transmitted to a cloud server. The cloud server then determines abnormal behavior information based on the passenger identification information and key point information of various parts of the body, and issues abnormal behavior warnings based on this abnormal behavior information. This embodiment, by determining abnormal behavior information based on passenger identification information and key point information of various parts of the body, and issuing abnormal behavior warnings based on this abnormal behavior information, can accurately identify passengers exhibiting abnormal behavior, obtain abnormal behavior information and the corresponding passengers, and thus issue abnormal behavior warnings.

[0087] refer to Figure 5 , Figure 5 This is a flowchart illustrating the third embodiment of the behavior detection method based on artificial intelligence of the present invention.

[0088] In this embodiment, the AI-based behavior detection method includes:

[0089] Step S100: Obtain passenger characteristic information of the current passenger in the elevator to be monitored;

[0090] It should be noted that the execution entity in this embodiment can be a cloud server.

[0091] It is understood that the passenger feature information in this embodiment may include passenger identification information and key point information of various parts of the human body, which can be obtained from the edge detection device.

[0092] Step S200: Determine passenger behavior fusion features based on the passenger feature information and the preset spatiotemporal fusion model;

[0093] It should be noted that the preset spatiotemporal fusion model refers to a pre-set model that fuses the features of passengers. Through this model, the behavioral features of different passengers in consecutive image frames can be obtained, i.e., passenger behavior fusion features.

[0094] Step S300: Perform behavioral analysis on the passenger behavior fusion features to obtain passenger behavior analysis results, and issue abnormal behavior warnings based on the passenger behavior analysis results.

[0095] Understandably, by analyzing the fusion characteristics of passenger behavior, we can obtain passenger behavior analysis results, that is, the current behavior of passengers, and issue abnormal behavior warnings when the current behavior of passengers is abnormal.

[0096] Furthermore, in order to accurately perform abnormal behavior warnings, in this embodiment, step S300 includes: classifying the passenger behavior fusion features to obtain different types of passenger behavior fusion features; matching the different types of passenger behavior fusion features with abnormal types of passenger behavior fusion features; determining abnormal passenger behavior fusion features based on the matching results, and performing abnormal behavior warnings based on the abnormal passenger behavior fusion features.

[0097] It should be noted that the passenger behavior fusion features of abnormal types can be set in advance according to the actual situation. Specifically, they may include passengers prying open elevator doors, passengers jumping inside the elevator, and other abnormal behaviors. This embodiment does not impose specific restrictions on these.

[0098] Understandably, passenger behavior fusion features can be classified using support vector machine (SVM) classifiers, and the number of SVM classifiers is the same as the number of categories of passenger behavior fusion features that need to be classified.

[0099] In the specific implementation, when the matching result is successful, the abnormal passenger behavior fusion feature is the passenger behavior feature of the abnormal type; when the matching result is unsuccessful, it means that no abnormal passenger behavior fusion feature has appeared.

[0100] Furthermore, in this embodiment, the step of determining the abnormal passenger behavior fusion feature based on the matching result and issuing an abnormal behavior warning based on the abnormal passenger behavior fusion feature specifically includes: determining the abnormal passenger behavior fusion feature based on the matching result and determining the abnormal behavior based on the abnormal passenger behavior fusion feature; obtaining the number of occurrences of the abnormal behavior within a preset time period, and when the number of occurrences is greater than the preset number, determining the abnormal passenger corresponding to the abnormal behavior; and issuing an abnormal behavior warning based on the abnormal passenger.

[0101] It is understood that this embodiment can determine abnormal behavior based on the fusion features of abnormal passenger behavior, and then obtain the number of times the abnormal behavior occurs within a preset time period. The preset time period can be set according to the actual situation, such as 10 seconds, 15 seconds, etc. This embodiment does not impose specific restrictions on this.

[0102] In practice, an abnormal passenger is identified only when the number of abnormal behaviors exceeds a preset limit. Specifically, the preset number of abnormal passenger identifications can be 3, 4, etc., and this embodiment does not impose a specific limitation. After identifying the abnormal passenger, the elevator number of that passenger is obtained, and management personnel can take appropriate measures based on the elevator number.

[0103] This embodiment obtains passenger characteristic information of the current passengers in the elevator to be monitored, then determines passenger behavior fusion characteristics based on passenger characteristic information and a preset spatiotemporal fusion model, then performs behavior analysis on the passenger behavior fusion characteristics to obtain passenger behavior analysis results, and provides abnormal behavior warnings based on the passenger behavior analysis results.

[0104] refer to Figure 6 , Figure 6 This is a flowchart illustrating the fourth embodiment of the behavior detection method based on artificial intelligence of the present invention.

[0105] Based on the third embodiment described above, in this embodiment, step S200 includes:

[0106] Step S2001: Determine the key point information of each part of the human body of the same passenger in consecutive image frames and the key point information of each part of the human body of different passengers in the same image frame based on the key point information of each part of the human body and the passenger identification information.

[0107] It is understandable that the key point information of the human body parts of the same passenger in consecutive image frames refers to the key point information of the human body parts of the same passenger in consecutive time, while the key point information of the human body parts of different passengers in the same image frame refers to the key point information of the human body parts of different passengers in the same image frame.

[0108] Step S2002: Input the key point information of the human body parts of the same passenger in consecutive image frames and the key point information of the human body parts of different passengers in the same image frame into a preset spatiotemporal fusion model to obtain passenger behavior fusion features.

[0109] Understandably, the pre-defined spatiotemporal fusion model includes: a pre-defined long short-term memory network, a pre-defined convolutional neural network, and a pre-defined feature fusion neural network. The pre-defined long short-term memory network is a neural network in the time dimension, and the pre-defined convolutional neural network is a neural network in the spatial dimension. The pre-defined spatiotemporal fusion model can be constructed by connecting the pre-defined long short-term memory network and the pre-defined convolutional neural network in parallel, and then connecting them in series with the pre-defined feature fusion neural network.

[0110] Furthermore, in order to accurately determine the passenger behavior fusion features, in this embodiment, step S2002 includes: inputting the key point information of various parts of the human body of the same passenger in consecutive image frames into the preset long short-term memory network to obtain the first passenger behavior feature; inputting the key point information of various parts of the human body of different passengers in the same image frame into the preset convolutional neural network to obtain the second passenger behavior feature; and inputting the first passenger behavior feature and the second passenger behavior feature into the preset feature fusion neural network to obtain the passenger behavior fusion feature.

[0111] It should be noted that the first passenger behavior feature refers to the behavior features of the same passenger in a continuous time frame, while the second passenger behavior feature refers to the behavior features of different passengers in a single time frame.

[0112] Understandably, by inputting the first and second passenger behavior features into a preset feature fusion neural network, passenger behavior fusion features can be obtained, that is, the behavior features of different passengers in continuous time.

[0113] This embodiment determines the key point information of various body parts of the same passenger in consecutive image frames and the key point information of different passengers in the same image frame based on key point information of various body parts and passenger identification information. Then, this information is input into a preset spatiotemporal fusion model to obtain passenger behavior fusion features. This embodiment obtains passenger behavior fusion features, i.e., the behavioral characteristics of different passengers over continuous time, by inputting the key point information of various body parts of the same passenger in consecutive image frames and the key point information of different passengers in the same image frame into a preset spatiotemporal fusion model, thereby enabling accurate identification of passengers exhibiting abnormal behavior.

[0114] Furthermore, this embodiment of the invention also proposes a storage medium storing an AI-based behavior detection program, which, when executed by a processor, implements the steps of the AI-based behavior detection method described above.

[0115] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0116] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0117] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory / random access memory, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0118] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A behavior detection method based on artificial intelligence, applied to a terminal, and characterized in that, The behavior detection method based on artificial intelligence comprises: acquiring image frames corresponding to a to-be-processed video; performing image segmentation on the image frames to obtain passenger information in the image frames, the passenger information comprising passenger identification information and passenger body contour information; performing posture estimation on the passenger body contour information to obtain body part key point information, and transmitting the passenger identification information and the body part key point information to a cloud server, so that the cloud server performs abnormal behavior early warning according to the passenger identification information and the body part key point information, the body part key point information comprising three-dimensional coordinate information of body part key points. 2.The artificial intelligence-based behavior detection method of claim 1, wherein, The step of performing image segmentation on the image frames to obtain passenger information in the image frames specifically comprises: adding an identification to a passenger in the image frames to obtain passenger identification information; performing image segmentation on the image frames to obtain segmented regions, and extracting a human body region of interest from the segmented regions; performing edge extraction on the human body region of interest through a preset neural network to obtain passenger body contour information in the image frames. 3.A behavior detection method based on artificial intelligence, applied to a cloud server, characterized in that, The behavior detection method based on artificial intelligence comprises: acquiring passenger feature information of a current passenger in a to-be-monitored elevator, the passenger feature information comprising passenger identification information and body part key point information; determining passenger behavior fusion features according to the passenger feature information and a preset spatio-temporal fusion model; performing behavior analysis on the passenger behavior fusion features to obtain passenger behavior analysis results, and performing abnormal behavior early warning based on the passenger behavior analysis results; The step of determining passenger behavior fusion features according to the passenger feature information and a preset spatio-temporal fusion model specifically comprises: determining body part key point information of a same passenger in consecutive image frames and body part key point information of different passengers in a same image frame according to the body part key point information and the passenger identification information; inputting the body part key point information of the same passenger in consecutive image frames and the body part key point information of different passengers in a same image frame into a preset spatio-temporal fusion model to obtain passenger behavior fusion features.

4. The artificial intelligence-based behavior detection method of claim 3, wherein, The preset spatio-temporal fusion model comprises a preset long short-term memory network, a preset convolutional neural network, and a preset feature fusion neural network; The step of inputting the body part key point information of the same passenger in consecutive image frames and the body part key point information of different passengers in a same image frame into a preset spatio-temporal fusion model to obtain passenger behavior fusion features specifically comprises: inputting the body part key point information of the same passenger in consecutive image frames into the preset long short-term memory network to obtain first passenger behavior features; inputting the body part key point information of different passengers in a same image frame into the preset convolutional neural network to obtain second passenger behavior features; inputting the first passenger behavior features and the second passenger behavior features into the preset feature fusion neural network to obtain passenger behavior fusion features. 5.The artificial intelligence-based behavior detection method of claim 3, wherein, The step of performing behavior analysis on the passenger behavior fusion features, obtaining a passenger behavior analysis result, and performing abnormal behavior early warning based on the passenger behavior analysis result specifically includes: classifying the passenger behavior fusion features to obtain passenger behavior fusion features of different types; matching the passenger behavior fusion features of different types with abnormal passenger behavior fusion features; determining abnormal passenger behavior fusion features according to the matching result, and performing abnormal behavior early warning based on the abnormal passenger behavior fusion features. 6.The artificial intelligence-based behavior detection method of claim 5, wherein, The step of determining abnormal passenger behavior fusion features according to the matching result, and performing abnormal behavior early warning based on the abnormal passenger behavior fusion features specifically includes: determining abnormal passenger behavior fusion features according to the matching result, and determining abnormal behavior according to the abnormal passenger behavior fusion features; obtaining the number of occurrences of the abnormal behavior within a preset time length, and determining an abnormal passenger corresponding to the abnormal behavior when the number of occurrences is greater than a preset number; performing abnormal behavior early warning based on the abnormal passenger.

7. An artificial intelligence-based behavior detection device, characterized by, The behavior detection device based on artificial intelligence includes a memory, a processor, and a behavior detection program based on artificial intelligence stored on the memory and executable on the processor, and the behavior detection program based on artificial intelligence is configured to implement the behavior detection method based on artificial intelligence in any one of claims 1 to 2, or 3 to 6.

8. A storage medium, characterized by The storage medium stores a behavior detection program based on artificial intelligence, and the behavior detection program based on artificial intelligence is executed by the processor to implement the behavior detection method based on artificial intelligence in any one of claims 1 to 2, or 3 to 6.

Images