Dangerous behavior identification method and device, equipment and computer storage medium

Abstract

The invention belongs to the technical field of security and protection, and particularly relates to a dangerous behavior identification method and device, equipment and a computer storage medium. A to-be-recognized image is obtained by shooting a target place through a camera, and a first probability that a human body lifts a dangerous article is determined according to the to-be-recognized image. Meanwhile, according to the method, first point cloud data are obtained by scanning a target place through a laser radar, and a second probability that the personnel lift the dangerous goods is determined according to the first point cloud data. And finally, comprehensively considering the first probability and the second probability to judge whether the dangerous behavior that the personnel lift the dangerous goods exists or not. According to the method, image recognition and three-dimensional point cloud data are fused, abnormal behaviors in public places can be efficiently recognized, the technical problem that dangerous behaviors are difficult to predict in public place safety management is effectively solved, and therefore more reliable technical support is provided for public place safety management.

Description

Technical Field

[0001] The present application belongs to the field of security technology, and specifically relates to a method, device, equipment and computer storage medium for identifying dangerous behaviors. Background Art

[0002] With the acceleration of urbanization and the improvement of living standards, the number of public places has increased significantly, and their functions and services have become increasingly diversified. From shopping malls, transportation hubs to leisure places, educational institutions and medical institutions, they have become an indispensable and important part of the lives of urban residents.

[0003] Among them, good management of public places is of vital importance. It is not only related to the safety and health of the public, but also a key factor in enhancing the attractiveness and competitiveness of cities and promoting social harmony and progress.

[0004] However, due to the dense and highly mobile population in public places, as well as various potential safety hazards, the safety management of public places faces huge challenges. Summary of the invention

[0005] The present application provides a dangerous behavior identification method, device, equipment and computer storage medium, which are used to solve the problem of unpredictable dangerous behaviors in public place safety management.

[0006] In a first aspect, the present application provides a method for identifying dangerous behaviors, comprising:

[0007] The target location is photographed by a camera to obtain an image to be identified, and the target location is scanned by a laser radar to obtain first point cloud data;

[0008] Extracting a first human body recognition result and a first dangerous object recognition result from the image to be recognized, and determining a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result;

[0009] Determine the color of each point in the first point cloud data according to the image to be recognized, obtain second point cloud data with added color, extract a second human body recognition result and a second dangerous object recognition result from the second point cloud data, and determine a second probability that a person lifts a dangerous object according to the second human body recognition result and the second dangerous object recognition result;

[0010] According to the first probability and the second probability, it is determined whether there is a dangerous behavior of a person lifting a dangerous object.

[0011] Optionally, extracting a first human recognition result and a first dangerous object recognition result from the image to be recognized includes:

[0012] Inputting the image to be recognized into a pre-trained first recognition model to obtain a first human body recognition result output by the first recognition model, wherein the first human body recognition result includes a human body detection frame, a human body detection confidence, and human body skeleton key points;

[0013] The image to be identified is input into a pre-trained second identification model to obtain a first dangerous object identification result output by the second identification model, wherein the dangerous object identification result includes a dangerous object detection frame and a dangerous object detection confidence.

[0014] Optionally, determining a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result includes:

[0015] Eliminate data whose human detection confidence level is lower than a first threshold in the first human recognition result to obtain a first human recognition result after elimination processing;

[0016] Eliminate data whose dangerous goods detection confidence level is lower than a second threshold value in the first dangerous goods identification result to obtain a first dangerous goods identification result after elimination processing;

[0017] For each human body in the first human body recognition result that has been eliminated, an arm rectangular frame is determined with the wrist point and the elbow point in the human skeleton key points as diagonal points, and the center point of the arm rectangular frame is determined, and the arm rectangular frame is expanded while keeping the center point unchanged to obtain an expanded arm rectangular frame;

[0018] Determine the area intersection and union ratio of the expanded arm rectangular frame and each dangerous object detection frame in the first dangerous object identification result that has been eliminated, and if the area intersection and union ratio is greater than zero, determine whether the arm is raised according to the coordinates of the wrist point and the elbow point, and if the arm is raised, determine the degree of arm raising;

[0019] A first probability that the human body lifts the dangerous object is determined according to the human body detection confidence, the dangerous object detection confidence, the area intersection-over-union ratio, and the arm raising degree.

[0020] Optionally, determining the color of each point in the first point cloud data according to the image to be recognized includes:

[0021] Performing coordinate transformation on the first point cloud data to obtain two-dimensional coordinates of each point in the first point cloud data in a pixel coordinate system;

[0022] The color of the pixel point in the to-be-recognized image corresponding to the two-dimensional coordinates of each point in the pixel coordinate system is determined as the color of each point.

[0023] Optionally, extracting a second human body recognition result and a second dangerous goods recognition result from the second point cloud data includes:

[0024] The second point cloud data is input into a pre-trained third recognition model to obtain a target detection result output by the third recognition model, wherein the target detection result includes a three-dimensional detection box of the target, a target category, and a target detection confidence, wherein the target detection result whose target category is human is the second human recognition result, and the target detection result whose target category is dangerous goods is the second dangerous goods recognition result.

[0025] Optionally, determining a second probability that a person lifts a dangerous object according to the second human body recognition result and the second dangerous object recognition result includes:

[0026] Eliminate data whose target detection confidence level in the second human recognition result is lower than a third threshold value, to obtain a second human recognition result after elimination processing;

[0027] Eliminate data whose target detection confidence level is lower than a fourth threshold value in the second dangerous goods identification result to obtain a second dangerous goods identification result after elimination processing;

[0028] For each human body in the second human body recognition result after elimination processing, determine the volume intersection and union ratio of the three-dimensional detection frame of the human body and the three-dimensional detection frame of each dangerous object in the second dangerous object identification result after elimination processing; if the volume intersection and union ratio is greater than zero, determine the second probability that the human body lifts the dangerous object based on the target detection confidence of the human body, the target detection confidence of the dangerous object and the volume intersection and union ratio.

[0029] Optionally, determining whether there is a dangerous behavior of a human body lifting a dangerous object according to the first probability and the second probability includes:

[0030] The first probability and the second probability are fused by using DS evidence theory to obtain a fused probability that a human body lifts a dangerous object. If the fused probability is greater than a probability threshold, it is determined that there is a dangerous behavior of a human body lifting a dangerous object.

[0031] In a second aspect, the present application provides a device for identifying dangerous behaviors, comprising:

[0032] A shooting module is used to shoot a target location through a camera to obtain an image to be identified;

[0033] A scanning module, used for scanning the target location by a laser radar to obtain first point cloud data;

[0034] An extraction module, used to extract a first human recognition result and a first dangerous object recognition result from the image to be recognized;

[0035] A determination module, configured to determine a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result;

[0036] The determination module is further used to determine the color of each point in the first point cloud data according to the image to be recognized, so as to obtain second point cloud data with added color;

[0037] The extraction module is further used to extract a second human body recognition result and a second dangerous goods recognition result from the second point cloud data;

[0038] The determination module is further used to determine a second probability that a person lifts a dangerous object based on the second human recognition result and the second dangerous object recognition result;

[0039] The determination module is further used to determine whether there is a dangerous behavior of a person lifting a dangerous object based on the first probability and the second probability.

[0040] Optionally, the device further comprises: an input module;

[0041] The input module is used to input the image to be recognized into a pre-trained first recognition model to obtain a first human recognition result output by the first recognition model, wherein the first human recognition result includes a human detection frame, a human detection confidence, and human skeleton key points;

[0042] The input module is further used to input the image to be identified into a pre-trained second recognition model to obtain a first dangerous goods identification result output by the second recognition model, wherein the dangerous goods identification result includes a dangerous goods detection frame and a dangerous goods detection confidence level.

[0043] Optionally, the device further comprises: a rejection module;

[0044] The elimination module is used to eliminate data whose human detection confidence level is lower than a first threshold in the first human recognition result, so as to obtain a first human recognition result after elimination processing;

[0045] The elimination module is further used to eliminate data of the first dangerous article identification result whose dangerous article detection confidence level is lower than a second threshold value, so as to obtain a first dangerous article identification result after elimination processing;

[0046] The determination module is further used to determine an arm rectangular frame for each human body in the first human body recognition result that has been eliminated, using the wrist point and the elbow point in the human skeleton key points as diagonal points, and determine the center point of the arm rectangular frame;

[0047] The device further comprises: a processing module;

[0048] The processing module is used to expand the arm rectangular frame while keeping the center point unchanged to obtain an expanded arm rectangular frame;

[0049] The determination module is further used to determine the area intersection and union ratio of the expanded arm rectangular frame and each dangerous object detection frame in the first dangerous object identification result that has been eliminated, and if the area intersection and union ratio is greater than zero, determine whether the arm is raised according to the coordinates of the wrist point and the elbow point, and if the arm is raised, determine the degree of arm raising;

[0050] The determination module is specifically used to determine a first probability that a human body lifts a dangerous object based on the human body detection confidence, the dangerous object detection confidence, the area intersection-over-union ratio and the arm raising degree.

[0051] Optionally, the device further comprises: a conversion module;

[0052] The conversion module is used to perform coordinate conversion on the first point cloud data to obtain the two-dimensional coordinates of each point in the first point cloud data in a pixel coordinate system;

[0053] The determination module is specifically configured to determine the color of a pixel in the to-be-identified image corresponding to the two-dimensional coordinates of each point in the pixel coordinate system as the color of each point.

[0054] Optionally, the input module is also used to input the second point cloud data into a pre-trained third recognition model to obtain a target detection result output by the third recognition model, wherein the target detection result includes a three-dimensional detection box of the target, a target category, and a target detection confidence, wherein the target detection result whose target category is human is the second human recognition result, and the target detection result whose target category is dangerous goods is the second dangerous goods recognition result.

[0055] Optionally, the elimination module is further used to eliminate data whose target detection confidence level in the second human recognition result is lower than a third threshold, so as to obtain a second human recognition result after elimination processing;

[0056] The elimination module is further used to eliminate data in the second dangerous goods identification result whose target detection confidence is lower than a fourth threshold value, so as to obtain a second dangerous goods identification result after elimination processing;

[0057] The determination module is specifically used to determine, for each human body in the second human recognition result that has been eliminated, a volume intersection-and-union ratio of a three-dimensional detection frame of the human body and a three-dimensional detection frame of each dangerous object in the second dangerous object identification result that has been eliminated; if the volume intersection-and-union ratio is greater than zero, then determining a second probability that the human body lifts the dangerous object based on the target detection confidence of the human body, the target detection confidence of the dangerous object and the volume intersection-and-union ratio.

[0058] Optionally, the device further comprises: a fusion module;

[0059] The fusion module is used to fuse the first probability and the second probability using DS evidence theory to obtain a fused probability that a human body lifts a dangerous object;

[0060] The determination module is used to determine that there is a dangerous behavior of a human body lifting a dangerous object when it is determined that the fusion probability is greater than a probability threshold.

[0061] In a third aspect, the present application provides a dangerous behavior identification device, including:

[0062] Memory;

[0063] processor;

[0064] Wherein, the memory stores computer-executable instructions;

[0065] The processor executes the computer-executable instructions stored in the memory to implement the method for identifying dangerous behaviors as described in the first aspect and various possible implementations of the first aspect.

[0066] In a fourth aspect, the present application provides a computer storage medium having computer execution instructions stored thereon, wherein the computer execution instructions are executed by a processor to implement the method for identifying dangerous behaviors as described in the first aspect and various possible implementations of the first aspect.

[0067] In a fifth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the method for identifying dangerous behaviors as described above.

[0068] The dangerous behavior identification method provided by the present application obtains an image to be identified by photographing the target place with a camera, and determines the first probability that a person lifts a dangerous object based on the image to be identified. At the same time, the method also obtains the first point cloud data by scanning the target place with a laser radar, and determines the second probability that a person lifts a dangerous object based on the first point cloud data. Finally, the first probability and the second probability are comprehensively considered to determine whether there is a dangerous behavior of a person lifting a dangerous object. By integrating image recognition and three-dimensional point cloud data, the method can not only efficiently identify abnormal behaviors in public places, but also effectively solve technical problems in public place safety management that are difficult to foresee due to dangerous behaviors, thereby providing more reliable technical support for the safety management of public places. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0070] Figure 1 The process of the method for identifying dangerous behaviors provided for this application Figure 1 ;

[0071] Figure 2 The process of the method for identifying dangerous behaviors provided for this application Figure 2 ;

[0072] Figure 3 The process of the method for identifying dangerous behaviors provided for this application Figure 3 ;

[0073] Figure 4 The process of the method for identifying dangerous behaviors provided for this application Figure 4 ;

[0074] Figure 5 It is a schematic diagram of the structure of the dangerous behavior identification device provided by the present application;

[0075] Figure 6 It is a structural schematic diagram of the dangerous behavior identification device provided by this application.

[0076] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0077] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0078] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in sequences other than those illustrated or described herein, for example.

[0079] In the embodiments of the present application, the words "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0080] With the acceleration of urbanization and the improvement of people's living standards, the development of public places is booming. From shopping malls, airports, train stations to parks, schools, and hospitals, the number of various public places has increased dramatically, and their functions and services have become more and more diverse, becoming an indispensable part of the daily life of urban residents. These places not only carry people's multiple needs such as leisure, entertainment, learning, and medical treatment, but also serve as important platforms for social communication and cultural display, promoting the prosperity of urban culture and the increase of social interaction.

[0081] The importance of public places is self-evident. They are a direct reflection of the efficiency of urban operations and social vitality, and play a vital role in improving the quality of life of residents and promoting economic and social development. Good public place management can not only ensure the safety and health of the public, but also enhance the attractiveness and competitiveness of the city and promote social harmony and progress.

[0082] However, due to the dense population, high mobility, and various potential safety hazards, the safety management of public places faces huge challenges.

[0083] In response to the above problems, the present application provides a method for identifying dangerous behaviors. The target place is photographed by a camera to obtain an image to be identified, and a first probability that a person lifts a dangerous object is determined based on the image to be identified. At the same time, the method also obtains first point cloud data by scanning the target place through a lidar, and determines a second probability that a person lifts a dangerous object based on the first point cloud data. Finally, the first probability and the second probability are comprehensively considered to determine whether there is a dangerous behavior of a person lifting a dangerous object. By fusing image recognition and three-dimensional point cloud data, the method can not only efficiently identify abnormal behaviors in public places, but also effectively solve technical problems that are difficult to foresee in the safety management of public places due to dangerous behaviors, thereby providing more reliable technical support for the safety management of public places.

[0084] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0085] Figure 1 The process of the method for identifying dangerous behavior provided in this embodiment is as follows: Figure 1 .like Figure 1 As shown, the method for identifying dangerous behaviors provided in this embodiment includes:

[0086] S101: photographing a target location with a camera to obtain an image to be identified, and scanning the target location with a laser radar to obtain first point cloud data.

[0087] The target location may be, for example, a park, a shopping mall, or an office building, and this application does not impose any special restrictions on this.

[0088] The image to be identified is used to display a two-dimensional plane view of the target location. For example, if the image to be identified is the entrance of a shopping mall, the image to be identified displays a two-dimensional plane view of the entrance of the shopping mall.

[0089] The first point cloud data is used to display a three-dimensional stereoscopic view of the target location. For example, if the image to be recognized is a shopping mall entrance, the first point cloud data displays a three-dimensional stereoscopic view of the shopping mall entrance.

[0090] The purpose of this step is to obtain a two-dimensional plane view and a three-dimensional stereoscopic view of the target location at the same time.

[0091] It can be understood that the camera can obtain a two-dimensional plane view of the target place by capturing light information of the target place and converting it into a digital image.

[0092] LiDAR measures the distance and position information of each point in the target location by emitting lasers and receiving the reflected signals, thereby constructing a three-dimensional view of the target location.

[0093] Therefore, by using cameras and lidar to shoot and scan the target place, a two-dimensional plane view and a three-dimensional stereoscopic view of the target place can be obtained at the same time.

[0094] S102: extracting a first human body recognition result and a first dangerous object recognition result from the image to be recognized, and determining a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result.

[0095] The purpose of this step is to determine how many people are in the target location and whether there are any dangerous objects based on the image to be identified, and then determine which people are lifting the dangerous objects.

[0096] It is understandable that, since the target place is usually public and anyone can enter, the image to be identified of the target place captured by the camera may contain multiple people and multiple dangerous objects.

[0097] Therefore, in order to more accurately determine which people in the target place have lifted dangerous objects, it is necessary to identify which people are in the target place and whether there are dangerous objects from the image to be identified, and then determine which people in the target place have lifted dangerous objects based on the identification results.

[0098] S103: Determine the color of each point in the first point cloud data according to the image to be recognized, obtain second point cloud data with added color, extract a second human body recognition result and a second dangerous object recognition result from the second point cloud data, and determine a second probability that a person lifts a dangerous object according to the second human body recognition result and the second dangerous object recognition result.

[0099] The data format of each point in the first point cloud data is presented in the form of three-dimensional coordinates. For example, assume that the first point cloud data includes 150 points, of which two points are (1, 2, 3) and (5, 6, 7).

[0100] The data format of each point in the second point cloud data is composed of its position coordinates in three-dimensional space and information representing the color of the point. For example, assuming that a point in the second point cloud data is (1, 2, 3), and the color value corresponding to the point is displayed as red, then based on the above information, it can be determined that the data format of this point is (1, 2, 3) and the color value.

[0101] The purpose of this step is to add a corresponding color to each point in the first point cloud data, thereby determining how many people are in the target location and what dangerous items these people may hold, thereby determining the probability of a person lifting a dangerous item.

[0102] It is understandable that each point in the first point cloud data obtained by the laser radar scanning usually does not contain color information, which may result in the first point cloud data not accurately reflecting the actual presence of people or dangerous objects in the target scene. In contrast, the image to be identified taken by the camera can clearly capture the color information of each person or dangerous object.

[0103] Therefore, in order to make up for the defect that the laser radar cannot determine the color of each point, it is necessary to add a corresponding color to each point in the first point cloud data according to the color information in the image to be identified, so as to obtain the second point cloud data with color.

[0104] S104: Determine whether there is a dangerous behavior of a person lifting a dangerous object based on the first probability and the second probability.

[0105] The purpose of this step is to determine whether anyone in the target location has lifted any dangerous items.

[0106] It can be understood that the first probability is obtained based on the recognition results of the human body and the dangerous goods in the image to be recognized, and it reflects the possibility that the person visible in the image is holding the dangerous goods.

[0107] The second probability is based on the recognition results of human bodies and dangerous goods in the second point cloud data with added colors. It uses color information to enhance the accuracy of identifying people and dangerous goods.

[0108] Therefore, by comprehensively considering the first probability and the second probability, it means that the image recognition and three-dimensional point cloud data of the target place can be combined for consideration, which helps to more accurately determine whether someone has lifted a dangerous object in the target place.

[0109] The method for identifying dangerous behaviors provided in this embodiment first obtains an image to be identified by photographing the target place with a camera, and extracts the first human recognition result and the first dangerous object recognition result from the image to be identified, thereby determining the first probability that the human body lifts the dangerous object. At the same time, the method also obtains the first point cloud data by scanning the target place with a laser radar, and adds color information to these data points to obtain the second point cloud data. Subsequently, the second human recognition result and the second dangerous object recognition result are extracted from the second point cloud data to determine the second probability that the person lifts the dangerous object. Finally, a comprehensive judgment is made based on the first probability and the second probability whether there is a dangerous behavior of a person lifting a dangerous object. This method combines the advantages of image recognition and three-dimensional point cloud data, which not only improves the accuracy of recognition, but also solves the technical problems that are difficult to foresee in the safety management of public places due to dangerous behaviors, thereby providing more reliable and powerful technical support for the safety management of public places, and effectively improving the level of safety management.

[0110] Figure 2 The process of the method for identifying dangerous behavior provided in this embodiment is as follows: Figure 2 .like Figure 2 This embodiment is Figure 1 Based on the embodiment, the implementation process of the dangerous behavior identification method is described in detail. The dangerous behavior identification method provided in this embodiment includes:

[0111] S201: photographing a target location with a camera to obtain an image to be identified, and scanning the target location with a laser radar to obtain first point cloud data.

[0112] The explanation of step S201 is referenced from the explanation of the above embodiment and will not be repeated here.

[0113] S202: Input the image to be recognized into a pre-trained first recognition model to obtain a first human body recognition result output by the first recognition model, wherein the first human body recognition result includes a human body detection frame, a human body detection confidence, and human body skeleton key points.

[0114] The first recognition model may be, for example, a YOLOv8-pose model. The first recognition model is used to recognize a person in the image to be recognized.

[0115] The human body detection frame is used to represent the contour shape corresponding to the human body when the object in the image to be identified is a human body.

[0116] The human detection confidence is used to characterize the possibility of a person existing in the image to be identified. The higher the human detection confidence, the higher the possibility of a person existing in the image to be identified. Conversely, the lower the human detection confidence, the lower the possibility of a person existing in the image to be identified.

[0117] The human skeleton key points include: multiple human skeleton key points displayed in two-dimensional coordinates, namely (x1, y1) corresponding to "nose", (x2, y2) corresponding to "left eye", (x3, y3) corresponding to "right eye", (x4, y4) corresponding to "left ear", (x5, y5) corresponding to "right ear", (x6, y6) corresponding to "left shoulder", (x7, y7) corresponding to "right shoulder", (x8, y8) corresponding to "left elbow", (x9, y9) corresponding to "right elbow", (x10, y10) corresponding to "left wrist", (x11, y11) corresponding to "right wrist", (x12, y12) corresponding to "left hip", (x13, y13) corresponding to "right hip", (x14, y14) corresponding to "left knee", (x15, y15) corresponding to "right knee", (x16, y16) corresponding to "left ankle" and (x17, y17) corresponding to "right ankle".

[0118] The purpose of inputting the image to be identified into the pre-trained first recognition model in this step is to utilize the internal algorithm of the first recognition model to perform recognition processing on the image to be identified and obtain relevant information about people that may exist in the target place.

[0119] It is understandable that the image to be identified is obtained by shooting with a camera installed in the target place. Therefore, the image to be identified can truly reflect the actual situation in the target place, including the possible people and their movements.

[0120] The first recognition model is a pre-trained deep learning model that is specifically used for human posture estimation and recognition. Through its internal complex algorithms, the model can accurately analyze and process the input image, thereby accurately identifying the key bone points, human detection frames, and human detection confidence of the human body in the image.

[0121] Therefore, by taking the image to be identified as an input factor and inputting it into the pre-trained first recognition model, the first recognition model can perform human body recognition and posture recognition on the image to be identified, thereby obtaining relevant information of all people in the target place.

[0122] S203: Input the image to be recognized into a pre-trained second recognition model to obtain a first dangerous object recognition result output by the second recognition model, where the dangerous object recognition result includes a dangerous object detection frame and a dangerous object detection confidence.

[0123] The second recognition model may be, for example, a YOLOv5 model. The second recognition model is used to identify dangerous items in the target location.

[0124] The dangerous object detection frame is used to represent the contour shape corresponding to a knife when the dangerous object present in the target location is a knife, or the contour shape corresponding to a stick when the dangerous object present in the target location is a stick.

[0125] The dangerous goods detection frame includes, but is not limited to: the height of the dangerous goods detection frame, the width of the dangerous goods detection frame, and the coordinates of the center point of the dangerous goods detection frame displayed in two-dimensional coordinates.

[0126] Dangerous goods detection confidence is used to characterize the possibility of the existence of knives or sticks in the image to be identified. The higher the dangerous goods detection confidence, the higher the possibility of the existence of knives or sticks in the image to be identified. Conversely, the lower the dangerous goods detection confidence, the lower the possibility of the existence of knives or sticks in the image to be identified.

[0127] The purpose of inputting the image to be identified into the pre-trained second recognition model in this step is to utilize the internal algorithm of the second recognition model to process the image to be identified and obtain relevant information about dangerous goods that may exist in the target location.

[0128] It can be understood that the image to be identified is obtained by taking pictures with a camera installed in the target place, and the second identification model is a pre-trained model that can accurately identify dangerous objects in the target place.

[0129] Therefore, by taking the image to be identified as an input factor and inputting it into the pre-trained second recognition model, the second recognition model can identify the dangerous goods in the image to be identified, thereby obtaining relevant information of all dangerous goods in the target place.

[0130] S204: Determine a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result.

[0131] The explanation of step S204 refers to the explanation of the above embodiment, which will not be repeated here.

[0132] S205: performing coordinate transformation on the first point cloud data to obtain the two-dimensional coordinates of each point in the first point cloud data in a pixel coordinate system.

[0133] The purpose of performing coordinate transformation on the first point cloud data in this step is to map the point cloud data in the three-dimensional space onto the two-dimensional image plane.

[0134] This step may be, for example, to transform the coordinates of the first point cloud data using a camera calibration algorithm. For example, if the first point cloud data includes (13, 24, 68), the two-dimensional coordinates after transformation are (21, 45).

[0135] It is understandable that since the first point cloud data is obtained by laser radar scanning, and the laser radar can usually only obtain the position information of each point during the scanning process, and does not contain color information, this means that the first point cloud data may not fully and accurately reflect the objects with specific colors such as people or dangerous goods that actually exist in the target scene.

[0136] Therefore, in order to make up for the shortcomings of laser radar in acquiring color information, each point in the first point cloud data needs to be presented in pixel coordinates so that each coordinate point can be filled in according to the image to be identified captured by the camera.

[0137] S206: Determine the color of the pixel point in the to-be-recognized image corresponding to the two-dimensional coordinates of each point in the pixel coordinate system as the color of each point.

[0138] The purpose of this step is to assign color information to each point in the first point cloud data, so that the first point cloud data not only contains position information but also reflects the color characteristics of each point in the target scene.

[0139] It can be understood that, since the image to be identified is obtained by photographing the target place with a camera, it can clearly reflect the color information of the objects in the target place.

[0140] Therefore, after each point in the first point cloud data is presented in the form of two-dimensional coordinates, the corresponding pixel points can be found in the image to be identified based on these two-dimensional coordinates, and the color information of these pixel points can be obtained, thereby completing the assignment of corresponding colors to each point cloud data point.

[0141] S207: Input the second point cloud data into a pre-trained third recognition model to obtain a target detection result output by the third recognition model, wherein the target detection result includes a three-dimensional detection box of the target, a target category, and a target detection confidence level, wherein the target detection result whose target category is human is the second human recognition result, and the target detection result whose target category is dangerous goods is the second dangerous goods recognition result.

[0142] The third recognition model may be, for example, a PointPillars model. The third recognition model is used to perform recognition processing on the second point cloud data having colors.

[0143] When the target category is human, the target detection confidence is used to characterize the possibility of the existence of human in the target location. The higher the target detection confidence, the greater the possibility of the existence of human in the target location. Conversely, the lower the target detection confidence, the lower the possibility of the existence of human in the target location.

[0144] When the target category is dangerous goods, the target detection confidence is used to determine the possibility of dangerous goods in the target location. If the target detection confidence is higher, the possibility of dangerous goods in the target location is greater. Conversely, if the target detection confidence is lower, the possibility of dangerous goods in the target location is smaller.

[0145] The purpose of this step of inputting the second point cloud data into the pre-trained third recognition model is to use the third recognition model to perform deep learning and analysis on the point cloud data, so as to accurately detect the position, category and detection credibility of the target object (such as human body, dangerous goods, etc.) in three-dimensional space.

[0146] It can be understood that since each point in the second point cloud data only represents the position and color information of each point in space, it does not directly indicate the target object.

[0147] The pre-trained third recognition model can perform recognition and processing on the second point cloud data, and can not only recognize target detection results whose target category is human, but also recognize target detection results whose target category is dangerous goods.

[0148] Therefore, by inputting the second point cloud data into the pre-trained third recognition model, it is possible to accurately identify targets belonging to the human category and targets belonging to the dangerous goods category. And for each identified target, the model will also output its 3D detection box, target category (human or dangerous goods) and target detection confidence.

[0149] S208: Determine a second probability that the person lifts the dangerous object according to the second human body recognition result and the second dangerous object recognition result.

[0150] The explanation of step S208 refers to the explanation of the above embodiment, which will not be repeated here.

[0151] S209: The DS evidence theory is used to fuse the first probability and the second probability to obtain a fused probability that a human body lifts a dangerous object. If the fused probability is greater than a probability threshold, it is determined that there is a dangerous behavior of a human body lifting a dangerous object.

[0152] Among them, DS evidence theory (Dempster-Shafer evidence theory) is a mathematical tool for dealing with uncertainty and incomplete information. The core of this theory is to provide a method for reasoning and decision-making under imprecise or uncertain information. By defining concepts such as recognition framework and basic probability distribution, evidence from multiple different sources is integrated to obtain more reliable and comprehensive reasoning results. DS evidence theory can directly express "uncertain" and "unknown" information and retain this information during the reasoning process without knowing the prior probability, and has the advantage of intuitive and easy-to-understand prior data.

[0153] The probability threshold may be expressed, for example, by a numerical value, such as 0.5, or by a percentage, such as 50%. This application does not impose any special restrictions on this.

[0154] The purpose of this step is to fuse the first probability and the second probability by adopting the DS evidence theory to obtain a more accurate and comprehensive fusion probability, which is used to evaluate the possibility of a human body lifting a dangerous object.

[0155] It can be understood that DS evidence theory can fuse the first probability and the second probability, and through its unique synthesis rules and trust functions, it can effectively integrate information from different sources and reduce uncertainty.

[0156] Therefore, through the fusion processing of DS evidence theory, a more accurate and reliable fusion probability can be obtained, which can be used to accurately assess the risk of human lifting dangerous objects.

[0157] For example, suppose the probability threshold is 50% to determine whether a person has dangerous behavior of lifting dangerous objects. It is known that the first probability of a person is 0.6 and the second probability is 0.4. Based on the above information, the DS evidence theory can be used to fuse the first probability and the second probability, and the fused probability of a person lifting dangerous objects can be obtained to be 31.6%. Finally, it is determined that the person does not have dangerous behavior of lifting dangerous objects.

[0158] The method for identifying dangerous behaviors provided in this embodiment first captures an image of a target place through a camera, and uses the first and second recognition models trained in advance to respectively identify the human body and dangerous objects, and outputs recognition results including a human body detection frame, human body detection confidence, human body skeleton key points, and dangerous object detection frame and dangerous object detection confidence. Based on these results, the first probability that the human body lifts the dangerous object is calculated. At the same time, the first point cloud data acquired by the laser radar is transformed into coordinates, color information is assigned to each point, and the third recognition model trained in advance is input to obtain a target detection result including a target three-dimensional detection frame, a target category and a target detection confidence, thereby determining the second human body recognition result and the second dangerous object recognition result, and calculating the second probability accordingly. Finally, the DS evidence theory is used to fuse the two probabilities to obtain the fusion probability of the human body lifting the dangerous object. If the fusion probability exceeds the preset probability threshold, it is determined that there is a dangerous behavior of the human body lifting the dangerous object. By combining image recognition and point cloud data processing technology, this method not only significantly improves the accuracy of identifying dangerous behaviors in public places, but also effectively solves the technical problem of unpredictable dangerous behaviors that have long existed in the safety management of public places, thereby providing a more solid and reliable technical support for the safety management of personnel in public places.

[0159] Figure 3 The process of the method for identifying dangerous behavior provided in this embodiment is as follows: Figure 3 .like Figure 3 This embodiment is Figure 2 Based on the embodiment, the implementation process of determining the first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result is described in detail. The dangerous behavior recognition method provided in this embodiment includes:

[0160] S301: Eliminate data in the first human body recognition result whose human body detection confidence level is lower than a first threshold, and obtain a first human body recognition result after elimination processing.

[0161] The first threshold value may be expressed as a numerical value, such as 0.6, or as a percentage, such as 60%. This application does not impose any special restrictions on this.

[0162] The purpose of this step is to filter out data with low reliability from the first human body recognition result.

[0163] It is understandable that since the first recognition model is used to recognize people in the images to be recognized, and these images to be recognized are usually taken by cameras at the target location, the images taken by the cameras may contain not only people but also other irrelevant objects, such as cars, trees, etc.

[0164] During the recognition process of the first recognition model, the first recognition model may generate some misjudgments or recognition results with high uncertainty, and the human body detection confidence of these results may be low.

[0165] Therefore, in order to improve the accuracy of human body recognition, it is necessary to perform judgment verification, that is, to eliminate the data with human body detection confidence lower than the first threshold in the first human body recognition result, so as to obtain a more accurate and reliable human body recognition result.

[0166] For example, assume that the first threshold is 0.4. It is known that the first human recognition result includes: human detection confidence 0.14, human detection confidence 0.25, human detection confidence 0.56, and human detection confidence 0.45. Based on the above information, human detection confidence 0.14 and human detection confidence 0.25 can be eliminated to obtain the first human recognition result including human detection confidence 0.56 and human detection confidence 0.45.

[0167] S302: Eliminate data of the first dangerous article identification result whose dangerous article detection confidence level is lower than a second threshold value, and obtain a first dangerous article identification result after elimination processing.

[0168] The second threshold value may be expressed as a numerical value, such as 0.5, or as a percentage, such as 50%. This application does not impose any special restrictions on this.

[0169] The purpose of this step is to filter out data with low reliability from the first dangerous goods identification result.

[0170] It is understandable that since the second recognition model is used to recognize dangerous objects in the images to be recognized, and these images to be recognized are usually taken by cameras installed in the target place, the images taken by the cameras not only contain the existence of dangerous objects, but also may contain other irrelevant objects, such as trees.

[0171] During the recognition process of the second recognition model, the second recognition model may generate some misjudgments or recognition results with high uncertainty, and the dangerous goods detection confidence corresponding to these results is usually low.

[0172] Therefore, in order to improve the accuracy of dangerous goods identification, it is necessary to perform judgment verification, that is, to eliminate those data in the first dangerous goods identification result whose dangerous goods detection confidence is lower than the second threshold, so as to obtain a more accurate and reliable dangerous goods identification result.

[0173] For example, assuming that the second threshold is 50%. It is known that the first dangerous goods identification result includes: dangerous goods detection confidence 0.23, dangerous goods detection confidence 0.25, dangerous goods detection confidence 0.6, dangerous goods detection confidence 0.8. Based on the above information, dangerous goods detection confidence 0.23 and dangerous goods detection confidence 0.25 can be eliminated to obtain dangerous goods identification results including dangerous goods detection confidence 0.6 and dangerous goods detection confidence 0.8.

[0174] S303: For each human body in the first human body recognition result that has been eliminated, an arm rectangular frame is determined with the wrist point and the elbow point in the human skeleton key points as diagonal points, and the center point of the arm rectangular frame is determined. The arm rectangular frame is expanded while keeping the center point unchanged to obtain an expanded arm rectangular frame.

[0175] The arm rectangular frame includes: a left arm rectangular frame and a right arm rectangular frame. When the left wrist point and the left elbow point are diagonal points, the left arm rectangular frame can be formed; when the right wrist point and the right elbow point are diagonal points, the right arm rectangular frame can be formed.

[0176] The center points of the arm rectangle box include: the center point of the left arm rectangle box and the center point of the right arm rectangle box. For example, assuming that the left wrist point is (x10, y10), the left elbow point is (x8, y8), the right wrist point is (x11, y11), and the right elbow point is (x9, y9). Based on the above information, it can be determined that the center point of the left arm rectangle box is ((x10+x8) / 2, (y10+y8) / 2), and the center point of the right arm rectangle box is ((x11+x9) / 2, (y11+y9) / 2).

[0177] The purpose of this step is to extract the activity posture area of ​​each human arm from the human recognition results that have been eliminated. The purpose of this step of expansion is to increase the coverage of the arm rectangle frame so as to better determine whether there is a person lifting a dangerous object in the first human recognition result.

[0178] It can be understood that the wrist point and the elbow point can not only reflect the extension state of the arm, but also the bending state of the arm. Therefore, by using the wrist point and elbow point in each key point of the human skeleton as diagonal points to determine a rectangular frame, the approximate activity area of ​​the human arm can be obtained.

[0179] However, considering that the arm may carry objects or make subtle movements, in order to capture this information more comprehensively, the determined arm rectangular frame needs to be expanded. This not only makes it easier for the expanded rectangular frame to include the arm movements or objects carried, but also improves the recognition accuracy of arm movements.

[0180] S304: Determine the area intersection and union ratio of the expanded arm rectangular frame and each dangerous object detection frame in the first dangerous object identification result that has been eliminated. If the area intersection and union ratio is greater than zero, determine whether the arm is raised based on the coordinates of the wrist point and the elbow point, and if the arm is raised, determine the degree of arm raising.

[0181] The expanded arm rectangular frame includes: an expanded left arm rectangular frame and an expanded right arm rectangular frame.

[0182] The area intersection and union ratio is used to indicate whether there is an intersection area between the expanded arm rectangular frame and the dangerous goods detection frame. If the area intersection and union ratio is greater than 0, it indicates that there is an intersection area between the expanded arm rectangular frame and the dangerous goods detection frame. Conversely, if the area intersection and union ratio is not greater than 0, it indicates that there is no intersection area between the expanded arm rectangular frame and the dangerous goods detection frame.

[0183] The purpose of determining the area intersection-union ratio in this step is to determine which people are holding dangerous objects from the first human recognition results that have been eliminated.

[0184] Understandably, the expanded arm rectangle itself cannot directly reflect which people are holding dangerous objects. The dangerous object detection frame accurately identifies the range of dangerous objects.

[0185] Therefore, by calculating the intersection-and-union ratio of the expanded arm rectangular frame and each dangerous object detection frame in the first dangerous object identification result that has been eliminated, it is possible to effectively determine which people are holding dangerous objects.

[0186] Optionally, this application provides a possible implementation method, including:

[0187] In the first step, for each human body in the first human body recognition result that has been eliminated, according to the expanded arm rectangular frame of each human body, the upper left corner coordinates and the lower right corner coordinates of the expanded arm rectangular frame of each human body are determined.

[0188] Among them, if the expanded arm rectangular frame is an expanded left arm rectangular frame, the upper left corner coordinates and the lower right corner coordinates are determined according to the expanded left arm rectangular frame; if the expanded arm rectangular frame is an expanded right arm rectangular frame, the upper left corner coordinates and the lower right corner coordinates are determined according to the expanded right arm rectangular frame.

[0189] For example, it is known that a person's left wrist point is (2, 3) and the left elbow point is (2, 7). Based on the above information, we can first determine that the upper left corner coordinates of the left arm rectangular box formed with the left wrist point and the left elbow point as diagonal points are (2, 7) and the lower right corner coordinates are (2, 3). Then, we can determine that the upper left corner coordinates of the expanded left arm rectangular box are (1, 8) and the lower right corner coordinates are (3, 2).

[0190] In the second step, according to each dangerous object detection frame in the first dangerous object identification result that has been eliminated, the upper left corner coordinates and the lower right corner coordinates of each dangerous object detection frame are determined.

[0191] For example, if the height of a dangerous object detection frame is 10 cm, the width is 6 cm, and the center point coordinates are (5, 7), then based on the above information, it can be determined that the upper left corner coordinates of the dangerous object detection frame are (2, 12) and the upper right corner coordinates are (8, 2).

[0192] In the third step, for the upper left corner coordinates and lower right corner coordinates of any expanded arm rectangular frame of the human body, the upper left corner coordinates and lower right corner coordinates of each dangerous object detection frame are traversed respectively to obtain the upper left corner coordinates and lower right corner coordinates of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame.

[0193] For example, it is known that the coordinates of the upper left corner of a person's expanded left arm rectangle are (1, 8) and the coordinates of the lower right corner are (3, 2), and the coordinates of the upper left corner of a dangerous object detection frame are (2, 12) and the coordinates of the upper right corner are (8, 2). Based on the above information, it can be determined that the coordinates of the upper left corner of the intersection area between the expanded arm rectangle and the dangerous object detection frame are (2, 12) and the coordinates of the lower right corner are (3, 2).

[0194] In the fourth step, the height and width of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame are determined according to the upper left corner coordinates and the lower right corner coordinates of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame.

[0195] For example, it is known that the upper left corner coordinates of the intersection area between the expanded arm rectangle frame and a certain dangerous object detection frame are (2, 12) and the lower right corner coordinates are (3, 2). Based on the above information, it can be determined that the height of the intersection area between the expanded arm rectangle frame and the dangerous object detection frame is 10 cm and the width is 1 cm.

[0196] The fifth step is to determine whether the height of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame is greater than 0, and to determine whether the width of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame is greater than 0.

[0197] The purpose of this step is to determine which dangerous goods detection frame has an overlapping portion with the expanded arm rectangular frame on the two-dimensional plane.

[0198] It can be understood that the overlapping part usually means that the human arm and the dangerous object are in contact or close to each other in space, which may indicate that the dangerous object is held or touched by the arm. Therefore, in order to determine which dangerous object detection frame the expanded arm rectangular frame has an intersection area with, it is necessary to determine whether the height of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame is greater than 0, and whether the width of the intersection area between the expanded arm rectangular frame and each dangerous object detection frame is greater than 0.

[0199] For example, it is known that the height of the intersection area between the expanded arm rectangular frame and a certain dangerous goods detection frame is 10 cm and the width is 1 cm. Based on the above information, it can be determined that the height of the intersection area between the expanded arm rectangular frame and a certain dangerous goods detection frame is greater than 0, and it can be determined that the width of the intersection area between the expanded arm rectangular frame and a certain dangerous goods detection frame is greater than 0.

[0200] In the sixth step, when the height of the intersection area between the expanded arm rectangular frame and any dangerous object detection frame is greater than 0, and the width of the intersection area between the expanded arm rectangular frame and any dangerous object detection frame is greater than 0, the dangerous object detection frame is matched with the expanded arm rectangular frame.

[0201] It can be understood that when the expanded arm rectangle frame intersects with any dangerous object detection frame on a two-dimensional plane, it means that the two rectangle frames partially overlap each other. Therefore, it can be reasonably inferred that there is a correlation between the dangerous object detection frame and the rectangular frame of the human arm in terms of spatial position. Based on this correlation, it can be determined that the dangerous object is held by the person.

[0202] In the seventh step, according to the expanded arm rectangular frame and the dangerous goods detection frame corresponding to the expanded arm rectangular frame, the area intersection-combination ratio between the expanded arm rectangular frame and the dangerous goods detection frame is determined.

[0203] For example, it is known that the coordinates of the upper left corner of a person's expanded left arm rectangular frame are (1, 8), and the coordinates of the lower right corner are (3, 2), the coordinates of the upper left corner of a certain dangerous goods detection frame are (2, 12), and the coordinates of the upper right corner are (8, 2), and the coordinates of the upper left corner of the intersection area between the expanded arm rectangular frame and the dangerous goods detection frame are (2, 12), and the coordinates of the lower right corner are (3, 2). Based on the above information, we can first determine that the area of ​​the expanded arm rectangular frame is 12, the area of ​​the dangerous goods detection frame is 60, and the area of ​​the intersection area between the expanded arm rectangular frame and the dangerous goods detection frame is 10. Subsequently, the area between the expanded arm rectangular frame and the dangerous goods detection frame excluding the intersection area is determined to be 62. Finally, the area intersection-over-combination ratio between the expanded arm rectangular frame and the dangerous goods detection frame is determined to be 0.16.

[0204] In the eighth step, when the area intersection and union ratio between the expanded arm rectangular frame and the dangerous goods detection frame is greater than 0, determine whether the arm is raised according to the coordinates of the wrist point and the elbow point, and if the arm is raised, determine the degree of arm raising.

[0205] It is understandable that when the human body is in a natural standing or sitting position, the relative position relationship between the wrist point and the elbow point usually presents a relatively fixed pattern, that is, the wrist point is located below the elbow point and relatively close to the body. However, when the arm is raised, this relative position relationship will change, and the wrist point may rise above the elbow point or at least be on the same horizontal line as the elbow point, and the distance between the two will increase accordingly. Therefore, by monitoring the coordinate changes of the wrist point and the elbow point, it is possible to effectively determine whether the arm is in a raised state.

[0206] Optionally, the present application provides a method for determining the degree of arm raising, including:

[0207] Assume the wrist point coordinate is P w (x w ,y w ), the coordinates of the elbow point are P e (x e ,y e ).

[0208] Among them, R a Indicates the degree of arm raising, x e represents the horizontal coordinate of the elbow point, x w Indicates the horizontal coordinate of the wrist point, y e The ordinate of the elbow point, y w Indicates the ordinate of the wrist point.

[0209] The degree of arm raising is represented by a numerical value. The larger the numerical value, the greater the degree of arm raising. 0 means that the arm is not raised, and 1 means that the arm is raised and straight.

[0210] For example, suppose the left wrist point of a person is (2, 3) and the left elbow point is (2, 7). Based on the above information, it can be determined that the degree of arm lifting of the person is 1.

[0211] S305: Determine a first probability that a human body lifts a dangerous object according to the human body detection confidence, the dangerous object detection confidence, the area intersection-over-union ratio, and the arm lifting degree.

[0212] The first probability is used to characterize the degree of possibility that a human body lifts a dangerous object. The greater the first probability, the greater the possibility that a human body lifts a dangerous object. Conversely, the smaller the first probability, the smaller the possibility that a human body lifts a dangerous object.

[0213] The purpose of this step is to determine the probability of each person lifting a dangerous object from the first human recognition results that have been eliminated.

[0214] Optionally, the present application provides a specific implementation method for determining the first probability that a human body lifts a dangerous object based on the human body detection confidence, the dangerous object detection confidence, the area intersection and union ratio, and the degree of arm lifting: X=A*B*C*D

[0215] Among them, the first probability that a person lifts a dangerous object is represented by X, the confidence of human detection is represented by A, the confidence of dangerous object detection is represented by B, the area intersection ratio is represented by C, and the degree of arm lifting is represented by D.

[0216] For example, if the human body detection confidence of a person is 0.8, the dangerous object detection confidence is 0.9, the area intersection ratio is 0.4, and the arm raising degree is 0.5, then based on the above information, it can be determined that the first probability that the person has raised a dangerous object is 0.144.

[0217] The method for identifying dangerous behaviors provided in this embodiment first removes data with a human detection confidence lower than a first threshold value in the first human recognition result to obtain an optimized first human recognition result. Next, remove data with a dangerous goods detection confidence lower than a second threshold value in the first dangerous goods recognition result to obtain an optimized first dangerous goods recognition result. Then, for each human body in the optimized human recognition result, an arm rectangular frame is determined with the wrist point and the elbow point as diagonal points, and the frame is expanded to more accurately capture the arm area. Subsequently, the area intersection and union ratio of the expanded arm rectangular frame and each dangerous goods detection frame is calculated. If the area intersection and union ratio is greater than zero, it is determined whether the arm is raised according to the coordinates of the wrist point and the elbow point, and the degree of arm raising is further determined. Finally, the first probability of a human body raising a dangerous goods is calculated by comprehensively considering the human body detection confidence, the dangerous goods detection confidence, the area intersection and union ratio, and the degree of arm raising. This method effectively improves the accuracy and reliability of human body raising dangerous goods recognition by processing the recognition results of human bodies and dangerous goods, and introducing the expansion processing of the arm rectangular frame and the judgment of the degree of arm raising, and provides strong technical support for safety monitoring and risk assessment.

[0218] Figure 4 The process of the method for identifying dangerous behavior provided in this embodiment is as follows: Figure 4 .like Figure 4 This embodiment is Figure 2 Based on the embodiment, the implementation process of determining the second probability that a person lifts a dangerous object according to the second human recognition result and the second dangerous object recognition result is described in detail. The dangerous behavior recognition method provided in this embodiment includes:

[0219] S401: Eliminate data in the second human body recognition result whose target detection confidence is lower than a third threshold value, and obtain a second human body recognition result after elimination processing.

[0220] The third threshold value may be expressed as a numerical value, such as 0.5, or as a percentage, such as 50%. This application does not impose any special restrictions on this.

[0221] The purpose of this step is to filter out data with relatively low reliability from the second human recognition result.

[0222] It is understandable that since the third recognition model is used to recognize and process the second point cloud data with color, and these second point cloud data are usually scanned and photographed by the laser radar installed at the target location, the information scanned by the laser radar includes not only the existence of people, but also other irrelevant objects, such as trees, cars, fixed buildings, etc.

[0223] During the recognition process of the third recognition model, the third recognition model may produce some misjudgments or recognition results with high uncertainty, and the target detection confidence corresponding to these results is usually low.

[0224] Therefore, in order to improve the accuracy of human recognition, judgment verification is required, that is, those data in the second human recognition result whose target detection confidence is lower than the third threshold are eliminated, so as to obtain a more accurate and reliable second human recognition result.

[0225] For example, assuming that the third threshold is 0.5. It is known that the second human recognition result includes: target detection confidence 0.3, target detection confidence 0.25, target detection confidence 0.7, and target detection confidence 0.9. Based on the above information, target detection confidence 0.3 and target detection confidence 0.25 can be eliminated to obtain the second human recognition result including target detection confidence 0.7 and target detection confidence 0.9.

[0226] S402: Eliminate data in the second dangerous object identification result whose target detection confidence level is lower than a fourth threshold value, and obtain a second dangerous object identification result after elimination processing.

[0227] The fourth threshold value may be expressed as a numerical value, such as 0.7, or as a percentage, such as 70%. This application does not impose any special restrictions on this.

[0228] The purpose of this step is to filter out data with relatively low reliability from the second dangerous goods identification result.

[0229] It is understandable that since the third recognition model is used to recognize and process the second point cloud data with color, and these second point cloud data are usually scanned and photographed by the laser radar installed at the target location, the information scanned by the laser radar not only includes the existence of dangerous objects, but also includes other irrelevant objects, such as trees, cars, etc.

[0230] During the recognition process of the third recognition model, the third recognition model may produce some misjudgments or recognition results with high uncertainty, and the target detection confidence corresponding to these results is usually low.

[0231] Therefore, in order to improve the accuracy of dangerous goods identification, it is necessary to perform judgment verification, that is, to eliminate those data in the second dangerous goods identification result whose target detection confidence is lower than the fourth threshold, so as to obtain a more accurate and reliable second dangerous goods identification result.

[0232] For example, assuming that the fourth threshold is 60%. It is known that the second dangerous goods identification result includes: target detection confidence 0.61, target detection confidence 0.4, target detection confidence 0.45, target detection confidence 0.8. Based on the above information, the target detection confidence 0.4 and the target detection confidence 0.45 can be eliminated to obtain the second dangerous goods identification result including the target detection confidence 0.61 and the target detection confidence 0.8.

[0233] S403: For each human body in the second human body recognition result after elimination processing, determine the volume intersection and union ratio of the three-dimensional detection frame of the human body and the three-dimensional detection frame of each dangerous object in the second dangerous object recognition result after elimination processing; if the volume intersection and union ratio is greater than zero, determine the second probability that the human body lifts the dangerous object based on the target detection confidence of the human body, the target detection confidence of the dangerous object and the volume intersection and union ratio.

[0234] The second probability is used to characterize the possibility of a human body lifting a dangerous object. The larger the second probability, the greater the possibility of a human body lifting a dangerous object. Conversely, the smaller the second probability, the smaller the possibility of a human body lifting a dangerous object.

[0235] The purpose of this step is to find out from the second human body recognition result which people have the behavior of lifting dangerous objects, and to analyze the probability of these people lifting dangerous objects.

[0236] It can be understood that the 3D detection frame of the human body reflects the position of the human body in the 3D space, and the 3D detection frame of each dangerous object reflects the position of the dangerous object in the 3D space. Therefore, by calculating the 3D detection frame of the human body and the 3D detection frame of each dangerous object, it means that the degree of proximity or overlap between the human body and the dangerous object in space can be evaluated. When the volume intersection ratio is greater than zero, it directly means that in the 3D space, there is a certain degree of overlap or close contact between the human body and a certain dangerous object.

[0237] Optionally, the present application provides a possible implementation method for determining, for each human body in the second human body recognition result that has been eliminated, a volume intersection-combination ratio of a three-dimensional detection frame of the human body and a three-dimensional detection frame of each dangerous object in the second dangerous object recognition result that has been eliminated, including:

[0238] In the first step, for each human body in the second human body recognition result after elimination processing, according to the three-dimensional detection frame of each human body, the minimum position coordinates and the maximum position coordinates of the three-dimensional detection frame are determined.

[0239] The minimum position coordinate is used to represent the starting point of the three-dimensional detection frame of the human body, and the maximum position is used to represent the end point of the three-dimensional detection frame of the human body.

[0240] The purpose of this step is to obtain the starting and ending parts of the three-dimensional detection frame of each human body.

[0241] For example, suppose that after the elimination process, a 3D detection frame of a human body is identified, and its minimum position coordinates are (2, 3, 1) and the maximum position coordinates are (5, 7, 4). This means that the starting point of the 3D space region where the human body is located is (2, 3, 1) and the ending point is (5, 7, 4), thus obtaining the 3D detection frame of the human body.

[0242] In the second step, for each dangerous item in the second dangerous item identification result after elimination processing, according to its three-dimensional detection frame, the minimum position coordinates and the maximum position coordinates of the three-dimensional detection frame are determined.

[0243] The minimum position coordinate is used to represent the starting point of the dangerous goods, and the maximum position coordinate is used to represent the ending point of the dangerous goods.

[0244] For example, suppose a 3D detection frame of a dangerous object is identified, the minimum position coordinates are (4, 5, 2), and the maximum position coordinates are (7, 9, 5). This means that the spatial area of ​​the dangerous object is from (4, 5, 2) to (7, 9, 5), which is the 3D area where the dangerous object is located.

[0245] The third step is to traverse the 3D detection frames of all dangerous objects for each human body's 3D detection frame, calculate the intersection area between each human body's 3D detection frame and each dangerous object's 3D detection frame, and determine the minimum and maximum position coordinates of the intersection area.

[0246] The purpose of this step is to determine which three-dimensional detection frame of a dangerous object the three-dimensional detection frame of each human body intersects with, and then determine which dangerous object is related to which person.

[0247] For example, suppose the 3D detection frame of a human body is (2, 3, 1) to (5, 7, 4), and the 3D detection frame of a dangerous object is (4, 5, 2) to (7, 9, 5). Based on the above information, it can be determined that the minimum position coordinates of the intersection area are (4, 5, 2) and the maximum position coordinates are (5, 7, 4). Therefore, the intersection area is from (4, 5, 2) to (5, 7, 4), that is, the overlapping part of the two, so it can be determined that the human body is related to the dangerous object.

[0248] The fourth step is to determine whether the maximum position coordinate of the intersection area between the three-dimensional detection frame of each human body and the three-dimensional detection frame of each dangerous object is less than or equal to the minimum position coordinate of the intersection area.

[0249] The purpose of this step is to determine which three-dimensional detection frame the three-dimensional detection frame of the human body is related to.

[0250] For example, assuming that the minimum position coordinates of the intersection area between the three-dimensional detection frame of a human body and the three-dimensional detection frame of a dangerous object are (4, 5, 2), and the maximum position coordinates are (5, 7, 4), then based on the above information, it can be determined that the maximum position coordinates of the intersection area are less than or equal to the minimum position coordinates of the intersection area.

[0251] In the fifth step, when the maximum position coordinate of the intersection area is less than or equal to the minimum position coordinate, the volume of the intersection area between the three-dimensional detection frame of each human body and the three-dimensional detection frame of the corresponding dangerous object is calculated.

[0252] The purpose of this step is to determine the volume of the intersection between the three-dimensional detection frame of the human body and the three-dimensional detection frame of the associated dangerous object.

[0253] For example, assuming that the minimum position coordinates of the intersection area between a human body 3D detection frame and a dangerous object 3D detection frame are (4, 5, 2) and the maximum position coordinates are (5, 7, 4), then based on the above information, it can be determined that the volume of the intersection area is 4 cubic meters.

[0254] The sixth step is to calculate the volume of each human body's three-dimensional detection frame and the corresponding dangerous object's three-dimensional detection frame excluding the intersection area.

[0255] For example, suppose a 3D detection frame of a human body is (2, 3, 1) to (5, 7, 4), and a 3D detection frame of a dangerous object is (4, 5, 2) to (7, 9, 5). The volume of the intersection area between the 3D detection frame of the human body and the 3D detection frame of the dangerous object is 4 cubic meters. Based on the above information, the volume of the 3D detection frame of the human body and the volume of the 3D detection frame of the dangerous object can be determined to be 36 cubic meters, and then the volume of the 3D detection frame of the human body and the corresponding 3D detection frame of the dangerous object, excluding the intersection area, can be determined to be 32 cubic meters.

[0256] In the seventh step, the volume intersection ratio is calculated according to the volume of the intersection area between each human body's three-dimensional detection frame and the corresponding dangerous object's three-dimensional detection frame and the volume excluding the intersection area.

[0257] For example, assuming that the volume of the three-dimensional detection frame of the human body and the corresponding three-dimensional detection frame of the dangerous object excluding the intersection area is 32 cubic meters, and the volume of the intersection area between the three-dimensional detection frame of the human body and the corresponding three-dimensional detection frame of the dangerous object is 4 cubic meters, then based on the above information, it can be determined that the volume intersection ratio is 0.059.

[0258] Optionally, the present application provides a possible implementation method for determining the second probability that a human body lifts a dangerous object based on the target detection confidence of the human body, the target detection confidence of the dangerous object, and the volume intersection and union ratio, including: Pli =0.98*(W1P k +W2P P )+0.1*V IOU

[0259] Among them, P li represents the second probability of a person lifting a dangerous object, W1 represents the weight of the dangerous object, P k represents the confidence of target detection of dangerous objects, W2 represents the weight of human body, P P Represents the confidence of human target detection, V IOU It represents the volume intersection ratio.

[0260] For example, assuming that the weight of the dangerous object is 0.1, the target detection confidence of the dangerous object is 0.9, the weight of the human body is 0.2, the target detection confidence of the human body is 0.8, and the volume intersection ratio is 0.059. Based on the above information, it can be determined that the second probability of the human body lifting the dangerous object is 0.25.

[0261] The method for identifying dangerous behaviors provided in this embodiment first eliminates the data in the second human recognition result whose target detection confidence is lower than the third threshold value to obtain the optimized second human recognition result. Next, eliminate the data in the second dangerous goods recognition result whose target detection confidence is lower than the fourth threshold value to obtain the optimized second dangerous goods recognition result. Subsequently, for each human body in the optimized human recognition result, the volume intersection and union ratio of the three-dimensional detection frame of each dangerous goods is calculated. If the volume intersection and union ratio is greater than zero, it indicates that the human body and the dangerous goods overlap or are close in space. At this time, the second probability that the human body lifts the dangerous goods is comprehensively evaluated and determined by combining the target detection confidence of the human body, the target detection confidence of the dangerous goods and the volume intersection and union ratio. This method not only improves the accuracy of interactive identification of the human body and dangerous goods through the calculation of the volume intersection and union ratio of the three-dimensional detection frame and the probability evaluation of multiple factors, but also provides more reliable technical support for safety management.

[0262] Figure 5 This is a schematic diagram of the structure of the dangerous behavior identification device provided by this application. Figure 5 As shown, the present application provides a device for identifying dangerous behaviors, and the device 500 for identifying dangerous behaviors includes:

[0263] The shooting module 501 is used to shoot the target place through a camera to obtain an image to be identified;

[0264] A scanning module 502, configured to scan the target location using a laser radar to obtain first point cloud data;

[0265] An extraction module 503 is used to extract a first human recognition result and a first dangerous object recognition result from the image to be recognized;

[0266] A determination module 504, configured to determine a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result;

[0267] The determination module 504 is further configured to determine the color of each point in the first point cloud data according to the image to be recognized, so as to obtain second point cloud data with added color;

[0268] The extraction module 503 is further used to extract a second human recognition result and a second dangerous goods recognition result from the second point cloud data;

[0269] The determination module 504 is further configured to determine a second probability that a person lifts a dangerous object based on the second human recognition result and the second dangerous object recognition result;

[0270] The determination module 504 is further configured to determine, based on the first probability and the second probability, whether there is a dangerous behavior of a person lifting a dangerous object.

[0271] Optionally, the device further includes: an input module 505;

[0272] The input module 505 is used to input the image to be recognized into a pre-trained first recognition model to obtain a first human recognition result output by the first recognition model, wherein the first human recognition result includes a human detection frame, a human detection confidence, and human skeleton key points;

[0273] The input module 505 is further used to input the image to be identified into a pre-trained second recognition model to obtain a first dangerous goods identification result output by the second recognition model, wherein the dangerous goods identification result includes a dangerous goods detection frame and a dangerous goods detection confidence.

[0274] Optionally, the device further includes: a rejection module 506;

[0275] The elimination module 506 is used to eliminate data whose human detection confidence level is lower than a first threshold in the first human recognition result, so as to obtain a first human recognition result after elimination processing;

[0276] The elimination module 506 is further configured to eliminate data in the first dangerous article identification result whose dangerous article detection confidence level is lower than a second threshold value, so as to obtain a first dangerous article identification result after elimination processing;

[0277] The determination module 504 is further used to determine an arm rectangular frame for each human body in the first human body recognition result that has been eliminated, using the wrist point and the elbow point in the human skeleton key points as diagonal points, and determine the center point of the arm rectangular frame;

[0278] The device further includes: a processing module 507;

[0279] The processing module 507 is used to expand the arm rectangular frame while keeping the center point unchanged to obtain an expanded arm rectangular frame;

[0280] The determination module 504 is further configured to determine an area intersection-and-union ratio between the expanded arm rectangular frame and each dangerous object detection frame in the first dangerous object identification result that has been eliminated, and if the area intersection-and-union ratio is greater than zero, determine whether the arm is raised according to the coordinates of the wrist point and the elbow point, and if the arm is raised, determine the degree of arm raising;

[0281] The determination module 504 is specifically configured to determine a first probability that a human body lifts a dangerous object according to the human body detection confidence, the dangerous object detection confidence, the area intersection-over-union ratio, and the arm raising degree.

[0282] Optionally, the device further includes: a conversion module 508;

[0283] The conversion module 508 is used to perform coordinate conversion on the first point cloud data to obtain the two-dimensional coordinates of each point in the first point cloud data in a pixel coordinate system;

[0284] The determination module 504 is specifically configured to determine the color of the pixel in the to-be-recognized image corresponding to the two-dimensional coordinates of each point in the pixel coordinate system as the color of each point.

[0285] Optionally, the input module 505 is also used to input the second point cloud data into a pre-trained third recognition model to obtain a target detection result output by the third recognition model, wherein the target detection result includes a three-dimensional detection box of the target, a target category, and a target detection confidence, wherein the target detection result whose target category is human is the second human recognition result, and the target detection result whose target category is dangerous goods is the second dangerous goods recognition result.

[0286] Optionally, the elimination module 503 is further used to eliminate data whose target detection confidence level in the second human recognition result is lower than a third threshold, so as to obtain a second human recognition result after elimination processing;

[0287] The elimination module 503 is further used to eliminate data in the second dangerous goods identification result whose target detection confidence is lower than a fourth threshold, so as to obtain a second dangerous goods identification result after elimination processing;

[0288] The determination module 504 is specifically used to determine, for each human body in the second human body recognition result that has been eliminated, a volume intersection-and-union ratio between the three-dimensional detection frame of the human body and the three-dimensional detection frame of each dangerous object in the second dangerous object recognition result that has been eliminated; if the volume intersection-and-union ratio is greater than zero, then determining a second probability that the human body lifts the dangerous object based on the target detection confidence of the human body, the target detection confidence of the dangerous object and the volume intersection-and-union ratio.

[0289] Optionally, the device further includes: a fusion module 509;

[0290] The fusion module 509 is used to fuse the first probability and the second probability using DS evidence theory to obtain a fused probability that a human body lifts a dangerous object;

[0291] The determination module 504 is used to determine that there is a dangerous behavior of a human lifting a dangerous object when it is determined that the fusion probability is greater than a probability threshold.

[0292] Figure 6 This is a schematic diagram of the structure of the dangerous behavior identification device provided by this application. Figure 6 As shown, the present application provides a dangerous behavior identification device, and the dangerous behavior identification device 600 includes: a receiver 601, a transmitter 602, a processor 603 and a memory 604.

[0293] Receiver 601, used for receiving instructions and data;

[0294] A transmitter 602, used for sending instructions and data;

[0295] Memory 604, used to store computer-executable instructions;

[0296] The processor 603 is used to execute the computer-executable instructions stored in the memory 604 to implement the various steps performed by the dangerous behavior identification method in the above embodiment. For details, please refer to the relevant description in the above dangerous behavior identification method embodiment.

[0297] Optionally, the memory 604 may be independent or integrated with the processor 603 .

[0298] When the memory 604 is independently provided, the electronic device further includes a bus for connecting the memory 604 and the processor 603 .

[0299] The present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, a dangerous behavior identification method as performed by the dangerous behavior identification device described above is implemented.

[0300] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementations, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0301] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific embodiments, and the above embodiments are only used to illustrate the technical solution of the present application rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying dangerous behavior, characterized in that: The method comprises: The target location is photographed by a camera to obtain an image to be identified, and the target location is scanned by a laser radar to obtain first point cloud data; Extracting a first human body recognition result and a first dangerous object recognition result from the image to be recognized, and determining a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result; Determine the color of each point in the first point cloud data according to the image to be recognized, obtain second point cloud data with added color, extract a second human body recognition result and a second dangerous object recognition result from the second point cloud data, and determine a second probability that a person lifts a dangerous object according to the second human body recognition result and the second dangerous object recognition result; According to the first probability and the second probability, it is determined whether there is a dangerous behavior of a person lifting a dangerous object.

2. The method according to claim 1, characterized in that The extracting the first human body recognition result and the first dangerous goods recognition result from the image to be recognized includes: Inputting the image to be recognized into a pre-trained first recognition model to obtain a first human body recognition result output by the first recognition model, wherein the first human body recognition result includes a human body detection frame, a human body detection confidence, and human body skeleton key points; The image to be identified is input into a pre-trained second identification model to obtain a first dangerous object identification result output by the second identification model, wherein the dangerous object identification result includes a dangerous object detection frame and a dangerous object detection confidence.

3. The method according to claim 2, characterized in that The determining, according to the first human body recognition result and the first dangerous object recognition result, a first probability that a human body lifts a dangerous object comprises: Eliminate data whose human detection confidence level is lower than a first threshold in the first human recognition result to obtain a first human recognition result after elimination processing; Eliminate data whose dangerous goods detection confidence level is lower than a second threshold value in the first dangerous goods identification result to obtain a first dangerous goods identification result after elimination processing; For each human body in the first human body recognition result that has been eliminated, an arm rectangular frame is determined with the wrist point and the elbow point in the human skeleton key points as diagonal points, and the center point of the arm rectangular frame is determined, and the arm rectangular frame is expanded while keeping the center point unchanged to obtain an expanded arm rectangular frame; Determine the area intersection and union ratio of the expanded arm rectangular frame and each dangerous object detection frame in the first dangerous object identification result that has been eliminated, and if the area intersection and union ratio is greater than zero, determine whether the arm is raised according to the coordinates of the wrist point and the elbow point, and if the arm is raised, determine the degree of arm raising; A first probability that the human body lifts the dangerous object is determined according to the human body detection confidence, the dangerous object detection confidence, the area intersection-over-union ratio, and the arm raising degree.

4. The method according to any one of claims 1 to 3, characterized in that: The step of determining the color of each point in the first point cloud data according to the image to be recognized includes: Performing coordinate transformation on the first point cloud data to obtain two-dimensional coordinates of each point in the first point cloud data in a pixel coordinate system; The color of the pixel point in the to-be-recognized image corresponding to the two-dimensional coordinates of each point in the pixel coordinate system is determined as the color of each point.

5. The method according to any one of claims 1 to 3, characterized in that: The extracting of the second human body recognition result and the second dangerous goods recognition result from the second point cloud data includes: The second point cloud data is input into a pre-trained third recognition model to obtain a target detection result output by the third recognition model, wherein the target detection result includes a three-dimensional detection box of the target, a target category, and a target detection confidence, wherein the target detection result whose target category is human is the second human recognition result, and the target detection result whose target category is dangerous goods is the second dangerous goods recognition result.

6. The method according to claim 5, characterized in that The determining, according to the second human body recognition result and the second dangerous object recognition result, a second probability that a person lifts a dangerous object comprises: Eliminate data whose target detection confidence level in the second human recognition result is lower than a third threshold value, to obtain a second human recognition result after elimination processing; Eliminate data whose target detection confidence level is lower than a fourth threshold value in the second dangerous goods identification result to obtain a second dangerous goods identification result after elimination processing; For each human body in the second human body recognition result after elimination processing, determine the volume intersection and union ratio of the three-dimensional detection frame of the human body and the three-dimensional detection frame of each dangerous object in the second dangerous object identification result after elimination processing; if the volume intersection and union ratio is greater than zero, determine the second probability that the human body lifts the dangerous object based on the target detection confidence of the human body, the target detection confidence of the dangerous object and the volume intersection and union ratio.

7. The method according to any one of claims 1 to 3, characterized in that: The determining, based on the first probability and the second probability, whether there is a dangerous behavior of a human body lifting a dangerous object includes: The first probability and the second probability are fused by using DS evidence theory to obtain a fused probability that a human body lifts a dangerous object. If the fused probability is greater than a probability threshold, it is determined that there is a dangerous behavior of a human body lifting a dangerous object.

8. A device for identifying dangerous behavior, characterized in that: include: A shooting module is used to shoot a target location through a camera to obtain an image to be identified; A scanning module, used for scanning the target location by a laser radar to obtain first point cloud data; An extraction module, used to extract a first human recognition result and a first dangerous object recognition result from the image to be recognized; A determination module, configured to determine a first probability that a human body lifts a dangerous object according to the first human body recognition result and the first dangerous object recognition result; The determination module is further used to determine the color of each point in the first point cloud data according to the image to be recognized, so as to obtain second point cloud data with added color; The extraction module is further used to extract a second human body recognition result and a second dangerous goods recognition result from the second point cloud data; The determination module is further used to determine a second probability that a person lifts a dangerous object based on the second human recognition result and the second dangerous object recognition result; The determination module is further used to determine whether there is a dangerous behavior of a person lifting a dangerous object based on the first probability and the second probability.

9. A device for identifying dangerous behavior, characterized in that: include: Memory; processor; Wherein, the memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method for identifying dangerous behaviors according to any one of claims 1-7.

10. A computer storage medium, characterized in that: The computer storage medium stores computer executable instructions, which, when executed by a processor, are used to implement the dangerous behavior identification method according to any one of claims 1 to 7.

Images