Abnormal behavior detection method, device, electronic device, and storage medium

By conducting object detection and motion state analysis on the detection images in coal mine tunnel transportation, combined with overlapping degree and motion state information, the accurate identification and alarm of violations is achieved, and the problems of inaccurate detection and poor reliability in the prior art are solved. Improves detection efficiency and safety.

CN113674311BActive Publication Date: 2024-11-29ZHEJIANG DAHUA TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202110736796.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-06-30
Publication Date
2024-11-29
Estimated Expiration
2041-06-30

AI Technical Summary

Technical Problem

The detection of violations in coal mine tunnel transportation is not accurate enough, and the reliance on manual inspection is low efficiency and poor reliability. The existing technology cannot accurately judge the type of violation and record evidence, resulting in safety hazards and difficulty in obtaining evidence.

Method used

By detecting the detected image, obtaining the target position and motion state information, combining the overlap degree and motion state information, we can judge whether the behavior violates safety regulations, output alarm processing, and realize automatic detection and alarm.

Benefits of technology

Improves detection efficiency and reliability, can accurately identify behaviors that violate safety regulations without the need for multiple sensors, reduce costs, avoid misjudgment, and ensure safety and reliability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113674311B_ABST
    Figure CN113674311B_ABST
Patent Text Reader

Abstract

The present invention provides an abnormal behavior detection method, device, electronic device, and computer-readable storage medium. The abnormal behavior detection method includes: performing target detection on a detection image, obtaining the position information of a first detection target and the position information of a second detection target, and obtaining the motion state information of the first detection target and the motion state information of the second detection target; based on the position information of the first detection target and the position information of the second detection target combined with the motion state information of the first detection target and the motion state information of the second detection target, determining whether the behavior of the first detection target is abnormal behavior. This method can detect behaviors that violate safety regulations, with high detection efficiency and strong reliability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of image behavior analysis, and in particular to an abnormal behavior detection method, device, electronic equipment, and storage medium. Background Art

[0002] In some areas, abnormal behaviors can lead to major safety hazards, but the detection of some abnormal behaviors is not accurate enough. For example, in the transportation lanes of the coal mining industry, the characteristics of the lane transportation are: large transportation volume, complex lane routes, narrow lanes, and lane transportation operations must be carried out in strict accordance with relevant safety regulations. Illegal behaviors such as riding, kicking, and kicking hooks by underground lane operators are very likely to cause serious accidents with casualties.

[0003] At present, the inspection of illegal personnel in coal mine tunnel transportation mostly relies on manual inspection by safety supervision and monitoring rooms, which is easy to lead to missed inspections due to negligence or distraction of inspectors, and the manual inspection method is inefficient and unreliable. Summary of the invention

[0004] The present invention provides an abnormal behavior detection method, device, electronic equipment and storage medium. The method can detect abnormal behavior of a target with high detection efficiency and strong reliability.

[0005] To solve the above technical problems, the first technical solution provided by the present invention is: to provide a target detection method, including: performing target detection on a detection image, obtaining position information of a first detection target and position information of a second detection target, and obtaining motion state information of the first detection target and motion state information of the second detection target; based on the position information of the first detection target and the position information of the second detection target combined with the motion state information of the first detection target and the motion state information of the second detection target, determining whether the behavior of the first detection is an abnormal behavior.

[0006] Wherein, determining whether the behavior of the first detection target is abnormal behavior includes: if it is determined that the behavior of the first detection target violates preset safety regulations, then determining that the behavior is abnormal behavior; and performing alarm processing based on the detection image.

[0007] Among them, the step of determining whether the behavior of the first detection target is abnormal behavior based on the position information of the first detection target, the position information of the second detection target combined with the motion state information of the first detection target and the motion state information of the second detection target includes: determining the degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; determining whether the behavior of the first detection target is abnormal behavior based on the degree of overlap combined with the motion state information of the first detection target and the motion state information of the second detection target.

[0008] Among them, the step of determining the degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target includes: determining the intersection-and-union ratio of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; and determining the intersection-and-union ratio as the degree of overlap.

[0009] Among them, the step of determining the intersection-and-union ratio of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target includes: determining the intersection of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; determining the union of the first detection target and the second detection target based on the size of the first detection target, the size of the second detection target and the intersection; determining the intersection-and-union ratio based on the intersection and the union.

[0010] Among them, the step of determining whether the behavior of the first detection target is abnormal behavior based on the overlap degree combined with the motion state information of the first detection target and the motion state information of the second detection target includes: if it is determined that the overlap degree is within a preset range, and it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, then determining that the behavior of the first detection target is abnormal behavior, wherein the second detection target is a second detection target overlapping with the first detection target.

[0011] Wherein, the determination that the motion state information of the first detection target matches the motion state information of the second detection target includes: the motion state information includes motion speed, and if it is determined that the motion speed of the first detection target matches the motion speed of the second detection target, then it is determined that the motion state information of the first detection target matches the motion state information of the second detection target; or the motion state information includes motion trajectory, and if it is determined that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, then it is determined that the motion state information of the first detection target matches the motion state information of the second detection target; or the motion state information includes motion speed and motion trajectory, and if it is determined that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, and it is determined that the motion speed of the first detection target matches the motion speed of the second detection target, then it is determined that the motion state information of the first detection target matches the motion state information of the second detection target.

[0012] Among them, after determining that the degree of overlap is within a preset range, and before determining that the motion state information of the first detection target matches the motion state information of the second detection target, it also includes: determining the position information of the first detection target, and determining the position information of the second detection target adjacent to the first detection target; determining that the position relationship between the first detection target and the second detection target meets a preset position condition.

[0013] Among them, the determination that the positional relationship between the first detection target and the second detection target satisfies a preset position condition includes: if it is determined that the coordinates of the first direction of the upper left corner position of the first detection target are smaller than the coordinates of the first direction of the upper left corner position of the second detection target, and / or it is determined that the coordinates of the first direction of the lower right corner position of the first detection target are larger than the coordinates of the first direction of the lower right corner position of the second detection target, then it is determined that the positional relationship between the first detection target and the second detection target satisfies the preset position condition.

[0014] The first detection target includes a person, and the second detection target includes a vehicle.

[0015] To solve the above technical problems, the second technical solution provided by the present invention is: to provide an abnormal behavior detection device, including: a target detection module, used to perform target detection on a detection image, obtain the position information of a first detection target and the position information of a second detection target, and obtain the motion state information of the first detection target and the motion state information of the second detection target; a judgment module, used to determine whether the first detected behavior is an abnormal behavior based on the position information of the first detection target, the position information of the second detection target, combined with the motion state information of the first detection target and the motion state information of the second detection target.

[0016] To solve the above technical problems, the third technical solution provided by the present invention is: to provide an electronic device, comprising: a memory and a processor, wherein the memory stores program instructions, and the processor calls the program instructions from the memory to execute any of the above methods.

[0017] In order to solve the above technical problems, the fourth technical solution provided by the present invention is: providing a computer-readable storage medium storing a program file, and the program file can be executed to implement any of the above methods.

[0018] The beneficial effect of the present invention is different from the prior art. The present invention determines whether the safety regulations are violated based on the position information of the first detection target and the second detection target combined with the movement speed and / or movement trajectory of the first detection target and the second detection target; in response to the violation of the safety regulations, the result is output. This method can detect the behavior that violates the safety regulations, with high detection efficiency and strong reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0020] Figure 1 It is a flowchart of a first embodiment of the abnormal behavior detection method of the present invention;

[0021] Figure 2 for Figure 1 A schematic flow chart of an embodiment of step S12;

[0022] Figure 3a A schematic diagram of a first embodiment of the positional relationship between a person and a vehicle during a hook-kicking behavior;

[0023] Figure 3bA schematic diagram of a second embodiment of the positional relationship between a person and a vehicle during a hook-kicking behavior;

[0024] Figure 4 It is a structural schematic diagram of an embodiment of an abnormal behavior detection device of the present invention;

[0025] Figure 5 It is a structural schematic diagram of an embodiment of an electronic device of the present invention;

[0026] Figure 6 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present invention. DETAILED DESCRIPTION

[0027] Safety in production is the top priority of the coal mining industry, especially in the transportation tunnels. The characteristics of tunnel transportation are: large transportation volume, complex tunnel routes, and narrow tunnels. The tunnel transportation operation must be carried out in strict accordance with relevant safety regulations. Illegal behaviors such as riding, kicking, and kicking hooks by underground tunnel workers are very likely to cause serious accidents with casualties.

[0028] At present, the inspection of illegal personnel in coal mine tunnel transportation mostly relies on manual inspection by safety supervision and monitoring rooms, which is easy to lead to missed inspections due to negligence or distraction of inspectors, and the manual inspection method is inefficient and unreliable.

[0029] The prior art proposes a safety detection system for preventing locomotives from being lost, stolen, and rear-end collisions in underground coal mines, which includes a vehicle-mounted host and a rear vehicle detection device. The rear vehicle detection device collects vehicle-lost and vehicle-stealing signals in real time, and transmits them to the receiving end of the vehicle-mounted host of the local vehicle in a wireless form, and sends a wireless signal to the vehicle-mounted host of the rear vehicle at the same time; the vehicle-mounted host of the local vehicle receives the alarm signal sent by the rear vehicle detection device of the local vehicle and sends the alarm signal in the form of sound, light, and voice to remind the driver to take corresponding measures; when the vehicle-mounted host of the rear vehicle receives the anti-rear-end collision signal sent by the rear vehicle detection device of the front vehicle, it reminds the driver to slow down. This method can only determine whether a person is close to a vehicle, but cannot accurately determine the type of safety regulations that he has violated. In addition, when a violation occurs, it is not easy for the system to record evidence of the violation, resulting in difficulty in obtaining evidence.

[0030] The prior art provides an underground locomotive driving safety monitoring device, which consists of a detection safety sensor, a signal receiving and processing and alarm device, the signal receiving and processing and alarm device is installed in the locomotive, and the detection safety sensor is installed on the roof and taillight position of the locomotive to realize the detection and alarm of obstacles behind the vehicle and in front of the vehicle. This method requires multiple sensors, is costly, and cannot detect the pedaling behavior in the middle of the vehicle.

[0031] The present application proposes a target detection method, which can detect behaviors that violate safety regulations and can determine the types of violations of safety regulations. It does not require multiple sensors, has low cost, high detection efficiency, and strong reliability. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0032] See also Figure 1 , is a flow chart of the first embodiment of the target detection method of the present invention. Specifically, it includes:

[0033] Step S11: performing target detection on the detection image, obtaining position information of a first detection target and position information of a second detection target, and obtaining motion state information of the first detection target and motion state information of the second detection target.

[0034] Specifically, an image / video acquisition device, such as a monitoring device, is installed in the area where detection is required. Take the coal mine transportation channel as an example. The monitoring device is installed in the coal mine channel, and the installation angle of the monitoring device can be adjusted, as long as it can ensure that the monitoring device can detect the driving conditions of vehicles in the moving channel. The monitoring device can be an explosion-proof IP camera, and its imaging spectrum is visible light or infrared light.

[0035] Target detection is performed on the detection image acquired by the monitoring device to obtain the position information of the first detection target and the position information of the second detection target. In one embodiment, the target detection algorithm can be used to perform target detection on the detection image to obtain the position information of the first detection target and the position information of the second detection target. The target detection algorithm can be, for example, a neural network algorithm, including but not limited to the RCNN series, SSD, YOLO series, etc.

[0036] In a specific embodiment, the target detection algorithm can identify the identity information of the first detection target and the second detection target, and the detection frame coordinates of the first detection target and the second detection target. It can be understood that if it is necessary to detect whether a hook-kicking behavior occurs, the first detection target is a person and the second detection target is a vehicle, that is, the position information of the person and the vehicle is detected. Specifically, the target detection algorithm can detect the upper left corner coordinates and the lower right corner coordinates of the detection frames of the first detection target and the second detection target, and then obtain the position information of the first detection target and the position information of the second detection target.

[0037] In one embodiment, the first detection target and the second detection target may also be tracked based on their identity information to obtain motion state information of the first detection target and the second detection target. Specifically, the first detection target and the second detection target may be tracked using a target tracking algorithm, including but not limited to a SORT algorithm, a DeepSORT algorithm, and the like.

[0038] Step S12: Determine whether the behavior of the first detection target is abnormal behavior based on the position information of the first detection target, the position information of the second detection target, the motion state information of the first detection target, and the motion state information of the second detection target.

[0039] Specifically, whether the behavior of the first detection target is abnormal behavior is determined based on the position information of the first detection target, the position information of the second detection target, the motion state information of the first detection target, and the motion state information of the second detection target.

[0040] In a specific embodiment, the positional relationship between the first detection target and the second detection target can be derived based on the position information of the first detection target and the position information of the second detection target, and then combined with the motion state information of the first detection target and the motion state information of the second detection target, it is determined whether the hook kicking behavior occurs, that is, whether the behavior of the first detection target is abnormal behavior.

[0041] In one embodiment, if it is determined that the behavior of the first detection target violates the preset safety regulations, then the behavior of the first detection target is determined to be abnormal behavior. For example, if it is determined that the first detection target has a hook-kicking behavior, then the behavior of the first detection target is determined to violate the preset safety regulations. For another example, if it is determined that the first detection target has a climbing behavior, such as climbing a tree, then the behavior of the first detection target is determined to violate the preset safety regulations. For another example, if it is determined that the first detection target has a car-stealing behavior, then the behavior of the first detection target is determined to violate the preset safety regulations, without specific limitation.

[0042] The method described in this embodiment can determine whether the first detection target has abnormal behavior based on the position information and motion state information of the first detection target and the position information and motion state information of the second detection target. The method is efficient and reliable.

[0043] Please combine Figure 2 , Figure 2 The flowchart of a specific embodiment of step S12 includes:

[0044] Step S21: determining the degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target.

[0045] In one embodiment, the intersection-and-union ratio of the first detection target and the second detection target may be calculated and determined based on the position information of the first detection target and the position information of the second detection target, and the intersection-and-union ratio may be determined as the degree of overlap.

[0046] Specifically, the intersection-over-union ratio is calculated by calculating and determining the intersection of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target. The specific calculation method is as follows:

[0047] overlap=(min(x H2 ,x c2 )-max(x H1 ,x c1 ))×(min(y H2 ,y c2 )-max(y H1 ,y c1 )) (1);

[0048] Among them, overlap is the intersection, (x H1 ,y H1 ) is the upper left corner position information of the first detection target, (x H2 ,y H2 ) is the lower right corner position information of the first detection target, (x c1 ,y c1 ) is the upper left corner position information of the second detection target, (x c2 ,y c2 ) is the lower right corner position information of the second detection target.

[0049] Based on the size of the first detection target, the size of the second detection target and the intersection, the union of the first detection target and the second detection target is calculated. The specific calculation method is as follows:

[0050] union=w1×h1+w2×h2-overlap (2);

[0051] Among them, union is the union, w1 is the length of the detection box corresponding to the first detection target, h1 is the width of the detection box corresponding to the first detection target, w1 = x H2 -x H1 ; h1=y H2 -y H1 w2 is the length of the detection frame corresponding to the second detection target, h2 is the width of the detection frame corresponding to the second detection target, w2 = x c2 -x c1 ; h1=y c2 -y c1 .

[0052] The intersection-union ratio is calculated based on the intersection and union. The specific calculation is as follows:

[0053]

[0054] It should be noted that the intersection-over-union (IOU) ratio indicates the degree of overlap.

[0055] It should be noted that when performing target detection on the detection image, the upper left corner position information (x H1 ,y H1 ) and the lower right corner position information (x H2 ,y H2 ), the upper left corner position information of the second detection target can also be obtained (x c1 ,y c1 ) and the lower right corner position information (x c2 ,y c2 ). Based on the upper left corner position information of the first detected target (x H1 ,y H1 ) and the lower right corner position information (x H2 ,y H2 ) calculates the length w1 and width h1 of the detection box corresponding to the first detection target, based on the upper left corner position information (x c1 ,y c1 ) and the lower right corner position information (x c2 ,y c2 ) Calculate the length w2 and width h2 of the detection box corresponding to the second detection target.

[0056] Step S22: Determine whether the behavior of the first detection target is abnormal behavior based on the overlap degree and the motion state information of the first detection target and the motion state information of the second detection target.

[0057] Specifically, determine whether the degree of overlap (i.e., the intersection-over-union (IOU)) is within a preset range. If it is determined that the degree of overlap is within the preset range, compare the motion state information of the first detection target and the motion state information of the second detection target, and the second detection target is the second detection target that overlaps with the first detection target. Determine whether the motion state information of the first detection target matches the motion state information of the second detection target. If it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, then determine that the behavior of the first detection target is abnormal behavior.

[0058] Specifically, the preset range is (IOU0, IOU1), that is, if IOU0<IOU<IOU1, the motion state information of the first detection target is further compared with the motion state information of the second detection target to determine whether the motion state information of the first detection target matches the motion state information of the second detection target. If it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, it is determined that the behavior of the first detection target is abnormal behavior. Among them, IOU0 can take the value of 0, and IOU1 can take the value according to the actual application scenario, for example, it can take the value of 0.2, 0.3, 0.5, 0.8.

[0059] In a specific embodiment, the motion state information includes motion speed. That is, if IOU0<IOU<IOU1, the motion speed of the first detection target is further compared with the motion speed of the second detection target to determine whether the motion speed of the first detection target matches the motion speed of the second detection target. If it is determined that the motion speed of the first detection target matches the motion speed of the second detection target, it is determined that the behavior of the first detection target is abnormal behavior. It should be noted that the second detection target is a second detection target that overlaps with the first detection target.

[0060] In a specific embodiment, the motion state information includes a motion trajectory. That is, if IOU0<IOU<IOU1, the motion trajectory of the first detection target is further compared with the motion trajectory of the second detection target to determine whether the motion trajectory of the first detection target matches the motion trajectory of the second detection target. If it is determined that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, then the behavior of the first detection target is determined to be abnormal behavior. It should be noted that the second detection target is a second detection target that overlaps with the first detection target.

[0061] In a specific embodiment, the motion state information includes a motion trajectory and a motion speed. That is, if IOU0<IOU<IOU1, the motion trajectory of the first detection target is further compared with the motion trajectory of the second detection target, and the motion speed of the first detection target is compared with the motion speed of the second detection target to determine whether the motion trajectory of the first detection target matches the motion trajectory of the second detection target, and determine whether the motion speed of the first detection target matches the motion speed of the second detection target. If it is determined that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, and it is determined that the motion speed of the first detection target matches the motion speed of the second detection target, then it is determined that the behavior of the first detection target is abnormal behavior. It should be noted that the second detection target is a second detection target that overlaps with the first detection target.

[0062] In another embodiment, if the overlap degree is within a preset range, it is necessary to further determine the position information of the first detection target, and determine the position information of the second detection target adjacent to the first detection target, and determine whether the position relationship between the first detection target and the second detection target meets the preset position condition. If the position relationship between the first detection target and the second detection target meets the preset position condition, it is further determined whether the motion state information of the first detection target matches the motion state information of the second detection target. Specifically, it can be determined based on the position information of the first detection target and the second detection target whether the first detection target is located to the left of the second detection target, or to the right of the second detection target.

[0063] Specifically, in one embodiment, the coordinates of the upper left corner of the first detection target in the first direction (X direction) and the coordinates of the upper left corner of the second detection target (the second detection target adjacent to the first detection target) in the first direction (X direction) can be compared. Figure 3a As shown, the coordinates of the first direction of the upper left corner position A of the first detection target 31 are compared with the coordinates of the first direction of the upper left corner position A1 of the second detection target 32. If the coordinates of the first direction of the upper left corner position A of the first detection target 31 are less than the coordinates of the first direction of the upper left corner position A1 of the second detection target 33, it means that the first detection target 31 is located on the left side of the second detection target 32. At this time, it is determined that the positional relationship between the first detection target and the second detection target meets the preset position condition. It is further determined whether the motion state information of the first detection target matches the motion state information of the second detection target.

[0064] In another embodiment, the coordinates of the first direction (X direction) of the lower right corner position of the first detection target are compared with the coordinates of the first direction (X direction) of the lower right corner position of the second detection target. Figure 3b As shown, the coordinates of the first direction of the lower right corner position B of the first detection target 31 are compared with the coordinates of the first direction of the lower right corner position B1 of the second detection target 32. If the coordinates of the first direction of the lower right corner position B of the first detection target 31 are greater than the coordinates of the first direction of the lower right corner position B1 of the second detection target 32, it means that the first detection target 31 is located on the right side of the second detection target. At this time, it is determined that the positional relationship between the first detection target and the second detection target meets the preset position condition. It is further determined whether the motion state information of the first detection target matches the motion state information of the second detection target.

[0065] In one embodiment, when IOU≥IOU1, or IOU≤IOU0, the corresponding target may be ignored.

[0066] Specifically, through the method described above, if it is determined that the first detection target violates the preset safety regulations, an alarm is processed based on the detection image. The method for performing the alarm processing includes: displaying the detection image on a display screen; or, displaying the detection image on a display screen with prompts such as flashing lights, vibrations, and sounds; or obtaining N frames of images adjacent to the detection image as evidence and saving them locally, where N can be 2, 3, 5, 7, etc.; or, capturing a picture of the first detection target from the detection image and saving it locally or displaying it; or, capturing a picture of the first detection target from the detection image, identifying the first detection target, matching the identity information of the first detection target (such as name, ID card, etc.) and displaying it, without specific limitation.

[0067] The abnormal behavior detection method of the present application can accurately identify whether a hook-kicking behavior occurs based on the relative position and movement relationship between the person and the vehicle, avoid misjudgment, and can largely avoid false alarms caused by non-human target movements. There is no need to install additional sensors, reducing costs, and the detection efficiency is high and the reliability is strong.

[0068] See also Figure 4 , is a schematic diagram of the structure of an embodiment of an abnormal behavior detection device of the present invention, which specifically includes: a target detection module 41 and a judgment module 42.

[0069] The target detection module 41 is used to perform target detection on the detection image, obtain the position information of the first detection target and the position information of the second detection target, and obtain the motion state information of the first detection target and the motion state information of the second detection target.

[0070] Specifically, an image / video acquisition device, such as a monitoring device, is installed in the area where detection is required. Take the coal mine transportation channel as an example. The monitoring device is installed in the coal mine channel, and the installation angle of the monitoring device can be adjusted, as long as it can ensure that the monitoring device can detect the driving conditions of vehicles in the moving channel. The monitoring device can be an explosion-proof IP camera, and its imaging spectrum is visible light or infrared light.

[0071] The target detection module 41 performs target detection on the detection image acquired by the monitoring device, and then obtains the position information of the first detection target and the position information of the second detection target. In one embodiment, the target detection algorithm can be used to perform target detection on the detection image to obtain the position information of the first detection target and the position information of the second detection target. The target detection algorithm can be, for example, a neural network algorithm, including but not limited to the RCNN series, SSD, YOLO series, etc.

[0072] In a specific embodiment, the target detection module 41 can identify the identity information of the first detection target and the second detection target, and the detection frame coordinates of the first detection target and the second detection target using the target detection algorithm. It can be understood that if it is necessary to detect whether the hook-kicking behavior occurs, the first detection target is a person, and the second detection target is a vehicle, that is, the position information of the person and the vehicle is detected. Specifically, the target detection algorithm can detect the upper left corner coordinates and the lower right corner coordinates of the detection frames of the first detection target and the second detection target, and then obtain the position information of the first detection target and the position information of the second detection target.

[0073] In one embodiment, the target detection module 41 can also track the first detection target and the second detection target based on the identity information of the first detection target and the second detection target to obtain the motion state information of the first detection target and the motion state information of the second detection target. Specifically, the first detection target and the second detection target can be tracked using a target tracking algorithm, and the target tracking algorithm includes but is not limited to a SORT algorithm, a DeepSORT algorithm, and the like.

[0074] The judgment module 42 is used to determine whether the behavior of the first detection target is abnormal behavior based on the position information of the first detection target, the position information of the second detection target, the motion state information of the first detection target, and the motion state information of the second detection target.

[0075] In a specific embodiment, the judgment module 42 can derive the positional relationship between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target, and then determine whether the hook-kicking behavior occurs in combination with the motion state information of the first detection target and the motion state information of the second detection target, that is, determine whether the behavior of the first detection target is abnormal behavior.

[0076] In one embodiment, if the judgment module 42 determines that the behavior of the first detection target violates the preset safety regulations, the behavior of the first detection target is determined to be abnormal behavior. For example, if the judgment module 42 determines that the first detection target has a hook-kicking behavior, the behavior of the first detection target is determined to violate the preset safety regulations. For another example, if the judgment module 42 determines that the first detection target has a climbing behavior, such as climbing a tree, etc., the behavior of the first detection target is determined to violate the preset safety regulations. For another example, if the judgment module 42 determines that the first detection target has a car-stealing behavior, the behavior of the first detection target is determined to violate the preset safety regulations, without specific limitation.

[0077] The device described in this embodiment can determine whether the first detection target has abnormal behavior based on the position information and motion state information of the first detection target and the position information and motion state information of the second detection target. This method is highly efficient and reliable.

[0078] Specifically, the judgment module 42 determines the degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target, and determines whether the behavior of the first detection target is abnormal behavior based on the degree of overlap combined with the motion state information of the first detection target and the motion state information of the second detection target.

[0079] In one embodiment, the determination module 42 may calculate and determine an intersection-and-union ratio between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target, and determine the intersection-and-union ratio as the degree of overlap.

[0080] Specifically, the judgment module 42 judges whether the degree of overlap (i.e., the intersection-and-union ratio IOU) is within a preset range. If it is determined that the degree of overlap is within the preset range, the motion state information of the first detection target and the motion state information of the second detection target are compared to determine whether the motion state information of the first detection target matches the motion state information of the second detection target. If it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, it is determined that the behavior of the first detection target is abnormal behavior.

[0081] Specifically, the preset range is (IOU0, IOU1), that is, if the judgment result of the judgment module 42 is: IOU0<IOU<IOU1, the motion state information of the first detection target is further compared with the motion state information of the second detection target to determine whether the motion state information of the first detection target matches the motion state information of the second detection target. If it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, it is determined that the behavior of the first detection target is abnormal behavior. Among them, IOU0 can take the value of 0, and IOU1 can take the value according to the actual application scenario. For example, it can take the value of 0.2, 0.3, 0.5, or 0.8.

[0082] In a specific embodiment, the motion state information includes motion speed. That is, if IOU0<IOU<IOU1, the judgment module 42 further compares the motion speed of the first detection target with the motion speed of the second detection target to determine whether the motion speed of the first detection target matches the motion speed of the second detection target. If the judgment module 42 determines that the motion speed of the first detection target matches the motion speed of the second detection target, it is determined that the behavior of the first detection target is abnormal behavior. It should be noted that the second detection target is a second detection target that overlaps with the first detection target.

[0083] In a specific embodiment, the motion state information includes a motion trajectory. That is, if IOU0<IOU<IOU1, the judgment module 42 further compares the motion trajectory of the first detection target with the motion trajectory of the second detection target to determine whether the motion trajectory of the first detection target matches the motion trajectory of the second detection target. If the judgment module 42 determines that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, it is determined that the behavior of the first detection target is an abnormal behavior. It should be noted that the second detection target is a second detection target that overlaps with the first detection target.

[0084] In a specific embodiment, the motion state information includes a motion trajectory and a motion speed. That is, if IOU0<IOU<IOU1, the judgment module 42 further compares the motion trajectory of the first detection target with the motion trajectory of the second detection target, and compares the motion speed of the first detection target with the motion speed of the second detection target to determine whether the motion trajectory of the first detection target matches the motion trajectory of the second detection target, and determines whether the motion speed of the first detection target matches the motion speed of the second detection target. If the judgment module 42 determines that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, and determines that the motion speed of the first detection target matches the motion speed of the second detection target, then it is determined that the behavior of the first detection target is abnormal behavior. It should be noted that the second detection target is a second detection target that overlaps with the first detection target.

[0085] In another embodiment, if the judgment module 42 determines that the overlap degree is within the preset range, the judgment module 42 needs to further determine the position information of the first detection target, and determine the position information of the second detection target adjacent to the first detection target, and then determine whether the position relationship between the first detection target and the second detection target meets the preset position condition. If the judgment module 42 determines that the position relationship between the first detection target and the second detection target meets the preset position condition, it further determines whether the motion state information of the first detection target matches the motion state information of the second detection target. Specifically, it can be determined based on the position information of the first detection target and the second detection target that the first detection target is located to the left of the second detection target, or to the right of the second detection target.

[0086] Specifically,

[0087] The coordinates of the upper left corner of the first detection target in the first direction (X direction) can be compared with the coordinates of the upper left corner of the second detection target (the second detection target adjacent to the first detection target) in the first direction (X direction). Figure 3aAs shown, the coordinates of the first direction of the upper left corner position A of the first detection target 31 are compared with the coordinates of the first direction of the upper left corner position A1 of the second detection target 32. If the coordinates of the first direction of the upper left corner position A of the first detection target 31 are less than the coordinates of the first direction of the upper left corner position A1 of the second detection target 33, it means that the first detection target 31 is located on the left side of the second detection target 32. At this time, it is determined that the positional relationship between the first detection target and the second detection target meets the preset position condition. It is further determined whether the motion state information of the first detection target matches the motion state information of the second detection target.

[0088] In another embodiment, the coordinates of the first direction (X direction) of the lower right corner position of the first detection target are compared with the coordinates of the first direction (X direction) of the lower right corner position of the second detection target. Figure 3b As shown, the coordinates of the first direction of the lower right corner position B of the first detection target 31 are compared with the coordinates of the first direction of the lower right corner position B1 of the second detection target 32. If the coordinates of the first direction of the lower right corner position B of the first detection target 31 are greater than the coordinates of the first direction of the lower right corner position B1 of the second detection target 32, it means that the first detection target 31 is located on the right side of the second detection target. At this time, it is determined that the positional relationship between the first detection target and the second detection target meets the preset position condition. It is further determined whether the motion state information of the first detection target matches the motion state information of the second detection target.

[0089] In one embodiment, when IOU≥IOU1, or IOU≤IOU0, the corresponding target may be ignored.

[0090] Specifically, through the above-mentioned device, if it is determined that the first detection target violates the preset safety regulations, an alarm is processed based on the detection image. The method for performing the alarm processing includes: displaying the detection image on the display screen; or, displaying the detection image on the display screen with prompts such as flashing lights, vibrations, and sounds; or obtaining N frames of images adjacent to the detection image as evidence and saving them locally, where N can be 2, 3, 5, 7, etc.; or, capturing the image of the first detection target from the detection image and saving it locally or displaying it; or, capturing the image of the first detection target from the detection image, identifying the first detection target, matching and obtaining the identity information of the first detection target (such as name, ID card, etc.) and displaying it, without specific limitation.

[0091] The abnormal behavior detection device of the present application can accurately identify whether a hook-kicking behavior occurs based on the relative position and movement relationship between the person and the vehicle, avoid misjudgment, and can largely avoid false alarms caused by non-human target movements. There is no need to install additional sensors, reducing costs.

[0092] See also Figure 5, is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention, the electronic device includes a memory 202 and a processor 201 connected to each other.

[0093] The memory 202 is used to store program instructions for implementing any one of the above methods.

[0094] The processor 201 is used to execute program instructions stored in the memory 202 .

[0095] The processor 201 may also be referred to as a CPU (Central Processing Unit). The processor 201 may be an integrated circuit chip having signal processing capabilities. The processor 201 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0096] The memory 202 can be a memory stick, a TF card, etc., which can store all the information in the electronic device of the device, including the input raw data, computer programs, intermediate operation results and final operation results are all stored in the memory. It stores and retrieves information according to the location specified by the controller. With the memory, the electronic device has a memory function to ensure normal operation. The memory of the electronic device can be divided into main memory (internal memory) and auxiliary memory (external memory) according to its use, and there is also a classification method of dividing it into external memory and internal memory. External memory is usually a magnetic medium or an optical disk, etc., which can store information for a long time. Memory refers to the storage component on the motherboard, which is used to store the data and programs currently being executed, but is only used to temporarily store programs and data. If the power is turned off or the power is cut off, the data will be lost.

[0097] In the several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device implementation described above is only schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0098] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0099] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0100] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a system server, or a network device, etc.) or a processor to execute all or part of the steps of the various implementation methods of the present application.

[0101] See also Figure 6 , which is a schematic diagram of the structure of the computer-readable storage medium of the present invention. The storage medium of the present application stores a program file 203 that can implement all the above methods, wherein the program file 203 can be stored in the above storage medium in the form of a software product, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to execute all or part of the steps of each implementation method of the present application. The aforementioned storage device includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk, etc., which can store program codes, or terminal devices such as computers, servers, mobile phones, and tablets.

[0102] The above are only implementation modes of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for detecting abnormal behavior, characterized in that: include: Performing target detection on the detection image to obtain position information of a first detection target and position information of a second detection target, and obtaining motion state information of the first detection target and motion state information of the second detection target; Determining a degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; If it is determined that the overlap degree is within a preset range, further determining the position information of the first detection target, and determining the position information of the second detection target adjacent to the first detection target; If it is determined that the positional relationship between the first detection target and the second detection target meets the preset position condition, further determining whether the motion state information of the first detection target matches the motion state information of the second detection target; wherein the motion state information includes motion speed and motion trajectory; If it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, then the behavior of the first detection target is determined to be abnormal behavior, wherein the second detection target is a second detection target overlapping with the first detection target, and the abnormal behavior includes hook kicking behavior.

2. The method according to claim 1, characterized in that The determining whether the behavior of the first detection target is an abnormal behavior includes: If it is determined that the behavior of the first detection target violates the preset safety regulations, then determining that the behavior is abnormal behavior; Alarm processing is performed based on the detection image.

3. The method according to claim 1, characterized in that The step of determining the degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target includes: Determining an intersection-over-union ratio of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; The intersection-over-union ratio is determined as the overlap degree.

4. The method according to claim 3, characterized in that The step of determining the intersection-and-union ratio of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target includes: Determine an intersection of the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; Determine a union of the first detection target and the second detection target based on a size of the first detection target, a size of the second detection target, and the intersection; The intersection-to-union ratio is determined based on the intersection and the union.

5. The method according to claim 1, characterized in that The determining that the motion state information of the first detection target matches the motion state information of the second detection target includes: The motion state information includes motion speed and motion trajectory. If it is determined that the motion trajectory of the first detection target matches the motion trajectory of the second detection target, and it is determined that the motion speed of the first detection target matches the motion speed of the second detection target, then it is determined that the motion state information of the first detection target matches the motion state information of the second detection target.

6. The method according to claim 1, characterized in that The determining that the positional relationship between the first detection target and the second detection target meets a preset position condition includes: If it is determined that the coordinates of the first direction of the upper left corner position of the first detection target are smaller than the coordinates of the first direction of the upper left corner position of the second detection target, and / or it is determined that the coordinates of the first direction of the lower right corner position of the first detection target are greater than the coordinates of the first direction of the lower right corner position of the second detection target, then it is determined that the positional relationship between the first detection target and the second detection target satisfies the preset position condition.

7. The method according to any one of claims 1 to 6, characterized in that: The first detection target includes a person, and the second detection target includes a vehicle.

8. An abnormal behavior detection device, characterized in that: include: The target detection module is used to perform target detection on the detection image, obtain the position information of the first detection target and the position information of the second detection target, and obtain the motion state information of the first detection target and the motion state information of the second detection target; A judgment module, used to determine the degree of overlap between the first detection target and the second detection target based on the position information of the first detection target and the position information of the second detection target; if it is determined that the degree of overlap is within a preset range, further determine the position information of the first detection target, and determine the position information of the second detection target adjacent to the first detection target; if it is determined that the position relationship between the first detection target and the second detection target meets the preset position condition, further determine whether the motion state information of the first detection target matches the motion state information of the second detection target; wherein the motion state information includes motion speed and motion trajectory; if it is determined that the motion state information of the first detection target matches the motion state information of the second detection target, determine that the behavior of the first detection target is abnormal behavior, wherein the second detection target is the second detection target overlapping with the first detection target, and the abnormal behavior includes hooking behavior.

9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory stores program instructions, and the processor retrieves the program instructions from the memory to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A program file is stored, and the program file can be executed by a processor to implement the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Monitoring method and device for judging whether transport device carries worker

    CN107489454A

  • Human-vehicle trajectory monitoring and analyzing method, device and equipment and storage medium

    CN109635857A

  • Abnormal event detection method, device and system and electronic equipment

    CN109934075A

  • Coal mine tunnel monitor method and device and computer storage medium

    CN111425256A

  • Personnel behavior detection method and device, electronic equipment and storage medium

    CN111914656A