A method and system for identifying the behavior of people illegally bringing mobile phones into sensitive places

Through computer vision technology and neural network model, surveillance video is processed, combined with the analysis of overlapping area of ​​pedestrian trajectory and mobile phone storage area, the problem of low efficiency and poor binding force in identifying personnel illegally carrying mobile phones into sensitive places in the existing technology is solved, and the problem of poor binding force is achieved. High accuracy and efficient identification of violations.

CN114299448BActive Publication Date: 2024-12-13NO 719 RES INST CHINA SHIPBUILDING IND
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111616228.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-12-27
Publication Date
2024-12-13
Estimated Expiration
2041-12-27

AI Technical Summary

Technical Problem

The existing technology is inefficient when identifying that people carry mobile phones into sensitive places in violation of regulations, it has low efficiency, poor binding force, poor supervision effect, and high error and missed detection rates, which cannot be effectively applied.

Method used

Computer vision technology is used to process surveillance videos, judge whether to enter sensitive places through pedestrian trajectory, combine mobile phone detection and human posture analysis, and use neural network models to perform personnel positioning and target tracking, determine whether to carry a mobile phone, and use the hand area and the mobile phone storage area. The overlapping area is determined whether to store the mobile phone, and improve recognition accuracy and efficiency.

Benefits of technology

It improves identification accuracy and efficiency, reduces the occupation of human resources, ensures high accuracy, effectively solves the problem of investigating and handling violations, and reduces the false detection rate and missed detection rate.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114299448B_ABST
    Figure CN114299448B_ABST
Patent Text Reader

Abstract

The present invention provides a method and system for identifying the behavior of people illegally carrying mobile phones into sensitive places, which belongs to the field of security monitoring. The method is specifically: pre-processing the surveillance video of sensitive places to obtain images to be detected; locating the people in the images to be detected and forming a single person image set by segmentation; using pedestrian trajectories to determine whether they have entered sensitive places, if so, proceed to the next step, if not, end monitoring; judging whether the mobile phone is brought into sensitive places according to the mobile phone detection results, if so, record the violation; if not, proceed to the next step; judging whether the mobile phone is stored using the overlapping area of ​​the hand area and the mobile phone storage area, if so, judge that there is no violation, if not, record the violation; repeat the above steps until all personnel are monitored. The present invention can ensure a high degree of accuracy while simplifying the judgment process, and effectively solves the problem that it is difficult to detect people illegally carrying mobile phones into sensitive places.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of security monitoring, and more specifically, relates to a method and system for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place. Background Art

[0002] In order to ensure information security and equipment security, some sensitive places usually prohibit the use of mobile phones, so it is necessary to monitor whether people illegally bring mobile phones into sensitive places. At present, the methods for identifying people who illegally bring mobile phones into sensitive places mainly include: on-site monitoring, intelligent mobile phone storage cabinets and remote viewing of video surveillance.

[0003] Among them, on-site surveillance mainly relies on inspectors to supervise the storage of mobile phones. This method has high recognition accuracy and obvious effects, but it requires full-time personnel to be arranged at each entrance and exit for a long time. It is extremely inefficient and occupies a large amount of human resources. It is not suitable for large-scale promotion. At the same time, there is also the disadvantage of being unable to conduct post-verification. For intelligent mobile phone storage cabinets, CN111364875A discloses an intelligent safe and intelligent management system, and CN211950099U discloses an intelligent safe with face recognition function, but this method lacks strong constraints and has the problem of poor supervision effect. In remote video monitoring, the monitor can browse multiple videos in the monitoring room at the same time, but in the face of massive videos, the monitor is prone to visual fatigue and it is difficult to maintain a high degree of concentration for a long time, and there is a problem of unstable recognition accuracy. Although there is a mobile phone detection model in the prior art, which can detect whether a mobile phone appears in the monitoring screen to determine whether the personnel have violated the regulations and collected data into sensitive places, the technology has a high false detection rate and missed detection rate, and cannot be used in practice. Summary of the invention

[0004] In view of the defects of the prior art, the purpose of the present invention is to provide a method and system for identifying the behavior of people illegally bringing mobile phones into sensitive places, aiming to solve the problems of low efficiency, poor binding force and poor supervision effect of the existing determination methods.

[0005] To achieve the above object, the present invention provides a method for identifying the behavior of a person illegally carrying a mobile phone into a sensitive place, the method comprising the following steps:

[0006] S1 pre-processes the surveillance video of sensitive places to obtain the image to be detected;

[0007] S2 performs person positioning on the image to be detected obtained in step S1, and divides the image to be detected according to the result of the person positioning to form a single person image set;

[0008] S3 performs target tracking, mobile phone detection, and human key point detection on the current person's single-person image set to obtain pedestrian trajectories, mobile phone detection results, and human body postures;

[0009] S4 uses the pedestrian trajectory to determine whether the current person has entered a sensitive place. If so, proceed to step S5. If not, end the monitoring of the current person.

[0010] S5 selects the mobile phone detection results of the entrance and exit of the sensitive place according to the pedestrian trajectory to determine whether the current person brings a mobile phone into the sensitive place. If so, record the violation; if not, proceed to step S6;

[0011] S6 selects the human body posture at the mobile phone storage cabinet according to the pedestrian trajectory, and uses the overlapping area of ​​the hand area and the mobile phone storage area to determine whether the mobile phone is stored. If so, it is determined that the current person has no violation behavior, and if not, the violation behavior is recorded;

[0012] S7 repeats steps S3 to S6 until all personnel are monitored.

[0013] As a further preferred embodiment, in step S1, the monitoring video is decoded and transcoded to convert the video stream into a single-frame image, and a preset number of single-frame images are extracted by frame extraction technology to obtain the image to be detected.

[0014] As a further preferred embodiment, in step S2, a neural network model is used to locate the person in the image to be detected, and a matrix as shown below is output:

[0015]

[0016] In the formula, (x i1 ,y i1 ) is the upper left corner coordinate of the i-th person on the image to be detected, (x i2 ,y i2 ) is the lower right corner coordinate of the i-th person on the image to be detected. The i-th person is completely inside the rectangular bounding box formed by these two points. i ∈[0,1] represents the confidence of the i-th person. i Those whose confidence level is lower than the confidence threshold are directly eliminated.

[0017] As further preferred, in step S2, the confidence threshold is 0.3-0.4.

[0018] As a further preferred embodiment, in step S3, the single person image set is sent to a mobile phone detection model to detect whether there is a mobile phone in the image.

[0019] As a further preferred embodiment, in step S3, the single person image set is sent to a human key point detection model to obtain a human body posture.

[0020] As a further preferred embodiment, in step S5, the mobile phone location coordinates and confidence level are recorded while recording the violation.

[0021] As further preferred, step S6 includes the following sub-steps:

[0022] S61 determines a wrist key point according to the human body posture at the mobile phone storage cabinet, and makes a square with a side length of 50 to 150 pixels with the wrist key point as the center, which serves as the hand area;

[0023] S62 calculates the overlapping area of ​​the hand area and the mobile phone storage area;

[0024] S63 determines whether the overlapping area is greater than an area threshold. If so, it is determined that the current person has stored the mobile phone and there is no violation. If not, the violation is recorded.

[0025] As a further preferred embodiment, in step S63, the area threshold is 200 pixels to 300 pixels.

[0026] According to another aspect of the present invention, a system for identifying the behavior of a person illegally carrying a mobile phone into a sensitive place is provided, the system comprising a monitoring device, a video processing device and an alarm device, the monitoring device being used to obtain monitoring videos of sensitive places; the video processing device being used to implement the above-mentioned method for identifying the behavior of a person illegally carrying a mobile phone into a sensitive place; and the alarm device being used to sound an alarm when an illegal behavior is identified.

[0027] In general, the above technical solution conceived by the present invention has the following beneficial effects compared with the prior art:

[0028] 1. The present invention uses computer vision technology to process surveillance videos. First, pedestrian trajectories are used to determine whether the current person has entered a sensitive place, thereby simplifying the judgment process. Only the person who has entered the sensitive place is identified. Then, whether there is a mobile phone in the images of the entrance and exit of the sensitive place is used to preliminarily determine whether there is a violation. If there is no mobile phone, the overlapping area of ​​the hand area and the mobile phone storage area at the mobile phone storage cabinet is further used to determine whether the action of storing the mobile phone has been performed. This can more accurately determine whether the person has violated the rules. While simplifying the judgment process, it can ensure high accuracy, effectively solving the problem that it is difficult to detect and investigate people who illegally bring mobile phones into sensitive places, and greatly improving the recognition accuracy while effectively saving manpower and material resources;

[0029] 2. In particular, the present invention uses a neural network model to locate people in the image to be detected to obtain a single person image set, which can improve the judgment accuracy and the judgment efficiency;

[0030] 3. In addition, when judging whether to store a mobile phone, the present invention defines the hand area with the key point of the wrist as the center, and judges whether there is an action of storing the mobile phone through the overlapping area of ​​the hand area and the storage area, which simplifies the judgment process and ensures a high accuracy rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 It is a flow chart of a method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] like Figure 1 As shown, the present invention provides a method for identifying the behavior of a person illegally carrying a mobile phone into a sensitive place, and the method comprises the following steps:

[0034] S1 pre-processes the surveillance video of sensitive places. The surveillance video can cover the entrance and exit of sensitive places and the mobile phone storage cabinet. The surveillance video can be obtained in real time or cached historical surveillance data. During the pre-processing process, the video stream is converted into a single-frame image through decoding and transcoding, and a picture is extracted every certain number of frames through the frame extraction technology, so as to obtain a preset number of single-frame images as the images to be detected;

[0035] S2 performs person positioning on the image to be detected obtained in step S1, and divides the image to be detected according to the result of person positioning to form a single person image set;

[0036] S3 performs target tracking, mobile phone detection, and human key point detection on the current person's single-person image set to obtain pedestrian trajectories, mobile phone detection results, and human body postures;

[0037] S4 uses the pedestrian trajectory to determine whether the current person has entered a sensitive place. If so, proceed to step S5. If not, end the monitoring of the current person.

[0038] S5 selects the mobile phone detection results at the entrance and exit of sensitive places according to the pedestrian trajectory to determine whether the current person brings a mobile phone into the sensitive place. If so, the violation is recorded, and the time, camera ID, violation place, current video frame number, violator, mobile phone location coordinates and confidence are recorded; if not, proceed to step S6;

[0039] S6 selects the human body posture at the mobile phone storage cabinet according to the pedestrian trajectory, and uses the overlapping area of ​​the hand area and the mobile phone storage area to determine whether the mobile phone is stored. If so, it determines that the current person has no violation behavior. If not, it records the violation behavior, and records the time, camera ID, violation location, current video frame number, and violator;

[0040] S7 repeats steps S3 to S6 until all personnel are monitored.

[0041] Furthermore, in step S2, the neural network model is used to locate the person in the image to be detected, and the matrix shown in the following formula is output:

[0042]

[0043] In the formula, each row represents a person in the image to be detected, (x i1 ,y i1 ) is the upper left corner coordinate of the i-th person on the image to be detected, (x i2 ,y i2 ) is the lower right corner coordinate of the i-th person on the image to be detected. The i-th person is completely inside the rectangular bounding box formed by these two points. i ∈[0,1] represents the confidence of the i-th person. i Personnel with a confidence level lower than the confidence threshold are directly eliminated, wherein the confidence threshold is preferably 0.3-0.4, so as to reduce the amount of calculation and improve the processing speed; the image to be detected is segmented according to the result of personnel positioning to form a single person image set.

[0044] Further, in step S3, a neural network model is used for target tracking; and the single-person image set is sent to a mobile phone detection model to detect whether there is a mobile phone in the image, wherein the mobile phone detection model can be a target detection model in the prior art, which can accurately distinguish whether there is a specific target of a mobile phone in the detection image through training; at the same time, the single-person image set is sent to a human key point detection model to obtain the human body posture.

[0045] Further, step S6 includes the following sub-steps:

[0046] S61 determines the wrist key point according to the human body posture at the mobile phone storage cabinet, and makes a square with a side length of 50 to 150 pixels with the wrist key point as the center, which is used as the hand area; at the same time, the remaining key points in the human body posture can be used as auxiliary basis to determine the position relationship between the human body and the mobile phone storage cabinet, so as to further improve the accuracy of hand positioning;

[0047] S62 calculates the overlapping area of ​​the hand area and the mobile phone storage area;

[0048] S63 determines whether the overlapping area is greater than the area threshold. If so, it means that the current person interacts with the mobile phone storage area, and it is determined that the current person has no violation. If not, it means that the current person has not stored the mobile phone, and the violation is recorded.

[0049] Further, in step S63, the area threshold is 200 pixels to 300 pixels, so as to ensure the accuracy of the judgment.

[0050] According to another aspect of the present invention, a system for identifying the behavior of a person illegally carrying a mobile phone into a sensitive place is provided, the system comprising a monitoring device, a video processing device and an alarm device, the monitoring device is used to obtain monitoring videos of sensitive places, and can cover the entrance to the sensitive place, the exit of the sensitive place and the mobile phone storage cabinet, wherein the mobile phone storage cabinet is used to store mobile phones before entering the sensitive place, and uses existing ordinary mobile phone storage cabinets, which have the advantages of low cost and easy layout; the video processing device is used to implement the above-mentioned method for identifying the behavior of a person illegally carrying a mobile phone into a sensitive place, and can perform target tracking, mobile phone detection and human body key point detection; the alarm device is used to alarm when an illegal behavior is identified, and can push information to the person in charge.

[0051] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place, characterized in that: The method comprises the following steps: S1 pre-processes the surveillance video of sensitive places to obtain the image to be detected; S2 performs person positioning on the image to be detected obtained in step S1, and divides the image to be detected according to the result of the person positioning to form a single person image set; S3 performs target tracking, mobile phone detection, and human key point detection on the current person's single-person image set to obtain pedestrian trajectories, mobile phone detection results, and human body postures; S4 uses the pedestrian trajectory to determine whether the current person has entered a sensitive place. If so, proceed to step S5. If not, end the monitoring of the current person. S5 selects the mobile phone detection results of the entrance and exit of the sensitive place according to the pedestrian trajectory to determine whether the current person brings a mobile phone into the sensitive place. If so, record the violation; if not, proceed to step S6; S6 selects the human body posture at the mobile phone storage cabinet according to the pedestrian trajectory, and uses the overlapping area of ​​the hand area and the mobile phone storage area to determine whether the mobile phone is stored. If so, it is determined that the current person has no violation behavior. If not, the violation behavior is recorded, specifically: S61 determines a wrist key point according to the human body posture at the mobile phone storage cabinet, and makes a square with a side length of 50 to 150 pixels with the wrist key point as the center, which serves as the hand area; S62 calculates the overlapping area of ​​the hand area and the mobile phone storage area; S63 determines whether the overlapping area is greater than an area threshold. If so, it is determined that the current person has stored the mobile phone and there is no violation. If not, the violation is recorded. S7 repeats steps S3 to S6 until all personnel are monitored.

2. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 1, characterized in that: In step S1, the monitoring video is decoded and transcoded to convert the video stream into a single-frame image, and a preset number of single-frame images are extracted by frame extraction technology to obtain an image to be detected.

3. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 1, characterized in that: In step S2, the neural network model is used to locate the person in the image to be detected, and the following matrix is ​​output: In the formula, For the The upper left corner coordinates of a person on the image to be detected, For the The lower right corner coordinates of the person on the image to be detected, The person is completely inside the rectangular bounding box formed by these two points. Indicates The confidence level of each person is Those whose confidence level is lower than the confidence threshold are directly eliminated.

4. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 3, characterized in that: In step S2, the confidence threshold is 0.3-0.

4.

5. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 1, characterized in that: In step S3, the single person image set is fed into a mobile phone detection model to detect whether there is a mobile phone in the image.

6. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 1, characterized in that: In step S3, the single person image set is sent to a human key point detection model to obtain the human posture.

7. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 1, characterized in that: In step S5, the mobile phone location coordinates and confidence level are recorded while recording the violation.

8. The method for identifying the behavior of a person illegally bringing a mobile phone into a sensitive place as claimed in claim 1, characterized in that: In step S63, the area threshold is 200 pixels to 300 pixels.

9. A system for identifying the behavior of people illegally bringing mobile phones into sensitive places, characterized in that: The system includes a monitoring device, a video processing device and an alarm device. The monitoring device is used to obtain monitoring videos of sensitive places; the video processing device is used to implement the method for identifying the behavior of people illegally carrying mobile phones into sensitive places as described in any one of claims 1 to 8; and the alarm device is used to sound an alarm when an illegal behavior is identified.