System and method for detecting personnel intrusion based on multi-camera view cross

Abstract

The invention discloses a system and a method for detecting personnel intrusion based on multi-camera view crossing, and relates to the technical field of workshop safety environment monitoring, a visual perception module comprises a plurality of image acquisition devices arranged in a workshop, and realizes comprehensive coverage of pedestrian road occupying positions in the workshop; the personnel intrusion detection algorithm module carries out real-time processing on image information transmitted by the visual perception module, realizes identification and prediction on intrusion behaviors of personnel in a workshop, transmits an output prediction target to the alarm and response module, and carries out online alarm immediately through the switching value wireless unvarnished transmission module when the intrusion behaviors exist. An alarm signal is transmitted to the integration and control module so as to control emergency stop of the truss; the integration and control module is used for being responsible for overall integration and configuration work of the system. The system has the advantages of high recognition precision, complete view scene coverage, online alarm and the like, and provides powerful guarantee for safe production of workshops.

Description

Technical Field

[0001] The present invention belongs to the technical field of workshop environment monitoring, and in particular relates to a system and method for detecting human intrusion based on cross-viewing of multiple cameras. Background Art

[0002] With the rapid development of industrial automation and intelligent manufacturing, the demand for security monitoring in production environments such as copper plating automation workshops is increasing. The traditional single-camera monitoring system has obvious blind spots and cannot fully cover every corner and potential intrusion path in the workshop. This has led to the inability to effectively monitor some key areas, increasing safety hazards; due to interference from factors such as ambient light changes, object occlusion, and camera jitter, traditional monitoring systems often have a high false alarm rate. This not only wastes human and material resources, but may also cause staff to ignore alarm signals due to frequent false alarms; after detecting intrusion behavior, traditional monitoring systems usually require manual intervention for confirmation and processing, which leads to a lag in response speed. In an emergency, this lag may have serious consequences; in scenarios monitored by multiple cameras, how to achieve cross-camera target tracking is a difficult problem. Traditional methods often rely on complex image matching algorithms and a large amount of computing resources, resulting in unsatisfactory tracking effects and poor real-time performance. Summary of the invention

[0003] In view of the fact that the traditional manufacturing industry relies relatively heavily on human control for safety management, cannot accurately and timely make early predictions, and has defects and problems such as blind spots, high false alarm rate, and slow response speed, the present invention provides a system and method for detecting personnel intrusion based on multi-camera cross-view. The system arranges multiple high-definition cameras in the workshop to form cross-view coverage, and uses advanced image processing technology and target detection algorithms to achieve real-time monitoring of personnel activities in the workshop and intrusion detection. The system can not only effectively reduce blind spots in the field of vision and improve the accuracy and reliability of detection, but also quickly trigger an alarm mechanism when an intrusion is detected to ensure safe production in the workshop.

[0004] The solution adopted by the present invention to solve its technical problems is: a system for detecting human intrusion based on the cross-view of multiple cameras, including a visual perception module, a human intrusion detection algorithm module, an alarm and response module, a data transmission module, a switch quantity wireless transparent transmission module and an integration and control module; the visual perception module includes a plurality of image acquisition devices arranged in a workshop, and the cross-view setting is realized by point selection, so as to achieve comprehensive coverage of the pedestrian road positions in the workshop; the human intrusion detection algorithm module integrates the yolov7+ improved CRNN algorithm, and processes the image information transmitted by the visual perception module in real time, so as to realize the recognition and prediction of human intrusion behavior in the workshop, and transmit the output prediction target to the alarm and response module, and when there is intrusion behavior, an online alarm is immediately issued through the switch quantity wireless transparent transmission module, and the alarm signal is transmitted to the integration and control module to control the emergency stop of the truss; the integration and control module is used to be responsible for the overall integration and configuration of the system, so as to adjust the system parameters according to the actual situation of the workshop.

[0005] Furthermore, the image acquisition device includes four cameras arranged and managed in the copper plating automation workshop. The four cameras are respectively installed on tempered columns inside the copper plating automation workshop with an interval of 5 meters. Two of the cameras face the workshop trusses, and the other two face the personnel detection area.

[0006] Furthermore, the visual perception module transmits the collected target image information to the yolov7+ improved CRNN node in the form of SEI information table using the data transmission module and starts the YOLOV7 target detection algorithm for prediction, and outputs the predicted target for online alarm through the switch quantity wireless transparent transmission module.

[0007] Furthermore, the image input to the improved CRNN network is resized to 640x640, input into the backbone network, and then outputs three layers of feature maps of different sizes through the head layer network, and outputs the prediction results through Rep and Conv.

[0008] Furthermore, the backbone of the improved CRNN is actually a PAFPN structure, and the specific contents are as follows: first, for the 32-fold downsampled feature map C5 finally output by the backbone, the number of channels is changed from 1024 to 512 after SPPCSP; first, top down is fused with C4 and C3 to obtain P3, P4 and P5; then bottom-up is fused with P4 and P5; CNSM is introduced into the backbone module to perform convolutional networks in space and time, and ELAN is replaced by DWConv; the definition of the spatiotemporal network formula is to divide the mine tunnel into i*j regions, and the regional set of the network is defined as L={l1, l2, ..., l i*j}; At the same time, divide the time T into different time intervals, and define the interval T = {t s1 , t s2 ,...t sT}, use a two-dimensional array (a, b) to define the space-time coordinates; where a represents time and b represents area, locate the starting point of the journey of any object: s = (a s , b s ), e=(a e , , b e );

[0009] (1) Initial field of view flow

[0010] In a certain time interval t, the initial visual flow of each area End the visual flow

[0011] (2) Visual field flow

[0012] The visual field flow describes the dynamic interaction relationship between regions. At time t, the visual flow value from area i to area j is:

[0013]

[0014] Furthermore, the alarm and response module includes an audible and visual alarm arranged on a pedestrian pillar in a loading area of ​​a copper plating automation workshop, and the switch quantity wireless transparent transmission module includes a wireless receiving end and a wireless transmitting end, wherein the wireless transmitting end is installed in a control cabinet of the copper plating automation workshop, and the receiving end is installed next to the audible and visual alarm; the input contacts of the switch quantity wireless transparent transmission module are connected to the corresponding contacts of the controller PLC that controls the start and stop of the truss in the integration and control module, and when there is an intrusion, an online alarm is immediately issued through the switch quantity wireless transparent transmission module, and the alarm signal is transmitted to the integration and control module to control the emergency stop of the truss.

[0015] The present invention also provides a method for detecting human intrusion based on the intersection of multiple camera fields of view, comprising the following steps:

[0016] S1. Data collection: The visual perception module uses the camera to capture the image information in the workshop in real time, including personnel activities and rack status, and transmits the image information to the personnel intrusion detection algorithm module;

[0017] S2. Personnel intrusion detection: Real-time processing of image information transmitted by the visual perception module to accurately identify intrusion behavior of personnel in the workshop; once a person is detected entering the preset prohibited area, the system will immediately trigger the alarm mechanism;

[0018] S3, alarm and response: When the intrusion detection algorithm module identifies the intrusion behavior, the information is transmitted to the alarm and response module through the data transmission module. The module immediately activates the alarm mechanism and sends the alarm signal to the PLC controller that controls the start and stop of the truss through the wireless transparent transmission module to realize the emergency stop of the truss;

[0019] S4. Data recording and tracing: The integration and control module will take screenshots and save videos of intrusion events, and record the image information and behavior trajectory of the intruders for subsequent security analysis and tracing.

[0020] Compared with the prior art, the present invention has the following beneficial effects:

[0021] The system and method for detecting human intrusion based on cross-view of multiple cameras provided by the present invention realizes accurate detection and immediate response to human intrusion behavior in the copper plating automation workshop through the coordinated work of multiple modules such as visual perception, human intrusion detection, alarm and response, and system integration and configuration, which provides a strong guarantee for the safe production of the workshop and also includes the following advantages:

[0022] 1. High-precision recognition: Using the advanced YOLOv7 target detection algorithm, this method can quickly and accurately identify human targets in complex workshop environments. YOLOv7 has significantly improved speed and accuracy and is suitable for real-time monitoring systems. Combined with the improved CRNN algorithm, it further improves the ability to accurately predict and classify target locations, reducing false positives and false negatives.

[0023] 2. Full field of view coverage: Through the multi-camera layout and cross-field setting, this method can achieve full coverage of every corner and potential intrusion path in the copper plating automation workshop. This layout ensures no blind spot monitoring and improves the overall security of the system.

[0024] 3. Real-time response: Once a person intrusion is detected, the system can immediately trigger the alarm mechanism and send the alarm signal to the PLC controller that controls the start and stop of the rack through the wireless transparent transmission module. This real-time response capability helps to take quick measures to prevent potential safety accidents.

[0025] 4. High reliability: This method combines a variety of technical means, including high-definition imaging, target detection, wireless transparent transmission, etc. Each link has been carefully designed and optimized to ensure the overall reliability and stability of the system.

[0026] 5. Easy to expand and maintain: The system adopts modular design, and each module is relatively independent, which is convenient for expansion and upgrading according to actual needs. At the same time, the maintenance of the system is relatively simple, and only regular inspection and maintenance of each module is required.

[0027] 6. Rich data recording and tracing: The system can record screenshots and videos of intrusion events, providing strong support for subsequent security analysis and tracing. These data records not only help companies understand the safety status in the workshop, but also provide important reference for improving safety measures.

[0028] 7. Intelligent management: This method realizes the intelligent monitoring and management of personnel activities in the workshop, reducing the workload and cost of manual inspections. At the same time, through data analysis and mining, it can also provide enterprises with more in-depth safety management suggestions and decision-making support. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 A workflow diagram of a method for detecting human intrusion based on cross-views of multiple cameras;

[0030] Figure 2 This is a schematic diagram of the installation positions of various modules in a system for detecting human intrusion based on cross-views of multiple cameras;

[0031] Figure 3 Improved network architecture diagram for the algorithm of the present invention;

[0032] Figure 4 The CRNN network structure diagram is improved for the present invention.

[0033] In the figure: 1-sound and light alarm; 2-truss; 3-camera; 4-intrusion detection area; 5-receiving end of switch quantity wireless transparent transmission module; 6-transmitting end of switch quantity wireless transparent transmission module. DETAILED DESCRIPTION

[0034] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0035] Embodiment 1:

[0036] The present embodiment provides a system for detecting human intrusion based on the intersection of multiple camera fields of view. The system is intended for the safety control of the start and stop of the trusses in the copper plating automation workshop. By combining the intersection of multiple camera fields of view with advanced artificial intelligence algorithms (yolov7+improved CRNN), accurate detection and immediate response to human intrusion in the workshop can be achieved. The system mainly includes a visual perception module, a human intrusion detection algorithm module, an alarm and response module, a data transmission module, a switch quantity wireless transparent transmission module, and an integration and control module.

[0037] The visual perception module includes multiple image acquisition devices arranged in the workshop. In this embodiment, multiple cameras 3 are arranged and managed in the copper plating automation workshop, and the best camera installation position is selected according to the workshop layout and personnel activity rules to ensure that the cross-view area can effectively cover all potential personnel intrusion paths. Figure 2As shown, in the actual application of this embodiment, a total of 4 high-definition cameras are installed in the workshop to ensure the clarity and accuracy of the image information. The four cameras are installed on tempered columns 5 meters apart inside the copper plating automation workshop, two of which are facing the workshop trusses, and the other two are facing the personnel detection area. In this way, through reasonable point selection and cross-view setting, comprehensive coverage of pedestrian positions in the workshop is achieved. This module can capture image information in the workshop in real time, including personnel activities, rack status, etc., to provide basic data for subsequent personnel intrusion detection.

[0038] The personnel intrusion detection algorithm module is based on the Ubuntu 20.04 system and integrates the yolov7+ improved CRNN algorithm, which is used to process the image information transmitted by the visual perception module in real time, and realize the accurate recognition of the intrusion behavior of personnel in the workshop. Once a person is detected to enter the preset prohibited area, the intrusion detection area 4 is set, and the system will immediately trigger the alarm mechanism. Furthermore, the module uses the efficiency and accuracy of yolov7 to quickly identify and locate pedestrian targets in the image, and on the basis of yolov7, the module also introduces an improved CRNN (Convolutional Recurrent Neural Network) structure to make more detailed predictions and classifications of the target position. The module can improve the recognition accuracy and robustness of the algorithm in complex scenes by adjusting the network structure and parameters.

[0039] The visual perception module transmits the collected target image information to the yolov7+ improved CRNN node in the form of SEI information table using the data transmission module and starts the YOLOV7 target detection algorithm for prediction, and outputs the predicted target for online alarm through the switch wireless transparent transmission module.

[0040] Specifically, the image input to the improved CRNN network is resized to 640x640, input to the backbone network, and then outputs three layers of feature maps of different sizes through the head layer network, and outputs the prediction results through Rep and Conv. In this embodiment, the data set takes coco+ actual samples as an example, and the output is 1 category, and then each output (x, y, w, h, o) is the coordinate position and the background before and after, 3 refers to the number of anchors, so the output of each layer is (1+5)x3=18, and then multiplied by the size of the feature map is the final output. Backbone has 50 layers, Figure 3 Only the key layers are marked in the figure. Backbone is constructed by 4 convolutional layers. Figure 3As shown in the figure, CBS is mainly composed of Conv+BN+SiLU. Different colors are used in the figure to represent different sizes and strides. For example, (3,2) means that the convolution kernel size is 3 and the stride is 2. The configuration in config is as follows Figure 3 shown.

[0041] The Backbone of the above-mentioned improved CRNN is actually a PAFPN structure. The improved CRNN network structure is as follows Figure 4 As shown. The specific content is: First, for the 32-fold downsampled feature map C5 finally output by the backbone, and then after SPPCSP, the number of channels changes from 1024 to 512. First, fuse top down with C4 and C3 to obtain P3, P4 and P5; then fuse bottom-up with P4 and P5. In the Backbone module, CNSM will be introduced to perform convolutional networks in space and time, and ELAN will be replaced by DWConv. The definition of the spatiotemporal network formula is to divide the mine tunnel into i*j regions, and define the network region set as L={l1, l2,...., l i*j}. At the same time, the time T is divided into different time intervals, and the interval definition T={t s1 , t s2 ,...t sT}, use a two-dimensional array (a, b) to define the space-time coordinates. a represents time, b represents area, and locates the starting point of any object's journey: s = (a s , b s ), e=(a e , b e ).

[0042] (1) Initial field of view flow

[0043] In a certain time interval t, the initial visual flow of each area End the visual flow

[0044]

[0045] (2) Visual field flow

[0046] The visual field flow describes the dynamic interaction relationship between regions. At time t, the visual field flow value flows from area i to area j.

[0047]

[0048] In the above formula: s is the starting point coordinate, e is the end point coordinate, M is the coordinate domain constant, a e is the end time, t is the time domain, s tis the starting time, b s is the starting time of the area, b e is the end time of the area, l i , l j are area sets respectively, i and j are area block addresses respectively.

[0049] When the above-mentioned personnel intrusion detection algorithm module identifies the intrusion behavior of personnel, the alarm and response module will immediately start the alarm mechanism, and send the alarm signal to the PLC controller that controls the start and stop of the rack through the wireless transparent transmission module, so as to realize the emergency stop of the rack or corresponding safety measures. At the same time, the module will also record the screenshots and videos of the intrusion event to provide a basis for subsequent safety analysis and tracing. The alarm and response module includes an audible and visual alarm configured on the pedestrian column in the loading area of ​​the copper plating automation workshop, and the switch quantity wireless transparent transmission module includes a wireless receiving end 5 and a wireless transmitting end 6, wherein the wireless transmitting end is installed in the control cabinet of the copper plating automation workshop, and the receiving end is installed next to the audible and visual alarm 1. For the acquisition of the start and stop of the truss in the copper plating automation workshop, the input contact of the switch quantity wireless transparent transmission module is connected to the corresponding contact of the controller PLC that controls the start and stop of the truss in the integration and control module. When there is an intrusion behavior, the switch quantity wireless transparent transmission module will immediately make an online alarm, and transmit the alarm signal to the integration and control module to control the emergency stop of the truss 2.

[0050] The integration and control module is responsible for the overall integration and configuration of the system, including the installation and debugging of the camera, the optimization and adjustment of the algorithm parameters, the setting of the alarm threshold, etc., so as to form a complete and efficient personnel intrusion system. Through this module, users can easily adjust the system parameters according to the actual situation of the workshop to ensure the best operating state of the system, and improve the recognition accuracy and response speed of the system by optimizing and adjusting the algorithm parameters.

[0051] The present invention provides a system for detecting human intrusion based on the cross-viewing of multiple cameras. Through the collaborative work of multiple modules such as visual perception, human intrusion detection, alarm and response, and system integration and configuration, the system realizes accurate detection and immediate response to human intrusion in an automated copper plating workshop. The system has the advantages of high recognition accuracy, full coverage of visual scene, and online alarm, thereby providing a strong guarantee for safe production in the workshop.

[0052] Embodiment 2:

[0053] On the basis of Example 1, Figure 1 and Figure 2 As shown, this embodiment provides a method for detecting intrusion based on the intersection of multiple camera fields of view based on the system for detecting intrusion based on the intersection of multiple camera fields of view provided in the above embodiment 1, which specifically includes the following steps:

[0054] S1. Data Collection

[0055] In the copper plating automation workshop, multiple high-definition cameras are reasonably arranged according to the layout of the workshop and the rules of personnel activities. The module selects and adjusts the camera points according to the actual situation of the workshop to ensure that the field of view of each camera can be cross-covered to form a monitoring network without blind spots; the visual perception module uses the camera to capture the image information in the workshop in real time, including personnel activities, rack status, etc., and transmits the image information to the personnel intrusion detection algorithm module;

[0056] S2. Personnel intrusion detection

[0057] The intrusion detection algorithm module detects cross-view intrusion based on yolov7+ improved CRNN, and gives the location and size information of the target, combined with the cross-view information of multiple cameras, to further analyze and judge the detected target. If the target appears in the cross-view area of ​​multiple cameras, and its behavior conforms to the preset intrusion mode (such as approaching a prohibited area, climbing over a fence, etc.), it is judged as a human intrusion, and an online warning is issued, a screenshot is recorded, and a 10S video is saved;

[0058] S3. Alarm and response

[0059] Once human intrusion is detected, the system will immediately trigger the alarm mechanism to quickly notify relevant personnel for processing. At the same time, the system will send the alarm signal to the integration and control module through the switch quantity wireless transmission module, which is used to control the start and stop of the truss through the PLC controller; the PLC controller will execute corresponding safety measures based on the received signal, such as emergency stop of the truss operation.

[0060] S4. Data recording and traceability

[0061] The integration and control module will take screenshots and save 10s video of the intrusion event, and record the image information and behavior trajectory of the intruder. These records will serve as an important basis for subsequent security analysis and tracing.

[0062] The method for detecting human intrusion based on cross-view of multiple cameras provided in this embodiment realizes accurate detection and immediate response to human intrusion in the copper plating automation workshop through the coordinated work of multiple links such as camera layout, visual perception, human intrusion detection, alarm and response, and data recording and tracing. This system not only improves the safety of the workshop, but also provides enterprises with more intelligent and efficient security management means.

[0063] The above description is only a preferred embodiment of the present invention and does not limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A system for detecting human intrusion based on the intersection of multiple camera fields of view, characterized in that: It includes a visual perception module, a personnel intrusion detection algorithm module, an alarm and response module, a data transmission module, a switch quantity wireless transparent transmission module and an integration and control module; the visual perception module includes multiple image acquisition devices arranged in the workshop, and the field of view cross setting is realized through point selection to achieve comprehensive coverage of the pedestrian road positions in the workshop; the personnel intrusion detection algorithm module integrates the yolov7+ improved CRNN algorithm, and processes the image information transmitted by the visual perception module in real time to realize the recognition and prediction of personnel intrusion behavior in the workshop, and transmits the output prediction target to the alarm and response module. When there is intrusion behavior, an online alarm is immediately issued through the switch quantity wireless transparent transmission module, and the alarm signal is transmitted to the integration and control module to control the emergency stop of the truss; the integration and control module is responsible for the overall integration and configuration of the system to adjust the system parameters according to the actual situation of the workshop.

2. The system for detecting human intrusion based on cross-viewing of multiple cameras according to claim 1 is characterized in that: The image acquisition device includes four cameras arranged and managed in the copper plating automation workshop. The four cameras are respectively installed on tempered columns inside the copper plating automation workshop with an interval of 5 meters. Two of the four cameras face the workshop truss and the other two face the personnel detection area.

3. The system for detecting human intrusion based on cross-viewing of multiple cameras according to claim 1 is characterized in that: The visual perception module transmits the collected target image information to the yolov7+ improved CRNN node in the form of SEI information table using the data transmission module and starts the YOLOV7 target detection algorithm for prediction, and outputs the predicted target for online alarm through the switch wireless transparent transmission module.

4. The system for detecting human intrusion based on cross-viewing of multiple cameras according to claim 3 is characterized in that: The image input to the improved CRNN network is resized to 640x640, input into the backbone network, and then outputs three layers of feature maps of different sizes through the head layer network, and outputs the prediction results through Rep and Conv.

5. The system for detecting human intrusion based on cross-viewing of multiple cameras according to claim 4 is characterized in that: The backbone of the improved CRNN is actually a PAFPN structure, and the specific contents are as follows: first, for the 32-fold downsampled feature map C5 finally output by the backbone, the number of channels is changed from 1024 to 512 after SPPCSP; first, top-down is fused with C4 and C3 to obtain P3, P4 and P5; then bottom-up is fused with P4 and P5; CNSM is introduced into the backbone module to perform convolutional networks in space and time, and ELAN is replaced by DWConv; the definition of the spatiotemporal network formula is to divide the mine tunnel into i*j regions, and the region set of the network is defined as L={l1, l2, ..., l i*j }; At the same time, divide the time T into different time intervals, and define the interval T = {t s1 , t s2 , ..., t sT }, use a two-dimensional array (a, b) to define the space-time coordinates; where a represents time and b represents area, locate the starting point of the journey of any object: s = (a s , b s ), e=(a e , b e ); (1) Initial field of view flow In a certain time interval t, the initial visual flow of each area End the visual flow (2) Visual field flow The visual field flow describes the dynamic interaction relationship between regions. At time t, the visual flow value from area i to area j is:

6. The system for detecting human intrusion based on cross-viewing of multiple cameras according to claim 1 is characterized in that: The alarm and response module includes an audible and visual alarm arranged on a pedestrian pillar in a loading area of ​​a copper plating automation workshop, and the switch quantity wireless transparent transmission module includes a wireless receiving end and a wireless transmitting end, wherein the wireless transmitting end is installed in a control cabinet of the copper plating automation workshop, and the receiving end is installed next to the audible and visual alarm; the input contacts of the switch quantity wireless transparent transmission module are connected to the corresponding contacts of the controller PLC that controls the start and stop of the truss in the integration and control module, and when there is an intrusion, an online alarm is immediately issued through the switch quantity wireless transparent transmission module, and the alarm signal is transmitted to the integration and control module to control the emergency stop of the truss.

7. A method for detecting human intrusion based on the intersection of multiple camera fields of view, using the system for detecting human intrusion based on the intersection of multiple camera fields of view as described in claims 1 to 6, characterized in that: The following steps are involved: S1. Data collection: The visual perception module uses the camera to capture the image information in the workshop in real time, including personnel activities and rack status, and transmits the image information to the personnel intrusion detection algorithm module; S2. Personnel intrusion detection: Real-time processing of image information transmitted by the visual perception module to accurately identify intrusion behavior of personnel in the workshop; once a person is detected entering the preset prohibited area, the system will immediately trigger the alarm mechanism; S3, alarm and response: When the intrusion detection algorithm module identifies the intrusion behavior, the information is transmitted to the alarm and response module through the data transmission module. The module immediately activates the alarm mechanism and sends the alarm signal to the PLC controller that controls the start and stop of the truss through the wireless transparent transmission module to realize the emergency stop of the truss; S4. Data recording and tracing: The integration and control module will take screenshots and save videos of intrusion events, and record the image information and behavior trajectory of the intruders for subsequent security analysis and tracing.

Images