Visual security collaborative inspection system, computer equipment and storage medium

Abstract

A visual security collaborative inspection system, a computer device and a storage medium wherein a data acquisition module periodically monitors regional images by means of an inspection unit, and extracts people flow and face records. The model control unit deploys polling key points and procedures, the region segmentation module divides regions and constructs assembly line scheduling, and the middle layer unit stores marginal information and counts operation information. The database unit stores data and divides danger levels according to an action feedback scheme. According to the system, precise area monitoring, efficient data processing, reasonable task scheduling and intelligent cooperative early warning are integrally realized, comprehensive and intelligent powerful support is provided for security work, the security inspection efficiency and quality are effectively improved, and a safe environment is guaranteed. According to the invention, the communication unit with the middle layer is arranged, so that a target person or a target monitoring area can be covered when a monitoring system and an emergency occur, and the monitoring security efficiency and the inspection efficiency are further improved while monitoring dead angles are reduced.

Description

Technical Field

[0001] The present invention belongs to the technical field of security patrol inspection, and particularly relates to a visual security collaborative patrol inspection system, a computer device, and a storage medium. Background Art

[0002] In current society, due to the rapid development of Internet +, the security system not only responds quickly in the face of abnormalities, but also in the security inspection process, the combination of face recognition and visual technology recognition with intelligent security patrol inspection is also the basis for rapid response and early prevention today.

[0003] However, in the training process, especially in large-scale areas such as large chemical enterprises, schools, and special control areas, due to the back-and-forth rotation of the security monitoring itself and in the monitoring system, it is very easy to form monitoring blind spots and the monitoring images are too far away and blurred. In both the early prevention and the later rapid response processes, there are many situations where the blurred images lead to unrecognizable and untraceable situations, and the monitoring process is usually relatively fixed, unable to respond flexibly according to the actual situation, resulting in an increase in the resource cost required by the monitoring system and a low patrol inspection efficiency. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a visual security collaborative patrol inspection system, a computer device, and a storage medium. By setting a communication unit with an intermediate layer, the present invention can cover the target person or the target monitoring area in the monitoring system and in case of an emergency, while reducing the monitoring blind spots and further increasing the monitoring security efficiency and the patrol inspection efficiency.

[0005] To solve the above technical problem, the technical solution adopted by the present invention is: A visual security collaborative patrol inspection system includes a data acquisition module, the data acquisition module is electrically connected to a model control unit, an intermediate layer unit, a region segmentation module, and a model control unit respectively, and the model control unit is electrically connected to a database unit; wherein: The data acquisition module completes image monitoring within the time sequence of a period in the area through a patrol inspection unit composed of sensors and cameras, and extracts the number of people and facial records in the image; The model control unit is used to adjust and arrange the key observation targets and daily patrol inspection processes of the patrol inspection units in multiple ranges; The region segmentation module divides the area within a large range according to the patrol inspection units at different positions, and connects multiple patrol inspection units in sequence to form a pipeline-type one-way scheduling. In the scheduling process, it is always ensured that at least two patrol inspection units are included in the same patrol inspection area for real-time monitoring; Intermediate layer unit. An intermediate layer unit is provided between multiple inspection units. The intermediate layer unit is used to store the marginal information of the current inspection unit, combine with adjacent inspection units to statistically analyze the current operation information, and receive configuration information from the model control unit; Database unit. It saves the data of the passenger flow and facial records extracted by the data acquisition module, records behaviors and actions within the time sequence, and in case of abnormal actions, gives real-time feedback to the inspection unit. Preferably, the model control unit issues an inspection task process to different area segmentation modules. The inspection task process distributes the scheduling plan to the intermediate layer unit, and the intermediate layer unit re-combines and distributes according to the priority of the task process.

[0006] Preferably, the inspection task process is divided into multiple sub-model tasks. A complete pipeline training process is constructed through multiple sub-model tasks, and the input data batch of one sub-model task is divided into multiple micro-batches. At the same time, the calculation of each computing device is changed from continuously executing the forward of multiple micro-batches to continuously executing the backward of multiple micro-batches.

[0007] Preferably, the database unit includes an action feedback plan. When the data acquisition module identifies a target action, the risk level is divided according to the target action of the action feedback plan, and finally the feedback is completed by the inspection unit.

[0008] Preferably, the facial record of the data acquisition module completes facial emotion analysis based on the action feedback plan of the database unit, and at the same time combines with the target action to complete the time sequence combination. The process is as follows: Combine the target person in the areas divided by multiple different inspection units and area segmentation modules, and mark the abnormal time in chronological order; In the abnormal time of the target person, combine the time sequences of different facial emotion analysis conclusions; The database unit analyzes whether the risk level is reached. When the risk level is reached, a reminder is given to the target task, and route management is coordinated with the security equipment according to the warning plan; when the risk level is not reached, the abnormal information is transmitted to the model control unit, and a scheduling task is sent through the model control unit, and multiple inspection units continuously monitor the target person; The judgment of the risk level is based on the ConvNet model through the image recognition comparison algorithm and the time segmentation network algorithm to complete the basic judgment, and then different risk levels are classified according to the level division of the database unit.

[0009] A computer device includes a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the system functions described above are implemented.

[0010] A computer-readable storage medium stores a computer program thereon, and when the computer program is executed by a processor, the functions of the visual security collaborative patrol system are implemented.

[0011] A method for operating a visual security collaborative patrol system includes the following steps: S1: The patrol unit periodically conducts image monitoring within a specified area, extracts the pedestrian flow information and facial records in the images; S1.1 The cameras in the patrol unit perform image acquisition on the specified area at a preset time interval, such as every 5 minutes.

[0012] S1.2 After the images are acquired, the built-in image recognition algorithm is used to recognize the people in the images, and the pedestrian flow information is determined by counting the number of people.

[0013] S1.3 For each person, face recognition technology is used to extract facial features, record facial information, and perform a preliminary comparison with the existing facial data in the database to mark whether it is a known person.

[0014] S2: The large-scale monitoring area is divided into multiple small areas according to the location, and the patrol units are connected in sequence to form a pipeline-style one-way scheduling, ensuring that at least two patrol units in the same patrol area are monitoring in real time; S2.1 According to the Geographic Information System (GIS) or the actual on-site layout, the entire monitoring area is divided into multiple small areas according to the building layout, road direction, and functional zoning. For example, a large shopping mall is divided into different floors, different store areas, etc.

[0015] S2.2 Assign corresponding patrol units to each small area, and connect the patrol units in sequence according to the order of adjacent areas to form a pipeline-style one-way scheduling order.

[0016] S2.3 During the scheduling process, the working status of each patrol unit is monitored in real time. If a certain patrol unit fails, it is immediately allocated from the standby patrol unit to ensure that at least two patrol units in the same patrol area are always working properly.

[0017] S3: The intermediate layer unit collects the marginal information of the current patrol unit, combines it with the adjacent patrol units to count the operation information, receives the configuration information from the model control unit, preprocesses it, and then transmits it to the model control unit; S3.1 The intermediate layer unit periodically sends information requests to each connected patrol unit to obtain the marginal information such as the working status, power, and shooting range of each patrol unit.

[0018] S3.2 Integrate the marginal information of the current inspection unit with the information of adjacent inspection units, and statistically analyze the inspection operation of the entire area, such as the inspection coverage rate and inspection time interval of each area.

[0019] S3.3 Receive the configuration information sent by the model control unit, such as task priority adjustment, new inspection strategies, etc., conduct preliminary parsing and screening on this information, remove redundant information, and then transmit it to the model control unit.

[0020] S4: The model control unit adjusts the key observation targets and daily inspection processes of the inspection units, divides priorities according to the flow of people and importance in different areas, and assigns inspection tasks to each inspection unit; S4.1 Obtain the pedestrian flow data of each area in real time, combine historical data and current period characteristics, analyze and judge the pedestrian flow trend of each area, such as paying attention to traffic hub areas during peak commuting hours.

[0021] S4.2 Divide the priorities of inspection tasks according to the importance of the area, such as key areas such as bank vaults and data centers, as well as the flow of people.

[0022] S4.3 According to the priorities, formulate a detailed inspection task plan, assign tasks to the corresponding inspection units, and clarify the inspection time, route and key observation targets of each inspection unit.

[0023] S5: Use the data saved in the database unit to record the behavior and actions in time series. When an abnormal action occurs, it is immediately fed back to the inspection unit to strengthen the monitoring. The database unit saves effective monitoring schemes and deep learning algorithm data sets, and re-learns and updates the inspection unit based on the newly obtained data; S5.1 The database unit records the human behavior and action data collected by each inspection unit in chronological order and establishes a detailed behavior log.

[0024] S5.2 Use a preset abnormal behavior recognition model to conduct real-time analysis on the behavior log to judge whether there are abnormal actions, such as running, fighting, etc.

[0025] S5.3 Once an abnormal action is detected, immediately send an instruction to the relevant inspection unit, requiring it to adjust the shooting angle and range and strengthen the monitoring of the abnormal area.

[0026] S5.4 Regularly organize and update the monitoring schemes and deep learning algorithm data sets in the database, incorporate the newly collected data into the training, optimize the algorithm model, and improve the recognition ability of the inspection unit.

[0027] S6: When detecting abnormal persons or behaviors, increase the monitoring coverage of the target to avoid monitoring blind spots. If the target person exhibits dangerous behaviors, the indoor and outdoor patrol units will focus on covering and capturing images, and be locally scheduled by the intermediate layer unit. Meanwhile, notify the model control unit to take measures to eliminate the danger; S6.1 When the patrol unit detects an abnormal person (such as a stranger frequently wandering in a sensitive area) or an abnormal behavior, the intermediate layer unit immediately activates the emergency plan.

[0028] S6.2 Mobilize the surrounding patrol units, adjust the shooting angles and ranges, and form an all-round monitoring coverage of the abnormal target to ensure there are no monitoring blind spots.

[0029] S6.3 If the target person exhibits dangerous behaviors, such as hurting people with weapons, etc., the indoor and outdoor patrol units quickly converge and capture images of the target from multiple angles.

[0030] S6.4 The intermediate layer unit makes local scheduling according to the on-site situation, coordinates the work of each patrol unit, and at the same time promptly notifies the model control unit of the situation. The model control unit coordinates relevant departments to take measures to eliminate the danger.

[0031] S7: Analyze the emotions and behaviors of the target person, and based on the time series combination, judge whether the danger level is reached. When the danger level is reached, remind relevant personnel and implement route management; when it is not reached, transmit information to the model control unit to send a new scheduling task; S7.1 Use facial expression recognition and behavior analysis algorithms to analyze the emotions and behaviors of the target person in real time, and judge their emotional states (such as anger, anxiety, etc.) and behavior intentions (such as attack, escape, etc.).

[0032] S7.2 Combine the analysis results of emotions and behaviors at different time points, and analyze their changing trends according to the time series.

[0033] S7.3 According to the preset danger level assessment model, combined with the time series analysis results, judge whether the danger level is reached.

[0034] S7.4 If the danger level is reached, immediately send reminder information to relevant personnel such as security personnel and management personnel, and activate the route management plan to guide personnel evacuation or control the action route of the target person.

[0035] S7.5 If the danger level is not reached, transmit the relevant information to the model control unit, and the model control unit sends a new scheduling task according to the situation, such as strengthening the continuous monitoring of the target person.

[0036] S8: Divide the inspection task process into multiple sub-model tasks, and further split each sub-model task into micro-batches to improve the response speed. Control the inspection unit to perform specific tasks through the forward and reverse control of the intermediate layer unit; S8.1 According to the type and target of the inspection task, divide the entire inspection task process into multiple sub-model tasks. For example, divide the shopping mall inspection task into sub-model tasks such as entrance area inspection, store area inspection, and public area inspection.

[0037] S8.2 For each sub-model task, further split it into multiple micro-batches according to the data processing volume and time requirements. Each micro-batch contains a small amount of data processing tasks.

[0038] S8.3 The intermediate layer unit controls the inspection unit to execute the tasks of each micro-batch in sequence through forward (according to the normal scheduling order) and reverse (adjust the scheduling order according to special requirements) methods according to the task priority and real-time situation to ensure the efficient completion of the tasks.

[0039] S9: Use the image recognition comparison algorithm and the time segmentation network algorithm to determine the danger level, and formulate corresponding countermeasures according to different danger levels. The countermeasures include communication, assistance, or preventive intervention; S9.1 Use the image recognition comparison algorithm to compare the collected images with the standard images in the database to analyze whether the behavior actions of the person are abnormal.

[0040] S9.2 Use the time segmentation network algorithm to analyze the video data within a period of time, identify key events and behavior patterns, and further determine the degree of danger.

[0041] S9.3 According to the analysis results of the two algorithms, determine the danger level according to the preset danger level standard, such as dividing it into three levels: low, medium, and high.

[0042] S9.4 For different danger levels, formulate corresponding countermeasures: Low danger level: Arrange security personnel to communicate with the target person to understand the situation and eliminate potential risks.

[0043] Medium danger level: Organize relevant personnel to assist in handling, such as medical personnel ready to provide assistance at any time, and property personnel to assist in maintaining order.

[0044] High danger level: Immediately take preventive intervention measures, such as calling the police and controlling the actions of the target person.

[0045] S10: The model control unit continuously regulates the inspection process to ensure the optimal information transmission process, continuously monitors the behavior changes, and confirms that the behavior returns to normal.

[0046] S10.1 monitors the working status, information transmission and task execution progress of each inspection unit in real time, and collects various information fed back by the middle-layer units.

[0047] S10.2 Based on the collected information, analyze and determine the problems existing in the current inspection process, such as information transmission delay, unreasonable task allocation, etc.

[0048] S10.3 In response to existing problems, timely adjust the inspection process, optimize the information transmission path, and reallocate tasks to ensure the efficient operation of the inspection process.

[0049] S10.4 Continue to monitor behavioral changes in the area where the abnormal event occurred. When it is confirmed that the behavior has returned to normal, cancel the corresponding emergency plan and resume the normal inspection process.

[0050] The present invention can achieve the following beneficial effects: 1. The present invention provides a communication unit with an intermediate layer, which can provide coverage for target persons or target monitoring areas in the monitoring system and when an emergency occurs, thereby further increasing monitoring security efficiency and inspection efficiency while reducing monitoring blind spots.

[0051] 2. The present invention uses the regional segmentation module to reasonably divide a large area, and uses pipeline-type one-way scheduling to ensure that the same inspection area is always monitored in real time by at least two inspection units. This makes the monitoring of the entire area seamless and coherent, greatly improving the reliability and accuracy of monitoring, and being able to promptly detect any abnormal situation or suspicious target in the area, effectively improving the efficiency and quality of security monitoring, and reducing safety risks.

[0052] 3. The model control unit of the present invention can flexibly adjust and arrange the key observation targets and daily inspection processes of the inspection units in multiple ranges. It can accurately allocate monitoring resources according to the characteristics of different areas, risk levels and needs of specific time periods, realize continuous tracking of key targets and comprehensive inspections of the entire area, and enhance the pertinence and effectiveness of security work.

[0053] 4. The data acquisition module of the present invention can accurately extract the flow of people and facial records in the image, and record behaviors and actions in a time series. The database unit stores and analyzes these data, and divides the target actions into dangerous levels based on the image recognition comparison algorithm and the time segmentation network algorithm, combined with the action feedback scheme, which can not only accurately judge abnormal behaviors, but also take corresponding measures according to the degree of danger, such as timely feedback to the inspection unit for reminders or coordinated security equipment for route management, effectively preventing the occurrence of security incidents and ensuring the safety of personnel and property.

[0054] 5. The inspection task process of the present invention is divided into multiple sub-model tasks, and a complete pipeline training process is constructed. The input data batch is divided into multiple micro-batches for calculation. This design enables the system to make full use of computing resources, improve computing efficiency, speed up task processing, ensure the timeliness and efficiency of the inspection work, and avoid potential safety hazards caused by data processing delays. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The present invention will be further described below in conjunction with the drawings and embodiments: Figure 1 is a schematic structural diagram of the system of the present invention; Figure 2 is a schematic flow chart of the present invention; Figure 3 is a simulation diagram of the intermediate layer unit of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0056] As Figure 1-2 shown, the present invention provides a visual security collaborative inspection system, including: A data acquisition module that completes image monitoring within the periodic time sequence of the area through an inspection unit composed of sensors and cameras, and extracts the number of people and facial records in the image; A model control unit for adjusting and arranging the key observation targets and daily inspection processes of inspection units within multiple ranges; An area segmentation module that segments the area within a large range according to the inspection units at different positions, and connects multiple inspection units in sequence to form a pipeline-type one-way scheduling. During the scheduling process, it is always ensured that at least two inspection units are included in the same inspection area for real-time monitoring; An intermediate layer unit is provided between multiple inspection units. The intermediate layer unit is used to store the marginal information of the current inspection unit, combine it with adjacent inspection units to statistically analyze the current operation information, and receive configuration information from the model control unit; A database unit stores the data of the number of people and facial records extracted by the data acquisition module, records behaviors and actions within the time sequence, and feeds back to the inspection unit in real time when abnormal actions occur.

[0057] The inspection unit involved in the data acquisition module can be a fixed monitoring camera or a mobile portable camera, robot, drone and other devices. Face recognition algorithms and emotion recognition algorithms can be installed in the devices. That is, according to simple face recognition matching, the target population is divided into frequently appearing people and non-frequently appearing people. At the same time, according to different emotions, different appearing frequencies of the population are analyzed. For example, more emotion analysis is used for non-frequently appearing people.

[0058] The model control unit includes a database unit. All effective and reliable monitoring plans recorded by the model control unit are saved in the database unit. The database unit also saves a unified algorithm data set that has undergone deep learning. The data set is re-learned based on the data obtained by the data acquisition module according to the time period and is updated with the corresponding inspection unit. However, if the inspection unit itself runs image recognition or emotion analysis, the power consumption of the inspection unit will increase. The function of the inspection unit itself is mainly to collect security data. Using the model control unit to directly control the inspection unit point-to-point will also reduce the control efficiency. Therefore, an intermediate layer unit is added between multiple inspection units. The intermediate layer unit can connect the inspection units in an area, such as Figure 3 As shown, the message is pre-processed by the middle layer unit and then transmitted to the model control unit. Since the middle layer unit is within the data receiving range compared to the model control unit, when an emergency occurs, the middle layer unit can complete the control operation of the nearby inspection unit according to its own equipped scheme and its own algorithm; The added middle-layer unit can be a wireless connection or limited connection device for the inspection unit, which is directly affiliated with the local area. For example, when the monitoring range is divided according to the building, the middle-layer unit is on the top floor or middle floor of the building. When other adapter devices appear, such as drones for high-altitude security inspections, fixed decoupling can be used to make device connection more convenient. Only the middle-layer unit needs to be adapted to connect to the corridor monitoring devices on each floor of the building. If a target person wearing a peaked cap and a mask appears in the inspection area, the data acquisition module can increase the monitoring coverage of the target person according to the instructions of the middle layer unit to avoid forming a monitoring blind spot, and then try to analyze his behavior and emotions. When the target person exhibits dangerous behavior, the indoor and outdoor inspection units will focus on covering the monitoring images. Since the outdoor inspection unit can be an external security drone that is instantly connected, or monitoring equipment in other buildings, and multiple monitoring ranges are connected to the same middle layer unit, it is only necessary to dispatch through the local middle layer unit at this time, and inform the model control unit in real time to take measures to eliminate the danger; In the daily operation, the model control unit issues patrol tasks to different area segmentation modules. The priority of the issued patrol tasks is the lowest level, and at least two different security devices in the same area of the patrol plan should cover the image range simultaneously. Therefore, in the actual patrol task process, based on decision trees and support vector machines under the neural network, the road segments in the area are mainly divided into sections with high pedestrian flow, important sections, and sections with low pedestrian flow. In sections with high pedestrian flow and important sections, multiple security devices are required to cover the monitoring, while in sections with low pedestrian flow, only single or mobile monitoring forms are adopted. For example, monitoring devices with rotatable directions are set at stairways, and the monitoring is covered in real time by wide-angle and upper and lower floor monitoring devices to avoid the situation where a certain device is damaged, thereby generating a basic task scheduling process based on artificial intelligence selection.

[0059] The patrol task process will be divided into multiple sub-model tasks. A complete pipeline training process is constructed through multiple sub-model tasks, and the input data batch of a sub-model task is divided into multiple micro-batches. At the same time, the calculations of each computing device are from continuously executing multiple forward micro-batches and then continuously executing multiple backward micro-batches; that is, when performing the task process, a whole patrol task process will be split into sub-models within an area. For example, several monitoring areas with higher priorities are split into several sub-model tasks based on the intermediate layer unit, and then the patrol unit is controlled forward and backward by the sub-model tasks. Since it is micro-batch processing, the sub-model tasks have better discreteness, so the execution time will be much less than the batch processing interval, which will make the response of the monitoring device faster. Instead of directly stuffing the entire execution task to the monitoring device for operation, the monitoring device only needs to execute a single task of the intermediate layer unit.

[0060] The facial record of the data acquisition module completes facial emotion analysis based on the action feedback scheme of the database unit, and combines the target actions to complete the timing combination. The process is as follows: Combine the target person in the areas segmented by multiple different patrol units and area segmentation modules, and mark the abnormal time in chronological order; In the abnormal time of the target person, combine the timing of different facial emotion analysis conclusions; The database unit analyzes whether the danger level has been reached. When the danger level has been reached, a reminder is made for the target task, and route management is coordinated with the security equipment according to the warning scheme; when the danger level has not been reached, the abnormal information is transmitted to the model control unit, and a scheduling task is sent through the model control unit. Multiple patrol units continuously monitor the target person; The judgment of the danger level is based on the basic judgment of the ConvNet model through the image recognition and comparison algorithm and the time segmentation network algorithm, and then different danger levels are classified according to the level division of the database unit.

[0061] The image recognition comparison algorithm directly compares the face intercepted from the video image with the emotion pictures in the database after face cropping. This method can achieve quick recognition and comparison for regular customers, reducing the burden on the database unit and quickly identifying the facial emotions of the target object. For example, for the property owners in a community or the staff in a building, when there are situations different from their daily behaviors and regular emotions, such as venting anger on the elevator buttons or running abnormally on the stairs, early warnings will be issued directly. The time segmentation network algorithm mainly identifies non-regular customers and people with abnormal behaviors. It trains the video based on the ConvNet model. For example, behaviors such as multiple people fighting and group emotional finger-pointing in the video are classified as multiple-person dangers, which are the highest danger level; behaviors such as smashing things and a single person falling in the video are classified as single-person dangers, which are the medium danger level; for the dangerous areas that the objects in the video are going to, such as a person with a lost mood going to the rooftop, entering the electrical equipment room without safety measures, and carrying swimming equipment and preparing to enter the prohibited deep water area, such behaviors are unhappened and preventable behaviors, which are the low danger level. Based on the above different danger levels, the staff can set different response strategies. For example, they can communicate with the target person who is going to the rooftop with a lost mood in a timely manner, or assist and treat the target with single-person dangerous behaviors in a timely manner by carrying tools.

[0062] Specifically, in the patrol inspection process, under the control of the model control unit, information within the area is obtained based on the data acquisition module. The area segmentation module connects the boundaries of multiple areas in series to form an overall, and cooperates with the database unit and the intermediate layer unit to optimize the information transmission process.

[0063] In the patrol inspection process, multiple patrol inspection units are interconnected through the intermediate layer, making the multiple divided areas linked as a whole. When an abnormality occurs, it is processed according to the danger level prompt. During the processing process, it continues to monitor whether the behavior returns to normal, such as Figure 2 forming a patrol inspection process in this form; and for each change generated, the model control unit issues task messages targeted. Each task message will be decomposed into sub-model tasks and sent to the specific intermediate layer. After the intermediate layers exchange information, they control the adjacent monitoring areas to execute the sub-task information.

[0064] In another embodiment, the present invention provides a computer device, including a processor, a memory, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it realizes the system functions of the system embodiment.

[0065] In another embodiment, the present invention provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the system functions of the system embodiment are implemented.

[0066] A method for operating a visual security collaborative patrol system includes the following steps: S1: The patrol unit periodically conducts image monitoring within a specified area, extracts the pedestrian flow information and facial records in the images; S1.1 The cameras in the patrol unit perform image acquisition on the specified area at a preset time interval, such as every 5 minutes.

[0067] S1.2 After the images are acquired, the built-in image recognition algorithm is used to identify the people in the images, and the pedestrian flow information is determined by counting the number of people.

[0068] S1.3 For each person, face recognition technology is adopted to extract facial features, record facial information, and perform a preliminary comparison with the existing facial data in the database to mark whether it is a known person.

[0069] S2: The large-scale monitoring area is divided into multiple small areas according to the location, and the patrol units are connected in sequence to form a pipeline-style one-way scheduling, ensuring that at least two patrol units in the same patrol area are monitoring in real time; S2.1 According to the Geographic Information System (GIS) or the actual on-site layout, the entire monitoring area is divided into multiple small areas according to the building layout, road direction, and functional zoning. For example, a large shopping mall is divided into different floors, different store areas, etc.

[0070] S2.2 Corresponding patrol units are assigned to each small area, and the patrol units are connected in sequence according to the order of adjacent areas to form a pipeline-style one-way scheduling order.

[0071] S2.3 During the scheduling process, the working status of each patrol unit is monitored in real time. If a certain patrol unit fails, it is immediately allocated from the standby patrol units to ensure that at least two patrol units in the same patrol area are always working properly.

[0072] S3: The intermediate layer unit collects the marginal information of the current patrol unit, combines it with the adjacent patrol units to count the operation information, receives the configuration information from the model control unit, preprocesses it, and then transmits it to the model control unit; S3.1 The intermediate layer unit periodically sends information requests to each connected patrol unit to obtain the marginal information such as the working status, power, and shooting range of each patrol unit.

[0073] S3.2 Integrate the marginal information of the current inspection unit with the information of adjacent inspection units, and statistically analyze the operation of inspections in the entire area, such as the inspection coverage rate and inspection time interval of each area.

[0074] S3.3 Receive the configuration information sent by the model control unit, such as task priority adjustment, new inspection strategies, etc., perform preliminary parsing and screening on this information, remove redundant information, and then transmit it to the model control unit.

[0075] S4: The model control unit adjusts the key observation targets and daily inspection processes of the inspection units, divides priorities according to the pedestrian flow situation and importance of different areas, and assigns inspection tasks to each inspection unit; S4.1 Obtain the pedestrian flow data of each area in real time, analyze and judge the pedestrian flow trend of each area by combining historical data and the characteristics of the current period. For example, during the peak commuting hours, focus on traffic hub areas.

[0076] S4.2 Divide the priorities of inspection tasks according to the importance of the areas, such as key areas like bank vaults and data centers, as well as the pedestrian flow situation.

[0077] S4.3 According to the priorities, formulate a detailed inspection task plan, assign tasks to the corresponding inspection units, and clarify the inspection time, route, and key observation targets of each inspection unit.

[0078] S5: Use the data saved in the database unit to record the behavior and actions in chronological order. When an abnormal action occurs, it is immediately fed back to the inspection unit to strengthen monitoring. The database unit saves effective monitoring schemes and deep learning algorithm data sets, and re-learns and updates the inspection unit based on the newly obtained data; S5.1 The database unit records the data of the behavior and actions of each inspection unit collected in chronological order to establish a detailed behavior log.

[0079] S5.2 Use a preset abnormal behavior recognition model to perform real-time analysis on the behavior log to determine whether there are abnormal actions, such as running, fighting, etc.

[0080] S5.3 Once an abnormal action is detected, immediately send an instruction to the relevant inspection unit, requiring it to adjust the shooting angle and range to strengthen the monitoring of the abnormal area.

[0081] S5.4 Regularly organize and update the monitoring schemes and deep learning algorithm data sets in the database, incorporate the newly collected data into the training, optimize the algorithm model, and improve the recognition ability of the inspection unit.

[0082] S6: When detecting abnormal people or behaviors, increase the monitoring coverage of the target to avoid monitoring blind spots. If the target person exhibits dangerous behaviors, the indoor and outdoor patrol units will focus on covering and taking pictures, and be locally dispatched by the intermediate layer unit. Meanwhile, notify the model control unit to take measures to eliminate the danger; S6.1 When the patrol unit detects abnormal people (such as strangers frequently wandering in sensitive areas) or abnormal behaviors, the intermediate layer unit immediately activates the emergency plan.

[0083] S6.2 Mobilize the surrounding patrol units, adjust the shooting angles and ranges, and form an all-round monitoring coverage of the abnormal target to ensure there are no monitoring blind spots.

[0084] S6.3 If the target person exhibits dangerous behaviors, such as hurting people with weapons, etc., the indoor and outdoor patrol units will quickly gather and take pictures of the target from multiple angles.

[0085] S6.4 The intermediate layer unit makes local dispatching according to the on-site situation, coordinates the work of each patrol unit, and at the same time promptly notifies the model control unit of the situation. The model control unit coordinates relevant departments to take measures to eliminate the danger.

[0086] S7: Analyze the emotions and behaviors of the target person, and based on the time series combination, judge whether the danger level is reached. When the danger level is reached, remind relevant personnel and implement route management; when it is not reached, transmit information to the model control unit to send a new dispatching task; S7.1 Use facial expression recognition and behavior analysis algorithms to analyze the emotions and behaviors of the target person in real time, and judge their emotional states (such as anger, anxiety, etc.) and behavior intentions (such as attack, escape, etc.).

[0087] S7.2 Combine the analysis results of emotions and behaviors at different time points, and analyze their changing trends according to the time series.

[0088] S7.3 According to the preset danger level assessment model, combined with the time series analysis results, judge whether the danger level is reached.

[0089] S7.4 If the danger level is reached, immediately send reminder information to relevant personnel such as security personnel and management personnel, and activate the route management plan to guide personnel evacuation or control the action route of the target person.

[0090] S7.5 If the danger level is not reached, transmit the relevant information to the model control unit, and the model control unit sends a new dispatching task according to the situation, such as strengthening the continuous monitoring of the target person.

[0091] S8: Divide the inspection task process into multiple sub-model tasks, and further split each sub-model task into micro-batches to improve the response speed. Control the inspection unit to perform specific tasks through the forward and reverse control of the intermediate layer unit; S8.1 According to the type and objective of the inspection task, divide the entire inspection task process into multiple sub-model tasks. For example, divide the shopping mall inspection task into sub-model tasks such as entrance area inspection, store area inspection, and public area inspection.

[0092] S8.2 For each sub-model task, further split it into multiple micro-batches according to the data processing volume and time requirements. Each micro-batch contains a small amount of data processing tasks.

[0093] S8.3 The intermediate layer unit controls the inspection unit to execute the tasks of each micro-batch in sequence through forward (according to the normal scheduling order) and reverse (adjust the scheduling order according to special requirements) methods according to the task priority and real-time situation to ensure the efficient completion of the task.

[0094] S9: Use the image recognition and comparison algorithm and the time segmentation network algorithm to determine the danger level, and formulate corresponding countermeasures according to different danger levels. The countermeasures include communication, assistance, or preventive intervention; S9.1 Use the image recognition and comparison algorithm to compare the collected images with the standard images in the database to analyze whether the behavior actions of the person are abnormal.

[0095] S9.2 Use the time segmentation network algorithm to analyze the video data within a period of time, identify key events and behavior patterns, and further determine the degree of danger.

[0096] S9.3 According to the analysis results of the two algorithms, determine the danger level according to the preset danger level standard, such as dividing it into three levels: low, medium, and high.

[0097] S9.4 For different danger levels, formulate corresponding countermeasures: Low danger level: Arrange security personnel to communicate with the target person, understand the situation, and eliminate potential risks.

[0098] Medium danger level: Organize relevant personnel to assist in handling, such as medical personnel being ready to provide assistance at any time, and property personnel assisting in maintaining order.

[0099] High danger level: Immediately take preventive intervention measures, such as calling the police and controlling the actions of the target person.

[0100] S10: The model control unit continuously regulates the inspection process to ensure the optimal information transmission process, continuously monitors the behavior changes, and confirms that the behavior returns to normal.

[0101] S10.1 monitors the working status, information transmission and task execution progress of each inspection unit in real time, and collects various information fed back by the middle-layer units.

[0102] S10.2 Based on the collected information, analyze and determine the problems existing in the current inspection process, such as information transmission delay, unreasonable task allocation, etc.

[0103] S10.3 In response to existing problems, timely adjust the inspection process, optimize the information transmission path, and reallocate tasks to ensure the efficient operation of the inspection process.

[0104] S10.4 Continue to monitor behavioral changes in the area where the abnormal event occurred. When it is confirmed that the behavior has returned to normal, cancel the corresponding emergency plan and resume the normal inspection process.

[0105] The above embodiments are only preferred technical solutions of the present invention and should not be regarded as limiting the present invention. The protection scope of the present invention shall be the technical solutions recorded in the claims, including equivalent replacement solutions of the technical features in the technical solutions recorded in the claims. That is, equivalent replacement improvements within this scope are also within the protection scope of the present invention.

Claims

1. A visual security collaborative inspection system, characterized by: It includes a data acquisition module, which is electrically connected to the model control unit, the middle layer unit, the region segmentation module and the model control unit respectively, and the model control unit is electrically connected to the database unit; wherein: The data acquisition module uses the inspection unit composed of sensors and cameras to complete image monitoring in the area within a periodic time sequence and extract the flow of people and facial records in the image; Model control unit, used to adjust and arrange key observation targets and daily inspection processes of inspection units within multiple ranges; The area segmentation module divides the area within a large area according to the inspection units at different positions, and connects multiple inspection units in sequence to form a pipeline-style one-way scheduling. The scheduling process always ensures that at least two inspection units are included in the same inspection area for real-time monitoring; An intermediate layer unit is provided between the plurality of inspection units, and is used to store the marginal information of the current inspection unit, to combine with the adjacent inspection units to count the current operation information, and to receive the configuration information from the model control unit; The database unit saves the data of human traffic and facial records extracted by the data acquisition module, records behaviors and actions in a time series, and provides real-time feedback to the inspection unit when abnormal actions occur.

2. A visual security collaborative inspection system according to claim 1, characterized in that: The model control unit issues inspection task processes to different area segmentation modules. The inspection task processes distribute the scheduling plans to the middle-layer units, which then combine and distribute them again according to the priority of the task processes.

3. A visual security collaborative inspection system according to claim 2, characterized in that: The inspection task process will be divided into multiple sub-model tasks. A complete pipeline training process will be built through multiple sub-model tasks, and the input data batch of a sub-model task will be divided into multiple micro-batches. At the same time, the calculation of each computing device will be changed from continuously executing the forward direction of multiple micro-batches to continuously executing the reverse direction of multiple micro-batches.

4. A visual security collaborative inspection system according to claim 1, characterized in that: The database unit includes an action feedback scheme. When the data acquisition module identifies the target action, the danger level is divided according to the target action of the action feedback scheme, and finally the inspection unit completes the feedback.

5. A visual security collaborative inspection system according to claim 4, characterized in that: The facial record of the data acquisition module completes the facial emotion analysis based on the action feedback scheme of the database unit, and combines the target action to complete the timing combination. The process is as follows: The target person is combined in the areas divided by multiple different inspection units and area segmentation modules, and the abnormal time is marked in chronological order; Combine the timing of different facial emotion analysis conclusions during the abnormal time of the target person; The database unit analyzes whether the danger level has been reached. If the danger level has been reached, a reminder will be given to the target task, and route management will be coordinated with the security equipment according to the early warning plan. If the danger level has not been reached, the abnormal information will be transmitted to the model control unit, and the scheduling task will be sent through the model control unit, and multiple inspection units will monitor the target person uninterruptedly. The judgment of danger level is completed through image recognition comparison algorithm and time segmentation network algorithm based on ConvNet model, and then different danger levels are classified according to the level division of database units.

6. A computer device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the processor implements the system function described in any one of claims 1 to 5 when executing the computer program.

7. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the functions of a visual security collaborative inspection system as described in any one of claims 1 to 5.

8. The method for operating a visual security collaborative inspection system according to any one of claims 1 to 5, characterized in that: The following steps are involved: S1: The inspection unit periodically monitors images in the designated area and extracts the flow of people and facial records in the images; S2: The large monitoring area is subdivided into multiple small areas according to the location, and connected in sequence through inspection units to form a pipeline-style one-way scheduling, ensuring that at least two inspection units are monitoring in real time in the same inspection area; S3: The middle layer unit collects the marginal information of the current inspection unit, and combines the statistical operation information with the adjacent inspection units, receives the configuration information from the model control unit and transmits it to the model control unit after preprocessing; S4: The model control unit adjusts the key observation targets and daily inspection processes of the inspection units, and prioritizes and assigns inspection tasks to each inspection unit according to the flow of people and importance of different areas; S5: Use the data stored in the database unit to record the behaviors and actions in time series. When abnormal actions occur, feedback is given to the inspection unit in real time to strengthen monitoring. The database unit stores effective monitoring plans and deep learning algorithm data sets, and re-learns and updates the inspection unit based on the newly acquired data. S6: When an abnormal person or behavior is detected, the monitoring coverage of the target is increased to avoid blind spots. If the target person exhibits dangerous behavior, the indoor and outdoor inspection units will centrally cover the captured images and the middle layer unit will dispatch them locally. At the same time, the model control unit will be notified to take measures to eliminate the danger. S7: Analyze the target person's emotions and behaviors, and determine whether the danger level has been reached based on the time series combination. If the danger level has been reached, the relevant personnel will be reminded and route management will be implemented; if the danger level has not been reached, the information will be transmitted to the model control unit to send a new scheduling task; S8: Divide the inspection task flow into multiple sub-model tasks, and each sub-model task is further divided into micro-batch processing to improve the response speed, and control the inspection unit to perform specific tasks through the intermediate layer unit anterograde and retrograde; S9: Use image recognition comparison algorithm and time segmentation network algorithm to determine the danger level, and formulate corresponding response strategies according to different danger levels. The response strategies include communication, assistance or preventive intervention; S10: The model control unit continuously regulates the inspection process to ensure the optimal information transmission process, continuously monitors behavior changes, and confirms that the behavior returns to normal.

9. The method for operating a visual security collaborative inspection system according to claim 8, characterized in that: The following steps are involved: The sub-steps of S1 are: S1.1 the camera in the inspection unit collects an image of the designated area at a preset time interval; After S1.2 collects the image, it uses the built-in image recognition algorithm to identify the people in the image and determines the flow of people by counting the number of people; S1.3 For each person, facial recognition technology is used to extract facial features, record facial information, and make a preliminary comparison with the existing facial data in the database to mark whether it is a known person; The sub-steps of S2 are: S2.1 Based on the geographic information system or actual on-site layout, the entire monitoring area is divided into multiple small areas according to building layout, road direction and functional zoning; S2.2 allocates a corresponding inspection unit to each small area, and connects the inspection units in sequence according to the order of adjacent areas to form a pipeline-like one-way scheduling sequence; S2.3 During the dispatching process, the working status of each inspection unit is monitored in real time. If a certain inspection unit fails, it is immediately deployed from the spare inspection unit to ensure that there are always at least two inspection units working normally in the same inspection area; The sub-steps of S3 are: S3.1 The middle layer unit periodically sends information requests to each connected inspection unit to obtain marginal information such as the working status, power, and shooting range of each inspection unit; S3.2 integrates the marginal information of the current inspection unit with the information of the adjacent inspection units to statistically analyze the inspection operation status of the entire area; S3.3 receives the configuration information sent by the model control unit, performs preliminary analysis and screening on the information, removes redundant information, and then transmits it to the model control unit; The sub-steps of S4 are: S4.1 Obtain the flow data of people in each area in real time, and analyze and determine the flow trend of people in each area by combining historical data and the characteristics of the current period; S4.2 Prioritize inspection tasks based on the importance of the area, such as key areas such as bank vaults and data centers, and the flow of people; S4.3 Develop a detailed inspection task plan based on priority, assign tasks to corresponding inspection units, and specify the inspection time, route, and key observation targets for each inspection unit; The sub-steps of S5 are: S5.1 The database unit records the human behavior and action data collected by each inspection unit in chronological order and establishes a detailed behavior log; S5.2 uses the preset abnormal behavior recognition model to analyze the behavior log in real time to determine whether there are abnormal actions; S5.3 Once abnormal movement is detected, an instruction is immediately sent to the relevant inspection unit, requiring it to adjust the shooting angle and range to strengthen the monitoring of the abnormal area; S5.4 Regularly organize and update the monitoring plans and deep learning algorithm data sets in the database, incorporate newly collected data into training, optimize the algorithm model, and enhance the recognition capabilities of the inspection unit.

10. The method for operating a visual security collaborative inspection system according to claim 8, characterized in that: The following steps are involved: The sub-steps of S6 are: S6.1 When the inspection unit detects an abnormal person or abnormal behavior, the middle layer unit immediately activates the emergency plan; S6.2 mobilize the surrounding inspection units, adjust the shooting angle and range, and form a full range of monitoring coverage for abnormal targets to ensure that there are no blind spots in monitoring; S6.3 If the target person exhibits dangerous behavior, the indoor and outdoor inspection units will quickly converge to take images of the target from multiple angles; S6.4 The middle layer unit conducts local dispatch and coordinates the work of each inspection unit according to the on-site situation, and promptly informs the model control unit of the situation, which then coordinates relevant departments to take measures to eliminate the danger; The sub-steps of S7 are: S7.1 uses facial expression recognition and behavior analysis algorithms to analyze the target person's emotions and behaviors in real time to determine their emotional state and behavioral intentions; S7.2 Combine the emotion and behavior analysis results at different time points and analyze their changing trends according to the time series; S7.3 Determine whether the hazard level has been reached based on the preset hazard level assessment model and the time series analysis results; S7.4 If the danger level is reached, immediately send a reminder message to security personnel, management personnel and other relevant personnel, and start the route management plan to guide personnel to evacuate or control the target person's route; S7.5 If the danger level is not reached, the relevant information is transmitted to the model control unit, and the model control unit sends a new scheduling task according to the situation; The sub-steps of S8 are: S8.1 Divide the entire inspection task process into multiple sub-model tasks according to the type and goal of the inspection task. For example, divide the shopping mall inspection task into sub-model tasks including entrance area inspection, store area inspection, and public area inspection. S8.2 For each sub-model task, further split it into multiple micro-batches according to the data processing volume and time requirements, and each micro-batch contains a small amount of data processing tasks; The S8.3 middle-layer unit controls the inspection unit to execute each micro-batch task in sequence in a forward and retrograde manner according to the task priority and real-time situation to ensure efficient completion of the task; The sub-steps of S9 are: S9.1 uses image recognition and comparison algorithms to compare the collected images with standard images in the database to analyze whether the person's behavior is abnormal; S9.2 uses a time segmentation network algorithm to analyze video data over a period of time, identify key events and behavior patterns, and further determine the degree of danger; S9.3 Determine the hazard level based on the analysis results of the two algorithms and in accordance with the preset hazard level standards; S9.4 Develop corresponding response strategies for different risk levels: Low risk level: arrange security personnel to communicate with the target person to understand the situation and eliminate potential risks; Medium-risk level: Organize relevant personnel to assist in handling the situation, such as medical personnel ready to provide assistance at any time, and property personnel assist in maintaining order; High risk level: Take preventive intervention measures immediately; The sub-steps of S10 are: S10.1 monitors the working status, information transmission and task execution progress of each inspection unit in real time, and collects various information fed back by the middle-layer units; S10.2 Analyze and determine the problems existing in the current inspection process based on the collected information; S10.3 In response to existing problems, timely adjust the inspection process, optimize the information transmission path, and reallocate tasks to ensure the efficient operation of the inspection process; S10.4 Continue to monitor behavioral changes in the area where the abnormal event occurred. When it is confirmed that the behavior has returned to normal, cancel the corresponding emergency plan and resume the normal inspection process.

Images