A regional intrusion detection method and system based on multi-target tracking

Abstract

The present invention proposes a method and system for regional intrusion detection based on multi-target tracking, including obtaining a video stream from a monitoring device, collecting discrete continuous image frames from the video stream at intervals; setting a polygonal monitoring area, and setting the edge of the image frame where it is located as the personnel intrusion boundary; preprocessing the image frame, including reducing the image width and height in proportion, filling black pixels, adjusting the image size, and normalizing each image frame to obtain an input image; converting the input image into training image data, passing it into a single-step multi-frame detector for model parameter training, and obtaining an optimized detection model; performing personnel detection and feature extraction, obtaining a detected boundary box, locating and outputting the specific position of the personnel in the input image; multi-target tracking, obtaining personnel trajectories, personnel intrusion conditions, and calculating the number of personnel in the polygonal monitoring area. The present invention can accurately identify and continuously track each monitored individual.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a regional intrusion detection method and system based on multi-target tracking. Background Art

[0002] Regional intrusion detection technology aims to monitor whether there are unauthorized people or objects in a specific area to prevent illegal intrusion and various security accidents. This technology is widely used in many fields such as security protection, border control, and protection of important facilities. Traditional intrusion detection methods mainly rely on manual monitoring, supplemented by infrared, microwave, vibration and other sensors to detect temperature changes and movement. However, these methods have problems such as high false alarm rate, slow response speed and high labor cost.

[0003] With the advancement of computer technology, especially the development of artificial intelligence technology, regional intrusion detection has ushered in a new change. Among them, the application of artificial intelligence technology has significantly improved the detection efficiency and reduced the demand for human resources. Regional intrusion detection schemes based on artificial intelligence can be roughly divided into two categories: target detection-based methods and target tracking-based methods. The former mainly judges whether there is intrusion behavior by quickly identifying objects in the image, but due to the lack of in-depth analysis of dynamic information, it is easy to produce missed reports or false alarms, especially in complex backgrounds or high-density crowd environments; in contrast, the target tracking-based method can not only effectively reduce false detections by analyzing the historical trajectory of personnel, but also maintain a high detection stability in the face of special situations such as occlusion.

[0004] Nevertheless, traditional regional intrusion detection based on target tracking still faces some challenges. For example, when multiple targets are close, relying solely on position information for target matching may lead to incorrect associations, which in turn affects long-term stable tracking. In order to solve this problem, an improved solution needs to be proposed. Summary of the invention

[0005] In view of the deficiencies of the prior art, the present invention proposes a regional intrusion detection method based on multi-target tracking.

[0006] The first aspect of the present invention discloses a regional intrusion detection method based on multi-target tracking, comprising:

[0007] S1: Obtain a video stream from a monitoring device, and collect discrete continuous image frames from the video stream at intervals;

[0008] S2: Setting a polygonal monitoring area, where the edge of the image frame where the polygonal monitoring area is located is set as the boundary of human intrusion;

[0009] S3: preprocessing the image frames, including reducing the image width and height in proportion to each image frame, filling black pixels at the edges of the image, adjusting the image size, and normalizing the image to obtain an input image;

[0010] S4: converting the input image into training image data, and passing it into the single-step multi-frame detector for model parameter training to obtain an optimized detection model;

[0011] S5: inputting the input image into the single-step multi-frame detector, performing person detection and feature extraction, obtaining a detected bounding box, and locating and outputting a specific position of the person in the input image;

[0012] S6: Perform multi-target tracking, determine the relationship between the personnel and the polygonal monitoring area, obtain the personnel trajectory, personnel intrusion situation, and calculate the number of personnel in the polygonal monitoring area.

[0013] In an optional embodiment, the preprocessing of the image frames includes performing a processing of reducing the image width and height in proportion, filling the image edge with black pixels, adjusting the image size, and normalizing the image for each of the image frames to obtain an input image, including:

[0014] Each image frame is reduced in width and height in equal proportion, and black pixels are filled at the edges of the image frame, and the image size of the image frame is adjusted to 640×640. Then, the image frame is normalized so that the value of each pixel of the image frame is between 0 and 1. The normalized pixel p' of the i-th row and j-th column of the image frame is i,j , defined as Among them, p i,j represents the original pixel at row i and column j, p m represents the average pixel value of the scene, σ m Represents the standard deviation of scene pixels.

[0015] In an optional embodiment, the step of converting the input image into training image data and passing the data into a single-step multi-frame detector for model parameter training to obtain an optimized detection model includes:

[0016] S41: Divide the input image into a training set, a validation set and a test set in a ratio of 8:1:1, optimize the parameters of the detection model by fitting the training set, the validation set is used to evaluate the effect of the detection model during the training process, and the test set is used to test the performance of the detection model after the training is completed;

[0017] S42: Calculate the training loss of the detection model, where the training loss includes category loss and position loss, where the category loss is the difference loss between the predicted category and the true category in the detection model, and the position loss is the position loss between the predicted box and the actual bounding box in the detection model;

[0018] S43: The detection model is optimized and trained by the stochastic gradient descent method, and the learning rate warm-up method is used to dynamically adjust the learning rate of the detection model.

[0019] In an optional embodiment, the single-step multi-box detector includes a feature extractor and a detection head, the feature extractor adopts a residual module, including a main branch, a residual branch and an activation function, the main branch consists of a convolution layer, a batch normalization and a linear rectification function, and performs the following steps:

[0020] S51: The convolution layer extracts a feature map from the input image through weight matrices of 3×3 and 1×1;

[0021] S52: The batch normalization performs normalization processing on the feature map, and the formula is: Where f in Input feature map, f out is the output feature map, ∈ is 10 -6 , γ is the scaling factor, β is the offset factor, μ is the mean of the input features, σ 2 is the variance of the input feature;

[0022] S53: The calculation formula of the linear rectification function is as follows:

[0023] S54: Connecting the input feature map to the output feature map through the residual branch;

[0024] S55: adding the results of the main branch and the residual branch and then connecting them with the activation function to form a residual structure;

[0025] The detection head consists of a pooling layer and a convolutional layer, performing the following steps:

[0026] S55: the pooling layer uses average pooling with a window size of 2×2 to reduce the resolution of the output feature map to half of the input feature map;

[0027] S56: The convolution layer uses 3×3 convolution to fuse adjacent pixels on the output feature map, and then adjusts the number of feature map channels to 6n through 1×1 convolution, where n represents the number of prediction boxes at each pixel position in the output feature map, and 6 represents the horizontal coordinate of the center point of each prediction box, the vertical coordinate of the center point, the width, the height, the probability of belonging to the foreground, and the probability of belonging to the background.

[0028] In an optional embodiment, the performing of multi-target tracking, determining the relationship between a person and the polygonal monitoring area, obtaining the trajectory of the person, the intrusion of the person, and calculating the number of people in the polygonal monitoring area include:

[0029] S61: predicting the bounding box state of the current image frame according to the bounding box state of the previous image frame through Kalman filtering;

[0030] S62: intercepting a person image in the current image frame bounding box predicted in S61, intercepting a person image in the bounding box detected in S5, and inputting them into a person re-identification model to obtain a predicted person feature and a detected person feature vector;

[0031] S63: The pedestrian re-identification model adopts a residual network, including a residual module and an average pooling layer, and the average pooling layer achieves image compression by downsampling the output feature map;

[0032] S64: Calculate the matching cost matrix to obtain the similarity between the predicted person characteristics and the detected person characteristics;

[0033] S65: Match the detection personnel and the prediction personnel by using the Jock-Wolgent algorithm, and calculate the matching relationship between the two;

[0034] S66: Update the prediction value according to the matching result;

[0035] S67: Calculate the position of the person in the current image frame and update the movement trajectory;

[0036] S68: Detect and calculate the intrusion of people and their trajectories in the polygonal monitoring area.

[0037] In an optional embodiment, in S61, predicting the bounding box state of the current image frame according to the bounding box state of the previous image frame by using Kalman filtering includes:

[0038] The bounding box state vector x is defined as: in, Respectively represent the horizontal coordinate, vertical coordinate, width, height, horizontal coordinate change rate, vertical coordinate change rate, width change rate, and height change rate of the center point of the bounding box;

[0039] The state vector prediction equation is: in, represents the state vector of the kth frame predicted based on the k-1th frame, F k is the state transfer matrix of the kth frame, is the optimal estimate for the k-1th frame.

[0040] In an optional embodiment, in S64, calculating the matching cost matrix includes:

[0041] Calculate the cosine similarity between the predicted personnel features and the detected personnel features. The formula is: Among them, S i,j is the cosine similarity between the i-th identified person and the j-th predicted person, represents the i-th identification feature, represents the jth predicted personnel feature, ||·|| represents the modulus of the vector;

[0042] The matching cost of the i-th identified person and the j-th predicted person is cos i,j Defined as: cos i,j =1-S i,j , the matching cost is calculated for each combination of the identified personnel and the predicted personnel, and the matching cost matrix is ​​obtained.

[0043] In an optional embodiment, in S66, updating the prediction value according to the matching result includes:

[0044] The Kalman filter balances the influence of the predicted data and the observed data to obtain the Kalman gain matrix, which is used to determine the weight of the observed data when updating the state estimate. The Kalman gain matrix K of the kth frame is k The calculation formula is: Among them, P k|k-1 is the predicted error covariance matrix, H k is the observation matrix of the kth frame, R k is the measurement noise covariance matrix;

[0045] The state vector of the kth frame corrected by the measured value for:

[0046] In an optional embodiment, in S68, the detecting and calculating of the intrusion of personnel and the trajectory of personnel in the polygonal monitoring area includes:

[0047] S681: judging the relationship between the position of the motion trajectory in S67 in the current input image and the polygonal monitoring area in S2 by using the ray method; when the motion trajectory of the person is located inside the polygonal monitoring area in the current input image, it is judged as an area intrusion and an alarm signal is output;

[0048] S682: Calculate the number of people in the polygonal monitoring area. When the movement trajectory of a person is within the polygonal monitoring area in the current image frame and outside the polygonal monitoring area in the previous image frame, the number of people in the area is increased by 1, otherwise it is decreased by 1.

[0049] The second aspect of the present invention discloses a regional intrusion detection system based on multi-target tracking, comprising:

[0050] Edge computing monitoring module, used to obtain video streams, video encoding, perform algorithm reasoning, and output alarm signals;

[0051] A signal transceiver module, used to receive input signals from the edge computing monitoring device and output control signals;

[0052] An alarm module, used for giving an alarm according to the control signal;

[0053] A switch, used to forward the images acquired by the edge computing monitoring module;

[0054] The display is used to display the images acquired by the edge computing monitoring module, the movement trajectory of personnel, the polygonal monitoring area and the number of personnel in the polygonal monitoring area.

[0055] Compared with the prior art, the present invention has the following advantages:

[0056] The present invention uses deep learning technology to extract the characteristic information of the target from the image for precise matching, accurately identifying and continuously tracking each monitored individual. Through autonomous in-depth learning and training model recognition, the model can also be deployed to edge monitoring equipment to achieve low-cost and high-efficiency regional intrusion detection solutions. Therefore, this solution can not only improve the accuracy and stability of regional intrusions, but also greatly reduce the deployment and operation and maintenance costs of the system, providing users with a more advanced and reliable security option. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 It is a flow chart of the regional intrusion detection method based on multi-target tracking of the present invention;

[0059] Figure 2It is a structural schematic diagram of a regional intrusion detection system based on multi-target tracking of the present invention;

[0060] Figure 3 A schematic diagram of a regional intrusion detection device based on multi-target tracking according to the present invention;

[0061] Figure 4 A schematic diagram of a computer-readable storage medium according to the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. Based on the embodiments provided in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present application.

[0063] Obviously, the drawings described below are only some examples or embodiments of the present application. For ordinary technicians in this field, the present application can also be applied to other similar scenarios based on these drawings without creative work. In addition, it can also be understood that although the efforts made in this development process may be complicated and lengthy, for ordinary technicians in this field related to the content disclosed in this application, some changes in design, manufacturing or production based on the technical content disclosed in this application are just conventional technical means, and should not be understood as insufficient content disclosed in this application.

[0064] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.

[0065] Unless otherwise defined, the technical terms or scientific terms involved in this application should be understood by people with ordinary skills in the technical field to which this application belongs. The words "one", "a", "a", "the" and the like involved in this application do not indicate a quantitative limitation, and may represent the singular or plural. The terms "include", "comprise", "have" and any of their variations involved in this application are intended to cover non-exclusive inclusions; for example, a process, method, system, product or device that includes a series of steps or modules (units) is not limited to the listed steps or units, but may also include steps or units that are not listed, or may also include other steps or units inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the objects before and after are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0066] See also Figure 1 The embodiment of the present invention discloses a regional intrusion detection method based on multi-target tracking, the method comprising:

[0067] S1: Obtain a video stream from a monitoring device, and collect discrete continuous image frames from the video stream at intervals; this step is used to collect images of the monitoring area;

[0068] S2: Setting a polygonal monitoring area, where the edge of the image frame where the polygonal monitoring area is located is set as the boundary of human intrusion; specifically, the polygonal monitoring area is a small part of the module that is subject to key monitoring in the monitoring device, and whether a person has invaded the boundary is determined by the part;

[0069] S3: preprocessing the image frames, including reducing the image width and height in proportion to each image frame, filling black pixels at the edges of the image, adjusting the image size, and normalizing the image to obtain an input image;

[0070] S4: converting the input image into training image data, passing it into a single-step multi-frame detector for model parameter training, and obtaining an optimized detection model; specifically, it is necessary to first train the input image, pass it into the single-step multi-frame detector for model parameter training, until the optimized detection model is obtained; here, the optimized detection model is a single-step multi-frame detector;

[0071] S5: Input the input image into the single-step multi-frame detector, perform personnel detection and feature extraction, obtain the detected bounding box, locate and output the specific position of the person in the input image; specifically, extract the feature map and the bounding box state of the polygonal monitoring area and the detected bounding box from the input image, and capture a local image with the target person from the image. This step is used to locate the local area on the image where the person is located;

[0072] S6: Perform multi-target tracking, determine the relationship between the personnel and the polygonal monitoring area, obtain the personnel trajectory, personnel intrusion situation, and calculate the number of personnel in the polygonal monitoring area.

[0073] The present invention uses deep learning technology to extract the characteristic information of the target from the image for precise matching, accurately identifying and continuously tracking each monitored individual. Through autonomous in-depth learning and training model recognition, the model can also be deployed to edge monitoring equipment to achieve low-cost and high-efficiency regional intrusion detection solutions. Therefore, this solution can not only improve the accuracy and stability of regional intrusions, but also greatly reduce the deployment and operation and maintenance costs of the system, providing users with a more advanced and reliable security option.

[0074] In an optional embodiment, the preprocessing of the image frames includes performing a processing of reducing the image width and height in proportion, filling the image edge with black pixels, adjusting the image size, and normalizing the image for each of the image frames to obtain an input image, including:

[0075] Each image frame is reduced in width and height in equal proportion, and black pixels are filled at the edges of the image frame, and the image size of the image frame is adjusted to 640×640. Then, the image frame is normalized so that the value of each pixel of the image frame is between 0 and 1. The normalized pixel p' of the i-th row and j-th column of the image frame is i,j , defined as Among them, p i,j represents the original pixel at row i and column j, p m represents the average pixel value of the scene, σ m Represents the standard deviation of scene pixels. The calculation result is the input image. Specifically, p m and σ m The value is calculated by collecting a large number of images of the scene (ie, image frames captured in a video stream containing the scene) and then finding the average value of the pixels.

[0076] In an optional embodiment, the step of converting the input image into training image data and passing the data into a single-step multi-frame detector for model parameter training to obtain an optimized detection model includes:

[0077] S41: the input image obtained by preprocessing in S3 is divided into a training set, a validation set and a test set in a ratio of 8:1:1, the detection model is optimized by fitting the training set, the validation set is used to evaluate the effect of the detection model during the training process, and the test set is used to test the performance of the detection model after the training is completed;

[0078] S42: Calculate the training loss of the detection model, wherein the training loss includes a category loss and a position loss, wherein the category loss is the difference loss between the predicted category and the true category in the detection model, and the position loss is the position loss between the predicted box and the actual bounding box in the detection model; specifically, the category loss is calculated using cross entropy; and the position loss is calculated using intersection-over-union ratio;

[0079] S43: The detection model is optimized and trained through the stochastic gradient descent method, and the learning rate warm-up method is used to dynamically adjust the learning rate of the detection model. This method can ensure the stability in the initial stage of training.

[0080] In an optional embodiment, the single-step multi-box detector includes a feature extractor and a detection head, the feature extractor adopts a residual module, including a main branch, a residual branch and an activation function, the main branch consists of a convolution layer, a batch normalization and a linear rectification function, and performs the following steps:

[0081] S51: The convolution layer extracts a feature map from the input image through weight matrices of 3×3 and 1×1;

[0082] S52: The batch normalization performs normalization processing on the feature map, and the formula is: Where f in Input feature map, f out is the output feature map, ∈ is 10 -6 , γ is the scaling factor, β is the offset factor, μ is the mean of the input features, σ 2 is the variance of the input feature;

[0083] S53: The calculation formula of the linear rectification function is as follows: This step is used to increase the nonlinear fitting ability of the detection model;

[0084] S54: Connecting the input feature map to the output feature map through the residual branch;

[0085] S55: adding the results of the main branch and the residual branch and then connecting them with the activation function to form a residual structure;

[0086] Specifically, the feature extractor is used to extract an input feature map from the input image and perform normalization processing and calculation, and the detection head is used to detect the position of the person in the input image, as follows:

[0087] The detection head consists of a pooling layer and a convolutional layer, performing the following steps:

[0088] S55: the pooling layer uses average pooling with a window size of 2×2 to reduce the resolution of the output feature map to half of the input feature map;

[0089] S56: The convolution layer uses 3×3 convolution to fuse adjacent pixels on the output feature map, and then adjusts the number of feature map channels to 6n through 1×1 convolution, where n represents the number of prediction boxes at each pixel position in the output feature map, and 6 represents the horizontal coordinate of the center point of each prediction box, the vertical coordinate of the center point, the width, the height, the probability of belonging to the foreground, and the probability of belonging to the background.

[0090] In an optional embodiment, the performing of multi-target tracking, determining the relationship between a person and the polygonal monitoring area, obtaining the trajectory of the person, the intrusion of the person, and calculating the number of people in the polygonal monitoring area include:

[0091] S61: predicting the bounding box state of the current image frame according to the bounding box state of the previous image frame through Kalman filtering;

[0092] In an optional embodiment, in S61, predicting the bounding box state of the current image frame according to the bounding box state of the previous image frame by using Kalman filtering includes:

[0093] The bounding box state vector x is defined as: in, Respectively represent the horizontal coordinate, vertical coordinate, width, height, horizontal coordinate change rate, vertical coordinate change rate, width change rate, and height change rate of the center point of the bounding box;

[0094] The state vector prediction equation is: in, represents the state vector of the kth frame predicted based on the k-1th frame, F k is the state transfer matrix of the kth frame, is the optimal estimate for the k-1th frame.

[0095] This step is used to predict the bounding box.

[0096] S62: intercepting a person image in the current image frame bounding box predicted in S61, intercepting a person image in the bounding box detected in S5, and inputting them into a person re-identification model to obtain a predicted person feature and a detected person feature vector;

[0097] Furthermore, this step can also use the center point distance between the predicted person bounding box and the detected person bounding box as the matching cost. This method can be faster in information response time and improve the efficiency of recognition and tracking.

[0098] S63: The pedestrian re-identification model adopts a residual network, including a residual module and an average pooling layer, and the average pooling layer achieves image compression by downsampling the output feature map; specifically, the image block can be replaced by calculating the average value to obtain image compression;

[0099] S64: Calculate the matching cost matrix to obtain the similarity between the predicted person characteristics and the detected person characteristics;

[0100] In an optional embodiment, in S64, calculating the matching cost matrix includes:

[0101] Calculate the cosine similarity between the predicted personnel features and the detected personnel features. The formula is: Among them, S i,j is the cosine similarity between the i-th identified person and the j-th predicted person, represents the i-th identification feature, represents the j-th predicted personnel feature, ||·|| represents the modulus of the vector;

[0102] The matching cost of the i-th identified person and the j-th predicted person is cos i,j Defined as: cos i,j =1-S i,j , the matching cost is calculated for each combination of the identified personnel and the predicted personnel, and the matching cost matrix is ​​obtained.

[0103] S65: Match the detection person and the prediction person by using the Jock-Walgent algorithm, and calculate the matching relationship between the two; specifically, the Jock-Walgent algorithm continuously creates potential matching points without affecting the optimal solution, and finally obtains the matching relationship with the minimum total cost;

[0104] S66: Update the prediction value according to the matching result;

[0105] In an optional embodiment, in S66, updating the prediction value according to the matching result includes:

[0106] The Kalman filter balances the influence of the predicted data and the observed data to obtain the Kalman gain matrix, which is used to determine the weight of the observed data when updating the state estimate. The Kalman gain matrix K of the kth frame is k The calculation formula is: Among them, P k|k-1 is the predicted error covariance matrix, H k is the observation matrix of the kth frame, R k is the measurement noise covariance matrix;

[0107] The state vector of the kth frame corrected by the measured value for:

[0108] S67: Calculate the position of the person in the current image frame and update the movement trajectory;

[0109] S68: Detect and calculate the intrusion of people and their trajectories in the polygonal monitoring area.

[0110] In an optional embodiment, in S68, detecting and calculating the intrusion of personnel and the trajectory of personnel in the polygonal monitoring area includes:

[0111] S681: judging the relationship between the position of the motion trajectory in S67 in the current input image and the polygonal monitoring area in S2 by using the ray method; when the motion trajectory of the person is located inside the polygonal monitoring area in the current input image, it is judged as an area intrusion and an alarm signal is output;

[0112] S682: Calculate the number of people in the polygonal monitoring area. When the movement trajectory of a person is within the polygonal monitoring area in the current image frame and outside the polygonal monitoring area in the previous image frame, the number of people in the area is increased by 1, otherwise it is decreased by 1.

[0113] The present invention proposes a regional intrusion detection method based on multi-target tracking. The method is an innovative solution that uses deep learning technology to extract the characteristic information of the target from the image for precise matching. Even when multiple targets are closely adjacent, the method can accurately identify and continuously track each individual; in addition, the present invention also supports seamless tracking across cameras, that is, it can correctly identify the same target between different surveillance cameras, further enhancing the robustness and practicality of the system. In terms of deployment, the design of the present invention fully considers cost-effectiveness and ease of implementation. It does not need to rely on expensive high-performance computing resources, such as dedicated graphics card servers, but can be easily integrated into existing monitoring systems, realizing a low-cost, high-efficiency regional intrusion detection solution. In summary, the present invention not only improves the accuracy and stability of regional intrusion detection, but also greatly reduces the deployment and operation and maintenance costs of the system, providing users with a more advanced and reliable security option.

[0114] See also Figure 2 The second aspect of the present invention discloses a regional intrusion detection system based on multi-target tracking, comprising:

[0115] Edge computing monitoring module, used to obtain video streams, video encoding, perform algorithm reasoning, and output alarm signals; specifically, this module can include multiple cameras and support cross-camera information linkage;

[0116] A signal transceiver module, used to receive input signals from the edge computing monitoring device and output control signals; specifically, the module can use a terminal board;

[0117] An alarm module is used to give an alarm according to the control signal; specifically, a green light is on in a normal state, and a red light is on to give an alarm when there is an intrusion;

[0118] A switch, used to forward the images acquired by the edge computing monitoring module;

[0119] The display is used to display the images acquired by the edge computing monitoring module, the movement trajectory of personnel, the polygonal monitoring area and the number of personnel in the polygonal monitoring area.

[0120] In the solution of the present invention, terminal-side deployment is achieved, with flexible installation and low cost; it provides functions such as picture monitoring, motion trajectory recording, intrusion alarm, and personnel counting, and users can easily obtain monitoring data; it can run in real time, with both accuracy and efficiency.

[0121] See also Figure 3 The third aspect of the present invention discloses a regional intrusion detection device based on multi-target tracking, comprising:

[0122] at least one processor, and

[0123] a memory communicatively connected to the at least one processor; wherein,

[0124] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the regional intrusion detection method based on multi-target tracking as described in any one of the first aspects of the present invention.

[0125] The computer device may be a terminal, and the computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a regional intrusion detection method based on multi-target tracking is implemented. The display screen of the computer device may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a key, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse, etc.

[0126] See also Figure 4 The fourth aspect of the present invention discloses a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the regional intrusion detection method based on multi-target tracking as described in any one of the first aspect of the present invention.

[0127] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of the regional intrusion detection method based on multi-target tracking. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0128] Alternatively, if the above modules of the present invention are implemented in the form of software function modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0129] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A regional intrusion detection method based on multi-target tracking, characterized in that: The method comprises: S1: Obtain a video stream from a monitoring device, and collect discrete continuous image frames from the video stream at intervals; S2: Setting a polygonal monitoring area, where the edge of the image frame where the polygonal monitoring area is located is set as the boundary of human intrusion; S3: preprocessing the image frames, including reducing the image width and height in proportion to each image frame, filling black pixels at the edges of the image, adjusting the image size, and normalizing the image to obtain an input image; S4: converting the input image into training image data, and passing it into the single-step multi-frame detector for model parameter training to obtain an optimized detection model; S5: inputting the input image into the single-step multi-frame detector, performing person detection and feature extraction, obtaining a detected bounding box, and locating and outputting a specific position of the person in the input image; S6: Perform multi-target tracking, determine the relationship between the personnel and the polygonal monitoring area, obtain the personnel trajectory, personnel intrusion situation, and calculate the number of personnel in the polygonal monitoring area.

2. The method for regional intrusion detection based on multi-target tracking according to claim 1 is characterized in that: The preprocessing of the image frames includes performing a processing of reducing the image width and height in proportion, filling the image edge with black pixels, adjusting the image size, and normalizing the image for each image frame to obtain an input image, including: Each image frame is reduced in width and height in equal proportion, and black pixels are filled at the edges of the image frame, and the image size of the image frame is adjusted to 640×640. Then, the image frame is normalized so that the value of each pixel of the image frame is between 0 and 1. The normalized pixel p' of the i-th row and j-th column of the image frame is i,j , defined as Among them, p i,j represents the original pixel at row i and column j, p m represents the average pixel value of the scene, σ m Represents the standard deviation of scene pixels.

3. The regional intrusion detection method based on multi-target tracking according to claim 2 is characterized in that: The step of converting the input image into training image data and passing it into the single-step multi-frame detector for model parameter training to obtain an optimized detection model includes: S41: Divide the input image into a training set, a validation set and a test set in a ratio of 8:1:1, optimize the parameters of the detection model by fitting the training set, the validation set is used to evaluate the effect of the detection model during the training process, and the test set is used to test the performance of the detection model after the training is completed; S42: Calculate the training loss of the detection model, where the training loss includes category loss and position loss, where the category loss is the difference loss between the predicted category and the true category in the detection model, and the position loss is the position loss between the predicted box and the actual bounding box in the detection model; S43: The detection model is optimized and trained by the stochastic gradient descent method, and the learning rate warm-up method is used to dynamically adjust the learning rate of the detection model.

4. The method for regional intrusion detection based on multi-target tracking according to claim 3 is characterized in that: The single-step multi-frame detector includes a feature extractor and a detection head. The feature extractor adopts a residual module, including a main branch, a residual branch and an activation function. The main branch consists of a convolution layer, a batch normalization and a linear rectification function, and performs the following steps: S51: The convolution layer extracts a feature map from the input image through weight matrices of 3×3 and 1×1; S52: The batch normalization performs normalization processing on the feature map, and the formula is: Where f in Input feature map, f out is the output feature map, ∈ is 10 -6 , γ is the scaling factor, β is the offset factor, μ is the mean of the input features, σ 2 is the variance of the input feature; S53: The calculation formula of the linear rectification function is as follows: S54: Connecting the input feature map to the output feature map through the residual branch; S55: adding the results of the main branch and the residual branch and then connecting them with the activation function to form a residual structure; The detection head consists of a pooling layer and a convolutional layer, performing the following steps: S55: the pooling layer uses average pooling with a window size of 2×2 to reduce the resolution of the output feature map to half of the input feature map; S56: The convolution layer uses 3×3 convolution to fuse adjacent pixels on the output feature map, and then adjusts the number of feature map channels to 6n through 1×1 convolution, where n represents the number of prediction boxes at each pixel position in the output feature map, and 6 represents the horizontal coordinate of the center point of each prediction box, the vertical coordinate of the center point, the width, the height, the probability of belonging to the foreground, and the probability of belonging to the background.

5. The method for regional intrusion detection based on multi-target tracking according to claim 4 is characterized in that: The multi-target tracking, determining the relationship between a person and the polygonal monitoring area, obtaining the person's trajectory, the person's intrusion situation, and calculating the number of people in the polygonal monitoring area include: S61: predicting the bounding box state of the current image frame according to the bounding box state of the previous image frame through Kalman filtering; S62: intercepting a person image in the current image frame bounding box predicted in S61, intercepting a person image in the bounding box detected in S5, and inputting them into a person re-identification model to obtain a predicted person feature and a detected person feature vector; S63: The pedestrian re-identification model adopts a residual network, including a residual module and an average pooling layer, and the average pooling layer achieves image compression by downsampling the output feature map; S64: Calculate the matching cost matrix to obtain the similarity between the predicted person characteristics and the detected person characteristics; S65: Match the detection personnel and the prediction personnel by using the Jock-Wolgent algorithm, and calculate the matching relationship between the two; S66: Update the prediction value according to the matching result; S67: Calculate the position of the person in the current image frame and update the movement trajectory; S68: Detect and calculate the intrusion of people and their trajectories in the polygonal monitoring area.

6. The method for regional intrusion detection based on multi-target tracking according to claim 5 is characterized in that: In S61, predicting the bounding box state of the current image frame according to the bounding box state of the previous image frame by using Kalman filtering includes: The bounding box state vector x is defined as: Among them, x c ,y c ,w,h, Respectively represent the horizontal coordinate, vertical coordinate, width, height, horizontal coordinate change rate, vertical coordinate change rate, width change rate, and height change rate of the center point of the bounding box; The state vector prediction equation is: in, represents the state vector of the kth frame predicted based on the k-1th frame, F k is the state transfer matrix of the kth frame, is the optimal estimate for the k-1th frame.

7. The method for regional intrusion detection based on multi-target tracking according to claim 6 is characterized in that: In S64, the calculation of the matching cost matrix includes: Calculate the cosine similarity between the predicted personnel features and the detected personnel features. The formula is: Among them, S i,j is the cosine similarity between the i-th identified person and the j-th predicted person, represents the i-th identification feature, represents the jth predicted personnel feature, ||·|| represents the modulus of the vector; The matching cost of the i-th identified person and the j-th predicted person is cos i,j Defined as: cos i,j =1-S i,j , the matching cost is calculated for each combination of the identified personnel and the predicted personnel, and the matching cost matrix is ​​obtained.

8. The method for regional intrusion detection based on multi-target tracking according to claim 7 is characterized in that: In S66, updating the prediction value according to the matching result includes: The Kalman filter balances the influence of the predicted data and the observed data to obtain the Kalman gain matrix, which is used to determine the weight of the observed data when updating the state estimate. The Kalman gain matrix K of the kth frame is k The calculation formula is: Among them, P k|k-1 is the predicted error covariance matrix, H k is the observation matrix of the kth frame, R k is the measurement noise covariance matrix; The state vector of the kth frame corrected by the measured value for:

9. The method for regional intrusion detection based on multi-target tracking according to claim 8, characterized in that: In S68, the detecting and calculating of the intrusion of people and the trajectory of people in the polygonal monitoring area includes: S681: judging the relationship between the position of the motion trajectory in S67 in the current input image and the polygonal monitoring area in S2 by using the ray method; when the motion trajectory of the person is located inside the polygonal monitoring area in the current input image, it is judged as an area intrusion and an alarm signal is output; S682: Calculate the number of people in the polygonal monitoring area. When the movement trajectory of a person is within the polygonal monitoring area in the current image frame and outside the polygonal monitoring area in the previous image frame, the number of people in the area is increased by 1, otherwise it is decreased by 1.

10. A regional intrusion detection system based on multi-target tracking, characterized in that: include: Edge computing monitoring module, used to obtain video streams, video encoding, perform algorithm reasoning, and output alarm signals; A signal transceiver module, used to receive input signals from the edge computing monitoring device and output control signals; An alarm module, used for giving an alarm according to the control signal; A switch, used to forward the images acquired by the edge computing monitoring module; The display is used to display the images acquired by the edge computing monitoring module, the movement trajectory of personnel, the polygonal monitoring area and the number of personnel in the polygonal monitoring area.

Images