Personnel intrusion detection method and system based on target detection algorithm

Abstract

The invention provides a personnel intrusion detection method and system based on a target detection algorithm, and relates to the technical field of safety monitoring, and the method comprises the steps: firstly, defining an intrusion attention region and a judgment rule based on a panoramic layout image of a to-be-monitored scene, then obtaining a real-time monitoring video stream, and extracting a to-be-detected video frame sequence; and calling a pre-trained personnel target detection algorithm to carry out frame-by-frame detection on the video frame sequence to obtain personnel target detection information, carrying out region association behavior characteristic analysis based on an intrusion attention region and the detection information, finally carrying out intrusion judgment according to behavior characteristics and judgment rules, and generating a personnel intrusion detection result. Therefore, the accuracy and reliability of personnel intrusion detection can be improved, and misjudgment and missed judgment are reduced.

Description

Technical Field

[0001] This invention relates to the field of security monitoring technology, and more specifically, to a method and system for detecting personnel intrusion based on a target detection algorithm. Background Technology

[0002] In various security-critical scenarios, such as around important facilities, private property, and specific work areas, intrusion detection is a crucial element in ensuring security. Traditional intrusion detection methods primarily rely on manual patrols or simple physical security devices. Manual patrols not only consume significant manpower but also struggle to achieve continuous, real-time monitoring, often resulting in blind spots and undetected intrusions. Simple physical security devices, such as fences and access control systems, only provide limited deterrence and cannot effectively detect or warn against intrusion attempts such as climbing or vandalism.

[0003] With the development of video surveillance technology, video-based intrusion detection methods have been gradually applied. However, most existing video-based detection methods simply set up fixed detection areas, and when a person enters the area, it is considered an intrusion, lacking in-depth analysis of the person's behavioral characteristics. This crude detection method is prone to misjudgments; for example, legitimate personnel moving normally near the detection area may be mistakenly identified as intruders. Furthermore, existing methods fail to adequately consider the dynamic relationship between people and the detection area, such as movement trends and dwell time, resulting in low accuracy and reliability, and failing to meet the needs of practical security protection. Summary of the Invention

[0004] In view of the aforementioned problems, and in conjunction with the first aspect of the present invention, embodiments of the present invention provide a method for personnel intrusion detection based on a target detection algorithm, the method comprising: Based on the panoramic layout image of the scene to be monitored, the intrusion concern area and intrusion judgment reference rules of the scene to be monitored are defined. The intrusion concern area is a specific spatial range in the scene to be monitored that needs to be detected for personnel intrusion. The intrusion judgment reference rules are a set of behavioral judgment criteria used to determine whether personnel behavior constitutes intrusion. Acquire the real-time monitoring video stream of the scene to be monitored, and extract the sequence of video frames to be detected with continuous temporal correlation from the real-time monitoring video stream; A pre-trained human target detection algorithm is invoked to perform frame-by-frame human target detection processing on the video frame sequence to be detected, thereby obtaining human target detection information in the video frame sequence to be detected. The human target detection information includes the position information of the human target in the video frame and the appearance feature information of the human target. Based on the intrusion concern area and the personnel target detection information, the personnel target in the video frame sequence to be detected is analyzed for regional association behavior features to obtain the regional association behavior features of the personnel target. The regional association behavior features are feature information describing the spatial positional relationship and movement trend between the personnel target and the intrusion concern area. Based on the regional association behavior characteristics and the intrusion determination reference rules, the behavior of personnel targets is processed for intrusion determination, and personnel intrusion detection results are generated. The personnel intrusion detection results are used to indicate whether there is personnel intrusion behavior that conforms to the intrusion determination reference rules in the monitored scene.

[0005] In another aspect, embodiments of the present invention also provide a personnel intrusion detection system based on a target detection algorithm, including a processor and a machine-readable storage medium connected to the processor. The machine-readable storage medium is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the machine-readable storage medium to implement the above-described method.

[0006] Based on the above, this embodiment of the invention defines the intrusion interest area and intrusion judgment reference rules based on the panoramic layout image of the scene to be monitored, obtains the real-time monitoring video stream of the scene to be monitored and extracts the video frame sequence to be detected, ensuring the real-time and continuous nature of the detection data, enabling timely capture of dynamic human behavior, calling a pre-trained human target detection algorithm to perform frame-by-frame human target detection processing, and obtaining human target detection information including location and appearance features. Based on the intrusion interest area and human target detection information, regional association behavior feature analysis is performed, which can deeply explore the spatial positional relationship and movement trend between human targets and the intrusion interest area, and obtain more representative regional association behavior features. Finally, intrusion judgment processing is performed according to the regional association behavior features and intrusion judgment reference rules. The generated human intrusion detection results can accurately indicate whether there is human intrusion behavior that conforms to the rules in the scene to be monitored, effectively improving the accuracy and reliability of human intrusion detection and reducing the occurrence of false positives and false negatives. Attached Figure Description

[0007] Figure 1 This is a schematic diagram of the execution flow of the personnel intrusion detection method based on the target detection algorithm provided in the embodiment of the present invention.

[0008] Figure 2 This is a schematic diagram of exemplary hardware and software components of a personnel intrusion detection system based on a target detection algorithm provided in an embodiment of the present invention. Detailed Implementation

[0009] The present invention will now be described in detail with reference to the accompanying drawings. Figure 1This is a flowchart illustrating a personnel intrusion detection method based on a target detection algorithm according to an embodiment of the present invention. The following is a detailed description of the personnel intrusion detection method based on the target detection algorithm.

[0010] Step S110: Based on the panoramic layout image of the scene to be monitored, define the intrusion concern area and intrusion judgment reference rules of the scene to be monitored. The intrusion concern area is a specific spatial range in the scene to be monitored that needs to be detected for personnel intrusion. The intrusion judgment reference rules are a set of behavioral judgment criteria used to determine whether personnel behavior constitutes intrusion.

[0011] Step S111: Obtain a panoramic layout image of the scene to be monitored, wherein the panoramic layout image is an image that fully presents the spatial structure of the scene to be monitored.

[0012] At power work sites, acquiring panoramic images requires covering all areas potentially at risk of intrusion. First, the installation locations of the monitoring equipment must be determined, ensuring there are no blind spots, especially in critical areas such as high-voltage equipment areas and cable tunnels. High-definition cameras with wide-angle capabilities should be selected, with lenses capable of covering a sufficiently wide field of view to reduce the number of devices required and ensure the integrity of the image stitching.

[0013] During shooting, camera parameters need to be adjusted, including white balance and exposure time, to adapt to the lighting conditions on site. Whether it's a bright daytime with ample natural light or a dimly lit night, image clarity and color accuracy must be ensured. For nighttime scenes, the camera's night vision function can be activated, using infrared illumination to ensure image details are discernible.

[0014] After shooting, the images captured by each camera are preprocessed. Preprocessing includes image denoising, which uses a Gaussian filtering algorithm to smooth noise in the image and reduce the impact of sensor noise on image quality; and image enhancement, which adjusts contrast and brightness to make the edges of objects in the image clearer, facilitating subsequent stitching processing.

[0015] Image stitching employs feature point matching technology. First, feature points are extracted from each image; these feature points can be unique elements such as the edges of devices or markings on the ground. Then, the similarity of feature points across different images is calculated to find matching feature point pairs, which are used to determine the relative positional relationships between the images.

[0016] Based on their relative positions, the images undergo geometric transformations, projecting multiple images onto the same coordinate system. In overlapping areas, a fade-in / fade-out fusion algorithm is used to ensure a natural transition and eliminate noticeable stitching artifacts. The final panoramic layout image must fully represent all areas of the power operation site, including equipment distribution, pathways, fence locations, and other spatial structural information.

[0017] Step S112: In the panoramic layout image, at least one closed polygonal region is defined as the initial region of interest by manual annotation or preset region generation rules. The boundary of the initial region of interest is determined by the vertex coordinates of multiple vertices in the pixel coordinate system of the panoramic layout image.

[0018] When manually marking initial areas of concern, this must be done by technicians familiar with safety regulations for power work sites. Using a dedicated image annotation tool, the technicians open a panoramic layout image and, based on the on-site safety risk assessment, determine areas requiring focused monitoring, such as the area around high-voltage switchgear and transformer platforms.

[0019] During the annotation process, technicians click on the image with a mouse to define the vertices of the region. Each vertex must be positioned precisely on a feature point of the region's boundary, such as a fence corner or the edge of equipment. For irregularly shaped regions, the number of vertices needs to be increased to ensure that the closed polygon accurately fits the actual boundary of the region. For example, for a circular equipment area, multiple vertices can be used to approximate the circular boundary.

[0020] Preset region generation rules are applicable to regions with relatively regular shapes or fixed characteristics. For example, for a rectangular cable trench region, four vertices can be automatically generated based on its start and end coordinates in the image, forming a closed rectangular polygon. The rules need to be set in conjunction with the actual characteristics of the region, such as the length and width ratio of the region, and its relative position to surrounding equipment.

[0021] After generating the initial area of ​​interest, it needs to be validated. This involves comparing the labeled area with the actual area on site to check for omissions or mislabeling. For areas that do not meet the requirements, they should be re-labeled or the generation rules adjusted until the initial area of ​​interest accurately reflects the spatial range that needs to be monitored.

[0022] Step S113: Set regional attributes for the initial focus area. The regional attribute settings include assigning a unique regional identifier to each initial focus area, setting regional priority, and specifying the time range during which people are allowed to appear in the specified area.

[0023] When assigning area identifiers, the established coding rules must be followed to ensure that the area identifiers both reflect the area type and guarantee uniqueness. The coding rules may include the area's location information, area type code, etc. For example, the identifier for a high-voltage equipment area may include the words "high-voltage" and the area's on-site number.

[0024] Setting area priorities should be based on the severity of potential risks within the area. Risk assessments consider factors such as the voltage level of the equipment, its operating status, and the potential consequences of accidental personnel trespassing. Areas containing high-voltage live equipment are given the highest priority due to their highest risk; areas with risks but relatively minor consequences, such as passageways above cable trenches, are given lower priority.

[0025] When specifying the permitted time range for personnel, the work plan and safety management regulations for electrical operations must be consulted. For areas requiring regular maintenance, the permitted time range for personnel is the time period scheduled in the maintenance plan; for areas where unauthorized personnel are prohibited from entering, personnel are not allowed outside of the designated authorized work times. The time range must be set precisely to specific time periods, such as a specific period from morning to afternoon on a given day, and stored in a standard time format for easy system recognition and comparison.

[0026] Step S114: Determine the initial area of ​​interest with the set area attributes as the intrusion area of ​​interest, and store the area identifier, vertex coordinate set, area priority and time range limit of the intrusion area of ​​interest in the area configuration file.

[0027] Before determining the area of ​​concern for intrusion, the initial area of ​​concern with its attributes set must be reviewed again to ensure that the boundaries of the area are accurate and the attribute settings are reasonable. After the review is approved, it is officially designated as the area of ​​concern for intrusion.

[0028] The region configuration files are stored in a predefined format, such as XML or JSON, which are easy for computers to read and parse. Within the file, information about each region of interest is presented as a separate entry, including the region identifier, a set of vertex coordinates (x and y coordinates of each vertex), region priority, and allowed time range.

[0029] During storage, data validation is necessary to ensure its integrity and correct format. For example, check if vertex coordinates are complete and if the time range format meets requirements. After storage, the region configuration file must be backed up to prevent data loss. Simultaneously, file access permissions should be set, allowing only authorized administrators to modify file contents, thus ensuring data security.

[0030] Step S115: Based on the security management requirements of the scenario to be monitored, formulate intrusion judgment reference rules. The intrusion judgment reference rules include the duration threshold of personnel entering the intrusion concern area, the movement trajectory pattern of personnel in the intrusion concern area, the behavior action type of personnel in the intrusion concern area, and the interaction method between personnel and the boundary of the intrusion concern area.

[0031] When setting duration thresholds, the risk level of the area and the actual needs of the operation must be considered. For high-risk areas, such as high-voltage equipment areas, the duration threshold should be set shorter; once personnel enter and remain for more than this time, it may constitute an intrusion. For areas with relatively low risk, the threshold can be appropriately extended. At the same time, it is necessary to distinguish between the time authorized personnel spend during normal operations and the time unauthorized personnel spend to avoid misjudgments.

[0032] The development of movement trajectory patterns is based on the analysis of historical intrusion events and normal operational behaviors. Suspicious trajectory patterns include aimless wandering within the area, frequent approach to hazardous equipment, and attempts to avoid surveillance cameras. For example, in high-voltage equipment areas, if a person's trajectory repeatedly passes through the live parts of the equipment, it is considered a suspicious pattern.

[0033] The determination of the type of behavior must be based on the safety regulations for electrical work, clarifying which behaviors are prohibited and dangerous. For example, climbing equipment, touching live parts, and using open flames will all be considered behaviors that may constitute intrusion.

[0034] Among the interactions between personnel and the boundary of an area, behaviors that qualify as intrusion include climbing over fences, damaging boundary facilities, and entering through unauthorized entrances. The criteria for determining these interactions need to be specific and clear, facilitating system-wide judgment through image recognition.

[0035] Step S116: Associate the intrusion determination reference rule with the corresponding area identifier of the intrusion concern area, and store the associated intrusion determination reference rule in the rule database to form an intrusion determination reference rule set that corresponds one-to-one with the intrusion concern area.

[0036] During the association and binding process, a correspondence is established between intrusion judgment reference rules and intrusion-related areas of interest through area identifiers. Each area identifier corresponds to a complete set of rules, including duration thresholds, movement trajectory patterns, behavior action types, and boundary interaction methods.

[0037] The rules database uses a relational database architecture and features a dedicated table structure to store rule information. The tables contain fields such as region identifier, rule type, and rule content. All corresponding rules can be quickly retrieved using the region identifier.

[0038] During storage, the validity of rules must be verified to ensure there are no conflicts and that they meet security management requirements. For example, different rules for the same area must not contradict each other in their decision conditions. After storage, the rule database needs to be backed up and maintained regularly to ensure data reliability and security. Simultaneously, an update interface for rules should be provided to facilitate administrators in modifying or adding rules as needed.

[0039] Step S120: Obtain the real-time monitoring video stream of the scene to be monitored, and extract the sequence of video frames to be detected with continuous temporal correlation from the real-time monitoring video stream.

[0040] Step S121: Collect real-time monitoring video streams by deploying multiple monitoring devices in the scene to be monitored. The monitoring devices continuously capture images of the scene according to a preset sampling frame rate.

[0041] The deployment location of monitoring equipment needs to be planned based on the layout of the scene to be monitored and the distribution of areas of interest for intrusion detection. Monitoring equipment should be installed around and at the entrance of each area of ​​interest to ensure clear capture of personnel activities within and around the area. For areas with limited visibility, multiple devices can be used to capture images simultaneously, avoiding blind spots.

[0042] When setting the sampling frame rate, a balance must be struck between video smoothness and data volume. An excessively high sampling frame rate leads to excessive data volume, increasing storage and processing burdens; an excessively low frame rate may miss crucial personnel actions. Selecting an appropriate sampling frame rate based on the actual conditions of the power operation site ensures clear recording of personnel behavior while keeping the data volume within a reasonable range.

[0043] The surveillance equipment must maintain stable operation during recording. It must possess autofocus and auto-exposure functions to adapt to changes in ambient lighting conditions. Simultaneously, the equipment must have anti-interference capabilities to prevent electromagnetic interference and other factors from affecting video signal quality. The real-time monitoring video stream is transmitted to the backend processing system via a dedicated network, employing encryption technology during transmission to ensure video data security.

[0044] Step S122: Perform frame extraction processing on the real-time monitoring video stream, extracting video frames from the real-time monitoring video stream in chronological order to form an initial video frame sequence.

[0045] Frame extraction is performed by the video processing module, which reads video frame data sequentially from the real-time monitoring video stream. During extraction, a timestamp is added to each video frame, accurate to milliseconds, to record the capture time of the video frame.

[0046] The initial video frame sequence is arranged in chronological order of timestamps, forming a continuous sequence. Each video frame in the sequence contains its corresponding monitoring device identifier, timestamp, and image data. For ease of management and subsequent processing, the video frame data is stored as files in a designated directory, with filenames containing the device identifier and timestamp information.

[0047] During the extraction process, the integrity of the video frames needs to be checked. If any video frame data is found to be corrupted or missing, it should be marked promptly, and an attempt should be made to extract it again from the video stream. Simultaneously, the transmission status of the video stream should be monitored. If network interruptions occur, the video frames extracted during the interruption period should be supplemented after the connection is restored to ensure the continuity of the initial video frame sequence.

[0048] Step S123: Perform redundant frame removal processing on the initial video frame sequence. By calculating the similarity between adjacent video frames, remove redundant video frames with a similarity higher than a preset similarity threshold.

[0049] When calculating the similarity between adjacent video frames, a pixel-value-based comparison method is used. First, adjacent frames are converted to grayscale to reduce computation. Then, the image is divided into several small blocks, and the average pixel value of each block is calculated. The similarity between two frames is calculated by comparing the average pixel values ​​of corresponding blocks.

[0050] When setting a similarity threshold, the dynamic changes in the scene should be considered. For areas with less human activity and slow scene changes, the threshold can be set higher to remove more redundant frames; for areas with frequent human activity and rapid scene changes, the threshold should be set lower to avoid accidentally deleting keyframes.

[0051] During redundant frame removal, adjacent video frames are compared one by one. If the similarity exceeds a threshold, the subsequent frame is determined to be redundant and removed from the sequence. The retained video frames are still arranged in chronological order to ensure the temporal continuity of the sequence is not affected. After processing, the number of redundant frames removed and the number of retained video frames are recorded to facilitate evaluation of the processing effect.

[0052] Step S124: Arrange the video frames after redundant frame removal in chronological order to form a sequence of video frames to be detected with continuous temporal correlation.

[0053] After redundant frame removal, the remaining video frames are organized. The timestamps of each video frame are checked to ensure they are in the correct order and without any time discrepancies. If any video frames are found to be out of sequence, their positions within the sequence must be readjusted.

[0054] The sequence of video frames to be detected is stored in a linked list structure. Each node contains the image data, timestamp, and corresponding monitoring device identifier of the video frame. The linked list structure facilitates subsequent traversal and processing, allowing for quick access to any video frame in the sequence.

[0055] To improve the efficiency of subsequent processing, video frames in the sequence can be preprocessed, such as by standardizing image size and converting color space. Standardizing image size adjusts all video frames to the same resolution, facilitating object detection algorithms. Converting color space converts RGB images to grayscale, reducing data volume. The preprocessed video frames retain their original temporal correlation, ensuring the sequence accurately reflects the on-site personnel activities.

[0056] Step S130: Call the pre-trained personnel target detection algorithm to perform frame-by-frame personnel target detection processing on the video frame sequence to be detected, and obtain personnel target detection information in the video frame sequence to be detected. The personnel target detection information includes the position information of the personnel target in the video frame and the appearance feature information of the personnel target.

[0057] Step S131: Convert each video frame in the video frame sequence to be detected into standard image data that matches the input format of the personnel target detection algorithm. The standard image data has a preset image size and color channel mode.

[0058] When resizing images, the preset image size must be determined according to the requirements of the personnel target detection algorithm. If the size of the video frame to be detected does not match the preset size, an image scaling algorithm is used for adjustment. During scaling, the aspect ratio of the image is maintained to avoid image distortion. For blank areas generated after scaling, a filling method is used, with the fill color being black or another neutral color.

[0059] When converting color channel modes, if the video frame to be detected is in RGB mode, but the algorithm requires BGR mode, then the channel order should be converted. This ensures that the converted color channel mode matches the algorithm's input requirements, avoiding any impact on detection results due to channel mismatch.

[0060] During the conversion process, the image data is normalized to convert pixel values ​​to the range required by the algorithm. The normalization process uses a linear transformation to convert pixel values ​​from the 0-255 range to the 0-1 range, facilitating algorithm calculation and convergence. After conversion, the standard image data is stored as an array and used as input to the person target detection algorithm.

[0061] Step S132: Call the feature extraction backbone network of the personnel target detection algorithm to perform deep feature extraction processing on the standard image data to generate a multi-scale image feature map, which contains image semantic features and detail features at different levels.

[0062] The feature extraction backbone network consists of multiple convolutional layers, pooling layers, and activation function layers. Standard image data is first input into the first convolutional layer, which performs convolution operations on the image using multiple convolutional kernels to extract low-level features such as edges and textures. The size and number of convolutional kernels are set according to the feature extraction requirements, with each kernel corresponding to a specific feature pattern.

[0063] After processing by the convolutional layers, the image data enters the pooling layers. The pooling layers use either max pooling or average pooling to downsample the image, reducing the size of the feature maps while preserving important feature information. Pooling operations improve the translation invariance of features, making the algorithm less sensitive to changes in the position of people or other objects.

[0064] The activation function layer uses the ReLU function to perform a non-linear transformation on the features after convolution and pooling, increasing the network's expressive power. Through alternating processing of multiple convolutional layers, pooling layers, and activation function layers, the network gradually extracts high-level semantic features of the image.

[0065] As the number of network layers increases, the size of the generated feature maps gradually decreases, while the level of abstraction of the features gradually increases. Feature maps at different levels constitute multi-scale image feature maps, where low-level feature maps contain rich detail information and are suitable for detecting small-sized human targets; high-level feature maps contain more semantic information and are suitable for detecting large-sized human targets.

[0066] Step S1321: Input the standard image data into the initial convolutional layer of the feature extraction backbone network, and perform convolutional filtering and downsampling processing on the standard image data through the initial convolutional layer to generate the first-level feature map.

[0067] The initial convolutional layer contains multiple convolutional kernels, each with a fixed size, such as 3x3. The number of convolutional kernels is determined by the number of features to be extracted; more kernels result in richer extracted features, but also increase the computational cost.

[0068] During convolutional filtering, each convolutional kernel slides across the standard image data, performing convolution operations with the pixel values ​​of the corresponding region to obtain the feature values ​​of that region. Zero-padding is used during the operation to ensure that the size of the convolved feature map is consistent with the size of the input image.

[0069] Downsampling is performed after convolutional filtering, using a convolution operation with a stride of 2. Downsampling reduces the size of the feature map, lowers the computational cost of subsequent processing, and expands the receptive field, enabling the network to capture a wider range of feature information.

[0070] After processing by the initial convolutional layer, the first-level feature map is generated. This first-level feature map retains more image details, is slightly smaller in size than standard image data, and contains mainly low-level features such as image edges and color variations.

[0071] Step S1322: Input the first-level feature map into the first residual module group of the feature extraction backbone network. The first residual module group contains multiple cascaded residual units. Each residual unit performs feature enhancement and dimensionality improvement processing on the input feature map through a skip connection structure and outputs a second-level feature map. The number of feature channels in the second-level feature map is greater than the number of feature channels in the first-level feature map.

[0072] The first residual module group consists of multiple residual units connected in series. Each residual unit contains two convolutional layers and one skip connection. The first convolutional layer has a 1x1 kernel size to reduce the number of channels in the feature map and reduce the computational cost. The second convolutional layer has a 3x3 kernel size to extract more complex features. Then, a 1x1 convolutional layer is used to restore the number of channels.

[0073] Skip connections directly add the input and output of the residual unit. This structure alleviates the vanishing gradient problem in deep networks, making the network easier to train. Through skip connections, important information from the input feature map can be directly passed to subsequent layers, preventing feature loss during the transfer process.

[0074] Feature enhancement is achieved through convolutional operations. Each residual unit further extracts and integrates features from the input first-level feature map, enhancing important features and suppressing irrelevant noise. Dimensionality enhancement is achieved by increasing the number of convolutional kernels, making the second-level feature map have more feature channels than the first-level feature map, thus expressing richer feature information. In a power operation scenario, the first-level feature map may contain basic features such as ground texture and equipment edges in the monitoring image. After processing by the first residual module group, the second-level feature map can more clearly present the contour features of areas where personnel may be present, as well as the distinguishing features between equipment and the ground.

[0075] When reducing the number of channels, the 1x1 convolutional layer in each residual unit can filter features based on their importance in the first-level feature map, retaining channels highly relevant to the target person and removing those primarily reflecting background noise. The 3x3 convolutional layer then further extracts more complex combined features based on these filtered features, such as fragments of a person's limb contours or specific structural combinations of equipment. Subsequent 1x1 convolutional layers, when restoring the number of channels, can allocate the extracted complex features to new channels, ensuring each channel carries specific feature information.

[0076] During the pass-through process, skip connections can element-wise add the input feature map and the convolutionally processed feature map. This addition is not a simple numerical superposition, but rather a fusion of information along the feature dimension, ensuring that key details in the original features are not weakened during multiple convolution operations. For example, the first-level feature map might contain a blurry outline of a person. After convolution, this edge might become clearer, but some details might be lost. Skip connections can supplement the original edge information, so that the final output second-level feature map retains the original details while enhancing key features.

[0077] The number of residual units in the first residual module group is determined based on the complexity of the first-level feature map. In the monitoring screen of a power operation site, if the background is complex, including various equipment and terrain changes, the number of residual units will be increased accordingly to ensure that features can be fully extracted and enhanced. Each residual unit is connected in series, with the output of the previous residual unit serving as the input of the next residual unit. After multiple processing steps, the first-level feature map is gradually transformed into a second-level feature map.

[0078] With an increased number of feature channels in the second-level feature map, more feature information about personnel targets can be accommodated, such as the texture features of clothing and the preliminary outline features of tools carried. These features play an important role in accurately detecting personnel targets in complex power operation environments, improving target detection accuracy and reducing false detections caused by background interference.

[0079] Step S1323: Input the second-level feature map into the second residual module group of the feature extraction backbone network. The second residual module group performs further feature abstraction and spatial dimensionality reduction on the second-level feature map to generate a third-level feature map. The spatial size of the third-level feature map is smaller than that of the second-level feature map.

[0080] The second residual module group also consists of multiple residual units, but its internal convolutional layer parameter settings differ from those of the first residual module group. In terms of feature abstraction, the convolutional layers of the second residual module group employ a larger receptive field, enabling them to capture a wider range of feature relationships in the second-level feature map. For example, in a power work site, the second-level feature map might contain local features such as the head, torso, and limbs of a person. After processing by the second residual module group, these local features are integrated into more complete human body structure features, achieving further feature abstraction.

[0081] Spatial dimensionality reduction is achieved by setting a stride greater than 1 in the convolutional layer. As the convolutional kernel slides across the second-level feature map, increasing the stride reduces the spatial size of the output feature map. This dimensionality reduction doesn't simply reduce the number of pixels; rather, it reduces the amount of data in the feature map while preserving key feature information, thus improving the efficiency of subsequent processing. For example, the second-level feature map may contain some repetitive background features; spatial dimensionality reduction can remove this redundant spatial information, allowing the third-level feature map to focus more on regions where people or other objects might be present.

[0082] Each residual unit also employs a skip connection structure during feature abstraction and spatial dimensionality reduction. However, unlike the first residual module group, the skip connections here require appropriate adjustments to the input feature map to match the spatial dimensions of the convolutionally processed feature map. This adjustment is typically achieved through a 1x1 convolutional layer with a suitable stride, reducing the spatial dimensionality of the input feature map before adding it to the convolutionally processed feature map. This ensures both the effectiveness of the skip connections and adaptability to changes in spatial dimensions.

[0083] In the scenario of a power operation site, after processing by the second residual module group, the third-level feature map can more clearly present the relative positional relationship between personnel and surrounding equipment. For example, the positional association between personnel and large equipment such as transformers and transmission towers will be reflected in the third-level feature map.

[0084] After the spatial size of the third-level feature map is reduced, each pixel represents a larger area of ​​the original image, reflecting more macroscopic scene information. This makes the third-level feature map more advantageous when detecting people at a distance, because people at a distance occupy less space in the image, and the feature map after spatial dimensionality reduction can better capture their overall features.

[0085] Step S1324: Input the third-level feature map into the third residual module group of the feature extraction backbone network, and perform multi-receptive field feature extraction on the third-level feature map through multiple convolution operations with different kernel sizes to generate a fourth-level feature map, which contains local image features at different scales.

[0086] The third residual module group is designed with a focus on multi-receptive field feature extraction. It contains multiple parallel convolutional branches, each using convolutional kernels of different sizes. The selection of these convolutional kernel sizes is determined based on the different scales at which personnel targets may appear in the power operation site. For example, for personnel targets at close range, smaller convolutional kernels are used to extract more detailed local features; for personnel targets at a distance, larger convolutional kernels are used to capture more comprehensive overall features.

[0087] When processing the third-level feature map, each convolutional branch independently performs a convolution operation on the feature map. The output feature maps from each branch are then concatenated along the channel dimension to form a preliminary intermediate feature map. This multi-receptive-field fusion method enables the fourth-level feature map to simultaneously contain local image features at different scales. For example, in a power work site, it can capture both the detailed features of a worker's hands operating tools (small scale) and the overall posture features of the worker within the work area (large scale).

[0088] The residual units in the third residual module group also have skip connections, and their connection method is similar to that of the second residual module group. The input feature map needs to be adjusted according to the size of the feature map after the convolution operation. The role of the skip connections here is to fuse the basic features in the third-level feature map with the features extracted from multiple receptive fields, so as to avoid losing important original feature information during multi-scale feature extraction.

[0089] The generated fourth-level feature map is richer in feature representation and can adapt to various scale changes of personnel targets at power operation sites. Whether the person is close to the monitoring camera or a distant person at the edge of the frame, the fourth-level feature map contains corresponding feature information.

[0090] Step S1325: Input the fourth-level feature map into the fourth residual module group of the feature extraction backbone network, perform global context information modeling processing on the fourth-level feature map, and generate the fifth-level feature map.

[0091] The fourth residual module group introduces an attention mechanism to model global contextual information. This attention mechanism enables the network to automatically focus on regions in the fourth-level feature map that are more important for personnel target detection, while suppressing features in irrelevant regions. In power work sites, global contextual information includes the overall relationship between personnel targets and their surrounding environment, such as whether personnel are near high-voltage equipment or at the edge of a restricted work area.

[0092] The attention mechanism is implemented through three steps: feature compression, feature excitation, and feature recalibration. Feature compression uses global average pooling to compress each channel of the fourth-level feature map into a single value, representing the global information of that channel's feature. Feature excitation processes the compressed value using fully connected layers and activation functions to generate weight coefficients for each channel, the magnitude of which indicates the importance of the feature in that channel. Feature recalibration multiplies the weight coefficients by the corresponding channels of the fourth-level feature map, strengthening important feature channels and weakening less important ones.

[0093] During processing, the residual units of the fourth residual module group add the feature map processed by the attention mechanism to the fourth-level feature map passed through skip connections. Skip connections ensure that global structural information in the original feature map is not lost, while the attention mechanism enhances the feature representation of key regions. For example, in the fourth-level feature map of a power work site, the features of personnel areas near high-voltage transmission towers are given higher weights and strengthened, while the features of distant background areas are given lower weights, thus making the generated fifth-level feature map more focused on key personnel target areas.

[0094] Global contextual information modeling enables the fifth-level feature map to incorporate broader scene information, allowing for an understanding of the location and status of personnel targets throughout the power operation site. For example, the fifth-level feature map can determine whether personnel are active in a normal work area or have approached a dangerous high-voltage equipment area, which is of great significance for subsequent regional behavioral feature analysis.

[0095] Step S1326: Combine the first-level feature map, the second-level feature map, the third-level feature map, the fourth-level feature map, and the fifth-level feature map into a multi-scale image feature map. The feature maps of different levels in the multi-scale image feature map have different resolutions in the spatial dimension and different feature representation capabilities in the channel dimension.

[0096] The combination of multi-scale image feature maps is not simply a matter of stacking feature maps from five levels, but rather a fusion process using a Feature Pyramid Network (FPN) structure. The FPN structure consists of three parts: bottom-up, top-down, and lateral connections. The bottom-up part comprises feature maps generated by the four residual modules mentioned earlier. From the first to the fifth level, the spatial resolution of the feature maps gradually decreases while the semantic information gradually increases.

[0097] The top-down approach starts from the fifth-level feature map and uses upsampling to enlarge the high-semantic-information feature map to the same spatial size as the lower-level feature maps. Bilinear interpolation is used for upsampling to ensure that the feature map does not lose too much feature information during the enlargement process. Lateral concatenation then joins the bottom-up and top-down feature maps along the channel dimension, resulting in a concatenated feature map that includes both low-level detailed features and high-level semantic features.

[0098] In power operation field applications, the multi-scale image feature maps fused by the FPN structure retain rich details in the first-level feature map, such as the color details of clothing and the texture of tools, making them suitable for detecting small targets of people at close range. The fifth-level feature map contains strong semantic information, suitable for detecting large targets of people at a distance or judging the overall behavioral trends of people. The feature maps of different levels complement each other, forming a comprehensive feature representation that can meet the needs of accurately detecting personnel targets of different sizes and locations in complex power operation environments.

[0099] The combined multi-scale image feature maps are then fed into the subsequent region proposal network to improve the accuracy and reliability of candidate bounding boxes.

[0100] Step S133: The region proposal network of the personnel target detection algorithm is used to generate potential personnel target regions on the multi-scale image feature map to obtain multiple candidate personnel target bounding boxes. Each candidate personnel target bounding box has coordinate position and size information in the standard image data.

[0101] The main function of the region proposal network is to filter regions that may contain people from multi-scale image feature maps. Its processing combines the characteristics of multi-scale feature maps, generating corresponding candidate bounding boxes for feature maps of different levels to accommodate people targets of different sizes.

[0102] Step S1331: Perform anchor box generation processing with preset scale and aspect ratio on each level feature map of the multi-scale image feature map, and generate multiple initial anchor boxes with different sizes and shapes at each pixel position of each level feature map.

[0103] For each level of feature map, different preset scales are set based on its spatial resolution and the size of the corresponding actual image region. Lower-level feature maps correspond to small regions in the actual image, so their preset scales are smaller; higher-level feature maps correspond to large regions in the actual image, so their preset scales are larger. For example, in the multi-scale feature maps of a power operation site, the preset scale of the initial anchor box on the first-level feature map is suitable for detecting local regions of people who are relatively close in the image, while the preset scale of the initial anchor box on the fifth-level feature map is suitable for detecting the entire region of people who are relatively far away in the image.

[0104] The aspect ratio setting takes into account the different postures that a person might present in the image, typically including aspect ratios corresponding to postures such as standing horizontally, standing vertically, and crouching. At each pixel location, multiple initial anchor boxes are generated according to a preset scale and aspect ratio. These anchor boxes cover various possible areas around that pixel location where a person might be present.

[0105] The initial anchor boxes are generated based on the pixel coordinates of the feature map, with each anchor box represented by the coordinates of its top-left and bottom-right corners. During the generation process, it is ensured that the anchor boxes are evenly distributed on the feature map to avoid missed detection areas, while also avoiding excessive computation due to too many duplicate anchor boxes.

[0106] Step S1332: Perform feature mapping processing on the feature map of each layer through the convolutional layer of the region proposal network to generate a category score and bounding box offset corresponding to the number of initial anchor boxes. The category score is used to represent the probability that the initial anchor box contains a person target, and the bounding box offset is used to adjust the position and size of the initial anchor box.

[0107] The convolutional layers of the region proposal network perform convolution operations on the feature maps of each layer. The number of convolutional kernels is related to the number of initial anchor boxes and the parameters that need to be predicted for each anchor box. For each initial anchor box, the convolutional layer outputs a class score and a set of bounding box offsets.

[0108] The category score is generated by classifying the features in the feature map, and its value reflects the probability that the initial anchor box contains a person target. In the feature map of a power operation site, if the area covered by an initial anchor box contains human features, such as human body outlines or head features, the corresponding category score will be higher; if it covers the background area, such as equipment or the ground, the category score will be lower.

[0109] The bounding box offset includes adjustment parameters for the center coordinates and width and height of the initial anchor box. These parameters are learned by analyzing the positional differences between the anchor box and the actual person target in the feature map. For example, when the top-left corner coordinates of the initial anchor box deviate from the top-left corner coordinates of the actual person target, the bounding box offset indicates how to adjust those coordinates to bring the anchor box closer to the actual target.

[0110] Step S1333: Filter the initial anchor boxes according to the category score, and retain the initial anchor boxes with category scores greater than the preset category threshold as candidate anchor boxes.

[0111] Setting the preset category threshold needs to balance the detection rate and false detection rate. In power operation sites, in order to avoid missing potential personnel targets as much as possible, the preset category threshold should not be set too high, but it should also not be set too low to avoid too many invalid candidate anchor boxes.

[0112] During the screening process, the category score of each initial anchor box is compared with a preset category threshold. Only initial anchor boxes with scores higher than the threshold are retained as candidate anchor boxes. These candidate anchor boxes are considered to be areas more likely to contain human targets.

[0113] Step S1334: Use the bounding box offset to perform coordinate correction processing on the candidate anchor box to obtain the adjusted candidate bounding box. The position and size of the adjusted candidate bounding box are closer to the boundary of the real person target.

[0114] The application of bounding box offsets is based on the feature map coordinate system. Adjusting the center coordinates of candidate anchor boxes involves combining the initial anchor box's center coordinates with its width or height and the corresponding offset. For example, the adjusted center horizontal coordinate is the initial anchor box center horizontal coordinate plus the product of the initial anchor box width and the horizontal offset; the adjusted center vertical coordinate is the initial anchor box center vertical coordinate plus the product of the initial anchor box height and the vertical offset.

[0115] Adjusting the width and height of candidate anchor frames involves multiplying the initial anchor frame width or height by an exponential function of the offset. This adjustment method ensures that the width and height are always positive and achieves smooth adjustment as the offset changes.

[0116] Through the above correction process, the candidate anchor boxes can more accurately surround real personnel targets. At power work sites, personnel may be in various postures, such as bending over or walking, and the initial anchor boxes may not perfectly match the actual outline of the personnel. After the bounding box offset is corrected, the adjusted candidate bounding boxes can better fit the edges of the personnel's body, improving the accuracy of subsequent target detection.

[0117] Step S1335: Map the adjusted candidate bounding boxes generated on the feature maps of different levels to the pixel coordinate system of the standard image data to obtain multiple candidate target bounding boxes in the standard image data space.

[0118] Because there is a scaling relationship between feature maps at different levels and standard image data, it is necessary to transform the adjusted candidate bounding boxes from the feature map coordinate system to the pixel coordinate system of the standard image data. This transformation is achieved by multiplying the coordinates on the feature map by a corresponding scaling factor, which is determined by the size ratio between the feature map and the standard image data.

[0119] For example, if the width of a certain level feature map is a fraction of the width of the standard image data, then the horizontal coordinates of the candidate bounding boxes on that feature map need to be multiplied by the scaling factor to obtain the horizontal coordinates in the standard image data. The conversion of the vertical coordinates is similar.

[0120] In standard image data from power operation sites, the mapped candidate personnel target bounding boxes accurately reflect the areas where personnel targets may exist in the actual monitoring images, further reducing the deviation between the position and size of these bounding boxes and the actual personnel targets.

[0121] Step S1336: Perform cross-level merging processing on the mapped candidate target bounding boxes to eliminate duplicate candidate target bounding boxes caused by the generation of multi-scale feature maps, forming a final set of multiple candidate target bounding boxes.

[0122] Since multi-scale feature maps may generate multiple candidate bounding boxes for the same person target, cross-level merging is required. The merging process is achieved by calculating the degree of overlap between different candidate bounding boxes. When the overlap between two candidate bounding boxes exceeds a preset threshold, they are considered to point to the same person target, and the candidate bounding box with higher quality (such as one with a higher category score) is retained.

[0123] At a power operation site, for the same person in the image, candidate bounding boxes may be generated on feature maps at multiple levels. After cross-level merging, only one candidate bounding box that can most accurately surround the person is retained, avoiding repeated detection of the same target in subsequent processing and improving detection efficiency.

[0124] The merged candidate bounding box set contains all regions in the standard image data that may contain human targets. These bounding boxes will be input into the subsequent non-maximum suppression processing step to further filter out more accurate target bounding boxes.

[0125] Step S134: Perform non-maximum suppression processing on the multiple candidate target bounding boxes, remove redundant candidate target bounding boxes with overlap exceeding a preset threshold, and retain candidate target bounding boxes with confidence scores greater than a set confidence score threshold as target bounding boxes.

[0126] The purpose of nonmaximum suppression is to filter out the most accurate feature information from the set of candidate personnel bounding boxes. In the scenario of a power operation site, personnel targets may wear different colored work clothes and carry different tools. These appearance features are of great significance for subsequent personnel identification and behavior analysis.

[0127] When extracting clothing color features, the image region within the effective bounding box is first converted from the RGB color space to the HSV color space. This conversion better separates color and brightness information, reducing the impact of lighting changes on color recognition. In the HSV space, the H channel value (hue) and S channel value (saturation) of each pixel are extracted, while the V channel value (lightness) is ignored. Then, the H channel values ​​are statistically analyzed, dividing the hue range into multiple intervals, each corresponding to a basic color. The percentage of pixels in each interval is calculated, and the colors corresponding to the intervals with the highest percentages are selected as the main clothing color features of the person target. For example, if the pixel percentage corresponding to the blue hue interval is the highest within the effective bounding box, followed by gray, then blue and gray are used as the clothing color features of the person target.

[0128] Body contour feature extraction is achieved through edge detection and contour fitting. The Canny edge detection algorithm is used to process the image within the effective bounding box to obtain the edge pixels of the person target. Then, a polygon approximation algorithm is used to fit these edge points, resulting in a simplified polygon contour. The number of vertices, the length ratio of each side, and the concavity / convexity of the contour collectively constitute the body contour features. In power work sites, personnel may be in different postures such as standing, bending, and walking. These posture changes lead to differences in body contour features, and these differences must be preserved during extraction.

[0129] Feature extraction of carried items primarily focuses on whether a person is carrying tools and the approximate shape of the tools. Gradient changes in the image are calculated by analyzing the grayscale values ​​within the effective bounding box. If a region with a drastic gradient change exists on the person's hand or side of their body, and the shape of this region does not conform to the characteristics of a human body part, it is identified as potentially carrying an item. Further morphological processing, such as erosion and dilation operations, is applied to this region to remove noise interference and obtain the approximate outline of the item. Based on parameters such as the aspect ratio and area of ​​the outline, the type of item is preliminarily determined, such as a long, narrow tool (possibly a wrench or screwdriver), a square item (possibly a toolbox), etc., and this information is used as the features of the carried item.

[0130] Step S137: Combine the bounding box coordinates of the effective bounding box and the corresponding appearance feature information into personnel target detection information, and sort and store the personnel target detection information according to the time order of the video frames to be detected in the video frame sequence to be detected.

[0131] The bounding box coordinates (coordinates of the top-left and bottom-right corners in the standard image data pixel coordinate system) of each valid bounding box are associated with the extracted appearance feature information (clothing color features, body contour features, and carried item features) to form a person target detection information. Each information also includes the timestamp of the corresponding video frame to be detected, indicating when the person target appeared.

[0132] Based on the chronological order of the video frame sequence to be detected, all personnel target detection information is stored in a data structure, such as a linked list or array. During storage, a node is created for the personnel target detection information corresponding to each video frame, and the nodes are connected in chronological order according to their timestamps. This sorted storage method facilitates the subsequent extraction of the position and feature changes of the same personnel target in consecutive video frames. For example, in multiple consecutive video frames, the personnel target detection information of the same personnel target is arranged in chronological order, which can clearly show its position movement and posture changes.

[0133] Step S140: Based on the intrusion interest area and the personnel target detection information, perform regional association behavior feature analysis on the personnel targets in the video frame sequence to be detected to obtain the regional association behavior features of the personnel targets. The regional association behavior features are feature information describing the spatial positional relationship and movement trend between the personnel targets and the intrusion interest area.

[0134] In power operation sites, the spatial relationship and movement trend between personnel targets and areas of interest for intrusion (such as high-voltage transmission tower areas, transformer operating areas, etc.) are crucial for determining the presence of intrusion behavior. Area-related behavioral characteristic analysis aims to extract these relationships and trend information from personnel target detection data.

[0135] Step S141: Transform the effective bounding box coordinates in the personnel target detection information from the image coordinate system of the video frame to be detected to the pixel coordinate system of the panoramic layout image to obtain the actual position coordinates of the personnel target in the panoramic layout image.

[0136] The image coordinate system of the video frame to be detected has its origin at the top left corner of the video frame, while the pixel coordinate system of the panoramic layout image has its own origin and scale. In order to unify the positional information, coordinate transformation is required.

[0137] First, the position and shooting parameters of the camera corresponding to the video frame to be detected within the panoramic layout image are determined. Each camera has its own installation coordinates, shooting direction, focal length, and other parameters within the panoramic layout image. Based on these parameters, an imaging model of the camera is established, which describes how three-dimensional spatial points in the panoramic layout image are projected onto the two-dimensional video frame captured by the camera.

[0138] Then, using the inverse transformation of the imaging model, the coordinates of the effective bounding box (coordinates in the image coordinate system of the video frame to be detected) are transformed to the pixel coordinate system of the panoramic layout image. During the transformation, the distortion correction parameters of the camera need to be considered to correct the coordinates and eliminate the positional deviation caused by lens distortion. For example, after the transformation, the coordinates of a certain effective bounding box in the video frame to be detected correspond to a certain position near the high-voltage transmission tower area in the panoramic layout image, which is the actual position coordinate of the person target in the panoramic layout image.

[0139] Step S142: Determine whether the actual location coordinates fall within the closed polygonal area of ​​the intrusion concern area, and determine the spatial attribution relationship between the personnel target and the intrusion concern area. The spatial attribution relationship includes the personnel target being located inside the intrusion concern area, the personnel target being located outside the intrusion concern area, and the personnel target intersecting with the boundary of the intrusion concern area.

[0140] The ray casting method is used to determine whether the actual position coordinates are within the closed polygon region. A virtual ray is cast horizontally to the right from the actual position coordinates, and the number of intersections between the ray and the boundary of the closed polygon is counted. If the number of intersections is odd, the actual position coordinates are considered to be inside the region; if the number is even, they are considered to be outside the region; if the actual position coordinates fall exactly on the boundary line of the polygon, they are considered to intersect the boundary.

[0141] In the panoramic layout image of a power work site, for the closed polygon GY-001 (high-voltage transmission tower area), if the actual location coordinates of a person target, after being determined by the ray method, have 3 (odd) intersection points, then the person target is located inside GY-001; if the number of intersection points is 2 (even), then the person target is located outside GY-001; if the coordinates fall on a side of the polygon, then the person target intersects with the boundary of GY-001. The same method applies to BYQ-002 (transformer operating area) and DLG-003 (cable laying trench).

[0142] Step S143: Based on the personnel target detection information of multiple consecutive frames in the video frame sequence to be detected, extract the actual position coordinate sequence of the same personnel target, perform trajectory fitting processing on the actual position coordinate sequence, and obtain the movement trajectory curve of the personnel target.

[0143] The same person or target will have continuous positional changes in consecutive video frames to be detected. Extracting this positional information and fitting a trajectory can reflect its movement path and trend.

[0144] Step S1431: Perform personnel target matching processing on the personnel target detection information of adjacent video frames in the video frame sequence to be detected. Determine the correspondence of the same personnel target in different video frames by calculating the similarity of appearance feature information and position distance similarity of personnel targets, and establish the identity identifier of personnel targets.

[0145] The similarity calculation of appearance feature information is performed from three aspects: clothing color, body shape outline, and carried items. For clothing color features, the hue and saturation differences of the main clothing colors of the target person in two video frames are calculated; the smaller the difference, the higher the similarity. For body shape outline features, the vertex matching degree and side length ratio differences of the two outline polygons are calculated; the higher the matching degree and the smaller the difference, the higher the similarity. For carried items features, if both target persons carry items and the item outlines are similar, or if neither carries items, the similarity is high; otherwise, the similarity is low. The similarity of these three aspects is weighted and averaged to obtain the comprehensive similarity of appearance feature information.

[0146] Location distance similarity is calculated by the Euclidean distance between the actual coordinates of a person's location in two video frames. The smaller the distance, the closer the locations are, and the higher the location distance similarity. The Euclidean distance is normalized so that its value ranges from 0 to 1, consistent with the value range of appearance feature information similarity.

[0147] A matching threshold is set. When the overall similarity (a weighted sum of appearance feature similarity and location distance similarity) exceeds this threshold, the people in the two video frames are determined to be the same target. A unique identifier, such as ID-001, ID-002, etc., is assigned to each identified person for subsequent trajectory tracking.

[0148] Step S1432: Based on the identity identifier, the actual location coordinates of the same person target are arranged in the time order of the video frames in which they are located to form a sequence of actual location coordinates of the person target. Each element in the sequence of actual location coordinates contains a timestamp and the corresponding actual location coordinates.

[0149] In the sequence of video frames to be detected, all personnel target detection information is traversed and grouped according to identity identifiers. For each group corresponding to an identity identifier, the actual position coordinates within the group are arranged in ascending order of the video frame timestamps. Each arranged element contains the timestamp (accurate to milliseconds) and the actual position coordinates (coordinates in the pixel coordinate system of the panoramic layout image) corresponding to that actual position coordinate. For example, the actual position coordinate sequence corresponding to ID-001 might be [(t1, (x1, y1)), (t2, (x2, y2)), ..., (tn, (xn, yn))], where t1 <t2<...<tn。

[0150] Step S1433: Perform outlier detection processing on the actual position coordinate sequence to identify and remove abnormal position coordinates that deviate from the normal trajectory due to detection errors.

[0151] An outlier detection method based on standard deviation is used. First, the mean and standard deviation of the x-coordinate and the mean and standard deviation of the y-coordinate in the actual location coordinate sequence are calculated. For each actual location coordinate (x, y) in the sequence, if the difference between x and the x-mean exceeds a certain multiple of the x-standard deviation, or the difference between y and the y-mean exceeds a certain multiple of the y-standard deviation, then that coordinate is determined to be an outlier.

[0152] At power work sites, deviations in the detection position of personnel targets may occur due to factors such as camera angle and changes in lighting. For example, in the actual location coordinate sequence of ID-001, if the difference between the x-value of a certain coordinate and the average value is much greater than several times the standard deviation of x, and the coordinates before and after the timestamp corresponding to that coordinate are relatively concentrated, then that coordinate is determined to be an outlier and is removed from the sequence.

[0153] Step S1434: Use a polynomial curve fitting algorithm to perform curve fitting on the actual position coordinate sequence after removing outliers to obtain a smooth initial trajectory curve.

[0154] Polynomial curve fitting algorithms aim to make the fitted curve as close as possible to the points in the actual position coordinate sequence by selecting an appropriate polynomial order. The choice of order needs to balance fitting accuracy and curve smoothness. Too high an order may lead to overfitting (unnecessary fluctuations), while too low an order may result in insufficient fitting accuracy.

[0155] For the actual location coordinate sequence of ID-001 after outlier removal, a third-order polynomial is assumed to be used for fitting. Let the polynomial equations be x = at³ + bt² + ct + d and y = et³ + ft² + gt + h, where t is a continuous variable after timestamp conversion, and a, b, c, d, e, f, g, and h are the coefficients to be determined. These coefficients are calculated using the least squares method to minimize the sum of squares of the deviations between the fitted curve and the actual location coordinate sequence, thus obtaining the initial trajectory curve. This initial trajectory curve smoothly connects all actual location coordinate points, eliminating small fluctuations caused by detection errors.

[0156] Step S1435: Perform curvature analysis on the initial trajectory curve, identify inflection points and straight line segments in the trajectory curve, and divide the initial trajectory curve into multiple trajectory segments according to the position of the inflection points.

[0157] Curvature analysis is achieved by calculating the curvature value at each point on the trajectory curve. The curvature value reflects the degree of curvature of the curve; the larger the curvature value, the more pronounced the curvature. If the curvature value is zero or close to zero, the curve is close to a straight line. A curvature threshold is set, and when the curvature value at a point on the curve exceeds this threshold, it is determined to be an inflection point.

[0158] In the initial trajectory curve of ID-001, if the curvature value of a certain segment suddenly increases and exceeds a threshold, that point is an inflection point. Based on these inflection points, the initial trajectory curve is divided into multiple continuous trajectory segments, each of which may be a straight line segment or a curve segment with a small degree of curvature. For example, the trajectory of ID-001 may be divided into: a straight line segment from point A to inflection point B, a curve segment from inflection point B to inflection point C, a straight line segment from inflection point C to point D, etc.

[0159] Step S1436: Calculate the trajectory parameters of each trajectory segment, and optimize and adjust the initial trajectory curve based on the trajectory parameters to obtain a movement trajectory curve that can accurately reflect the movement pattern of the personnel target. The trajectory parameters include trajectory length, average movement speed, and direction change angle.

[0160] The trajectory length is obtained by calculating the sum of the distances between all points on the trajectory segment. The average speed is the ratio of the trajectory length to the time interval corresponding to that segment. The angle of change of direction is obtained by calculating the angle between the tangent directions at the start and end points of the trajectory segment.

[0161] Based on these trajectory parameters, the initial trajectory curve is optimized. If the average moving speed of a certain trajectory segment is much greater than that of other segments, and the trajectory length of that segment is relatively short, it may be due to detection errors. In this case, the segment needs to be smoothed to reduce its curvature. The optimized movement trajectory curve can more accurately reflect the movement pattern of personnel targets at the power work site, such as the entire path of ID-001 from the work area entrance to the transformer operating area and then leaving.

[0162] Step S144: Calculate the movement direction vector and movement speed characteristics of the personnel target based on the movement trajectory curve. The movement direction vector represents the position change trend of the personnel target per unit time, and the movement speed characteristics represent the speed of the position change of the personnel target.

[0163] The movement direction vector is obtained by calculating the difference between the position coordinates of two adjacent time points on the movement trajectory curve. Let the position coordinates of a certain time point t1 on the trajectory curve be (x1, y1), and the position coordinates of the adjacent time point t2 (t2>t1) be (x2, y2). Then the movement direction vector is (x2-x1, y2-y1). The direction of this movement direction vector represents the direction of movement of the personnel target from t1 to t2, and the ratio of the magnitude of the movement direction vector to (t2-t1) reflects the speed of movement.

[0164] Movement speed characteristics include instantaneous speed and average speed. Instantaneous speed is the ratio of the magnitude of the movement direction vector to the time interval (t2-t1). Average speed is the ratio of the length of a trajectory segment to the time interval corresponding to that segment. In power work sites, the movement speed of personnel may vary. Extracting these speed characteristics can help determine the stability and purpose of their movement; for example, rushing quickly towards high-voltage equipment areas may be dangerous.

[0165] Step S145: Analyze the intersection of the movement trajectory curve and the boundary of the intrusion concern area to determine the coordinates and intersection time of the boundary intersection point when the personnel target approaches or leaves the intrusion concern area.

[0166] The coordinates of the intersection points are obtained by simultaneously solving the equations of the movement trajectory curve and the closed polygon boundary of the intrusion area of ​​interest. For each intersection point, the corresponding intersection time point is determined based on the time parameter of the movement trajectory curve.

[0167] For example, the trajectory curve of ID-001 intersects the closed polygon boundary of GY-001 (the high-voltage transmission tower area) at two points. The coordinates of these two points are obtained by solving the equations of the trajectory curve and the equations of each side of the polygon boundary. Then, based on the time parameters of the trajectory curve, the time when ID-001 reaches the first intersection point (the intersection point when it enters GY-001) and the time when it reaches the second intersection point (the intersection point when it leaves GY-001) are determined. The same method can be used to determine the coordinates and intersection times of the intersection points with the boundaries of BYQ-002 and DLG-003.

[0168] Step S146: Combining the spatial affiliation relationship, movement trajectory curve, movement direction vector, movement speed characteristics, boundary intersection coordinates and intersection time points, construct the regional association behavior characteristics of the personnel target. The regional association behavior characteristics can comprehensively reflect the dynamic behavior process of the personnel target relative to the intrusion interest area.

[0169] The extracted features are integrated to form a set of regional association behavior features with multiple dimensions. For ID-001, its regional association behavior features include: spatial affiliation with GY-001, BYQ-002, and DLG-003 at different time points; complete movement trajectory curves; movement direction vectors for each time period; movement speed features such as instantaneous speed and average speed; and coordinates of intersection points with the boundaries of each intrusion interest area and the corresponding intersection time points.

[0170] These characteristics collectively describe the dynamic behavior of ID-001 relative to each intrusion concern area at the power work site, including when it enters BYQ-002, its movement path within the area, its movement speed, whether it approaches hazardous equipment, and when it leaves.

[0171] Step S150: Based on the regional association behavior characteristics and the intrusion determination reference rules, perform intrusion determination processing on the behavior of the personnel target to generate personnel intrusion detection results. The personnel intrusion detection results are used to indicate whether there is personnel intrusion behavior that conforms to the intrusion determination reference rules in the monitored scene.

[0172] Based on the combined regional behavioral characteristics and the preset intrusion judgment reference rules, the behavior of personnel targets is analyzed and judged to determine whether an intrusion has occurred.

[0173] Step S151: Retrieve the intrusion judgment reference rule set corresponding to the intrusion concern area from the rule database.

[0174] Based on the area identifier of the target personnel's current location or the intrusion-related area of ​​interest, the system queries the rule database for the corresponding set of intrusion judgment reference rules. For example, when analyzing the behavior of ID-001, if its area-related behavioral characteristics involve GY-001, the system retrieves the set of intrusion judgment reference rules associated with the area identifier (GY-001) of GY-001; if it involves BYQ-002, the system retrieves the set of rules associated with the area identifier (BYQ-002) of BYQ-002.

[0175] Step S152: Compare the spatial affiliation relationship in the area association behavior characteristics with the area entry conditions in the intrusion determination reference rules to determine whether the personnel target meets the preliminary conditions for entering the intrusion concern area.

[0176] Area entry conditions include the permitted time range and the authorization status of personnel. For example, the intrusion judgment reference rules for GY-001 stipulate that the permitted time range is a specific period on the first Tuesday of each month, and personnel must hold a high-voltage work permit. The entry time into GY-001 in the spatial affiliation of ID-001 is compared with the permitted time range. If it is not within the permitted time range, the preliminary entry conditions are met (i.e., there is suspicion of intrusion); if it is within the permitted time range, but it cannot be confirmed that the person holds a high-voltage work permit, the preliminary conditions are also met.

[0177] Step S153: When the personnel target meets the preliminary conditions, extract the duration of the personnel target being located inside the intrusion concern area from the regional association behavior features, and compare it with the duration threshold in the intrusion judgment reference rules.

[0178] The duration during which a target person remains within the area of ​​interest for intrusion is the time difference between the intersection of their departure and entry points. For example, if ID-001 enters GY-001 at time t1 and leaves at time t2, the duration is t2-t1. This duration is compared to the duration threshold set for this area of ​​interest in the intrusion determination reference rules. If the duration exceeds the threshold, it is recorded as non-compliant in this judgment; otherwise, it is recorded as compliant.

[0179] In the GY-001 area of ​​the power work site, assume its duration threshold is a specific duration. After the duration of ID-001 is calculated, it is compared with this threshold. If the duration of ID-001 within GY-001 exceeds the set threshold, then based on the duration judgment dimension, the behavior of the target person is suspected of intrusion.

[0180] Step S154: Analyze the movement trajectory curve and movement direction vector in the regional association behavior characteristics to determine whether the movement trajectory pattern of the personnel target in the intrusion interest area conforms to the preset suspicious trajectory pattern in the intrusion judgment reference rules.

[0181] First, the movement trajectory curve is segmented, dividing it into multiple trajectory segments according to the direction of change and the points of stillness. Each trajectory segment has its corresponding start point, end point, direction of movement, and duration.

[0182] Next, features of each trajectory segment are extracted, including the length of the trajectory segment, the angle of change in the direction of movement, and the change in movement speed within the trajectory segment. These features are then compared with the features corresponding to the pre-defined suspicious trajectory patterns in the intrusion determination reference rules.

[0183] For example, in the GY-001 area, the preset suspicious trajectory patterns include frequent back-and-forth travel around transmission towers and prolonged stays near high-voltage equipment. If the movement trajectory curve of ID-001 shows multiple trajectory segments that move back and forth between two transmission towers with large directional changes and long periods of stay near the transmission towers, then its movement trajectory pattern can be determined to match the suspicious trajectory pattern.

[0184] Step S155: Combine the appearance feature information in the regional association behavior features to identify the type of behavior of the target personnel in the intrusion concern area, and determine whether there are any dangerous behaviors prohibited by the intrusion judgment reference rules.

[0185] Physical characteristics include the color of clothing, whether the person is carrying tools, and their body shape. By analyzing this information and combining it with changes in the person's posture in video frames, the type of their behavior can be identified.

[0186] At power work sites, prohibited dangerous behaviors according to the intrusion determination reference rules include climbing equipment, touching high-voltage terminals, and using open flames around equipment. For example, if ID-001's appearance shows that it is not carrying any insulated tools and appears to be climbing a transmission tower support in the video frame, then it can be determined that it is engaging in prohibited dangerous behaviors.

[0187] Step S156: Based on the interaction between the personnel target and the boundary of the intrusion concern area, determine whether the personnel target is attempting to cross, damage the boundary, or evade monitoring.

[0188] By analyzing the intersection of the movement trajectory curve of the personnel target with the area boundary, and the relative positional relationship between the personnel and boundary facilities in the video frame, the interaction mode is determined.

[0189] If a person's movement repeatedly approaches the boundary fence, and the video frames show them climbing the fence or engaging in acts of damaging it, such as dismantling fence components, then it can be determined that they are attempting to cross or damage the boundary. Furthermore, if a person's movement deliberately avoids the camera's field of view, exhibiting a tendency to evade surveillance, this is also an interaction pattern that requires attention.

[0190] Step S157: Perform a comprehensive weighted evaluation on the various judgment results, and calculate the intrusion risk score of the personnel target according to the preset weight allocation rules. The intrusion risk score comprehensively reflects the possibility that the behavior of the personnel target constitutes an intrusion.

[0191] Different weights are assigned to each judgment result, and the magnitude of the weight is determined according to the importance of the judgment item in the intrusion determination. For example, the item of "the existence of prohibited dangerous behavior" has a relatively high weight, while the item of "the duration exceeds the threshold" has a lower weight.

[0192] Each judgment result is calculated according to its corresponding weight. If a judgment result is not met, that item is scored according to its corresponding weight; if it is met, no score is awarded. The scores of all items are added together to obtain the intrusion risk score of the personnel target.

[0193] For example, in the GY-001 area, the weight for prohibited dangerous actions is 0.3, the weight for duration exceeding the threshold is 0.2, the weight for movement trajectory pattern matching a suspicious pattern is 0.2, the weight for boundary breaching behavior is 0.2, and the weight for other items is 0.1. ID-001 does not meet the criteria for prohibited dangerous actions and duration exceeding the threshold, so its intrusion risk score is 0.3 + 0.2 = 0.5.

[0194] Step S158: Compare the intrusion risk score with a preset intrusion determination threshold. When the intrusion risk score is greater than the intrusion determination threshold, the behavior of the target person is determined to constitute an intrusion, and a personnel intrusion detection result containing the identity of the intruder target, the coordinates of the intrusion location, the intrusion timestamp, and a description of the intrusion behavior is generated. When the intrusion risk score is less than or equal to the intrusion determination threshold, the behavior of the target person is determined not to constitute an intrusion, and a normal behavior detection result is generated.

[0195] The preset intrusion detection threshold is determined based on the security level of the area of ​​interest; the higher the security level, the lower the threshold. In the GY-001 area, due to its high security level, the intrusion detection threshold is set to a lower value.

[0196] When ID-001's intrusion risk score is 0.5, and the intrusion threshold for this area is 0.4, its behavior is deemed an intrusion because 0.5 > 0.4. The generated personnel intrusion detection result includes ID-001's identification, its specific location coordinates within the GY-001 area, the timestamp of the intrusion, and a description of its intrusion behavior, such as "staying in the high-voltage transmission tower area for too long and exhibiting climbing behavior."

[0197] If the intrusion risk score of a person is less than or equal to the intrusion judgment threshold, a normal behavior detection result is generated, recording information such as the person's identity, location, and time, indicating that their behavior does not constitute an intrusion.

[0198] For example, the method may also include: Step S160: Perform event information encapsulation processing on the generated personnel intrusion detection results, and convert the intruder target's identity, intrusion location coordinates, intrusion timestamp, and intrusion behavior description in the personnel intrusion detection results into standard intrusion event records according to a preset data format.

[0199] Standard intrusion event logs use a predefined data format, which, in addition to the information mentioned above, also includes a unique identifier for the event and a region identifier where the event occurred. This format facilitates subsequent storage, retrieval, and analysis.

[0200] For example, for an intrusion event with ID-001, the standard intrusion event log will have a unique event number, such as "EVT-001", a zone identifier of "GY-001", and detailed intrusion-related information.

[0201] Step S161: Associate and fuse the standard intrusion event records with the geographic information data of the scene to be monitored to supplement the actual geographical location description and surrounding environmental features corresponding to the intrusion location coordinates.

[0202] The geographic information data of the monitored scene includes the specific geographical location of each area, the distribution of surrounding equipment, terrain features, etc. The intrusion location coordinates in the standard intrusion event record are matched with the geographic information data to obtain the actual geographical location description corresponding to the location, such as "3 meters away from the No. 1 transmission tower in the GY-001 area", as well as the surrounding environmental features, such as "there is a fence around it, and a low-voltage distribution cabinet is 5 meters to the east".

[0203] Step S162: Based on the regional priority of the intrusion concern area and the severity of the intrusion behavior in the personnel intrusion detection results, classify the standard intrusion event records into different urgency levels and assign different urgency level labels.

[0204] The higher the area priority, the more serious the intrusion, and the higher the corresponding emergency level. In power work sites, the GY-001 area has the highest priority. If an intrusion involving climbing high-voltage equipment occurs in this area, its emergency level will be set to the highest level, such as "Level 1 Emergency".

[0205] Step S163: Send the standard intrusion event record with the urgency level identifier to the management platform of the monitoring center and trigger the alarm response mechanism of the corresponding level. The alarm response mechanism includes audible and visual alarms, notification of security personnel, and activation of emergency response plan.

[0206] After receiving a standard intrusion event record, the management platform activates the corresponding alarm response mechanism based on the urgency level. For a Level 1 emergency, it can immediately trigger an audible and visual alarm at the monitoring center, simultaneously sending a notification to the mobile terminals of security personnel, containing detailed information about the intrusion event, and activating the emergency response plan, such as dispatching personnel to the scene for emergency handling.

[0207] Step S164: Store and archive the personnel intrusion detection results and the corresponding standard intrusion event records to establish an intrusion event history database.

[0208] The results of intrusion detection and the corresponding standard intrusion event records are stored in the database in chronological order, facilitating subsequent querying, statistics, and analysis. The historical database provides data support for security management; by analyzing historical intrusion events, intrusion patterns can be summarized, and intrusion detection rules and prevention measures can be optimized.

[0209] Figure 2 The illustration shows exemplary hardware and software components of a person intrusion detection system 100 based on a target detection algorithm, which can implement the ideas of this application, according to some embodiments of this application. For example, a processor 120 can be used in the person intrusion detection system 100 based on a target detection algorithm and to perform the functions in this application.

[0210] For example, the personnel intrusion detection system 100 based on target detection algorithms may include a network port 110 connected to a network, one or more processors 120 for executing program instructions, a communication bus 130, and various forms of storage media 140, such as a disk, ROM, or RAM, or any combination thereof. Exemplarily, the personnel intrusion detection system 100 based on target detection algorithms may also include program instructions stored in ROM, RAM, or other types of non-transitory storage media, or any combination thereof. The methods of this application can be implemented according to these program instructions. The personnel intrusion detection system 100 based on target detection algorithms also includes an I / O interface 150 between the computer and other input / output devices.

[0211] Furthermore, this embodiment of the invention also provides a readable storage medium, wherein computer-executable instructions are preset in the readable storage medium, and when the processor executes the computer-executable instructions, the above-mentioned personnel intrusion detection method based on the target detection algorithm is implemented.

[0212] It should be noted that, in order to simplify the description of the present invention and thus help to understand one or more embodiments of the invention, multiple features may sometimes be grouped into one embodiment, drawing or description thereof in the foregoing description of the embodiments of the present invention.

Claims

1. A method for detecting personnel intrusion based on a target detection algorithm, characterized in that, The method includes: Based on the panoramic layout image of the scene to be monitored, the intrusion concern area and intrusion judgment reference rules of the scene to be monitored are defined. The intrusion concern area is a specific spatial range in the scene to be monitored that needs to be detected for personnel intrusion. The intrusion judgment reference rules are a set of behavioral judgment criteria used to determine whether personnel behavior constitutes intrusion. Acquire the real-time monitoring video stream of the scene to be monitored, and extract the sequence of video frames to be detected with continuous temporal correlation from the real-time monitoring video stream; A pre-trained human target detection algorithm is invoked to perform frame-by-frame human target detection processing on the video frame sequence to be detected, thereby obtaining human target detection information in the video frame sequence to be detected. The human target detection information includes the position information of the human target in the video frame and the appearance feature information of the human target. Based on the intrusion concern area and the personnel target detection information, the personnel target in the video frame sequence to be detected is analyzed for regional association behavior features to obtain the regional association behavior features of the personnel target. The regional association behavior features are feature information describing the spatial positional relationship and movement trend between the personnel target and the intrusion concern area. Based on the regional association behavior characteristics and the intrusion determination reference rules, the behavior of personnel targets is processed for intrusion determination, and personnel intrusion detection results are generated. The personnel intrusion detection results are used to indicate whether there is personnel intrusion behavior that conforms to the intrusion determination reference rules in the monitored scene.

2. The personnel intrusion detection method based on target detection algorithm according to claim 1, characterized in that, The panoramic layout image based on the scene to be monitored defines the intrusion interest area and intrusion judgment reference rules for the scene to be monitored, including: Acquire a panoramic layout image of the scene to be monitored, wherein the panoramic layout image is an image that fully presents the spatial structure of the scene to be monitored; In the panoramic layout image, at least one closed polygonal region is defined as the initial region of interest by manual annotation or preset region generation rules. The boundary of the initial region of interest is determined by the vertex coordinates of multiple vertices in the pixel coordinate system of the panoramic layout image. The initial focus area is configured with regional attributes, including assigning a unique regional identifier to each initial focus area, setting regional priority, and specifying a time range limit for the appearance of people within the area. The initial area of ​​interest with the area attributes set is identified as the intrusion area of ​​interest, and the area identifier, vertex coordinate set, area priority and time range limit of the intrusion area of ​​interest are stored in the area configuration file. Based on the security management requirements of the monitored scenario, intrusion judgment reference rules are formulated. The intrusion judgment reference rules include the duration threshold of personnel entering the intrusion concern area, the movement trajectory pattern of personnel in the intrusion concern area, the behavior action type of personnel in the intrusion concern area, and the interaction method between personnel and the boundary of the intrusion concern area. The intrusion determination reference rules are associated and bound with the corresponding area identifiers of the intrusion concern areas, and the associated and bound intrusion determination reference rules are stored in the rule database to form a set of intrusion determination reference rules that correspond one-to-one with the intrusion concern areas.

3. The personnel intrusion detection method based on target detection algorithm according to claim 1, characterized in that, The process of calling a pre-trained personnel target detection algorithm to perform frame-by-frame personnel target detection processing on the video frame sequence to be detected, and obtaining personnel target detection information in the video frame sequence to be detected, includes: Each video frame in the sequence of video frames to be detected is converted into standard image data that matches the input format of the personnel target detection algorithm. The standard image data has a preset image size and color channel mode. The feature extraction backbone network of the personnel target detection algorithm is invoked to perform deep feature extraction processing on the standard image data to generate a multi-scale image feature map, which contains image semantic features and detail features at different levels. The region proposal network of the personnel target detection algorithm performs potential personnel target region generation processing on the multi-scale image feature map to obtain multiple candidate personnel target bounding boxes. Each candidate personnel target bounding box has coordinate position and size information in standard image data. Non-maximum suppression is performed on the multiple candidate target bounding boxes to remove redundant candidate target bounding boxes with an overlap exceeding a preset threshold, and candidate target bounding boxes with a confidence level greater than a set confidence threshold are retained as target bounding boxes. The classification and regression head of the personnel target detection algorithm is invoked to classify and refine the regional features corresponding to the target bounding box, and to determine the category probability and bounding box coordinates of the personnel target within each target bounding box. The category probability is used to represent the probability that the target bounding box contains a personnel target. The target bounding box with a category probability greater than a preset probability threshold is determined as a valid bounding box containing the person target, and the appearance feature information of the person target within the valid bounding box is extracted. The appearance feature information includes the clothing color feature, body shape feature and carried item feature of the person target. The bounding box coordinates of the effective bounding box and the corresponding appearance feature information are combined into personnel target detection information, and the personnel target detection information is sorted and stored according to the time order of the video frames to be detected in the video frame sequence to be detected.

4. The personnel intrusion detection method based on target detection algorithm according to claim 3, characterized in that, The feature extraction backbone network that invokes the personnel target detection algorithm performs deep feature extraction processing on the standard image data to generate multi-scale image feature maps, including: The standard image data is input into the initial convolutional layer of the feature extraction backbone network. The standard image data is then subjected to convolutional filtering and downsampling through the initial convolutional layer to generate a first-level feature map. The first-level feature map is input into the first residual module group of the feature extraction backbone network. The first residual module group contains multiple cascaded residual units. Each residual unit performs feature enhancement and dimensionality improvement on the input feature map through a skip connection structure and outputs a second-level feature map. The number of feature channels in the second-level feature map is greater than the number of feature channels in the first-level feature map. The second-level feature map is input into the second residual module group of the feature extraction backbone network. The second residual module group performs further feature abstraction and spatial dimensionality reduction on the second-level feature map to generate a third-level feature map. The spatial size of the third-level feature map is smaller than that of the second-level feature map. The third-level feature map is input into the third residual module group of the feature extraction backbone network. Multiple convolution operations with different kernel sizes are used to extract multiple receptive field features from the third-level feature map to generate a fourth-level feature map. The fourth-level feature map contains local image features at different scales. The fourth-level feature map is input into the fourth residual module group of the feature extraction backbone network to perform global context information modeling processing on the fourth-level feature map and generate the fifth-level feature map. The first-level feature map, the second-level feature map, the third-level feature map, the fourth-level feature map, and the fifth-level feature map are combined into a multi-scale image feature map. The feature maps of different levels in the multi-scale image feature map have different resolutions in the spatial dimension and different feature representation capabilities in the channel dimension.

5. The personnel intrusion detection method based on target detection algorithm according to claim 3, characterized in that, The region proposal network of the personnel target detection algorithm performs potential personnel target region generation processing on the multi-scale image feature map to obtain multiple candidate personnel target bounding boxes, including: Anchor boxes with preset scale and aspect ratio are generated for each level feature map in the multi-scale image feature map, and multiple initial anchor boxes with different sizes and shapes are generated at each pixel position of each level feature map. The convolutional layers of the region proposal network perform feature mapping processing on the feature maps of each level to generate class scores and bounding box offsets corresponding to the number of initial anchor boxes. The class scores are used to represent the probability that the initial anchor boxes contain human targets, and the bounding box offsets are used to adjust the position and size of the initial anchor boxes. The initial anchor boxes are filtered based on the category scores, and the initial anchor boxes with category scores greater than a preset category threshold are retained as candidate anchor boxes. The candidate anchor box is corrected by using the bounding box offset to obtain an adjusted candidate bounding box. The position and size of the adjusted candidate bounding box are closer to the boundary of the real person target. The adjusted candidate bounding boxes generated on feature maps at different levels are mapped to the pixel coordinate system of the standard image data to obtain multiple candidate target bounding boxes in the standard image data space. The mapped candidate bounding boxes are merged across levels to eliminate duplicate candidate bounding boxes caused by the generation of multi-scale feature maps, forming a final set of multiple candidate bounding boxes.

6. The personnel intrusion detection method based on target detection algorithm according to claim 1, characterized in that, Based on the intrusion interest area and the personnel target detection information, the method performs region-related behavior feature analysis on the personnel targets in the video frame sequence to be detected, and obtains the region-related behavior features of the personnel targets, including: The effective bounding box coordinates in the personnel target detection information are transformed from the image coordinate system of the video frame to be detected to the pixel coordinate system of the panoramic layout image to obtain the actual position coordinates of the personnel target in the panoramic layout image. Determine whether the actual location coordinates fall within the closed polygonal area of ​​the intrusion concern area, and determine the spatial attribution relationship between the personnel target and the intrusion concern area. The spatial attribution relationship includes the personnel target being located inside the intrusion concern area, the personnel target being located outside the intrusion concern area, and the personnel target intersecting the boundary of the intrusion concern area. Based on the personnel target detection information of multiple consecutive frames in the video frame sequence to be detected, the actual position coordinate sequence of the same personnel target is extracted, and the actual position coordinate sequence is subjected to trajectory fitting processing to obtain the movement trajectory curve of the personnel target. The movement direction vector and movement speed characteristics of the personnel target are calculated based on the movement trajectory curve. The movement direction vector represents the position change trend of the personnel target per unit time, and the movement speed characteristics represent the speed of the position change of the personnel target. Analyze the intersection points of the movement trajectory curve and the boundary of the intrusion concern area to determine the coordinates and time points of the boundary intersection points when the personnel target approaches or leaves the intrusion concern area; By combining the spatial attribution relationship, movement trajectory curve, movement direction vector, movement speed characteristics, boundary intersection coordinates and intersection time points, a regional association behavior feature of the personnel target is constructed. The regional association behavior feature can comprehensively reflect the dynamic behavior process of the personnel target relative to the intrusion interest area.

7. The personnel intrusion detection method based on target detection algorithm according to claim 6, characterized in that, The step of extracting the actual position coordinate sequence of the same person target based on the person target detection information of multiple consecutive frames in the video frame sequence to be detected, and performing trajectory fitting processing on the actual position coordinate sequence to obtain the movement trajectory curve of the person target includes: Personnel target detection information of adjacent video frames in the video frame sequence to be detected is subjected to personnel target matching processing. By calculating the similarity of appearance feature information and position distance similarity of personnel targets, the correspondence of the same personnel target in different video frames is determined, and the identity identifier of personnel targets is established. Based on the identity identifier, the actual location coordinates of the same person target are arranged in the time order of the video frames in which they are located, forming a sequence of actual location coordinates of the person target. Each element in the sequence of actual location coordinates contains a timestamp and the corresponding actual location coordinates. Anomaly detection processing is performed on the actual position coordinate sequence to identify and remove abnormal position coordinates that deviate from the normal trajectory due to detection errors; A polynomial curve fitting algorithm is used to perform curve fitting on the actual position coordinate sequence after outlier removal to generate a smooth initial trajectory curve. Curvature analysis is performed on the initial trajectory curve to identify inflection points and straight line segments in the trajectory curve, and the initial trajectory curve is divided into multiple trajectory segments according to the position of the inflection points; The trajectory parameters of each trajectory segment are calculated, and the initial trajectory curve is optimized and adjusted based on the trajectory parameters to obtain a movement trajectory curve that can accurately reflect the movement pattern of the personnel target. The trajectory parameters include trajectory length, average movement speed, and angle of change of direction.

8. The personnel intrusion detection method based on target detection algorithm according to claim 1, characterized in that, The step of performing intrusion detection processing on the behavior of personnel targets based on the regional association behavior characteristics and the intrusion determination reference rules, and generating personnel intrusion detection results, includes: Retrieve the set of intrusion judgment reference rules corresponding to the intrusion concern area from the rule database; The spatial affiliation relationship in the area-related behavioral characteristics is compared with the area entry conditions in the intrusion determination reference rules to determine whether the personnel target meets the preliminary conditions for entering the intrusion concern area. When the personnel target meets the preliminary conditions, extract the duration of the personnel target being located inside the intrusion concern area from the regional association behavior features, and compare it with the duration threshold in the intrusion judgment reference rules. Analyze the movement trajectory curves and movement direction vectors in the regional association behavior characteristics to determine whether the movement trajectory pattern of the personnel target in the intrusion interest area conforms to the preset suspicious trajectory pattern in the intrusion judgment reference rules; By combining the appearance feature information in the regional association behavior characteristics, the types of human target behavior actions in the intrusion concern area are identified, and it is determined whether there are dangerous behaviors prohibited by the intrusion judgment reference rules. Based on the interaction between personnel targets and the boundary of the intrusion concern area, determine whether personnel targets are attempting to cross, damage the boundary, or evade monitoring. The results of each judgment are comprehensively and weighted, and the intrusion risk score of the personnel target is calculated according to the preset weight allocation rules. The intrusion risk score comprehensively reflects the possibility that the behavior of the personnel target constitutes an intrusion. The intrusion risk score is compared with a preset intrusion determination threshold. When the intrusion risk score is greater than the intrusion determination threshold, the behavior of the target person is determined to constitute an intrusion, and a personnel intrusion detection result containing the identity of the intruder target, the coordinates of the intrusion location, the intrusion timestamp, and a description of the intrusion behavior is generated. When the intrusion risk score is less than or equal to the intrusion determination threshold, the behavior of the target person is determined not to constitute an intrusion, and a normal behavior detection result is generated.

9. The personnel intrusion detection method based on target detection algorithm according to claim 8, characterized in that, The analysis of the movement trajectory curves and movement direction vectors in the associated behavioral characteristics of the analysis area determines whether the movement trajectory pattern of the target person within the intrusion interest area matches the preset suspicious trajectory pattern in the intrusion determination reference rules, including: The trajectory curve is subjected to trajectory morphology feature extraction, and the total length, average curvature, rate of curvature change, trajectory enclosed area and direction angle change range of the trajectory curve are calculated as trajectory morphology parameters. The extracted trajectory morphology parameters are compared with the trajectory morphology parameter templates corresponding to the preset suspicious trajectory patterns in the intrusion judgment reference rules to calculate the similarity and obtain the trajectory morphology similarity value. Based on the movement direction vector sequence, the frequency of direction change and the direction entropy value of the personnel target in the intrusion concern area are calculated. The frequency of direction change represents the number of times the movement direction changes per unit time, and the direction entropy value represents the degree of uncertainty of the movement direction. The direction change frequency and direction entropy value are compared with the corresponding direction feature threshold in the suspicious trajectory pattern to determine whether the movement direction feature of the personnel target matches the suspicious trajectory feature. Analyze the distance relationship between the movement trajectory curve and the location of critical facilities within the area of ​​concern for intrusion, calculate the shortest distance from the movement trajectory curve to each critical facility, and determine whether there are trajectory segments that are close to the critical facilities; Based on the trajectory shape similarity value, the comparison results of directional features, and the proximity of key facilities, a comprehensive judgment is made as to whether the movement trajectory pattern of the personnel target belongs to a suspicious trajectory pattern. When the trajectory shape similarity value is greater than the preset similarity threshold, the directional features meet the suspicious trajectory features, or there is a trajectory segment close to the key facilities, the movement trajectory pattern is determined to be a suspicious trajectory pattern.

10. A personnel intrusion detection system based on a target detection algorithm, characterized in that... The device includes a processor and a memory, the memory being connected to the processor. The memory is used to store programs, instructions, or code, and the processor is used to execute the programs, instructions, or code in the memory to implement the personnel intrusion detection method based on the target detection algorithm as described in any one of claims 1-9.

Images