Warning area intrusion alarm method, device and equipment based on computer vision

Abstract

The invention provides a warning area intrusion alarm method, device and equipment based on computer vision, and the method comprises the steps: obtaining a first to-be-detected image of a to-be-detected area, inputting the first to-be-detected image into a pre-trained intrusion detection model, and obtaining target labeling information corresponding to the first to-be-detected image, and if the target labeling information indicates that the target object exists in the to-be-detected area, determining whether the target object is an intrusion object based on the first position information of the target object and a warning area in the to-be-detected area, and if the target object is the intrusion object, triggering an alarm. According to the technical scheme, the image of the to-be-detected area is acquired and input to the pre-trained intrusion detection model, whether the target object has the intrusion behavior is judged according to the model output result, and if the target object is determined to be an intrusion object, the alarm is triggered. The technical effects of real-time monitoring and accurate and efficient detection of the intrusion condition of the warning area are realized.

Description

Technical Field

[0001] The present application relates to the technical field of substations, and in particular to a method, device, and equipment for alarming intrusion into a warning area based on computer vision. Background Art

[0002] The safety protection of substations is an important measure to ensure the normal operation of substation equipment, avoid accidents, and protect personnel and equipment. In order to ensure the safety of substations, real-time monitoring and alarm of the warning area around the substation are required.

[0003] In the existing technology, on-site supervision and remote video monitoring are used to monitor the intrusion of the warning area around the substation. However, there are technical problems such as low efficiency and low real-time performance of the intrusion monitoring of the warning area around the substation. Summary of the Invention

[0004] The embodiments of the present application provide a computer vision-based warning area intrusion alarm method, device and equipment to solve the problems of low efficiency and low real-time performance of warning area intrusion monitoring around substations in the existing technology.

[0005] In a first aspect, an embodiment of the present application provides a method for alarming an intrusion in a warning area based on computer vision, comprising:

[0006] Acquire a first image to be detected of the area to be detected;

[0007] Inputting the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, wherein the target annotation information is used to indicate whether a target object exists in the area to be detected and first position information of the target object;

[0008] If the target annotation information indicates that the target object exists in the area to be detected, determining whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected;

[0009] If the target object is the intrusion object, an alarm is triggered.

[0010] In a possible implementation, before inputting the first image to be detected into a pre-trained intrusion detection model to obtain target labeling information corresponding to the first image to be detected, the method includes:

[0011] Obtaining a first data set and annotation information corresponding to the first data set, wherein the first data set includes: a plurality of first images containing objects, and annotation information corresponding to each first image, the annotation information being used to indicate the object in the first image and location information of the object;

[0012] performing data augmentation processing on a plurality of first images in the first data set to obtain a second data set, wherein the second data set includes: a plurality of second images, wherein for each second image, the second image is an image that occludes any portion of an object in any of the first images;

[0013] The YOLOv5 model is trained according to the first data set, the second data set, and the annotation information corresponding to the first data set to obtain the intrusion detection model.

[0014] In a possible implementation, the YOLOv5 model includes: a Backbone module, a Neck module, and a prediction layer, wherein the prediction layer includes: a medium target layer and a small target layer;

[0015] Accordingly, the step of inputting the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected includes:

[0016] Inputting the first image to be detected into the Backbone module for feature extraction processing to obtain a multi-scale feature map, wherein the Backbone module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence;

[0017] Inputting the multi-scale feature map into the Neck module for feature fusion processing to obtain a dual-scale feature map, wherein the Neck module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence;

[0018] The dual-scale feature map is input into the prediction layer for prediction processing to obtain target labeling information corresponding to the first image to be detected.

[0019] In a possible implementation, determining whether the target object is an intrusion object based on the first location information of the target object and the warning area in the area to be detected includes:

[0020] determining whether the target object is an intrusion object based on intersection-over-union data of the first position information and the warning area;

[0021] Alternatively, a second image to be detected at a next moment after the first image to be detected is obtained, and whether the target object is the intrusion object is determined based on the second image to be detected, the first position information, and the warning area.

[0022] In a possible implementation, determining whether the target object is the intrusion object based on intersection-over-union data between the first location information and the warning area includes:

[0023] If the intersection-over-union ratio data of the first position information and the warning area is greater than or equal to a preset threshold, determining that the target object is the intrusion object;

[0024] If the intersection-over-union ratio data of the first position information and the warning area is smaller than a preset threshold, it is determined that the target object is not the intrusion object.

[0025] In a possible implementation, determining whether the target object is an intrusion object based on the second image to be detected, the first location information, and the warning area includes:

[0026] S1, determining a line to be detected based on a center point of second position information corresponding to the second image to be detected and a center point of the first position information;

[0027] S2, determining a target circle to be detected with the center of the line to be detected as the center and a preset threshold as the radius;

[0028] S3, if the target circle to be detected intersects the warning area, determining that the target object is the intruder;

[0029] S4: If there is no intersection between the target circle to be detected and the warning area, it is determined that the target object is not the intrusion object.

[0030] In one possible implementation, the method further includes:

[0031] When the target object is not the intrusion object, the second image to be detected is used as a new first image to be detected, the image to be detected at the next moment of the second image to be detected is used as a new second image to be detected, and steps S1 to S4 are repeated.

[0032] In a second aspect, an embodiment of the present application provides a warning area intrusion alarm device based on computer vision, comprising:

[0033] An acquisition module, configured to acquire a first image to be detected of the area to be detected;

[0034] a processing module, configured to input the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, wherein the target annotation information is used to indicate whether a target object exists in the area to be detected and first position information of the target object;

[0035] a determination module, configured to determine whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected if the target annotation information indicates that the target object exists in the area to be detected;

[0036] The trigger module is used to trigger an alarm if the target object is the intrusion object.

[0037] In a possible implementation, before inputting the first image to be detected into a pre-trained intrusion detection model to obtain target labeling information corresponding to the first image to be detected, the processing module is specifically configured to:

[0038] Obtaining a first data set and annotation information corresponding to the first data set, wherein the first data set includes: a plurality of first images containing objects, and annotation information corresponding to each first image, the annotation information being used to indicate the object in the first image and location information of the object;

[0039] performing data augmentation processing on a plurality of first images in the first data set to obtain a second data set, wherein the second data set includes: a plurality of second images, wherein for each second image, the second image is an image that occludes any portion of an object in any of the first images;

[0040] The YOLOv5 model is trained according to the first data set, the second data set, and the annotation information corresponding to the first data set to obtain the intrusion detection model.

[0041] In a possible implementation, the YOLOv5 model includes: a Backbone module, a Neck module, and a prediction layer, wherein the prediction layer includes: a medium target layer and a small target layer;

[0042] Accordingly, the processing module is specifically configured to:

[0043] Inputting the first image to be detected into the Backbone module for feature extraction processing to obtain a multi-scale feature map, wherein the Backbone module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence;

[0044] Inputting the multi-scale feature map into the Neck module for feature fusion processing to obtain a dual-scale feature map, wherein the Neck module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence;

[0045] The dual-scale feature map is input into the prediction layer for prediction processing to obtain target labeling information corresponding to the first image to be detected.

[0046] In a possible implementation, the determining module is specifically configured to:

[0047] determining whether the target object is an intrusion object based on intersection-over-union data of the first position information and the warning area;

[0048] Alternatively, a second image to be detected at a next moment after the first image to be detected is obtained, and whether the target object is the intrusion object is determined based on the second image to be detected, the first position information, and the warning area.

[0049] In a possible implementation, the determination module determines whether the target object is the intrusion object based on the intersection-over-union data of the first location information and the warning area, specifically for:

[0050] If the intersection-over-union ratio data of the first position information and the warning area is greater than or equal to a preset threshold, determining that the target object is the intrusion object;

[0051] If the intersection-over-union ratio data of the first position information and the warning area is smaller than a preset threshold, it is determined that the target object is not the intrusion object.

[0052] In a possible implementation, the determination module determines whether the target object is an intrusion object based on the second image to be detected, the first location information, and the warning area, specifically for:

[0053] S1, determining a line to be detected based on a center point of second position information corresponding to the second image to be detected and a center point of the first position information;

[0054] S2, determining a target circle to be detected with the center of the line to be detected as the center and a preset threshold as the radius;

[0055] S3, if the target circle to be detected intersects the warning area, determining that the target object is the intruder;

[0056] S4: If there is no intersection between the target circle to be detected and the warning area, it is determined that the target object is not the intrusion object.

[0057] In a possible implementation, the determining module is further configured to:

[0058] When the target object is not the intrusion object, the second image to be detected is used as a new first image to be detected, the image to be detected at the next moment after the second image to be detected is used as a new second image to be detected, and steps S1 to S4 are repeated.

[0059] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0060] The memory stores computer-executable instructions;

[0061] The processor executes the computer-executable instructions stored in the memory to implement the method as described in the first aspect or any one of the above methods.

[0062] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect or any of the above methods.

[0063] In a fifth aspect, an embodiment of the present application provides a computer program, wherein the computer program product includes a computer program, wherein the computer program is stored in a computer-readable storage medium, and at least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, the method described in the first aspect or any one of the methods can be implemented.

[0064] The embodiments of the present application provide a computer vision-based warning area intrusion alarm method, device, and equipment. The method first acquires a first image of the area to be detected, then inputs the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected. If the target annotation information indicates that a target object exists in the area to be detected, based on the first position information of the target object and the warning area in the area to be detected, it is determined whether the target object is an intrusion object. If the target object is an intrusion object, an alarm is triggered. This technical solution acquires an image of the area to be detected and inputs it into a pre-trained intrusion detection model. Based on the output of the model, it determines whether the target object has committed an intrusion. If the target object is confirmed to be an intrusion object, an alarm is triggered, thereby achieving real-time monitoring and accurate detection of the warning area and improving the intelligence and efficiency of security prevention in the warning area. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0066] Figure 1 A schematic diagram of the architecture of the intrusion detection model provided in an embodiment of the present application;

[0067] Figure 2 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 1 ;

[0068] Figure 3 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 2 ;

[0069] Figure 4 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 3 ;

[0070] Figure 5 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 4 ;

[0071] Figure 6 Schematic diagram of the process of the warning area intrusion method based on computer vision provided in the embodiment of the present application Figure 5 ;

[0072] Figure 7 A schematic diagram of the structure of a warning area intrusion alarm device based on computer vision provided in an embodiment of the present application;

[0073] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application.

[0074] The above drawings illustrate specific embodiments of the present application, which will be described in more detail below. These drawings and the textual description are not intended to limit the scope of the present application in any way, but rather to illustrate the concepts of the present application to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0075] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0076] Before introducing the embodiments of the present application, the application background of the embodiments of the present application is first explained:

[0077] Currently, substation security primarily relies on a combination of technical and personnel defenses. Among these technical defenses, security fences are a key measure, designed to prevent unauthorized access. However, fences are not permanent installations and are susceptible to human influence, posing a risk of vandalism or removal. Furthermore, in cross-operation environments, fences can be accidentally removed by operators, further reducing security.

[0078] Civil defense measures primarily rely on on-site supervision and remote video surveillance. However, these monitoring methods often focus on post-event investigations and have limited preventative capabilities. Most existing measures only address construction workers' arbitrary entry into designated areas and other violations after the fact, making it difficult to prevent them in a timely manner and increasing safety risks.

[0079] The safety protection of substations is an important measure to ensure the normal operation of substation equipment, avoid accidents, and protect personnel and equipment. In order to ensure the safety of substations, real-time monitoring of the warning area around the substation is required.

[0080] In summary, the above-mentioned combination of technical defense measures and human defense measures to monitor intrusion into the warning area around the substation has technical problems such as low efficiency and low real-time performance of intrusion monitoring in the warning area around the substation.

[0081] In response to the technical problems existing in the prior art, the inventors of this application have come up with the following idea: to address the problems of low efficiency and low real-time performance of intrusion monitoring in the warning area around the substation, computer vision and target detection technology are used to achieve target detection in the warning area. If there is an intrusion, an alarm is triggered, thereby achieving the purpose of real-time and efficient determination of the intrusion situation in the warning area, thereby improving the detection efficiency and real-time performance of the warning area. Specifically, the image to be detected is input into a pre-trained intrusion detection model for prediction to obtain the annotation information of the target object in the image to be detected. Whether the target object is an intrusion object is determined based on the position information in the annotation information and the warning area in the area to be detected. If the target object is an intrusion object, an alarm is triggered. In some feasible implementations, the intrusion detection model is a YOLOv5 model with a large target layer removed, which improves the calculation speed of the model and thereby improves the real-time performance of intrusion detection in the warning area. The intrusion detection model obtained by improving the YOLOv5 model by adding an iterative residual multi-head attention module (iRMB) and replacing ordinary convolution with a deformable convolution network (DCN) can extract deeper features of the image to be detected, thereby improving the detection efficiency of the intrusion detection model. Based on the relationship between the annotation information of the target object and the warning area obtained by the intrusion detection model, it is determined whether there is an intrusion.

[0082] Specifically, Figure 1 The schematic diagram of the architecture of the intrusion detection model provided in the embodiment of the present application is as follows: Figure 1As shown in FIG, the architecture of the intrusion detection model includes: an input module, a backbone network module, a neck module, and a prediction layer.

[0083] The role of the input module is to receive raw data (usually images or video frames) and perform preprocessing such as resizing, normalization, and enhancement (such as rotation, translation, and cropping) on it, so as to convert the input data into a format suitable for model training.

[0084] Backbone module: extracts low-level and high-level features from the input data. These features usually include low-level features such as edges, textures, and shapes, as well as high-level features such as object contours, colors, and categories.

[0085] For example, the Backbone module includes a focus layer (English: Focus) to increase the model's perception and efficiency by separating input channels before the convolution operation; the cross-stage partial network (CSP) improves computational efficiency and reduces the amount of computation by sharing some feature information between different stages of the network. CSP1_X refers to a CSP structure that includes a convolution-batch normalization-leaky linear rectifier unit (Convolution-BatchNorm-Leaky Relu, CBL), X residual components (English: Res unit), a convolutional layer (Conv), a concatenation layer (Concat), a batch normalization (BN), and a leaky relu layer.

[0086] Among them, the residual unit (Residual Unit, Res unit) includes two CBL layers and a feature accumulation layer, which is to solve the gradient disappearance or gradient explosion problems that may occur in deep neural network training; CSP2_X refers to the CSP structure that does not contain the Res unit layer (such as Figure 1 shown).

[0087] In addition, the Spatial Pyramid Pooling (SPP) layer maintains the spatial information of the image by performing pooling operations at multiple scales and does not require resizing the input image, thereby enhancing the flexibility of the detection model.

[0088] Neck module: further processes and fuses the features extracted by the backbone network to generate more effective and richer feature representations.

[0089] Among them, the upsampling layer enlarges the size or resolution of the data in some way.

[0090] Prediction layer: Responsible for predicting specific tasks based on the features obtained from the Backbone module and the Neck module. In the embodiment of the present application, the prediction layer generally generates prediction results about the target object category, location, etc.

[0091] Parts not described in detail are disclosed by the following embodiments.

[0092] The technical solution of the present application is described in detail below through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0093] It is worth noting that the application fields of the computer vision-based warning area intrusion alarm method, device and equipment of this application are not limited.

[0094] Among them, the executor of this application is the monitoring system, and the specific monitoring system can be electronic equipment such as servers and terminal devices.

[0095] Figure 2 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 1 ,like Figure 2 As shown, the method may include the following steps:

[0096] Step 21: Acquire a first image to be detected of the area to be detected;

[0097] In this step, the monitoring system acquires in real time the first image to be detected of the area to be detected captured by the image capture device.

[0098] Among them, the area to be detected is the area that can be monitored by the monitoring camera installed in the substation, and the first image to be detected is the image corresponding to the first frame of the video obtained by frame extraction based on the real-time video stream collected by the image acquisition device.

[0099] For example, the images captured by the image acquisition device may contain some unnecessary noise, which reduces the image quality and affects the detection effect of downstream tasks.

[0100] In one possible implementation, the embodiment of the present application uses a median filter method to effectively filter out noise data in the image. For all pixel values in the neighborhood of a given pixel point (x, y) in the first image to be detected, it is represented as a pixel set W. The output of the median filter can be expressed as:

[0101] g(x,y)=med{f(xk,yl)|(k,l)∈W}

[0102] Wherein, f(x, y) is the original image (that is, the first image to be detected), g(x, y) is the image after median filtering, W is a two-dimensional convolution template, usually a 3×3 or 5×5 square area, but can also be a linear, circular or other different shapes, k refers to the horizontal position offset of a given pixel point in the first image to be detected, l refers to the vertical position offset of a given pixel point in the first image to be detected, and the embodiment of the present application adopts the median value in the 3×3 convolution template.

[0103] Step 22: Input the first image to be detected into a pre-trained intrusion detection model to obtain target labeling information corresponding to the first image to be detected.

[0104] The target annotation information is used to indicate whether a target object exists in the area to be detected and the first position information of the target object;

[0105] In this step, the first image to be detected is input into a pre-trained intrusion detection model for processing. The intrusion detection model identifies the target object in the image based on the features of the pre-trained image and generates target annotation information, providing accurate data support for subsequent judgment.

[0106] In a possible implementation, the target annotation information indicates that the target object category is “person”, and the position information of the detection box of the target object includes the upper left corner coordinates (100, 100) and the lower right corner coordinates (300, 300).

[0107] Step 23: If the target marking information indicates that there is a target object in the area to be detected, determine whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected.

[0108] In this step, the monitoring system determines whether the target labeling information indicates that there is a target object in the area to be detected. If it is confirmed that there is a target object, the monitoring system will determine whether the target object has invaded the area based on the positional relationship between the first position information of the target object and the warning area. If there is an area invasion, the monitoring system will determine that the target object is an intrusion object.

[0109] Among them, the area to be detected is the area that can be monitored by the monitoring camera installed in the substation, and the warning area is the area within the preset range from the substation drawn in the area to be detected in the monitoring camera visualization interface.

[0110] In one possible implementation, the preset range may be 1 meter.

[0111] For example, the area intrusion determination may include the intersection-over-union determination method and the person tracking method. These two determination methods will be described in the following Figure 4and Figure 5 The corresponding parts are introduced in detail.

[0112] Step 24: If the target object is an intrusion object, trigger an alarm.

[0113] In this step, if the monitoring system determines that the target object is an intrusion object, the monitoring system will immediately trigger an alarm mechanism.

[0114] Among them, the above-mentioned alarm mechanism usually includes measures such as issuing alarm prompts, notifying security personnel, and recording images or videos, so as to quickly respond and take corresponding safety protection measures to ensure the safety of the warning area.

[0115] In a possible implementation, the alarm prompt may be “Danger area, do not enter”.

[0116] The computer vision-based warning area intrusion alarm method provided in an embodiment of the present application first acquires a first image of the area to be detected, then inputs the first image into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected. If the target annotation information indicates that a target object exists in the area to be detected, the method determines whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected. If the target object is an intrusion object, an alarm is triggered. This technical solution achieves the technical effect of real-time monitoring and accurate and efficient detection of intrusions in the warning area by acquiring an image of the area to be detected and inputting it into a pre-trained intrusion detection model. If the target object is an intrusion object, an alarm is triggered.

[0117] Based on the above embodiments, Figure 3 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 2 ,like Figure 3 As shown, before step 22, the following steps may be included:

[0118] Step 31: Obtain a first data set and annotation information corresponding to the first data set.

[0119] The first data set includes: a plurality of first images containing objects, and annotation information corresponding to each first image, where the annotation information is used to indicate the object and position information of the object in the first image;

[0120] In this step, a first data set is obtained, and at least one target object in an image in the first data set is annotated to obtain corresponding annotation information.

[0121] Exemplarily, the first data set contains multiple images, each of which contains one or more target objects. Each image has corresponding annotation information, which includes the category information and location information of the target object. The location information records the location information of each target object in the image in detail, and a rectangular box (English: bounding box) is usually used to represent the location of the target object. The accuracy of the annotation information is crucial for the subsequent model training.

[0122] Among them, the embodiment of the present application does not limit the size of the first image (ie, the model input image), and can be adjusted according to actual conditions.

[0123] Exemplarily, the size of the first image as the input image of the model can be 640*640*3.

[0124] In a possible implementation, the first data set is collected in the following three ways:

[0125] Method 1: Use handheld cameras to capture images of pedestrians from different angles. Handheld cameras include mobile phones and cameras to obtain images of pedestrians at construction sites and in different scenarios.

[0126] Method 2: Use a drone to capture pedestrian images at different angles, heights, and weather conditions. These weather conditions can include sunny, cloudy, foggy, and lightly rainy days to ensure the diversity of image data.

[0127] Method 3: Use fixed and mobile cameras at the substation to capture images of pedestrians at multiple angles, heights, and weather conditions.

[0128] The images collected by the above methods 1, 2, and 3 are cleaned, and pictures that do not meet the requirements are eliminated, including images without people and images with poor imaging quality, to ensure the validity of the data set and obtain the first data set.

[0129] In one possible implementation, the tag information may be determined in the following manner:

[0130] The LabelImg annotation software is used to annotate the target objects in the pictures in the first dataset with frames, and the corresponding Extensible Markup Language (XML) files are generated. The pictures in the first dataset and the corresponding XML files are associated and saved to obtain the annotation information corresponding to the first dataset.

[0131] Step 32: Perform data enhancement processing on the multiple first images in the first data set to obtain a second data set.

[0132] The second data set includes: a plurality of second images, and for each second image, the second image is an image that covers any part of an object in any first image;

[0133] In this step, multiple first images in the first data set are enhanced using data homogeneity enhancement and data hybrid enhancement to generate a second data set for subsequent model training, ensuring that the model can learn how to identify partially occluded or incompletely visible target objects, thereby improving the robustness of the intrusion detection model in actual environments.

[0134] In a possible implementation, data homogeneity enhancement is to enhance the first data set by using techniques such as flipping, rotating, scaling, shifting, and blurring.

[0135] Considering the complexity of substation equipment, camera placement, and shooting angles, the target object may be obscured by other objects, resulting in the loss of some features and affecting detection accuracy. Therefore, the first dataset after data homogeneity enhancement is further enhanced with data hybridization.

[0136] Existing cutout data hybrid enhancement has the following shortcomings:

[0137] a. The Cotout random occlusion area is for the entire image only. Sometimes the occlusion area does not cover the target object, resulting in invalid occlusion.

[0138] b. The Cutout occlusion area only considers square occlusion, but does not consider occlusions of different sizes and shapes, which does not meet the actual situation and results in low model performance.

[0139] Therefore, the present embodiment proposes the Cutout++ data mixing enhancement method to improve the generalization ability of the model, which specifically includes the following steps:

[0140] 1) Obtain the specific location information of all target object annotation boxes in the current image according to the annotation XML file (representing the coordinates of the upper left corner of the annotation box (x min ,y min ) and the lower right corner coordinate (x max ,y max )).

[0141] 2) According to the specific location information of the annotation box of each target object, a number of to-be-occluded areas of different sizes and shapes are randomly generated in the annotation box, and the to-be-occluded areas are filled with all zeros to occlude them.

[0142] Among them, areas of different sizes are areas covered by 5%, 10%, 15%, 20%, 25%, 30%, 35%, 40%, 45%, and 50% of the area of the annotation box; areas of different shapes can be squares, rectangles, triangles, circles, trapezoids, etc.

[0143] By limiting the size of the occluded area, we can avoid excessive occlusion affecting data quality. The Cutout++ data enhancement method can simulate occlusion more realistically.

[0144] In order to avoid the impact of filling 0 values on model training, the data should be normalized as follows:

[0145] The generated occlusion image is drawn from the upper left corner (x min ,y min ) starts filling, and moves at preset pixel intervals each time until it reaches the lower right corner (x max ,y max If the interval element between the last pixel and the second last pixel does not meet the preset pixel point, the last element is still blocked. The number of blocked images that can be generated for each first image is:

[0146]

[0147] in, Indicates rounding up (i.e., the smallest integer greater than or equal to x), and gap is the moving interval pixel point.

[0148] In a possible implementation, the preset pixel points are 8 pixel points (ie, gap=8).

[0149] The second data set is obtained by masking the annotation box in the annotation information of the first image in the first data set. Therefore, the annotation information of the second data set is completely identical to the annotation information of the first data set.

[0150] Step 33: Train the YOLOv5 model based on the first data set, the second data set, and the annotation information corresponding to the first data set to obtain an intrusion detection model.

[0151] In this step, the YOLOv5 model is trained using the first data set, the second data set, and the annotation information corresponding to the first data set. During the training process, the YOLOv5 model gradually grasps the characteristics and position distribution of the target object by learning the images and their annotation information in the first and second data sets.

[0152] Among them, the enhanced images in the second dataset help improve the model's recognition ability when facing occlusion or complex backgrounds, ensuring that the trained intrusion detection model has higher accuracy and robustness.

[0153] Optionally, the YOLOv5 model in step 33 includes: a Backbone module, a Neck module, and a prediction layer.

[0154] Among them, the prediction layer includes: medium target layer and small target layer;

[0155] In one possible implementation, the camera that collects the first image to be detected is generally installed at a certain distance from the warning area. Therefore, compared to the target object (usually a pedestrian) and the area of the warning area, if the warning area can be fully monitored, the camera's monitoring screen will be further scaled or the camera will be moved back, and the corresponding target object will become smaller in the monitoring screen. Therefore, the target object occupies fewer pixels in the image. Therefore, the embodiment of the present application mainly detects small and medium targets. The embodiment of the present application deletes the large target detection layer in the YOLOv5 model. This will not affect the accuracy of the detection, but will greatly improve the detection speed of the YOLOv5 model and reduce the configuration of hardware facilities.

[0156] Among them, the embodiment of the present application does not limit the size of the output images of the medium target layer and the small target layer, and can be adjusted accordingly according to actual conditions.

[0157] For example, the output image size corresponding to the medium target layer is 80*80*255, and the output image size corresponding to the small target layer is 40*40*255.

[0158] Accordingly, the first image to be detected is input into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, including the following implementation:

[0159] Step 1: Input the first image to be detected into the Backbone module for feature extraction to obtain a multi-scale feature map.

[0160] Among them, the Backbone module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module, which are arranged in sequence.

[0161] In this implementation, the input module of the YOLOv5 model inputs the first monitoring image into the Backbone module for feature extraction processing. After a series of operations such as convolutional layers, activation functions, and pooling layers, low-level and high-level features of the image are extracted to obtain multiple feature maps of different scales so that objects of different sizes can be identified.

[0162] Among them, the multi-scale feature map contains detailed information at different levels in the image, such as edges, textures, colors, shapes, etc., which helps in target detection in subsequent steps.

[0163] For example, the Backbone module of the YOLOv5 model used in the embodiment of the present application is improved as follows:

[0164] Improvement 1: Replace the ordinary convolution in the YOLOv5 model with DCN. The deformable convolution network learns the feature map through an additional convolution layer, and adds a position offset {Δp n |n=1,2,...,n},n∈|R|, the traditional convolutional network formula is:

[0165]

[0166] Among them, p0 is each position on the output feature map, p n is the relative position between the convolution kernels, w is the convolution kernel, y(p0) represents the output image feature of the deformable convolution network (i.e., the output feature value at a specific position p0), x(p0+p n ) means that in the input feature map x, the relative position p is added to the position p0 n To obtain the input value, this is the process of sampling the input feature map, that is, with p0 as the center, through the relative position p n Sampling, w(p n ) represents the weight parameter, which is used to adjust the relative position p n The input values at are weighted.

[0167] The deformable convolutional network adds an offset Δp to the regular network R n , allowing the convolution kernel to dynamically adjust the spatial sampling position on the input image (i.e., the first image to be detected). The convolution kernel can adaptively select different sampling points to better capture the shape, boundary, and detail information in the image. Especially when processing deformed, distorted, rotated, etc. images, it is more flexible than ordinary convolution, improving the model's ability to capture the shape and boundary of the target object, thereby improving the model's performance. The deformable convolutional network formula is:

[0168]

[0169] Where Δp n is the offset of each position in the feature map of the first image to be detected on the regular grid.

[0170] like Figure 1As shown in the figure, the second layer of ordinary convolution in the Backbone module is replaced by the first deformable convolution network, the fourth layer of ordinary convolution is replaced by the second deformable convolution network, the sixth layer of ordinary convolution is replaced by the third deformable convolution network, and the eighth layer of ordinary convolution is replaced by the fourth deformable convolution network.

[0171] Improvement 2: The Transformer encoder module is used to replace some convolutional blocks and cross-stage partial bottleneck (CSP bottleneck) modules in the original YOLOv5 to effectively capture global information and rich contextual information. The Transformer encoder block consists of two sublayers:

[0172] The first sub-layer is a multi-head attention layer, which is used to enhance the model's ability to focus on important features in the first image to be detected;

[0173] The second sub-layer is a fully connected layer, which is used to improve the richness of the feature representation in the first image to be detected, where the sub-layers of each fully connected layer are interconnected through residual connections;

[0174] Add the first Transformer encoder module (i.e. Figure 1 The Trans module in YOLOv5 is used to fully utilize the characteristics of low-resolution feature maps, thereby reducing computational and memory overhead and improving the overall network performance of the YOLOv5 model.

[0175] Improvement 3: The attention mechanism can effectively improve the model's ability to focus on the area corresponding to the target object in the first image to be detected, thereby enhancing the accuracy of target positioning. By emphasizing important features and suppressing irrelevant information, the attention mechanism improves the model's feature representation ability, enabling it to more accurately identify and distinguish targets. In addition, the attention mechanism can reduce the YOLOv5 model's misjudgment of background or non-target areas, enhance the recognition ability of small and difficult-to-detect targets, and thus reduce missed detections. The attention mechanism can also make the YOLOv5 model more robust in the face of challenges such as occlusion, lighting changes, and background interference, because the attention mechanism can focus more on the characteristics of the target itself.

[0176] In the third-to-last layer of the Backbone module (such as Figure 1The first iRMB module is added to the YOLOv5 architecture (as shown in the figure). The iRMB module combines the CNN's efficiency in modeling short-range dependencies with the Transformer's ability to dynamically model long-range interactions. This approach provides a balanced solution, enabling the YOLOv5 model to capture local features while understanding global context. This helps the model better focus on key areas in the image, thereby improving object detection accuracy and enhancing the model's ability to recognize objects.

[0177] Step 2: Input the multi-scale feature map into the Neck module for feature fusion processing to obtain a dual-scale feature map.

[0178] Among them, the Neck module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module, which are arranged in sequence.

[0179] In this implementation, the multi-scale feature map obtained after processing by the Backbone module is input into the Neck module for feature fusion processing, which can combine feature information of different scales. In this way, in the subsequent prediction stage, the model can more accurately identify targets of different sizes.

[0180] Among them, the goal of the Neck module is to enable the YOLOv5 model network to perform effective target detection at multiple scales by combining low-level features with high-level features.

[0181] like Figure 1 As shown, the fifth deformable convolution is added to the second layer of the Neck module, the second iRMB module and the third iRMB module are added to the sixth layer and the penultimate layer of the Neck module respectively, and the second Transformer module and the third Transformer module are added to the penultimate layer of the Neck module respectively.

[0182] Step 3: Input the dual-scale feature map into the prediction layer for prediction processing to obtain the target labeling information corresponding to the first image to be detected.

[0183] In this implementation, the dual-scale feature map fused by the Neck module is input into the prediction layer for the final prediction of target detection, and the target annotation information corresponding to the image to be detected is obtained.

[0184] Among them, the prediction layer usually uses convolution operations to generate the target category, location information (i.e., the bounding box coordinates of the target object), and the target confidence (i.e., the probability of the target belonging to the category). Based on these outputs, the YOLOv5 model can locate the target in the first image to be detected and generate corresponding annotation information for each target, such as category label and bounding box coordinates.

[0185] The computer vision-based warning area intrusion alarm method provided in an embodiment of the present application first obtains a first data set and the annotation information corresponding to the first data set, then performs data enhancement processing on multiple first images in the first data set to obtain a second data set, and finally trains a YOLOv5 model based on the first data set, the second data set, and the annotation information corresponding to the first data set to obtain an intrusion detection model. This technical solution achieves the technical effect of improving the accuracy and robustness of the intrusion detection model by obtaining and enhancing the first data set to generate the second data set, and training the YOLOv5 model based on the corresponding annotation information.

[0186] Based on the above embodiments, Figure 4 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 3 ,like Figure 4 As shown, step 23 may include the following steps:

[0187] Step 41: Determine whether the target object is an intrusion object based on the intersection-and-union ratio data of the first position information and the warning area;

[0188] In this step, the monitoring system determines the current position information of the target object based on the first position information of the target object, and calculates the intersection-and-union ratio data of the target object's position information and the warning area, thereby determining whether the target object is an intrusion object.

[0189] The intersection-over-union ratio refers to the ratio of the overlapping area of the target area and the warning area to the total area of the two. It reflects the degree of overlap between the target object and the warning area, and can be used to determine whether the target object is an intrusion object.

[0190] In a possible implementation, the first image to be detected is acquired every 0.5 seconds, and the intersection-over-union data of the first position information corresponding to the first image to be detected and the warning area is calculated.

[0191] Optionally, step 41 may be implemented as follows:

[0192] Step 1: If the intersection-over-union ratio of the first location information and the warning area is greater than or equal to a preset threshold, the target object is determined to be an intrusion object;

[0193] In this implementation, the relationship between the IoU data and a preset threshold is determined. If the calculated IoU data is greater than or equal to the preset threshold, the target object is considered to have entered the warning area and is an intruder. The monitoring system will trigger an intrusion alarm and perform subsequent processing, such as alarming, recording surveillance video, and real-time notification.

[0194] In a possible implementation, the preset threshold may be 0.7.

[0195] Step 2: If the intersection-over-union ratio of the first position information and the warning area is less than a preset threshold, it is determined that the target object is not an intrusion object.

[0196] In this implementation, if the intersection-union ratio data is less than the preset threshold, it means that the overlapping area between the target object and the warning area is small, indicating that the target object has not entered the warning area. Therefore, it is judged that the target object is not an intrusion object, and the monitoring system will not take any intrusion processing measures and continue to monitor other possible intrusion events.

[0197] In addition, step 41 can also be implemented as follows:

[0198] The first step is to obtain a second image to be detected at the next moment of the first image to be detected, and determine whether the target object is an intrusion object based on the second image to be detected, the first position information, and the warning area.

[0199] In this implementation, the monitoring system acquires a first image to be detected and calculates the target object's location information based on the target's location in that image. The monitoring system then acquires a second image to be detected at the next moment to capture the target object's possible motion trajectory or changes at the next moment. The time series of the first and second images to be detected provides dynamic information about the target object, making intrusion judgments more timely and accurate. By comparing the target position in the second image to the first location information, as well as the location of the warning zone, the monitoring system determines whether the target object has crossed the boundary of the warning zone. If the target object remains within the warning zone or continues to move toward it in the second image, the target object can be determined to be an intruder.

[0200] In a possible implementation, the next moment is a moment 0.5 seconds after the current moment.

[0201] Exemplarily, the center point of the prediction box of the first target object in the first image to be detected is stored in the first position in the preset queue, the center point of the prediction box of the first target object in the second image to be detected is stored in the second position in the preset queue, and the center point of the prediction box of the first target object in the third image to be detected is stored in the third position in the preset queue. The center points of the first target object at the three moments are connected into a line (that is, the center line of the person's movement), and it is determined whether the center line of the person's movement intersects with the warning area. If there is an intersection and the intersection is located within the person's movement circle with the midpoint of the person's movement center line as the center and a preset value as the radius, the target object is determined to be an intrusion object.

[0202] The preset queue length can be set according to the specific application scenario.

[0203] In a possible implementation, the preset queue length is 6, and the preset value is 8 pixels.

[0204] The embodiment of the present application provides a warning area intrusion alarm method based on computer vision. The method determines whether the target object is an intrusion object based on the intersection-and-union data of the first position information and the warning area, or obtains a second image to be detected at the next moment of the first image to be detected, and determines whether the target object is an intrusion object based on the second image to be detected, the first position information, and the warning area. This technical solution can accurately track the dynamic changes of the target position by combining the intersection-and-union data of the target object and the warning area to judge the intrusion analysis of static target objects, or by using dynamic analysis of continuous images, to adapt to real-time monitoring needs, effectively improve the accuracy and real-time response speed of intrusion detection, and flexibly adapt to different monitoring environments.

[0205] Based on the above embodiments, Figure 5 Schematic diagram of the process of the warning area intrusion alarm method based on computer vision provided in the embodiment of the present application Figure 4 ,like Figure 5 As shown, the first step in one implementation of step 41 may include the following steps:

[0206] S1, determining a line to be inspected based on a center point of second position information corresponding to a second image to be inspected and a center point of the first position information;

[0207] In this implementation, the monitoring system calculates the center point of the same target in the first image to be detected and the second image to be detected based on the target position information corresponding to the first image to be detected and the second image to be detected, and connects the center points of the same target in the first image to be detected and the second image to be detected into a line as the line to be detected.

[0208] Among them, the line to be detected represents the motion trajectory of the same target object at two consecutive moments, thereby providing a key spatial position reference for subsequent steps, ensuring that the motion direction and position of the same target object can be accurately described during the detection process.

[0209] S2, determining the target circle to be detected with the center of the line to be detected as the center and the preset threshold as the radius;

[0210] In this implementation, the target circle to be detected is determined by taking the midpoint of the line to be detected determined by the center point of the same target in the first image to be detected and the second image to be detected as the center of the circle and taking the preset threshold as the radius.

[0211] Among them, the target circle to be detected represents the possible impact range or the range of the target object approaching the warning area during the movement. By setting the radius, the sensitivity to the target object approaching the warning area can be flexibly adjusted according to actual needs to ensure the accuracy of detection.

[0212] In a possible implementation, the preset threshold is 6 pixels.

[0213] S3, if there is an intersection between the target circle to be detected and the warning area, the target object is determined to be an intruder;

[0214] In this implementation, if there is an intersection between the target circle to be detected and the warning area, it means that the target object has approached or entered the warning area and meets the intrusion conditions. At this time, the monitoring system will identify the target object as an intrusion object, trigger relevant alarms or protective measures, and respond to potential security threats in a timely manner.

[0215] S4: If there is no intersection between the target circle to be detected and the warning area, it is determined that the target object is not an intrusion object.

[0216] In this implementation, if the target circle to be detected does not intersect the warning area, it means that the target object has not entered the warning area and is still within the safe range. Therefore, the monitoring system will determine that the target object is not an intruder and continue to monitor the target object's behavior.

[0217] Among them, by determining whether there is an intersection between the target circle to be detected and the warning area, false alarms of intrusion into the warning area and unnecessary alarm triggering can be effectively avoided, thereby improving the intelligence level of the monitoring system.

[0218] Optionally, the first step can also be implemented as follows:

[0219] Step 1: When the target object is not an intrusion object, the second image to be detected is used as a new first image to be detected, the image to be detected at the next moment of the second image to be detected is used as a new second image to be detected, and steps S1 to S4 are repeated.

[0220] In this implementation, when the monitoring system determines that the target object is not an intruder, the monitoring system updates the current second image to be detected as the new first image to be detected, ensuring the latest tracking of the target's position. The system then obtains the next image of the second image to be detected and uses it as the new second image to be detected, repeating steps S1 to S4.

[0221] This approach allows the monitoring system to continuously update the target's location and, based on this, dynamically monitor its movement trajectory and its relationship to the warning area. This cyclical detection mechanism not only continuously tracks changes in the target's behavior but also ensures accurate judgment of intrusion risks at every point in time, enabling timely responses and maintaining system continuity and efficiency.

[0222] The embodiment of the present application provides a warning area intrusion alarm method based on computer vision. The method first determines the line to be detected based on the center point of the second position information corresponding to the second image to be detected and the center point of the first position information, and then determines the target circle to be detected with the center of the line to be detected as the center of the circle and the preset threshold as the radius. If there is an intersection between the target circle to be detected and the warning area, the target object is determined to be an intruder. Finally, if there is no intersection between the target circle to be detected and the warning area, the target object is determined to be an intruder. This technical solution determines the line to be detected by calculating the center point of the target position in the second image to be detected and the first image to be detected, thereby determining the target circle to be detected, and then judging whether the target circle has an intersection with the warning area. If there is an intersection, the target is determined to be an intruder. If there is no intersection, the target is considered not to be an intruder. The relationship between the target object and the warning area can be monitored in real time, and whether there is an intrusion risk in the warning area can be accurately judged, thereby effectively improving the real-time response capability of the monitoring system and the accuracy of intrusion detection in the warning area, and avoiding false alarms.

[0223] In one possible implementation, Figure 6 Schematic diagram of the process of the warning area intrusion method based on computer vision provided in the embodiment of the present application Figure 5 , specifically including the following steps:

[0224] Step 1. Get started.

[0225] Step 2: Set the warning area.

[0226] In this step, the warning area is manually set in the monitoring visualization interface.

[0227] Step 3: Target object detection.

[0228] In this step, the intrusion detection model detects all target objects that exist in the monitoring perspective.

[0229] Step 4: Detect the target object.

[0230] In this step, if the intrusion detection model detects the target object, then step 5 is executed; if the intrusion detection model does not detect the target object, then step 9 is executed.

[0231] Step 5: The target object is in the warning area.

[0232] In this step, the monitoring system determines whether the target object is in the warning area. If it is in the warning area, step 6 is executed; if it is not in the warning area, step 9 is executed.

[0233] Step 6: The warning area is invaded and an alarm is issued.

[0234] In this step, if the warning area receives an intrusion, the monitoring system will issue an alarm message.

[0235] Step 7: The alarm device sounds an alarm.

[0236] In this step, the alarm device receives the alarm information sent by the monitoring system and issues an alarm.

[0237] Step 8: Notify the operation staff on duty.

[0238] In this step, the alarm will notify the operation duty personnel through sound, and the duty personnel will check the intrusion situation in the warning area.

[0239] Step 9: No danger, silence.

[0240] Step 10. End.

[0241] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

[0242] Figure 7 This is a schematic diagram of the structure of the warning area intrusion alarm device based on computer vision provided in the embodiment of the present application. Figure 7 As shown, the device includes:

[0243] An acquisition module 71 is configured to acquire a first image to be detected of the area to be detected;

[0244] A processing module 72 is configured to input the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, where the target annotation information is used to indicate whether a target object exists in the area to be detected and first location information of the target object;

[0245] a determination module 73 for determining whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected if the target annotation information indicates that the target object exists in the area to be detected;

[0246] The trigger module 74 is configured to trigger an alarm if the target object is an intrusion object.

[0247] In a possible implementation, before inputting the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, the processing module 72 is specifically configured to:

[0248] Obtaining a first data set and annotation information corresponding to the first data set, where the first data set includes: a plurality of first images containing objects, and annotation information corresponding to each first image, where the annotation information is used to indicate the object and location information of the object in the first image;

[0249] Performing data enhancement processing on the plurality of first images in the first data set to obtain a second data set, wherein the second data set includes: a plurality of second images, each of the second images being an image that occludes any portion of an object in any of the first images;

[0250] The YOLOv5 model is trained according to the first data set, the second data set, and the annotation information corresponding to the first data set to obtain an intrusion detection model.

[0251] In one possible implementation, the YOLOv5 model includes: a Backbone module, a Neck module, and a prediction layer, where the prediction layer includes: a medium target layer and a small target layer;

[0252] Accordingly, the processing module 72 is specifically configured to:

[0253] Input the first image to be detected into the Backbone module for feature extraction processing to obtain a multi-scale feature map. The Backbone module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence;

[0254] The multi-scale feature map is input into the Neck module for feature fusion processing to obtain a dual-scale feature map. The Neck module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module.

[0255] The dual-scale feature map is input into the prediction layer for prediction processing to obtain the target labeling information corresponding to the first image to be detected.

[0256] In a possible implementation, the determination module 73 is specifically configured to:

[0257] determining whether the target object is an intrusion object based on the intersection-and-union ratio data of the first position information and the warning area;

[0258] Alternatively, a second image to be detected is acquired at a next moment after the first image to be detected, and whether the target object is an intrusion object is determined based on the second image to be detected, the first position information, and the warning area.

[0259] In a possible implementation, the determination module 73 determines whether the target object is an intrusion object based on the intersection-over-union ratio data of the first position information and the warning area, specifically for:

[0260] If the intersection-over-union ratio data of the first position information and the warning area is greater than or equal to a preset threshold, the target object is determined to be an intrusion object;

[0261] If the intersection-over-union ratio data of the first position information and the warning area is smaller than a preset threshold, it is determined that the target object is not an intrusion object.

[0262] In a possible implementation, the determination module 73 determines whether the target object is an intrusion object based on the second image to be detected, the first location information, and the warning area, specifically for:

[0263] S1, determining a line to be inspected based on a center point of second position information corresponding to a second image to be inspected and a center point of the first position information;

[0264] S2, determining the target circle to be detected with the center of the line to be detected as the center and the preset threshold as the radius;

[0265] S3, if there is an intersection between the target circle to be detected and the warning area, the target object is determined to be an intruder;

[0266] S4: If there is no intersection between the target circle to be detected and the warning area, it is determined that the target object is not an intrusion object.

[0267] In a possible implementation, the determination module 73 is further configured to:

[0268] When the target object is not an intrusion object, the second image to be detected is used as a new first image to be detected, the image to be detected at the next moment after the second image to be detected is used as a new second image to be detected, and steps S1 to S4 are repeated.

[0269] The device provided in the embodiments of the present application can be used to execute the determination method in any of the above embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0270] It should be noted that it should be understood that the division of the various modules of the above device is merely a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. Moreover, these modules can all be implemented in the form of software called by processing elements; they can also all be implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. In addition, these modules can be fully or partially integrated together or implemented independently. The processing element here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or by instructions in the form of software.

[0271] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 8 As shown, the electronic device may include: a processor 81, a memory 82, and computer program instructions stored in the memory 82 and executable on the processor 81. When the processor 81 executes the computer program instructions, the method provided in any of the aforementioned embodiments is implemented.

[0272] Optionally, the above-mentioned components of the electronic device may be connected via a system bus.

[0273] The memory 82 may be a separate storage unit or a storage unit integrated in the processor 81. The number of the processor 81 may be one or more.

[0274] It should be understood that the processor 81 can be a central processing unit (CPU), or other general-purpose processors 81, digital signal processors 81 (DSP), application-specific integrated circuits (ASIC), etc. The general-purpose processor 81 can be a microprocessor 81 or any conventional processor 81. The steps of the method disclosed in this application can be performed by the hardware processor 81 or by a combination of hardware and software modules in the processor 81.

[0275] The system bus can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, among others. A system bus can be divided into an address bus, a data bus, a control bus, and so on. For ease of illustration, the figure uses only one thick line, but this does not imply that there is only one bus or only one type of bus. Memory 82 may include random access memory 82 (RAM) and may also include non-volatile memory 82 (NVM), such as at least one disk storage device 82.

[0276] All or part of the steps of the above-mentioned method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a readable memory 82. When the program is executed, it performs the steps of the above-mentioned method embodiments; and the aforementioned memory 82 (storage medium) includes: read-only memory 82 (ROM), RAM, flash memory 82, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0277] The electronic device provided in the embodiments of the present application can be used to execute the method provided in any of the above method embodiments. Its implementation principles and technical effects are similar and will not be repeated here.

[0278] An embodiment of the present application provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on a computer, the computer executes the above method.

[0279] The computer-readable storage medium mentioned above may be implemented by any type of volatile or non-volatile memory device, or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The computer-readable storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0280] Optionally, a readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0281] An embodiment of the present application also provides a computer program product, which includes a computer program. The computer program is stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium. When the at least one processor executes the computer program, the above method can be implemented.

[0282] It should be understood that the present application is not limited to the exact structure described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A warning area intrusion alarm method based on computer vision, characterized in that: include: Acquire a first image to be detected of the area to be detected; Inputting the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, wherein the target annotation information is used to indicate whether a target object exists in the area to be detected and first position information of the target object; If the target annotation information indicates that the target object exists in the area to be detected, determining whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected; If the target object is the intrusion object, an alarm is triggered.

2. The method according to claim 1, characterized in that Before inputting the first image to be detected into a pre-trained intrusion detection model to obtain target labeling information corresponding to the first image to be detected, the method includes: Obtaining a first data set and annotation information corresponding to the first data set, where the first data set includes: a plurality of first images containing objects, and annotation information corresponding to each first image, the annotation information being used to indicate the object in the first image and location information of the object; performing data augmentation processing on a plurality of first images in the first data set to obtain a second data set, wherein the second data set includes: a plurality of second images, wherein for each second image, the second image is an image that occludes any portion of an object in any of the first images; The YOLOv5 model is trained according to the first data set, the second data set, and the annotation information corresponding to the first data set to obtain the intrusion detection model.

3. The method according to claim 2, characterized in that The YOLOv5 model includes: a Backbone module, a Neck module, and a prediction layer, and the prediction layer includes: a medium target layer and a small target layer; Accordingly, the step of inputting the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected includes: Inputting the first image to be detected into the Backbone module for feature extraction processing to obtain a multi-scale feature map, wherein the Backbone module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence; Inputting the multi-scale feature map into the Neck module for feature fusion processing to obtain a dual-scale feature map, wherein the Neck module includes a deformable convolution module, an iterative residual multi-head attention module, and a Transformer module arranged in sequence; The dual-scale feature map is input into the prediction layer for prediction processing to obtain target labeling information corresponding to the first image to be detected.

4. The method according to any one of claims 1 to 3, characterized in that The determining whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected includes: determining whether the target object is an intrusion object based on intersection-over-union data of the first position information and the warning area; Alternatively, a second image to be detected at a next moment after the first image to be detected is obtained, and whether the target object is the intrusion object is determined based on the second image to be detected, the first position information, and the warning area.

5. The method according to claim 4, characterized in that The determining whether the target object is the intrusion object according to the intersection-over-union data of the first position information and the warning area includes: If the intersection-over-union ratio data of the first position information and the warning area is greater than or equal to a preset threshold, determining that the target object is the intrusion object; If the intersection-over-union ratio data of the first position information and the warning area is smaller than a preset threshold, it is determined that the target object is not the intrusion object.

6. The method according to claim 4, characterized in that The determining whether the target object is an intrusion object according to the second image to be detected, the first position information, and the warning area includes: S1, determining a line to be detected based on a center point of second position information corresponding to the second image to be detected and a center point of the first position information; S2, determining a target circle to be detected with the center of the line to be detected as the center and a preset threshold as the radius; S3, if the target circle to be detected intersects the warning area, determining that the target object is the intruder; S4: If there is no intersection between the target circle to be detected and the warning area, it is determined that the target object is not the intrusion object.

7. The method according to claim 6, characterized in that The method further comprises: When the target object is not the intrusion object, the second image to be detected is used as a new first image to be detected, the image to be detected at the next moment of the second image to be detected is used as a new second image to be detected, and steps S1 to S4 are repeated.

8. A warning area intrusion alarm device based on computer vision, characterized in that: include: An acquisition module, configured to acquire a first image to be detected of the area to be detected; a processing module, configured to input the first image to be detected into a pre-trained intrusion detection model to obtain target annotation information corresponding to the first image to be detected, wherein the target annotation information is used to indicate whether a target object exists in the area to be detected and first position information of the target object; a determination module, configured to determine whether the target object is an intrusion object based on the first position information of the target object and the warning area in the area to be detected if the target annotation information indicates that the target object exists in the area to be detected; The trigger module is used to trigger an alarm if the target object is the intrusion object.

9. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images