Perimeter intrusion identification method, system and electronic equipment

Abstract

The present invention discloses a perimeter intrusion recognition method, system and electronic device, wherein the method comprises: acquiring an image according to a real-time video stream, setting a perimeter area on the image; automatically segmenting the image into a first image cluster based on the perimeter area; scaling the image in the first image cluster to obtain a second image cluster; inputting the second image cluster into a neural network, detecting whether a continuous multi-frame image of the second image cluster contains pedestrians invading the perimeter area, and outputting a detection and recognition result. The present invention improves the accuracy of detecting perimeter intrusions at a distance of sight, reduces false alarms caused by perimeter environmental interference, achieves the technical effect of reducing the false positive rate of perimeter alarms, and solves the problem that small targets cannot be recognized and the false positive rate of perimeter alarms is high in existing perimeter intrusion solutions.

Description

Technical Field

[0001] The present invention relates to the field of video analysis and recognition, and in particular to a perimeter intrusion recognition method, system and electronic equipment. Background Art

[0002] Traditional perimeter intrusion solutions often use leaky cables or infrared lasers to detect and identify the human body. These two methods have the same problem and are easily interfered by the environment. For example, the leaky cable method is easily interfered by metal objects or surrounding electromagnetic fields, and the infrared laser method is easily interfered by tree branches, wind and snow, etc.

[0003] When using a video-based perimeter intrusion solution to detect and identify the human body, an early warning reminder is issued by judging whether the human body has entered the virtual perimeter area. Common video-based perimeter intrusion solutions mainly identify the human body in a single picture, which cannot identify small targets and easily cause human body recognition errors. The alarm misjudgment rate is high, causing trouble to users.

[0004] There is currently no effective solution to the problems that existing perimeter intrusion solutions cannot identify small targets and have a high false positive rate for perimeter alarms. Summary of the invention

[0005] The main purpose of the present invention is to provide a perimeter intrusion identification method and system to solve the problems that small targets cannot be identified and the perimeter alarm misjudgment rate is high in existing perimeter intrusion solutions.

[0006] In order to achieve the above object, a first aspect of the present invention provides a perimeter intrusion identification method, comprising:

[0007] Acquire an image based on a real-time video stream, and set a perimeter area on the image;

[0008] Based on the perimeter area, automatically segmenting the image into a first image cluster;

[0009] Scaling the images in the first image cluster to obtain a second image cluster;

[0010] The second image cluster is input into a neural network, and whether a plurality of consecutive frames of images of the second image cluster contain pedestrians invading a peripheral area is detected, and a detection and recognition result is output.

[0011] Optionally, the automatically segmenting the image into a first image cluster based on the perimeter area includes:

[0012] Configuring a direction in which sight lines disappear within the perimeter area;

[0013] The image is automatically segmented into a plurality of pixel clusters according to the direction in which the line of sight disappears, to obtain a first image cluster.

[0014] Optionally, scaling the images in the first image cluster to obtain a second image cluster includes:

[0015] Determine the scaling ratio based on the detection scenario and requirements;

[0016] The image at the near end of the sight line in the first image cluster is reduced according to the scaling ratio, and the image at the far end is enlarged to obtain a second image cluster.

[0017] Optionally, inputting the second image cluster into a neural network, detecting whether a plurality of consecutive frames of images of the second image cluster contain pedestrians invading a peripheral area, and outputting a detection and recognition result includes:

[0018] Using a deep learning algorithm, detecting whether the image of the second image cluster contains a pedestrian;

[0019] If pedestrians are detected in the image, the vibe algorithm is used to perform background difference on the area where the pedestrians are detected, and the mobility in the area is determined by judging the difference image.

[0020] Furthermore, the method further comprises:

[0021] If it is detected that no pedestrian is contained in the image, the image is automatically segmented again based on the perimeter area.

[0022] Furthermore, the vibe algorithm is used to perform background difference on the area where the pedestrian is detected, and the mobility judgment in the area is performed by judging the difference image, including:

[0023] Determine whether a pedestrian is within the perimeter area;

[0024] If the pedestrian is within the perimeter area, determining whether the pedestrian is within the perimeter area of ​​the continuous multiple frames of images and whether there is a differential change in the images within the perimeter area;

[0025] If the pedestrian is within the perimeter area of ​​the continuous multiple frames of images and there is a differential change in the images within the perimeter area, an alarm is triggered and a snapshot is taken; otherwise, it is re-determined whether the pedestrian is within the perimeter area.

[0026] Furthermore, the method further comprises:

[0027] If the pedestrian is not within the perimeter area, the image is automatically segmented again based on the perimeter area.

[0028] A second aspect of the present invention provides a perimeter intrusion identification system, comprising:

[0029] A setting unit, configured to acquire an image according to a real-time video stream and set a perimeter area on the image;

[0030] a segmentation unit, configured to automatically segment the image into a first image cluster based on the perimeter area;

[0031] a scaling unit, configured to scale the images in the first image cluster to obtain a second image cluster;

[0032] The detection unit is used to input the second image cluster into the neural network, detect whether the continuous multiple frames of the second image cluster contain pedestrians invading the perimeter area, and output the detection and recognition results.

[0033] A third aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the perimeter intrusion identification method provided by any one of the first aspects.

[0034] The fourth aspect of the present invention provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the perimeter intrusion identification method provided in any one of the first aspects.

[0035] In the perimeter intrusion recognition method provided by the embodiment of the present invention, by scaling the images in the first image cluster, the difference in the proportion of pedestrians in different areas is reduced, small targets are accurately recognized, and the accuracy of detecting perimeter intrusions at a distance of the line of sight is improved;

[0036] By inputting the second image cluster into the neural network, it is detected whether the continuous multiple frames of the second image cluster contain pedestrians who have invaded the perimeter area, thereby reducing false alarms caused by peripheral environmental interference, achieving the technical effect of reducing the false positive rate of perimeter alarms, and solving the problem that small targets cannot be identified and the perimeter alarm false positive rate is high in existing perimeter intrusion solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1 A schematic diagram of a process flow of a perimeter intrusion identification method provided by an embodiment of the present invention;

[0039] Figure 2A schematic diagram of a peripheral area and a direction in which sight lines disappear provided by an embodiment of the present invention;

[0040] Figure 3 A flow chart of a pedestrian detection method based on Faster R-CNN provided in an embodiment of the present invention;

[0041] Figure 4 A flow chart of a perimeter intrusion identification method provided by an embodiment of the present invention;

[0042] Figure 5 A block diagram of a perimeter intrusion identification system provided by an embodiment of the present invention;

[0043] Figure 6 A block diagram of an electronic device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0044] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0045] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so as to describe the embodiments of the present invention described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0046] When a video-based perimeter intrusion solution is used to detect and identify the human body, an early warning reminder is issued by judging whether the human body has entered the virtual perimeter area. Common video-based perimeter intrusion solutions mainly identify the human body in a single picture, which cannot identify small targets and can easily cause human body recognition errors. The alarm misjudgment rate is high, causing trouble to users. The perimeter intrusion identification method provided by the present invention is applicable to scenarios including farmland security management scenarios, such as ranch biosafety management scenarios, to solve the problem of high ranch perimeter alarm misjudgment rate and inability to identify small targets.

[0047] An embodiment of the present invention provides a perimeter intrusion identification method, which is applied to edge computing devices, such as Figure 1 As shown, the method includes the following steps S101 to S104:

[0048] Step S101: acquiring an image according to a real-time video stream, and setting a perimeter area on the image;

[0049] Pre-set the camera, obtain the real-time video stream collected by the camera through the web page, and then obtain the image based on the real-time video stream, and divide the perimeter area on the image according to the needs.

[0050] Step S102: Based on the perimeter area, the image is automatically segmented into a first image cluster; in order to accurately detect pedestrian intrusion in the perimeter area, the image is automatically segmented to generate multiple sub-images, and the detection rate is improved by detecting each sub-image one by one.

[0051] Specifically, step S102 includes:

[0052] The direction in which the line of sight disappears is configured in the perimeter area; a schematic diagram of the perimeter area and the direction in which the line of sight disappears provided by the embodiment of the present invention is as follows Figure 2 shown.

[0053] The image is automatically segmented into a plurality of pixel clusters according to the direction in which the line of sight disappears, to obtain a first image cluster.

[0054] Step S103: scaling the images in the first image cluster to obtain a second image cluster;

[0055] Specifically, step S103 includes:

[0056] Determine the scaling ratio based on the detection scenario and requirements;

[0057] The reduction / enlargement ratio is determined according to different detection scenarios and requirements: when the camera lens and CCD (Charge Coupled Device) are the same, if the distance to be detected is farther, the more images need to be segmented and the larger the zoom ratio is; when the distance and CCD are the same, if the lens focal length is shorter, the more images need to be segmented and the larger the zoom ratio is.

[0058] According to the principle of near is big and far is small, the image at the near end of the sight line in the first image cluster is reduced according to the scaling ratio, and the image at the far end is enlarged to obtain the second image cluster.

[0059] The present invention scales the images in the first image cluster, reduces the near-end images, and enlarges the far-end images, thereby reducing the differences in the proportions of pedestrians in different areas, accurately identifying small targets, and improving the accuracy of detecting perimeter intrusions at a distance from the line of sight.

[0060] Step S104: input the second image cluster into a neural network, detect whether a plurality of consecutive frames of images of the second image cluster contain pedestrians invading the perimeter area, and output a detection and recognition result.

[0061] By inputting the second image cluster into the neural network, it is detected whether the continuous multiple frames of the second image cluster contain pedestrians who have invaded the perimeter area, thereby reducing false alarms caused by peripheral environmental interference, achieving the technical effect of reducing the false positive rate of perimeter alarms, and solving the problem that small targets cannot be identified and the perimeter alarm false positive rate is high in existing perimeter intrusion solutions.

[0062] Specifically, step S104 includes:

[0063] A deep learning algorithm is used to detect whether the images in the second image cluster contain pedestrians. The deep learning algorithm may be a target detection algorithm, including any one of R-CNN, Fast R-CNN, Faster R-CNN, SSD, YOLO v1, YOLO v2, YOLO v4, and YOLO v5. The present invention takes Faster R-CNN as an example to explain a specific scheme for pedestrian detection.

[0064] The flowchart of the pedestrian detection method based on Faster R-CNN used in the embodiment of the present invention is as follows: Figure 3 As shown in the figure, image is an image, conv layers are convolutional layers, feature maps are feature maps, Region Proposal Network (RPN) is a region generation network or a region candidate network, which is used to extract candidate boxes; ROI pooling is a neural network layer for target detection tasks, which performs a pooling operation on ROI (Region of Interest). The purpose of this operation is to use the pooling method for ROIs of different sizes in the input feature map to obtain an output feature map of a fixed size; classifier is a classifier;

[0065] Structurally, Faster R-CNN has integrated feature extraction, proposal extraction, bounding box regression (rect refine), and classification into one network, which greatly improves the overall performance, especially in detection speed.

[0066] If it is detected that there is no pedestrian in the image, the image is automatically segmented based on the peripheral area again. That is, if there is no pedestrian in the image, the process returns to the step of automatically segmenting the image based on the peripheral area.

[0067] If pedestrians are detected in the image, the vibe algorithm is used to perform background difference on the area where the pedestrians are detected, and the mobility in the area is judged by judging the difference image. The vibe algorithm is used to perform background difference operation, randomly select samples of pixels to be replaced, and randomly select neighboring pixels for update; when the model of pixel changes cannot be determined, the random update strategy is used to simulate the uncertainty of pixel changes.

[0068] Among them, the vibe algorithm is a foreground detection algorithm based on background update. Its principle is to establish a sample set of pixel points by extracting the pixel values ​​around the pixel point (x, y) and the previous pixel values, and then compare the pixel value at another frame (x, y) with the pixel value in the sample set. If the distance between it and the pixel value in the sample set is greater than a preset threshold, the pixel point is considered to be a foreground pixel point, otherwise it is a background pixel point.

[0069] The method of performing background difference on the area where pedestrians are detected by using the vibe algorithm and judging the mobility in the area by judging the difference image includes:

[0070] Determine whether a pedestrian is within the perimeter area;

[0071] If the pedestrian is not in the perimeter area, the image is automatically segmented based on the perimeter area again. That is, if the pedestrian is not in the perimeter area, the process returns to the step of automatically segmenting the image based on the perimeter area.

[0072] If the pedestrian is within the perimeter area, determining whether the pedestrian is within the perimeter area of ​​the continuous multiple frames of images and whether there is a differential change in the images within the perimeter area;

[0073] If a pedestrian is within the perimeter area of ​​the continuous multiple-frame images and there is a differential change in the images within the perimeter area, an alarm is triggered and a snapshot is taken;

[0074] Otherwise, re-determine whether the pedestrian is in the perimeter area. That is, if the pedestrian does not appear in the perimeter area of ​​the continuous multiple frames of images, or there is no differential change in the images in the perimeter area, return to the step of determining whether the pedestrian is in the perimeter area.

[0075] The vibe algorithm is used to perform background differentiation on the area where pedestrians are detected. If the pedestrian is within the perimeter area of ​​multiple consecutive frames of images and there is a differential change in the images within the perimeter area, an alarm is triggered to reduce false alarms caused by peripheral environmental interference, thereby achieving the technical effect of reducing the false positive rate of perimeter alarms and solving the problem that small targets cannot be identified and the false positive rate of perimeter alarms is high in existing perimeter intrusion solutions.

[0076] The flow chart of the perimeter intrusion identification method provided by the embodiment of the present invention is as follows: Figure 4 As shown, the following steps are included in sequence:

[0077] Set up perimeter areas;

[0078] According to the set perimeter area and the principle of near big and far small, the image is automatically segmented into regions and cut into image clusters;

[0079] The near-end image is reduced and the far-end image is enlarged to reduce the difference in the proportion of pedestrians in different areas;

[0080] Input the processed image clusters into the neural network for calculation;

[0081] Determine whether a pedestrian is detected and recognized; if not, re-divide the automatically segmented area into image clusters;

[0082] If yes, determine whether the pedestrian is in the perimeter area; if not, re-divide the automatically segmented area into image clusters;

[0083] If so, it is further determined whether multiple frames are continuously detected to be within the region and whether the images in the target region have differential changes;

[0084] If yes, an alarm is triggered and a photo is taken; if no, a new judgment is made as to whether the pedestrian is within the perimeter area.

[0085] From the above description, it can be seen that the present invention achieves the following technical effects:

[0086] The present invention scales the images in the first image cluster, shrinks the near-end images, and enlarges the far-end images, thereby reducing the differences in the proportions of pedestrians in different areas, accurately identifying small targets, and improving the accuracy of detecting intrusions at the perimeter of the distant sight.

[0087] By inputting the second image cluster into the neural network, it is detected whether the continuous multiple frames of the second image cluster contain pedestrians who have invaded the perimeter area, thereby reducing the false alarms caused by the interference of the perimeter environment, achieving the technical effect of reducing the false positive rate of perimeter alarms, and solving the problems of the inability to identify small targets and the high false positive rate of perimeter alarms in existing perimeter intrusion solutions.

[0088] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0089] The embodiment of the present invention also provides a perimeter intrusion identification system for implementing the above perimeter intrusion identification method, such as Figure 5 As shown, the system includes:

[0090] A setting unit 51, configured to acquire an image according to a real-time video stream and set a perimeter area on the image;

[0091] a segmentation unit 52, configured to automatically segment the image into a first image cluster based on the perimeter area;

[0092] A scaling unit 53, configured to scale the images in the first image cluster to obtain a second image cluster;

[0093] The detection unit 54 is used to input the second image cluster into a neural network, detect whether the continuous multiple frames of the second image cluster contain pedestrians invading the perimeter area, and output a detection and recognition result.

[0094] The embodiment of the present invention further provides an electronic device, such as Figure 6 As shown, the electronic device includes one or more processors 61 and a memory 62. Figure 6 A processor 61 is taken as an example.

[0095] The controller may further include: an input device 63 and an output device 64 .

[0096] The processor 61, the memory 62, the input device 63 and the output device 64 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0097] The processor 61 may be a central processing unit (CPU), or other general-purpose processors, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips. The general-purpose processor may be a microprocessor or any conventional processor.

[0098] The memory 62 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as program instructions / modules corresponding to the control method in the embodiment of the present invention. The processor 61 executes various functional applications and data processing of the server by running the non-transitory software programs, instructions and modules stored in the memory 62, that is, the perimeter intrusion identification method of the above method embodiment is implemented.

[0099] The memory 62 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of a processing device operated by the server, etc. In addition, the memory 62 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 62 may optionally include a memory remotely arranged relative to the processor 61, and these remote memories may be connected to a network connection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0100] The input device 63 can receive input digital or character information and generate key signal input related to user settings and function control of the processing device of the server. The output device 64 can include display devices such as a display screen.

[0101] One or more modules are stored in the memory 62, and when executed by one or more processors 61, the execution is as follows: Figure 1 The method shown.

[0102] Those skilled in the art can understand that the implementation of all or part of the processes in the above-mentioned embodiment method can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned motor control methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (FM), a hard disk drive (HDD), or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0103] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A perimeter intrusion identification method, characterized in that: include: Acquire an image based on a real-time video stream, and set a perimeter area on the image; Based on the perimeter area, the image is automatically segmented into a first image cluster, specifically: Configuring a direction in which sight lines disappear within the perimeter area; Automatically segmenting the image into multiple pixel clusters according to the direction in which the line of sight disappears, to obtain a first image cluster; Scaling the images in the first image cluster to obtain a second image cluster; including: determining a scaling ratio according to a detection scene and a requirement; reducing the images at the near end of the line of sight in the first image cluster according to the scaling ratio, and enlarging the images at the far end to obtain a second image cluster; The second image cluster is input into a neural network to detect whether a plurality of consecutive frames of images of the second image cluster contain pedestrians invading the perimeter area, and output a detection and recognition result, specifically: Using a deep learning algorithm, detecting whether the image of the second image cluster contains pedestrians; If a pedestrian is detected in the image, the vibe algorithm is used to perform background difference on the area where the pedestrian is detected, and the mobility in the area is judged by judging the difference image, specifically: judging whether the pedestrian is in the perimeter area; If the pedestrian is within the perimeter area, determining whether the pedestrian is within the perimeter area of ​​the continuous multiple frames of images and whether there is a differential change in the images within the perimeter area; If the pedestrian is within the perimeter area of ​​the continuous multiple-frame images and there is a differential change in the images within the perimeter area, an alarm is triggered and a snapshot is taken; otherwise, it is determined again whether the pedestrian is within the perimeter area; If the pedestrian is not within the perimeter area, automatically segmenting the image based on the perimeter area again; If it is detected that no pedestrian is contained in the image, the image is automatically segmented again based on the perimeter area.

2. A perimeter intrusion identification system, characterized in that: include: A setting unit, configured to acquire an image according to a real-time video stream and set a perimeter area on the image; The segmentation unit is used to automatically segment the image into a first image cluster based on the perimeter area, specifically: Configuring a direction in which sight lines disappear within the perimeter area; Automatically segmenting the image into multiple pixel clusters according to the direction in which the line of sight disappears, to obtain a first image cluster; A scaling unit is used to scale the images in the first image cluster to obtain a second image cluster; comprising: determining a scaling ratio according to the detection scene and requirements; reducing the image at the near end of the line of sight in the first image cluster according to the scaling ratio, and enlarging the image at the far end to obtain a second image cluster. A detection unit is used to input the second image cluster into a neural network, detect whether the continuous multiple-frame images of the second image cluster contain pedestrians who have invaded the perimeter area, and output a detection and recognition result; specifically: using a deep learning algorithm, detect whether the images of the second image cluster contain pedestrians; if it is detected that the images contain pedestrians, use a vibe algorithm to perform background difference on the area where the pedestrians are detected, and perform mobility judgment in the area by judging the differential image, specifically: judge whether the pedestrian is within the perimeter area; if the pedestrian is within the perimeter area, judge whether the pedestrian is within the perimeter area of ​​the continuous multiple-frame images and there is a differential change in the images within the perimeter area; if the pedestrian is within the perimeter area of ​​the continuous multiple-frame images and there is a differential change in the images within the perimeter area, an alarm is triggered and a snapshot is taken; otherwise, re-judge whether the pedestrian is within the perimeter area; if the pedestrian is not within the perimeter area, automatically segment the image based on the perimeter area again; if it is detected that the image does not contain pedestrians, automatically segment the image based on the perimeter area again.

3. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the perimeter intrusion identification method according to claim 1.

4. An electronic device, characterized in that: The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor executes the perimeter intrusion identification method described in claim 1.

Images