Intelligent perimeter security method and system based on multi-data fusion

Abstract

Multiple embodiments of this specification relate to the field of information technology, and specifically to a multi-data fusion intelligent perimeter security method and system. The method includes: acquiring multi-source sensor data; identifying the type of intrusion object crossing the perimeter through video data, and synchronously recording the detection values ​​of other sensors when the intrusion object crosses the perimeter, and identifying events or behaviors based on the detection values; constructing a multimodal feature data set based on the events and behaviors; generating a reference baseline for perimeter intrusion identification based on the multimodal feature data set; comparing subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times, and revising the reference baseline based on the comparison results; identifying perimeter intrusion based on the revised reference baseline, obtaining the intrusion object type, and generating an intrusion detection result based on the intrusion object type.

Description

Technical Field

[0001] Multiple embodiments of this specification relate to the field of information technology, and more particularly to a multi-data fusion intelligent perimeter security method and system. Background Art

[0002] With the development of society and the increasing demand for security, traditional perimeter security systems are no longer able to meet increasingly complex security needs. Early perimeter security systems relied primarily on physical barriers such as fences and walls, supplemented by simple alarm devices. However, these traditional methods are insufficient in the face of modern security threats, particularly with regard to detection accuracy, response speed, and false alarm rates.

[0003] In recent years, with the advancement of sensor technology and artificial intelligence, intelligent perimeter security systems based on multi-source sensor data fusion have become a research hotspot. By integrating multiple sensor types (such as video surveillance, infrared detectors, vibration sensors, sound sensors, fiber optic sensors, photoelectric beam detectors, and meteorological sensors), these new security systems can monitor the perimeter environment in real time and identify and warn of potential intrusions. However, practical applications still face several challenges. First, different types of sensors have varying operating principles and output formats, making it difficult to effectively fuse this heterogeneous data and extract useful information. Second, complex environmental factors (such as inclement weather) can affect sensor performance, leading to false alarms or missed alerts. Therefore, research is needed on intelligent perimeter security recognition technologies that utilize multi-data fusion. Summary of the Invention

[0004] Multiple embodiments of this specification describe a multi-data fusion intelligent perimeter security method and system.

[0005] In a first aspect, the embodiments of this specification provide a multi-data fusion intelligent perimeter security method, including the following steps:

[0006] Acquiring multi-source sensor data, wherein the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric beam detector data;

[0007] Identify the type of intruder crossing the perimeter through video data, and simultaneously record the detection values ​​of other sensors when the intruder crosses the perimeter, and identify the event or behavior based on the detection values;

[0008] Constructing a multimodal feature dataset based on the events and behaviors;

[0009] generating a reference baseline for perimeter intrusion identification based on the multimodal feature dataset;

[0010] Comparing subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times, and revising the reference baseline based on the comparison results;

[0011] Perimeter intrusion is identified based on the modified reference baseline to obtain the intrusion object type, and an intrusion detection result is generated according to the intrusion object type.

[0012] In a second aspect, the embodiments of this specification provide a multi-data fusion intelligent perimeter security system, including:

[0013] An acquisition module acquires multi-source sensor data, wherein the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, optical fiber sensor data, and photoelectric beam detector data;

[0014] An identification module that uses video data to identify the type of intruder that crosses the perimeter, simultaneously records the detection values ​​of other sensors when the intruder crosses the perimeter, and identifies the event or behavior based on the detection values;

[0015] A data set module constructs a multimodal feature data set based on the events and behaviors;

[0016] A baseline module, which generates a reference baseline for perimeter intrusion identification based on the multimodal feature dataset;

[0017] a correction module, comparing subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times, and correcting the reference baseline based on the comparison results;

[0018] The detection module identifies perimeter intrusion based on the modified reference baseline, obtains the intrusion object type, and generates an intrusion detection result according to the intrusion object type.

[0019] In a third aspect, embodiments of this specification provide an electronic device, including a processor and a memory;

[0020] The processor is connected to the memory;

[0021] The memory is used to store executable program code;

[0022] The processor reads the executable program code stored in the memory to run a program corresponding to the executable program code, so as to execute the method described in any one of the above aspects.

[0023] In a fourth aspect, an embodiment of this specification provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in any one of the above aspects is implemented.

[0024] In a fifth aspect, embodiments of this specification provide a computer program product, including a computer program, which implements the method described in any of the above aspects when executed by a processor.

[0025] The beneficial effects of the technical solutions provided by some embodiments of this specification include at least:

[0026] In various embodiments of this specification, a multi-data fusion intelligent perimeter security method and system is provided. By integrating multiple types of data, including video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric beam detector data, it is possible to more comprehensively capture the behavioral characteristics of intrusions, thereby improving the accuracy and reliability of identification. Different sensors have their own advantages in different environments and can adapt to different environmental conditions. Utilizing advanced technologies such as background modeling, foreground target extraction, and motion detection algorithms, combined with multimodal feature datasets and behavioral recognition models, it is possible to effectively distinguish actual threats from non-threatening events, reducing false alarms caused by environmental factors.

[0027] Other features and advantages of the various embodiments of this specification will be further disclosed in the following detailed description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of this specification, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] Figure 1 This is a schematic diagram of the perimeter security scenario provided in the embodiments of this specification.

[0030] Figure 2 This is a flow chart of the intelligent perimeter security method provided in the embodiments of this specification.

[0031] Figure 3 This is a schematic diagram of multi-source sensor data provided in the embodiments of this specification.

[0032] Figure 4 This is a schematic diagram of identifying the type of intrusion object provided in the embodiments of this specification.

[0033] Figure 5 A reference baseline diagram provided for the embodiments of this specification.

[0034] Figure 6 Schematic diagram of the intelligent perimeter security system provided in the embodiments of this specification.

[0035] Figure 7This is a schematic diagram of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0036] The following is an explanation and description of the technical solutions of the embodiments of this specification in conjunction with the drawings of the embodiments of this specification. However, the following embodiments are only preferred embodiments of this specification and are not exhaustive. Based on the embodiments in the implementation mode, other embodiments obtained by those skilled in the art without making any creative work are all within the scope of protection of this specification.

[0037] Throughout this specification, the claims, and the accompanying drawings, the terms "first," "second," "third," and the like are used to distinguish between different items, not to describe a particular order. Furthermore, the terms "including," "having," and any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or elements is not limited to the listed steps or elements but may optionally include steps or elements not listed, or may include other steps or elements inherent to the process, method, product, or apparatus.

[0038] In the following description, terms such as "inside", "outside", "up", "down", "left", "right", etc. that indicate directions or positional relationships are only used to facilitate the description of the embodiments and simplify the description. They do not indicate or imply that the devices or elements referred to must have a specific direction, be constructed and operate in a specific direction. Therefore, they should not be understood as limitations on this specification.

[0039] The data involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection of relevant data complies with the relevant laws, regulations and standards of relevant countries and regions.

[0040] Before introducing the technical solution in this specification, the application scenarios and related technologies of the technical solution are introduced.

[0041] Perimeter security involves implementing protective measures along the boundaries of a specific area to prevent unauthorized entry. It's widely used in residential areas, commercial parks, industrial plants, airports, and other locations requiring security control. Perimeter security solutions include traditional physical barriers such as solid walls, fences, and barbed wire to deter intrusion. Electronic fencing is another emerging technology that combines electric shock deterrents with alarms, triggering alerts when someone attempts to climb or break into the area. Video surveillance systems use cameras to monitor the perimeter in real time and record footage for subsequent investigations. Intrusion detection systems use devices such as vibration sensors, infrared detectors, and microwave radars to detect illegal trespassing. Intelligent analysis software uses artificial intelligence algorithms to analyze surveillance footage, automatically identify suspicious activity, and issue timely warnings. Infrared detectors can be infrared cameras, which have a wide detection range.

[0042] For example, please see the attached Figure 1 For example, consider a building with a wall and open lawn in front of it. An electronic fence can be installed on the wall to form a perimeter 12, which not only physically blocks the building but also issues a warning signal in the event of an intrusion attempt. Multiple high-definition cameras can be deployed inside and outside the wall to form a video surveillance network, ensuring that anyone approaching the building can be clearly observed. Furthermore, fiber optic sensors buried in the lawn, or infrared detectors or photoelectric beam detectors placed around the lawn edge, can be used to form an open space perimeter 11. Combined with intelligent analysis by the backend server 20, intrusion monitoring results are obtained.

[0043] First, this manual provides a multi-data fusion intelligent perimeter security method, please refer to the attached Figure 2 , including the steps of:

[0044] Step S1) Acquire multi-source sensor data, including video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric beam detector data. Figure 3 , is a schematic diagram of the sensor used in this embodiment.

[0045] Video surveillance cameras capture real-time images and video footage, providing intuitive visual information and forming video data. 22 Recommended video surveillance cameras should feature high-definition resolution, day / night operation, and a waterproof and dustproof design, making them suitable for outdoor use. Infrared detectors use infrared radiation emitted by objects to detect targets and generate infrared detection data. 23 Infrared detectors are passive, require no lighting, and can penetrate obstacles such as smoke. Examples include the Optex AX-130TN and Paradox PWX-445.

[0046] Vibration sensor data involves sensing vibrations caused by climbing, cutting, or other destructive activities, generating vibration sensor data 24. For perimeter security, vibration sensors with high sensitivity, adjustable thresholds, and compatibility with various surfaces are recommended. For example, the Ricochet MICRA-V and Fiber SenSysFD3220 are two commonly used models.

[0047] The sound sensor can collect sound and can also collect sound sensor data 25 even when the view is blocked. For example, it can identify abnormal sound patterns such as cracking or shouting. It is necessary to filter out environmental noise.

[0048] Fiber optic sensors use changes in optical signals to detect changes in physical quantities such as pressure and temperature, making them particularly suitable for long-distance monitoring. Due to their resistance to electromagnetic interference, long-distance monitoring, and high accuracy, fiber optic sensors are often buried at the perimeter of open spaces to detect the pressure generated by the perimeter and obtain fiber optic sensor data 26. A photoelectric cross-beam detector consists of a transmitter and a receiver. When the light beam between the two is interrupted, an alarm is triggered, and photoelectric cross-beam detector data 27 is obtained. These sensors offer the advantages of easy installation, fast response, and suitability for outdoor use. These sensors are ultimately connected to a server 20 via a controller 21. The method is executed by the server 20, or by both the server 20 and the controller 21. Alternatively, the server 20 may be omitted, and the method may be implemented solely by the controller 21.

[0049] Step S2) Identify the type of the intruder that crosses the perimeter through the video data, and simultaneously record the detection values ​​of other sensors when the intruder crosses the perimeter, and identify the event or behavior based on the detection values.

[0050] Before intrusion identification can be performed, data accumulation is required. When using video data to identify the type of intruders crossing the perimeter, it is recommended to perform the identification under conditions with good visibility and lighting. Identification should be suspended under conditions of obstructed vision or poor lighting. For example, identification of intruder types can be performed during clear daylight hours. This can be combined with proactively setting certain intrusion targets to allow for the collection of relevant video data. During the data accumulation phase, video data collection is suspended at night.

[0051] Among them, the method of identifying the type of intruder crossing the perimeter through video data includes:

[0052] Read the video data of the perimeter monitoring area, identify the background and build a background model;

[0053] extracting a foreground object from an image included in the video data according to the background model;

[0054] Using motion detection algorithms to identify moving foreground objects in the video image 31 and generate corresponding movement trajectories;

[0055] Performing image segmentation on the moving foreground target 31 based on the moving trajectory to extract the image area of ​​the moving foreground target 31;

[0056] According to the image area of ​​the moving foreground object 31, the type of the intrusion object is identified.

[0057] Identifying the background and building a background model from the acquired video data specifically involves obtaining image regions that are static in most frames of the video data, which are referred to as "normally static regions." Using an image recognition model known in the art, objects in the normally static regions are identified as background objects. These objects are characterized by their names, and images of the background objects are obtained based on their regions.

[0058] All frames of video data in which each background object is not static are obtained, recorded as dynamic frames. The position of the background object in the dynamic frames is identified, and the motion range of the background object is established based on the position of the background object in all dynamic frames. A background model is established based on the name, area, image, and motion range of all background objects identified in the normally static area.

[0059] Please see the attached Figure 4 For example, the grass, house, and dirt road in the image are all static image regions in most frames, and background objects can be identified through recognition. The flower bushes on either side of the house door are also static image regions in most frames, but they can move in windy conditions. By reading all frames of video data showing the flower bushes in a non-static state, the motion range of the flower bushes can be established. This ensures that in subsequent recognition, when the flower bushes are in motion, they will not be mistakenly identified as foreground objects.

[0060] Based on the established background model, foreground targets are extracted from the images contained in the video data. These foreground targets represent objects other than the background in the video, and a moving foreground target 31 is identified from these objects. Based on the image of the range of the moving foreground target 31, the type of intruding object is obtained with the help of the image recognition model. For example, the pedestrians, animals and birds appearing in the figure can be regarded as moving foreground targets 31 because they are basically in a moving state. Based on the image area of ​​the moving foreground target 31 in any frame, the type of intruding object can be obtained through image recognition. The trophy in the figure is only in a moving state in some frames, and is in a stationary state in a large number of subsequent frames. Therefore, although it is identified as a foreground target, it will not be identified as a moving foreground target 31.

[0061] When enough data is collected during the day, the detection values ​​of other sensors when the intruder crosses the perimeter and the events or behaviors identified based on the detection values ​​can be used to establish a multimodal feature dataset.

[0062] For example, when a person or animal is identified by video data as a perimeter intrusion, the infrared sensor can detect the heat emitted by the person or animal, and is still effective at night or in low visibility conditions. If a person or animal climbs a fence to enter the perimeter, the vibration sensor can detect the resulting ground or wall vibrations. Abnormal sounds such as footsteps, talking or other noises can be captured by the sound sensor, which is helpful for distinguishing human activities from other types of interference. Fiber optic sensors installed on fences or buried underground can detect tiny deformations and therefore can detect perimeter intrusions. Photoelectric beam detectors are usually used in pairs, with one emitting a beam and the other receiving. If an object blocks the beam, such as an intruder passing through, an alarm will be triggered.

[0063] For example, when a bird is identified as intruding a perimeter through video data, vibration sensors, fiber optic sensors, and photoelectric detection devices may not be particularly effective in responding to bird intrusions due to the special characteristics of birds. Infrared sensors can detect the heat emitted by birds. Sound sensors can detect the flapping of wings produced by flying birds. Occasionally, the sound of birds chirping can be detected. When a bird happens to pass through a photoelectric detection device, the photoelectric detection device can detect it. If the bird does not chirp or pass through a photoelectric detection device, it will be detected by video data and infrared detection data during the day and by infrared detection data at night. Objects accidentally thrown into the perimeter may be detected by video data, vibration sensor data, and sound sensor data.

[0064] When enough data is collected, it can cover enough intrusion target types and can also identify events or behaviors. Specifically, the method of identifying events or behaviors based on the detection values ​​includes:

[0065] When an intruder crosses the perimeter, the detection values ​​of multi-source sensor data are synchronized and aligned;

[0066] According to the preset event recognition model corresponding to each sensor, the event of the detection value generated by each sensor is identified respectively;

[0067] Arrange all events according to the timeline to form an event sequence;

[0068] randomly deleting one or more events in the event sequence multiple times to obtain multiple adjusted event sequences;

[0069] Obtaining identified behaviors based on responses of a pre-built behavior recognition model to the event sequence and the adjustment event sequence;

[0070] Events that have no impact on the identification of the behavior are recorded as separate events;

[0071] All behaviors and individual events are identified as a result of the detection value identifying the event or behavior.

[0072] In this embodiment, each sensor corresponds to an event recognition model. For example, vibration sensor data obtained by a vibration sensor installed on a wall can identify events such as being hit, climbed, or leaned against based on vibration characteristics such as frequency, amplitude, frequency, and duration. Vibration sensor data obtained by a vibration sensor installed on a low fence at the edge of a garden can identify events such as being hit, climbed over, leaned against, or stepped on based on vibration characteristics such as frequency, amplitude, frequency, and duration.

[0073] The optical fiber sensor data corresponding to the optical fiber sensor set up underground can detect trampling events and ground vibration events.

[0074] The event recognition model corresponding to the vibration sensor data obtained by the vibration sensors installed on the walls and low fences can be obtained by using a machine learning model and training with sample data manually labeled with events. Alternatively, other technologies known in the art can be used.

[0075] Sensors corresponding to other types of multi-source sensor data are also obtained using machine learning models trained with sample data after manually annotated events, or using model building techniques disclosed in the art.

[0076] For example, when there is a certain number of multi-source sensor data indicating pedestrians intruding a perimeter set on a lawn, identified by video data, the detection value events generated by each sensor other than the video surveillance camera are identified separately. All events are sorted along a timeline, and the resulting event sequence may include:

[0077] Event sequence 1: {Infrared detection data: heat source passes [event], vibration sensor data: no [event], sound sensor data: no [event], fiber optic sensor data: passes [event], photoelectric beam detector data: blocked [event]};

[0078] Event sequence 2: {Infrared detection data: heat source passed [event], vibration sensor data: passed [event], sound sensor data: no [event], fiber optic sensor data: passed [event], photoelectric beam detector data: no [event]};

[0079] Event sequence 3: {Infrared detection data: heat source passes [event], vibration sensor data: no [event], sound sensor data: voice [event], fiber optic sensor data: passes [event], photoelectric beam detector data: blocked [event]}.

[0080] The input of the behavior recognition model is an event sequence, and the output is a behavior. Inputting the aforementioned event sequences 1-3 into the behavior recognition model can all produce pedestrian intrusion behavior. By deleting one or more events from event sequence 1, the resulting event sequence 1-adjusted sequence 1 is: Event sequence 1-adjusted sequence 1: {Infrared detection data: No [event], Vibration sensor data: No [event], Sound sensor data: No [event], Fiber optic sensor data: Passed [event], Photoelectric beam detector data: Blocked [event]}. When event sequence 1-adjusted sequence 1 is input into the behavior recognition model, the behavior recognition model will output no behavior. That is, the same behavior can correspond to multiple event sequences. Some event sequences may not correspond to any behavior, meaning the behavior recognition model outputs no behavior.

[0081] The establishment of the behavior recognition model also uses the establishment of a machine learning model, and obtains sample data formed by the event sequence of manually labeled behaviors and the adjusted event sequence for training.

[0082] According to event sequences 1-3, the individual events that can be obtained are photoelectric detector data: occlusion [event], vibration sensor data: passage [event], and sound sensor data: voice [event].

[0083] Step S3) Construct a multimodal feature dataset based on the events and behaviors.

[0084] The method for constructing a multimodal feature dataset based on the events and behaviors includes:

[0085] Read all event sequences corresponding to the same behavior and adjust the event sequence;

[0086] Obtaining an intersection of several event sequences;

[0087] The intersection is regarded as a multimodal feature data.

[0088] By taking the intersection of three event sequences corresponding to pedestrian intrusions, also identified using video data, we obtain a multimodal feature data set: Pedestrian intrusion behavior - {Infrared detection data: Heat source passing [event], Vibration sensor data: No [event], Sound sensor data: No [event], Fiber optic sensor data: Passing [event], Photoelectric beam detector data: No [event]}. This indicates that when a pedestrian intrudes, the infrared detection data and fiber optic sensor data must be reflected; the other multi-source sensor data can remain unchanged.

[0089] Because pedestrians are tall, they may not trigger a change in the photoelectric beam detector data. Similarly, when an animal intrusion into the perimeter is detected through video data, the corresponding multimodal feature data is: Animal Intrusion Behavior - {Infrared Detection Data: Heat Source Passed [Event], Vibration Sensor Data: No [Event], Sound Sensor Data: No [Event], Fiber Optic Sensor Data: No [Event], Photoelectric Detector Data: Occlusion [Event]}. Because some animals are light, they may not necessarily trigger a change in the fiber optic sensor data, but they will definitely trigger changes in the infrared detection data and the photoelectric beam detector data.

[0090] Similarly, when a bird intrusion into a perimeter is detected through video data, the corresponding multimodal feature data is: {Infrared detection data: Heat source passing [event], Vibration sensor data: No [event], Sound sensor data: Wing flapping [event], Fiber optic sensor data: No [event], Photoelectric beam detector data: No [event]}. The behavior associated with this multimodal feature data is: Bird intrusion into the perimeter.

[0091] Step S4) generating a reference baseline for perimeter intrusion identification based on the multimodal feature dataset.

[0092] The method for generating a reference baseline for perimeter intrusion identification based on the multimodal feature dataset includes:

[0093] Associating multimodal feature data with corresponding behaviors as a reference point;

[0094] Obtaining a reference baseline according to all reference points corresponding to the multimodal feature dataset;

[0095] The method of comparing the subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times and correcting the reference baseline based on the comparison results includes:

[0096] Obtaining a detection value generated by each sensor of the multi-source sensor data;

[0097] According to the preset event recognition model corresponding to each sensor, the events of the detection values ​​generated by each sensor are respectively identified, and then the event sequence is obtained and the event sequence is adjusted;

[0098] Identify sequences of events and the behavior that adjusts them;

[0099] Comparing with the reference baseline to obtain all reference points corresponding to the same behavior, and obtaining multimodal feature data of the reference points;

[0100] Obtaining the intersection of all event sequences and adjustment event sequences corresponding to the same behavior, and using the intersection as updated multimodal feature data;

[0101] The reference baseline is corrected according to all updated multimodal feature data.

[0102] For example, please see the attached Figure 5 Multimodal feature data: {Infrared detection data: Heat source passing [event], Vibration sensor data: No [event], Sound sensor data: No [event], Fiber optic sensor data: Passing [event], Photoelectric beam detector data: No [event]}, associated behavior: Pedestrian intrusion into the perimeter, which is one reference point. Multimodal feature data: {Infrared detection data: Heat source passing [event], Vibration sensor data: No [event], Sound sensor data: No [event], Fiber optic sensor data: No [event], Photoelectric beam detector data: Blocking [event]}, associated behavior: Animal intrusion into the perimeter, which is another reference point. Multimodal feature data: {Infrared detection data: Heat source passing [event], Vibration sensor data: No [event], Sound sensor data: No [event], Fiber optic sensor data: No [event], Photoelectric beam detector data: No [event]}, associated behavior: Bird intrusion into the perimeter, which is another reference point.

[0103] Step S5) Compare the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times, and modify the reference baseline based on the comparison results.

[0104] For example, if a photoelectric beam detector malfunctions and consistently fails to detect an occlusion event caused by an animal intruding the perimeter, the video surveillance camera can still detect the animal's intrusion. The corresponding multimodal feature data generated will show "Photoelectric beam detector data: No [event]. After a period of time, the reference point corresponding to the animal intrusion will be updated to {Infrared detection data: Heat source passing [event], Vibration sensor data: No [event], Sound sensor data: No [event], Fiber optic sensor data: No [event], Photoelectric beam detector data: No [event]}." This removes the requirement for photoelectric beam detector data to generate an event.

[0105] Step S6) Identify perimeter intrusion based on the corrected reference baseline, obtain the intrusion object type, and generate an intrusion detection result according to the intrusion object type.

[0106] The method for identifying perimeter intrusion based on the modified reference baseline, obtaining the intrusion object type, and generating the intrusion detection result according to the intrusion object type includes:

[0107] Read multi-source sensor data of a preset duration, and obtain the event corresponding to each sensor in the multi-source sensor data based on a preset event recognition model corresponding to each sensor;

[0108] Arrange all events according to the time axis to form an event sequence, and then obtain multiple adjustment event sequences;

[0109] Comparing the event sequence and the plurality of adjusted event sequences with reference points in the reference baseline to obtain matching reference points;

[0110] obtaining the type of the intruded object according to the reference point;

[0111] The object type is compared with a preset exclusion object type table. When the object type does not belong to the exclusion object type table, an intrusion detection result is generated as an intrusion existence and an intrusion alarm is issued.

[0112] For example, when a bird invades a perimeter set on a lawn, the resulting event sequence is {infrared detection data: heat source passing [event], vibration sensor data: no [event], sound sensor data: wings flapping [event], fiber optic sensor data: no [event], photoelectric beam detector data: occlusion [event]}. Directly matching the event sequence to reference points in the reference baseline requires fuzzy matching to obtain matching events, but fuzzy matching has low accuracy. Using an adjusted event sequence, the following adjusted event sequences may be generated: {infrared detection data: no [event], vibration sensor data: no [event], sound sensor data: wings flapping [event], fiber optic sensor data: no [event], photoelectric beam detector data: no [event]}, and {infrared detection data: heat source passing [event], vibration sensor data: no [event], sound sensor data: wings flapping [event], fiber optic sensor data: no [event], photoelectric beam detector data: no [event]}. The second adjusted event sequence can then accurately match the result of the bird's perimeter intrusion behavior. Therefore, comparing the event sequence and multiple adjusted event sequences with the reference points in the reference baseline has higher accuracy.

[0113] When a bird intrusion into the perimeter is detected, the exclusion object type table clearly indicates that the bird is included. Therefore, the intrusion detection result is "no intrusion" and no intrusion alarm is issued. When a pedestrian intrusion into the perimeter is detected, the intrusion detection result is "present intrusion" and an intrusion alarm is issued. This improves the accuracy of perimeter intrusion detection and reduces false alarm rates.

[0114] On the other hand, please see the attached Figure 6 , a multi-data fusion intelligent perimeter security system, including:

[0115] An acquisition module 100 acquires multi-source sensor data, wherein the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, optical fiber sensor data, and photoelectric beam detector data;

[0116] Identification module 200, which identifies the type of intruder crossing the perimeter through video data, and simultaneously records the detection values ​​of other sensors when the intruder crosses the perimeter, and identifies the event or behavior based on the detection values;

[0117] The data set module 300 constructs a multimodal feature data set based on the events and behaviors;

[0118] A baseline module 400 generates a reference baseline for perimeter intrusion identification based on the multimodal feature dataset;

[0119] The correction module 500 compares the subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times, and corrects the reference baseline based on the comparison results;

[0120] The detection module 600 identifies perimeter intrusion based on the modified reference baseline, obtains the intrusion object type, and generates an intrusion detection result according to the intrusion object type.

[0121] See also Figure 7 A schematic structural diagram of an electronic device provided in an embodiment of this specification is shown.

[0122] like Figure 7As shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. The communication bus 1102 may be used to implement communication between the aforementioned components. The user interface 1103 may include buttons, and optionally may also include a standard wired interface or a wireless interface. The network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc. The processor 1101 may include one or more processing cores. The processor 1101 utilizes various interfaces and circuits to connect the various components within the entire electronic device 1100. By running or executing instructions, programs, code sets, or instruction sets stored in the memory 1105 and accessing data stored in the memory 1105, it performs various functions of the routing device 1100 and processes data. Optionally, the processor 1101 may be implemented in hardware using at least one of a DSP, an FPGA, and a PLA. The processor 1101 may integrate one or a combination of a CPU, a GPU, and a modem. The CPU primarily processes the operating system, user interface, and applications; the GPU is responsible for rendering and drawing content displayed on the display; and the modem handles wireless communications.

[0123] It is understandable that the above-mentioned modem may not be integrated into the processor 1101, but may be implemented by a separate chip.

[0124] Memory 1105 may include either RAM or ROM. Optionally, memory 1105 may include non-transitory computer-readable media. Memory 1105 may be used to store instructions, programs, codes, code sets, or instruction sets. Memory 1105 may include a program storage area and a data storage area. The program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, sound playback function, image playback function, etc.), instructions for implementing the aforementioned method embodiments, etc.; the data storage area may store data related to the aforementioned method embodiments, etc. Memory 1105 may also optionally be at least one storage device located remotely from the aforementioned processor 1101. Memory 1105, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and application programs. Processor 1101 may be configured to invoke the application programs stored in memory 1105 and execute the methods described in the aforementioned embodiments.

[0125] The embodiments of this specification also provide a computer-readable storage medium having instructions stored therein that, when executed on a computer or processor, cause the computer or processor to perform the steps of the aforementioned embodiments. If the components of the aforementioned electronic device are implemented as software functional units and sold or used as independent products, they may be stored in the computer-readable storage medium.

[0126] The embodiments of this specification also provide a computer program product, including a computer program, which implements multiple steps in the above embodiments when executed by a processor.

[0127] In the absence of conflict, the technical features in this embodiment and implementation scheme can be combined arbitrarily.

[0128] In the above embodiments, all or part of the embodiments may be implemented using software, hardware, firmware, or any combination thereof. When implemented using software, all or part of the embodiments may be implemented in the form of a computer program product. The computer program product comprises multiple computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this specification are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted via the computer-readable storage medium. The computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium accessible by a computer or a data storage device such as a server or data center that integrates multiple available media. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a digital versatile disc (DVD)), or a semiconductor medium (eg, a solid state drive (SSD)).

[0129] When implemented via hardware or firmware, the aforementioned method flow is programmed into the hardware circuit to obtain the corresponding hardware circuit structure and realize the corresponding function. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit, whose logical function is determined by the user's device programming. Designers can "integrate" a digital system on a PLD through self-programming, eliminating the need for chip manufacturers to design and manufacture dedicated integrated circuit chips. Moreover, today, instead of manually manufacturing integrated circuit chips, this programming is often performed using "logic compiler" software. This is similar to the software compiler used in program development. Before compilation, the original code must also be written in a specific programming language, called a hardware description language (HDL). There are not just one HDL, but many. Those skilled in the art will also understand that simply by programming the method flow in one of the aforementioned hardware description languages ​​and programming it into the integrated circuit, a hardware circuit that implements the logical method flow can be easily obtained.

[0130] The embodiments described above are merely preferred embodiments of this specification and are not intended to limit the scope of this specification. Without departing from the design spirit of this specification, various modifications and improvements made to the technical solutions of this specification by ordinary technicians in this field should fall within the scope of protection determined by the claims of this specification.

Claims

1. An intelligent perimeter security method based on multi-data fusion, characterized in that: Including steps: Acquiring multi-source sensor data, wherein the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric beam detector data; Identify the type of intruder crossing the perimeter through video data, and simultaneously record the detection values ​​of other sensors when the intruder crosses the perimeter, and identify the event or behavior based on the detection values; Constructing a multimodal feature dataset based on the events and behaviors; generating a reference baseline for perimeter intrusion identification based on the multimodal feature dataset; Comparing subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times, and revising the reference baseline based on the comparison results; Identify perimeter intrusion based on the modified reference baseline, obtain the intrusion object type, and generate an intrusion detection result according to the intrusion object type; The method for identifying an event or behavior according to the detection value includes: When an intruder crosses the perimeter, the detection values ​​of multi-source sensor data are synchronized and aligned; According to the preset event recognition model corresponding to each sensor, the event of the detection value generated by each sensor is identified respectively; Arrange all events according to the timeline to form an event sequence; randomly deleting one or more events in the event sequence multiple times to obtain multiple adjusted event sequences; Obtaining identified behaviors based on responses of a pre-built behavior recognition model to the event sequence and the adjustment event sequence; Events that have no impact on the identification of the behavior are recorded as separate events; All identified behaviors and individual events are used as the result of the detection value identifying the event or behavior; The method for generating a reference baseline for perimeter intrusion identification based on the multimodal feature dataset includes: Associating multimodal feature data with corresponding behaviors as a reference point; A reference baseline is obtained according to all reference points corresponding to the multimodal feature dataset.

2. The multi-data fusion intelligent perimeter security method according to claim 1, characterized in that: Methods for identifying the type of intruders that cross the perimeter using video data include: Read the video data of the perimeter monitoring area, identify the background and build a background model; extracting a foreground object from an image included in the video data according to the background model; Use motion detection algorithm to identify moving foreground targets in the video image and generate corresponding movement trajectories; Perform image segmentation on the moving foreground target based on the moving trajectory and extract the image area of ​​the moving foreground target; The type of intrusion object is identified based on the image area of ​​the moving foreground target.

3. The intelligent perimeter security method based on multi-data fusion according to claim 1, characterized in that: Based on the events and behaviors, the method of constructing a multimodal feature dataset includes: Read all event sequences corresponding to the same behavior and adjust the event sequence; Obtaining an intersection of several of the event sequences; The intersection is regarded as a multimodal feature data.

4. The multi-data fusion intelligent perimeter security method according to claim 3, characterized in that: The method of comparing the subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times and correcting the reference baseline based on the comparison results includes: Obtaining a detection value generated by each sensor of the multi-source sensor data; According to the preset event recognition model corresponding to each sensor, the events of the detection values ​​generated by each sensor are respectively identified, and then the event sequence is obtained and the event sequence is adjusted; Identify sequences of events and the actions that adjust them; Comparing with the reference baseline to obtain all reference points corresponding to the same behavior, and obtaining multimodal feature data of the reference points; Obtaining the intersection of all event sequences and adjustment event sequences corresponding to the same behavior, and using the intersection as updated multimodal feature data; The reference baseline is corrected according to all updated multimodal feature data.

5. The multi-data fusion intelligent perimeter security method according to claim 1, characterized in that: The method for identifying perimeter intrusion based on the modified reference baseline, obtaining the intrusion object type, and generating the intrusion detection result according to the intrusion object type includes: Read multi-source sensor data of a preset duration, and obtain the event corresponding to each sensor in the multi-source sensor data based on a preset event recognition model corresponding to each sensor; Arrange all events according to the time axis to form an event sequence, and then obtain multiple adjustment event sequences; Comparing the event sequence and the plurality of adjusted event sequences with reference points in the reference baseline to obtain matching reference points; obtaining the type of the intruded object according to the reference point; The object type is compared with a preset exclusion object type table. When the object type does not belong to the exclusion object type table, an intrusion detection result is generated as an intrusion existence and an intrusion alarm is issued.

6. Multi-data fusion intelligent perimeter security system, characterized by: include: An acquisition module acquires multi-source sensor data, wherein the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, optical fiber sensor data, and photoelectric beam detector data; An identification module that uses video data to identify the type of intruder that crosses the perimeter, simultaneously records the detection values ​​of other sensors when the intruder crosses the perimeter, and identifies the event or behavior based on the detection values; A data set module constructs a multimodal feature data set based on the events and behaviors; A baseline module, which generates a reference baseline for perimeter intrusion identification based on the multimodal feature dataset; a correction module, comparing subsequently collected multi-source sensor data of the same intrusion object type with the reference baseline multiple times, and correcting the reference baseline based on the comparison results; a detection module, which identifies perimeter intrusion based on the modified reference baseline, obtains the intrusion object type, and generates an intrusion detection result according to the intrusion object type; The method for identifying an event or behavior according to the detection value includes: When an intruder crosses the perimeter, the detection values ​​of multi-source sensor data are synchronized and aligned; According to the preset event recognition model corresponding to each sensor, the event of the detection value generated by each sensor is identified respectively; Arrange all events according to the timeline to form an event sequence; randomly deleting one or more events in the event sequence multiple times to obtain multiple adjusted event sequences; Obtaining identified behaviors based on responses of a pre-built behavior recognition model to the event sequence and the adjustment event sequence; Events that have no impact on the identification of the behavior are recorded as separate events; All identified behaviors and individual events are used as the result of the detection value identifying the event or behavior; The method for generating a reference baseline for perimeter intrusion identification based on the multimodal feature dataset includes: Associating multimodal feature data with corresponding behaviors as a reference point; A reference baseline is obtained according to all reference points corresponding to the multimodal feature dataset.

7. An electronic device, characterized in that including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

9. Computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images