Multi-data fusion intelligent perimeter security method and system

Abstract

The embodiment of the invention relates to the technical field of information, in particular to an intelligent perimeter security and protection method and system based on multi-data fusion. The method comprises the following steps: acquiring multi-source sensor data; identifying the type of an intrusion object passing through the perimeter through the video data, synchronously recording detection values of other sensors when the intrusion object passes through the perimeter, and identifying an event or a behavior according to the detection values; constructing a multi-modal feature data set according to the event and the behavior; based on the multi-modal feature data set, generating a reference baseline for perimeter intrusion identification; comparing subsequently acquired multi-source sensor data of the same invasion object type with the reference baseline for multiple times, and correcting the reference baseline according to a comparison result; and recognizing perimeter intrusion based on the corrected reference baseline to obtain an intrusion object type, and generating an intrusion detection result according to the intrusion object type.

Description

Technical Field

[0001] Multiple embodiments of this specification relate to the field of information technology, and specifically to an intelligent perimeter security method and system for multi-data fusion. Background Art

[0002] With the development of society and the improvement of security requirements, traditional perimeter security systems have become difficult to meet the increasingly complex security needs. Early perimeter security mainly relied on physical barriers such as fences and walls, supplemented by simple alarm devices. However, these traditional means are inadequate in the face of modern security threats, especially in terms of detection accuracy, response speed, and false alarm rate.

[0003] In recent years, with the development of sensor technology and artificial intelligence, intelligent perimeter security systems based on multi-source sensor data fusion have gradually become a research hotspot. This new type of security system can integrate various types of sensors (such as video surveillance, infrared detectors, vibration sensors, sound sensors, fiber optic sensors, photoelectric interlock detectors, and meteorological sensors) to monitor the perimeter environment in real time and identify and warn of potential intrusion behaviors. Nevertheless, several challenges still exist in practical applications. First, different types of sensors have different working principles and output formats, and how to effectively fuse this heterogeneous data and extract useful information is a difficult problem. Second, complex environmental factors (such as adverse weather conditions) may affect the working performance of sensors, leading to false alarms or missed alarms. Therefore, it is necessary to study the technology of intelligent perimeter security identification for multi-data fusion. Summary of the Invention

[0004] Multiple embodiments of this specification describe an intelligent perimeter security method and system for multi-data fusion.

[0005] In a first aspect, an embodiment of this specification provides an intelligent perimeter security method for multi-data fusion, including the steps of: Obtain multi-source sensor data, where the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric interlock detector data; Identify the type of intrusion object crossing the perimeter through the video data, and synchronously record the detection values of other sensors when the intrusion object crosses the perimeter, and identify events or behaviors based on the detection values; Construct a multi-modal feature data set according to the events and behaviors; Generate a reference baseline for perimeter intrusion identification based on the multi-modal feature data set; Compare the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times, and correct the reference baseline according to the comparison results; Identify perimeter intrusion based on the corrected reference baseline, obtain the type of the intrusion object, and generate an intrusion detection result according to the type of the intrusion object.

[0006] In a second aspect, an embodiment of the present specification provides an intelligent perimeter security system with multi-data fusion, including: An acquisition module that acquires multi-source sensor data, where the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and optoelectronic pair detector data; An identification module that identifies the type of intrusion object crossing the perimeter through video data, and synchronously records the detection values of other sensors when the intrusion object crosses the perimeter, and identifies events or behaviors according to the detection values; A data set module that constructs a multi-modal feature data set according to the events and behaviors; A baseline module that generates a reference baseline for perimeter intrusion identification based on the multi-modal feature data set; A correction module that compares the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times, and corrects the reference baseline according to the comparison results; A detection module that identifies perimeter intrusion based on the corrected reference baseline, obtains the type of the intrusion object, and generates an intrusion detection result according to the type of the intrusion object.

[0007] In a third aspect, an embodiment of the present specification provides an electronic device, including a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method described in any of the above aspects.

[0008] In a fourth aspect, an embodiment of the present specification provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the method described in any of the above aspects is implemented.

[0009] In a fifth aspect, an embodiment of the present specification provides a computer program product, including a computer program, and when the computer program is executed by a processor, the method described in any of the above aspects is implemented.

[0010] The beneficial effects brought by the technical solutions provided by some embodiments of the present specification at least include: In multiple embodiments of this specification, the provided intelligent perimeter security method and system for multi-data fusion can capture the behavioral characteristics of intrusion more comprehensively by integrating various types of data such as video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and optoelectronic pair detector data, improving the accuracy and reliability of recognition. Different sensors have their own advantages in different environments and can adapt to different environmental conditions. By using advanced technologies such as background modeling, foreground object extraction, and motion detection algorithms, combined with multi-modal feature datasets and behavior recognition models, actual threats and non-threat events can be effectively distinguished, reducing false alarms caused by environmental factors.

[0011] Other features and advantages of multiple embodiments of this specification will be further revealed in the following detailed description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] To more clearly illustrate the technical solutions in the embodiments of this specification, the drawings required for the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of this specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0013] Figure 1 It is a schematic diagram of the perimeter security scenario provided for the embodiments of this specification.

[0014] Figure 2 It is a schematic diagram of the process flow of the intelligent perimeter security method provided for the embodiments of this specification.

[0015] Figure 3 It is a schematic diagram of multi-source sensor data provided for the embodiments of this specification.

[0016] Figure 4 It is a schematic diagram of the recognition of the type of intrusion object provided for the embodiments of this specification.

[0017] Figure 5 It is a schematic diagram of the reference baseline provided for the embodiments of this specification.

[0018] Figure 6 It is a schematic diagram of the intelligent perimeter security system provided for the embodiments of this specification.

[0019] Figure 7 It is a schematic diagram of the electronic device provided for the embodiments of this specification. DETAILED DESCRIPTION

[0020] The technical solutions of the embodiments of this specification will be explained and illustrated below with reference to the accompanying drawings of the embodiments of this specification. However, the following embodiments are only the preferred embodiments of this specification and not all of them. Based on the embodiments in the implementation manners, other embodiments obtained by those skilled in the art without creative efforts all fall within the protection scope of this specification.

[0021] Terms such as "first", "second", "third", etc. in the specification, claims and the above accompanying drawings of this specification are used to distinguish different objects rather than to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally further includes steps or units not listed, or optionally further includes other steps or units inherent to these processes, methods, products or devices.

[0022] In the following description, terms indicating orientation or positional relationships such as "inner", "outer", "upper", "lower", "left", "right", etc. are only for the convenience of describing the embodiments and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be construed as a limitation of this specification.

[0023] The data involved in this application are all information and data authorized by users or fully authorized by all parties, and the collection of relevant data complies with the relevant laws, regulations and standards of relevant countries and regions.

[0024] Before introducing the technical solutions described in this specification, the application scenarios of the technical solutions and related technologies will be introduced.

[0025] Perimeter security refers to setting up protective measures on the boundary of a specific area to prevent unauthorized personnel or vehicles from entering. Perimeter security is widely used in places that require security control such as residential areas, commercial parks, factory areas, airports, etc. Related solutions for perimeter security include traditional physical barriers such as solid walls, fences, barbed wire, etc. for preventing illegal intrusion. An electronic fence is another new technology that combines an electric shock deterrence and an alarm function, and will trigger an alarm when someone tries to climb or damage it. A video surveillance system uses cameras to monitor the perimeter situation in real time and record video materials for post-event investigation. An intrusion detection system monitors whether there is any illegal crossing behavior through devices such as vibration sensors, infrared detectors, microwave radars, etc. Intelligent analysis software uses artificial intelligence algorithms to analyze the surveillance images to automatically identify suspicious behaviors and give early warnings in a timely manner. The infrared detector can be an infrared camera and has a relatively large detection range.

[0026] Exemplarily, please refer to the appendix Figure 1, taking a building with a wall and an empty grassland in front of the door as an example. An electronic fence can be installed on the wall to form a perimeter 12 of the wall, which not only plays a physical blocking role but also emits a warning signal when an intrusion attempt occurs. Multiple high-definition cameras can be arranged inside and outside the wall to form a video surveillance network to ensure that people approaching the building can be clearly observed. Further, fiber optic sensors buried under the grass or infrared detectors or photoelectric beam detectors arranged around the edge of the grassland are used to form an open space perimeter 11. Combining the intelligent analysis of the background server 20, the result of intrusion monitoring can be obtained.

[0027] First, this specification provides an intelligent perimeter security method for multi-data fusion. Please refer to the appendix Figure 2 , including the steps: Step S1) Obtain multi-source sensor data, where the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric beam detector data. Please refer to the appendix Figure 3 , which is a schematic diagram of the sensors used in this embodiment.

[0028] The video surveillance camera captures real-time images and videos, provides intuitive visual information, and forms video data 22. It is recommended to use devices with high-definition resolution, day and night mode, and waterproof and dustproof design for the video surveillance camera to collect data, so as to adapt to the outdoor use environment. The infrared detector uses the infrared radiation emitted by an object to detect the target and obtains infrared detection data 23. The infrared detector has the characteristics of passive operation without illumination and can penetrate obstacles such as smoke. Exemplarily, models such as Optex AX-130TN and Paradox PWX-445 are used.

[0029] The vibration sensor data involves that the vibration sensor can sense the vibration caused by climbing, cutting or other destructive behaviors and obtain vibration sensor data 24. For vibration sensors used in perimeter security, it is recommended to use sensors with high sensitivity, adjustable threshold, and suitable for various surfaces. Exemplarily, Ricochet MICRA-V and Fiber SenSys FD3220 are two commonly used models.

[0030] The sound sensor can collect sounds and can also collect sound sensor data 25 when the view is blocked. Exemplarily, it can identify abnormal sound patterns such as breaking sounds or shouting sounds, etc. It is necessary to filter the environmental noise.

[0031] Optical fiber sensors use changes in optical signals to detect changes in physical quantities such as pressure and temperature, and are especially suitable for long-distance monitoring. Due to their characteristics of anti-electromagnetic interference, long-distance monitoring, and high precision, optical fiber sensors are often buried at the perimeter of open spaces to detect the pressure generated when passing through the perimeter and obtain the optical fiber sensor data 26. The photoelectric opposed-beam detector consists of a transmitting end and a receiving end. When the beam between the two is interrupted, an alarm is triggered to obtain the photoelectric opposed-beam detector data 27. It has the advantages of simple installation, fast response, and suitability for outdoor use. These sensors are finally connected to the server 20 through the controller 21. The server 20 runs this method, or the server 20 and the controller 21 jointly complete this method. Or the server 20 is omitted, and this method is only implemented by the controller 21.

[0032] Step S2) Identify the type of intrusion object crossing the perimeter through video data, and synchronously record the detection values of other sensors when the intrusion object crosses the perimeter, and identify events or behaviors based on the detection values.

[0033] Before intrusion recognition, data accumulation is required. When identifying the type of intrusion object crossing the perimeter through video data, it is recommended to choose an environment with good vision, that is, lighting conditions. When the vision is blocked or the lighting is poor, the recognition is suspended. Exemplarily, the type of intrusion object is recognized on a sunny day. At this time, some objects can be actively set for intrusion so that the video data can collect corresponding data. During the data accumulation stage, video data collection is suspended at night.

[0034] Among them, the method for identifying the type of intrusion object crossing the perimeter through video data includes: Read the video data of the perimeter monitoring area, identify the background, and establish a background model; Extract the foreground object from the images included in the video data according to the background model; Use a motion detection algorithm to identify the moving foreground object 31 in the video frame and generate a corresponding moving trajectory; Based on the moving trajectory, perform image segmentation on the moving foreground object 31 and extract the image area of the moving foreground object 31; According to the image area of the moving foreground object 31, identify the type of intrusion object.

[0035] Identify the background and establish a background model from the acquired video data, specifically including: obtaining the image area that is in a stationary state in most frames in the video data, denoted as the frequently stationary area. Use the publicly available image recognition model in the field to identify the objects in the frequently stationary area as background objects. Characterize these objects with the names of the background objects, and at the same time, obtain the images of the background objects according to the areas of the background objects.

[0036] Obtain all frames of the video data when each background object is in a non-static state, denoted as dynamic frames. Identify the position of the background object in the dynamic frames, and establish the movement range of the background object based on the positions of the background object in all the dynamic frames. Establish a background model based on the names, regions, images, and movement ranges of all the background objects in the identified constant-static regions.

[0037] Please refer to the attached Figure 4 , Exemplarily, the grassland, house, and dirt road in the figure are all image regions that are in a static state in most frames, and background objects can be obtained through identification. The flower beds on both sides in front of the house in the figure also belong to image regions that are in a static state in most frames, but in the case of wind, the flower beds will move. By reading all the frames of the video data when the flower beds are in a non-static state, the movement range of the flower beds is established. Thus, in subsequent identification, when the flower beds move, they will not be misidentified as foreground targets.

[0038] Based on the established background model, foreground targets are extracted from the images included in the video data. These foreground targets represent the objects other than the background in the video, and the moving foreground target 31 is identified from these objects. According to the image of the range of the moving foreground target 31, with the help of an image recognition model, the type of the intrusion object is obtained. Exemplarily, pedestrians, animals, and birds appearing in the figure are basically in a moving state and can be regarded as the moving foreground target 31. According to the image region of the moving foreground target 31 in any frame, the type of the intrusion object can be obtained through image recognition. The trophy in the figure is only in a moving state in some frames and is in a static state in a large number of subsequent frames. Therefore, although it is identified as a foreground target, it will not be identified as the moving foreground target 31.

[0039] When enough data is collected during the day, with the help of the detection values of other sensors synchronously recorded when the intrusion object crosses the perimeter, and the events or behaviors identified according to the detection values, a multi-modal feature data set can be established.

[0040] , Exemplarily, when a person or an animal is identified by the video data as having invaded the perimeter, an infrared sensor can detect the heat emitted by the human body or the animal and is still effective at night or under low visibility conditions. If a person or an animal climbs over the fence and enters the perimeter, a vibration sensor can detect the resulting ground or wall vibration. Abnormal sounds such as footsteps, voices, or other noises can be captured by a sound sensor, which is very helpful for distinguishing human activities from other types of interference. Fiber optic sensors installed on the fence or buried underground can detect minute deformations and thus can detect events of invading the perimeter. Photoelectric beam detectors are usually used in pairs, one emitting a beam and the other receiving it. If an object blocks the beam, such as an intruder passing through, an alarm will be triggered.

[0041] For example, when birds are identified by video data as invading the perimeter, due to the particularity of birds, vibration sensors, fiber optic sensors, and photoelectric detection detectors may not be particularly effective in responding to bird intrusions. Infrared sensors can detect the heat emitted by birds. Sound sensors can detect the flapping of wings produced by flying birds. Occasionally, the sound of birds chirping can be detected. When birds just pass through the photoelectric detection detector, the photoelectric detection detector can detect it. When birds do not chirp or pass through the photoelectric detection detector, they will be detected by video data and infrared detection data during the day and by infrared detection data at night. Objects that are accidentally thrown into the perimeter may be detected by video data, vibration sensor data, and sound sensor data.

[0042] When the collected data is sufficient, it can cover enough intrusion object types and can also identify events or behaviors. Specifically, the method of identifying events or behaviors based on the detection values ​​includes: When the intruder crosses the perimeter, the detection values ​​of the multi-source sensor data are synchronized and aligned; According to the preset event recognition model corresponding to each sensor, the event of the detection value generated by each sensor is respectively recognized; Arrange all events according to the timeline to form an event sequence; randomly deleting one or more events in the event sequence multiple times to obtain multiple adjusted event sequences; Obtaining identified behaviors according to the responses of the pre-built behavior recognition model to the event sequence and the adjustment event sequence; Events that have no effect on the identification of the behavior are obtained and recorded as separate events; All behaviors and individual events are identified as a result of the detection value identifying the event or behavior.

[0043] In this embodiment, each sensor corresponds to an event recognition model. For example, the vibration sensor data obtained by the vibration sensor installed on the fence can identify events such as the fence being hit, the fence being climbed, and the fence being leaned on based on the vibration frequency, amplitude, number of times, duration, and other characteristics. The vibration sensor data obtained by the vibration sensor installed on the low fence at the edge of the garden can identify events such as the fence being hit, the fence being crossed, the fence being leaned on, and the fence being stepped on based on the vibration frequency, amplitude, number of times, duration, and other characteristics.

[0044] The optical fiber sensor corresponding to the optical fiber sensor data set underground can detect trampling events and ground vibration events.

[0045] The vibration sensor data obtained by vibration sensors installed on the perimeter wall and low fence, and the corresponding event recognition model can be obtained by using a machine learning model and training it with sample data manually labeled with events. Or other publicly disclosed technologies in the field can be used.

[0046] For sensors corresponding to other types of multi-source sensor data, a machine learning model trained with sample data after manually labeling events is also used to obtain them. Or the model establishment technologies publicly disclosed in the field can be used.

[0047] Exemplarily, when there is a certain amount of multi-source sensor data when pedestrians identified by video data invade the perimeter set on the grass, for each sensor other than the video surveillance camera, the events generated by the detected values are respectively identified, and all the events are sorted according to the time axis. The possible event sequences formed may be: Event sequence 1: {Infrared detection data: heat source passes through [event], vibration sensor data: no [event], sound sensor data: no [event], fiber optic sensor data: passes through [event], optoelectronic pair detector data: occlusion [event]}; Event sequence 2: {Infrared detection data: heat source passes through [event], vibration sensor data: passes through [event], sound sensor data: no [event], fiber optic sensor data: passes through [event], optoelectronic pair detector data: no [event]}; Event sequence 3: {Infrared detection data: heat source passes through [event], vibration sensor data: no [event], sound sensor data: voice [event], fiber optic sensor data: passes through [event], optoelectronic pair detector data: occlusion [event]}.

[0048] The input of the behavior recognition model is the event sequence, and the output is the behavior. Inputting the aforementioned event sequences 1-3 into the behavior recognition model can all obtain the behavior of pedestrian invasion. By deleting one or more events from event sequence 1, event sequence 1 - adjusted sequence 1 is obtained: Event sequence 1 - adjusted sequence 1: {Infrared detection data: no [event], vibration sensor data: no [event], sound sensor data: no [event], fiber optic sensor data: passes through [event], optoelectronic pair detector data: occlusion [event]}. Then when event sequence 1 - adjusted sequence 1 is input into the behavior recognition model, the behavior recognition model will output a result of no behavior. That is, the same behavior can correspond to multiple event sequences. Some event sequences may not correspond to a behavior, that is, the behavior recognition model outputs no behavior.

[0049] The establishment of the behavior recognition model also uses the establishment of a machine learning model and is trained with sample data formed by event sequences manually labeled with behaviors and adjusted event sequences.

[0050] Based on event sequences 1-3, the individual events that can be obtained are: photoelectric pair detector data: occlusion [event], vibration sensor data: pass [event], and sound sensor data: voice [event].

[0051] Step S3) Construct a multi-modal feature data set according to the events and behaviors.

[0052] Among them, the method for constructing a multi-modal feature data set according to the events and behaviors includes: Read all event sequences corresponding to the same behavior and adjust the event sequences; Obtain the intersection of several of the event sequences; Use the intersection as a multi-modal feature data.

[0053] Through three event sequences corresponding to the intrusion of a pedestrian into the perimeter identified from video data as well, after taking the intersection, a multi-modal feature data obtained is: pedestrian intrusion into the perimeter behavior - {infrared detection data: heat source body passes [event], vibration sensor data: none [event], sound sensor data: none [event], fiber optic sensor data: passes [event], photoelectric pair detector data: none [event]}. It means that when a pedestrian intrudes, the infrared detection data and fiber optic sensor data must be reflected, and the other multi-source sensor data can remain unchanged.

[0054] Because pedestrians are relatively tall, they may not trigger changes in the photoelectric pair detector data. Similarly, when an animal intrusion into the perimeter is identified from video data, a corresponding multi-modal feature data is: animal intrusion into the perimeter behavior - {infrared detection data: heat source body passes [event], vibration sensor data: none [event], sound sensor data: none [event], fiber optic sensor data: none [event], photoelectric pair detector data: occlusion [event]}. Since the weight of some animals is relatively light, they may not trigger changes in the fiber optic sensor data, but will necessarily trigger changes in the infrared detection data and photoelectric pair detector data.

[0055] Similarly, when a bird intrusion into the perimeter is identified from video data, a corresponding multi-modal feature data is: {infrared detection data: heat source body passes [event], vibration sensor data: none [event], sound sensor data: wing flapping sound [event], fiber optic sensor data: none [event], photoelectric pair detector data: none [event]}. The behavior associated with this multi-modal feature data is: bird intrusion into the perimeter.

[0056] Step S4) Generate a reference baseline for perimeter intrusion recognition based on the multi-modal feature data set.

[0057] The method for generating a reference baseline for perimeter intrusion recognition based on the multi-modal feature data set includes: Associate the multi-modal feature data with the corresponding behavior as a reference point; Obtain a reference baseline based on all the reference points corresponding to the multi-modal feature data set; A method for comparing the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times and correcting the reference baseline according to the comparison results includes: Obtain the detection values generated by each sensor of the multi-source sensor data; According to the preset event recognition model corresponding to each type of sensor, respectively identify the events of the detection values generated by each sensor, and then obtain an event sequence and an adjusted event sequence; Identify the behavior of the event sequence and the adjusted event sequence; Compare with the reference baseline to obtain all the reference points corresponding to the same behavior, and obtain the multi-modal feature data of the reference points; Obtain the intersection of all the event sequences and adjusted event sequences corresponding to the same behavior, and use the intersection as an updated multi-modal feature data; Correct the reference baseline according to all the updated multi-modal feature data.

[0058] Exemplarily, please refer to the appendix Figure 5 , multi-modal feature data: {Infrared detection data: heat source passes through [event], vibration sensor data: no [event], sound sensor data: no [event], fiber optic sensor data: passes through [event], optoelectronic pair detector data: no [event]}, associated behavior: pedestrian intrudes into the perimeter, which is a reference point. Multi-modal feature data: {Infrared detection data: heat source passes through [event], vibration sensor data: no [event], sound sensor data: no [event], fiber optic sensor data: no [event], optoelectronic pair detector data: occlusion [event]}, associated behavior: animal intrudes into the perimeter, which is another reference point. Multi-modal feature data: {Infrared detection data: heat source passes through [event], vibration sensor data: no [event], sound sensor data: no [event], fiber optic sensor data: no [event], optoelectronic pair detector data: no [event]}, associated behavior: bird intrudes into the perimeter, which is yet another reference point.

[0059] Step S5) Compare the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times and correct the reference baseline according to the comparison results.

[0060] Exemplarily, when the photoelectric pair-beam detector fails and always fails to recognize the occlusion event generated when an animal invades the perimeter, the video surveillance camera can still recognize the behavior of the animal invasion at this time. In the corresponding multi-modal feature data generated, the data of the photoelectric pair-beam detector: none [event]. After a period of time, the reference point corresponding to the animal invading the perimeter will be updated to {infrared detection data: heat source body passing through [event], vibration sensor data: none [event], sound sensor data: none [event], fiber optic sensor data: none [event], photoelectric pair-beam detector data: none [event]}. That is, the requirement for the photoelectric pair-beam detector data to generate an event is deleted.

[0061] Step S6) Identify the perimeter intrusion based on the corrected reference baseline, obtain the type of the intrusion object, and generate an intrusion detection result according to the type of the intrusion object.

[0062] The method for identifying the perimeter intrusion based on the corrected reference baseline, obtaining the type of the intrusion object, and generating an intrusion detection result includes: Read the multi-source sensor data for a preset duration, and obtain the event corresponding to each sensor in the multi-source sensor data according to the preset event recognition model corresponding to each sensor. Sort all the events according to the time axis to form an event sequence, and then obtain multiple adjusted event sequences. Compare the event sequence and the multiple adjusted event sequences with the reference points in the reference baseline respectively to obtain the matching reference points. Obtain the type of the intrusion object according to the reference points. Compare the object type with the preset exclusion object type table. When the object type does not belong to the exclusion object type table, generate an intrusion detection result that there is an intrusion and issue an intrusion alarm.

[0063] Exemplarily, when a bird invades the perimeter set on the grassland, the generated event sequence is {Infrared detection data: heat source passes through [event], Vibration sensor data: none [event], Sound sensor data: wing flapping sound [event], Fiber optic sensor data: none [event], Photoelectric pair detector data: occlusion [event]}. When directly using the event sequence to match the reference points in the reference baseline, a fuzzy matching method needs to be adopted to obtain the matching events, but the accuracy during fuzzy matching is relatively low. If the event sequence is adjusted, the possible generated adjusted event sequences may be: {Infrared detection data: none [event], Vibration sensor data: none [event], Sound sensor data: wing flapping sound [event], Fiber optic sensor data: none [event], Photoelectric pair detector data: none [event]}, and {Infrared detection data: heat source passes through [event], Vibration sensor data: none [event], Sound sensor data: wing flapping sound [event], Fiber optic sensor data: none [event], Photoelectric pair detector data: none [event]}. Then the second adjusted event sequence can match the result of the bird invading the perimeter behavior in an exact matching manner. Therefore, comparing the event sequence and multiple adjusted event sequences with the reference points in the reference baseline respectively has higher accuracy.

[0064] When the result of the bird invading the perimeter behavior is recognized, according to the comparison with the exclusion object type table, obviously the bird is in the exclusion object type table. Therefore, finally, the intrusion detection result will be no intrusion, and no intrusion alarm will be issued. When the result of the pedestrian invading the perimeter behavior is recognized, the intrusion detection result will be intrusion exists, and an intrusion alarm will be issued. Thus, the accuracy of perimeter intrusion recognition can be improved, and the false alarm rate can be reduced.

[0065] On the other hand, please refer to the appendix Figure 6 , The intelligent perimeter security system with multi-data fusion includes: An acquisition module 100 that acquires multi-source sensor data, where the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and photoelectric pair detector data; An identification module 200 that identifies the type of intrusion object crossing the perimeter through video data, and synchronously records the detection values of other sensors when the intrusion object crosses the perimeter, and identifies events or behaviors according to the detection values; A data set module 300 that constructs a multi-modal feature data set according to the events and behaviors; A baseline module 400 that generates a reference baseline for perimeter intrusion recognition based on the multi-modal feature data set; A correction module 500 that compares the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times, and corrects the reference baseline according to the comparison results; The detection module 600 identifies perimeter intrusion based on the corrected reference baseline, obtains the type of the intrusion object, and generates an intrusion detection result according to the type of the intrusion object.

[0066] Please refer to Figure 7 the schematic structural diagram of an electronic device provided by the embodiments of the present specification shown below.

[0067] As Figure 7 shown, the electronic device 1100 may include: at least one processor 1101, at least one network interface 1104, a user interface 1103, a memory 1105, and at least one communication bus 1102. Among them, the communication bus 1102 can be used to realize the connection and communication of the above-mentioned various components. Among them, the user interface 1103 may include buttons, and the optional user interface may further include a standard wired interface and a wireless interface. Among them, the network interface 1104 may include, but is not limited to, a Bluetooth module, an NFC module, a Wi-Fi module, etc. Among them, the processor 1101 may include one or more processing cores. The processor 1101 connects various parts within the entire electronic device 1100 through various interfaces and lines, runs or executes instructions, programs, code sets or instruction sets stored in the memory 1105, and calls the data stored in the memory 1105 to execute various functions of the routing device 1100 and process data. Optionally, the processor 1101 may be implemented in at least one hardware form of DSP, FPGA, or PLA. The processor 1101 may integrate one or several combinations of a CPU, a GPU, and a modem, etc. Among them, the CPU mainly processes the operating system, the user interface, and application programs, etc.; the GPU is responsible for rendering and drawing the content to be displayed on the display screen; the modem is used to process wireless communication.

[0068] It can be understood that the above-mentioned modem may also not be integrated into the processor 1101 and is implemented separately by a chip.

[0069] Among them, the memory 1105 may include a RAM or a ROM. Optionally, the memory 1105 includes a non-transitory computer-readable medium. The memory 1105 can be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1105 may include a program storage area and a data storage area. Among them, the program storage area may store instructions for implementing the operating system, instructions for at least one function (such as touch function, sound playback function, image playback function, etc.), instructions for implementing the above-mentioned method embodiments, etc.; the data storage area may store the data involved in the above-mentioned method embodiments. Optionally, the memory 1105 may also be at least one storage device located far from the aforementioned processor 1101. The memory 1105, as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an application program. The processor 1101 may be used to call the application program stored in the memory 1105 and execute the methods in the above-mentioned multiple embodiments.

[0070] An embodiment of this specification also provides a computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are run on a computer or a processor, the computer or the processor is caused to execute multiple steps in the above-mentioned embodiments. If each component module of the above-mentioned electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in the computer-readable storage medium.

[0071] An embodiment of this specification also provides a computer program product, including a computer program. When the computer program is executed by a processor, multiple steps in the above-mentioned embodiments are implemented.

[0072] Without conflict, the technical features in this embodiment and the implementation solutions can be combined arbitrarily.

[0073] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes a plurality of computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this specification are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, Digital Subscriber Line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes a plurality of available media integrated. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a Digital Versatile Disc (DVD)), or a semiconductor medium (for example, a Solid State Disk (SSD)), etc.

[0074] When implemented by hardware or firmware, the foregoing method flow is programmed into a hardware circuit to obtain a corresponding hardware circuit structure and implement the corresponding function. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit, and its logical function is determined by the user programming the device. A designer can program by himself to "integrate" a digital system on a PLD, without having to ask a chip manufacturer to design and manufacture a dedicated integrated circuit chip. Moreover, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL), and there is not only one kind of HDL, but many kinds. Those skilled in the art should also be clear that only by slightly logically programming the method flow with the above several hardware description languages and programming it into an integrated circuit can the hardware circuit implementing the logical method flow be easily obtained.

[0075] The embodiments described above are only described in terms of the preferred embodiments of this specification, and do not limit the scope of this specification. Without departing from the design spirit of this specification, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of this specification shall fall within the protection scope determined by the claims of this specification.

Claims

1. An intelligent perimeter security method for multi-data fusion, characterized in that Including the steps: Obtain multi-source sensor data, where the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and optoelectronic pair-beam detector data; Identify the type of intrusion object crossing the perimeter through the video data, and simultaneously record the detection values of other sensors when the intrusion object crosses the perimeter, and identify events or behaviors based on the detection values; Construct a multi-modal feature dataset according to the events and behaviors; Generate a reference baseline for perimeter intrusion recognition based on the multi-modal feature dataset; Compare the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times, and correct the reference baseline according to the comparison results; Identify perimeter intrusion based on the corrected reference baseline, obtain the type of intrusion object, and generate an intrusion detection result according to the type of intrusion object.

2. The intelligent perimeter security method for multi-data fusion according to claim 1, characterized in that: The method for identifying the type of intrusion object crossing the perimeter through video data includes: Read the video data of the perimeter monitoring area, identify the background and establish a background model; Extract the foreground target from the images included in the video data according to the background model; Use a motion detection algorithm to identify the moving foreground targets in the video frame and generate corresponding moving trajectories; Perform image segmentation on the moving foreground targets based on the moving trajectories, and extract the image regions of the moving foreground targets; Identify the type of intrusion object according to the image regions of the moving foreground targets.

3. The intelligent perimeter security method for multi-data fusion according to claim 1 or 2, characterized in that: The method for identifying events or behaviors based on the detection values includes: Synchronize the detection values of the multi-source sensor data in time and align the data when the intrusion object crosses the perimeter; Identify the events of the detection values generated by each sensor respectively according to the preset event recognition model corresponding to each sensor; Sort all the events according to the time axis to form an event sequence; Randomly delete one or more events in the event sequence multiple times to obtain multiple adjusted event sequences; Obtain the recognized behaviors according to the responses of the event sequence and the adjusted event sequences to the pre-constructed behavior recognition model; Obtain the events that have no influence on the recognition of the behaviors, and record them as individual events; Take all the recognized behaviors and individual events as the results of the events or behaviors recognized by the detection values.

4. The intelligent perimeter security method for multi-data fusion according to claim 3, characterized in that: The method for constructing a multi-modal feature dataset according to the events and behaviors includes: Read all the event sequences and adjusted event sequences corresponding to the same behavior; Obtain the intersections of several of the event sequences; Take the intersections as a multi-modal feature data.

5. The intelligent perimeter security method for multi-data fusion according to claim 4, characterized in that: The method for generating a reference baseline for perimeter intrusion recognition based on the multi-modal feature dataset includes: Associate the multi-modal feature data with the corresponding behaviors as a reference point; Obtain a reference baseline based on all the reference points corresponding to the multi-modal feature dataset; The method of comparing the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times and correcting the reference baseline according to the comparison results includes: Obtain the detection values generated by each sensor of the multi-source sensor data; According to the preset event recognition model corresponding to each sensor, respectively identify the events of the detection values generated by each sensor, and then obtain an event sequence and an adjusted event sequence; Identify the behaviors of the event sequence and the adjusted event sequence; Compare with the reference baseline to obtain all the reference points corresponding to the same behavior, and obtain the multi-modal feature data of the reference points; Obtain the intersection of all the event sequences and the adjusted event sequences corresponding to the same behavior, and use the intersection as an updated multi-modal feature data; Correct the reference baseline according to all the updated multi-modal feature data.

6. The intelligent perimeter security method for multi-data fusion according to claim 3, characterized in that The method of identifying perimeter intrusion based on the corrected reference baseline, obtaining the type of the intrusion object, and generating an intrusion detection result according to the type of the intrusion object includes: Read the multi-source sensor data for a preset duration, and according to the preset event recognition model corresponding to each sensor, obtain the events corresponding to each sensor in the multi-source sensor data; Sort all the events according to the time axis to form an event sequence, and then obtain a plurality of adjusted event sequences; Compare the event sequence and the plurality of adjusted event sequences with the reference points in the reference baseline respectively to obtain the matching reference points; Obtain the type of the intrusion object according to the reference points; Compare the object type with the preset exclusion object type table. When the object type does not belong to the exclusion object type table, generate an intrusion detection result that there is an intrusion and issue an intrusion alarm.

7. The intelligent perimeter security system with multi-data fusion is characterized in that, Includes: An acquisition module that acquires multi-source sensor data, where the multi-source sensor data includes video data, infrared detection data, vibration sensor data, sound sensor data, fiber optic sensor data, and optoelectronic pair detector data; An identification module that identifies the type of intrusion object crossing the perimeter through video data, and synchronously records the detection values of other sensors when the intrusion object crosses the perimeter, and identifies events or behaviors according to the detection values; A dataset module that constructs a multi-modal feature dataset according to the events and behaviors; A baseline module that generates a reference baseline for perimeter intrusion identification based on the multi-modal feature dataset; A correction module that compares the multi-source sensor data of the same intrusion object type collected subsequently with the reference baseline multiple times and corrects the reference baseline according to the comparison results; A detection module that identifies perimeter intrusion based on the corrected reference baseline, obtains the type of the intrusion object, and generates an intrusion detection result according to the type of the intrusion object.

8. An electronic device, characterized in that, Includes a processor and a memory; The processor is connected to the memory; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to execute the method according to any one of claims 1-6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1-6 is implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the method according to any one of claims 1-6 is implemented.

Images