Perimeter intrusion identification method and system based on video joint acquisition

Abstract

The present invention provides a perimeter intrusion identification method and system based on video joint acquisition, which relates to the field of data processing technology. The method comprises: collecting environmental brightness information of a first monitoring area through a photoreceptor, independently identifying a first infrared imaging set and a first monitoring image set, obtaining the distribution coordinates of the intrusion object, and activating an infrared pan-tilt imager array or a binocular pan-tilt camera array in combination with the environmental brightness information to lock the intrusion object for joint tracking, obtaining a moving coordinate sequence of the intrusion object, executing training of an intrusion behavior identification channel, generating an intrusion behavior trigger probability, and when the intrusion behavior trigger probability meets a trigger probability threshold, sending the intrusion behavior identification result and the real-time coordinates of the intrusion object to a perimeter intrusion management terminal, thereby solving the technical problem that it is difficult to realize multi-camera joint evaluation in traditional perimeter intrusion identification, resulting in poor identification accuracy, and realizing the improvement of the perimeter intrusion identification rate based on the joint acquisition of visible light and infrared video.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a perimeter intrusion recognition method and system based on video joint acquisition. Background Art

[0002] With the development of video recognition technology, especially the development of perimeter intrusion recognition, perimeter intrusion detection usually refers to monitoring a linear boundary and generating an alarm when an illegal act of crossing the boundary is found. Perimeter intrusion detection is subject to various conditions such as terrain, topography, climate, and meteorology, so the detection requirements are high and the difficulty is also great. The existing perimeter intrusion detection products are few in variety and have little room for choice. In addition, traditional perimeter intrusion recognition is difficult to achieve multi-camera joint evaluation, resulting in technical problems such as poor recognition accuracy. Summary of the invention

[0003] The present application provides a perimeter intrusion recognition method and system based on video joint acquisition, which is used to solve the technical problem that traditional perimeter intrusion recognition in the prior art is difficult to achieve multi-camera joint evaluation, resulting in poor recognition accuracy.

[0004] In view of the above problems, the present application provides a perimeter intrusion identification method and system based on video joint acquisition.

[0005] In a first aspect, the present application provides a perimeter intrusion identification method based on video joint acquisition, the method comprising: collecting ambient brightness information of a first monitoring area through a photoreceptor; when the ambient brightness information is less than or equal to a first brightness threshold, activating an infrared pan-tilt imager array for monitoring to obtain a first infrared imaging set; when the ambient brightness information is greater than the first brightness threshold, activating a binocular pan-tilt camera array for monitoring to obtain a first monitoring image set; independently identifying the first infrared imaging set and the first monitoring image set to obtain the distribution coordinates of the intrusion object; according to the distribution coordinates of the intrusion object, activating the infrared pan-tilt imager array or the binocular pan-tilt camera array in combination with the ambient brightness information to lock the intrusion object for joint tracking to obtain a moving coordinate sequence of the intrusion object; according to the moving coordinate sequence of the intrusion object, performing training of the intrusion behavior recognition channel to generate an intrusion behavior trigger probability; when the intrusion behavior trigger probability meets the trigger probability threshold, activating the sound and light alarm device to alarm, and at the same time sending the intrusion behavior recognition result and the real-time coordinates of the intrusion object to the perimeter intrusion management terminal.

[0006] In the second aspect, the present application provides a perimeter intrusion identification system based on video joint acquisition, and the system includes: one or more technical solutions provided in the present application, which have at least the following technical effects or advantages: an information acquisition module, which is used to collect environmental brightness information of a first monitoring area through a photoreceptor; a first monitoring module, which is used to activate the infrared pan-tilt imager array for monitoring when the environmental brightness information is less than or equal to a first brightness threshold, and obtain a first infrared imaging set; a second monitoring module, which is used to activate the binocular pan-tilt camera array for monitoring when the environmental brightness information is greater than the first brightness threshold, and obtain a first monitoring image set; an independent recognition module, which is used to identify the first infrared image. The image set is independently identified with the first monitoring image set to obtain the distribution coordinates of the intrusion object; a joint tracking module is used to activate the infrared pan-tilt imager array or the binocular pan-tilt camera array to lock the intrusion object for joint tracking according to the distribution coordinates of the intrusion object and in combination with the ambient brightness information to obtain a moving coordinate sequence of the intrusion object; a first training module is used to perform training of the intrusion behavior recognition channel according to the moving coordinate sequence of the intrusion object to generate an intrusion behavior trigger probability; an alarm module is used to activate the sound and light alarm device to alarm when the intrusion behavior trigger probability meets the trigger probability threshold, and at the same time send the intrusion behavior recognition result and the real-time coordinates of the intrusion object to the perimeter intrusion management terminal.

[0007] The perimeter intrusion identification method and system based on video joint acquisition provided in the present application relate to the field of data processing technology, solve the technical problem that traditional perimeter intrusion identification in the prior art is difficult to achieve multi-camera joint evaluation, resulting in poor recognition accuracy, and realize the improvement of perimeter intrusion recognition rate based on visible light and infrared video joint acquisition. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1 A schematic diagram of the process of perimeter intrusion identification method based on video joint acquisition is provided for this application;

[0009] Figure 2 A schematic diagram of the distribution coordinates of intrusion objects in a perimeter intrusion identification method based on video joint acquisition is provided for this application;

[0010] Figure 3 A structural schematic diagram of a perimeter intrusion identification system based on video joint acquisition is provided for this application.

[0011] Explanation of the accompanying drawings: information collection module 1, first monitoring module 2, second monitoring module 3, independent identification module 4, joint tracking module 5, first training module 6, alarm module 7. DETAILED DESCRIPTION

[0012] The present application provides a perimeter intrusion identification method and system based on video joint acquisition, which is used to solve the technical problem that the acquisition method for perimeter intrusion identification in the prior art is single, resulting in a low perimeter intrusion identification rate.

[0013] Embodiment 1

[0014] like Figure 1 As shown, the embodiment of the present application provides a perimeter intrusion identification method based on video joint acquisition, the method comprising:

[0015] Step A100: collecting ambient brightness information of the first monitoring area through a photoreceptor;

[0016] In the present application, the perimeter intrusion identification method based on video joint acquisition provided in the embodiment of the present application is applied to a perimeter intrusion identification system based on video joint acquisition. The perimeter intrusion identification system based on video joint acquisition is communicatively connected with a photoreceptor, and the photoreceptor is used to collect ambient brightness parameters.

[0017] Furthermore, in order to better identify perimeter intrusions, it is necessary to collect the ambient brightness in the first monitoring area through the photoreceptor connected to the system. The first monitoring area refers to the area within the perimeter that needs to be monitored. The ambient brightness information collected by the photoreceptor can be that when the photosensitive material in the photoreceptor is irradiated with light of appropriate wavelength, the current will increase with the increase of light intensity, thereby realizing photoelectric conversion, and obtaining ambient light intensity data, ambient light color data, ambient light direction and other data in the first monitoring area, and summarizing them and recording them as ambient brightness information for output, which serves as an important reference for the later recognition and collection of perimeter intrusion based on video.

[0018] Step A200: When the ambient brightness information is less than or equal to a first brightness threshold, the infrared pan-tilt imager array is activated for monitoring to obtain a first infrared imaging set;

[0019] In this application, in order to improve the accuracy of identifying intruders in the process of perimeter intrusion identification, it is necessary to compare and judge the ambient brightness information with the first brightness threshold based on the ambient brightness information collected above. The first brightness threshold is preset based on the average ambient brightness of the first monitoring area in the historical period. Further, when the ambient brightness information is less than or equal to the first brightness threshold, the infrared pan-tilt imager array is activated.

[0020] The infrared pan-tilt imager array is communicatively connected to a perimeter intrusion identification system based on video joint acquisition. The infrared pan-tilt imager array is obtained by arranging a plurality of infrared pan-tilt imagers at equal intervals in a first monitoring area, and the infrared pan-tilt imager array is used to monitor and collect infrared radiation energy distribution graphic parameters in the first monitoring area, thereby generating a first infrared imaging set based on the infrared pan-tilt imager, wherein the first infrared imaging set includes a plurality of infrared thermal images in the first monitoring area, thereby ensuring the recognition and acquisition of perimeter intrusion based on video joint acquisition.

[0021] Step A300: When the environment brightness information is greater than the first brightness threshold, the binocular pan-tilt camera array is activated for monitoring to obtain a first monitoring image set;

[0022] Furthermore, step A300 of the present application also includes:

[0023] Step A310: When the environment brightness information is greater than the first brightness threshold and greater than the second brightness threshold, activate the binocular pan-tilt camera array for monitoring to obtain a first monitoring image set;

[0024] Step A320: When the environment brightness information is greater than the first brightness threshold and less than or equal to the second brightness threshold, activate the laser fill light component, and perform monitoring in combination with the binocular pan-tilt camera array to obtain a first monitoring image set;

[0025] Furthermore, step A300 of the present application also includes:

[0026] Step A330: obtaining the coordinates of the first monitoring area of ​​the first binocular pan-tilt camera of the binocular pan-tilt camera array;

[0027] Step A340: extracting a plurality of vegetation coverage areas according to the coordinates of the first monitoring area, wherein any vegetation coverage area has a vegetation coverage width label, a vegetation coverage height label, and a vegetation coverage length label;

[0028] Step A350: constructing a vegetation coverage three-dimensional model according to the vegetation coverage width label, the vegetation coverage height label and the vegetation coverage length label;

[0029] Step A360: Based on the historical intrusion transaction set, the minimum intrusion object width, the minimum intrusion object height and the minimum intrusion object length are configured to build an intrusion object threshold model;

[0030] Step A370: Compare the intrusion object threshold model and the vegetation coverage three-dimensional model. When the vegetation coverage three-dimensional model can block the intrusion object threshold model, activate the first infrared pan-tilt imager of the first monitoring area coordinates for auxiliary monitoring, obtain an infrared monitoring auxiliary image, and add it to the first monitoring image set of the first monitoring area coordinates.

[0031] In the present application, in order to more accurately identify perimeter intrusions, it is necessary to activate the binocular pan-tilt camera array based on the deployed infrared pan-tilt imager array monitoring the first monitoring area to obtain the first infrared imaging set when the ambient brightness information is greater than the first brightness threshold. The binocular pan-tilt camera array is communicated with the perimeter intrusion identification system based on video joint acquisition, and the binocular pan-tilt camera array is used to collect image parameters of environmental intrusion objects.

[0032] Furthermore, when the ambient brightness information is greater than the first brightness threshold and also greater than the second brightness threshold, the binocular pan-tilt camera array is activated for monitoring. The second brightness threshold refers to the average of the highest brightness value of the infrared pan-tilt imager array and the lowest brightness of the binocular pan-tilt camera, and the second brightness threshold is greater than the first brightness threshold. At this time, it is deemed that the binocular pan-tilt camera array can be used to perform video intelligent recognition color imaging of the first monitoring area at the current brightness, and its visual effect is good, thereby compensating for the poor resolution of the thermal imaging image, and the obtained color imaging is summarized and recorded as the first monitoring image set. Furthermore, when the ambient brightness information is greater than the first brightness threshold and less than or equal to the second brightness threshold, the binocular pan-tilt camera array is activated for monitoring. The connected laser fill light component is activated, the fill light operation is performed on the first monitoring area through the laser fill light group, and the first monitoring area is monitored for intrusion in combination with the binocular pan-tilt camera array. The monitoring coordinates in the first monitoring area are marked while the intrusion is detected, which means that the first monitoring area coordinates corresponding to the first binocular pan-tilt camera in the binocular pan-tilt camera array are first marked, the first binocular pan-tilt camera is a camera arbitrarily selected and determined in the binocular pan-tilt camera array, the first monitoring area coordinates are generated by constructing the area coordinate axis according to the area monitored by the first binocular pan-tilt camera, and further, according to the first monitoring area coordinates, the first monitoring area having Extracting vegetation areas refers to storing and recording areas with static irregular objects in the first monitoring area, generating multiple vegetation coverage areas in the first monitoring area, and any vegetation coverage area has a vegetation coverage width label, a vegetation coverage height label and a vegetation coverage length label. The vegetation coverage width label is used to mark the widest distance of vegetation coverage, the vegetation coverage height label is used to mark the height of the vegetation in the vegetation coverage area from the horizontal plane, and the vegetation coverage length label is used to mark the longest distance of vegetation coverage. Furthermore, based on the vegetation coverage width label, the vegetation coverage height label and the vegetation coverage length label, the vegetation coverage width, the vegetation coverage height and the vegetation coverage length are coordinate-characterized by an iterative nearest point algorithm. The point alignment refers to iteratively finding the optimal point cloud transformation matrix between the width of vegetation coverage, the height of vegetation coverage and the length of vegetation coverage to minimize the distance between them, and constructing a three-dimensional vegetation coverage model based on the registered data. At the same time, it takes the historical invasion transaction set as the basis. The historical invasion transaction is the event that identifies the existence of intrusion objects in the first monitoring area within the historical period, and extracts the minimum width of the intrusion object, the minimum height of the intrusion object and the minimum length of the intrusion object in the historical invasion transaction set. The minimum width of the intrusion object refers to the data with the smallest width in the historical intrusion object, the minimum height of the intrusion object refers to the data with the smallest height in the historical intrusion object, and the minimum length of the intrusion object refers to the data with the smallest length in the historical intrusion object.On this basis, the width data, height data, and length data of the historical intrusion objects are aligned in the spatial coordinate system of the first monitoring area. The alignment process is the same as the above alignment, so as to complete the construction of the intrusion object threshold model. Finally, the coordinate feature points of the constructed intrusion object threshold model and the vegetation coverage three-dimensional model are compared. When the area demarcated by the connection of the vegetation feature points in the vegetation coverage three-dimensional model is larger than the area demarcated by the connection of the intrusion object feature points in the intrusion object threshold model, it is considered that the vegetation feature points in the vegetation coverage three-dimensional model can cover the intrusion object feature points in the intrusion object threshold model. At this time, the binocular pan-tilt camera array has a blind spot in the process of intrusion monitoring. It is necessary to activate the first infrared pan-tilt imager of the first monitoring area coordinates to assist the binocular pan-tilt camera array in intrusion monitoring, so as to obtain the infrared monitoring auxiliary image generated by the first infrared pan-tilt imager, and at the same time, add the infrared monitoring auxiliary image to the first monitoring image set of the first monitoring area coordinates to improve the monitoring accuracy of the binocular pan-tilt camera, and lay a solid foundation for the subsequent recognition of perimeter intrusion based on video joint. ,

[0033] Step A400: independently identifying the first infrared imaging set and the first monitoring image set to obtain distribution coordinates of intrusion objects;

[0034] Furthermore, if Figure 2 As shown, step A400 of the present application also includes:

[0035] Step A410: traversing the first infrared imaging set and the first monitoring image set, configuring a reference infrared imaging set and a reference monitoring image set, wherein the reference infrared imaging set and the reference monitoring image set are monitoring images without intrusion objects;

[0036] Step A420: Activate the intrusion object judgment channel, traverse the first infrared imaging set and the first monitoring image set, combine the reference infrared imaging set and the reference monitoring image set for identification, and obtain the intrusion object distribution coordinates.

[0037] Furthermore, step A420 of the present application includes:

[0038] Step A421: traverse the first infrared imaging set and the first monitoring image set to extract a first image to be identified and a first reference image;

[0039] Step A422: When the first image to be identified is an infrared image, the first feature extraction path of the infrared response node of the intrusion object judgment channel is activated to receive the first image to be identified, and the second feature extraction path of the infrared response node of the intrusion object judgment channel is activated to receive the first reference image for abnormal identification to obtain a morphological deviation area, wherein the infrared response node is a twin neural network;

[0040] Step A423: When the first image to be identified is a non-infrared image, the third feature extraction path of the conventional image response node of the intrusion object judgment channel is activated to receive the first image to be identified, and the fourth feature extraction path of the conventional image response node of the intrusion object judgment channel is activated to receive the first reference image for abnormal identification to obtain a morphological deviation area, wherein the conventional image response node is a twin neural network;

[0041] Step A424: Based on the PTZ coordinate system, the morphological deviation area is located to generate the distribution coordinates of the intrusion object.

[0042] In the present application, in order to better determine the distribution coordinates of intrusion objects within the perimeter, it is necessary to independently identify the first infrared imaging set obtained by monitoring through the infrared pan-tilt imager array and the first monitoring image set obtained by monitoring through the binocular pan-tilt camera array, which means that after traversing the infrared imaging images in the first infrared imaging set and the video images in the first monitoring image set in turn, the images when there are no intrusion objects are extracted and integrated to generate a benchmark infrared imaging set and a benchmark monitoring image set, that is, the benchmark infrared imaging set and the benchmark monitoring image set are both monitoring images without intrusion objects. Furthermore, after activating the intrusion object judgment channel, the first infrared imaging set is identified. Each image data node included in the external imaging set and the first monitoring image set is accessed in sequence, and at the same time, the reference infrared imaging set and the reference monitoring image set are combined for traversal identification. The process can be to traverse each image data included in the first infrared imaging set and the first monitoring image set, randomly extract an image from the image set where it is not determined whether there is an intruder as the first image to be identified, and randomly extract an image from the image set where it is determined that there is no intruder as the first reference image. When the first image to be identified is an infrared image, the first feature extraction path of the infrared response node in the intrusion object judgment channel is activated to receive the first image to be identified, and the infrared response node is activated to receive the first image to be identified. The response node is a data node for extracting image features from infrared images, and at the same time, the second feature extraction path of the infrared response node of the intrusion object judgment channel is activated to receive the first reference image to mark the first image to be identified as abnormal, and generate a morphological deviation area according to the abnormal mark, wherein the infrared response node is a twin neural network in the first monitoring area, and the construction process of the twin neural network can be that the twin neural network refers to a coupling framework established based on two artificial neural networks, and the twin neural network is further constructed, wherein the input data of the twin neural network includes the configuration parameters obtained by inputting the first selection data into the detection device selection unit to select the detection device, and the twin neural network includes an infrared response node corresponding to the infrared pan-tilt imager array and a conventional image response node corresponding to the binocular pan-tilt camera array, so that the first constructed data set is obtained through the infrared response node, and the second constructed data set is obtained through the conventional image response node, and the twin neural network is a neural network model in machine learning that can continuously perform self-iterative optimization, and the twin neural network is obtained through training data sets and supervision data sets, wherein each group of training data in the first constructed data set and the second constructed data set includes a training data set and a supervision data set corresponding to the configuration parameters, and the supervision data set is supervision data that corresponds one to one to the training data set.

[0043] Furthermore, the twin neural network construction process is as follows: each set of training data in the training data set is input into the twin neural network, and the output supervision adjustment of the twin neural network is performed through the supervision data corresponding to this set of training data. When the output result of the twin neural network is consistent with the supervision data, the training of the current group is completed, and when all the training data in the training data set are trained, the twin neural network training is completed.

[0044] In order to ensure the convergence and accuracy of the twin neural network, its convergence process can be that when the output data in the twin neural network converges to a point and approaches a certain value, it is considered convergence. Its accuracy can be tested by testing the twin neural network with a test data set. For example, the test accuracy can be set to 80%. When the test accuracy of the test data set meets 80%, the twin neural network is constructed.

[0045] Similarly, when the first image to be identified is a non-infrared image, the third feature extraction path of the conventional image response node of the intrusion object judgment channel is activated to receive the first image to be identified, and at the same time, the fourth feature extraction path of the conventional image response node of the intrusion object judgment channel is activated to receive the first reference image for abnormal identification, and a morphological deviation area is generated according to the abnormal identification in the image, wherein the conventional image response node is also obtained by the twin neural network calculation in the first monitoring area, so as to locate the morphological deviation area in the gimbal coordinate system, and the gimbal coordinate system is based on the characteristic coordinate points that overlap in the infrared gimbal imager array and the binocular gimbal camera array in the first monitoring area as a reference, so as to generate the intrusion object distribution coordinates according to the alignment coordinates, and the intrusion object distribution coordinates are all in the gimbal coordinate system, which has a limited effect on realizing the identification of perimeter intrusion based on video joint.

[0046] Step A500: according to the distribution coordinates of the intrusion object and in combination with the ambient brightness information, the infrared pan-tilt imager array or the binocular pan-tilt camera array is activated to lock the intrusion object for joint tracking, and a moving coordinate sequence of the intrusion object is obtained;

[0047] Furthermore, step A500 of the present application also includes:

[0048] Step A510: Based on the distribution coordinates of the intruding object, the intruding object is locked on the first monitoring device for tracking and monitoring, and a moving coordinate sequence of the intruding object in the first area is obtained, wherein the first monitoring device is an infrared pan-tilt imager and / or a binocular pan-tilt camera;

[0049] Step A520: When the intruding object leaves the first area and enters the second area, the second monitoring device is activated to lock the intruding object for tracking and monitoring, and a moving coordinate sequence of the intruding object in the second area is obtained;

[0050] Step A530: According to the monitoring sequence, the first region intrusion object movement coordinate sequence, the second region intrusion object movement coordinate sequence, and the Kth region intrusion object movement coordinate sequence are jointly stored to obtain the intrusion object movement coordinate sequence.

[0051] In the present application, the above-obtained distribution coordinates of the intruding object are used as reference data, combined with the ambient brightness information to activate the infrared pan-tilt imager array and / or the binocular pan-tilt camera array, and then the intruding object is locked and jointly tracked, which means that on the basis of the distribution coordinates of the intruding object, the intruding object is locked by the first monitoring device for tracking and monitoring. The first monitoring device can be any one of the infrared pan-tilt imager and binocular pan-tilt camera, and the intruding object is dynamically and continuously monitored. According to the monitoring coordinate points of the intruding object in each monitoring device, a moving coordinate sequence of the intruding object in the first area is generated. Furthermore, when the intruding object leaves the first area and enters the second area, the first area and the second area are adjacent and connected but different equal block areas, thereby activating the second area. The monitoring device locks on the intrusion object for tracking and monitoring, which means that the coordinate points of the intrusion object entering the second area are locked by the second monitoring device to generate a moving coordinate sequence of the intrusion object in the second area. Furthermore, according to the monitoring time sequence of the intrusion object, the moving coordinate sequence of the intrusion object in the first area, the moving coordinate sequence of the intrusion object in the second area, and the moving coordinate sequence of the intrusion object in the Kth area are jointly stored, which means that the moving coordinate sequence of the intrusion object in the first area, the moving coordinate sequence of the intrusion object in the second area, and the moving coordinate sequence of the intrusion object in the Kth area are sorted according to the monitoring time sequence and then stored in the system in bit order, and the moving coordinate sequence of the intrusion object is generated according to the stored data, so as to serve as reference data for the subsequent identification of perimeter intrusion based on video.

[0052] Step A600: performing training of an intrusion behavior recognition channel according to the intrusion object movement coordinate sequence to generate an intrusion behavior trigger probability;

[0053] Furthermore, step A600 of the present application also includes:

[0054] Step A610: configuring an intrusion behavior mobile sensitive path;

[0055] Step A620: collecting an intrusion transaction object movement coordinate sequence data set and a regular object movement coordinate sequence data set, mixing them, and generating an intrusion behavior recognition training data set;

[0056] Step A630: Configure the intrusion behavior trigger probability evaluation function:

[0057] Y i =(y1,y2,…y l ,…,y Q );

[0058]

[0059] Among them, P(X0) represents the intrusion behavior triggering probability of the moving coordinate sequence of the intrusion object, Y i The moving coordinate sequence that represents the sensitive moving path of the i-th intrusion behavior, y l The lth coordinate of the mobile coordinate sequence representing the mobile sensitive path of the i-th intrusion behavior, X0 represents the mobile coordinate sequence of the intrusion object, Q represents the number of coordinates of the mobile sensitive path of the i-th intrusion behavior, and M represents the number of mobile sensitive paths of the intrusion behavior;

[0060] Step A640: Based on the intrusion behavior trigger probability evaluation function and the intrusion behavior mobile sensitive path, construct an intrusion behavior evaluation rule, perform unsupervised training on the intrusion behavior identification training data set, generate the intrusion behavior identification channel, analyze the intrusion object movement coordinate sequence, and generate the intrusion behavior trigger probability.

[0061] In the present application, in order to improve the accuracy of perimeter intrusion identification, it is necessary to use the intrusion object movement coordinate sequence as the basis to perform training on the intrusion behavior identification channel, which means first configuring the intrusion behavior movement sensitive path, which refers to recording the abnormal intrusion path in the first monitoring area. The path may include a climbing path, a path over a railing, etc. Further, the intrusion transaction object movement coordinate sequence data set and the conventional object movement coordinate sequence data set are collected and mixed. The intrusion transaction object movement coordinate sequence data set refers to a data set recording the dynamic coordinate points identified as intrusion objects in the first monitoring area, and the conventional object movement coordinate sequence data set refers to a data set recording the dynamic coordinate points of mobile objects identified as normal state in the first monitoring area. On this basis, an intrusion behavior identification training data set is generated. Further, an intrusion behavior trigger probability evaluation function and an intrusion behavior movement sensitive path are used to construct an intrusion behavior evaluation rule. The configured intrusion behavior trigger probability evaluation function is as follows:

[0062] Y i =(y1,y2,…y l ,…,y Q );

[0063]

[0064] Among them, P(X0) represents the intrusion behavior triggering probability of the moving coordinate sequence of the intrusion object, Y i The moving coordinate sequence that represents the sensitive moving path of the i-th intrusion behavior, y lThe lth coordinate of the mobile coordinate sequence representing the mobile sensitive path of the i-th intrusion behavior, X0 represents the mobile coordinate sequence of the intrusion object, Q represents the number of coordinates of the mobile sensitive path of the i-th intrusion behavior, and M represents the number of mobile sensitive paths of the intrusion behavior;

[0065] The intrusion level of the behavior of the intrusion object in the first monitoring area is evaluated by the moving coordinate sequence of the intrusion object and the moving coordinate sequence of the sensitive path of the intrusion behavior. The intrusion level evaluation is based on the degree of invasion impact of the intrusion object in the invasion area. The higher the evaluation level, the higher the invasion impact. The intrusion behavior trigger probability of the moving coordinate sequence of the intrusion object is obtained, and at the same time, the intrusion behavior recognition training data set is unsupervisedly trained, which means that the intrusion behavior recognition training data set is automatically clustered into several categories, and the categories with large similarity are clustered together to complete the division of the intrusion behavior recognition training data set, thereby generating an intrusion behavior recognition channel, and analyzing the movement behavior of the intrusion object on the moving coordinate sequence of the intrusion object in the intrusion behavior recognition channel to generate the intrusion behavior trigger probability, thereby improving the accuracy of the subsequent recognition of perimeter intrusion based on video joint.

[0066] Step A700: When the intrusion behavior trigger probability meets the trigger probability threshold, the sound and light alarm device is activated to alarm, and the intrusion behavior identification result and the real-time coordinates of the intrusion object are sent to the perimeter intrusion management terminal.

[0067] In the present application, the trigger probability of the intrusion behavior is first judged, which means comparing the trigger probability of the intrusion behavior with the trigger probability threshold. The trigger probability threshold is set according to the historical maximum trigger probability of the intrusion trigger in the first monitoring area. When the trigger probability of the intrusion behavior meets the trigger probability threshold, it is considered that the current intrusion behavior has the greatest impact on the first monitoring area. Therefore, it is necessary to activate the sound and light alarm device, and use the sound and light alarm device to perform a combined light alarm and sound alarm on the first monitoring area. At the same time, the intrusion behavior recognition result of the intrusion object corresponding to the current intrusion behavior and the real-time coordinates of the intrusion object are determined in the first monitoring area, and the intrusion behavior recognition result and the real-time coordinates of the intrusion object are integrated and sent to the perimeter intrusion management terminal to notify the control personnel corresponding to the first monitoring area to perform intrusion control on the intrusion object, so as to ensure better identification of perimeter intrusion based on video in the later stage.

[0068] In summary, the perimeter intrusion identification method based on video joint acquisition provided in the embodiment of the present application includes at least the following technical effects: improving the perimeter intrusion identification rate based on the joint acquisition of visible light and infrared videos.

[0069] Embodiment 2

[0070] Based on the same inventive concept as the perimeter intrusion identification method based on video joint acquisition in the aforementioned embodiment, Figure 3 As shown, the present application provides a perimeter intrusion identification system based on video joint acquisition, the system comprising:

[0071] An information collection module 1, the information collection module 1 is used to collect the ambient brightness information of the first monitoring area through a photoreceptor;

[0072] A first monitoring module 2, wherein the first monitoring module 2 is used to activate the infrared pan-tilt imager array for monitoring when the ambient brightness information is less than or equal to a first brightness threshold, and obtain a first infrared imaging set;

[0073] A second monitoring module 3, wherein the second monitoring module 3 is used to activate the binocular pan-tilt camera array for monitoring when the environmental brightness information is greater than the first brightness threshold, and obtain a first monitoring image set;

[0074] An independent identification module 4, the independent identification module 4 is used to independently identify the first infrared imaging set and the first monitoring image set to obtain the distribution coordinates of the intrusion object;

[0075] A joint tracking module 5 is used to activate the infrared pan-tilt imager array or the binocular pan-tilt camera array to lock the intrusion object for joint tracking according to the distribution coordinates of the intrusion object and in combination with the ambient brightness information, so as to obtain a moving coordinate sequence of the intrusion object;

[0076] A first training module 6, the first training module 6 is used to perform training of an intrusion behavior recognition channel according to the intrusion object movement coordinate sequence, and generate an intrusion behavior trigger probability;

[0077] The alarm module 7 is used to activate the sound and light alarm device to alarm when the intrusion behavior trigger probability meets the trigger probability threshold, and at the same time send the intrusion behavior identification result and the real-time coordinates of the intrusion object to the perimeter intrusion management terminal.

[0078] Furthermore, the system also includes:

[0079] A first judgment module, wherein the first judgment module is used to activate the binocular pan-tilt camera array for monitoring to obtain a first monitoring image set when the ambient brightness information is greater than the first brightness threshold and greater than a second brightness threshold;

[0080] a second judgment module, wherein the second judgment module is used to activate the laser fill light component and perform monitoring in combination with the binocular pan-tilt camera array to obtain a first monitoring image set when the ambient brightness information is greater than the first brightness threshold and less than or equal to the second brightness threshold;

[0081] The second brightness threshold is greater than the first brightness threshold.

[0082] Furthermore, the system also includes:

[0083] A coordinate acquisition module, the coordinate acquisition module is used to obtain the coordinates of a first monitoring area of ​​a first binocular pan-tilt camera of the binocular pan-tilt camera array;

[0084] A region extraction module, the region extraction module is used to extract a plurality of vegetation coverage areas according to the coordinates of the first monitoring area, wherein any vegetation coverage area has a vegetation coverage width label, a vegetation coverage height label and a vegetation coverage length label;

[0085] A first model building module, the first model building module is used to build a vegetation coverage three-dimensional model according to the vegetation coverage width label, the vegetation coverage height label and the vegetation coverage length label;

[0086] A second model building module, the second model building module is used to configure a minimum intrusion object width, a minimum intrusion object height, and a minimum intrusion object length based on a historical intrusion transaction set, and build an intrusion object threshold model;

[0087] An auxiliary monitoring module is used to compare the intrusion object threshold model with the vegetation coverage three-dimensional model. When the vegetation coverage three-dimensional model can block the intrusion object threshold model, the first infrared pan-tilt imager of the first monitoring area coordinate is activated for auxiliary monitoring, and an infrared monitoring auxiliary image is obtained, which is added to the first monitoring image set of the first monitoring area coordinate.

[0088] Furthermore, the system also includes:

[0089] a first traversal module, the first traversal module being used to traverse the first infrared imaging set and the first monitoring image set, and configure a reference infrared imaging set and a reference monitoring image set, wherein the reference infrared imaging set and the reference monitoring image set are monitoring images without intrusion objects;

[0090] An identification module is used to activate an intrusion object judgment channel, traverse the first infrared imaging set and the first monitoring image set, combine the reference infrared imaging set and the reference monitoring image set for identification, and obtain the distribution coordinates of the intrusion object.

[0091] Furthermore, the system also includes:

[0092] A second traversal module, the second traversal module is used to traverse the first infrared imaging set and the first monitoring image set to extract a first image to be identified and a first reference image;

[0093] A first abnormality identification module, wherein the first abnormality identification module is used to activate the first feature extraction path of the infrared response node of the intrusion object judgment channel to receive the first image to be identified when the first image to be identified is an infrared image, and activate the second feature extraction path of the infrared response node of the intrusion object judgment channel to receive the first reference image for abnormality identification to obtain a morphological deviation area, wherein the infrared response node is a twin neural network;

[0094] A second abnormality identification module, wherein the second abnormality identification module is used for, when the first image to be identified is a non-infrared image, activating the third feature extraction path of the conventional image response node of the intrusion object judgment channel to receive the first image to be identified, activating the fourth feature extraction path of the conventional image response node of the intrusion object judgment channel to receive the first reference image for abnormality identification, and obtaining a morphological deviation area, wherein the conventional image response node is a twin neural network;

[0095] A coordinate positioning module is used to locate the morphological deviation area based on a pan-tilt coordinate system to generate the distribution coordinates of the intrusion object.

[0096] Furthermore, the system also includes:

[0097] A first tracking and monitoring module, the first tracking and monitoring module is used to lock the intrusion object on the first monitoring device for tracking and monitoring based on the distribution coordinates of the intrusion object, and obtain a moving coordinate sequence of the intrusion object in the first area, wherein the first monitoring device is an infrared pan-tilt imager and / or a binocular pan-tilt camera;

[0098] A second tracking and monitoring module, wherein the second tracking and monitoring module is used to activate a second monitoring device to lock the intruder for tracking and monitoring when the intruder leaves the first area and moves to the second area, and obtain a moving coordinate sequence of the intruder in the second area;

[0099] A joint storage module is used to jointly store the first area intrusion object movement coordinate sequence, the second area intrusion object movement coordinate sequence, and the Kth area intrusion object movement coordinate sequence according to the monitoring timing to obtain the intrusion object movement coordinate sequence.

[0100] Furthermore, the system also includes:

[0101] A path configuration module, wherein the path configuration module is used to configure an intrusion behavior movement sensitive path;

[0102] A mixing module, the mixing module is used to collect an intrusion transaction object movement coordinate sequence data set and a conventional object movement coordinate sequence data set for mixing to generate an intrusion behavior recognition training data set;

[0103] Function module, which is used to configure the intrusion behavior trigger probability assessment function:

[0104] Y i =(y1,y2,…y l ,…,y Q );

[0105]

[0106] Among them, P(X0) represents the intrusion behavior triggering probability of the moving coordinate sequence of the intrusion object, Y i The moving coordinate sequence that represents the sensitive moving path of the i-th intrusion behavior, y l The lth coordinate of the mobile coordinate sequence representing the mobile sensitive path of the i-th intrusion behavior, X0 represents the mobile coordinate sequence of the intrusion object, Q represents the number of coordinates of the mobile sensitive path of the i-th intrusion behavior, and M represents the number of mobile sensitive paths of the intrusion behavior;

[0107] An unsupervised training module is used to construct an intrusion behavior evaluation rule based on the intrusion behavior trigger probability evaluation function and the intrusion behavior mobile sensitive path, perform unsupervised training on the intrusion behavior identification training data set, generate the intrusion behavior identification channel, analyze the intrusion object mobile coordinate sequence, and generate the intrusion behavior trigger probability.

[0108] Through the above-mentioned detailed description of the perimeter intrusion identification method based on video joint acquisition in this specification, those skilled in the art can clearly understand the perimeter intrusion identification system based on video joint acquisition in this embodiment. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part.

[0109] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A perimeter intrusion identification method based on video joint acquisition, characterized in that: include: Collecting ambient brightness information of the first monitoring area through a photoreceptor; When the ambient brightness information is less than or equal to a first brightness threshold, activating the infrared pan-tilt imager array for monitoring to obtain a first infrared imaging set; When the environment brightness information is greater than the first brightness threshold, activating the binocular pan-tilt camera array for monitoring to obtain a first monitoring image set; Independently identifying the first infrared imaging set and the first monitoring image set to obtain distribution coordinates of intrusion objects; According to the distribution coordinates of the intrusion object, combined with the ambient brightness information, the infrared pan-tilt imager array or the binocular pan-tilt camera array is activated to lock the intrusion object for joint tracking to obtain a moving coordinate sequence of the intrusion object; According to the intrusion object movement coordinate sequence, the intrusion behavior recognition channel is trained to generate an intrusion behavior trigger probability; When the intrusion behavior trigger probability meets the trigger probability threshold, the sound and light alarm device is activated to sound an alarm, and the intrusion behavior recognition result and the real-time coordinates of the intrusion object are sent to the perimeter intrusion management terminal; According to the intrusion object movement coordinate sequence, the intrusion behavior recognition channel is trained to generate the intrusion behavior trigger probability, including: Configure intrusion behavior mobile sensitive paths; Collect the intrusion transaction object movement coordinate sequence data set and the regular object movement coordinate sequence data set, mix them, and generate the intrusion behavior recognition training data set; Configure the intrusion behavior trigger probability evaluation function: Y i =(y1,y2,…y l ,…,y Q ); Among them, P(X0) represents the intrusion behavior triggering probability of the moving coordinate sequence of the intrusion object, Y i The moving coordinate sequence that represents the sensitive moving path of the i-th intrusion behavior, y l The lth coordinate of the mobile coordinate sequence representing the mobile sensitive path of the i-th intrusion behavior, X0 represents the mobile coordinate sequence of the intrusion object, Q represents the number of coordinates of the mobile sensitive path of the i-th intrusion behavior, and M represents the number of mobile sensitive paths of the intrusion behavior; According to the intrusion behavior trigger probability evaluation function and the intrusion behavior mobile sensitive path, an intrusion behavior evaluation rule is constructed, the intrusion behavior identification training data set is unsupervisedly trained to generate the intrusion behavior identification channel, and the intrusion object mobile coordinate sequence is analyzed to generate the intrusion behavior trigger probability.

2. The method according to claim 1, characterized in that When the environment brightness information is greater than the first brightness threshold, activating the binocular pan-tilt camera array for monitoring to obtain a first monitoring image set, further comprising: When the environment brightness information is greater than the first brightness threshold and greater than the second brightness threshold, activating the binocular pan-tilt camera array for monitoring to obtain a first monitoring image set; When the environment brightness information is greater than the first brightness threshold and less than or equal to the second brightness threshold, the laser fill light component is activated, and monitoring is performed in combination with the binocular pan-tilt camera array to obtain a first monitoring image set; The second brightness threshold is greater than the first brightness threshold.

3. The method according to claim 1, characterized in that When the environment brightness information is greater than the first brightness threshold, activating the binocular pan-tilt camera array for monitoring to obtain a first monitoring image set, further comprising: Obtaining the coordinates of a first monitoring area of ​​a first binocular pan-tilt camera of the binocular pan-tilt camera array; Extracting a plurality of vegetation coverage areas according to the coordinates of the first monitoring area, wherein any vegetation coverage area has a vegetation coverage width label, a vegetation coverage height label, and a vegetation coverage length label; Constructing a three-dimensional vegetation coverage model according to the vegetation coverage width label, the vegetation coverage height label and the vegetation coverage length label; Based on the historical intrusion transaction set, the minimum width, height and length of the intrusion object are configured to build an intrusion object threshold model; The intrusion object threshold model and the vegetation coverage three-dimensional model are compared. When the vegetation coverage three-dimensional model can block the intrusion object threshold model, the first infrared pan-tilt imager of the first monitoring area coordinate is activated for auxiliary monitoring, and an infrared monitoring auxiliary image is obtained, which is added to the first monitoring image set of the first monitoring area coordinate.

4. The method according to claim 1, characterized in that Independently identifying the first infrared imaging set and the first monitoring image set to obtain the distribution coordinates of the intrusion object includes: Traversing the first infrared imaging set and the first monitoring image set, configuring a reference infrared imaging set and a reference monitoring image set, wherein the reference infrared imaging set and the reference monitoring image set are monitoring images without intrusion objects; The intrusion object judgment channel is activated, the first infrared imaging set and the first monitoring image set are traversed, and the reference infrared imaging set and the reference monitoring image set are combined for identification to obtain the distribution coordinates of the intrusion object.

5. The method according to claim 4, characterized in that Activating the intrusion object judgment channel, traversing the first infrared imaging set and the first monitoring image set, combining the reference infrared imaging set and the reference monitoring image set for identification, and obtaining the intrusion object distribution coordinates, including: Traversing the first infrared imaging set and the first monitoring image set, extracting a first image to be identified and a first reference image; When the first image to be identified is an infrared image, the first feature extraction path of the infrared response node of the intrusion object judgment channel is activated to receive the first image to be identified, and the second feature extraction path of the infrared response node of the intrusion object judgment channel is activated to receive the first reference image for abnormal identification to obtain a morphological deviation area, wherein the infrared response node is a twin neural network; When the first image to be identified is a non-infrared image, the third feature extraction path of the conventional image response node of the intrusion object judgment channel is activated to receive the first image to be identified, and the fourth feature extraction path of the conventional image response node of the intrusion object judgment channel is activated to receive the first reference image for abnormal identification to obtain a morphological deviation area, wherein the conventional image response node is a twin neural network; Based on the pan-tilt coordinate system, the morphological deviation area is located to generate the distribution coordinates of the intrusion object.

6. The method according to claim 1, characterized in that According to the distribution coordinates of the intrusion object, in combination with the ambient brightness information, the infrared pan-tilt imager array or the binocular pan-tilt camera array is activated to lock the intrusion object for joint tracking, and a moving coordinate sequence of the intrusion object is obtained, including: Based on the distribution coordinates of the intrusion object, the intrusion object is locked on the first monitoring device for tracking and monitoring, and a moving coordinate sequence of the intrusion object in the first area is obtained, wherein the first monitoring device is an infrared pan-tilt imager and / or a binocular pan-tilt camera; When the intruding object leaves the first area and enters the second area, the second monitoring device is activated to lock the intruding object for tracking and monitoring, and a moving coordinate sequence of the intruding object in the second area is obtained; According to the monitoring sequence, the first region intrusion object movement coordinate sequence, the second region intrusion object movement coordinate sequence, and the Kth region intrusion object movement coordinate sequence are jointly stored to obtain the intrusion object movement coordinate sequence.

7. Perimeter intrusion identification system based on video joint acquisition, characterized by: The system is used to execute the perimeter intrusion identification method based on video joint acquisition according to any one of claims 1 to 6, the system comprising: An information collection module, the information collection module is used to collect the ambient brightness information of the first monitoring area through a photoreceptor; A first monitoring module, wherein the first monitoring module is used to activate the infrared pan-tilt imager array for monitoring when the ambient brightness information is less than or equal to a first brightness threshold, and obtain a first infrared imaging set; A second monitoring module, wherein the second monitoring module is used to activate the binocular pan-tilt camera array for monitoring when the environmental brightness information is greater than the first brightness threshold, and obtain a first monitoring image set; An independent identification module, the independent identification module is used to independently identify the first infrared imaging set and the first monitoring image set to obtain the distribution coordinates of the intrusion object; A joint tracking module, wherein the joint tracking module is used to activate the infrared pan-tilt imager array or the binocular pan-tilt camera array to lock the intrusion object for joint tracking according to the distribution coordinates of the intrusion object and in combination with the ambient brightness information, so as to obtain a moving coordinate sequence of the intrusion object; A first training module, the first training module is used to perform intrusion behavior recognition channel training according to the intrusion object movement coordinate sequence, and generate an intrusion behavior trigger probability; The alarm module is used to activate the sound and light alarm device to alarm when the intrusion behavior trigger probability meets the trigger probability threshold, and at the same time send the intrusion behavior identification result and the real-time coordinates of the intrusion object to the perimeter intrusion management terminal.

Images