Intrusion response method, apparatus, device and readable storage medium

Abstract

The application provides an intrusion response method, device and equipment and a readable storage medium, and relates to the field of intrusion monitoring. The method comprises the following steps: determining a target feature contour according to an infrared thermal image frame and a visible light image frame corresponding to any synchronization moment; generating a first intrusion response in the case that the target feature contour and a first monitoring area in a to-be-monitored area overlap, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold, the first intrusion response being used for indicating and informing a monitoring personnel; generating a second intrusion response in the case that the target feature contour and a second monitoring area in the to-be-monitored area overlap, the second intrusion response being used for indicating and informing an intruder; and the monitoring level of the first monitoring area is less than the monitoring level of the second monitoring area. The application generates different responses in different monitoring areas, which not only improves the accuracy of suspicious personnel detection, but also improves the accuracy and rationality of the intrusion response.

Description

Technical Field

[0001] This invention relates to the field of intrusion monitoring, and more particularly to an intrusion response method, apparatus, device, and readable storage medium. Background Technology

[0002] Currently, in the field of unmanned monitoring, it is impossible to distinguish suspicious outlines and generate different intrusion responses based on the movement trajectory of suspicious outlines in the monitoring screen. Intrusion responses in a single monitoring area often lead to false alarms or missed alarms. Summary of the Invention

[0003] This invention provides an intrusion response method, apparatus, device, and readable storage medium to address the technical deficiency in the prior art that it is impossible to achieve accurate intrusion response to suspicious outlines in unattended monitoring. It provides a technical solution to divide the monitoring area into intrusion zones corresponding to intrusion levels and respond to different intrusion notifications according to different intrusion levels.

[0004] In a first aspect, the present invention provides an intrusion response method, comprising:

[0005] The target feature contour is determined based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization moment;

[0006] If the target feature contour overlaps with the first monitoring area in the area to be monitored, and the similarity between the target feature contour and the preset edge contour is greater than a preset threshold, a first intrusion response is generated. The first intrusion response is used to instruct and inform the monitoring personnel.

[0007] If the target feature contour overlaps with a second monitoring area in the area to be monitored, a second intrusion response is generated, which is used to notify the intruder.

[0008] The monitoring level of the first monitoring area is lower than that of the second monitoring area.

[0009] According to the intrusion response method provided by the present invention, before determining the target feature contour, it further includes:

[0010] Infrared thermal video is acquired using an infrared thermal imager, and visible light video is acquired using a visible light camera.

[0011] The infrared thermal video and the visible light video are divided according to different times, and the infrared thermal image frames corresponding to all times and the visible light image frames corresponding to each time are obtained.

[0012] According to the intrusion response method provided by the present invention, determining the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization moment includes:

[0013] When the brightness of the area to be monitored is greater than or equal to the preset brightness, the target feature contour is determined based on the infrared thermal image frame and visible light image frame corresponding to any synchronization time, and based on the meteorological information of the monitored area.

[0014] When the brightness of the area to be monitored is less than the preset brightness, the target feature contour is determined based on the infrared thermal imaging in the infrared thermal image frame.

[0015] The meteorological information includes both sunny and cloudy weather conditions.

[0016] According to the intrusion response method provided by the present invention, determining the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization time, and based on the meteorological information of the monitored area, includes:

[0017] When the weather information indicates a clear day, a visible light feature contour is determined based on the visible light image frame, and an infrared thermal feature contour is determined based on the infrared thermal image frame. If the overlap between the visible light feature contour and the infrared thermal feature contour is greater than a first overlap, the visible light feature contour is determined to be the target feature contour.

[0018] When the weather information indicates a non-sunny day, an infrared thermal feature contour is determined based on an infrared thermal image frame, and a visible light feature contour is determined based on a visible light image frame. If the overlap between the infrared thermal feature contour and the visible light feature contour is greater than a second overlap, the infrared thermal feature contour is determined to be the target feature contour.

[0019] The first overlap is greater than the second overlap;

[0020] The non-sunny weather conditions include rainy weather, snowy weather, foggy weather, and sandstorm weather.

[0021] According to the intrusion response method provided by the present invention, after determining the target feature contour, the method further includes:

[0022] A first marker corresponding to the target feature contour is generated in the display screen of the area to be monitored;

[0023] The first marker is used to identify pedestrians, animals, or vehicles.

[0024] According to the intrusion response method provided by the present invention, after generating the first intrusion response, it further includes:

[0025] Based on the optical flow field tracking algorithm, infrared thermal image frames corresponding to all times and visible light image frames corresponding to each time are processed to obtain pedestrian motion information corresponding to the target feature contour.

[0026] A second marker corresponding to the target feature contour is generated in the display screen of the area to be monitored;

[0027] The second marker is used to characterize the pedestrian movement information;

[0028] The pedestrian movement information includes the direction of pedestrian movement and the speed of pedestrian movement.

[0029] According to the intrusion response method provided by the present invention, the area to be monitored is an airport runway or subway track, or a stadium entrance;

[0030] In the case where the area to be monitored is an airport runway or a subway track, the first monitoring area is a waiting area, the second monitoring area is a danger area, and the second intrusion response also includes a voice prompt, which is used to inform the intruder to leave the danger area.

[0031] When the area to be monitored is the entrance of the venue, the first monitoring area is a passing area, the second monitoring area is a warning area, and the second intrusion response also includes a light prompt, which is used to inform the intruder that they have entered the warning area.

[0032] Secondly, an intrusion response device is provided, comprising:

[0033] Determining unit: used to determine the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization time;

[0034] First generation unit: used to generate a first intrusion response when the target feature contour overlaps with a first monitoring area in the area to be monitored, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold. The first intrusion response is used to instruct and inform monitoring personnel.

[0035] The second generation unit is used to generate a second intrusion response when the target feature contour overlaps with the second monitoring area in the area to be monitored. The second intrusion response is used to instruct and notify the intruder.

[0036] The monitoring level of the first monitoring area is lower than that of the second monitoring area.

[0037] Thirdly, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the intrusion response method when executing the program.

[0038] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements an intrusion response method as described above.

[0039] This invention provides an intrusion response method, apparatus, device, and readable storage medium. By acquiring infrared thermal image frames and visible light image frames at a synchronized time in real time, this invention determines the target feature contour. When the target feature contour appears in the monitored area and is determined to be a suspicious person, the monitoring personnel will be alerted, but the suspicious person will not be notified. Only when the suspicious person further penetrates into a second monitored area will the intrusion behavior be notified to the intruder. By generating different responses in different monitored areas, this invention not only improves the accuracy of suspicious person detection but also enhances the accuracy and rationality of intrusion response. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0041] Figure 1 This is one of the flowcharts of the intrusion response method provided by the present invention;

[0042] Figure 2 This is the second flowchart of the intrusion response method provided by the present invention;

[0043] Figure 3 This is one of the flowcharts for determining the contour of a target feature provided by the present invention;

[0044] Figure 4 This is the second flowchart illustrating the process of determining the contour of a target feature provided by the present invention;

[0045] Figure 5 This is the third flowchart of the intrusion response method provided by the present invention;

[0046] Figure 6 This is a schematic diagram of the intrusion response device provided by the present invention;

[0047] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] Currently, real-time monitoring of moving targets, especially pedestrians and vehicles, is crucial. For critical intrusion prevention sites such as airport runways, subway tracks, museums, and exhibition halls, real-time monitoring and management deployment are indispensable. Infrared tracking systems, which passively receive infrared radiation from targets to detect, track, and identify them, offer advantages such as good concealment, strong resistance to electromagnetic interference, small size, light weight, and high mobility, and are gradually becoming an important component of modern defense systems. Currently, there is no technology that fully utilizes the imaging characteristics of visible light and infrared thermal acquisition to solve the problem of generating different responses in different monitoring areas, thus failing to meet the application requirements of pedestrian detection and tracking. To address these technical problems, this invention provides an intrusion response method, device, equipment, and readable storage medium. Figure 1 This is one of the flowcharts of the intrusion response method provided by the present invention. The present invention discloses an intrusion response method, including:

[0050] Step 101: Determine the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization moment;

[0051] Step 102: If the target feature contour overlaps with the first monitoring area in the area to be monitored, and the similarity between the target feature contour and the preset edge contour is greater than a preset threshold, a first intrusion response is generated. The first intrusion response is used to instruct and inform the monitoring personnel.

[0052] Step 103: If the target feature contour overlaps with the second monitoring area in the area to be monitored, a second intrusion response is generated. The second intrusion response is used to notify the intruder.

[0053] The monitoring level of the first monitoring area is lower than that of the second monitoring area.

[0054] In step 101, the target feature contour is determined based on the infrared thermal image frame and visible light image frame corresponding to any synchronization moment. For the same monitoring area, the present invention uses two different image acquisition devices to acquire infrared thermal video and visible light image video in real time, and converts the infrared thermal video into image frames and the visible light image video into image frames, thereby obtaining all image frames corresponding to the infrared thermal video and all image frames corresponding to the visible light image video, and finally obtaining the infrared thermal image frame and visible light image frame corresponding to each synchronization moment.

[0055] This invention can acquire target feature contours by adopting different feature extraction strategies based on the geographical location, climate, brightness, and visibility of the area to be monitored. For example, in low visibility conditions, a feature extraction strategy that primarily uses infrared thermal image frames and secondarily uses visible light image frames is adopted, while in high visibility conditions, a feature extraction strategy that primarily uses visible light image frames and secondarily uses infrared thermal image frames is adopted.

[0056] In step 102, the present invention differs from the prior art in that it can only monitor the complete picture of a monitoring area. Instead, it divides a monitoring area into two monitoring areas, namely a first monitoring area and a second monitoring area. The first monitoring area and the second monitoring area are spliced ​​together to form the monitoring area. Different intrusion response strategies are adopted for the two different monitoring areas.

[0057] Optionally, the monitoring level of the first monitoring area is lower than that of the second monitoring area. For example, corresponding to a museum entrance, two different image acquisition devices are set up to acquire infrared thermal image frames and visible light image frames at any synchronous moment in real time. In actual display, the monitoring images of the area to be monitored are divided according to a preset ratio based on the horizontal line. The upper part of the screen shows the museum entrance, while the lower part of the screen does not show the museum entrance and can be a corridor. In this case, the monitoring area corresponding to the lower part of the screen is the first monitoring area, and the monitoring area corresponding to the upper part of the screen is the second monitoring area. Since intruders must pass through the first monitoring area and then the second monitoring area to enter the museum entrance, an actual intrusion will only occur if the intruder enters the second monitoring area. Since staff may only pass through the first monitoring area and not go to the second monitoring area, the monitoring level of the first monitoring area is lower than that of the second monitoring area.

[0058] Furthermore, if the target feature contour overlaps with the first monitoring area in the area to be monitored, it is considered that part or all of the contour of a suspicious object has entered the first monitoring area. In order to rule out that the target feature contour may be a bird, dog, cat or other animal, the present invention also needs to further detect whether the target feature contour is a suspicious person, that is, to determine the similarity between the target feature contour and the preset edge contour. The present invention can use the existing contour similarity comparison tool OpenCV, or improve the accuracy of the similarity comparison between the target feature contour and the preset edge contour through translation, rotation, affine, perspective and other means.

[0059] If the target feature contour overlaps with the first monitoring area in the area to be monitored, and the similarity between the target feature contour and the preset edge contour is less than or equal to a preset threshold, then the target feature contour is considered not to be a suspicious person, but possibly just a passing animal. In this case, there is no need to generate a first intrusion response. If the target feature contour overlaps with the first monitoring area in the area to be monitored, and the similarity between the target feature contour and the preset edge contour is greater than a preset threshold, then the target feature contour is considered to be a suspicious person. In this case, it is necessary to record the appearance of the suspicious person and further track the subsequent movements of the suspicious person to generate a first intrusion response. The first intrusion response is used to instruct and inform the monitoring personnel. This invention can use SMS, ringing, warning lights, or vibration to inform the monitoring personnel and remind them to pay attention.

[0060] In this invention, when the target feature contour does not overlap with the first monitoring area in the area to be monitored, and the similarity between the target feature contour and the preset edge contour is greater than a preset threshold, i.e., when the target feature contour is determined to be a suspicious person, the suspicious person is warned. This is because the monitoring level of the first monitoring area is lower than that of the second monitoring area. That is, entering the first monitoring area does not mean that the suspicious person is an intruder. Therefore, in order to avoid disturbing other passers-by, there is no need to use sound, light or other means to warn the suspicious person.

[0061] In step 103, if the target feature contour overlaps with the second monitoring area in the area to be monitored, a second intrusion response is generated. The second intrusion response is used to notify the intruder. After step 102, the present invention will continuously monitor the next movement route of the suspected person. Once it is found that the target feature contour overlaps with the second monitoring area in the area to be monitored, it is considered that the target feature contour has intruded into the second monitoring area, and the target feature contour is determined to be an intruder. At this time, in order to prevent the intruder from lingering in the second monitoring area or entering other areas, it is necessary to notify the intruder, that is, generate a second intrusion response. The present invention can use an audible alarm to remind the intruder to leave, or use a flashing light to attract the intruder's attention. If it is an indoor environment, it can also turn off the lights to prevent the intruder from continuing to move forward. It can also use some interlocking mechanisms, such as closing or locking the door from the second monitoring area to other prohibited areas, or cutting off the power to other prohibited areas, etc., to prevent the intruder from entering other prohibited areas, thereby improving the monitoring security factor.

[0062] Optionally, the area to be monitored is an airport runway, subway tracks, or stadium entrance;

[0063] In the case where the area to be monitored is an airport runway or a subway track, the first monitoring area is a waiting area, the second monitoring area is a danger area, and the second intrusion response also includes a voice prompt, which is used to inform the intruder to leave the danger area.

[0064] When the area to be monitored is the entrance of the venue, the first monitoring area is a passing area, the second monitoring area is a warning area, and the second intrusion response also includes a light prompt, which is used to inform the intruder that they have entered the warning area.

[0065] This invention provides two common application scenarios for the disclosed technical solution, but this does not mean that the invention can only be applied to the above two scenarios. Specifically, when the area to be monitored is an airport runway or a subway track, the area to be monitored is set as a waiting area for people waiting for trains or buses, and a dangerous area for airplanes or trains arriving at pick-up and drop-off points. In this case, the camera angle of the image acquisition device is adjusted so that the first monitoring area and the second monitoring area are displayed in the area to be monitored. In other embodiments, after acquiring the area to be monitored, a dividing line for separating the first monitoring area and the second monitoring area can be determined. Since the image acquisition device cannot move during image acquisition, after determining the dividing line, subsequent frames of images can be processed based on the dividing line. More specifically, when it is determined that an intruder has entered the second monitoring area, a second intrusion response is generated. The second intrusion response also includes a voice prompt, which is used to inform the intruder to leave the dangerous area.

[0066] Optionally, when the area to be monitored is the entrance of the venue, the area to be monitored is set as a passage area for people to pass through and a warning area for reaching the interior of the venue. In order to prevent intrusion by other non-staff members, the present invention can also generate a second intrusion response when it is determined that an intruder has entered the second monitoring area. The second intrusion response also includes a light prompt, which is used to inform the intruder that he has entered the warning area.

[0067] More specifically, to prevent false alarms when authorized personnel enter the second monitoring area, the present invention can also set a preset walking route in the second monitoring area. If the authorized personnel enter the second monitoring area according to the preset walking route, the generation of the second intrusion response will not be triggered. As a variation, the present invention can also set the preset walking route in the first monitoring area so that the authorized personnel obtain the qualification to enter the second monitoring area in the first monitoring area, so that the generation of the second intrusion response will not be triggered when entering the second monitoring area.

[0068] This invention provides a method for detecting and tracking intruders based on a dual-view camera. It can improve the detection rate and accuracy of target contour regions in images combining visible light and infrared images. Furthermore, it can detect and identify moving targets in a multispectral, efficient, and all-weather manner and track them. This fully combines the advantages of both visible light and infrared images to achieve automatic detection and tracking of pedestrians in all weather conditions.

[0069] This invention provides an intrusion response method, apparatus, device, and readable storage medium. By acquiring infrared thermal image frames and visible light image frames at a synchronized time in real time, this invention determines the target feature contour. When the target feature contour appears in the monitored area and is determined to be a suspicious person, the monitoring personnel will be alerted, but the suspicious person will not be notified. Only when the suspicious person further penetrates into a second monitored area will the intrusion behavior be notified to the intruder. By generating different responses in different monitored areas, this invention not only improves the accuracy of suspicious person detection but also enhances the accuracy and rationality of intrusion response.

[0070] Figure 2 This is a second flowchart of the intrusion response method provided by the present invention. Before determining the target feature contour, it also includes:

[0071] Step 201: Acquire infrared thermal video using an infrared thermal imager and visible light video using a visible light camera;

[0072] Step 202: Divide the infrared thermal video and the visible light video according to different times, and obtain the infrared thermal image frames corresponding to all times and the visible light image frames corresponding to each time.

[0073] In step 201, the present invention can achieve uninterrupted real-time monitoring around the clock, and can also select a preset time period for monitoring according to the needs of the monitoring personnel. In order to record the infrared thermal video and the visible light video, after acquiring the infrared thermal video and the visible light video, the present invention can also store the infrared thermal video and the visible light video, and verify the trigger probability and accuracy of the intrusion response through manual verification, and provide strong support for subsequent monitoring technology upgrades.

[0074] In step 202, those skilled in the art will understand that an image frame is the smallest unit that makes up a video. After acquiring the infrared thermal video and the visible light video, the present invention processes the infrared thermal video and the visible light video, and at each moment, determines the corresponding infrared thermal image frame and the visible light image frame, and finally obtains the infrared thermal image frames corresponding to all moments and the visible light image frames corresponding to each moment.

[0075] Figure 3 This is one of the flowcharts for determining the target feature contour provided by the present invention. The step of determining the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization moment includes:

[0076] Step 1011: When the brightness of the area to be monitored is greater than or equal to the preset brightness, the target feature contour is determined based on the infrared thermal image frame and visible light image frame corresponding to any synchronization time, and based on the meteorological information of the monitored area.

[0077] Step 1012: When the brightness of the area to be monitored is less than the preset brightness, determine the target feature contour based on the infrared thermal imaging in the infrared thermal image frame;

[0078] The meteorological information includes both sunny and cloudy weather conditions.

[0079] In step 1011, the present invention combines the brightness of the area to be monitored. When the brightness is high, it can more accurately distinguish whether the target feature contour is a suspicious person. Specifically, when the brightness of the area to be monitored is greater than or equal to a preset brightness, both infrared thermal image frames and visible light image frames can clearly obtain the target feature contour. Thus, the present invention uses a combination of the two image frames to improve the ability to distinguish the target feature contour. At the same time, sometimes although the brightness is high, the visibility is still very low, such as in rainy or snowy weather. The meteorological information includes sunny and non-sunny weather conditions. The present invention also combines meteorological information to further improve the ability to distinguish the target feature contour.

[0080] In step 1012, if the brightness of the area to be monitored is less than the preset brightness, it is assumed that the visibility of the area to be monitored is low due to indoor brightness adjustment or day and night changes in the outdoor monitoring environment. In this case, the reference significance of the visible light image frame is low. In this case, the present invention uses infrared thermal imaging in the infrared thermal image frame to determine the target feature contour, thereby reducing the computational complexity, shortening the resolution time, and improving the recognition efficiency.

[0081] Figure 4 This is the second schematic diagram of the process for determining the feature contour of a target provided by the present invention. The step of determining the feature contour of the target based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization time, and based on the meteorological information of the monitored area, includes:

[0082] Step 10111: When the weather information indicates a clear day, determine the visible light feature contour based on the visible light image frame, determine the infrared thermal feature contour based on the infrared thermal image frame, and determine the visible light feature contour as the target feature contour when the overlap between the visible light feature contour and the infrared thermal feature contour is greater than a first overlap.

[0083] Step 10112: When the weather information indicates a non-sunny weather condition, determine the infrared thermal feature contour based on the infrared thermal image frame, determine the visible light feature contour based on the visible light image frame, and determine the infrared thermal feature contour as the target feature contour when the overlap between the infrared thermal feature contour and the visible light feature contour is greater than a second overlap.

[0084] The first overlap is greater than the second overlap;

[0085] The non-sunny weather conditions include rainy weather, snowy weather, foggy weather, and sandstorm weather.

[0086] In step 10111, the present invention can use a positioning system to call the weather conditions under the current location of the meteorological station. The meteorological information includes sunny and cloudy conditions. Here, sunny conditions can optionally refer to conditions with high visibility, and cloudy conditions can optionally refer to conditions with low visibility. Therefore, cloudy conditions include rainy, snowy, foggy, and sandstorm conditions. Even if the brightness is sufficient, the image captured by the image acquisition system contains rain, snow, fog, haze, and sand, resulting in an unclear image, which is not conducive to using visible light feature contours as the main feature.

[0087] Optionally, when the weather information indicates a clear day, the visible light feature contour can be determined based on the visible light image frame, and the infrared thermal feature contour can be determined based on the infrared thermal image frame. The visible light feature contour is used as the primary feature contour and compared with the infrared thermal feature contour. If the overlap between the visible light feature contour and the infrared thermal feature contour is greater than a first overlap, the visible light feature contour is determined to be the target feature contour. The first overlap can be 80% or 90%, which mainly ensures the authenticity of the obtained target feature contour through a dual determination method.

[0088] In step 10112, when the weather information indicates a non-sunny weather condition, the infrared thermal feature contour can be determined based on the infrared thermal image frame. Although the visible light image frame can also show the visible light feature contour, it may not be clear enough. Therefore, the infrared thermal feature contour is used as the primary indicator, with the visible light feature contour as an auxiliary indicator, to determine the authenticity of the infrared thermal feature contour. If the overlap between the infrared thermal feature contour and the visible light feature contour is greater than a second overlap, the infrared thermal feature contour is determined to be the target feature contour. The first overlap is greater than the second overlap, which can be 70% or 60%. Since the weight of determining the target feature contour using the infrared thermal feature contour is higher in a non-sunny weather condition, the judgment standard for the overlap between the infrared thermal feature contour and the visible light feature contour is lower than the judgment standard in step 10111.

[0089] Figure 5 This is the third flowchart of the intrusion response method provided by the present invention. After determining the target feature contour, it also includes:

[0090] A first marker corresponding to the target feature contour is generated in the display screen of the area to be monitored;

[0091] The first marker is used to identify pedestrians, animals, or vehicles.

[0092] In order to enable monitoring personnel to see the intrusion target more intuitively, this invention can generate a marker on the display screen of the monitored area when the intrusion response is implemented, so that the monitoring personnel can more easily observe the anomalies on the display screen.

[0093] After determining the target feature contour, it is determined that the target feature contour is a suspicious object. However, since it is not possible to distinguish whether the suspicious object is a person or other animal at this time, a first mark corresponding to the target feature contour is generated in the display screen of the area to be monitored. The first mark is used to represent pedestrians, animals or vehicles.

[0094] Optionally, after generating the first intrusion response, the following steps are also included:

[0095] Step 301: Based on the optical flow field tracking algorithm, process the infrared thermal image frames corresponding to all times and the visible light image frames corresponding to each time to obtain the pedestrian motion information corresponding to the target feature contour;

[0096] Step 302: Generate a second marker corresponding to the target feature contour in the display screen of the area to be monitored;

[0097] The second marker is used to characterize the pedestrian movement information;

[0098] The pedestrian movement information includes the direction of pedestrian movement and the speed of pedestrian movement.

[0099] Those skilled in the art will understand that by using an optical flow field tracking algorithm to process infrared thermal image frames corresponding to all times and visible light image frames corresponding to each time, it is possible to obtain pedestrian movement information corresponding to the target feature contour. The pedestrian movement information includes the pedestrian movement direction and the pedestrian movement speed. At this time, since the target feature contour has been identified as a suspicious person, in order to further monitor the movement of the suspicious person more intuitively, a second mark corresponding to the target feature contour is generated in the display screen of the area to be monitored. As the target feature contour moves in the display screen of the area to be monitored, the second mark moves accordingly.

[0100] Figure 6This is a schematic diagram of the intrusion response device provided by the present invention. The present invention also provides an intrusion response device, including a determination unit 1: used to determine the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization moment. The working principle of the determination unit 1 can be referred to the aforementioned step 101, and will not be repeated here.

[0101] The intrusion response device further includes a first generation unit 2: used to generate a first intrusion response when the target feature contour overlaps with a first monitoring area in the area to be monitored, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold. The first intrusion response is used to instruct and inform the monitoring personnel. The working principle of the first generation unit 2 can be referred to the aforementioned step 102, and will not be repeated here.

[0102] The intrusion response device further includes a second generation unit 3: used to generate a second intrusion response when the target feature contour overlaps with the second monitoring area in the area to be monitored. The second intrusion response is used to instruct and notify the intruder. The working principle of the second generation unit 3 can be referred to the aforementioned step 103, and will not be repeated here.

[0103] The monitoring level of the first monitoring area is lower than that of the second monitoring area.

[0104] This invention provides an intrusion response method, apparatus, device, and readable storage medium. By acquiring infrared thermal image frames and visible light image frames at a synchronized time in real time, this invention determines the target feature contour. When the target feature contour appears in the monitored area and is determined to be a suspicious person, the monitoring personnel will be alerted, but the suspicious person will not be notified. Only when the suspicious person further penetrates into a second monitored area will the intrusion behavior be notified to the intruder. By generating different responses in different monitored areas, this invention not only improves the accuracy of suspicious person detection but also enhances the accuracy and rationality of intrusion response.

[0105] According to another aspect of the present invention, the present invention also provides an intrusion response system, including an image acquisition system and a visual analysis system. The image acquisition system controls a visible light camera and an infrared thermal imager to acquire environmental images through acquisition software, and then transmits the acquired images to the visual analysis system via wired / wireless transmission. The visual analysis system, based on the image information acquired by the visible light camera and the infrared thermal imager, outputs detection results through personnel detection and transmits the results to a monitoring system.

[0106] This invention employs a person intrusion detection algorithm. Based on environmental information of the detection area, it establishes an environmental model under normal conditions. First, it detects abnormal temperature areas based on infrared images, performs edge detection on high-temperature areas to correspond to the corresponding areas in the visible light image, and then constructs a pedestrian model, performing pedestrian detection by comparing the pedestrian model with the intrusion detection algorithm. This invention also utilizes a dual-video pedestrian tracking algorithm, employing an optical flow field tracking algorithm based on visible light and infrared contours to achieve automatic pedestrian tracking, thus solving the problem of high computational complexity associated with simply using detection algorithms.

[0107] The intrusion response system also includes a monitoring system capable of displaying and monitoring visible light and infrared video, as well as intrusion detection results. The monitoring system also possesses corresponding visible light and infrared thermal imager configuration functions and algorithm import functions. This invention solves the problem of visible light cameras being affected by light conditions and unable to be used at night, and the problem of relying solely on infrared thermal imagers being unable to distinguish between people and animals. It achieves all-weather, accurate intrusion detection and automatic tracking, and can be widely used in areas preventing unauthorized entry, such as airport runways, subway tracks, museums, and exhibition halls. Simultaneously, this invention also improves the reliability of pedestrian detection in terms of application effectiveness. Based on video tracking algorithms, it improves the continuity of detection, reduces the dependence of online monitoring on personnel, and enables automatic detection and early warning.

[0108] Figure 7 This is a schematic diagram of the structure of the electronic device provided by the present invention. For example... Figure 7 As shown, the electronic device may include a processor 710, a communications interface 720, a memory 730, and a communication bus 740, wherein the processor 710, communications interface 720, and memory 730 communicate with each other via the communication bus 740. The processor 710 can call logical instructions in the memory 730 to execute an intrusion response method. This method includes: determining a target feature contour based on an infrared thermal image frame and a visible light image frame corresponding to any synchronization moment; generating a first intrusion response when the target feature contour overlaps with a first monitoring area in the area to be monitored, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold, the first intrusion response being used to instruct monitoring personnel; generating a second intrusion response when the target feature contour overlaps with a second monitoring area in the area to be monitored, the second intrusion response being used to instruct intruders; the monitoring level of the first monitoring area is lower than the monitoring level of the second monitoring area.

[0109] Furthermore, the logical instructions in the aforementioned memory 730 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0110] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute an intrusion response method provided by the above methods. The method includes: determining a target feature contour based on an infrared thermal image frame and a visible light image frame corresponding to any synchronization moment; generating a first intrusion response when the target feature contour overlaps with a first monitoring area in the area to be monitored, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold, the first intrusion response being used to instruct and notify monitoring personnel; generating a second intrusion response when the target feature contour overlaps with a second monitoring area in the area to be monitored, the second intrusion response being used to instruct and notify intruders; the monitoring level of the first monitoring area is lower than the monitoring level of the second monitoring area.

[0111] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the intrusion response method provided by the above methods. The method includes: determining a target feature contour based on an infrared thermal image frame and a visible light image frame corresponding to any synchronization moment; generating a first intrusion response when the target feature contour overlaps with a first monitoring area in the area to be monitored, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold, the first intrusion response being used to instruct monitoring personnel; generating a second intrusion response when the target feature contour overlaps with a second monitoring area in the area to be monitored, the second intrusion response being used to instruct intruders; the monitoring level of the first monitoring area is lower than the monitoring level of the second monitoring area.

[0112] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0113] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0114] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An intrusion response method, characterized in that, include: The target feature contour is determined based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization moment; If the target feature contour overlaps with the first monitoring area in the area to be monitored, and the similarity between the target feature contour and the preset edge contour is greater than a preset threshold, a first intrusion response is generated. The first intrusion response is used to instruct and inform the monitoring personnel. If the target feature contour overlaps with the second monitoring area in the area to be monitored, a second intrusion response is generated. The second intrusion response is used to notify the intruder. A preset walking route is set in the second monitoring area. If an authorized staff member enters the second monitoring area according to the preset walking route, the generation of the second intrusion response will not be triggered. The monitoring level of the first monitoring area is lower than that of the second monitoring area; The step of determining the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization time includes: When the brightness of the monitored area is greater than or equal to a preset brightness, the target feature contour is determined based on the infrared thermal image frame and visible light image frame corresponding to any synchronization time, and based on the meteorological information of the monitored area. Specifically, this includes: When the weather information indicates a clear day, a visible light feature contour is determined based on the visible light image frame, and an infrared thermal feature contour is determined based on the infrared thermal image frame. If the overlap between the visible light feature contour and the infrared thermal feature contour is greater than a first overlap, the visible light feature contour is determined to be the target feature contour. When the weather information indicates a non-sunny day, an infrared thermal feature contour is determined based on an infrared thermal image frame, and a visible light feature contour is determined based on a visible light image frame. If the overlap between the infrared thermal feature contour and the visible light feature contour is greater than a second overlap, the infrared thermal feature contour is determined to be the target feature contour. The first degree of overlap is greater than the second degree of overlap.

2. The intrusion response method according to claim 1, characterized in that, Before determining the target feature contour, the following steps are also included: Infrared thermal video is acquired using an infrared thermal imager, and visible light video is acquired using a visible light camera. The infrared thermal video and the visible light video are divided according to different times, and the infrared thermal image frames corresponding to all times and the visible light image frames corresponding to each time are obtained.

3. The intrusion response method according to claim 1, characterized in that, The step of determining the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization time further includes: When the brightness of the area to be monitored is less than the preset brightness, the target feature contour is determined based on the infrared thermal imaging in the infrared thermal image frame.

4. The intrusion response method according to claim 1, characterized in that, After determining the target feature contour, the process also includes: A first marker corresponding to the target feature contour is generated in the display screen of the area to be monitored; The first marker is used to identify pedestrians, animals, or vehicles.

5. The intrusion response method according to claim 4, characterized in that, After generating the first intrusion response, the process also includes: Based on the optical flow field tracking algorithm, infrared thermal image frames corresponding to all times and visible light image frames corresponding to each time are processed to obtain pedestrian motion information corresponding to the target feature contour. A second marker corresponding to the target feature contour is generated in the display screen of the area to be monitored; The second marker is used to characterize the pedestrian movement information; The pedestrian movement information includes the direction of pedestrian movement and the speed of pedestrian movement.

6. The intrusion response method according to claim 1, characterized in that, The areas to be monitored are airport runways, subway tracks, and stadium entrances. In the case where the area to be monitored is an airport runway or a subway track, the first monitoring area is a waiting area, the second monitoring area is a danger area, and the second intrusion response also includes a voice prompt, which is used to inform the intruder to leave the danger area. When the area to be monitored is the entrance of the venue, the first monitoring area is a passing area, the second monitoring area is a warning area, and the second intrusion response also includes a light prompt, which is used to inform the intruder that they have entered the warning area.

7. An intrusion response device, characterized in that, include: Determining unit: used to determine the target feature contour based on the infrared thermal image frame and the visible light image frame corresponding to any synchronization time; First generation unit: used to generate a first intrusion response when the target feature contour overlaps with a first monitoring area in the area to be monitored, and the similarity between the target feature contour and a preset edge contour is greater than a preset threshold. The first intrusion response is used to instruct and inform monitoring personnel. The second generation unit is used to generate a second intrusion response when the target feature contour overlaps with the second monitoring area in the area to be monitored. The second intrusion response is used to inform the intruder and to set a preset walking route in the second monitoring area. If the authorized staff enters the second monitoring area according to the preset walking route, the generation of the second intrusion response will not be triggered. The monitoring level of the first monitoring area is lower than that of the second monitoring area; The determining unit is specifically used for: When the brightness of the monitored area is greater than or equal to a preset brightness, the target feature contour is determined based on the infrared thermal image frame and visible light image frame corresponding to any synchronization time, and based on the meteorological information of the monitored area. Specifically, this includes: When the weather information indicates a clear day, a visible light feature contour is determined based on the visible light image frame, and an infrared thermal feature contour is determined based on the infrared thermal image frame. If the overlap between the visible light feature contour and the infrared thermal feature contour is greater than a first overlap, the visible light feature contour is determined to be the target feature contour. When the weather information indicates a non-sunny day, an infrared thermal feature contour is determined based on an infrared thermal image frame, and a visible light feature contour is determined based on a visible light image frame. If the overlap between the infrared thermal feature contour and the visible light feature contour is greater than a second overlap, the infrared thermal feature contour is determined to be the target feature contour. The first degree of overlap is greater than the second degree of overlap.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the intrusion response method as described in any one of claims 1-6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the intrusion response method as described in any one of claims 1-6.

Images