Long-range laser radar and multi-camera serial fusion intrusion target detection and positioning device and method

Abstract

The present invention provides an intrusion target detection and positioning device that utilizes a serial fusion of a long-range laser radar and multiple cameras. The device comprises, at one deployment location, a first short-focus camera, a second short-focus camera, a first medium-focus camera, a second medium-focus camera, a long-range laser radar, and a fusion gateway. At one deployment location, the first short-focus camera and the first medium-focus camera are oriented in the same direction as the long-range laser radar; the second short-focus camera and the second medium-focus camera are oriented oppositely to the long-range laser radar. Furthermore, the first short-focus camera and the first medium-focus camera at one deployment location are positioned opposite the second short-focus camera and the second medium-focus camera at another adjacent deployment location. The present invention also provides an intrusion target detection and positioning method that utilizes a serial fusion of a long-range laser radar and multiple cameras. The present invention can reduce the computing power requirements of an intrusion target detection fusion algorithm.

Description

Technical Field

[0001] The present invention relates to the field of intelligent security, and in particular to an intrusion target detection and positioning device and method that integrates a long-range laser radar and multiple cameras in series for outdoor open environments and sparse target scenes. Background Art

[0002] As an important sensor in the fields of autonomous driving and three-dimensional mapping, LiDAR has three-dimensional ranging and point cloud scanning capabilities, and can obtain the distance, direction, and altitude of the target relative to the LiDAR. With the increasing number of pre-installed assisted driving and intelligent driving systems, the cost of LiDAR is rapidly decreasing, laying the foundation for the introduction of LiDAR for intrusion target detection in the intelligent security field. However, the relatively low-cost (semi-) solid-state LiDAR in the traditional intelligent driving field has problems such as the sparse point cloud of targets at close and distant distances, especially human targets. Direct use for intrusion target detection faces the problem of high cost due to the large number of LiDARs deployed per unit length, such as per kilometer. Therefore, it is necessary to use LiDAR with a small field of view and long range to reduce the overall cost of the intrusion target detection system.

[0003] Since the field of view angle and effective target detection distance of long-range lidar are usually significantly different from those of cameras, how to specifically select and design the matching number, focal length, and deployment method of cameras based on indicators such as the field of view angle and detection distance of long-range lidar, while taking into account the requirements of the two-dimensional image target detection algorithm for the minimum resolution of the detected target, needs further optimization.

[0004] Because laser point clouds are relatively sparse and lack essential texture information, the autonomous driving field often uses multimodal fusion methods that combine laser point clouds and multiple cameras to improve target detection performance, while also providing perception redundancy and enhancing the reliability of the perception system. Specific fusion methods typically include fusion at the raw data level (also known as front-end fusion or data-level fusion), fusion at the deep neural network (DNN) feature level (also known as deep fusion or feature-level fusion), and fusion of target detection results (also known as back-end fusion or target-level fusion). Regardless of the method, both require the camera-based deep neural network target detection algorithm and the laser point cloud processing algorithm to run in parallel, a process known as parallel fusion. This parallel fusion mode, on the one hand, results in high computational complexity and computing power requirements, which in turn increases system cost. On the other hand, it results in excessively high overall power consumption, making it unsuitable for applications lacking power supply infrastructure. Summary of the Invention

[0005] To address at least one technical problem in the prior art, embodiments of the present invention provide an intrusion target detection and positioning device and method that utilizes a serial fusion of a long-range laser radar and multiple cameras. This method employs a serial fusion mode that, after pre-detecting and locating intrusion targets based on laser point cloud data, decodes the corresponding camera's video stream in real time and performs two-dimensional image target classification. This reduces the computing power requirements of the intrusion target detection fusion algorithm and the overall power consumption of the intrusion target detection and positioning device. To achieve the above technical objectives, the technical solutions employed in the embodiments of the present invention are:

[0006] In a first aspect, an embodiment of the present invention provides an intrusion target detection and positioning device that integrates a long-range laser radar and multiple cameras in series, comprising: a first short-focus camera, a second short-focus camera, a first medium-focus camera, a second medium-focus camera, a long-range laser radar, and a fusion gateway at a deployment location;

[0007] At one deployment location, the first short-focus camera and the first medium-focus camera are oriented in the same direction as the long-range laser radar; the second short-focus camera and the second medium-focus camera are oriented opposite to the long-range laser radar; and the first short-focus camera and the first medium-focus camera at one deployment location are respectively arranged opposite to the second short-focus camera and the second medium-focus camera at another adjacent deployment location; the long-range laser radar at one deployment location faces the long-range laser radar at another adjacent deployment location;

[0008] The deployment interval of the two intrusion target detection and positioning devices is less than or equal to the distance at which the long-range laser radar can accurately detect human intrusion targets; within a distance half of the deployment interval, the first short-focus camera and the second short-focus camera are used for target detection within the short-range range, and the first medium-focus camera and the second medium-focus camera are used for target detection within the medium-range range; the short-range range and the medium-range range are connected;

[0009] The first short-focus camera, the second short-focus camera, the first medium-focus camera, the second medium-focus camera and the long-range laser radar are respectively connected to a fusion gateway.

[0010] Furthermore, the fusion gateway includes a main control chip; the main control chip is connected to the first short-focus camera, the second short-focus camera, the first medium-focus camera, the second medium-focus camera and the long-range laser radar through the Ethernet port; the main control chip is connected to the memory and the mobile communication module through the PCIe interface.

[0011] Furthermore, the main control chip includes a neural processor NPU and a video processor VPU.

[0012] Furthermore, the main control chip is connected to the first short-focus camera, the second short-focus camera, the first medium-focus camera, and the second medium-focus camera through the Gigabit Ethernet port provided by the Gigabit Ethernet interface expansion chip.

[0013] Furthermore, the main control chip is also connected to an uplink optical terminal through a Gigabit Ethernet port.

[0014] In a second aspect, an embodiment of the present invention provides an intrusion target detection and positioning method using a long-range laser radar and multiple cameras in serial fusion, comprising the following steps:

[0015] Step S10: During installation and deployment, a three-dimensional laser radar mapping algorithm is first used to construct a static three-dimensional map of the environment for each long-range laser radar monitoring area.

[0016] The LiDAR 3D mapping algorithm uses the normal distribution transform (NDT). First, the 3D space of the mapping area is divided into cubic voxels (Voxels) with fixed side lengths. Then, each point in a frame of point cloud data scanned by the LiDAR is assigned to a corresponding cubic voxel according to its coordinate value. Based on the points in the point cloud data that fall within the cubic voxel, the mean and variance of the point coordinate values ​​within each cubic voxel are calculated and saved.

[0017] Step S20, setting the shape of the long-range laser radar monitoring area, and filtering out the static environment three-dimensional map data outside the monitoring area according to the shape of the monitoring area;

[0018] Step S30: Using the long-range laser radar and the camera extrinsic matrix, based on the set monitoring area shape, generate rectangular areas corresponding to the monitoring areas at different positions in each camera image by distance segmentation, which are called regions of interest;

[0019] Step S40: After the installation and deployment of the intrusion target detection and positioning device is completed; each intrusion target detection and positioning device is restarted; the fusion gateway loads a static environment three-dimensional map of the corresponding monitoring area;

[0020] Step S50: After the long-range LiDAR is operational, the fusion gateway receives a frame of point cloud data sent by the long-range LiDAR. The mean and variance of the point coordinate values ​​within each cubic voxel (Voxel) are calculated and compared with the mean and variance of the point coordinate values ​​within the corresponding cubic voxel in the stored static environment 3D map. If the difference exceeds a specified threshold, the presence of an intruder is determined within the cubic voxel.

[0021] If there is no intrusion target, repeat step S50; if there is an intrusion target, proceed to step S60;

[0022] In step S60, the fusion gateway selects the corresponding camera's video stream for decoding based on the voxel location of the intrusion target. The fusion gateway extracts frames from the decoded video stream and crops the image based on the distance-segmented regions of interest generated in step S30. Only the image corresponding to the region of interest is fed into the 2D image target detection algorithm in the fusion gateway for target classification and detection.

[0023] Step S70: When the detected target is of the target category of interest, the fusion gateway stores a video of a specified duration and a specified number of on-site pictures;

[0024] When the detected target is not the target category of interest, the process returns to step S50.

[0025] Furthermore, in step S10, the average number of points and the variance of the number of points in each frame of point cloud data falling into the cubic voxel Voxel are counted for each cubic voxel, and are saved as part of the static environment three-dimensional map;

[0026] In step S50, the fusion gateway also counts the number of points falling into each cubic voxel Voxel and calculates the variance of the number of points, and compares it with the average number of points and the variance of the number of points in the corresponding cubic voxel Voxel in the stored static environment three-dimensional map. When the difference exceeds the corresponding specified threshold, and the difference between the calculated mean value and variance of the point coordinate values ​​in each cubic voxel Voxel and the mean value and variance of the point coordinate values ​​in the corresponding cubic voxel Voxel in the stored static environment three-dimensional map also exceeds the corresponding specified threshold, it is determined that an intrusion target exists in the cubic voxel Voxel.

[0027] Furthermore, in step S10, for a cubic voxel Voxel in which no point falls, the mean value of the point coordinate value is set to 0, and the variance of the point coordinate value is set to -1.

[0028] Optionally, during the execution of steps S60 and S70, the fusion gateway no longer receives point cloud data from the long-range lidar and does not perform the operation of step S50.

[0029] The technical solution provided by the embodiment of the present invention has the following beneficial effects:

[0030] 1) Targeted selection of medium- to low-cost, small-field-of-view (high-point-cloud-density) long-range LiDARs to achieve long-range detection of small intrusion targets such as the human body, thereby reducing the number of LiDARs deployed per unit length.

[0031] 2) For open outdoor environments and sparse target scenes, a comprehensive design of the serial fusion mode of long-range lidar and multiple cameras, the generation of regions of interest by distance segmentation, and image cropping based on the regions of interest are used to reduce the computing power requirements of the two-dimensional image target detection algorithm. This not only avoids the cost increase brought by the introduction of high-computing power computing units, but also realizes the time-sharing operation of the fusion gateway laser point cloud processing computing power and the two-dimensional image target detection algorithm. It can even realize the on-demand startup of the long-range lidar and multiple cameras, reducing the operating power consumption of the intrusion target detection and positioning device and improving the working life of the long-range lidar and multiple cameras.

[0032] 3) To address the characteristics of long-range lidar with high point cloud density, the average number of points falling within a cubic voxel in each frame of point cloud data and its variance are introduced into the static environment 3D map. This can significantly reduce the jitter of the point cloud count caused by lidar measurement errors and fixed-size voxel segmentation, as well as false alarms caused by randomly distributed noise generated in the environment under rainy, snowy, and foggy weather conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of the deployment of an intrusion target detection and positioning device in an embodiment of the present invention.

[0034] Figure 2 Schematic diagram of the structure of an intrusion target detection and positioning device in an embodiment of the present invention.

[0035] Figure 3 This is a flow chart of the intrusion target detection and positioning method in an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0037] In the description of the embodiments of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate the description of the present invention and simplify the description. They do not indicate or imply that the devices or components referred to must have a specific orientation, be constructed, or operate in a specific orientation. Therefore, they should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0038] In the description of the embodiments of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; they can refer to internal connections between two components; and they can refer to wireless connections or wired connections. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on specific circumstances.

[0039] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0040] like Figure 1 As shown, an embodiment of the present invention provides an intrusion target detection and positioning device that integrates a long-range laser radar and multiple cameras in series (hereinafter referred to as the intrusion target detection and positioning device), which includes: a first short-focus camera 1, a second short-focus camera 2, a first medium-focus camera 3, a second medium-focus camera 4, a long-range laser radar 5, and a fusion gateway 6 at a deployment location;

[0041] At one deployment location, the first short-focus camera 1 and the first medium-focus camera 3 are oriented in the same direction as the long-range laser radar 5; the second short-focus camera 2 and the second medium-focus camera 4 are oriented opposite to the long-range laser radar 5; and the first short-focus camera 1 and the first medium-focus camera 3 at one deployment location are respectively arranged opposite to the second short-focus camera 2 and the second medium-focus camera 4 at another adjacent deployment location; the long-range laser radar 5 at one deployment location faces the long-range laser radar 5 at another adjacent deployment location;

[0042] The deployment interval of the two intrusion target detection and positioning devices is less than or equal to the distance at which the long-range laser radar 5 can accurately detect human intrusion targets; within a distance half of the deployment interval, the first short-focus camera 1 and the second short-focus camera 2 are used for target detection within the short-range range, and the first medium-focus camera 3 and the second medium-focus camera 4 are used for target detection within the medium-range range; the short-range range and the medium-range range are connected;

[0043] The first short-focus camera 1 , the second short-focus camera 2 , the first medium-focus camera 3 , the second medium-focus camera 4 and the long-range laser radar 5 are respectively connected to a fusion gateway 6 .

[0044] exist Figure 1 In a specific embodiment shown, the deployment interval of the intrusion target detection and positioning device is 160 meters, that is, an intrusion target detection and positioning device is deployed every 160 meters. The distance of 160 meters corresponds to the distance at which the long-range laser radar 5 can accurately detect the human intrusion target; Figure 1 The direction of the middle arrow corresponds to the direction of the long-range laser radar 5 and each camera; the first short-focus camera 1 and the second short-focus camera 2 are responsible for target detection in the short-range range of 0 to 30 meters, and the first medium-focus camera 3 and the second medium-focus camera 4 are responsible for target detection in the medium-range range of 30 to 80 meters; thus, full coverage of the 160-meter area is achieved through the four cameras facing opposite directions in two adjacent intrusion target detection and positioning devices.

[0045] Specifically, if Figure 2 As shown, the fusion gateway 6 includes a main control chip; the main control chip is connected to the first short-focus camera 1, the second short-focus camera 2, the first medium-focus camera 3, the second medium-focus camera 4 and the long-range laser radar 5 respectively through the Ethernet port; the main control chip is connected to the memory and the mobile communication module through the PCIe interface; specifically, the main control chip in this embodiment adopts a multi-core ARM main control chip such as Rockchip's RK3566 or RK3588; the memory adopts a large-capacity solid-state drive (SSD) to realize local storage of the target intrusion event video and on-site screenshots; the mobile communication module adopts an extended 4G / 5G module to realize the upload of the target intrusion event video;

[0046] Furthermore, the main control chip includes a neural processor NPU and a video processor VPU; taking the currently lowest-priced Rockchip RK3566 as an example, it supports up to 8GB LPDDR4 high-speed memory and has a neural processor NPU (Neural Process Unit) with a computing power of 0.8TOPS@INT8, which can run various two-dimensional image target detection algorithms such as YOLO; it also includes a video processor VPU (Video Process Unit) that supports H.265, H264, and VP9 format videos with a resolution of 4096x2304 at 60 frames per second hard decoding and H.265 and H264 format videos with a resolution of 1920x1080 at 60 frames per second hard encoding.

[0047] Furthermore, the main control chip is connected to the first short-focus camera 1, the second short-focus camera 2, the first medium-focus camera 3 and the second medium-focus camera 4 through the Gigabit Ethernet port provided by the Gigabit Ethernet interface expansion chip; Rockchip RK3566 has two built-in 10M / 100M / 1000M adaptive Ethernet ports and a PCIe3.0 / 2.1 physical interface, which can be expanded to 4 or more Gigabit Ethernet ports through Realtek's RTL8211 series chip.

[0048] Optionally, the main control chip is further connected to an uplink optical terminal via a Gigabit Ethernet port to implement a data upload function based on optical fiber.

[0049] Specifically, the long-range LiDAR 5 uses the Livox Tele-15 LiDAR, which features a narrow field of view and long detection range. Its operating wavelength is 905nm, with a range accuracy of 2cm and a detection range of 320m at 10% reflectivity, significantly exceeding the detection range of 905nm automotive (semi-)solid-state LiDARs. The Tele-15 has a horizontal field of view of 14.5° and a vertical field of view of 16.2°. This narrow field of view ensures high point cloud density, allowing a person facing the LiDAR to still receive at least 30 return points at a distance of 200 meters.

[0050] The embodiment of the present invention further provides a method for detecting and locating intruder targets by serially fusing a long-range laser radar with multiple cameras, comprising the following steps:

[0051] Step S10: During installation and deployment, a three-dimensional laser radar mapping algorithm is first used to construct a static three-dimensional map of the environment for each long-range laser radar monitoring area.

[0052] The LiDAR 3D mapping algorithm uses the Normal Distribution Transform (NDT). It first divides the 3D space of the mapping area into cubic voxels (Voxels) with a fixed side length (e.g., 0.5 meters). Each point in a frame of point cloud data scanned by the LiDAR is then assigned to a corresponding cubic voxel based on its coordinates (including X-axis, Y-axis, and Z-axis coordinates). The mean and variance of the point coordinates within each cubic voxel are calculated and saved based on the points in the point cloud data that fall within the cubic voxel.

[0053] In this step, for any cubic voxel that does not have any point falling into it, the point coordinate value mean is set to 0 and the point coordinate value variance is set to -1;

[0054] In order to improve the robustness of the map and reduce the impact of laser point cloud measurement noise, a multi-frame fusion method can be used, that is, the point cloud data returned by multiple laser radar scans can be used for NDT mapping;

[0055] To reduce the noise point cloud caused by adverse weather conditions such as rain, snow, and fog, during the mapping process, the average number of points and the variance of the number of points that fall into each cubic voxel in each frame of point cloud data are counted and saved as part of the static environment 3D map.

[0056] Step S20, setting the shape of the long-range laser radar monitoring area, and filtering out the static environment three-dimensional map data outside the monitoring area according to the shape of the monitoring area;

[0057] Through this step, the storage space occupied by the static environment 3D map data can be reduced and the map loading speed can be accelerated;

[0058] Step S30: Using the long-range laser radar and the camera extrinsic matrix, based on the set monitoring area shape, generate rectangular areas corresponding to the monitoring areas at different positions in each camera image by distance segmentation, which are called regions of interest;

[0059] For example, the 0-15 meter monitoring area corresponds to the area of ​​interest in the first short-focus camera 1, the 15-30 meter monitoring area corresponds to the area of ​​interest in the first short-focus camera 1, the 30-50 meter monitoring area corresponds to the area of ​​interest in the first medium-focus camera 3, and the 50-80 meter monitoring area corresponds to the area of ​​interest in the first medium-focus camera 3, etc.

[0060] The above steps S10 to S30 are performed when the intrusion target detection and positioning device is deployed on site for the first time;

[0061] Step S40: After the installation and deployment of the intrusion target detection and positioning device is completed; each intrusion target detection and positioning device is restarted; the fusion gateway loads a static environment three-dimensional map of the corresponding monitoring area;

[0062] Step S50: After the long-range LiDAR is operational, the fusion gateway receives a frame of point cloud data sent by the long-range LiDAR. The mean and variance of the point coordinate values ​​within each cubic voxel (Voxel) are calculated and compared with the mean and variance of the point coordinate values ​​within the corresponding cubic voxel in the stored static environment 3D map. If the difference exceeds a specified threshold, the presence of an intruder is determined within the cubic voxel.

[0063] Since the voxel where the intrusion target is located can also be known, the intrusion target can also be accurately located;

[0064] If there is no intrusion target, repeat step S50; if there is an intrusion target, proceed to step S60;

[0065] In this step, the fusion gateway can also count the number of points falling into each cubic voxel and calculate the variance of the number of points. The number of points is compared with the average number of points and the variance of the number of points in the corresponding cubic voxel in the stored static environment 3D map. When the difference exceeds the corresponding specified threshold, and the difference between the calculated mean value and variance of the point coordinate values ​​in each cubic voxel and the mean value and variance of the point coordinate values ​​in the corresponding cubic voxel in the stored static environment 3D map also exceeds the corresponding specified threshold, it is determined that an intrusion target exists in the cubic voxel.

[0066] In step S60, the fusion gateway selects the corresponding camera's video stream for decoding based on the voxel location of the intrusion target. The fusion gateway extracts frames from the decoded video stream, for example, 1 to 2 frames per second, and simultaneously crops the image based on the distance-segmented regions of interest generated in step S30. Only the image corresponding to the region of interest is fed into the 2D image target detection algorithm in the fusion gateway for target classification and detection.

[0067] In this embodiment, the video stream is hard-decoded by the video processor VPU, and the neural processor NPU runs a two-dimensional image target detection algorithm such as YOLO to perform target classification and detection;

[0068] Step S70: When the detected target is a target of interest, such as a person, the fusion gateway stores a video of a specified duration and a specified number of on-site pictures; and can upload the real-time video of the scene to the control center according to the configuration option;

[0069] When the detected target is not the target category of interest, the process returns to step S50.

[0070] Optionally, during the execution of steps S60 and S70, the fusion gateway no longer receives point cloud data from the long-range lidar and does not perform the operation of step S50; this can reduce the overall power consumption of the intrusion target detection and positioning device and save the computing power of the fusion gateway main control chip, so as to retain sufficient computing power to quickly complete the post-processing work of the two-dimensional image target detection algorithm.

[0071] Finally, it should be noted that the above specific implementation methods are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that the technical solutions of the present invention can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A long-range laser radar and multi-camera serial fusion intrusion target detection and positioning device, characterized in that: The system comprises, at a deployment location, a first short-focus camera (1), a second short-focus camera (2), a first medium-focus camera (3), a second medium-focus camera (4), a long-range laser radar (5), and a fusion gateway (6); At a deployment position, the orientation of the first short-focus camera (1) and the first medium-focus camera (3) is the same as that of the long-range laser radar (5); the orientation of the second short-focus camera (2) and the second medium-focus camera (4) is opposite to that of the long-range laser radar (5); and the first short-focus camera (1) and the first medium-focus camera (3) at a deployment position are respectively arranged opposite to the second short-focus camera (2) and the second medium-focus camera (4) at another adjacent deployment position; the long-range laser radar (5) at one deployment position faces the long-range laser radar (5) at another adjacent deployment position; The deployment interval of the two intrusion target detection and positioning devices is less than or equal to the distance at which the long-range laser radar (5) can accurately detect the human intrusion target; within a distance half of the deployment interval, the first short-focus camera (1) and the second short-focus camera (2) are used for target detection within the short-range range, and the first medium-focus camera (3) and the second medium-focus camera (4) are used for target detection within the medium-range range; the short-range range and the medium-range range are connected; The first short-focus camera (1), the second short-focus camera (2), the first medium-focus camera (3), the second medium-focus camera (4), and the long-range laser radar (5) are respectively connected to a fusion gateway (6); The intrusion target detection and positioning device using a long-range laser radar and multiple cameras in series is applicable to an intrusion target detection and positioning method using a long-range laser radar and multiple cameras in series, and the method comprises the following steps: Step S10: During installation and deployment, a three-dimensional laser radar mapping algorithm is first used to construct a static three-dimensional map of the environment for each long-range laser radar monitoring area. The LiDAR 3D mapping algorithm uses the normal distribution transform (NDT). First, the 3D space of the mapping area is divided into cubic voxels (Voxels) with fixed side lengths. Then, each point in a frame of point cloud data scanned by the LiDAR is assigned to a corresponding cubic voxel according to its coordinate value. Based on the points in the point cloud data that fall within the cubic voxel, the mean and variance of the point coordinate values ​​within each cubic voxel are calculated and saved. Step S20, setting the shape of the long-range laser radar monitoring area, and filtering out the static environment three-dimensional map data outside the monitoring area according to the shape of the monitoring area; Step S30: Using the long-range laser radar and the camera extrinsic matrix, based on the set monitoring area shape, generate rectangular areas corresponding to the monitoring areas at different positions in each camera image by distance segmentation, which are called regions of interest; Step S40: After the installation and deployment of the intrusion target detection and positioning device is completed; each intrusion target detection and positioning device is restarted; the fusion gateway loads a static environment three-dimensional map of the corresponding monitoring area; Step S50: After the long-range LiDAR is operational, the fusion gateway receives a frame of point cloud data sent by the long-range LiDAR. The mean and variance of the point coordinate values ​​within each cubic voxel (Voxel) are calculated and compared with the mean and variance of the point coordinate values ​​within the corresponding cubic voxel in the stored static environment 3D map. If the difference exceeds a specified threshold, the presence of an intruder is determined within the cubic voxel. If there is no intrusion target, repeat step S50; if there is an intrusion target, proceed to step S60; In step S60, the fusion gateway selects the corresponding camera's video stream for decoding based on the voxel location of the intrusion target. The fusion gateway extracts frames from the decoded video stream and crops the image based on the distance-segmented regions of interest generated in step S30. Only the image corresponding to the region of interest is fed into the 2D image target detection algorithm in the fusion gateway for target classification and detection. Step S70: When the detected target is of the target category of interest, the fusion gateway stores a video of a specified duration and a specified number of on-site pictures; When the detected target is not the target category of interest, the process returns to step S50.

2. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 1 is characterized in that: The fusion gateway (6) comprises a main control chip; the main control chip is respectively connected to the first short-focus camera (1), the second short-focus camera (2), the first medium-focus camera (3), the second medium-focus camera (4) and the long-range laser radar (5) via an Ethernet port; the main control chip is connected to the memory and the mobile communication module via a PCIe interface.

3. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 2 is characterized in that: The main control chip includes a neural processor NPU and a video processor VPU.

4. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 2 is characterized in that: The main control chip is connected to the first short-focus camera (1), the second short-focus camera (2), the first medium-focus camera (3) and the second medium-focus camera (4) via the Gigabit Ethernet port provided by the Gigabit Ethernet interface expansion chip.

5. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 2 is characterized in that: The main control chip is also connected to the uplink optical terminal through the Gigabit Ethernet port.

6. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 1 is characterized in that: In step S10, the average number of points and the variance of the number of points in each frame of point cloud data falling into the cubic voxel Voxel are counted for each cubic voxel, and saved as part of the static environment three-dimensional map; In step S50, the fusion gateway also counts the number of points falling into each cubic voxel Voxel and calculates the variance of the number of points, and compares it with the average number of points and the variance of the number of points in the corresponding cubic voxel Voxel in the stored static environment three-dimensional map. When the difference exceeds the corresponding specified threshold, and the difference between the calculated mean value and variance of the point coordinate values ​​in each cubic voxel Voxel and the mean value and variance of the point coordinate values ​​in the corresponding cubic voxel Voxel in the stored static environment three-dimensional map also exceeds the corresponding specified threshold, it is determined that an intrusion target exists in the cubic voxel Voxel.

7. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 1 is characterized in that: In step S10 , for a cubic voxel Voxel in which no point falls, the mean value of the point coordinate value is set to 0, and the variance of the point coordinate value is set to -1.

8. The intrusion target detection and positioning device of the long-range laser radar and multi-camera serial fusion according to claim 1 is characterized in that: During the execution of steps S60 and S70, the fusion gateway no longer receives point cloud data from the long-range lidar and does not perform the operation of step S50.

Images