Security monitoring method and device, electronic equipment and storage medium

Abstract

The invention provides a security monitoring method. The method comprises the following steps: acquiring three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space; based on the three-dimensional point cloud data, projecting each three-dimensional point to a plurality of two-dimensional projection planes to obtain corresponding projection points of the three-dimensional points in the plurality of two-dimensional projection planes, each three-dimensional point corresponding to one projection point in one two-dimensional projection plane; determining target three-dimensional points located in the second three-dimensional space and the number of the target three-dimensional points based on projection points corresponding to the three-dimensional points in the plurality of two-dimensional projection planes; and if the number of the target three-dimensional points is greater than a preset number threshold, outputting a safety control signal. The method solves the problems that an existing method is long in operation time and cannot accurately process point cloud on the rear side of a projection plane.

Description

Technical Field

[0001] The present invention relates to the field of visual technology, and in particular to a security monitoring method, device, electronic equipment and storage medium. Background Art

[0002] Stereo safety sensors, also known as Time-of-Flight (ToF) cameras or 3D TOF cameras, are mainly used for stereo protection in the industrial field. Their working principle is to calculate the distance information by measuring the time it takes for light to be emitted from the stereo safety sensor to the object in the detection area and then reflected back to the camera, thereby obtaining the three-dimensional spatial data of the scene. When a target object invades the detected area, it will output a safety control signal to the host computer or relay, thereby controlling the shutdown of the working equipment.

[0003] At present, when dividing the detection area, the traditional method sets the detection area as a cube defined by vertices ABCDEFGH with the optical center O of the stereo safety sensor as the center, and the six quadrangular pyramids (OABEH, OCDFG, OABCD, OEFGH, OADEF, OBCGH) in the cube as sub-areas for judgment. Since it is necessary to process the multi-sided pyramid area connected to the optical center of the sensor, the calculation complexity is high each time the two-dimensional projection surface is transformed, which leads to a long operation time and the inability to accurately process the point cloud problem behind the projection plane. Summary of the invention

[0004] The embodiment of the present invention provides a security monitoring method that can accurately process the point cloud problem behind the projection plane. By obtaining the three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space; based on the three-dimensional point cloud data, each three-dimensional point is projected to multiple two-dimensional projection planes to obtain the projection points corresponding to the three-dimensional point in the multiple two-dimensional projection planes, and according to the projection points corresponding to the three-dimensional point in the multiple two-dimensional projection planes, the target three-dimensional point and the number of target three-dimensional points in the second three-dimensional space are determined, and when the number of target three-dimensional points is greater than a preset number threshold, a security control signal is output, which solves the problem that the existing method has a long operation time and cannot accurately process the point cloud problem behind the projection plane.

[0005] In a first aspect, an embodiment of the present invention provides a security monitoring method, the method comprising the following steps:

[0006] Obtaining three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space;

[0007] Based on the three-dimensional point cloud data, project each of the three-dimensional points onto a plurality of two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, wherein each of the three-dimensional points corresponds to one projection point in one of the two-dimensional projection planes;

[0008] Determine, based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, target three-dimensional points located in the second three-dimensional space and the number of the target three-dimensional points;

[0009] If the number of the target three-dimensional points is greater than a preset number threshold, a safety control signal is output.

[0010] Optionally, determining a target three-dimensional point located in the second three-dimensional space based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes includes:

[0011] In each of the two-dimensional projection planes, determining whether the projection point corresponding to each of the three-dimensional points is located in a reference projection area corresponding to the two-dimensional projection plane, the reference projection area being a projection area of ​​a second three-dimensional space in the two-dimensional projection plane, and the second three-dimensional space is located in the first three-dimensional space;

[0012] If the projection points of the three-dimensional point in each of the two-dimensional projection planes are located in the reference projection area corresponding to the two-dimensional projection plane, the three-dimensional point is determined to be a target three-dimensional point located in the second three-dimensional space.

[0013] Optionally, the three-dimensional point cloud data includes three-dimensional coordinate values ​​of the three-dimensional points, and projecting each of the three-dimensional points onto a plurality of two-dimensional projection planes based on the three-dimensional point cloud data includes:

[0014] Determining the normal direction of the two-dimensional projection plane;

[0015] In the normal direction, based on the three-dimensional coordinate value of the three-dimensional point, a two-dimensional coordinate value of the three-dimensional point on the two-dimensional projection plane is calculated;

[0016] The three-dimensional point is projected onto a plurality of two-dimensional projection planes based on the two-dimensional coordinate value of the three-dimensional point on the two-dimensional projection plane.

[0017] Optionally, determining, in each of the two-dimensional projection planes, whether the projection point corresponding to each of the three-dimensional points is located in a reference projection area corresponding to the two-dimensional projection plane includes:

[0018] In the two-dimensional projection plane, determine at least one group of boundary vectors of the reference projection area and a starting point corresponding to each group of boundary vectors, wherein a group of boundary vectors includes two boundary vectors with a common starting point, and each boundary vector is collinear with a boundary of the reference projection area;

[0019] Based on the starting point and the projection point corresponding to the three-dimensional point, determining a projection vector corresponding to the three-dimensional point;

[0020] It is determined, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane.

[0021] Optionally, determining, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane includes:

[0022] For a set of the boundary vectors, calculating a first cross product between two of the boundary vectors, and calculating a second cross product between the boundary vector and the projection vector;

[0023] Based on the first cross product and the second cross product, it is determined whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane.

[0024] Optionally, determining the projection vector corresponding to the three-dimensional point based on the starting point and the projection point corresponding to the three-dimensional point includes:

[0025] In the two-dimensional projection plane, clustering all the projection points according to the two-dimensional distances between the projection points to obtain at least one cluster, wherein the cluster includes at least one projection point;

[0026] If the cluster includes only one projection point, determining a projection vector corresponding to the three-dimensional point based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the projection point;

[0027] If the cluster includes a plurality of the projection points, a target circle corresponding to the cluster is determined with the centroid of the cluster as the center of the circle and the largest two-dimensional distance radius, and each cluster corresponds to one target circle;

[0028] Draw a tangent line to the target circle through the starting point, and determine a first tangent point and a second tangent point on the target circle corresponding to the starting point;

[0029] Based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the first tangent point, a first projection vector corresponding to all three-dimensional points in the cluster cluster is determined, and based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the second tangent point, a second projection vector corresponding to all three-dimensional points in the cluster cluster is determined.

[0030] Optionally, determining, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane includes:

[0031] For a set of the boundary vectors, calculating a first cross product between two of the boundary vectors, calculating a third cross product between the boundary vector and the first projection vector, and calculating a fourth cross product between the boundary vector and the second projection vector;

[0032] Based on the product of the first cross product and the third cross product and the product of the first cross product and the fourth cross product, it is determined whether the projection points corresponding to all three-dimensional points in the cluster are located in the reference projection area corresponding to the two-dimensional projection plane.

[0033] Optionally, the acquiring of the second three-dimensional space also includes a third three-dimensional space, and the method further includes:

[0034] The point cloud data in the third three-dimensional space is eliminated.

[0035] In a second aspect, an embodiment of the present invention further provides a security monitoring device, the security monitoring device comprising:

[0036] An acquisition module, used to acquire three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space;

[0037] A projection module, configured to project each of the three-dimensional points onto a plurality of two-dimensional projection planes based on the three-dimensional point cloud data, to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, wherein each of the three-dimensional points corresponds to one projection point in one of the two-dimensional projection planes;

[0038] A determination module, configured to determine a target three-dimensional point located in a second three-dimensional space and the number of the target three-dimensional points based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes;

[0039] The safety control module is used to output a safety control signal if the number of the target three-dimensional points is greater than a preset number threshold.

[0040] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the security monitoring method provided in the embodiment of the present invention when executing the computer program.

[0041] In an embodiment of the present invention, three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space is obtained; based on the three-dimensional point cloud data, each three-dimensional point is projected onto multiple two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, and each three-dimensional point corresponds to one projection point in one two-dimensional projection plane; based on the projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, the target three-dimensional point and the number of target three-dimensional points located in the second three-dimensional space are determined; if the number of target three-dimensional points is greater than a preset number threshold, a safety control signal is output. The present invention obtains three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space; based on the three-dimensional point cloud data, each three-dimensional point is projected onto multiple two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, and according to the projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, the target three-dimensional point and the number of target three-dimensional points located in the second three-dimensional space are determined, and when the number of target three-dimensional points is greater than the preset number threshold, a safety control signal is output, thereby solving the problem that the existing method has a long operation time and cannot accurately process the point cloud behind the projection plane. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 is a flow chart of a security monitoring method provided by an embodiment of the present invention;

[0044] Figure 2 is a structural schematic diagram of a security monitoring device provided by an embodiment of the present invention;

[0045] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] like Figure 1 As shown, Figure 1: is a flow chart of a security monitoring method provided by an embodiment of the present invention, the security monitoring method comprising the steps of:

[0048] 101. Obtain three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space.

[0049] In an embodiment of the present invention, the security monitoring method can be applied to a security monitoring system, which includes a host computer and a point cloud data acquisition device, which can be a three-dimensional camera, a depth camera or a time-of-flight camera, such as a stereo security sensor. The point cloud data acquisition device is used to collect point cloud data of the corresponding area.

[0050] The above security monitoring method is mainly used for the host computer of the security monitoring system. The above point cloud data can be understood as the three-dimensional coordinate data (x, y, z) of each pixel point in the depth map collected by the stereo security sensor.

[0051] The above-mentioned point cloud data can also be called 3D point cloud data. The collection of point cloud data in this embodiment can be based on the iTOF ranging principle. Through CW modulation driven by VCSEL, VCSEL modulates light with a frequency of f. Because it is necessary to analyze the phase difference between the reflected light and the emitted light to calculate the distance, in order to avoid multiple solutions, only the distance of the target within 1 / 2 wavelength can be measured, that is, the maximum ranging is 1 / 2 wavelength of the modulated light wave. The two frequencies of 15Mhz and 120Mhz adopted in this example, the theoretical maximum detection distance at 15Mhz frequency is 10m, and the theoretical maximum detection distance at 120Mhz frequency is 1.25m. The theoretical maximum detection distance of dual-frequency rotation is the least common multiple of the theoretical maximum detection distances corresponding to the two frequencies, that is, the greatest common multiple of 1.5m and 10m, that is, the theoretical maximum detection distance under the conditions of 15Mhz and 120Mhz rotation is 10m.

[0052] The distortion problem of the depth map is solved one by one through the camera intrinsic calibration. In the intrinsic calibration module, the common "pinhole + distortion" model in computer vision can be used to characterize the imaging system parameters such as lens distortion and principal point offset. The parameters include:

[0053] 1. Main point c x 、c y , represents the pixel coordinates of the intersection of the lens optical axis and the imaging target surface.

[0054] 2. Focal length f x 、f y , which represents the ratio of the lens focal length to the pixel size.

[0055] 3. Distortion parameters k1, k2, k 3；, p1, p2, where k1, k2, k3 are parameters representing the radial distortion of the lens, and p1 and p2 are parameters representing the tangential distortion of the lens. In the internal parameter calibration module, the calibration method is used to solve the model parameters. The camera is controlled to collect the intensity image of the calibration plate. Given the coordinates of the marking points in the calibration plate, the features of the marking points in the image are extracted to obtain the image coordinates of the marking points, the objective function is established, and the LM method is used for optimization and solution. Finally, the principal point c is obtained. x 、c y , focal length f x 、f y , and distortion parameters k1, k2, k3, p1, p2, and write the calibration results into the camera.

[0056] During the depth calibration (phase calibration) process, the measurement accuracy of TOF is affected by many factors. The depth calibration module mainly considers three items:

[0057] 1. When measuring an object with a distance of zero, when a zero-phase signal is transmitted, the delay difference between the phase of the signal received by RX (RF receiver) and the zero-phase signal is called zero drift.

[0058] 2. Characterize the zero drift that indicates the difference in the initial exposure time of different pixels, i.e., the fixed phase template noise.

[0059] 3. The deviation between the light waveform emitted by the TX (radio frequency transmitter) and the cosine signal causes harmonic errors that vary with distance.

[0060] The absolute errors introduced by the above items are usually in the order of centimeters. To ensure the accuracy of the distance measurement results, the above errors of each stereo safety sensor must be calibrated one by one before leaving the factory.

[0061] In order to calibrate the above errors, it is necessary to know the real distance of the object to be measured, collect TOF images at the same time, and calculate the actual measured distance. An error model is established between the two, and the model parameters are solved. In specific implementation, the stereo safety sensor is placed on the track table, and 90% diffuse reflection planes are photographed at different distances to collect TOF data. The distance between the stereo safety sensor and the diffuse reflection plane given by the track table and the internal parameter calibration model are used to calculate the real distance of each pixel from the diffuse reflection plane. Call the depth calibration algorithm to process the collected TOF data, calculate the actual measurement distance, and optimize the model parameters.

[0062] The depth calculation algorithm completes the calculation from the raw image to the tap image, the tap image to the intensity image, the amplitude image, the phase image, the phase image to the distance image, the distance image to the depth image, and the depth image to the point cloud image. The depth calculation algorithm also has the following two functions:

[0063] 1. In the process of calculating the phase, the depth calculation algorithm needs to call the calibration results to compensate the calculated phase to eliminate the system error and obtain higher measurement accuracy.

[0064] 2. Because the stereo safety sensor works in dual-frequency mode, the depth calculation algorithm needs to fuse the dual-frequency data and output a higher quality measurement result after fusion.

[0065] The depth filtering algorithm is effective at different stages of the depth calculation algorithm, processing the intermediate quantities of the calculation, filtering out errors and low signal-to-noise ratio results, and ensuring the quality of the output results. The depth filtering algorithm consists of five modules:

[0066] 1. Temporal filtering module: averages multiple frames of the depth map to improve the signal-to-noise ratio. You can set the temporal filtering algorithm on and off, as well as the fusion weight of the current frame.

[0067] 2. Spatial filtering module: Perform spatial Gaussian filtering on the 1Q image to improve the signal-to-noise ratio. You can set the spatial filtering algorithm on and off, as well as the size of the Gaussian filtering window.

[0068] 3. Amplitude filter module: Set a certain threshold to filter out pixel depths with amplitudes below this threshold. You can set the amplitude filter algorithm on and off, as well as the amplitude filter threshold.

[0069] 4. Fusion error filter module: Calculate the dual-frequency fusion error, and when the phase is greater than the set value, filter out the corresponding pixel depth. You can set the fusion error filter algorithm to be on or off, as well as the size of the interval value.

[0070] 5. Flying point filter module: convert the depth into point cloud, calculate the distance between the point and the surrounding points, and filter out the corresponding pixel depth when the minimum distance is greater than the set value. You can set the flying point filter to be on or off, as well as the size of the distance value.

[0071] The following is an example of point cloud generation:

[0072] Through CW modulation driven by VCSEL, VCSEL modulates two frequencies of light, 15MHz and 120MHz, respectively. The two frequencies of light are exposed under the illumination conditions to obtain energy integral images of four phases of 0°, 180°, 90°, and 270° at each frequency. Therefore, 8 energy integral images need to be collected together:

[0073] Rawdata120_0: 0° phase energy integration diagram under 120Mhz frequency modulated light.

[0074] Rawdata120_90: 90° phase energy integration diagram under 120Mhz frequency modulated light.

[0075] Rawdata120_180: 180° phase energy integration diagram under 120Mhz frequency modulated light.

[0076] Rawdata120_270: 270° phase energy integration diagram under 120Mhz frequency modulated light.

[0077] Rawdata15_0: 0° phase energy integration diagram under 15Mhz frequency modulated light.

[0078] Rawdata15_90: 90° phase energy integration diagram under 15Mhz frequency modulated light.

[0079] Rawdata15_180: 180° phase energy integration diagram under 15Mhz frequency modulated light.

[0080] Rawdata15_270: 270° phase energy integral diagram under 15Mhz frequency modulated light.

[0081] Dual frequency rotation:

[0082] 1. First, use 120MHz to take 4 pictures. Each picture has 2 phases (2 taps). The exposure time of each picture is the set exposure value (eg. the exposure is set to 1000us, 4 times of taking pictures is 4000us, and there is a waiting time between two pictures, the minimum is about 4ms).

[0083] 2. Then 15MHz, take 4 pictures, each picture has 2 phases (2 taps), and the exposure time of each picture is the set exposure value (eg. the exposure is set to 1000us, 4 photos are taken for 4 times, and there is a waiting time between two pictures, the minimum is about 4ms).

[0084] 3. At each modulation frequency, the tap phase sequence of the four figures is:

[0085] The tap sequence is (0°, 180°) → (90°, 270°) → (180°, 0°) → (270°, 90°).

[0086] The value of each pixel of Rawdata at a certain frequency and phase represents the integral of the target reflected energy at the direction corresponding to this pixel. For the same pixel, the integral energy Q0, Q 90 , Q 180 , Q 270 , the phase offset value of this point can be calculated

[0087]

[0088] Q0 represents the energy integral of the 0 phase of the pixel.

[0089] Q 90 Represents the energy integral of the 0 phase of the pixel.

[0090] Q 180 Represents the energy integral of the 0 phase of the pixel.

[0091] Q 270 Represents the energy integral of the 0 phase of the pixel.

[0092] According to the phase of the pixel The detection distance at this frequency can be calculated, and the corresponding distances d1 and d2 can be calculated for the two frequencies respectively.

[0093]

[0094] The final distance is obtained by dual-frequency solution.

[0095] (1) Calculate the dual-frequency integrated frequency.

[0096] f_max is the greatest common divisor of frequency f1 and frequency f2: when f1 = 15Mhz, f2 = 120Mhz, f_max = 15Mhz

[0097] (2) Calculate M_f1 and M_f2 as follows:

[0098] M_f1=f1 / f_max

[0099] M_f2=f2 / f_max

[0100] Calculate A_f1, A_f2 according to the following formula:

[0101]

[0102] Calculate ω according to the following formula:

[0103]

[0104] The final dual-frequency solution distance is:

[0105] d=d1·ω+d2·(1-ω)

[0106] Depth map generation:

[0107] ①According to the results of phase calibration, the relationship between the actual detected phase and the theoretical phase is obtained;

[0108] ② Based on the current actual detected phase, the current true phase can be calculated and converted into the calculated distance, which is the actual detection distance value.

[0109] Depth map converted into point cloud:

[0110] (1) Based on the calibration of the camera’s intrinsic parameters, the azimuth angle corresponding to each pixel can be obtained.

[0111] (2) Based on the depth d and azimuth, the three-dimensional coordinates x, y, and z can be calculated.

[0112] The above three-dimensional space can be understood as a three-dimensional coordinate system used to describe the position and shape of an object. The three-dimensional space is composed of three dimensions: length, width and height. The first three-dimensional space can be a camera monitoring area. The detection area can be set as a cube defined by vertices ABCDEFGH with the camera optical center O as the center, and six tetrahedrons (OABEH, OCDFG, OABCD, OEFGH, OADEF, OBCGH) in the cube as sub-areas.

[0113] The above 3D point cloud data can be understood as a set of points with coordinates in 3D space, each of which represents a position on the surface of an object. 3D point cloud data is a collection of points distributed in a large number of 3D spaces, used to represent the surface or shape information of an object.

[0114] 102. Based on the three-dimensional point cloud data, each three-dimensional point is projected onto a plurality of two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes.

[0115] In the embodiment of the present invention, each three-dimensional point corresponds to a projection point in a two-dimensional projection plane.

[0116] The above projection can be understood as a process of mapping points in three-dimensional space to a two-dimensional surface.

[0117] For each three-dimensional point, there will be a corresponding point on each two-dimensional plane it is projected onto. This point is the projection of the three-dimensional point on the two-dimensional plane.

[0118] The above two-dimensional projection plane can be understood as the position of a three-dimensional point on the two-dimensional plane under the projection perspective.

[0119] In a possible embodiment, each point in the three-dimensional space is projected onto a two-dimensional plane. Assuming the plane is z=1, the projection point P′(x′, y′) of the three-dimensional point P(x, y, z) can be calculated by the following formula:

[0120]

[0121] Through the above formula, the three-dimensional point P (x, y, z) is projected onto the two-dimensional plane z = 1 to obtain the corresponding two-dimensional coordinates P'(x', y').

[0122] It should be noted that for each 3D point, there is a corresponding projection point on each selected 2D projection plane. For example, if each 3D point is projected onto three different planes, then each 3D point has three corresponding projection points on the three different planes.

[0123] 103. Determine a target three-dimensional point and the number of the target three-dimensional points located in the second three-dimensional space based on projection points corresponding to the three-dimensional points in multiple two-dimensional projection planes.

[0124] In an embodiment of the present invention, the target three-dimensional points and the number of target three-dimensional points located in the second three-dimensional space can be determined through the relationship between the projection points corresponding to the three-dimensional points in multiple two-dimensional projection planes and the three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space.

[0125] The two-dimensional projection plane may be the plane where each face of the second three-dimensional space is located, the second three-dimensional space may be a protection zone or an alarm zone drawn by a user, or the second three-dimensional space may be a hexahedral space. If the projections of a three-dimensional point are all within the projection range of each two-dimensional projection plane corresponding to the second three-dimensional space, it can be determined that the three-dimensional point is located in the second three-dimensional space.

[0126] 104. If the number of target three-dimensional points is greater than a preset number threshold, a safety control signal is output.

[0127] In the embodiment of the present invention, the above-mentioned preset quantity threshold is a quantity threshold preset by the system.

[0128] The above safety control signal is a signal used to control the operating state, ensuring that the system operates in a safe state or that the control system stops when a danger is detected.

[0129] When the number of target three-dimensional points detected is greater than the preset threshold, it may be caused by the existence of multiple objects, complex depth information, etc. In order to ensure the stability and safety of the system, a safety control signal will be output to trigger a safety prompt.

[0130] The above safety prompts may be warning messages, alarm sounds or other forms of reminders, the purpose of which is to make users or system operators aware of possible abnormal situations or potential risks.

[0131] The present invention projects points in three-dimensional space onto a two-dimensional plane and uses a two-dimensional vector intersection method to determine whether the point is within the detection area. This can solve the problem that when dividing the detection area, the existing method needs to process the polygonal pyramid area connected to the optical center of the camera, and the calculation complexity is high each time the two-dimensional projection plane is transformed. The operation time is long and the point cloud behind the projection plane cannot be accurately processed.

[0132] In an embodiment of the present invention, three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space is obtained; based on the three-dimensional point cloud data, each three-dimensional point is projected onto multiple two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, and each three-dimensional point corresponds to one projection point in one two-dimensional projection plane; based on the projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, the target three-dimensional point and the number of target three-dimensional points located in the second three-dimensional space are determined; if the number of target three-dimensional points is greater than a preset number threshold, a safety control signal is output. The present invention obtains three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space; based on the three-dimensional point cloud data, each three-dimensional point is projected onto multiple two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, and according to the projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, the target three-dimensional point and the number of target three-dimensional points located in the second three-dimensional space are determined, and when the number of target three-dimensional points is greater than the preset number threshold, a safety control signal is output, thereby solving the problem that the existing method has a long operation time and cannot accurately process the point cloud behind the projection plane.

[0133] It is understandable that in the specific implementation of this application, point cloud data, task data, equipment data and other related data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0134] Optionally, in the step of determining a target three-dimensional point located in the second three-dimensional space based on the projection points corresponding to the three-dimensional point in multiple two-dimensional projection planes, it can be determined in each two-dimensional projection plane whether the projection points corresponding to each three-dimensional point are located within a reference projection area corresponding to the two-dimensional projection plane; if the projection points of the three-dimensional point in each two-dimensional projection plane are all located within the reference projection area corresponding to the two-dimensional projection plane, the three-dimensional point is determined to be a target three-dimensional point located in the second three-dimensional space.

[0135] In an embodiment of the present invention, the reference projection area is a projection area of ​​the second three-dimensional space in the two-dimensional projection plane. The second three-dimensional space is located in the first three-dimensional space.

[0136] The above projection can be understood as a process of mapping points in three-dimensional space to a two-dimensional surface.

[0137] The above-mentioned reference projection area can be understood as the mapping area corresponding to the three-dimensional space on the two-dimensional plane. The reference projection area is the projection of the second three-dimensional space on a specific two-dimensional projection plane. If all the projection points of a three-dimensional point fall within the reference projection area, then this three-dimensional point is considered to be the target three-dimensional point of the second three-dimensional space.

[0138] Optionally, the three-dimensional point cloud data includes three-dimensional coordinate values ​​of the three-dimensional points. In the step of projecting each three-dimensional point onto multiple two-dimensional projection planes based on the three-dimensional point cloud data, the normal direction of the two-dimensional projection plane can be determined; in the normal direction, based on the three-dimensional coordinate value of the three-dimensional point, the two-dimensional coordinate value of the three-dimensional point in the two-dimensional projection plane is calculated; based on the two-dimensional coordinate value of the three-dimensional point in the two-dimensional projection plane, the three-dimensional point is projected onto multiple two-dimensional projection planes.

[0139] In the embodiment of the present invention, the normal direction is the direction of a straight line perpendicular to the projection plane. For example, if a three-dimensional point is located on a two-dimensional plane, the normal direction of the two-dimensional plane is the vector pointing from the three-dimensional point to the perpendicular direction of the plane.

[0140] The distance formula from a point to a plane can be used to calculate the two-dimensional coordinate value of a three-dimensional point on a two-dimensional projection plane.

[0141] It should be noted that for a point in three-dimensional space, there will be corresponding projection points in multiple two-dimensional planes. The two-dimensional projection plane needs to have a certain normal direction. According to the normal direction of the projection plane, the normal direction of the projection plane and the coordinate value of the three-dimensional point can be used to calculate the projection coordinates of the point in the three-dimensional space on each two-dimensional plane.

[0142] Optionally, in each two-dimensional projection plane, in the step of determining whether the projection point corresponding to each three-dimensional point is located in the reference projection area corresponding to the two-dimensional projection plane, at least one set of boundary vectors of the reference projection area and the starting point corresponding to each set of boundary vectors can be determined in the two-dimensional projection plane; based on the starting point and the projection point corresponding to the three-dimensional point, the projection vector corresponding to the three-dimensional point is determined; based on the projection vector and at least one set of boundary vectors, it is determined whether the projection point corresponding to the three-dimensional point is located in the reference projection area corresponding to the two-dimensional projection plane.

[0143] In an embodiment of the present invention, a set of boundary vectors includes two boundary vectors with a common starting point, and each boundary vector is collinear with a boundary of the reference projection area. The above boundary vectors can be understood as vectors of the boundary of the projection area. The shape and range of the projection area can be determined by the boundary vectors. The above starting point can be the center of the projection area, a fixed point or other points.

[0144] For each point in the three-dimensional space, it is necessary to calculate the projection vector corresponding to the three-dimensional point based on at least one set of boundary vectors of the reference projection area and the starting point corresponding to each set of boundary vectors. This projection vector describes the positional relationship of the three-dimensional point on the two-dimensional projection plane. Using the projection vector and the boundary vector, it can be determined whether the projection point of the three-dimensional point on the two-dimensional projection plane is located in the reference projection area.

[0145] Optionally, in the step of determining whether the projection point corresponding to the three-dimensional point is located in the reference projection area corresponding to the two-dimensional projection plane based on the projection vector and at least one set of boundary vectors, a first cross product between two boundary vectors and a second cross product between the boundary vector and the projection vector can be calculated for a set of boundary vectors; based on the first cross product and the second cross product, it is determined whether the projection point corresponding to the three-dimensional point is located in the reference projection area corresponding to the two-dimensional projection plane.

[0146] In the embodiment of the present invention, the cross product may be understood as the directed area of ​​a parallelogram with two vectors as sides.

[0147] The cross product above returns the product of two vectors, resulting in a vector perpendicular to the plane formed by the two vectors. The purpose of calculating the cross product is to obtain information about the direction and position of the vector.

[0148] The first cross product may be a cross product between two boundary vectors, and the second cross product may be a cross product between a boundary vector and a projection vector.

[0149] It should be noted that the present invention can determine whether the projection point of a three-dimensional point on a two-dimensional projection plane is located within the projection reference area by using vector operations. Assuming that the plane is z=1, the projection point of the three-dimensional point P(x, y, z) is P′(x′, y′), if the point P′(x′, y′) is on the boundaries AB and CD, the formula for the cross product between the two boundary vectors is as follows:

[0150] Among them, cross represents the cross product of two-dimensional vectors, and is the boundary vector, is the vector between the projection point and the boundary starting point. By calculating the cross product between the two boundary vectors in each quadrilateral, it can be determined whether the projection points corresponding to all three-dimensional points in the cluster are located in the reference projection area corresponding to the two-dimensional projection plane.

[0151] For each three-dimensional point and each enhanced reference area, a judgment will be made between the enhanced vector and the boundary vector. If all judgments are passed, it can be determined that all the projection points of the three-dimensional point are located in the reference projection area corresponding to the two-dimensional projection plane, and then it can be determined that the three-dimensional point is located in the second three-dimensional area.

[0152] Optionally, in the step of determining the projection vector corresponding to the three-dimensional point based on the projection point corresponding to the starting point and the three-dimensional point, all the projection points can be clustered in the two-dimensional projection plane according to the two-dimensional distance between the projection points to obtain at least one cluster cluster; if the cluster cluster includes only one projection point, the projection vector corresponding to the three-dimensional point is determined based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the projection point; if the cluster cluster includes multiple projection points, the target circle corresponding to the cluster cluster is determined with the center of mass of the cluster cluster as the center of the circle and the largest two-dimensional distance radius; a tangent line of the target circle is made through the starting point, and a first tangent point and a second tangent point corresponding to the starting point are determined on the target circle; based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the first tangent point, the first projection vector corresponding to all the three-dimensional points in the cluster cluster is determined, and based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the second tangent point, the second projection vector corresponding to all the three-dimensional points in the cluster cluster is determined.

[0153] In the embodiment of the present invention, the clusters include at least one projection point, and each cluster corresponds to a target circle. The clusters can be understood as a group of similar projection points that are close to each other in a two-dimensional projection plane.

[0154] In the two-dimensional projection plane, we first need to determine all the projection points and measure their similarity based on the two-dimensional distance between the projection points. If the distance between two projection points is close, they are considered similar and can be classified into one category. Projection points with similar characteristics are clustered together to form different clusters. Each cluster represents a group of projection points that are close to each other on the two-dimensional projection plane. The distance between projection points within each cluster is close, while the distance between projection points in different clusters is far.

[0155] The above-mentioned projection vector can be understood as a vector obtained by projecting a vector along a certain direction onto another plane or space.

[0156] For example, if the cluster contains only one projection point, the projection point in the three-dimensional space is point P(x, y, z), and its projection point on the two-dimensional plane is Q(a, b). The coordinate values ​​of these two points can be used to calculate the vector from P to Q. Specifically, this vector can be expressed as u→==(QP), where QP=(ax,by,0).

[0157] The centroid can be understood as the average position of all points in the cluster. For example, in a two-dimensional plane, if a cluster contains n points, the coordinates of its centroid are the sum of the coordinates of these n points divided by n. The two-dimensional distance can be understood as the distance between two points on a two-dimensional plane.

[0158] The above target circle can be understood as the target area corresponding to the cluster.

[0159] The above tangent line can be understood as a straight line that has only one common point with the circle.

[0160] It should be noted that if the cluster only includes one projection point, the projection point can be used directly to determine the projection vector corresponding to the three-dimensional point. If the cluster includes multiple projection lines, the centroid of the cluster can be used as the center of the circle, and the target circle corresponding to the cluster can be determined with the largest two-dimensional distance radius. The starting point can be used as the tangent of the target circle, and two tangent points corresponding to the starting point can be determined on the target circle to determine the projection vector.

[0161] Through clustering processing, multiple three-dimensional points with close distances can be judged in batches, thereby reducing the amount of calculation of projection vectors and improving the calculation speed.

[0162] Optionally, in the step of determining whether the projection point corresponding to the three-dimensional point is located in the reference projection area corresponding to the two-dimensional projection plane based on the projection vector and at least one set of boundary vectors, the first cross product between two boundary vectors can be calculated for a set of boundary vectors, the third cross product between the boundary vector and the first projection vector can be calculated, and the fourth cross product between the boundary vector and the second projection vector can be calculated; based on the product of the first cross product and the third cross product, and the product of the first cross product and the fourth cross product, it is determined whether the projection points corresponding to all three-dimensional points in the cluster are located in the reference projection area corresponding to the two-dimensional projection plane.

[0163] In an embodiment of the present invention, the first cross product is a cross product between two boundary vectors; the third cross product is a cross product between a boundary vector and a first projection vector; and the fourth cross product is a cross product between a boundary vector and a second projection vector.

[0164] It should be noted that the intersection of two-dimensional vectors can be calculated to determine whether the projection points corresponding to all three-dimensional points in the cluster are located in the reference projection area corresponding to the two-dimensional projection plane. It can be understood that a set of boundaries is defined by vectors v1 and v2, and the intersection of the projection point P′ to each side of the quadrilateral can be calculated. By calculating whether there is an intersection with the vector of the boundary, if the point P′ is located in the intersection of all four sides, then the point is inside the quadrilateral, otherwise it may be outside. In a cluster, if it is possible to be outside, the cross product of the projection vector and the boundary vector can be determined for each projection point in the cluster one by one.

[0165] Optionally, the second three-dimensional space also includes a third three-dimensional space, and the point cloud data in the third three-dimensional space may be removed.

[0166] In an embodiment of the present invention, the third three-dimensional space can be a shielding area. Taking a robot or a mechanical arm as an example, the working area of ​​the robot or the mechanical arm is the shielding area, and the area outside the shielding area is the protection area or the alarm area. When point cloud data appears in the protection area or the alarm area, it means that an object invades the protection area or the alarm area from the shielding area during the operation of the robot or the mechanical arm. A safety control signal can be output to the host computer or the relay to control the corresponding working equipment to stop working. Point cloud data is a collection of a large number of points in a three-dimensional space, which is used to represent the surface or shape of an object. In some cases, the point cloud data in the shielding area can be eliminated to reduce the amount of calculation. In other cases, the point cloud data in the shielding area can be not eliminated to protect the integrity of the point cloud data and improve the accuracy of target detection. Specifically, if the target is only in the area of ​​interest, the point cloud data in the shielding area is eliminated; if the target is in the area of ​​interest and the target is in the shielding area, the point cloud data in the shielding area is protected.

[0167] The above elimination may be elimination of noise, redundant or irrelevant data, etc.

[0168] like Figure 2 As shown, an embodiment of the present invention provides a security monitoring device, the security monitoring device comprising:

[0169] An acquisition module 201 is used to acquire three-dimensional point cloud data corresponding to all three-dimensional points in a first three-dimensional space;

[0170] A projection module 202 is used to project each of the three-dimensional points onto a plurality of two-dimensional projection planes based on the three-dimensional point cloud data to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, wherein each of the three-dimensional points corresponds to one projection point in one of the two-dimensional projection planes;

[0171] A determination module 203, configured to determine a target three-dimensional point located in a second three-dimensional space and the number of the target three-dimensional points based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes;

[0172] The safety control module 204 is configured to output a safety control signal if the number of the target three-dimensional points is greater than a preset number threshold.

[0173] Optionally, the determination module 203 is also used to determine, in each of the two-dimensional projection planes, whether the projection point corresponding to each of the three-dimensional points is located within a reference projection area corresponding to the two-dimensional projection plane, where the reference projection area is a projection area of ​​a second three-dimensional space in the two-dimensional projection plane, and the second three-dimensional space is located in the first three-dimensional space; if the projection points of the three-dimensional point in each of the two-dimensional projection planes are located within the reference projection area corresponding to the two-dimensional projection plane, then the three-dimensional point is determined to be a target three-dimensional point located in the second three-dimensional space.

[0174] Optionally, the three-dimensional point cloud data includes three-dimensional coordinate values ​​of the three-dimensional points, and the projection module 202 is also used to determine the normal direction of the two-dimensional projection plane; in the normal direction, based on the three-dimensional coordinate values ​​of the three-dimensional points, the two-dimensional coordinate values ​​of the three-dimensional points in the two-dimensional projection plane are calculated; based on the two-dimensional coordinate values ​​of the three-dimensional points in the two-dimensional projection plane, the three-dimensional points are projected onto multiple two-dimensional projection planes.

[0175] Optionally, the determination module 203 is also used to determine at least one group of boundary vectors of the reference projection area and a starting point corresponding to each group of boundary vectors in the two-dimensional projection plane, a group of boundary vectors including two boundary vectors with a common starting point, and each boundary vector is collinear with a boundary of the reference projection area; based on the starting point and the projection point corresponding to the three-dimensional point, determine the projection vector corresponding to the three-dimensional point; and determine whether the projection point corresponding to the three-dimensional point is located in the reference projection area corresponding to the two-dimensional projection plane based on the projection vector and at least one group of boundary vectors.

[0176] Optionally, the determination module 203 is also used to calculate, for a group of the boundary vectors, a first cross product between two of the boundary vectors, and a second cross product between the boundary vector and the projection vector; based on the first cross product and the second cross product, determine whether the projection point corresponding to the three-dimensional point is located within the reference projection area corresponding to the two-dimensional projection plane.

[0177] Optionally, the determination module 203 clusters all the projection points in the two-dimensional projection plane according to the two-dimensional distance between the projection points to obtain at least one cluster, and the cluster includes at least one projection point; if the cluster includes only one projection point, then based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the projection point, the projection vector corresponding to the three-dimensional point is determined; if the cluster includes multiple projection points, with the centroid of the cluster as the center of the circle and the largest two-dimensional distance radius, the target circle corresponding to the cluster is determined, and each cluster corresponds to one target circle; a tangent to the target circle is made through the starting point, and a first tangent point and a second tangent point corresponding to the starting point are determined on the target circle; based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the first tangent point, the first projection vector corresponding to all the three-dimensional points in the cluster is determined, and based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the second tangent point, the second projection vector corresponding to all the three-dimensional points in the cluster is determined.

[0178] Optionally, the determination module 203 is also used to calculate, for a group of the boundary vectors, a first cross product between two of the boundary vectors, a third cross product between the boundary vector and the first projection vector, and a fourth cross product between the boundary vector and the second projection vector; based on the product of the first cross product and the third cross product, and the product of the first cross product and the fourth cross product, determine whether the projection points corresponding to all three-dimensional points in the cluster are located within the reference projection area corresponding to the two-dimensional projection plane.

[0179] Optionally, the acquiring of the second three-dimensional space also includes a third three-dimensional space, and the device further includes:

[0180] The elimination module eliminates the point cloud data in the third three-dimensional space.

[0181] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, including a processor, and the processor can execute any one of the above security monitoring methods.

[0182] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing a security monitoring method stored in the memory 302 and capable of running on the processor 301, wherein:

[0183] The processor 301 runs the computer program of the security monitoring method stored in the memory 302 to perform the following steps:

[0184] Obtaining three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space;

[0185] Based on the three-dimensional point cloud data, project each of the three-dimensional points onto a plurality of two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, wherein each of the three-dimensional points corresponds to one projection point in one of the two-dimensional projection planes;

[0186] Determine, based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, target three-dimensional points located in the second three-dimensional space and the number of the target three-dimensional points;

[0187] If the number of the target three-dimensional points is greater than a preset number threshold, a safety control signal is output.

[0188] Optionally, the processor 301 determines the target three-dimensional point located in the second three-dimensional space based on the projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, including:

[0189] In each of the two-dimensional projection planes, determining whether the projection point corresponding to each of the three-dimensional points is located in a reference projection area corresponding to the two-dimensional projection plane, the reference projection area being a projection area of ​​a second three-dimensional space in the two-dimensional projection plane, and the second three-dimensional space is located in the first three-dimensional space;

[0190] If the projection points of the three-dimensional point in each of the two-dimensional projection planes are located in the reference projection area corresponding to the two-dimensional projection plane, the three-dimensional point is determined to be a target three-dimensional point located in the second three-dimensional space.

[0191] Optionally, the three-dimensional point cloud data includes three-dimensional coordinate values ​​of the three-dimensional points, and the processor 301 executes the projecting of each of the three-dimensional points onto a plurality of two-dimensional projection planes based on the three-dimensional point cloud data, including:

[0192] Determining the normal direction of the two-dimensional projection plane;

[0193] In the normal direction, based on the three-dimensional coordinate value of the three-dimensional point, a two-dimensional coordinate value of the three-dimensional point on the two-dimensional projection plane is calculated;

[0194] The three-dimensional point is projected onto a plurality of two-dimensional projection planes based on the two-dimensional coordinate value of the three-dimensional point on the two-dimensional projection plane.

[0195] Optionally, the determining, in each of the two-dimensional projection planes, by the processor 301, whether the projection point corresponding to each of the three-dimensional points is located in a reference projection area corresponding to the two-dimensional projection plane includes:

[0196] In the two-dimensional projection plane, determine at least one group of boundary vectors of the reference projection area and a starting point corresponding to each group of boundary vectors, wherein a group of boundary vectors includes two boundary vectors with a common starting point, and each boundary vector is collinear with a boundary of the reference projection area;

[0197] Based on the starting point and the projection point corresponding to the three-dimensional point, determining a projection vector corresponding to the three-dimensional point;

[0198] It is determined, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane.

[0199] Optionally, the determining, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane performed by the processor 301 includes:

[0200] For a set of the boundary vectors, calculating a first cross product between two of the boundary vectors, and calculating a second cross product between the boundary vector and the projection vector;

[0201] Based on the first cross product and the second cross product, it is determined whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane.

[0202] Optionally, the processor 301 performs the step of determining, based on the starting point and the projection point corresponding to the three-dimensional point, a projection vector corresponding to the three-dimensional point, including:

[0203] In the two-dimensional projection plane, clustering all the projection points according to the two-dimensional distances between the projection points to obtain at least one cluster, wherein the cluster includes at least one projection point;

[0204] If the cluster includes only one projection point, determining a projection vector corresponding to the three-dimensional point based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the projection point;

[0205] If the cluster includes a plurality of the projection points, a target circle corresponding to the cluster is determined with the centroid of the cluster as the center of the circle and the largest two-dimensional distance radius, and each cluster corresponds to one target circle;

[0206] Draw a tangent line to the target circle through the starting point, and determine a first tangent point and a second tangent point on the target circle corresponding to the starting point;

[0207] Based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the first tangent point, a first projection vector corresponding to all three-dimensional points in the cluster cluster is determined, and based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the second tangent point, a second projection vector corresponding to all three-dimensional points in the cluster cluster is determined.

[0208] Optionally, the determining, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane performed by the processor 301 includes:

[0209] For a set of the boundary vectors, calculating a first cross product between two of the boundary vectors, calculating a third cross product between the boundary vector and the first projection vector, and calculating a fourth cross product between the boundary vector and the second projection vector;

[0210] Based on the product of the first cross product and the third cross product and the product of the first cross product and the fourth cross product, it is determined whether the projection points corresponding to all three-dimensional points in the cluster are located in the reference projection area corresponding to the two-dimensional projection plane.

[0211] Optionally, the acquiring of the second three-dimensional space also includes a third three-dimensional space, and the method executed by the processor 301 further includes:

[0212] The point cloud data in the third three-dimensional space is eliminated.

[0213] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0214] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A security monitoring method, characterized in that: The method comprises the following steps: Obtaining three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space; Based on the three-dimensional point cloud data, project each of the three-dimensional points onto a plurality of two-dimensional projection planes to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, wherein each of the three-dimensional points corresponds to one projection point in one of the two-dimensional projection planes; Determine, based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, target three-dimensional points located in the second three-dimensional space and the number of the target three-dimensional points; If the number of the target three-dimensional points is greater than a preset number threshold, a safety control signal is output.

2. The security monitoring method according to claim 1, characterized in that: Determining a target three-dimensional point located in a second three-dimensional space based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes includes: In each of the two-dimensional projection planes, determining whether the projection point corresponding to each of the three-dimensional points is located in a reference projection area corresponding to the two-dimensional projection plane, the reference projection area being a projection area of ​​a second three-dimensional space in the two-dimensional projection plane, and the second three-dimensional space is located in the first three-dimensional space; If the projection points of the three-dimensional point in each of the two-dimensional projection planes are located in the reference projection area corresponding to the two-dimensional projection plane, the three-dimensional point is determined to be a target three-dimensional point located in the second three-dimensional space.

3. The security monitoring method according to claim 2, characterized in that: The three-dimensional point cloud data includes three-dimensional coordinate values ​​of the three-dimensional points, and projecting each of the three-dimensional points onto a plurality of two-dimensional projection planes based on the three-dimensional point cloud data includes: Determining the normal direction of the two-dimensional projection plane; In the normal direction, based on the three-dimensional coordinate value of the three-dimensional point, a two-dimensional coordinate value of the three-dimensional point on the two-dimensional projection plane is calculated; The three-dimensional point is projected onto a plurality of two-dimensional projection planes based on the two-dimensional coordinate value of the three-dimensional point on the two-dimensional projection plane.

4. The security monitoring method according to claim 2, characterized in that: The step of determining, in each of the two-dimensional projection planes, whether the projection point corresponding to each of the three-dimensional points is located within a reference projection area corresponding to the two-dimensional projection plane comprises: In the two-dimensional projection plane, determine at least one group of boundary vectors of the reference projection area and a starting point corresponding to each group of boundary vectors, wherein a group of boundary vectors includes two boundary vectors with a common starting point, and each boundary vector is collinear with a boundary of the reference projection area; Based on the starting point and the projection point corresponding to the three-dimensional point, determining a projection vector corresponding to the three-dimensional point; It is determined, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane.

5. The security monitoring method according to claim 4, characterized in that: The determining, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located within the reference projection area corresponding to the two-dimensional projection plane comprises: For a set of the boundary vectors, calculating a first cross product between two of the boundary vectors, and calculating a second cross product between the boundary vector and the projection vector; Based on the first cross product and the second cross product, it is determined whether the projection point corresponding to the three-dimensional point is located in a reference projection area corresponding to the two-dimensional projection plane.

6. The security monitoring method according to claim 4, characterized in that: The determining, based on the projection point corresponding to the starting point and the three-dimensional point, a projection vector corresponding to the three-dimensional point comprises: In the two-dimensional projection plane, clustering all the projection points according to the two-dimensional distances between the projection points to obtain at least one cluster, wherein the cluster includes at least one projection point; If the cluster includes only one projection point, determining a projection vector corresponding to the three-dimensional point based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the projection point; If the cluster includes a plurality of the projection points, a target circle corresponding to the cluster is determined with the centroid of the cluster as the center of the circle and the largest two-dimensional distance radius, and each cluster corresponds to one target circle; Draw a tangent line to the target circle through the starting point, and determine a first tangent point and a second tangent point on the target circle corresponding to the starting point; Based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the first tangent point, a first projection vector corresponding to all three-dimensional points in the cluster cluster is determined, and based on the two-dimensional coordinate value of the starting point and the two-dimensional coordinate value of the second tangent point, a second projection vector corresponding to all three-dimensional points in the cluster cluster is determined.

7. The security monitoring method according to claim 6, characterized in that: The determining, according to the projection vector and at least one group of the boundary vectors, whether the projection point corresponding to the three-dimensional point is located within the reference projection area corresponding to the two-dimensional projection plane comprises: For a set of the boundary vectors, calculating a first cross product between two of the boundary vectors, calculating a third cross product between the boundary vector and the first projection vector, and calculating a fourth cross product between the boundary vector and the second projection vector; Based on the product of the first cross product and the third cross product and the product of the first cross product and the fourth cross product, it is determined whether the projection points corresponding to all three-dimensional points in the cluster are located in the reference projection area corresponding to the two-dimensional projection plane.

8. The security monitoring method according to any one of claims 1 to 7, characterized in that: The acquiring of the second three-dimensional space also includes a third three-dimensional space, and the method further includes: The point cloud data in the third three-dimensional space is eliminated.

9. A security monitoring device, characterized in that: The safety monitoring device comprises: An acquisition module, used to acquire three-dimensional point cloud data corresponding to all three-dimensional points in the first three-dimensional space; A projection module, configured to project each of the three-dimensional points onto a plurality of two-dimensional projection planes based on the three-dimensional point cloud data, to obtain projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes, wherein each of the three-dimensional points corresponds to one projection point in one of the two-dimensional projection planes; A determination module, configured to determine a target three-dimensional point located in a second three-dimensional space and the number of the target three-dimensional points based on projection points corresponding to the three-dimensional point in the plurality of two-dimensional projection planes; The safety control module is used to output a safety control signal if the number of the target three-dimensional points is greater than a preset number threshold.

10. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the security monitoring method according to any one of claims 1 to 8 when executing the computer program.

Images