Safety monitoring method based on contour recognition and related equipment

Abstract

The invention provides a safety monitoring method based on contour recognition and related equipment, and the method comprises the steps: obtaining first point cloud data corresponding to a current frame and second point cloud data corresponding to a reference frame, and enabling the point cloud positions in the first point cloud data to be in one-to-one correspondence with the point cloud positions in the second point cloud data; for the first point cloud data and the second point cloud data, carrying out point-by-point cloud calculation on a distance value to obtain a distance value at each point cloud position; if the distance value at the point cloud position is greater than or equal to a preset distance threshold value, determining that the point cloud position is a change point; and triggering and outputting a safety control signal based on the change point. According to the method, the first point cloud data corresponding to the current frame and the second point cloud data corresponding to the reference frame are acquired, the distance values are calculated point by point for the first point cloud data and the second point cloud data, the distance value of each point cloud position is obtained, and the safety control signal is output according to the change point, so that the safety and the stability of the system are effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of visual technology, and in particular to a safety monitoring method based on contour recognition and related equipment. Background Art

[0002] Stereo safety sensors, also known as Time-of-Flight (ToF) cameras or 3D TOF cameras, are mainly used for stereo protection in the industrial field. Their working principle is to calculate the distance information by measuring the time it takes for light to be emitted from the stereo safety sensor to the object in the detection area and then reflected back to the camera, thereby obtaining the three-dimensional spatial data of the scene. When a target object invades the detected area, it will output a safety control signal to the host computer or relay, thereby controlling the shutdown of the working equipment.

[0003] At present, traditional methods use point cloud data to identify targets when monitoring the detection area. However, due to object movement or environmental changes, the point cloud data of the detection area will also change, which will interfere with the identification of targets in the detection area, resulting in inaccurate target identification results and failure to detect system abnormalities in a timely manner. Summary of the invention

[0004] The embodiment of the present invention provides a safety monitoring method based on contour recognition, which can timely detect the continuous position change of the monitored object, timely detect the abnormality of the system, and improve the safety and stability of the system. By comprehensively acquiring the first point cloud data corresponding to the current frame and the second point cloud data corresponding to the reference frame, the cloud distance value is calculated point by point for the first point cloud data and the second point cloud data, the distance value at each point cloud position is obtained, and the safety control signal is output according to the change point, which solves the problem that the existing method fails to timely detect the continuous position change of the monitored object and the abnormality of the system.

[0005] In a first aspect, an embodiment of the present invention provides a safety monitoring method based on contour recognition, the method comprising the following steps:

[0006] Acquire first point cloud data corresponding to the current frame and second point cloud data corresponding to the reference frame, wherein the point cloud positions in the first point cloud data correspond one-to-one to the point cloud positions in the second point cloud data;

[0007] For the first point cloud data and the second point cloud data, computing distance values ​​point by point to obtain a distance value at each point cloud position;

[0008] If the distance value at the point cloud position is greater than or equal to a preset distance threshold, determining the point cloud position as a change point;

[0009] Based on the change point, a safety control signal is output.

[0010] Optionally, outputting a safety control signal based on the change point includes:

[0011] If the number of the change points in the current frame is greater than a preset number threshold, a safety control signal is output.

[0012] Optionally, outputting a safety control signal based on the change point includes:

[0013] Based on the change point, determining whether a change in the contour between the current frame and the reference frame is an unacceptable change;

[0014] If the profile change is an unacceptable change, a safety control signal is output.

[0015] Optionally, judging whether a contour change between the current frame and the reference frame is an unacceptable change based on the change point includes:

[0016] Based on the change point, extracting contour features in the first point cloud data to obtain current contour features corresponding to the current frame;

[0017] Calculating the correlation between the current contour feature and the reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature, wherein the reference contour feature is the contour feature corresponding to the current frame;

[0018] Based on the correlation value and a preset correlation threshold, it is determined whether a contour change between the current frame and the reference frame is an unacceptable change.

[0019] Optionally, performing correlation calculation on the current contour feature and the reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature includes:

[0020] Performing contour matching between the current contour feature and the reference contour feature to obtain a matching degree between the current contour feature and the reference contour feature;

[0021] Based on the matching degree, a correlation value between the current contour feature and the reference contour feature is determined, and the higher the matching degree, the greater the correlation value.

[0022] Optionally, judging whether a contour change between the current frame and the reference frame is an unacceptable change based on the correlation value and a preset correlation threshold includes:

[0023] If the correlation value is greater than or equal to a preset correlation threshold, determining that the contour change between the current frame and the reference frame is not an unacceptable change;

[0024] If the correlation value is less than a preset correlation threshold, it is determined that the contour change between the current frame and the reference frame is an unacceptable change.

[0025] Optionally, the current frame and the reference frame are both provided with shielding areas of the same size and position, and the method further includes:

[0026] If all the targets are outside the shielding area, the point cloud data in the shielding area is eliminated;

[0027] If a portion of the target is outside the shielding area and another portion of the target is within the shielding area, the point cloud data in the shielding area is retained.

[0028] In a second aspect, an embodiment of the present invention further provides a safety monitoring device based on contour recognition, the safety monitoring device based on contour recognition comprising:

[0029] An acquisition module, used to acquire first point cloud data corresponding to a current frame and second point cloud data corresponding to a reference frame, wherein a point cloud position in the first point cloud data corresponds to a point cloud position in the second point cloud data in a one-to-one correspondence;

[0030] A first processing module is used to calculate the distance value of the first point cloud data and the second point cloud data point by point to obtain the distance value at each point cloud position;

[0031] A determination module, configured to determine that the point cloud position is a change point if the distance value at the point cloud position is greater than or equal to a preset distance threshold;

[0032] The safety control module is used to output a safety control signal based on the change point.

[0033] In a third aspect, an embodiment of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps of the security monitoring method based on contour recognition provided in an embodiment of the present invention are implemented.

[0034] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of the security monitoring method based on contour recognition provided in an embodiment of the invention are implemented.

[0035] In an embodiment of the present invention, first point cloud data corresponding to the current frame and second point cloud data corresponding to the reference frame are obtained, and the point cloud positions in the first point cloud data correspond to the point cloud positions in the second point cloud data one by one; for the first point cloud data and the second point cloud data, the cloud distance value is calculated point by point to obtain the distance value at each point cloud position; if the distance value at the point cloud position is greater than or equal to a preset distance threshold, the point cloud position is determined to be a change point; based on the change point, a safety control signal is output. The present invention obtains the first point cloud data corresponding to the current frame and the second point cloud data corresponding to the reference frame, and for the first point cloud data and the second point cloud data, the cloud distance value is calculated point by point to obtain the distance value at each point cloud position, and outputs a safety control signal based on the change point, thereby solving the problems of the existing methods that fail to detect the continuous position changes of the monitored object in a timely manner, and fail to detect system abnormalities in a timely manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 is a flow chart of a safety monitoring method based on contour recognition provided by an embodiment of the present invention;

[0038] Figure 2 is a structural schematic diagram of a safety monitoring device based on contour recognition provided by an embodiment of the present invention;

[0039] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0041] like Figure 1 As shown, Figure 1 : is a flow chart of a safety monitoring method based on contour recognition provided by an embodiment of the present invention, the safety monitoring method based on contour recognition comprises the steps of:

[0042] 101. Obtain first point cloud data corresponding to a current frame and second point cloud data corresponding to a reference frame.

[0043] In an embodiment of the present invention, the security monitoring method can be applied to a security monitoring system, which includes a host computer and a point cloud data acquisition device, which can be a three-dimensional camera, a depth camera or a time-of-flight camera, such as a stereo security sensor. The point cloud data acquisition device is used to collect point cloud data of the corresponding area.

[0044] The above security monitoring method is mainly used for the host computer of the security monitoring system. The above point cloud data can be understood as the three-dimensional coordinate data (x, y, z) of each pixel point in the depth map collected by the stereo security sensor.

[0045] The above-mentioned point cloud data can also be called 3D point cloud data. The collection of point cloud data in this embodiment can be based on the iTOF ranging principle. Through CW modulation driven by VCSEL, VCSEL modulates light with a frequency of f. Because it is necessary to analyze the phase difference between the reflected light and the emitted light to calculate the distance, in order to avoid multiple solutions, only the distance of the target within 1 / 2 wavelength can be measured, that is, the maximum ranging is 1 / 2 wavelength of the modulated light wave. The two frequencies of 15Mhz and 120Mhz adopted in this example, the theoretical maximum detection distance at 15Mhz frequency is 10m, and the theoretical maximum detection distance at 120Mhz frequency is 1.25m. The theoretical maximum detection distance of dual-frequency rotation is the least common multiple of the theoretical maximum detection distances corresponding to the two frequencies, that is, the greatest common multiple of 1.5m and 10m, that is, the theoretical maximum detection distance under the conditions of 15Mhz and 120Mhz rotation is 10m.

[0046] The distortion problem of the depth map is solved one by one through the camera intrinsic calibration. In the intrinsic calibration module, the common "pinhole + distortion" model in computer vision can be used to characterize the imaging system parameters such as lens distortion and principal point offset. The parameters include:

[0047] 1. Main point c x 、c y , represents the pixel coordinates of the intersection of the lens optical axis and the imaging target surface.

[0048] 2. Focal length f x 、f y , which represents the ratio of the lens focal length to the pixel size.

[0049] 3. Distortion parameters k1, k2, k 3；, p1, p2, where k1, k2, k3 are parameters representing the radial distortion of the lens, and p1 and p2 are parameters representing the tangential distortion of the lens. In the internal parameter calibration module, the calibration method is used to solve the model parameters. The camera is controlled to collect the intensity image of the calibration plate. Given the coordinates of the marking points in the calibration plate, the features of the marking points in the image are extracted to obtain the image coordinates of the marking points, the objective function is established, and the LM method is used for optimization and solution. Finally, the principal point c is obtained. x 、c y , focal length f x 、f y , and distortion parameters k1, k2, k3, p1, p2, and write the calibration results into the camera.

[0050] During the depth calibration (phase calibration) process, the measurement accuracy of TOF is affected by many factors. The depth calibration module mainly considers three items:

[0051] 1. When measuring an object with a distance of zero, when a zero-phase signal is transmitted, the delay difference between the phase of the signal received by RX (RF receiver) and the zero-phase signal is called zero drift.

[0052] 2. Characterize the zero drift that indicates the difference in the initial exposure time of different pixels, i.e., the fixed phase template noise.

[0053] 3. The deviation between the light waveform emitted by the TX (radio frequency transmitter) and the cosine signal causes harmonic errors that vary with distance.

[0054] The absolute errors introduced by the above items are usually in the order of centimeters. To ensure the accuracy of the distance measurement results, the above errors of each stereo safety sensor must be calibrated one by one before leaving the factory.

[0055] In order to calibrate the above errors, it is necessary to know the real distance of the object to be measured, collect TOF images at the same time, and calculate the actual measured distance. An error model is established between the two, and the model parameters are solved. In specific implementation, the stereo safety sensor is placed on the track table, and the 90% diffuse reflection plane is photographed at different distances to collect TOF data. The distance between the stereo safety sensor and the diffuse reflection plane given by the track table and the internal parameter calibration model are used to calculate the real distance of each pixel from the diffuse reflection plane. Call the depth calibration algorithm to process the collected TOF data, calculate the actual measurement distance, and optimize the model parameters.

[0056] The depth calculation algorithm completes the calculation from the raw image to the tap image, the tap image to the intensity image, the amplitude image, the phase image, the phase image to the distance image, the distance image to the depth image, and the depth image to the point cloud image. The depth calculation algorithm also has the following two functions:

[0057] 1. In the process of calculating the phase, the depth calculation algorithm needs to call the calibration results to compensate the calculated phase to eliminate the system error and obtain higher measurement accuracy.

[0058] 2. Because the stereo safety sensor works in dual-frequency mode, the depth calculation algorithm needs to fuse the dual-frequency data and output a higher quality measurement result after fusion.

[0059] The depth filtering algorithm is effective at different stages of the depth calculation algorithm, processing the intermediate quantities of the calculation, filtering out errors and low signal-to-noise ratio results, and ensuring the quality of the output results. The depth filtering algorithm consists of five modules:

[0060] 1. Temporal filtering module: averages multiple frames of the depth map to improve the signal-to-noise ratio. You can set the temporal filtering algorithm on and off, as well as the fusion weight of the current frame.

[0061] 2. Spatial filtering module: Perform spatial Gaussian filtering on the 1Q image to improve the signal-to-noise ratio. You can set the spatial filtering algorithm on and off, as well as the size of the Gaussian filtering window.

[0062] 3. Amplitude filter module: Set a certain threshold to filter out pixel depths with amplitudes below this threshold. You can set the amplitude filter algorithm on and off, as well as the amplitude filter threshold.

[0063] 4. Fusion error filter module: Calculate the dual-frequency fusion error. When the phase is greater than the set value, filter out the corresponding pixel depth. You can set the fusion error filter algorithm to open or close, as well as the size of the interval.

[0064] 5. Flying point filter module: convert the depth into point cloud, calculate the distance between the point and the surrounding points, and filter out the corresponding pixel depth when the minimum distance is greater than the set value. You can set the flying point filter to be on or off, as well as the size of the distance value.

[0065] The following is an example of point cloud generation:

[0066] Through CW modulation driven by VCSEL, VCSEL modulates two frequencies of light, 15MHz and 120MHz, respectively. The two frequencies of light are exposed under the illumination conditions to obtain energy integral images of four phases of 0°, 180°, 90°, and 270° at each frequency. Therefore, 8 energy integral images need to be collected together:

[0067] Rawdata120_0: 0° phase energy integration diagram under 120Mhz frequency modulated light.

[0068] Rawdata120_90: 90° phase energy integration diagram under 120Mhz frequency modulated light.

[0069] Rawdata120_180: 180° phase energy integration diagram under 120Mhz frequency modulated light.

[0070] Rawdata120_270: 270° phase energy integration diagram under 120Mhz frequency modulated light.

[0071] Rawdata15_0: 0° phase energy integration diagram under 15Mhz frequency modulated light.

[0072] Rawdata15_90: 90° phase energy integration diagram under 15Mhz frequency modulated light.

[0073] Rawdata15_180: 180° phase energy integration diagram under 15Mhz frequency modulated light.

[0074] Rawdata15_270: 270° phase energy integral diagram under 15Mhz frequency modulated light.

[0075] Dual frequency rotation:

[0076] 1. First, use 120MHz to take 4 pictures. Each picture has 2 phases (2 taps). The exposure time of each picture is the set exposure value (eg. the exposure is set to 1000us, 4 times of taking pictures is 4000us, and there is a waiting time between two pictures, the minimum is about 4ms).

[0077] 2. Then 15MHz, take 4 pictures, each picture has 2 phases (2 taps), and the exposure time of each picture is the set exposure value (eg. the exposure is set to 1000us, 4 photos are taken for 4 times, and there is a waiting time between two pictures, the minimum is about 4ms).

[0078] 3. At each modulation frequency, the tap phase sequence of the four figures is:

[0079] The tap sequence is (0°, 180°) → (90°, 270°) → (180°, 0°) → (270°, 90°).

[0080] The value of each pixel of Rawdata at a certain frequency and phase represents the integral of the target reflected energy at the direction corresponding to this pixel. For the same pixel, the integral energy Q0, Q 90 , Q 180 , Q 270 , the phase offset value of this point can be calculated

[0081]

[0082] Q0 represents the energy integral of the 0 phase of the pixel.

[0083] Q 90 Represents the energy integral of the 0 phase of the pixel.

[0084] Q 180 Represents the energy integral of the 0 phase of the pixel.

[0085] Q 270 Represents the energy integral of the 0 phase of the pixel.

[0086] According to the phase of the pixel The detection distance at this frequency can be calculated, and the corresponding distances d1 and d2 can be calculated for the two frequencies respectively.

[0087]

[0088] The final distance is obtained by dual-frequency solution.

[0089] (1) Calculate the dual-frequency integrated frequency.

[0090] f_max is the greatest common divisor of frequency f1 and frequency f2: when f1 = 15Mhz, f2 = 120Mhz, f_max = 15Mhz

[0091] (2) Calculate M_f1 and M_f2 as follows:

[0092] M_f1=f1 / f_max

[0093] Mf2=f2 / f_max

[0094] Calculate A_f1, A_f2 according to the following formula:

[0095]

[0096] Calculate ω according to the following formula:

[0097]

[0098] The final dual-frequency solution distance is:

[0099] d=d1·ω+d2·(1-ω)

[0100] Depth map generation:

[0101] ①According to the results of phase calibration, the relationship between the actual detected phase and the theoretical phase is obtained;

[0102] ② According to the current actual detected phase, the current real phase can be calculated and converted into the calculated distance, which is the actual detection distance value.

[0103] Depth map converted into point cloud:

[0104] (1) Based on the calibration of the camera’s intrinsic parameters, the azimuth angle corresponding to each pixel can be obtained.

[0105] (2) Based on the depth d and azimuth, the three-dimensional coordinates x, y, and z can be calculated.

[0106] The above three-dimensional space can be understood as a three-dimensional coordinate system used to describe the position and shape of an object. The three-dimensional space is composed of three dimensions: length, width and height. The first three-dimensional space can be a camera monitoring area. The detection area can be set as a cube defined by vertices ABCDEFGH with the camera optical center O as the center, and six tetrahedrons (OABEH, OCDFG, OABCD, OEFGH, OADEF, OBCGH) in the cube as sub-areas.

[0107] The point cloud positions in the first point cloud data correspond to the point cloud positions in the second point cloud data one by one. Each point in the current frame has a point in the reference frame that matches its position.

[0108] The above current frame can be understood as a frame in an image sequence, or a frame captured by a sensor in real time.

[0109] The first point cloud data is point cloud data extracted from the current frame. The reference frame is a predefined frame used for comparison with the current frame. The second point cloud data is point cloud data extracted from the reference frame.

[0110] In one possible embodiment, for example, a surveillance video sequence needs to be processed, the current frame may be a frame in the video, and the reference frame is a reference stable frame. The first point cloud data may be a three-dimensional representation of an object, environment, or other object in the current frame, and the second point cloud data may be a three-dimensional representation of a corresponding object in the reference frame.

[0111] 102. For the first point cloud data and the second point cloud data, calculate the distance value point by point to obtain the distance value at each point cloud position.

[0112] In the embodiment of the present invention, the above-mentioned point-by-point cloud computing distance value can be understood as finding the nearest point in the second point cloud data for each point of the first point cloud data and calculating the Euclidean distance between the two points. The nearest neighbor search algorithm can be used to perform point-by-point cloud computing distance values ​​on the first point cloud data and the second point cloud data to obtain the distance value at each point cloud position.

[0113] The above distance value can reflect the similarity and difference between the two point clouds. The smaller the distance value, the greater the similarity between the two point clouds; conversely, the greater the difference between the two point clouds.

[0114] 103. If the distance value at the point cloud position is greater than or equal to a preset distance threshold, the point cloud position is determined to be a change point.

[0115] In the embodiment of the present invention, the preset distance threshold is a pre-set distance threshold used to determine whether there is a significant distance difference between the relative distances between each point in the first point cloud data and each point in the second point cloud data.

[0116] The distance value at the point cloud position can be calculated. If the distance value at the point cloud position is greater than or equal to a preset distance threshold, it can be considered that there is a significant distance difference between the first point cloud data and the second point cloud data, thereby determining that the point cloud position is a change point.

[0117] 104. Based on the change point, output a safety control signal.

[0118] In the embodiment of the present invention, a corresponding safety control signal may be output according to the change point, and the safety control signal is used to ensure the stability and safety of the system.

[0119] The above safety control signal may be a safety control signal in the form of voice, flashing warning light, stop command, etc.

[0120] In an embodiment of the present invention, first point cloud data corresponding to the current frame and second point cloud data corresponding to the reference frame are obtained, and the point cloud positions in the first point cloud data correspond to the point cloud positions in the second point cloud data one by one; for the first point cloud data and the second point cloud data, the cloud distance value is calculated point by point to obtain the distance value at each point cloud position; if the distance value at the point cloud position is greater than or equal to a preset distance threshold, the point cloud position is determined to be a change point; based on the change point, a safety control signal is output. The present invention obtains the first point cloud data corresponding to the current frame and the second point cloud data corresponding to the reference frame, and for the first point cloud data and the second point cloud data, the cloud distance value is calculated point by point to obtain the distance value at each point cloud position, and outputs a safety control signal based on the change point, thereby solving the problems of the existing methods that fail to detect the continuous position changes of the monitored object in a timely manner, and fail to detect system abnormalities in a timely manner.

[0121] It is understandable that in the specific implementation of this application, point cloud data, image data, device data and other related data are involved. When the embodiments in this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0122] Optionally, in the step of outputting a safety control signal based on the change points, if the number of change points in the current frame is greater than a preset number threshold, a safety control signal is output.

[0123] In the embodiment of the present invention, the number of change points in the current frame can be understood as the number of pixels detected in the current frame image that have changed compared with the previous frame. The number of change points in the current frame can be determined by calculating the pixel motion between adjacent frames.

[0124] The above-mentioned preset quantity threshold is a quantity threshold preset by the system.

[0125] The above safety control signal may be a safety control signal in the form of voice, flashing warning light, stop command, etc.

[0126] In a possible embodiment, for example, in a surveillance video, if the number of change points in the current frame is greater than a preset number threshold, it can be determined that an abnormality has occurred, and a safety control signal is output to remind or take stop action.

[0127] Optionally, in the step of outputting a safety control signal based on the change point, it can be determined based on the change point whether the contour change between the current frame and the reference frame is an unacceptable change; if the contour change is an unacceptable change, a safety control signal is output.

[0128] In the embodiment of the present invention, the above-mentioned contour change can be understood as the appearance change between the current frame and the reference frame.

[0129] The above safety control signal may be a safety control signal in the form of voice, flashing warning light, stop command, etc.

[0130] Based on the change points in the current frame, it can be determined whether the contour change between the current frame and the reference frame is an unacceptable change. If the detected contour change is an unacceptable change, it can be explained that the abnormal change may be caused by motion or other interference. The system will output a safety control signal to ensure the stability and safety of the system.

[0131] Optionally, in the step of judging whether the contour change between the current frame and the reference frame is an unacceptable change based on the change point, contour feature extraction can be performed in the first point cloud data based on the change point to obtain the current contour feature corresponding to the current frame; the correlation between the current contour feature and the reference contour feature is calculated to obtain the correlation value between the current contour feature and the reference contour feature; and based on the correlation value and a preset correlation threshold, judging whether the contour change between the current frame and the reference frame is an unacceptable change.

[0132] In the embodiment of the present invention, the reference contour feature is a contour feature corresponding to the current frame.

[0133] The first point cloud data is point cloud data extracted from the current frame.

[0134] The above-mentioned contour feature extraction can be understood as a process of identifying and extracting the contour features of the object from the first point cloud data. The contour features can be features such as the shape, size, and position of the object.

[0135] The current contour feature may be an edge, shape, contour line, etc. of an object in the current frame. The reference contour feature is a predefined contour feature.

[0136] The above correlation calculation can be understood as a process of calculating the similarity or difference between the current contour feature and the reference contour feature. The correlation calculation is used to determine whether there is a correlation between the current contour feature and the reference contour feature. The above correlation calculation can use the Pearson correlation coefficient, the Spearman rank correlation coefficient, etc.

[0137] The above correlation value refers to the correlation measure between the current profile feature and the reference profile feature. If the correlation value is high, it can be determined that the current system state is stable; if the correlation value is low, it can be determined that the system has abnormalities and noise interference.

[0138] The above preset correlation threshold is a correlation threshold preset by the system, which is used to determine whether the correlation between the two frames is high enough. If the correlation value between the current contour feature and the reference contour feature is lower than the preset correlation threshold, it can be determined whether the contour change between the current frame and the reference frame is an unacceptable change; if the correlation value between the current contour feature and the reference contour feature is lower than the preset correlation threshold but higher than the preset correlation threshold, it can be determined that the contour change between the current frame and the reference frame is an acceptable change.

[0139] Optionally, in the step of calculating the correlation between the current contour feature and the reference contour feature to obtain the correlation value between the current contour feature and the reference contour feature, the current contour feature and the reference contour feature can be contour matched to obtain the degree of matching between the current contour feature and the reference contour feature; based on the degree of matching, the correlation value between the current contour feature and the reference contour feature is determined.

[0140] In the embodiment of the present invention, the above-mentioned contour matching can be understood as a process of comparing the current contour feature with the reference contour feature. Through the contour comparison, the similarity or difference between the current contour feature and the reference contour feature can be determined. The current contour feature and the reference contour feature can be compared using a contour matching algorithm.

[0141] The matching degree between the current contour feature and the reference contour feature reflects the similarity between the current contour feature and the reference contour feature. For example, the matching degree can be a score between 0 and 1, where 1 indicates a complete match and 0 indicates a complete mismatch. The higher the matching degree, the greater the correlation value.

[0142] Optionally, in the step of determining whether the contour change between the current frame and the reference frame is an unacceptable change based on the correlation value and a preset correlation threshold, if the correlation value is greater than or equal to the preset correlation threshold, then the contour change between the current frame and the reference frame is determined not to be an unacceptable change; if the correlation value is less than the preset correlation threshold, then the contour change between the current frame and the reference frame is determined to be an unacceptable change.

[0143] In the embodiment of the present invention, the preset correlation threshold is a correlation threshold preset by the system and is used to measure whether the similarity between the current contour feature and the reference contour feature is an unacceptable change.

[0144] Furthermore, if the correlation value is greater than or equal to a preset correlation threshold, it can be determined that the contour change between the current frame and the reference frame is small or within an acceptable range, and the contour change between the current frame and the reference frame is not an unacceptable change; if the correlation value is less than the preset correlation threshold, it can be determined that the contour change between the current frame and the reference frame is large or exceeds the acceptable range, and the contour change between the current frame and the reference frame is an unacceptable change.

[0145] Optionally, a shielding area of ​​the same size and position is set in the current frame and the reference frame. If the entire target is outside the shielding area, the point cloud data in the shielding area is discarded; if part of the target is outside the shielding area and the other part of the target is inside the shielding area, the point cloud data in the shielding area is retained.

[0146] In an embodiment of the present invention, taking a robot or a robotic arm as an example, the working area of ​​the robot or the robotic arm is a shielding area, and the area outside the shielding area is a protection area or an alarm area. When point cloud data appears in the protection area or the alarm area, it means that an object has invaded the protection area or the alarm area from the shielding area during the working process of the robot or the robotic arm, and a safety control signal can be output to the host computer or computer, thereby controlling the corresponding working equipment to stop working.

[0147] It should be noted that when all targets are outside the shielding area, the point cloud data in the shielding area can be eliminated to reduce the amount of calculation. When part of the target is outside the shielding area and the other part of the target is inside the shielding area, the point cloud data in the shielding area may not be eliminated to protect the integrity of the point cloud data and improve the accuracy of target detection.

[0148] like Figure 2 As shown, an embodiment of the present invention provides a safety monitoring device based on contour recognition, and the safety monitoring device based on contour recognition includes:

[0149] An acquisition module 201 is used to acquire first point cloud data corresponding to a current frame and second point cloud data corresponding to a reference frame, wherein the point cloud positions in the first point cloud data correspond to the point cloud positions in the second point cloud data in a one-to-one manner;

[0150] A first processing module 202 is used to calculate the distance value of each point cloud position for the first point cloud data and the second point cloud data.

[0151] A determination module 203, configured to determine that the point cloud position is a change point if the distance value at the point cloud position is greater than or equal to a preset distance threshold;

[0152] The safety control module 204 is configured to output a safety control signal based on the change point.

[0153] Optionally, the safety control module 204 is further configured to output a safety control signal if the number of the change points in the current frame is greater than a preset number threshold.

[0154] Optionally, the safety control module 204 is further configured to determine, based on the change point, whether a contour change between the current frame and the reference frame is an unacceptable change; if the contour change is an unacceptable change, output a safety control signal.

[0155] Optionally, the security control module 204 is also used to extract contour features in the first point cloud data based on the change point to obtain a current contour feature corresponding to the current frame; calculate the correlation between the current contour feature and a reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature, wherein the reference contour feature is the contour feature corresponding to the current frame; and determine whether the contour change between the current frame and the reference frame is an unacceptable change based on the correlation value and a preset correlation threshold.

[0156] Optionally, the security control module 204 is also used to perform contour matching on the current contour feature and the reference contour feature to obtain a matching degree between the current contour feature and the reference contour feature; based on the matching degree, determine a correlation value between the current contour feature and the reference contour feature, and the higher the matching degree, the greater the correlation value.

[0157] Optionally, the security control module 204 is also used to determine that the contour change between the current frame and the reference frame is not an unacceptable change if the correlation value is greater than or equal to a preset correlation threshold; if the correlation value is less than the preset correlation threshold, then determine that the contour change between the current frame and the reference frame is an unacceptable change.

[0158] Optionally, the current frame and the reference frame are both provided with shielding areas of the same size and position, and the device further includes:

[0159] A second processing module is used to remove the point cloud data in the shielding area if all the targets are outside the shielding area;

[0160] The third processing module is used to retain the point cloud data in the shielding area if a part of the target is outside the shielding area and another part of the target is inside the shielding area.

[0161] like Figure 3 As shown, an embodiment of the present invention further provides an electronic device, including a processor, and the processor can execute any of the above-mentioned security monitoring methods based on contour recognition.

[0162] Specifically, it includes a processor 301 and a memory 302, and a computer program for executing a safety monitoring method based on contour recognition, which is stored in the memory 302 and can be run on the processor 301, wherein:

[0163] The processor 301 runs the computer program of the safety monitoring method based on contour recognition stored in the memory 302, and performs the following steps:

[0164] Acquire first point cloud data corresponding to the current frame and second point cloud data corresponding to the reference frame, wherein the point cloud positions in the first point cloud data correspond one-to-one to the point cloud positions in the second point cloud data;

[0165] For the first point cloud data and the second point cloud data, computing distance values ​​point by point to obtain a distance value at each point cloud position;

[0166] If the distance value at the point cloud position is greater than or equal to a preset distance threshold, determining the point cloud position as a change point;

[0167] Based on the change point, a safety control signal is output.

[0168] Optionally, the outputting of the safety control signal based on the change point performed by the processor 301 includes:

[0169] If the number of the change points in the current frame is greater than a preset number threshold, a safety control signal is output.

[0170] Optionally, the outputting of the safety control signal based on the change point performed by the processor 301 includes:

[0171] Based on the change point, determining whether a change in the contour between the current frame and the reference frame is an unacceptable change;

[0172] If the profile change is an unacceptable change, a safety control signal is output.

[0173] Optionally, the processor 301 performs the step of determining whether a contour change between the current frame and the reference frame is an unacceptable change based on the change point, including:

[0174] Based on the change point, extracting contour features in the first point cloud data to obtain current contour features corresponding to the current frame;

[0175] Calculating the correlation between the current contour feature and the reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature, wherein the reference contour feature is the contour feature corresponding to the current frame;

[0176] Based on the correlation value and a preset correlation threshold, it is determined whether a contour change between the current frame and the reference frame is an unacceptable change.

[0177] Optionally, the processor 301 performs correlation calculation on the current contour feature and the reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature, including:

[0178] Performing contour matching between the current contour feature and the reference contour feature to obtain a matching degree between the current contour feature and the reference contour feature;

[0179] Based on the matching degree, a correlation value between the current contour feature and the reference contour feature is determined, and the higher the matching degree, the greater the correlation value.

[0180] Optionally, the determining, based on the correlation value and a preset correlation threshold, whether a contour change between the current frame and the reference frame is an unacceptable change, performed by the processor 301, includes:

[0181] If the correlation value is greater than or equal to a preset correlation threshold, determining that the contour change between the current frame and the reference frame is not an unacceptable change;

[0182] If the correlation value is less than a preset correlation threshold, it is determined that the contour change between the current frame and the reference frame is an unacceptable change.

[0183] Optionally, the current frame and the reference frame are both provided with shielding areas of the same size and position, and the method executed by the processor 301 further includes:

[0184] If all the targets are outside the shielding area, the point cloud data in the shielding area is eliminated;

[0185] If a portion of the target is outside the shielding area and another portion of the target is within the shielding area, the point cloud data in the shielding area is retained.

[0186] An embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the various processes of the security monitoring method based on contour recognition provided by the embodiment of the present invention are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0187] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0188] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope of the present invention.

Claims

1. A safety monitoring method based on contour recognition, characterized in that: The method comprises the following steps: Acquire first point cloud data corresponding to the current frame and second point cloud data corresponding to the reference frame, wherein the point cloud positions in the first point cloud data correspond one-to-one to the point cloud positions in the second point cloud data; For the first point cloud data and the second point cloud data, computing distance values ​​point by point to obtain a distance value at each point cloud position; If the distance value at the point cloud position is greater than or equal to a preset distance threshold, determining the point cloud position as a change point; Based on the change point, a safety control signal is output.

2. The safety monitoring method based on contour recognition according to claim 1, characterized in that: The step of outputting a safety control signal based on the change point comprises: If the number of the change points in the current frame is greater than a preset number threshold, a safety control signal is output.

3. The safety monitoring method based on contour recognition according to claim 1, characterized in that: The step of outputting a safety control signal based on the change point comprises: Based on the change point, determining whether a change in the contour between the current frame and the reference frame is an unacceptable change; If the profile change is an unacceptable change, a safety control signal is output.

4. The safety monitoring method based on contour recognition according to claim 3, characterized in that: The determining, based on the change point, whether a contour change between the current frame and the reference frame is an unacceptable change comprises: Based on the change point, extracting contour features in the first point cloud data to obtain current contour features corresponding to the current frame; Calculating the correlation between the current contour feature and the reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature, wherein the reference contour feature is the contour feature corresponding to the current frame; Based on the correlation value and a preset correlation threshold, it is determined whether a contour change between the current frame and the reference frame is an unacceptable change.

5. The safety monitoring method based on contour recognition according to claim 4, characterized in that: The step of calculating the correlation between the current contour feature and the reference contour feature to obtain a correlation value between the current contour feature and the reference contour feature includes: Performing contour matching between the current contour feature and the reference contour feature to obtain a matching degree between the current contour feature and the reference contour feature; Based on the matching degree, a correlation value between the current contour feature and the reference contour feature is determined, and the higher the matching degree, the greater the correlation value.

6. The safety monitoring method based on contour recognition according to claim 4, characterized in that: The determining, based on the correlation value and a preset correlation threshold, whether a contour change between the current frame and the reference frame is an unacceptable change includes: If the correlation value is greater than or equal to a preset correlation threshold, determining that the contour change between the current frame and the reference frame is not an unacceptable change; If the correlation value is less than a preset correlation threshold, it is determined that the contour change between the current frame and the reference frame is an unacceptable change.

7. The safety monitoring method based on contour recognition according to any one of claims 1 to 6, characterized in that: The current frame and the reference frame are both provided with shielding areas of the same size and position, and the method further comprises: If all the targets are outside the shielding area, the point cloud data in the shielding area is eliminated; If a portion of the target is outside the shielding area and another portion of the target is within the shielding area, the point cloud data in the shielding area is retained.

8. A safety monitoring device based on contour recognition, characterized in that: The safety monitoring device based on contour recognition comprises: An acquisition module, used to acquire first point cloud data corresponding to a current frame and second point cloud data corresponding to a reference frame, wherein a point cloud position in the first point cloud data corresponds to a point cloud position in the second point cloud data in a one-to-one correspondence; A first processing module is used to calculate the distance value of the first point cloud data and the second point cloud data point by point to obtain the distance value at each point cloud position; A determination module, configured to determine that the point cloud position is a change point if the distance value at the point cloud position is greater than or equal to a preset distance threshold; The safety control module is used to output a safety control signal based on the change point.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the steps in the safety monitoring method based on contour recognition as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps in the safety monitoring method based on contour recognition as described in any one of claims 1 to 7 are implemented.

Images