Flying dust detection method and device, electronic equipment and readable storage medium

Abstract

The invention relates to a flying dust detection method and device, electronic equipment and a readable storage medium. The method comprises the following steps: respectively acquiring a radar point cloud and a depth map of a current frame through a radar and a depth camera; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group; updating a frame queue of the current frame based on the parameter group, the frame queue being formed by a plurality of parameter groups corresponding to continuous frames, and the frame queue comprising the parameter group of the current frame; and determining a fusion score of the parameter group of the current frame, and determining a target detection result of the current frame based on the fusion scores of all the parameter groups in the frame queue. By adopting the method, the dust raising environment can be accurately detected.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of perception detection, in particular to a dust-raising detection method and device, electronic equipment and readable storage medium. BACKGROUND

[0002] The existing cleaning robot, especially the dry cleaning robot, will inevitably produce a large amount of dust in the air when working in heavy dust scenes such as warehouses, underground garages and outdoor environments. In these harsh environments with high dust, if the robot continues to work, the dust will cause serious interference to the key sensors such as laser radar, and even failure, so that the perception data is unavailable, the robot positioning is lost, and the environmental perception is wrong, ultimately causing the robot to be unable to plan a path, unable to move forward, or even directly stuck. Therefore, there is an urgent need for a dust-raising detection method to enable the robot to perceive the dust-raising situation of the current environment in real time and take appropriate measures to ensure the reliability of the operation. SUMMARY

[0003] Therefore, it is necessary to provide a dust-raising detection method, device, electronic equipment and readable storage medium to accurately detect the dust-raising environment for subsequent operations.

[0004] In a first aspect, the present application provides a dust-raising detection method, which comprises:

[0005] acquiring a radar point cloud and a depth map of a current frame by a radar and a depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group;

[0006] updating a frame queue of the current frame based on the parameter group, the frame queue being formed by a plurality of parameter groups corresponding to consecutive frames, and the frame queue including the parameter group of the current frame;

[0007] determining a fusion score of the parameter group of the current frame, and determining a target detection result of the current frame based on the fusion scores of all parameter groups in the frame queue.

[0008] In a second aspect, the present application further provides a dust-raising detection device, which comprises:

[0009] an acquisition unit configured to acquire a radar point cloud and a depth map of a current frame by a radar and a depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group;

[0010] an updating unit configured to update a frame queue of the current frame based on the parameter group, the frame queue being formed by a plurality of parameter groups corresponding to consecutive frames, and the frame queue including the parameter group of the current frame;

[0011] determine a fusion score of the parameter group of the current frame, and determine a target detection result of the current frame based on the fusion scores of all parameter groups in the frame queue.

[0012] In a third aspect, the present application also provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method described above when executing the computer program.

[0013] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the method described above when executed by a processor.

[0014] The dust detection method, device, electronic device and computer readable storage medium described above. The method obtains a radar point cloud and a depth map of a current frame by a radar and a depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group; a frame queue of the current frame is updated based on the parameter group, the frame queue is formed by a plurality of parameter groups corresponding to consecutive frames, and the frame queue includes the parameter group of the current frame; and a target detection result of the current frame is determined based on fusion scores of all parameter groups in the frame queue. The radar point cloud and the depth map of the current frame are synchronously obtained by the radar and the depth camera, the parameter group is formed in time and space alignment, and the parameter groups of consecutive frames are constructed into the frame queue. In a dust environment, the depth camera is easily affected by particle scattering and the like, resulting in distortion or loss of the depth map, while the radar point cloud has strong penetration and can still provide stable spatial information. The system performs multi-frame comprehensive judgment based on the fusion scores of all parameter groups in the frame queue, not only uses current frame data, but also fuses information of historical frames, thereby effectively suppressing instantaneous interference caused by dust and improving the accuracy of environmental judgment in dust and other bad visual conditions. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the description of the embodiments of the present application or the related art. Obviously, the drawings in the following description only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0016] Figure 1 A flowchart of the dust detection method in an embodiment;

[0017] Figure 2 A flowchart of determining the fusion score of the parameter group of the current frame in the dust detection method in an embodiment;

[0018] Figure 3A flowchart of a process of matching a depth map and a radar point cloud based on a parameter set of a current frame in a dust detection method in an embodiment to determine a matching result;

[0019] Figure 4 A flowchart of a process of determining a matching result based on a camera two-dimensional point set and a radar two-dimensional point set in a dust detection method in an embodiment;

[0020] Figure 5 A flowchart of a process of determining a first calculation result in a dust detection method in an embodiment;

[0021] Figure 6 A flowchart of a process of determining a target detection result of a current frame based on fusion scores of all parameter sets in a frame queue in a dust detection method in an embodiment;

[0022] Figure 7 A flowchart of a process of determining a target detection result based on a real-time detection result, a corresponding time point and an effective time period of a historical dust detection result in a dust detection method in an embodiment;

[0023] Figure 8 A structural block diagram of a dust detection device in an embodiment;

[0024] Figure 9 An internal structural diagram of a computer device in an embodiment;

[0025] Figure 10 An internal structural diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0026] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0027] It should be noted that the terms "first", "second" and the like used in the present application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "include" and "have" and any variations thereof used in the present application are intended to cover non-exclusive inclusion. The term "multiple" used in the present application refers to two or more. The term "and / or" used in the present application refers to one of the options or any combination of multiple options.

[0028] In the field of health cleaning, when the existing cleaning robot works in heavy dust scenes such as warehouse, underground garage and outdoor, the cleaning mechanism of the robot will inevitably generate a large amount of dust in the air. In these harsh environments with high dust, if the robot continues to work, the dust will cause serious interference or even failure to the key sensors such as laser radar, so that the perception data is unavailable, the robot positioning is lost, and the environmental perception is wrong, which eventually causes the robot to be unable to plan a path, unable to move forward, or even directly stuck.

[0029] Therefore, in order to solve the above problems, the present application provides a dust detection method.

[0030] The dust detection method provided by the embodiments of the present application can be applied to electronic devices such as robots and detection devices. The electronic device can work in various scenes, including but not limited to heavy dust environments such as warehouses and basements. If applied to a robot, the robot can use the current detection result for subsequent processing (such as stopping work, shielding dust operation, etc.); if applied to a detection device, the detection device transmits the detected result to at least one robot or electronic device in the scene, so that the corresponding robot or electronic device performs subsequent operations.

[0031] In an exemplary embodiment, as shown in Figure 1 A dust detection method is provided. The method is applied to a robot as an example for illustration, but this does not represent a limitation on the application range of the method. The dust detection method includes the following steps S110 to S310.

[0032] Step S110: Obtain the radar point cloud and the depth map of the current frame by radar and depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group.

[0033] Understandably, the embodiment simultaneously acquires the current environment map by using radar and depth camera, and the radar acquires radar point cloud and the depth camera acquires depth map. Among them, the radar point cloud and the depth map photographed by the depth camera and the radar at the same time are taken as the same frame. The same time here refers to that the collection systems of the depth camera and the radar are triggered at the same time within the time accuracy of microseconds or nanoseconds, that is, the error between the first actual time of the depth camera and the second actual time of the radar at the same time is very small (microseconds and nanoseconds), and the first actual time of the depth camera and the second actual time of the radar can be regarded as the same time. For example, if the depth camera acquires a depth map at 12:10 microseconds (10 microseconds after 12:00), and the radar acquires a radar point cloud at 12:00 (12:00), the depth map acquired at 12:10 microseconds and the radar point cloud acquired at 12:00 can be regarded as the same frame. At the same time, the depth map acquired at 12:10 microseconds and the radar point cloud acquired at 12:00 correspond to each other and form a parameter group. One parameter group includes one radar point cloud and one depth map of the same frame.

[0034] Of course, it should also be noted that, in fact, the radar and the depth camera usually have different sampling frequencies due to different hardware characteristics (for example, the radar may be 10 Hz and the depth camera may be 30 Hz), and the radar and the depth camera each have independent time stamps and coordinate systems. Therefore, the above-mentioned "same time" does not mean that the two are completely synchronized in physical time, but means that after time alignment and / or space registration (including time synchronization and coordinate transformation), a set of multi-modal data is regarded as synchronized and aligned at the system processing level, and constitutes a logical frame and a corresponding parameter group. This processing is the basis for subsequent construction of a frame queue, calculation of a fusion score, and improvement of detection accuracy in complex environments such as dust lifting.

[0035] Among them, time alignment refers to that the system selects a pair of radar point cloud and depth map with the smallest time interval from the original data streams of the two sensors through hardware triggering (such as a synchronization signal) or software interpolation / matching algorithms (such as nearest neighbor timestamp matching, linear interpolation, etc.), and regards it as the observation at the "same time". For example, if the radar collects a frame at t=1.02s and the depth camera collects at t=1.018s or t=1.022s, the one with the smallest time difference is selected, and the depth map or the point cloud may be slightly time compensated to align them in time. Space registration refers to converting the radar point cloud from the radar coordinate system to the depth camera coordinate system (or a unified world coordinate system) by using the pre-calibrated external parameters (i.e., the rotation matrix and translation vector between the radar and the depth camera), so that the two correspond in space, thereby realizing pixel-level or voxel-level fusion.

[0036] At step S210, the frame queue of the current frame is updated based on the parameter groups, the frame queue is formed by the parameter groups corresponding to the continuous frames, and the frame queue includes the parameter group of the current frame.

[0037] Understandably, the frame queue is updated in real time, and is formed by the parameter group corresponding to the frame at the current time and all parameter groups corresponding to the continuous multiple historical frames before the current time. Alternatively, the number of parameter groups in the frame queue is n, and the value range of n can be 2-20, of course, other values can also be used, which is not limited here. If n is 10, it means that 10 parameter groups corresponding to 10 continuous frames, one of which corresponds to the current time, and the other 9 parameter groups correspond to historical times at the current time.

[0038] Demonstratively, if the interval time between each frame is 100 milliseconds, and the frame queue includes 3 continuous frames. At 12:00 200 milliseconds, the frame queue is composed of the parameter group at 12:00, the parameter group at 12:00 100 milliseconds, and the parameter group at 12:00 200 milliseconds, that is, it includes 3 radar point clouds and 3 depth maps.

[0039] Alternatively, if a frame does not meet the requirements, the frame is skipped, and the previous frame and the next frame of the frame are used as continuous frames.

[0040] At step S310, the fusion score of the parameter group of the current frame is determined, and the target detection result of the current frame is determined based on the fusion scores of all parameter groups in the frame queue.

[0041] Understandably, each parameter group corresponds to a fusion score, that is, each parameter group in the frame queue corresponds to a fusion score, and the current environment of the current frame can be determined based on the fusion scores of all parameter groups in the frame queue. That is, the target detection result of the current frame is to determine whether the environment of the current frame is a dust environment.

[0042] The embodiment forms a corresponding parameter group by fusing the radar point cloud and the depth map, constructs a frame queue by using continuous frames, and performs target detection in combination with the fusion scores of the parameter groups. In the dust environment, the depth camera is easily disturbed to cause the depth map to be distorted, while the radar point cloud has strong penetration and can still provide reliable spatial information. By dynamically evaluating the fusion score and combining the reliability of the two modalities, the target detection accuracy in the dust and other bad visual conditions is improved.

[0043] In some embodiments, as shown in Figure 2

[0044] At step S220, the depth map and the radar point cloud of the parameter group of the current frame are matched to determine the matching result.

[0045] ​Understandably, due to the scanning principle of the depth camera and the radar, the results detected by the depth camera and the radar are different for the small particles such as dust (there may be some dust detected by the radar, but not detected by the depth camera). Therefore, in order to judge more accurately, the parameters obtained by the depth camera and the radar need to be matched as a reference for subsequent result judgment.

[0046] In step S230, the first calculation result is determined based on the radar point cloud and the preset decision, wherein the preset decision includes dust grid ratio calculation and / or dust-to-radar distance variance calculation.

[0047] In step S240, the second calculation result is determined based on the curvature calculation between the dusts in the radar point cloud.

[0048] In step S250, the third calculation result is determined based on the internal point calculation in the radar body in the radar point cloud.

[0049] Understandably, for the radar, a variety of different ways can be used to judge whether the current frame is in the dust environment, and the judgment result is used as a reference for subsequent judgment. Among them, the dust grid ratio calculation is to project the radar point cloud in a certain range (for example, the range of dust that the radar can detect in the air around the robot) into a grid to judge the proportion of the radar point cloud in the grid. Since there is an irregular distance between the dusts, the curvature feature between the dusts can be used to judge whether it is dust. Due to the false echoes caused by the structure, reflection, shielding or algorithm error of the radar itself, the coordinate position is the internal point of the body coordinate system after coordinate conversion, so the internal point in the radar body can also be calculated as a reference for subsequent dust judgment.

[0050] In step S260, the fusion score of the parameter group of the current frame is determined according to at least one of the matching result, the first calculation result, the second calculation result and the third calculation result.

[0051] Optionally, the matching result, the first calculation result, the second calculation result and the third calculation result each correspond to a weight, and then the fusion score of the corresponding frame can be calculated according to each result and the weight corresponding to each result. Then, according to the comparison between the fusion score and the fusion score threshold, the fusion score judgment result corresponding to the current frame can be obtained.

[0052] Optionally, if the matching result, the first calculation result, the second calculation result and the third calculation result are 0, 0, 1 and 1 respectively, the fusion score is 0.7, which is greater than the fusion score threshold 0.6, indicating that the fusion score judgment result of the current frame is dust.

[0053] Understandably, the fusion score of the parameter group of the current frame can be comprehensively judged by any one or more of the above-mentioned various ways.

[0054] The embodiment describes the distribution density, geometric shape and abnormal echo characteristics of dust through multiple independent radar point cloud analysis methods (including dust grid ratio, point cloud curvature, internal points in the body), enhances the discrimination ability of dust from different angles, and avoids misjudgment of a single indicator. In addition, the difference in dust response (such as radar detectable and depth camera failure) is also used as an auxiliary basis by using depth map and radar point cloud matching. Through the matching result, the consistency or divergence between sensors is identified, which provides a key reference for fusion decision and improves the robustness in unreliable depth data scenarios. Finally, the matching result is combined with at least one of the multiple radar calculation results (first, second and third calculation results) to dynamically generate a fusion score, so that the dust judgment does not depend on a single feature, but is based on multi-feature cooperation to improve accuracy and adaptability.

[0055] In some embodiments, as shown in FIG. 2B, step S220 includes steps S221-S224. Figure 3

[0056] Step S221 converts the depth map into a depth point cloud.

[0057] Step S222 determines a camera two-dimensional point set based on the depth point cloud.

[0058] Step S223 determines a radar two-dimensional point set based on the radar point cloud.

[0059] Step S224 determines a matching result based on the camera two-dimensional point set and the radar two-dimensional point set.

[0060] Understandably, for the depth camera, the depth map needs to be converted into a corresponding depth point cloud, and then the depth point cloud needs to be compressed into a camera two-dimensional point set. This matching method not only makes the parameters obtained by the depth camera and the radar in the same dimension, but also reduces the amount of calculation by reducing the dimension.

[0061] ​Optionally, before determining the camera two-dimensional point set and before determining the radar two-dimensional point set, at least one of the filtering parameters based on the height, angle, distance, etc. of the radar is needed to filter the depth point cloud and the radar point cloud respectively, and the corresponding point cloud of the target region is obtained (for example, filtering the depth point cloud based on the height of the radar to determine the target depth point cloud), so that the required region parameters can be concentrated for calculation, reducing the computing power, and removing unnecessary interference parameters, increasing the accuracy of the judgment.

[0062] In this embodiment, the depth map and the radar point cloud are projected onto the same two-dimensional plane respectively to generate the camera and radar 2D point sets, and then the matching is completed in the plane: on the one hand, the 3D space is reduced to 2D, which greatly reduces the calculation amount; on the other hand, the depth scale and texture edge are still retained after dimension reduction, which improves the robustness of the overall matching.

[0063] In some embodiments, as shown in FIG. 22A, step S224 includes steps S2241-S2244. Figure 4

[0064] Step S2241, selecting the two-dimensional point set with fewer points in the camera two-dimensional point set and the radar two-dimensional point set as the reference point set, and the other two-dimensional point set as the matching point set.

[0065] Understandably, in the matching process, one two-dimensional point set is taken as the reference point set, and the other point set is taken as the matching point set, and the matching point set is matched with the reference point set. It is necessary to find out whether there are obstacle points (i.e. dust) in the same actual position of the two point sets. When the depth camera and the radar both detect that there are obstacle points in the actual position, it indicates that there is dust in the actual position. Due to the difference between the monitoring results of the depth camera and the radar, in order to reduce the computing power, the two-dimensional point set with fewer points is taken as the reference point set, and the other two-dimensional point set with more points is taken as the matching point set.

[0066] Step S2242, determining the reference region range corresponding to each point in the reference point set based on each point in the reference point set; wherein each reference region range corresponds to a matching region range in the matching point set.

[0067] Understandably, the actual position regions corresponding to the camera two-dimensional point set and the radar two-dimensional point set are the same. Therefore, any small region in one of the two-dimensional point sets has a corresponding small region in the other two-dimensional point set, that is, the same actual position region corresponds to two sub-regions (including a sub-region of the reference point set and a sub-region of the matching point set) in the reference point set and the matching point set respectively. Each sub-region corresponds to an obstacle point, which is a point in the reference point set or the matching point set.

[0068] ​Optionally, the reference region range can be a circular region formed with a preset radius with the corresponding point as the center. Of course, it can also be a region range of other shapes (for example, a rectangular region, a triangular region, an irregular region, etc.).

[0069] Step S2243, based on each pair of corresponding reference region range and matching region range, determine the number of matching successes of the point.

[0070] Step S2244, based on the total number of points in the reference point set and the number of matching successes, determine the matching result.

[0071] Understandably, when the corresponding two sub-regions both include obstacle points, it means that the matching of the two sub-regions is successful. If one of the corresponding two sub-regions does not include obstacle points, it means that the matching of the two sub-regions fails. In this way, the number of matching successes can be obtained. Then the matching rate can be calculated according to the number of matching successes and the total number of points in the reference point set. According to the matching rate and the preset matching rate threshold, the matching result is determined.

[0072] Demonstratively, if the total number of points in the reference point set is 20, the number of matching successes is 7, and the matching rate is 35%. If the preset matching rate threshold is 50%, the matching result is a dust-free environment. Optionally, the matching result of 0 indicates that the current environment is determined to be a dust-free environment according to the current matching method, and the matching result of 1 indicates that the current environment is determined to be a dusty environment according to the current matching method.

[0073] In some embodiments, as shown in Figure 5 Step S230 includes steps S231-S235.

[0074] Step S231, determine the grid map corresponding to the radar point cloud.

[0075] Step S232, obtain the number of target grids in the grid map that are not less than the preset probability threshold, and determine the grid ratio value of the number of target grids to the total number of grids in the grid map.

[0076] It can be understood that the radar point cloud is projected into the grid to form a grid map. Of course, the radar point cloud in a certain range (for example, the range of dust that the radar can detect in the air around the robot) is also projected into the grid to form a grid map. Each value in the grid map represents the probability of obstacles at the corresponding position. If it is a 2D radar, the probability is 0 or 1, and the preset probability threshold is 1. When the probability value in the grid map is 1, it indicates that there is an obstacle at the corresponding position of the current grid, that is, dust raising. When it is 0, it indicates that there is no obstacle at the corresponding position of the current grid, that is, no dust raising. If it is a 3D radar, the preset probability threshold can be 0.5. Of course, it can also be other numerical values, which are not limited here. The target grid indicates that there is dust raising at the corresponding actual position. In order to avoid misjudgment, it is necessary to preliminarily confirm that there is dust raising in the current environment only when the target grid and the grid ratio value of all grids are greater than a certain threshold.

[0077] Step S233, determining the grid ratio value times judgment result of the current frame based on the grid ratio values of all the first number of continuous frames.

[0078] It can be understood that if the judgment is made according to a single frame, it may be disturbed by some false points, so the result of single frame judgment is not very accurate. In order to make the result more accurate, the grid ratio value results of continuous multiple frames need to be obtained for comprehensive judgment. Optionally, the value of the first number can be 1, 2, 3, 4, 5, 6, 7, 8 or 9. Of course, it can also be other values, which are not limited here.

[0079] Step S234, determining the distance variance based on the distances from all target grids to the radar.

[0080] It can be understood that the trajectory of each dust raising is irregular, and the distance from each dust raising to the radar is uneven and has a large gap. In order to further judge whether the current environment is a dust raising environment, the distance from each dust raising to the radar also needs to be judged. In the grid map corresponding to the same frame, the distance variance of the current frame is determined according to the distance from each target grid in the grid map to the radar.

[0081] Step S235, determining the distance variance times judgment result of the current frame according to all the distance variances of the second number of continuous frames.

[0082] It can be understood that the above describes the calculation method of the distance variance of each frame. According to all the distance variances corresponding to multiple continuous frames, the total number of distance variances that meet a certain condition among these distance variances can be determined, and the total number is taken as the distance variance times judgment result. The distance variance times judgment result is taken as the basis for subsequent judgment of dust raising. Optionally, the value of the second number can be 1, 2, 3, 4, 5, 6, 7, 8 or 9. Of course, it can also be other values, which are not limited here.

[0083] Step S236, determining an average distance variance judgment result according to all distance variances of the third number of continuous frames.

[0084] Understandably, according to all distance variances corresponding to the plurality of continuous frames, a distance variance average value of the distance variances can also be determined. The average distance variance judgment result is determined according to the distance variance average value. The average distance variance judgment result is used as a basis for subsequent judgment of dust. Alternatively, the third number can be 1, 2, 3, 4, 5, 6, 7, 8 or 9. Of course, it can also be other values, which are not limited here.

[0085] Step S237, determining a first calculation result based on at least one of the grid ratio times judgment result, the distance variance times judgment result and the average distance variance judgment result.

[0086] Understandably, the different dimension judgment methods are comprehensively judged to determine the first calculation result of the current frame, so that the first calculation result is more accurate and has stronger reliability.

[0087] Alternatively, the grid ratio times judgment result, the distance variance times judgment result and the average distance variance judgment result each correspond to a weight. According to the output judgment result corresponding to each method and the corresponding weight, a comprehensive judgment value can be calculated. When the comprehensive judgment value is greater than or equal to a preset comprehensive judgment threshold, the first calculation result is 1, indicating that there is dust; when the comprehensive judgment value is less than the preset comprehensive judgment threshold, the first calculation result is 0, indicating that there is no dust. Alternatively, the weights corresponding to the grid ratio times judgment result, the distance variance times judgment result and the average distance variance judgment result can be 0.3, 0.4 and 0.3, respectively. Alternatively, the preset comprehensive judgment threshold can have a value range of 0.5-0.8, which is not limited here. Alternatively, the grid ratio times judgment result, the distance variance times judgment result and the average distance variance judgment result are each 0 or 1. If 1 is output, it indicates that the judgment result of the corresponding method is that there is dust, and 0 indicates that the judgment result of the corresponding method is that there is no dust.

[0088] The embodiment first converts the radar point cloud into a grid map, which is convenient for quantitative analysis of the distribution density of dust in space, makes the subsequent statistics (such as the number of target grids and the proportion) have a unified spatial reference, and improves the calculation stability. Then, by setting a preset probability threshold, the target grid with high reliability is counted, the noise points or low confidence echoes are effectively filtered, and the recognition accuracy of the real dust area is improved. Then, by judging whether the current is continuously in a high dust state (number of times) through the grid ratio of the first number of continuous frames, the false judgment caused by single frame mutation is avoided; the number of times is judged through the distance variance of the second number of continuous frames to avoid the instability of the dust in the distance dimension; the average distance variance of the third number of continuous frames is calculated to reflect the overall spatial dispersion characteristics of the dust. Finally, at least one of the grid ratio number, the distance variance number and the average distance variance is used to generate a first calculation result, that is, the result is calculated by one or more dimensions, so that the result is more accurate.

[0089] Therefore, the embodiment significantly improves the accuracy and anti-interference ability of judging the dust environment based on the radar point cloud without introducing external information.

[0090] In some embodiments, step S233 comprises:

[0091] In each frame of the first number of continuous frames; when the grid ratio is greater than the preset grid ratio, the grid ratio number cumulative value is determined to be one; when the grid ratio is not greater than the preset grid ratio, the grid ratio number cumulative value is determined to be one.

[0092] Based on the grid ratio number cumulative value of the current frame and the preset grid ratio number threshold, a grid ratio number judgment result of the current frame is determined.

[0093] Understandably, the first number of continuous frames includes the current frame, that is, the current frame and the multiple historical frames before the current frame form the first number of continuous frames. The embodiment mainly determines that the grid ratio number judgment result of the current frame is dust when the first number of continuous frames meet the preset grid ratio number threshold (equivalent to judging that all frames with dust are closer to the current frame). If the method of continuously meeting the preset grid ratio number threshold is used for judgment, the result of this method is no dust when one frame does not meet the preset grid ratio number threshold, which makes the judgment result inaccurate. If the method of the total number of frames meeting the preset grid ratio number threshold in the first number of continuous frames is used for judgment, it is still inaccurate because there are frames with dust before and frames without dust after, including the current frame.

[0094] The embodiment increases the grid ratio times accumulation value by one when the grid ratio is greater than the preset grid ratio, and decreases the grid ratio times accumulation value by one when the grid ratio is less than the preset grid ratio. This method provides a dynamic adjustment mechanism, so that the system can quickly respond to the change of the dust state. Compared with a simple binary judgment (yes or no), this accumulation and decrease method can better reflect the gradual change in the actual environment. In addition, the grid ratio times judgment result is determined based on the grid ratio times accumulation value of the current frame and the preset grid ratio times threshold, instead of relying on the data of a single frame, which enhances the robustness of the system. Even if there is a false judgment in some frames due to noise or other factors, this mechanism can effectively avoid the influence of a single abnormal frame on the overall judgment.

[0095] Furthermore, the embodiment emphasizes the concept of "near-sighted continuity", that is, more attention is paid to whether the continuous frames close to the current frame meet the preset condition, rather than whether all the first number of continuous frames meet the preset condition. The advantage of this design is to improve the sensitivity to the current environmental change while maintaining high judgment accuracy. Because if all frames are required to meet the preset condition, any fluctuation may lead to false judgment. In addition, the embodiment uses the total number of frames that meet the preset grid ratio times threshold to make the judgment, so that not only the number of frames that meet the condition is considered, but also the distribution of these frames in the entire sequence, especially the state closer to the current frame, so as to more accurately reflect whether there is dust in the current environment.

[0096] Therefore, the method of the present scheme dynamically adjusts the grid ratio times accumulation value, and combines the preset grid ratio times threshold to determine whether the current frame is in the dust environment, which not only ensures the sensitivity to the environmental change, but also improves the stability and accuracy of the judgment result. In addition, the importance of data close to the current frame is emphasized, which helps to reduce the error caused by historical data, so that the judgment is more accurate and reliable.

[0097] In some embodiments, step S235 comprises:

[0098] In each frame of the second number of continuous frames; when the distance variance is greater than the preset variance threshold, the distance variance times accumulation value is determined to be increased by one; when the distance variance is not greater than the preset variance threshold, the distance variance times accumulation value is determined to be decreased by one.

[0099] Based on the distance variance times accumulation value of the current frame and the preset variance times threshold, a distance variance times judgment result of the current frame is determined.

[0100] Understandably, the second number of continuous frames includes the current frame, that is, the current frame and a plurality of historical frames before the current frame form the second number of continuous frames. The beneficial effects can be referred to the above description. However, it should be noted that one is judged by the cumulative number of grid ratio values greater than a certain value, and in this embodiment, it is judged by the cumulative number of distance variances greater than a certain value. The two angles of judgment are different, and both can be used as the basis for judgment, that is, judgment is made through multiple dimensions to increase the reliability and accuracy of the judgment result.

[0101] In this embodiment, the distance variance is judged frame by frame in the second number of continuous frames, and the cumulative value of the distance variance frequency is formed by using the cumulative method of "greater than the threshold value, then plus one, otherwise minus one". This mechanism avoids the misjudgment caused by relying only on the single-frame distance variance, making the judgment result smoother and more stable. The second number of continuous frames includes the current frame and its adjacent historical frames, and the change trend of the cumulative value can more sensitively reflect whether the current environment continues to exist in a high distance variance state (that is, the dust particles are irregularly distributed), thereby enhancing the adaptability to the current scene. If a certain frame accidentally appears a higher variance due to noise or other non-dust factors, the cumulative value will decrease if the subsequent frames return to normal, thereby preventing misjudgment due to individual abnormal frames; on the contrary, if multiple frames have a high variance, the cumulative value will steadily increase, reliably triggering dust judgment.

[0102] In some embodiments, step S236 comprises:

[0103] Based on all distance variances of the third number of continuous frames, a distance variance average value is determined; the third number is not greater than the number of continuous frames corresponding to the frame queue.

[0104] Based on the distance variance average value and a preset variance average threshold value, an average distance variance judgment result is determined.

[0105] Understandably, the third number of continuous frames includes the current frame, that is, the current frame and a plurality of historical frames before the current frame form the third number of continuous frames. In this embodiment, the average value of multiple frames is greater than a certain value to determine, which uses a different dimension than the above-mentioned scheme. It should be noted that the continuous frames corresponding to each frame are different. Therefore, the average value corresponding to each frame is also different. For example, if there is a continuous frame queue: the first frame, the second frame, the third frame and the fourth frame. If the third number is 3, for the third frame, all distance variances of the continuous frames corresponding to the third frame include the distance variance of the first frame, the distance variance of the second frame and the distance variance of the third frame, and then the distance variance average value corresponding to the third frame is calculated according to the three distance variances; for the fourth frame, all distance variances of the continuous frames corresponding to the fourth frame include the distance variance of the second frame, the distance variance of the third frame and the distance variance of the fourth frame, and then the distance variance average value corresponding to the fourth frame is calculated according to the three distance variances.

[0106] In this embodiment, by averaging the distance variance of the third number of continuous frames, abnormal fluctuations caused by single frames due to transient interference (such as accidental reflection, sensor jitter, etc.) are effectively suppressed, making the dust judgment basis more stable and reliable. In addition, by limiting the third number to be less than the total number of frames in the frame queue, it is ensured that the average calculation is performed on only a few frames close to the current time, ensuring a timely response to the current environmental state. At the same time, the average judgment result is combined with other judgment results based on the number of accumulations (such as the number of grid ratio times and the number of distance variances), and the dynamic change trend is further considered from the overall average, enhancing the comprehensiveness of multi-dimensional fusion judgment.

[0107] Therefore, the present scheme provides a stable, efficient and physically meaningful dust auxiliary criterion by averaging the distance variance of the local continuous frames at the current time and comparing with the threshold value without exceeding the frame queue range.

[0108] In some embodiments, step S240 comprises:

[0109] Based on the radar point cloud, a plurality of point cloud groups are obtained by clustering.

[0110] The curvature of each point cloud group is determined.

[0111] Based on the total number of curvatures greater than the preset curvature threshold, a second calculation result is determined.

[0112] Understandably, the radar needs to move (such as rotate, etc.) in the process of acquiring the point cloud, and the acquired point cloud is relatively large. Therefore, in order to more accurately reflect the irregularity (greater curvature) between the dust through the curvature between the local point clouds, it is necessary to cluster all point clouds according to the position. After clustering, each point cloud group corresponds to a local curvature. When the local curvature is too large, it is highly probable that the corresponding position is dust. However, the local curvature of a point cloud group cannot accurately determine the dust environment, so it is necessary to determine the dust environment according to the local curvature of multiple point cloud groups, so that the judgment result is more accurate. Only when the total number of all curvatures greater than the preset curvature threshold is greater than a certain value, the current environment is determined to be a dust environment.

[0113] In this embodiment, the radar point cloud data is processed by clustering, which can effectively group the spatially adjacent points into the same group, improving the accuracy. The curvature of each point cloud group is calculated, which can reflect the geometric complexity or surface change degree of the local area.

[0114] In some embodiments, step S250 comprises:

[0115] Based on the radar point cloud, the number of internal points of the radar in the fuselage is determined.

[0116] When the number of internal points is greater than the preset internal point threshold, the third calculation result is determined as 1.

[0117] When the number of internal points is not greater than the preset internal point threshold, the third calculation result is determined as 0.

[0118] Understandably, when there is more dust raising, the laser emitted by the radar will reflect and block false echoes, and the coordinate position after coordinate conversion is inside the body coordinate system. Therefore, when the number of internal points is large, it indicates that the current environment is likely to have dust raising.

[0119] Alternatively, the preset internal point threshold can be in the range of 10-20, for example, 15. Of course, it can also be other value ranges, which are not limited here.

[0120] In this embodiment, by counting the number of point clouds located inside the fuselage (i.e. within the range of the radar structure), it can be determined whether there are a large number of point clouds inside the radar body. This usually indicates that the radar installation position is unreasonable, there is shielding, or the surrounding structure reflects strongly. Since the probability of radar installation position being unreasonable is small, the probability of shielding or surrounding structure reflection causing this phenomenon is large. Therefore, this can be used as one of the important bases in judging the heavy dust scene.

[0121] In some embodiments, as shown in FIG. 3, Figure 6 Based on the fusion scores of all parameter groups in the frame queue, the target detection result of the current frame is determined, including steps S320-S330.

[0122] Step S320, based on the fusion scores of all parameter groups in the frame queue, determine the real-time detection result of the frame queue and the corresponding time of the real-time detection result.

[0123] Step S330, based on the real-time detection result, the corresponding time and the effective time period of the historical dust detection result, determine the target detection result and update the effective time period.

[0124] Understandably, the above illustrates the fusion score calculation process corresponding to each parameter group. If the target detection result of the corresponding frame is directly determined according to the fusion score corresponding to each parameter group, the phenomenon of the result jumping back and forth for several seconds will occur, which not only makes the output result unstable, but also easily causes the robot to be stuck. Therefore, the embodiment increases the buffer time period to improve the stability of the output result. Wherein, the corresponding fusion score of each frame is output, and the real-time detection result of the corresponding frame is determined according to the fusion score. Taking the current frame as an example, the real-time detection result corresponding to the current frame (also the real-time detection result of the current frame queue) and the corresponding time of the real-time detection result can be determined based on the fusion scores of all parameter groups in the frame queue. Wherein, the corresponding time of the real-time detection result can be selected as the time corresponding to the current frame, which is not limited here. Wherein, the historical dust detection result is the dust detection result corresponding to the historical frame before the current frame, and the current frame corresponds to the real-time detection result.

[0125] It should be noted that since each frame corresponds to a frame queue, in order to avoid the fusion score of a single frame caused by accident and some interference, the fusion score of a single frame will not be selected as the real-time detection result corresponding to the single frame. Instead, all fusion scores in the frame queue corresponding to each frame are selected to determine together.

[0126] Optionally, the fusion score judgment result of j frames accumulated in the frame queue is determined as having dust, and the fusion score judgment result of the last i consecutive frames in the frame queue is determined as having dust, and the real-time detection result of the corresponding frame is determined as having dust. Wherein, j is a positive integer and less than or equal to the total number of frames in the frame queue, and i is a positive integer and less than or equal to the total number of frames in the frame queue. For example, when there are 10 frames in the frame queue, the fusion score judgment result of 7 frames accumulated in the frame queue is determined as having dust, and the fusion score judgment result of the last 3 consecutive frames in the frame queue is determined as having dust, which indicates that the real-time detection result of the current frame is having dust.

[0127] In the embodiment, by fusing the fusion scores of all parameter groups in the frame queue, not only the accuracy and anti-interference ability of dust detection are improved, but also the real-time detection result and its accurate corresponding time are determined simultaneously, ensuring that the event can be traced in the time dimension; at the same time, the effective time period of the historical dust detection result is combined to judge and dynamically update the effective time period of the current result, avoiding frequent jumping of the detection result and realizing the continuity of dust detection.

[0128] In some embodiments, as shown in Figure 7 step S330 includes steps S331-S333.

[0129] In step S331, in the case that the real-time detection result is having dust, the target detection result is determined as having dust, and the effective time period is updated based on the corresponding time.

[0130] Understandably, when the real-time detection result is dust raising, and the corresponding effective time period is valid at the current time, it is determined that the target detection result is dust raising, and the effective time period is updated to the preset effective time period. Wherein, the historical dust raising detection result refers to the detection result judged as dust raising. If the real-time detection result is no dust raising, the historical dust raising detection result does not need to be updated, and it can be automatically waited until the corresponding effective time period expires.

[0131] When the real-time detection result is dust raising, and the corresponding effective time period is invalid at the current time, it is determined that the target detection result is dust raising, and the effective time period is updated to the preset effective time period after the current time.

[0132] Exemplarily, if the current time is 12 o'clock, the real-time detection result is dust raising, the preset effective time period is 30 seconds, the corresponding real-time detection result is dust raising, and the corresponding effective time period of the historical dust raising detection result is 11:59:50 to 12:20. The updated effective time period is 12:20 to 12:50, and a new cycle of 30 seconds is performed again.

[0133] Optionally, each time the effective time period is updated, the time of the effective time period is updated to the preset effective time period, such as 10 seconds, 20 seconds, 30 seconds, 40 seconds, 50 seconds, 60 seconds, etc.

[0134] Step S332, in the case that the real-time detection result is no dust raising, and the corresponding time is within the effective time period, it is determined that the target detection result is dust raising.

[0135] Understandably, when the real-time detection result is no dust raising, and the corresponding effective time period is valid at the current time, it is determined that the target detection result is dust raising, and the effective time period does not need to be updated.

[0136] Exemplarily, if the current time is 12 o'clock, the real-time detection result is no dust raising, the preset effective time period is 30 seconds, the corresponding real-time detection result is dust raising, and the corresponding effective time period of the historical dust raising detection result is 11:59:50 to 12:20. The corresponding effective time period of the historical dust raising detection result is still 11:59:50 to 12:20.

[0137] Step S333, in the case that the real-time detection result is no dust raising, and the corresponding time is outside the effective time period, it is determined that the target detection result is no dust raising.

[0138] Understandably, the historical dust detection result may expire at a certain historical frame, and if the detection result corresponding to the subsequent historical frame is no dust, there is no need to update the valid time period and the historical dust detection result. Since it needs to be stationary for a period of time from having dust to no dust, if the current frame detects dust, the detection result in the subsequent period of time may also be dust. The update of the valid time period can ensure that the target detection result output in the subsequent period of time is unchanged, avoiding the target detection result output jumping back and forth. The problem of the output result jumping back and forth in a short period of time is effectively solved.

[0139] The embodiment of the application improves the accuracy of the judgment by comprehensively confirming whether the current working environment is a dust environment in multiple dimensions. At the same time, without adding other sensors, the detection speed is improved, avoiding the situation that the robot has been dead when dust is detected due to slow detection speed, and also reducing the hardware cost.

[0140] For the convenience of understanding, a detailed embodiment is given below.

[0141] I. Acquisition stage:

[0142] The radar point cloud and the depth map of the current frame are acquired by the radar and the depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group.

[0143] The frame queue of the current frame is updated based on the parameter group, the frame queue is formed by a plurality of parameter groups corresponding to consecutive frames, and the frame queue includes the parameter group of the current frame.

[0144] II. Matching stage:

[0145] The depth map is converted into a depth point cloud, and a camera two-dimensional point set is determined based on the depth point cloud;

[0146] A radar two-dimensional point set is determined based on the radar point cloud;

[0147] The two-dimensional point set with fewer points in the camera two-dimensional point set and the radar two-dimensional point set is selected as the reference point set, and the other two-dimensional point set is selected as the matching point set;

[0148] For each point in the reference point set, a reference region range corresponding to each point in the reference point set is determined; wherein each reference region range has a matching region range in the matching point set;

[0149] Based on each pair of corresponding reference region range and matching region range, the number of successful matches of points is determined;

[0150] Step 9, based on the total number of points in the reference point set and the number of successful matches, the matching result is determined.

[0151] III. First calculation result determination stage:

[0152] determine a grid map corresponding to the radar point cloud;

[0153] obtain a number of target grids in the grid map that are not less than a preset probability threshold, and determine a grid ratio value of the number of target grids to a number of all grids in the grid map;

[0154] determine a grid ratio value frequency determination result of the current frame based on all grid ratio values of the first number of continuous frames;

[0155] determine a distance variance based on distances of all target grids to the radar;

[0156] determine a distance variance frequency determination result of the current frame based on all distance variances of the second number of continuous frames;

[0157] determine an average distance variance determination result based on all distance variances of the third number of continuous frames;

[0158] determine a first calculation result based on at least one of the grid ratio value frequency determination result, the distance variance frequency determination result, and the average distance variance determination result.

[0159] IV. Second calculation result determination stage:

[0160] perform clustering based on the radar point cloud to obtain a plurality of point cloud groups, and determine a curvature of each point cloud group;

[0161] determine a second calculation result based on a total number of curvatures greater than a preset curvature threshold.

[0162] V. Third calculation result determination stage:

[0163] determine a number of internal points of the radar in the fuselage based on the radar point cloud;

[0164] when the number of internal points is greater than a preset internal point threshold, determine the third calculation result as 1; when the number of internal points is not greater than the preset internal point threshold, determine the third calculation result as 0.

[0165] VI. Fusion score determination stage:

[0166] determine a fusion score of the parameter group of the current frame based on at least one of the matching result, the first calculation result, the second calculation result, and the third calculation result.

[0167] VII. Target detection result determination stage:

[0168] determine a real-time detection result of the frame queue and a corresponding time of the real-time detection result based on fusion scores of all parameter groups in the frame queue;

[0169] In a case where the real-time detection result is dust raising, the target detection result is determined as dust raising, and the valid time period is updated based on the corresponding moment;

[0170] In a case where the real-time detection result is no dust raising and the corresponding moment is within the valid time period, the target detection result is determined as dust raising. In a case where the real-time detection result is no dust raising and the corresponding moment is outside the valid time period, the target detection result is determined as no dust raising.

[0171] It should be understood that, although each step in the flowchart involved in each of the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in each of the above embodiments can include multiple steps or stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least part of other steps or steps or stages in other steps. It can be understood that the steps in different embodiments can be freely combined as needed, and various non-contradictory schemes formed by combination are within the scope of protection of the present application.

[0172] The present application also provides a dust detection device 400, as shown in Figure 8 The dust detection device 400 comprises:

[0173] The acquisition unit 410 is configured to acquire a radar point cloud and a depth map of a current frame by a radar and a depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group.

[0174] The update unit 420 is configured to update a frame queue of the current frame based on the parameter group, the frame queue being formed by a plurality of parameter groups corresponding to consecutive frames, and the frame queue comprising the parameter group of the current frame.

[0175] The determination unit 430 is configured to determine a target detection result of the current frame based on fusion scores of all parameter groups in the frame queue.

[0176] The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in one or more dust detection device embodiments provided below can refer to the limitations of the dust detection method in the above text, and will not be repeated here.

[0177] The modules in the dust detection device can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in hardware form, or stored in a memory in the computer device in software form, so that the processor can call and execute the operations of the modules.

[0178] In an exemplary embodiment, an electronic device, which can be a robot, has an internal structure diagram as shown in Figure 9 The electronic device includes a processor, a memory, an input / output interface (I / O), and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the electronic device is configured to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the electronic device is configured to store data of a dust detection method. The input / output interface of the electronic device is configured to exchange information between the processor and external devices. The communication interface of the electronic device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a dust detection method.

[0179] In an exemplary embodiment, an electronic device, which can be a terminal, has an internal structure diagram as shown in Figure 10As shown in the figure. The electronic device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the electronic device is used to provide computing and control capability. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The input / output interface of the electronic device is used to exchange information between the processor and external devices. The communication interface of the electronic device is used to communicate with external terminals in a wired or wireless manner. The wireless manner can be realized through WIFI, mobile cellular network, near field communication (NFC) or other technologies. The computer program is executed by the processor to realize a dust detection method. The display unit of the electronic device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the electronic device can be a touch layer overlaid on the display screen, or a key, trackball or touchpad arranged on the shell of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0180] Those skilled in the art can understand that, Figures 9-10 The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device can include more or fewer components than those shown in the figure, or combine certain components, or have a different component arrangement.

[0181] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by the processor to realize the steps of the above method.

[0182] In one embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by the processor to realize the steps of the above method.

[0183] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0184] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when executed, can include the processes of the above-mentioned embodiment methods. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. The non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. The volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., without being limited thereto.

[0185] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0186] The above-described embodiments are merely illustrative of several embodiments of the present application, and the description is relatively specific and detailed, but should not be understood as a limitation on the scope of the patent. It should be noted that for those skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are all within the scope of the present application. Therefore, the scope of protection of the present application should be subject to the appended claims.

Claims

1. A method for detecting dust pollution, characterized in that, The method includes: The radar point cloud and depth map of the current frame are acquired by radar and depth camera respectively; wherein the radar point cloud and depth map of the same frame correspond to each other and form a parameter group; The frame queue of the current frame is updated based on the parameter group, the frame queue being formed by multiple parameter groups corresponding to consecutive frames, and the frame queue including the parameter group of the current frame; Determine the fusion score of the parameter group of the current frame, and based on the fusion scores of all parameter groups in the frame queue, determine the target detection result of the current frame.

2. The method according to claim 1, characterized in that, Determining the fusion score of the parameter group of the current frame includes: The depth map of the parameter group of the current frame and the radar point cloud are matched to determine the matching result; Calculations are performed based on the radar point cloud and preset decisions to determine a first calculation result, wherein the preset decisions include the calculation of the proportion of dust grid and / or the calculation of the variance of the distance from dust to the radar. Based on the radar point cloud, the curvature of the dust-generating area is calculated to determine the second calculation result; Based on the radar point cloud, calculate the internal points within the radar body to determine the third calculation result; The fusion score of the parameter group of the current frame is determined based on at least one of the matching result, the first calculation result, the second calculation result, and the third calculation result.

3. The method according to claim 2, characterized in that, The process of matching the depth map based on the parameter group of the current frame with the radar point cloud to determine the matching result includes: Convert the depth map into a depth point cloud; Based on the depth point cloud, determine the camera's two-dimensional point set; Based on the radar point cloud, a two-dimensional point set for the radar is determined; The matching result is determined based on the camera's two-dimensional point set and the radar's two-dimensional point set.

4. The method according to claim 3, characterized in that, Determining the matching result based on the camera's two-dimensional point set and the radar's two-dimensional point set includes: Select the two-dimensional point set of the camera and the two-dimensional point set of the radar with fewer points as the reference point set, and the other two-dimensional point set as the matching point set; Based on each point in the set of reference points, a reference region range corresponding to each point in the set of reference points is determined; wherein, each reference region range corresponds to a matching region range in the set of matching points. Based on each pair of corresponding reference region ranges and matching region ranges, determine the number of successfully matched points; The matching result is determined based on the total number of reference point clusters and the number of successful matches.

5. The method according to claim 2, characterized in that, The calculation based on the radar point cloud and preset decisions to determine the first calculation result includes: Determine the grid map corresponding to the radar point cloud; Obtain the number of target grid cells in the grid image that is not less than a preset probability threshold, and determine the grid ratio of the number of target grid cells to the total number of grid cells in the grid image; Based on all the raster ratio values ​​of a first number of consecutive frames, determine the number of times the raster ratio value of the current frame is determined; Based on the distances from all the target grids to the radar, the range variance is determined; the range variance count of the current frame is determined based on the range variances of a second number of consecutive frames; the average range variance is determined based on the range variances of a third number of consecutive frames; wherein the first number of consecutive frames, the second number of consecutive frames, and the third number of consecutive frames all include the current frame; The first calculation result is determined based on at least one of the grid ratio determination result, the distance variance determination result, and the average distance variance determination result.

6. The method according to claim 5, characterized in that, The determination of the number of times the raster ratio of the current frame is determined based on all raster ratios of a first number of consecutive frames includes: In each of the first number of consecutive frames; when the raster ratio is greater than a preset raster ratio, the cumulative value of the raster ratio count is incremented by one; when the raster ratio is not greater than the preset raster ratio, the cumulative value of the raster ratio count is decremented by one. The result of the raster ratio count for the current frame is determined based on the cumulative value of the raster ratio count for the current frame and the preset raster ratio count threshold.

7. The method according to claim 5, characterized in that, The determination of the number of distance variance counts for the current frame based on the distance variances of all consecutive frames of the second number includes: In each of the second number of consecutive frames; when the distance variance is greater than a preset variance threshold, the cumulative value of the distance variance count is incremented by one; when the distance variance is not greater than the preset variance threshold, the cumulative value of the distance variance count is decremented by one. The distance variance count judgment result for the current frame is determined based on the cumulative value of the distance variance count in the current frame and the preset variance count threshold.

8. The method according to claim 5, characterized in that, Based on the distance variances of all consecutive frames in the third number of frames, determine the average distance variance judgment result, including: The average distance variance is determined based on the distance variances of all consecutive frames of a third number; the third number is not greater than the number of consecutive frames corresponding to the frame queue. The average distance variance is determined based on the average distance variance and the preset average variance threshold.

9. The method according to claim 2, characterized in that, The calculation of the curvature of the dust field based on the radar point cloud to determine the second calculation result includes: Clustering is performed on the radar point cloud to obtain several point cloud groups; Determine the curvature of each point cloud group; The second calculation result is determined based on the total number of curvatures that are greater than a preset curvature threshold.

10. The method according to claim 2, characterized in that, The calculation of internal points within the radar body based on the radar point cloud, and the determination of the third calculation result, include: Based on the radar point cloud, determine the number of radar points inside the fuselage; When the number of internal points is greater than the preset internal point threshold, the third calculation result is determined to be 1; When the number of internal points is not greater than the preset internal point threshold, the third calculation result is determined to be 0.

11. The method according to claim 1, characterized in that, The determination of the target detection result for the current frame based on the fusion score of all parameter groups in the frame queue includes: Based on the fusion scores of all parameter groups in the frame queue, the real-time detection result of the frame queue and the corresponding time of the real-time detection result are determined. Based on the real-time detection results, the corresponding time, and the effective time period of historical dust detection results, the target detection result is determined and the effective time period is updated.

12. The method according to claim 11, characterized in that, The process of determining the target detection result and updating the effective time period based on the real-time detection result, the corresponding time, and the effective time period of historical dust detection results includes: If the real-time detection result indicates dust, the target detection result is determined to be dust, and the effective time period is updated based on the corresponding time. If the real-time detection result is no dust and the corresponding time is within the effective time period, then the target detection result is determined to be dusty. If the real-time detection result is no dust and the corresponding time is outside the effective time period, the target detection result is determined to be no dust.

13. A dust detection device, characterized in that, include: The acquisition unit is used to acquire the radar point cloud and depth map of the current frame through radar and depth camera respectively; wherein the radar point cloud and the depth map of the same frame correspond to each other and form a parameter group; An update unit is used to update the frame queue of the current frame based on the parameter group, wherein the frame queue is formed by multiple parameter groups corresponding to consecutive frames, and the frame queue includes the parameter group of the current frame. The determining unit is used to determine the fusion score of the parameter group of the current frame, and to determine the target detection result of the current frame based on the fusion scores of all parameter groups in the frame queue.

14. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

Images