Weather condition detection method, device, equipment and storage medium

Abstract

This disclosure provides a weather condition detection method, apparatus, device, and storage medium, which can be applied to the fields of vehicle networking, intelligent driving, and smart cities. The method includes: processing an initial point cloud dataset based on preset transformation rules to obtain a candidate depth image containing a set of candidate point cloud pixels, wherein the initial point cloud dataset includes data obtained after detecting a space to be detected by a target detection device; selecting a first target point cloud pixel from the candidate point cloud pixel set based on the candidate point cloud pixel spacing between different candidate point cloud pixels, wherein the first target point cloud pixel represents at least some features of a target object in the space to be detected; and determining abnormal weather conditions in the space to be detected based on the first target point cloud pixel.

Description

Technical Field

[0001] This disclosure relates to the fields of vehicle networking, intelligent driving, and smart cities, and more specifically, to a weather condition detection method, apparatus, device, storage medium, and program product. Background Technology

[0002] With the rapid development of technology, intelligent driver assistance technology based on detection devices such as LiDAR has been widely applied in passenger vehicles to improve vehicle safety and driving convenience. Simultaneously, intelligent driver assistance technology can also be applied to various scenarios such as unmanned goods transportation, demonstrating its wide range of applications. Among related technologies, intelligent driver assistance technology uses detection devices such as LiDAR to detect targets in the surrounding environment, such as obstacles, and determines target objects around the vehicle based on the detection results. This allows the vehicle to be controlled to achieve autonomous driving or assisted driving functions based on the detected target objects.

[0003] In realizing the concept disclosed herein, the inventors discovered at least the following problems in the related technologies: Under abnormal weather conditions, such as rain or snow, intelligent assisted driving technologies often incorrectly identify raindrops or snowflakes as obstacles, leading to subsequent erroneous driving states such as emergency avoidance. Furthermore, the related technologies lack a method to accurately detect abnormal weather, causing intelligent assisted driving systems to frequently enter erroneous driving states. Summary of the Invention

[0004] In view of this, the present disclosure provides a weather condition detection method, apparatus, equipment, storage medium, and program product.

[0005] One aspect of this disclosure provides a weather condition detection method, comprising:

[0006] The initial point cloud dataset is processed based on a preset transformation rule to obtain a candidate depth image containing a candidate point cloud pixel set. The initial point cloud dataset includes data obtained after the target detection device detects the space to be detected.

[0007] Based on the candidate point cloud pixel set and the candidate point cloud pixel spacing between different candidate point cloud pixels, a first target point cloud pixel is selected from the candidate point cloud pixel set, wherein the first target point cloud pixel represents at least some features of the target object in the detection space; and

[0008] Based on the aforementioned first target point cloud pixels, the abnormal weather conditions in the aforementioned space to be detected are determined.

[0009] According to embodiments of this disclosure, the above-mentioned weather condition detection method further includes:

[0010] Based on the candidate point cloud pixel parameters of adjacent candidate point cloud pixels in the above candidate depth image, calculate the candidate point cloud pixel spacing between adjacent candidate point cloud pixels.

[0011] Specifically, based on the candidate point cloud pixel set mentioned above, the candidate point cloud pixel spacing between different candidate point cloud pixels, the selection of the first target point cloud pixels from the candidate point cloud pixel set includes:

[0012] Based on the comparison results between the above candidate point cloud pixel spacing and the filtering spacing threshold corresponding to the above candidate point cloud pixel spacing, the candidate point cloud pixel corresponding to the target point cloud pixel spacing is determined as the above first target point cloud pixel, wherein the above target point cloud pixel spacing is the candidate point cloud pixel spacing that satisfies the preset condition in the comparison result with the corresponding filtering spacing threshold.

[0013] According to an embodiment of this disclosure, the candidate point cloud pixel spacing includes the first direction candidate point cloud pixel spacing between candidate point cloud pixels adjacent in the first direction and the second direction candidate point cloud pixel spacing between candidate point cloud pixels adjacent in the second direction in the candidate depth image, wherein the first direction and the second direction are two intersecting directions in the candidate depth image.

[0014] Based on the comparison results between the above candidate point cloud pixel spacing and the screening spacing threshold corresponding to the above candidate point cloud pixel spacing, the candidate point cloud pixels corresponding to the target point cloud pixel spacing are determined as the above first target point cloud pixels, including:

[0015] Based on the above first direction candidate point cloud pixel spacing and the first direction comparison result corresponding to the filtering spacing threshold of the above first direction candidate point cloud pixel spacing, the first direction candidate point cloud pixel spacing that satisfies the above preset condition is determined as the first direction target point cloud pixel spacing.

[0016] Delete the target point cloud pixels in the first direction from the above candidate point cloud pixel set to obtain the target candidate point cloud pixel set, wherein the target point cloud pixels in the first direction are candidate point cloud pixels corresponding to the distance between the target point cloud pixels in the first direction;

[0017] Based on the second-direction candidate point cloud pixel spacing between adjacent target candidate point cloud pixels in the aforementioned target candidate point cloud pixel set, and the second-direction comparison result corresponding to the filtering spacing threshold of the aforementioned second-direction candidate point cloud pixel spacing, the second-direction candidate point cloud pixel spacing whose second-direction comparison result satisfies the aforementioned preset condition is determined as the second-direction target point cloud pixel spacing; and

[0018] From the above candidate point cloud pixel set, select the first target point cloud pixels that correspond to the first direction target point cloud pixel spacing and the second direction target point cloud pixel spacing, respectively.

[0019] According to embodiments of this disclosure, the filtering distance threshold corresponding to the pixel spacing of the candidate point cloud is calculated in the following manner:

[0020] Determine the reference candidate point cloud pixels that constitute the above-mentioned candidate point cloud pixel spacing, wherein the reference candidate point cloud pixels are the candidate point cloud pixels that are closest to the above-mentioned target detection device among the candidate point cloud pixels that constitute the above-mentioned candidate point cloud pixel spacing;

[0021] Based on the reference pixel position of the above reference candidate point cloud pixel, the reference distance between the above reference candidate point cloud pixel and the adjacent ray is calculated, wherein the above adjacent ray is the ray corresponding to each of the other candidate point cloud pixels adjacent to the above reference candidate point cloud pixel;

[0022] The above baseline spacing is processed based on a preset filtering spacing rule in order to calculate the filtering spacing threshold corresponding to the pixel spacing of the above candidate point cloud.

[0023] According to embodiments of this disclosure, calculating the reference distance between the reference candidate point cloud pixel and its adjacent ray based on the reference pixel position of the reference candidate point cloud pixel includes:

[0024] Based on the reference pixel depth position of the above-mentioned reference candidate point cloud pixels, determine the virtual candidate point cloud pixels with the above-mentioned reference pixel depth position on the above-mentioned adjacent rays;

[0025] Using the aforementioned reference pixel depth position as a preset radius, and based on the angle between the reference ray and the aforementioned adjacent rays, calculate the reference arc distance between the aforementioned reference candidate point cloud pixel and the aforementioned virtual candidate point cloud pixel, wherein the aforementioned reference arc distance is used as the aforementioned reference spacing; or

[0026] Using the aforementioned reference pixel depth position as the side length of a preset triangle, and based on the angle between the aforementioned reference ray and the aforementioned adjacent ray, the reference straight-line distance between the aforementioned reference candidate point cloud pixel and the aforementioned virtual candidate point cloud pixel is calculated, wherein the aforementioned reference straight-line distance is used as the aforementioned reference spacing.

[0027] The aforementioned reference ray is the ray corresponding to the aforementioned reference candidate point cloud pixel.

[0028] According to embodiments of this disclosure, processing an initial point cloud dataset based on preset transformation rules to obtain a candidate depth image containing a candidate point cloud pixel set includes:

[0029] Based on the detection performance parameters of the aforementioned target detection device, candidate point cloud regions are determined;

[0030] Based on the aforementioned candidate point cloud regions and the initial point cloud positions of the initial point cloud data in the aforementioned initial point cloud dataset, candidate point cloud data located within the aforementioned candidate point cloud regions are determined from the aforementioned initial point cloud dataset; and

[0031] The candidate point cloud data is transformed to obtain a candidate depth image containing the pixel set of the candidate point cloud.

[0032] According to embodiments of this disclosure, the candidate point cloud region is determined based on the detection performance parameters of the target detection device described above, including:

[0033] The first detection range and the second detection range are determined based on the aforementioned detection performance parameters; and

[0034] Based on the difference between the first detection distance and the second detection distance in the space to be detected, candidate point cloud regions in the space to be detected are determined.

[0035] According to embodiments of this disclosure, the above-mentioned detection performance parameters include at least one of the following:

[0036] Angular resolution, detection distance, number of output points.

[0037] According to embodiments of this disclosure, determining the abnormal weather conditions of the space to be detected based on the aforementioned first target point cloud pixels includes:

[0038] The number of second targets is determined based on the difference between the number of candidate point cloud pixels in the aforementioned candidate point cloud pixel set and the number of first targets in the aforementioned first target point cloud pixel set; and

[0039] If the number of the second target is greater than or equal to the preset weather screening threshold, the weather conditions in the space to be detected will be identified as abnormal weather conditions.

[0040] According to embodiments of this disclosure, the above-mentioned weather condition detection method further includes:

[0041] Filter out the candidate point cloud pixels in the above candidate point cloud pixel set that are the same as the second target point cloud pixels to obtain the target detection point cloud pixel set, wherein the above second target point cloud pixels are the other candidate point cloud pixels in the above candidate point cloud pixel set, excluding the above first target point cloud pixels;

[0042] Based on the target detection point cloud pixels in the aforementioned target detection point cloud pixel set, target detection point cloud data corresponding to the aforementioned target detection point cloud pixels are selected from the aforementioned initial point cloud dataset; and

[0043] Based on the aforementioned target detection point cloud data, target object detection is performed on the aforementioned space to be detected.

[0044] According to embodiments of this disclosure, the target detection device includes at least one of the following:

[0045] LiDAR detection device, millimeter-wave radar detection device.

[0046] According to embodiments of this disclosure, the aforementioned abnormal weather conditions include at least one of the following:

[0047] Rainfall, snowfall, hail, and blowing sand.

[0048] Another aspect of this disclosure also provides a weather condition detection device, comprising:

[0049] The point cloud data conversion module is used to process the initial point cloud dataset based on a preset conversion rule to obtain a candidate depth image containing a candidate point cloud pixel set. The initial point cloud dataset includes data obtained after the target detection device detects the space to be detected.

[0050] The first filtering module is configured to filter first target point cloud pixels from the candidate point cloud pixel set based on the candidate point cloud pixel spacing between different candidate point cloud pixels, wherein the first target point cloud pixels represent at least some features of the target object in the detection space; and

[0051] The weather condition determination module is used to determine abnormal weather conditions in the space to be detected based on the first target point cloud pixels.

[0052] Another aspect of this disclosure provides an electronic device comprising:

[0053] One or more processors;

[0054] Memory, used to store one or more programs.

[0055] When the one or more programs are executed by the one or more processors, the one or more processors implement the weather condition detection method as described above.

[0056] Another aspect of this disclosure provides a computer-readable storage medium storing computer-executable instructions, which, when executed, are used to implement the weather condition detection method described above.

[0057] Another aspect of this disclosure provides a computer program product including computer-executable instructions that, when executed, implement the weather condition detection method described above.

[0058] According to embodiments of this disclosure, by processing the initial point cloud dataset detected by the target detection device according to a preset conversion rule, and by selecting first target point cloud pixels that can at least partially characterize the target object from the candidate point cloud pixel set obtained after processing based on the candidate point cloud pixel spacing, factors unrelated to the weather conditions in the initial point cloud dataset can be effectively analyzed. Thus, factors unrelated to the weather conditions in the detection space can be accurately filtered out based on the first target number of the first target point cloud pixels, thereby at least partially improving the detection accuracy of the weather conditions in the detection space. Furthermore, based on the detected abnormal weather conditions, the technical effect of improving the working stability of the relevant intelligent driving assistance system can be achieved. Attached Figure Description

[0059] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0060] Figure 1 This illustration schematically shows an exemplary system architecture for applying weather condition detection methods and apparatus according to embodiments of the present disclosure;

[0061] Figure 2 A flowchart illustrating a weather condition detection method according to an embodiment of the present disclosure is shown schematically.

[0062] Figure 3 The flowchart illustrates a process of processing an initial point cloud dataset according to a preset transformation rule to obtain a candidate depth image containing a candidate point cloud pixel set, according to an embodiment of the present disclosure.

[0063] Figure 4 The illustration shows a flowchart of determining a candidate point cloud pixel corresponding to a target point cloud pixel as a first target point cloud pixel based on a comparison result between the candidate point cloud pixel spacing and a filtering spacing threshold corresponding to the candidate point cloud pixel spacing, according to an embodiment of the present disclosure.

[0064] Figure 5 A schematic diagram illustrating the calculation of the filtering spacing threshold according to an embodiment of the present disclosure is shown.

[0065] Figure 6 The flowchart of a weather condition detection method according to another embodiment of the present disclosure is illustrated schematically;

[0066] Figure 7 A block diagram schematically illustrates a weather condition detection device according to an embodiment of the present disclosure; and

[0067] Figure 8 A block diagram of an electronic device suitable for implementing a weather condition detection method according to an embodiment of the present disclosure is shown schematically. Detailed Implementation

[0068] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0069] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0070] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0071] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0072] In the technical solution disclosed herein, the user's authorization or consent is obtained before acquiring or collecting the user's personal information.

[0073] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information comply with the provisions of relevant laws and regulations, necessary confidentiality measures have been taken, and there is no violation of public order and good morals.

[0074] In relevant intelligent driver assistance systems, target detection devices such as LiDAR and millimeter-wave radar are typically used to detect objects in the target space. Based on the detected objects, the system can control vehicles to perform emergency avoidance maneuvers and other operational transitions. However, in abnormal weather conditions such as rain or snow, target detection devices are affected by raindrops, snowflakes, and other abnormal weather factors, generating noise interference. This noise can trigger the intelligent driver assistance system to make collision judgments, leading to incorrect braking or emergency stopping control signals. When the intelligent driver assistance system is informed of abnormal weather conditions such as rain or snow in the target space, it will use different strategies to cope. However, there is no optimal method for judging abnormal weather conditions. For example, weather conditions can be manually input into the intelligent driver assistance system, or images can be captured by a camera and then processed using visual algorithms to determine rain or snow conditions.

[0075] However, the aforementioned methods for judging weather conditions have poor real-time performance and low accuracy. For example, using visual algorithms to judge rain or snow weather by taking pictures with a camera: this method has high requirements for camera image quality and algorithm, especially in rainy weather, when it is difficult for the camera to take clear pictures and the algorithm to accurately identify the weather, resulting in low accuracy in weather judgment.

[0076] This is because rainy and snowy weather conditions are complex, making it difficult for intelligent driver assistance systems to automatically perceive the current weather conditions. Tiny raindrops or snowflakes are difficult for cameras to capture and remove. While target acquisition devices such as lidar collect point cloud data of the space to be detected in rainy and snowy weather, intelligent driver assistance systems struggle to accurately distinguish between noisy point clouds formed by rain and snow and real obstacles.

[0077] Embodiments of this disclosure provide a weather condition detection method, apparatus, device, storage medium, and program product. The weather condition detection method includes: processing an initial point cloud dataset based on preset transformation rules to obtain a candidate depth image containing a candidate point cloud pixel set, wherein the initial point cloud dataset includes data obtained after probing a space to be detected by a target detection device; selecting a first target point cloud pixel from the candidate point cloud pixel set based on the candidate point cloud pixel spacing between different candidate point cloud pixels, wherein the first target point cloud pixel represents at least some features of a target object in the space to be detected; and determining abnormal weather conditions in the space to be detected based on the first target point cloud pixel.

[0078] According to embodiments of this disclosure, by processing the initial point cloud dataset detected by the target detection device according to a preset conversion rule, and by selecting first target point cloud pixels that can at least partially characterize the target object from the candidate point cloud pixel set obtained after processing based on the candidate point cloud pixel spacing, factors unrelated to the weather conditions in the initial point cloud dataset can be effectively analyzed. Thus, factors unrelated to the weather conditions in the detection space can be accurately filtered out based on the first target number of the first target point cloud pixels, thereby at least partially improving the detection accuracy of the weather conditions in the detection space. Furthermore, based on the detected abnormal weather conditions, the technical effect of improving the working stability of the relevant intelligent driving assistance system can be achieved.

[0079] It should be noted that the application scenarios of the embodiments of this disclosure can include applications in intelligent transportation tools such as unmanned vehicles and drones to control or assist the operation of intelligent transportation tools. However, it is not limited to this, and can also be applied to application scenarios such as urban intelligent security monitoring systems, for example, in intelligent detection devices such as security monitoring cameras. The embodiments of this disclosure do not limit the specific application scenarios of weather condition detection methods, devices, equipment, and storage media, and those skilled in the art can select them according to actual needs.

[0080] Figure 1 This illustration schematically depicts an exemplary system architecture to which weather condition detection methods and apparatus can be applied according to embodiments of this disclosure. It should be noted that... Figure 1 The examples shown are merely examples of system architectures that can be applied to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0081] like Figure 1 As shown, the system architecture 100 according to this embodiment may include a vehicle 101, a target object 102, a raindrop 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the vehicle 101 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0082] Vehicle 101 may include a vehicle equipped with target detection devices such as lidar, and target object 102 may be a moving object such as a pedestrian in the detection space 110. Alternatively, the target object may also be a fixed moving object in the detection space. Accordingly, vehicle 101 may have electronic devices capable of processing the initial point cloud dataset detected by the target detection device, including but not limited to electronic devices such as chips and processors for data processing.

[0083] It should be understood that a target detection device installed on vehicle 101 can be used to detect the space 110 to obtain an initial point cloud dataset. Alternatively, another target detection device installed outside vehicle 101 can be used to detect the space 110 to obtain an initial point cloud dataset, and then send the initial point cloud dataset to vehicle 101 and / or server 105 to implement the weather condition detection method provided in this embodiment of the present disclosure.

[0084] Vehicle 101 interacts with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on vehicle 101.

[0085] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using terminal devices (for example only). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0086] It should be noted that the weather condition detection method provided in this embodiment can generally be executed by electronic equipment in vehicle 101. Correspondingly, the weather condition detection device provided in this embodiment can generally be installed in vehicle 101. Alternatively, the weather condition detection method provided in this embodiment can be executed by server 105 capable of communicating with vehicle 101. Correspondingly, the weather condition detection device provided in this embodiment can also be installed in server 105. The weather condition detection method provided in this embodiment can also be executed by a server or server cluster different from server 105 and capable of communicating with vehicle 101 and / or server 105. Alternatively, the weather condition detection device provided in this embodiment can also be installed in a server or server cluster different from server 105 and capable of communicating with vehicle 101 and / or server 105.

[0087] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0088] Figure 2 A flowchart illustrating a weather condition detection method according to an embodiment of the present disclosure is shown schematically.

[0089] like Figure 2 As shown, the weather condition detection method may include operations S210 to S230.

[0090] In operation S210, the initial point cloud dataset is processed based on a preset transformation rule to obtain a candidate depth image containing a candidate point cloud pixel set. The initial point cloud dataset includes data obtained after the target detection device detects the space to be detected.

[0091] According to embodiments of this disclosure, the initial point cloud dataset can represent any target to be detected in the detection space, such as target objects like obstacles to be detected, as well as noise objects like raindrops and snowflakes in abnormal weather conditions.

[0092] According to embodiments of this disclosure, the target detection device may include a device capable of detecting target objects in the space to be detected, such as a lidar device. The embodiments of this disclosure do not limit the specific type of target detection device, and those skilled in the art can select one according to actual needs.

[0093] According to embodiments of this disclosure, the candidate point cloud pixels included in the candidate point cloud pixel set can constitute a candidate depth image. These candidate point cloud pixels can be pixel data obtained by converting part or all of the initial point cloud dataset using point cloud data conversion methods in related technologies.

[0094] It should be noted that the embodiments of this disclosure do not limit the specific data transformation method for converting the initial point cloud data to candidate point cloud pixels. For example, it may include a coordinate transformation method based on the camera intrinsic parameter matrix to convert the initial point cloud data to candidate point cloud pixels, but it is not limited to this. Those skilled in the art can choose a specific data transformation method according to actual needs.

[0095] In operation S220, based on the candidate point cloud pixel spacing between different candidate point cloud pixels in the candidate point cloud pixel set, a first target point cloud pixel is selected from the candidate point cloud pixel set, wherein the first target point cloud pixel represents at least some features of the target object in the detection space.

[0096] It should be noted that the candidate point cloud pixel spacing can be calculated by calculating the point cloud pixel positions of different candidate point cloud pixels, or it can be calculated by calculating the point cloud data positions of the initial point cloud data corresponding to the candidate point cloud pixels. The embodiments of this disclosure do not limit the specific calculation method of the candidate point cloud pixel spacing, and those skilled in the art can choose according to actual needs.

[0097] In operation S230, based on the first target point cloud pixels, abnormal weather conditions in the space to be detected are determined.

[0098] According to embodiments of this disclosure, the candidate point cloud pixel spacing between different candidate point cloud pixels can characterize the distance between different initial point cloud data corresponding to the candidate point cloud pixel. By using the candidate point cloud pixel spacing, the first target point cloud pixels that are relatively densely distributed in the candidate depth image can be selected. Thus, the first target point cloud pixel can be set as the target object in the detection space. In this way, the target object in the candidate depth image can be accurately selected by using the first target point cloud pixel, eliminating the interference of the target object in the candidate depth image. Furthermore, the point cloud pixels in the candidate depth image other than the first target point cloud pixel can be analyzed more accurately, improving the detection accuracy for noisy objects such as raindrops and snowflakes, and improving the detection accuracy of abnormal weather conditions.

[0099] According to embodiments of this disclosure, by processing the initial point cloud dataset detected by the target detection device according to a preset conversion rule, and by selecting first target point cloud pixels that can at least partially characterize the target object from the candidate point cloud pixel set obtained after processing based on the candidate point cloud pixel spacing, factors unrelated to the weather conditions in the initial point cloud dataset can be effectively analyzed. Thus, factors unrelated to the weather conditions in the detection space can be accurately filtered out based on the first target number of the first target point cloud pixels, thereby at least partially improving the detection accuracy of the weather conditions in the detection space. Furthermore, based on the detected abnormal weather conditions, the technical effect of improving the working stability of the relevant intelligent driving assistance system can be achieved.

[0100] According to embodiments of this disclosure, the target detection device includes at least one of the following:

[0101] LiDAR detection device, millimeter-wave radar detection device.

[0102] According to embodiments of this disclosure, the lidar detection device may include any type of lidar detection device, such as phased array lidar in related technologies. The embodiments of this disclosure do not limit the specific device type of the lidar detection device. Correspondingly, the embodiments of this disclosure do not limit the specific device type of the millimeter-wave radar detection device, and those skilled in the art can select one according to actual needs.

[0103] It should be noted that the embodiments disclosed herein do not limit the specific installation location of the target detection device or the specific performance parameters of the target detection device, and those skilled in the art can make selections according to actual needs.

[0104] According to embodiments of this disclosure, abnormal weather conditions include at least one of the following:

[0105] Rainfall, snowfall, hail, and blowing sand.

[0106] It should be understood that when there are abnormal weather conditions in the space to be detected, the point cloud pixels in the candidate depth image, excluding the first target point cloud pixels, can represent at least part of the noisy objects such as raindrops, snowflakes, hail or dust under abnormal weather conditions.

[0107] Figure 3 The flowchart illustrates a process of processing an initial point cloud dataset according to a preset transformation rule to obtain a candidate depth image containing a candidate point cloud pixel set, based on an embodiment of the present disclosure.

[0108] like Figure 3 As shown, operation S210, which processes the initial point cloud dataset based on a preset transformation rule to obtain a candidate depth image containing a candidate point cloud pixel set, may include operations S310 to S330.

[0109] In operation S310, candidate point cloud regions are determined based on the detection performance parameters of the target detection device.

[0110] In operation S320, based on the candidate point cloud region and the initial point cloud positions of the initial point cloud data in the initial point cloud dataset, candidate point cloud data located in the candidate point cloud region are determined from the initial point cloud dataset.

[0111] In operation S330, the candidate point cloud data is transformed to obtain a candidate depth image containing the candidate point cloud pixel set.

[0112] According to embodiments of this disclosure, detection performance parameters can affect the detection results, such as the quality and quantity of point cloud data in the initial point cloud dataset obtained by the target detection device. For example, a target detection device with stronger detection performance, as indicated by the detection performance parameters, can detect a larger quantity of point cloud data, while a target detection device with weaker detection performance, as indicated by the detection performance parameters, can detect a relatively smaller quantity of point cloud data. Appropriate candidate point cloud regions can be selected in the detection space based on the detection performance parameters to balance the impact of the detection performance parameters on subsequent weather condition detection results.

[0113] According to embodiments of this disclosure, the initial point cloud position of each initial point cloud data can be represented by its respective point cloud data coordinates. Candidate point cloud data located within a candidate point cloud region can then be selected from the initial point cloud dataset using candidate point cloud regions.

[0114] It should be noted that the embodiments disclosed herein do not limit the specific method of data conversion, and those skilled in the art can design it according to actual needs.

[0115] In one embodiment of this disclosure, candidate point cloud data can be converted into candidate point cloud pixels based on the following formula (1).

[0116] (1)

[0117] In formula (1), (u, v) are the coordinates of the candidate point cloud pixels (i.e., the position of the candidate point cloud pixels), (x, y, z) represent the coordinates of the candidate point cloud data (i.e., the position of the candidate point cloud data), row_scal represents the image height of the candidate depth image, which can be the same as the number of ray beams of the target detection device. For example, for a lidar detection device with 16 ray beams, row_scal=16; col_scale represents the image width of the candidate depth image, which can be the same as the number of scanning beams after each ray beam of the lidar scans for one cycle. R represents the distance between the candidate point cloud data and the target detection device, which can be expressed by formula (2).

[0118] (2);

[0119] FOV up FOV represents the angle from the horizontal position of the target detection device to its uppermost field of view.

[0120] According to embodiments of this disclosure, the detection performance parameters include at least one of the following:

[0121] Angular resolution, detection distance, number of output points.

[0122] According to embodiments of this disclosure, when the target detection device is a lidar detection device, the angular resolution can be the scanning interval angle of the scanning laser beams emitted by the lidar. The angular resolution can include horizontal angular resolution and / or vertical angular resolution.

[0123] According to embodiments of this disclosure, when the target detection device is a lidar detection device, the number of output points may include the number of laser points emitted by the lidar per second, i.e., the number of laser rays.

[0124] According to embodiments of this disclosure, operation S310, determining the candidate point cloud region based on the detection performance parameters of the target detection device, may include the following operations:

[0125] The first detection distance and the second detection distance are determined based on the detection performance parameters; and the candidate point cloud region in the space to be detected is determined based on the difference between the first detection distance and the second detection distance in the space to be detected.

[0126] In one embodiment of this disclosure, a first detection distance and a second detection distance can be determined based on the angular resolution, thereby selecting a candidate point cloud region corresponding to the angular resolution in the space to be detected. This candidate point cloud region can be determined based on the difference distance; that is, when 'a' represents the first detection distance and 'b' represents the second detection distance, the difference distance 'r' can satisfy 'b≤r≤a', and thus the candidate point cloud region in the space to be detected can be determined based on the difference distance 'r'.

[0127] It should be noted that when the angular resolution is small, a smaller difference distance is selected to determine a relatively small candidate point cloud region, and when the angular resolution is large, a larger difference distance is selected to determine a relatively large candidate point cloud region, thereby balancing the error caused by the different angular resolutions.

[0128] According to embodiments of this disclosure, the difference distance can also be determined based on other detection performance parameters. The embodiments of this disclosure do not limit the type of specific detection performance parameters for determining the difference distance.

[0129] According to embodiments of this disclosure, by determining candidate point cloud regions in the space to be detected based on difference distances, the candidate point cloud regions can filter out initial point cloud data representing noisy objects that are close to the target detection device in the initial point cloud dataset, and filter out initial point cloud data representing target objects that are far from the target detection device in the initial point cloud dataset. This improves the accuracy of the subsequently determined first target point cloud pixels in representing the target object, avoids confusing the target object and the noise object in the space to be detected, and thus improves the detection accuracy of weather conditions.

[0130] According to embodiments of this disclosure, the weather condition detection method may further include the following operations:

[0131] Based on the candidate point cloud pixel parameters of adjacent candidate point cloud pixels in the candidate depth image, the candidate point cloud pixel spacing between adjacent candidate point cloud pixels is calculated.

[0132] Operation S220, which involves selecting the first target point cloud pixel from the candidate point cloud pixel set based on the distance between different candidate point cloud pixels, may include the following operations:

[0133] Based on the comparison result between the candidate point cloud pixel spacing and the filtering spacing threshold corresponding to the candidate point cloud pixel spacing, the candidate point cloud pixel corresponding to the target point cloud pixel spacing is determined as the first target point cloud pixel, wherein the target point cloud pixel spacing is the candidate point cloud pixel spacing that satisfies the preset condition when the comparison result with the corresponding filtering spacing threshold is obtained.

[0134] According to embodiments of this disclosure, the candidate point cloud pixel parameters may include the pixel coordinates (u, v) of each candidate point cloud pixel. Since the angle between adjacent detection rays in the scanning detection rays emitted by the target detection device is fixed, i.e., the angular resolution is fixed, the farther away from the target detection device, the farther the distance between the initial point cloud data detected by two adjacent detection rays, and the larger the inherent spacing between the point cloud pixels formed after the initial point cloud data is projected onto the sphere. Therefore, the corresponding filtering spacing threshold can be determined based on the point cloud pixel coordinates of the candidate point cloud pixels corresponding to the candidate point cloud pixel spacing, in order to balance the error caused by the positional difference of adjacent candidate point cloud pixels from the target detection device.

[0135] According to embodiments of this disclosure, the pixel spacing of the target point cloud can be less than or less than or equal to the filtering spacing threshold, that is, the comparison result can satisfy less than or less than or equal to the threshold, thereby filtering out adjacent candidate point cloud pixels that are closer in distance in the candidate depth image. Then, the sparseness or compactness of the candidate point cloud pixels can be represented by the comparison result, and the first target point cloud pixels that are more compact and represent the target object can be filtered out, thereby improving the accuracy of determining the first target point cloud pixels.

[0136] According to embodiments of this disclosure, the candidate point cloud pixel spacing includes a first-direction candidate point cloud pixel spacing between candidate point cloud pixels adjacent in a first direction and a second-direction candidate point cloud pixel spacing between candidate point cloud pixels adjacent in a second direction in the candidate depth image, wherein the first direction and the second direction are two intersecting directions in the candidate depth image.

[0137] Figure 4 The illustration shows a flowchart of determining a candidate point cloud pixel corresponding to a target point cloud pixel as a first target point cloud pixel based on a comparison result between the candidate point cloud pixel spacing and a filtering spacing threshold corresponding to the candidate point cloud pixel spacing, according to an embodiment of the present disclosure.

[0138] like Figure 4 As shown, in the above operation, determining the candidate point cloud pixel corresponding to the target point cloud pixel as the first target point cloud pixel based on the comparison result between the candidate point cloud pixel spacing and the filtering spacing threshold corresponding to the candidate point cloud pixel spacing may include operations S410~S440.

[0139] In operation S410, based on the first direction candidate point cloud pixel spacing and the first direction comparison result corresponding to the filtering spacing threshold of the first direction candidate point cloud pixel spacing, the first direction candidate point cloud pixel spacing that meets the preset condition is determined as the first direction target point cloud pixel spacing.

[0140] In operation S420, the target point cloud pixels in the first direction are deleted from the candidate point cloud pixel set to obtain the target candidate point cloud pixel set, wherein the target point cloud pixels in the first direction are the candidate point cloud pixels corresponding to the distance between the target point cloud pixels in the first direction.

[0141] In operation S430, based on the second direction candidate point cloud pixel spacing between adjacent target candidate point cloud pixels in the target candidate point cloud pixel set, and the second direction comparison result corresponding to the filtering spacing threshold of the second direction candidate point cloud pixel spacing, the second direction candidate point cloud pixel spacing that meets the preset condition is determined as the second direction target point cloud pixel spacing.

[0142] In operation S440, the first target point cloud pixels corresponding to the first direction target point cloud pixel spacing and the second direction target point cloud pixel spacing are respectively selected from the candidate point cloud pixel set.

[0143] According to the embodiments of this disclosure, there may be a directional angle between the first direction and the second direction. The directional angle can be any angle value. The embodiments of this disclosure do not limit the specific angle value of the directional angle, as long as the first direction and the second direction intersect. Those skilled in the art can make the selection according to actual needs.

[0144] In one embodiment of this disclosure, the first direction and the second direction can be two mutually perpendicular directions, such as the horizontal and vertical directions in the candidate depth image. Therefore, adjacent candidate point cloud pixels in the candidate depth image can be traversed in the horizontal direction. The candidate point cloud pixel spacing in the first direction, which is less than or equal to a corresponding screening threshold, is determined as the first direction target point cloud pixel spacing. Similarly, the candidate point cloud pixel spacing in the second direction, which is less than or equal to a corresponding screening threshold, is determined as the second direction target point cloud pixel spacing. This allows for the selection of the first target point cloud pixel from the set of mutually selected point cloud pixels.

[0145] According to embodiments of this disclosure, by deleting target point cloud pixels in the first direction from the first candidate point cloud pixel set, the computational load for calculating the spacing between candidate point cloud pixels in the second direction can be reduced, saving computational overhead and thereby improving the overall efficiency of weather condition detection.

[0146] According to embodiments of this disclosure, the filtering distance threshold corresponding to the pixel spacing of the candidate point cloud is calculated in the following manner:

[0147] A baseline candidate point cloud pixel constituting the candidate point cloud pixel spacing is determined, wherein the baseline candidate point cloud pixel is the candidate point cloud pixel closest to the target detection device among the candidate point cloud pixels constituting the candidate point cloud pixel spacing; based on the baseline pixel position of the baseline candidate point cloud pixel, the baseline spacing between the baseline candidate point cloud pixel and its adjacent rays is calculated, wherein the adjacent rays are the rays corresponding to other candidate point cloud pixels adjacent to the baseline candidate point cloud pixel; the baseline spacing is processed based on a preset screening spacing rule in order to calculate the screening spacing threshold corresponding to the candidate point cloud pixel spacing.

[0148] According to embodiments of this disclosure, since candidate point cloud pixels are obtained by data conversion of initial point cloud data, the ray corresponding to the candidate point cloud pixel can be determined based on the ray corresponding to each of the initial point cloud data.

[0149] According to embodiments of this disclosure, the location of a reference pixel can be represented by the pixel coordinates of a reference candidate point cloud pixel.

[0150] According to embodiments of this disclosure, calculating the reference distance between a reference candidate point cloud pixel and an adjacent ray based on the reference pixel position of the reference candidate point cloud pixel may include the following operations:

[0151] Based on the reference pixel depth position of the reference candidate point cloud pixel, determine the virtual candidate point cloud pixels with the reference pixel depth position on the adjacent ray.

[0152] Using the reference pixel depth position as a preset radius, and based on the angle between the reference ray and adjacent rays, calculate the reference arc distance between the reference candidate point cloud pixels and the virtual candidate point cloud pixels, where the reference arc distance serves as the reference spacing; or

[0153] Using the reference pixel depth position as the side length of a preset triangle, the reference straight-line distance between the reference candidate point cloud pixel and the virtual candidate point cloud pixel is calculated based on the angle between the reference ray and the adjacent ray. The reference straight-line distance is used as the reference spacing.

[0154] Among them, the reference ray is the ray corresponding to the reference candidate point cloud pixel.

[0155] Figure 5 A schematic diagram illustrating the calculation of the filtering spacing threshold according to an embodiment of the present disclosure is shown.

[0156] like Figure 5As shown in Figure (a), the candidate point cloud region 510 can contain candidate point cloud data corresponding to horizontally adjacent candidate point cloud pixels in the candidate depth image. The candidate point cloud data in the candidate point cloud region 510 can correspond to adjacent candidate point cloud pixels A and B. The ray corresponding to candidate point cloud pixel A is ray 511, and the ray corresponding to candidate point cloud pixel B is ray 512. Candidate point cloud pixel A can be determined as the reference candidate point cloud pixel. The virtual candidate point cloud pixel C on ray 512 has the same reference pixel depth position (i.e., distance from the target detection device) as candidate point cloud pixel A.

[0157] By using the reference pixel depth position and the angular resolution α of the target detection device, the reference arc distance 51AC can be calculated, and thus the reference spacing determined by the reference arc distance can be obtained.

[0158] Alternatively, the baseline straight distance 52AC can be calculated using the following formula (3).

[0159] 52AC = 2×r×sin(α); (3)

[0160] In formula (3), r can represent the distance between candidate point cloud pixel A and the target detection device.

[0161] like Figure 5 As shown in Figure (b), the candidate point cloud region 520 can contain candidate point cloud data corresponding to adjacent candidate point cloud pixels in the horizontal direction of the candidate depth image. The candidate point cloud data in the candidate point cloud region 520 can correspond to adjacent candidate point cloud pixels C and D. The ray corresponding to candidate point cloud pixel C is ray 521, and the ray corresponding to candidate point cloud pixel D is ray 522. The angle between ray 521 and ray 522 is γ.

[0162] Using the same or similar methods, the baseline straight-line distance and / or baseline arc distance corresponding to candidate point cloud pixel C and candidate point cloud pixel D can be calculated using the corresponding algorithm in formula (3), that is, the baseline spacing corresponding to candidate point cloud pixel C and candidate point cloud pixel D can be determined.

[0163] After determining the baseline spacing, the screening spacing threshold can be calculated based on the following formula (4).

[0164] m = k·d; (4)

[0165] In formula (4), m represents the screening interval threshold, k represents the preset screening interval rule parameter, and d represents the baseline interval.

[0166] According to embodiments of this disclosure, reference is made to Figure 5As shown in Figure (a), when the distance between candidate point cloud pixel A and candidate point cloud pixel B is greater than the corresponding filtering interval threshold, candidate point cloud pixel A and candidate point cloud pixel B can be identified as point cloud pixels representing noise objects such as raindrops and snowflakes in the space to be detected. Accordingly, in Figure 5 In Figure (b), if the distance between candidate point cloud pixel C and candidate point cloud pixel D is less than or equal to the corresponding filtering interval threshold, candidate point cloud pixel C and candidate point cloud pixel D can be determined as the first target point cloud pixel. Furthermore, it can be intuitively seen that candidate point cloud pixel C and candidate point cloud pixel D can represent some features of the target object 531.

[0167] It should be noted that in practical applications, the angle between adjacent rays emitted by a target detection device is usually small. Figure 5 The placement of the intermediate rays is only for clearly illustrating the weather detection method provided in the embodiments of this disclosure, and does not represent the actual interval between the rays.

[0168] It should be understood that, for multi-line lidar detection devices, the number of ray beams of the lidar detection device can represent the vertical height of the candidate depth image, and the length range of the laser beam after scanning one cycle in the horizontal direction can be used as the horizontal width of the candidate depth image.

[0169] According to embodiments of this disclosure, operation S230, determining abnormal weather conditions in the space to be detected based on the first target point cloud pixels, may include the following operations:

[0170] The number of second targets is determined based on the difference between the number of candidate point cloud pixels in the candidate point cloud pixel set and the number of first targets in the first target point cloud pixel set; and if the number of second targets is greater than or equal to a preset weather screening threshold, the weather conditions in the space to be detected are determined as abnormal weather conditions.

[0171] According to embodiments of this disclosure, based on the difference between the number of candidate point cloud pixels and the number of first targets, a second target number of second target point cloud pixels representing at least some noise objects such as raindrops in the space to be detected can be obtained. Therefore, if the number of second targets is greater than or equal to a preset weather filtering threshold, the second target point cloud representing noise objects in the space to be detected can be determined.

[0172] According to the embodiments of this disclosure, the preset weather screening threshold can be pre-set. For example, the specific value range of the preset weather screening threshold can be set according to the performance parameters of the corresponding target detection device, the candidate point cloud region characterizing the candidate depth image, and other factors. The embodiments of this disclosure do not limit the specific value range of the preset weather screening threshold.

[0173] In one embodiment of this disclosure, the target detection device is a lidar detection device. The initial preset weather filtering threshold can be corrected based on the amount of noise point cloud data detected by the lidar detection device, thereby obtaining the preset weather filtering threshold.

[0174] According to embodiments of this disclosure, when the target detection device detects multiple frames of initial point cloud datasets, the same or similar weather condition detection method is applied to each frame of initial point cloud dataset to obtain the first target number corresponding to each frame of initial point cloud dataset, and then the second target number corresponding to each frame of initial point cloud dataset is obtained. After accumulating the second target numbers corresponding to each of the multiple frames of initial point cloud datasets, the obtained cumulative second target number is compared with the cumulative preset weather filtering threshold. Thus, if the cumulative second target number is greater than or equal to the cumulative preset weather filtering threshold, the weather condition in the space to be detected can be determined as an abnormal weather condition.

[0175] According to embodiments of this disclosure, the anomalous level of abnormal weather conditions can also be classified based on the difference between the second target quantity (or the cumulative second target quantity) and a preset weather screening threshold (cumulative preset weather screening threshold). For example, after calculating the difference between the second target quantity (or the cumulative second target quantity) and the preset weather screening threshold (cumulative preset weather screening threshold), this difference is used as the abnormal weather detection result. If the abnormal weather detection result is greater than the first-level result, the anomalous level of the abnormal weather condition is determined to be first-level abnormality. If the abnormal weather detection result is between the first-level result and the second-level result, the anomalous level can be determined to be second-level. Correspondingly, the anomalous level can also be associated with meteorological levels in relevant weather forecasting fields, thereby accurately determining the meteorological level corresponding to the abnormal weather in the space to be detected, so that intelligent driving assistance systems or related algorithm systems can adjust algorithm strategies in a timely manner, improving the stability and adaptability of vehicle control in relevant autonomous driving and intelligent assisted driving fields.

[0176] Figure 6 The flowchart of a weather condition detection method according to another embodiment of the present disclosure is illustrated schematically.

[0177] like Figure 6 As shown, the weather condition detection method may also include operations S610 to S630.

[0178] In operation S610, candidate point cloud pixels that are the same as the second target point cloud pixels in the candidate point cloud pixel set are filtered out to obtain the target detection point cloud pixel set, wherein the second target point cloud pixels are the other candidate point cloud pixels in the candidate point cloud pixel set, excluding the first target point cloud pixels.

[0179] In operation S620, target detection point cloud data corresponding to the target detection point cloud pixels are selected from the initial point cloud dataset based on the target detection point cloud pixels in the target detection point cloud pixel set.

[0180] When operating the S630, target object detection is performed in the space to be detected based on target detection point cloud data.

[0181] According to embodiments of this disclosure, the second target point cloud pixels can characterize noisy objects in the space to be detected. By filtering out the second target point cloud pixels, noise data in the space to be detected can be filtered out at least partially. Thus, the target detection point cloud data corresponding to the target detection point cloud pixel set can be used to detect the target object. This can at least partially avoid the interference caused by noise objects such as raindrops and snowflakes to the target object detection under abnormal weather conditions, improve the detection accuracy of the target object in the space to be detected, and realize the operational stability of the intelligent assisted driving system and the unmanned intelligent control system.

[0182] Figure 7 A block diagram of a weather condition detection device according to an embodiment of the present disclosure is shown schematically.

[0183] like Figure 7 As shown, the weather condition detection device 700 includes a point cloud data conversion module 710, a first filtering module 720, and a weather condition determination module 730.

[0184] The point cloud data conversion module 710 is used to process the initial point cloud dataset based on a preset conversion rule to obtain a candidate depth image containing a candidate point cloud pixel set. The initial point cloud dataset includes data obtained after the target detection device detects the space to be detected.

[0185] The first screening module 720 is used to screen out a first target point cloud pixel from the candidate point cloud pixel set based on the candidate point cloud pixel spacing between different candidate point cloud pixels in the candidate point cloud pixel set, wherein the first target point cloud pixel represents at least some features of the target object in the detection space.

[0186] The weather condition determination module 730 is used to determine abnormal weather conditions in the space to be detected based on the first target point cloud pixels.

[0187] According to embodiments of this disclosure, the weather condition detection device further includes a first calculation module.

[0188] The first calculation module is used to calculate the candidate point cloud pixel spacing between adjacent candidate point cloud pixels based on the candidate point cloud pixel parameters of each adjacent candidate point cloud pixel in the candidate depth image.

[0189] The first screening module includes: a first determination submodule.

[0190] The first determining unit is used to determine the candidate point cloud pixel corresponding to the target point cloud pixel as the first target point cloud pixel based on the comparison result between the candidate point cloud pixel spacing and the filtering spacing threshold corresponding to the candidate point cloud pixel spacing. The target point cloud pixel spacing is the candidate point cloud pixel spacing that satisfies the preset condition when the comparison result with the corresponding filtering spacing threshold is obtained.

[0191] According to embodiments of this disclosure, the candidate point cloud pixel spacing includes a first-direction candidate point cloud pixel spacing between candidate point cloud pixels adjacent in a first direction and a second-direction candidate point cloud pixel spacing between candidate point cloud pixels adjacent in a second direction in the candidate depth image, wherein the first direction and the second direction are two intersecting directions in the candidate depth image.

[0192] The first determination submodule includes: a first determination unit, a first deletion unit, a second determination unit, and a first filtering unit.

[0193] The first determining unit is used to determine the first direction candidate point cloud pixel spacing that meets the preset conditions as the first direction target point cloud pixel spacing based on the first direction candidate point cloud pixel spacing and the first direction comparison result corresponding to the first direction candidate point cloud pixel spacing.

[0194] The first deletion unit is used to delete the target point cloud pixels in the first direction from the candidate point cloud pixel set to obtain the target candidate point cloud pixel set, wherein the target point cloud pixels in the first direction are the candidate point cloud pixels corresponding to the distance between the target point cloud pixels in the first direction.

[0195] The second determining unit is used to determine the second direction candidate point cloud pixel spacing that meets the preset conditions as the second direction target point cloud pixel spacing based on the second direction candidate point cloud pixel spacing between adjacent target candidate point cloud pixels in the target candidate point cloud pixel set and the second direction comparison result corresponding to the filtering spacing threshold of the second direction candidate point cloud pixel spacing.

[0196] The first filtering unit is used to filter out the first target point cloud pixels from the candidate point cloud pixel set, which correspond to the first target point cloud pixel spacing in the first direction and the second target point cloud pixel spacing in the second direction.

[0197] According to embodiments of this disclosure, the filtering distance threshold corresponding to the pixel spacing of the candidate point cloud is calculated in the following manner:

[0198] A baseline candidate point cloud pixel constituting the candidate point cloud pixel spacing is determined, wherein the baseline candidate point cloud pixel is the candidate point cloud pixel closest to the target detection device among the candidate point cloud pixels constituting the candidate point cloud pixel spacing; based on the baseline pixel position of the baseline candidate point cloud pixel, the baseline spacing between the baseline candidate point cloud pixel and its adjacent rays is calculated, wherein the adjacent rays are the rays corresponding to other candidate point cloud pixels adjacent to the baseline candidate point cloud pixel; the baseline spacing is processed based on a preset screening spacing rule in order to calculate the screening spacing threshold corresponding to the candidate point cloud pixel spacing.

[0199] According to embodiments of this disclosure, calculating the reference spacing between a reference candidate point cloud pixel and an adjacent ray based on the reference pixel position of the reference candidate point cloud pixel includes:

[0200] Based on the reference pixel depth position of the reference candidate point cloud pixel, determine the virtual candidate point cloud pixel with the reference pixel depth position on the adjacent ray; using the reference pixel depth position as a preset radius, calculate the reference arc distance between the reference candidate point cloud pixel and the virtual candidate point cloud pixel based on the angle between the reference ray and the adjacent ray, where the reference arc distance is used as the reference spacing; or using the reference pixel depth position as a preset triangle side length, calculate the reference straight line distance between the reference candidate point cloud pixel and the virtual candidate point cloud pixel based on the angle between the reference ray and the adjacent ray, where the reference straight line distance is used as the reference spacing; wherein, the reference ray is the ray corresponding to the reference candidate point cloud pixel.

[0201] According to embodiments of this disclosure, the point cloud data conversion module includes: a second determining submodule, a third determining submodule, and a point cloud data conversion submodule.

[0202] The second determination submodule is used to determine the candidate point cloud region based on the detection performance parameters of the target detection device.

[0203] The third determination submodule is used to determine the candidate point cloud data located in the candidate point cloud region from the initial point cloud dataset based on the candidate point cloud region and the initial point cloud positions of the initial point cloud data in the initial point cloud dataset.

[0204] The point cloud data conversion submodule is used to convert candidate point cloud data to obtain a candidate depth image containing the candidate point cloud pixel set.

[0205] According to embodiments of this disclosure, the second determining submodule includes: a detection distance determining unit and a point cloud region determining unit.

[0206] The detection range determination unit is used to determine the first detection range and the second detection range based on the detection performance parameters.

[0207] The point cloud region determination unit is used to determine candidate point cloud regions in the space to be detected based on the difference between the first detection distance and the second detection distance in the space to be detected.

[0208] According to embodiments of this disclosure, the detection performance parameters include at least one of the following:

[0209] Angular resolution, detection distance, number of output points.

[0210] According to embodiments of this disclosure, the weather condition determination module includes a fourth determination submodule and a fifth determination submodule.

[0211] The fourth determining submodule is used to determine the number of second targets based on the difference between the number of candidate point cloud pixels in the candidate point cloud pixel set and the number of first targets in the first target point cloud pixels.

[0212] The fifth determination submodule is used to determine the weather conditions in the space to be detected as abnormal weather conditions when the number of second targets is greater than or equal to the preset weather screening threshold.

[0213] According to embodiments of this disclosure, the weather condition detection device further includes: a point cloud pixel filtering module, a target point cloud data filtering module, and a target object detection module.

[0214] The point cloud pixel filtering module is used to filter out candidate point cloud pixels that are the same as the second target point cloud pixels in the candidate point cloud pixel set, so as to obtain the target detection point cloud pixel set. The second target point cloud pixels are the other candidate point cloud pixels in the candidate point cloud pixel set, excluding the first target point cloud pixels.

[0215] The target point cloud data filtering module is used to filter target detection point cloud data corresponding to the target detection point cloud pixels in the target detection point cloud pixel set from the initial point cloud dataset.

[0216] The target object detection module performs target object detection in the space to be detected based on the target detection point cloud data.

[0217] According to embodiments of this disclosure, the target detection device includes at least one of the following:

[0218] LiDAR detection device, millimeter-wave radar detection device.

[0219] According to embodiments of this disclosure, abnormal weather conditions include at least one of the following:

[0220] Rainfall, snowfall, hail, and blowing sand.

[0221] Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure, or at least part of the functions of any one or more of them, can be implemented in one module. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be implemented by dividing them into multiple modules. Any one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as hardware circuitry, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Array (PLA), a System-on-Chip, a System-on-a-Substrate, a System-on-Package, an Application-Specific Integrated Circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, one or more of the modules, submodules, units, and subunits according to embodiments of the present disclosure can be at least partially implemented as computer program modules, which, when run, can perform corresponding functions.

[0222] For example, any multiple of the point cloud data conversion module 710, the first filtering module 720, and the weather condition determination module 730 can be combined into one module / submodule / submodule / unit / subunit, or any one of these modules / submodules / units / subunits can be split into multiple modules / submodules / units / subunits. Alternatively, at least part of the functionality of one or more of these modules / submodules / units / subunits can be combined with at least part of the functionality of other modules / submodules / units / subunits and implemented in one module / submodule / unit / subunit. According to embodiments of this disclosure, at least one of the submodules can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the submodules may be implemented at least partially as a computer program module, which, when run, can perform corresponding functions.

[0223] It should be noted that the weather condition detection device part in the embodiments of this disclosure corresponds to the weather condition detection method part in the embodiments of this disclosure. For a detailed description of the weather condition detection device part, please refer to the weather condition detection method part, which will not be repeated here.

[0224] Figure 8A block diagram of an electronic device suitable for implementing a weather condition detection method according to an embodiment of the present disclosure is shown schematically. Figure 8 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0225] like Figure 8 As shown, an electronic device 800 according to an embodiment of this disclosure includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this disclosure.

[0226] RAM 803 stores various programs and data required for the operation of electronic device 800. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in said one or more memories.

[0227] According to embodiments of this disclosure, the electronic device 800 may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device 800 may also include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0228] According to embodiments of this disclosure, the method flow according to embodiments of this disclosure can be implemented as a computer software program. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of embodiments of this disclosure. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0229] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0230] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium. Examples include, but are not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0231] For example, according to embodiments of this disclosure, a computer-readable storage medium may include the ROM 802 and / or RAM 803 described above and / or one or more memories other than ROM 802 and RAM 803.

[0232] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods provided in the embodiments of this disclosure. When the computer program product is run on an electronic device, the program code is used to enable the electronic device to implement the weather condition detection method provided in the embodiments of this disclosure.

[0233] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0234] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0235] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0236] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions. Those skilled in the art will understand that the features recited in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not expressly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure may be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0237] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A weather condition detection method, comprising: processing an initial point cloud data set based on a preset conversion rule to obtain a candidate depth image containing a candidate point cloud pixel set, wherein the initial point cloud data set comprises data obtained after a target detection device detects a space to be detected; determining a candidate point cloud pixel corresponding to a target point cloud pixel interval as a first target point cloud pixel according to a comparison result of a candidate point cloud pixel interval between adjacent candidate point cloud pixels in the candidate depth image and a screening interval threshold corresponding to the candidate point cloud pixel interval, wherein the target point cloud pixel interval is a candidate point cloud pixel interval whose comparison result with the corresponding screening interval threshold satisfies a preset condition, and the first target point cloud pixel represents at least part of a feature of a target object in the space to be detected; determining an abnormal weather condition of the space to be detected based on the first target point cloud pixel; wherein the screening interval threshold corresponding to the candidate point cloud pixel interval is calculated by: determining a reference candidate point cloud pixel constituting the candidate point cloud pixel interval, wherein the reference candidate point cloud pixel is a candidate point cloud pixel constituting the candidate point cloud pixel interval closest to the target detection device; calculating a reference interval between the reference candidate point cloud pixel and an adjacent ray according to a reference pixel position of the reference candidate point cloud pixel, wherein the adjacent ray is a ray corresponding to each of other candidate point cloud pixels adjacent to the reference candidate point cloud pixel; processing the reference interval based on a preset screening interval rule to calculate the screening interval threshold corresponding to the candidate point cloud pixel interval. 2.The method of claim 1, further comprising: calculating a candidate point cloud pixel interval between adjacent candidate point cloud pixels according to candidate point cloud pixel parameters of the adjacent candidate point cloud pixels in the candidate depth image.

3. The method of claim 2, wherein, The candidate point cloud pixel interval comprises a first direction candidate point cloud pixel interval between candidate point cloud pixels adjacent in a first direction and a second direction candidate point cloud pixel interval between candidate point cloud pixels adjacent in a second direction, the first direction and the second direction being two directions intersecting in the candidate depth image; determining the candidate point cloud pixel corresponding to the target point cloud pixel interval as the first target point cloud pixel according to a comparison result of the candidate point cloud pixel interval and the screening interval threshold corresponding to the candidate point cloud pixel interval comprises: determining a first direction target point cloud pixel interval according to a first direction comparison result of the first direction candidate point cloud pixel interval and a screening interval threshold corresponding to the first direction candidate point cloud pixel interval, the first direction comparison result satisfying the preset condition; deleting a first direction target point cloud pixel in the candidate point cloud pixel set to obtain a target candidate point cloud pixel set, wherein the first direction target point cloud pixel is a candidate point cloud pixel corresponding to the first direction target point cloud pixel interval. According to the target candidate point cloud pixel set, a second direction candidate point cloud pixel interval between adjacent target candidate point cloud pixels, and a second direction comparison result of a screening interval threshold corresponding to the second direction candidate point cloud pixel interval, a second direction candidate point cloud pixel interval satisfying the preset condition in the second direction comparison result is determined as a second direction target point cloud pixel interval; and From the candidate point cloud pixel set, a first target point cloud pixel corresponding to each of the first direction target point cloud pixel interval and the second direction target point cloud pixel interval is screened out respectively.

4. The method of claim 2, wherein, According to the reference pixel position of the reference candidate point cloud pixel, calculating a reference interval between the reference candidate point cloud pixel and an adjacent ray comprises: According to the reference pixel depth position of the reference candidate point cloud pixel, determining a virtual candidate point cloud pixel on the adjacent ray having the reference pixel depth position; Taking the reference pixel depth position as a preset radius, according to an included angle between the reference ray and the adjacent ray, calculating a reference circular arc distance between the reference candidate point cloud pixel and the virtual candidate point cloud pixel, wherein the reference circular arc distance is taken as the reference interval; or Taking the reference pixel depth position as a preset triangle side length, according to the included angle between the reference ray and the adjacent ray, calculating a reference straight line distance between the reference candidate point cloud pixel and the virtual candidate point cloud pixel, wherein the reference straight line distance is taken as the reference interval; Wherein, the reference ray is a ray corresponding to the reference candidate point cloud pixel.

5. The method of claim 1, wherein, Processing an initial point cloud data set based on a preset conversion rule to obtain a candidate depth image containing a candidate point cloud pixel set comprises: According to the detection performance parameters of the target detection device, determining a candidate point cloud region; According to the candidate point cloud region, and initial point cloud positions of initial point cloud data in the initial point cloud data set respectively, determining candidate point cloud data located in the candidate point cloud region from the initial point cloud data set; and Converting the candidate point cloud data to obtain the candidate depth image containing the candidate point cloud pixel set.

6. The method of claim 5, wherein, According to the detection performance parameters of the target detection device, determining a candidate point cloud region comprises: According to the detection performance parameters, determining a first detection distance and a second detection distance; and Based on a difference distance of the first detection distance and the second detection distance in the to-be-detected space, determining a candidate point cloud region in the to-be-detected space.

7. The method of claim 5, wherein, The detection performance parameters include at least one of the following: Angular resolution, detection distance, and out-point number.

8. The method of claim 1, wherein, Based on the first target point cloud pixel, determining an abnormal weather condition of the to-be-detected space comprises: According to a difference between a candidate point cloud pixel number of a candidate point cloud pixel in the candidate point cloud pixel set and a first target number of the first target point cloud pixel, determining a second target number; and In a case where the second target number is greater than or equal to a preset weather screening threshold, determining a weather condition in the to-be-detected space as an abnormal weather condition.

9. The method of claim 8, further comprising: filtering out candidate point cloud pixels same as a second target point cloud pixel from the candidate point cloud pixel set, to obtain a target detection point cloud pixel set, wherein the second target point cloud pixel is a candidate point cloud pixel in the candidate point cloud pixel set other than the first target point cloud pixel; screening target detection point cloud data corresponding to the target detection point cloud pixel from the initial point cloud data set according to the target detection point cloud pixel in the target detection point cloud pixel set; and performing target object detection on the space to be detected based on the target detection point cloud data.

10. The method of any one of claims 1 to 9, wherein, The target detection device includes at least one of: a laser radar detection device, a millimeter wave radar detection device.

11. The method according to any one of claims 1 to 9, wherein, The abnormal weather condition includes at least one of: rainy weather condition, snowy weather condition, hail weather condition, sandstorm weather condition.

12. A weather condition detection device, comprising: a point cloud data conversion module configured to process an initial point cloud data set based on a preset conversion rule to obtain a candidate depth image including a candidate point cloud pixel set, wherein the initial point cloud data set includes data obtained by detecting a space to be detected by a target detection device; a first screening module configured to screen a first target point cloud pixel from the candidate point cloud pixel set according to a candidate point cloud pixel distance between different candidate point cloud pixels in the candidate point cloud pixel set, wherein the first target point cloud pixel represents at least part of a feature of a target object in the space to be detected; and a weather condition determination module configured to determine an abnormal weather condition of the space to be detected based on the first target point cloud pixel. The first screening module is configured to: determine a candidate point cloud pixel corresponding to a target point cloud pixel distance as the first target point cloud pixel according to a comparison result of a candidate point cloud pixel distance between adjacent candidate point cloud pixels in the candidate depth image and a screening distance threshold corresponding to the candidate point cloud pixel distance, wherein the target point cloud pixel distance is a candidate point cloud pixel distance corresponding to a comparison result of the screening distance threshold satisfying a preset condition; wherein the screening distance threshold corresponding to the candidate point cloud pixel distance is calculated by: determining a reference candidate point cloud pixel constituting the candidate point cloud pixel distance, wherein the reference candidate point cloud pixel is a candidate point cloud pixel closest to the target detection device among candidate point cloud pixels constituting the candidate point cloud pixel distance; calculating a reference distance between the reference candidate point cloud pixel and an adjacent ray according to a reference pixel position of the reference candidate point cloud pixel, wherein the adjacent ray is a ray corresponding to each of other candidate point cloud pixels adjacent to the reference candidate point cloud pixel; processing the reference distance based on a preset screening distance rule to calculate the screening distance threshold corresponding to the candidate point cloud pixel distance.

13. An electronic device, comprising: one or more processors; a memory for storing one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to carry out the method of any one of claims 1 to 11.

14. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to carry out the method of any one of claims 1 to 11.

15. A computer program product comprising a computer program that, when executed by a processor, carries out the method according to any one of claims 1 to 11.

Images