Radar lens stain detection device suitable for mine environment

Abstract

The invention provides a radar lens stain detection method and device suitable for a mine environment, and the method comprises the steps: obtaining the prior information of a radar scanning window, and the prior information comprises the curvature and roughness of each point of the radar scanning window in a clean state; acquiring three-dimensional point cloud data and an infrared image of the radar scanning window, which are respectively acquired by the structured light camera and the infrared camera; converting the infrared image into a binary image, and communicating a black area of the binary image to obtain an area to be detected; based on the three-dimensional point cloud data of the radar scanning window, obtaining the three-dimensional point cloud data of the to-be-detected area; the curvature and roughness of each point in the three-dimensional point cloud data of the to-be-detected area are calculated; determining whether the radar lens is polluted by stains or not based on the curvature and roughness of each point in the three-dimensional point cloud data of the to-be-detected area and the prior information; the problem that radar lens stain detection does not have universality in the prior art is solved.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a radar lens stain detection device suitable for a mine environment. Background Art

[0002] In a mine environment, radar can be used to collect point clouds on site, and target recognition and other operations can be performed through point clouds. Due to the poor mine environment, the radar lens may be covered by pollutants such as dust and water mist, which will affect the effectiveness of radar data collection.

[0003] In the prior art, radar lens stain detection is performed by controlling different laser beams to be excited in sequence, and then processing and distinguishing according to the difference in voltage signals obtained by the receiving device. The disadvantage of this method is that it is only applied to non-coaxial laser radars and is not universal. Summary of the invention

[0004] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0005] To this end, the first purpose of the present application is to propose a radar lens stain detection method suitable for a mine environment, so as to achieve universal radar lens stain detection and solve the problem that radar lens stain detection in related technologies is not universal.

[0006] The second objective of the present application is to provide a radar lens stain detection device suitable for a mine environment.

[0007] The third objective of the present application is to provide another radar lens stain detection device suitable for a mine environment.

[0008] A fourth objective of the present application is to provide a computer-readable storage medium.

[0009] A fifth object of the present application is to provide a computer program product.

[0010] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a radar lens stain detection method suitable for a mine environment, comprising:

[0011] Acquiring prior information of the radar scanning window, wherein the prior information includes the curvature and roughness of each point of the radar scanning window in a clean state;

[0012] Acquire three-dimensional point cloud data and infrared images of the radar scanning window collected by a structured light camera and an infrared camera respectively;

[0013] Converting the infrared image into a binary image, and connecting the black area of ​​the binary image to obtain the area to be detected;

[0014] Based on the three-dimensional point cloud data of the radar scanning window, the three-dimensional point cloud data of the area to be detected is obtained; and the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected are calculated;

[0015] Based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the priori information, it is determined whether the radar lens is contaminated by stains.

[0016] In some implementations, the obtaining the prior information of the radar scanning window includes:

[0017] Collecting data of the radar scanning window in a clean state by using the structured light camera to obtain three-dimensional point cloud data of the clean radar scanning window;

[0018] Based on the three-dimensional point cloud data of the clean radar scanning window, the curvature and roughness of each point of the radar scanning window in a clean state are calculated.

[0019] In some implementations, the step of calculating the curvature and roughness of each point of the radar scanning window in a clean state based on the three-dimensional point cloud data of the clean radar scanning window comprises:

[0020] Based on the three-dimensional point cloud data of the cleaning radar scanning window, the curvature of each point is obtained by using a principal component analysis method on the neighborhood of each point;

[0021] Based on the three-dimensional point cloud data of the cleaning radar scanning window, the neighborhood of each point is fitted by least squares fitting to obtain a fitting error, and the fitting error is used as the roughness of the corresponding point to obtain the roughness of each point.

[0022] In some implementations, after calculating the curvature and roughness of each point of the radar scanning window in a clean state based on the three-dimensional point cloud data of the clean radar scanning window, the method further comprises:

[0023] Each point of the radar scanning window in a clean state and the curvature and roughness of each point are stored in a multidimensional search tree.

[0024] In some implementations, converting the infrared image into a binary image and connecting the black area of ​​the binary image to obtain the area to be detected includes:

[0025] converting the infrared image into a temperature map;

[0026] Mapping the temperature map into a grayscale map;

[0027] Processing the grayscale image by adaptive binarization to obtain a binary image;

[0028] The black areas in the binary image are connected by a region growing algorithm to obtain the area to be detected.

[0029] In some implementations, the determining whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the prior information includes:

[0030] Determine the a priori three points closest to each point of the area to be detected from the a priori information, and obtain the curvature and roughness of each of the a priori three points;

[0031] Calculating the average curvature and the average roughness of the area to be detected based on the curvature and the roughness of each point in the three-dimensional point cloud data of the area to be detected;

[0032] Calculate the average curvature and the average roughness of the prior area based on the curvature and the roughness of each of the three prior points;

[0033] Calculating a first difference between an average value of curvature of the area to be detected and an average value of curvature of the prior area, and a second difference between an average value of roughness of the area to be detected and an average value of roughness of the prior area;

[0034] Based on the first difference and the second difference, it is determined whether the radar lens is contaminated by dirt.

[0035] In some implementations, determining whether the radar lens is contaminated by dirt based on the first difference and the second difference includes:

[0036] Determining whether the first difference is greater than a curvature change threshold and whether the second difference is greater than a roughness change threshold;

[0037] When the first difference is greater than a curvature change threshold and the second difference is greater than a roughness change threshold, it is determined that the radar lens is contaminated by dirt.

[0038] To achieve the above-mentioned purpose, the second embodiment of the present application proposes a radar lens stain detection device suitable for a mine environment, comprising:

[0039] A data processing module, used for acquiring prior information of the radar scanning window, wherein the prior information includes the curvature and roughness of each point of the radar scanning window in a clean state;

[0040] A data acquisition module, used to acquire the three-dimensional point cloud data and infrared image of the radar scanning window collected by a structured light camera and an infrared camera respectively;

[0041] An image processing module, used for converting the infrared image into a binary image, and connecting the black area of ​​the binary image to obtain an area to be detected;

[0042] The data processing module is further used to obtain the three-dimensional point cloud data of the area to be detected based on the three-dimensional point cloud data of the radar scanning window; and calculate the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected;

[0043] The data processing module is further used to determine whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the priori information.

[0044] In some implementations, when acquiring the prior information of the radar scanning window, the data processing module is used to:

[0045] Collecting data of the radar scanning window in a clean state by using the structured light camera to obtain three-dimensional point cloud data of the clean radar scanning window;

[0046] Based on the three-dimensional point cloud data of the clean radar scanning window, the curvature and roughness of each point of the radar scanning window in a clean state are calculated.

[0047] In some implementations, when the data processing module calculates the curvature and roughness of each point of the radar scanning window in a clean state based on the three-dimensional point cloud data of the clean radar scanning window, it is used to:

[0048] Based on the three-dimensional point cloud data of the cleaning radar scanning window, the curvature of each point is obtained by using a principal component analysis method on the neighborhood of each point;

[0049] Based on the three-dimensional point cloud data of the cleaning radar scanning window, the neighborhood of each point is fitted by least squares fitting to obtain a fitting error, and the fitting error is used as the roughness of the corresponding point to obtain the roughness of each point.

[0050] In some implementations, the data processing module is further configured to:

[0051] Each point of the radar scanning window in a clean state and the curvature and roughness of each point are stored in a multidimensional search tree.

[0052] In some implementations, the image processing module is specifically configured to:

[0053] converting the infrared image into a temperature map;

[0054] Mapping the temperature map into a grayscale map;

[0055] Processing the grayscale image by adaptive binarization to obtain a binary image;

[0056] The black areas in the binary image are connected by a region growing algorithm to obtain the area to be detected.

[0057] In some implementations, when the data processing module determines whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the prior information, it is configured to:

[0058] Determine the a priori three points closest to each point of the area to be detected from the a priori information, and obtain the curvature and roughness of each of the a priori three points;

[0059] Calculating the average curvature and the average roughness of the area to be detected based on the curvature and the roughness of each point in the three-dimensional point cloud data of the area to be detected;

[0060] Calculate the average curvature and the average roughness of the prior area based on the curvature and the roughness of each of the three prior points;

[0061] Calculating a first difference between an average value of curvature of the area to be detected and an average value of curvature of the prior area, and a second difference between an average value of roughness of the area to be detected and an average value of roughness of the prior area;

[0062] Based on the first difference and the second difference, it is determined whether the radar lens is contaminated by dirt.

[0063] In some implementations, when determining whether the radar lens is contaminated by dirt based on the first difference and the second difference, the data processing module is configured to:

[0064] Determining whether the first difference is greater than a curvature change threshold and whether the second difference is greater than a roughness change threshold;

[0065] When the first difference is greater than a curvature change threshold and the second difference is greater than a roughness change threshold, it is determined that the radar lens is contaminated by dirt.

[0066] To achieve the above-mentioned purpose, the third aspect of the present application proposes a radar lens stain detection device suitable for a mine environment, comprising a detection camera, a rotatable bracket and a processor, wherein the detection camera comprises a fixedly connected infrared camera and a structured light camera, a lens protection cover is provided on the outer side of the lens of the detection camera, the detection camera is communicatively connected to the processor, and one end of the rotatable bracket is fixedly connected to the detection camera; during detection, the lens of the detection camera faces the radar to be detected, and after the detection is completed, the rotatable bracket moves the detection camera outside the field of view of the radar to be detected by rotating;

[0067] The structured light camera and the infrared camera are used to respectively acquire the three-dimensional point cloud data and the infrared image of the radar scanning window of the radar to be detected;

[0068] The processor is used to implement the method described in the first aspect based on the three-dimensional point cloud data and infrared images collected by the detection camera.

[0069] To achieve the above-mentioned purpose, the fourth aspect of the present application proposes a computer-readable storage medium, in which computer-readable storage medium is stored computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the method described in the first aspect.

[0070] To achieve the above-mentioned purpose, the fifth aspect of the present application proposes a computer program product, including a computer program, which implements the method described in the first aspect when executed by a processor.

[0071] The radar lens stain detection method, device, electronic device and storage medium provided by the present application are applicable to a mine environment. The curvature and roughness of the point cloud data of the radar lens area in a clean state and the curvature and roughness of the point cloud data of the radar lens area during the current detection are obtained in advance by a structured camera and an infrared camera, and the point cloud data features in a clean state are compared with the point cloud data features during the current detection. The radar lens is determined to be contaminated by stains based on the data changes. The radar lens stain detection method of the present invention can be used for stain detection of radar lenses in a mine environment, is applicable to any type of radar equipment, and is universal; once stains are detected, the stains can be removed immediately by manual or automatic cleaning devices.

[0072] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0073] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0074] Figure 1 A schematic flow chart of a radar lens stain detection method applicable to a mine environment provided in an embodiment of the present application;

[0075] Figure 2 A schematic diagram of the installation of a detection camera provided in an embodiment of the present application;

[0076] Figure 3 A block diagram of a radar lens stain detection device suitable for a mine environment provided in an embodiment of the present application;

[0077] Figure 4 A block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0078] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0079] The following describes the radar lens stain detection method, device and equipment applicable to a mine environment according to an embodiment of the present application with reference to the accompanying drawings.

[0080] Figure 1 A schematic flow chart of a radar lens stain detection method suitable for a mine environment provided in an embodiment of the present application.

[0081] It should be noted that the executor of the radar lens stain detection method suitable for mine environment of the embodiment of the present application is the radar lens stain detection device suitable for mine environment of the embodiment of the present application. The radar lens stain detection device suitable for mine environment can be configured in an electronic device so that the electronic device can perform the radar lens stain detection function suitable for mine environment.

[0082] like Figure 1 As shown, the radar lens stain detection method suitable for a mine environment comprises the following steps:

[0083] Step S101, obtaining prior information of the radar scanning window, the prior information including the curvature and roughness of each point of the radar scanning window in a clean state.

[0084] As an implementation method, a method for obtaining prior information of a radar scanning window includes:

[0085] The structured light camera is used to collect data of the radar scanning window in a clean state to obtain three-dimensional point cloud data of the clean radar scanning window; based on the three-dimensional point cloud data of the clean radar scanning window, the curvature and roughness of each point of the radar scanning window in the clean state are calculated.

[0086] In some implementations, a method for calculating the curvature and roughness of each point of a radar scanning window in a clean state based on three-dimensional point cloud data of a clean radar scanning window includes: based on the three-dimensional point cloud data of the clean radar scanning window, using a principal component analysis method on the neighborhood of each point to obtain the curvature of each point; based on the three-dimensional point cloud data of the clean radar scanning window, performing least squares fitting on the neighborhood of each point to obtain a fitting error, and using the fitting error as the roughness of the corresponding point to obtain the roughness of each point.

[0087] In some implementations, after calculating the curvature and roughness of each point of the radar scanning window in a clean state based on the three-dimensional point cloud data of the clean radar scanning window, the method includes: storing each point of the radar scanning window in the clean state and the curvature and roughness of each point in a multidimensional search tree.

[0088] As an implementation method, the camera is used to obtain prior information, such as Figure 2 As shown, the detection camera 1 includes a fixedly connected infrared camera and a structured light camera, and a lens protection cover 4 is provided on the outside of the lens of the detection camera 1; the detection camera 1 is installed on the radar 2 through a rotatable bracket 3. When the radar 2 works normally, the detection camera 1 is placed outside the field of view of the radar 2 by rotating the rotatable bracket 3 and covered with a mechanical lens protection cover 4; when obtaining prior information, ensure that the radar lens is in a clean state, rotate the rotatable bracket 3 so that the lens of the detection camera 1 faces the radar 2 to collect data. After data collection, the detection camera 1 is retracted and then covered with the lens protection cover 4.

[0089] Exemplarily, when the radar lens is clean, a detection camera is used to collect data from the radar, extract the three-dimensional information of the radar scanning window, and perform statistics on the curvature and roughness of each point in the radar scanning window; wherein the curvature is obtained by using the principal component analysis method for the neighborhood of the point; the roughness is calculated by performing the least squares fitting on the neighborhood of the point, calculating the fitting error, and recording the fitting error as the roughness. In order to facilitate the search of each point and the curvature and roughness of each point in the subsequent steps, each point in the radar scanning window and its curvature and roughness are stored in a multidimensional search tree, such as in KDTree, recorded as a priori KD tree.

[0090] This step is mainly to obtain prior information and prepare for data comparison in subsequent steps.

[0091] Step S102, obtaining three-dimensional point cloud data and infrared images of the radar scanning window collected by the structured light camera and the infrared camera respectively.

[0092] For example, at fixed intervals, such as 2 hours, a rotatable bracket can be used to adjust the detection camera to the front of the radar and the lens protection cover of the detection camera can be opened to collect the three-dimensional point cloud data and infrared image of the radar scanning window through the structured light camera and the infrared camera respectively.

[0093] Step S103, converting the infrared image into a binary image, and connecting the black area of ​​the binary image to obtain the area to be detected.

[0094] As an implementation method, an infrared image is converted into a binary image, and the black area of ​​the binary image is connected to obtain a method for obtaining a region to be detected; the method includes: converting the infrared image into a temperature map; mapping the temperature map into a grayscale map; processing the grayscale map by adaptive binarization to obtain a binary image; and connecting the black area in the binary image by a region growing algorithm to obtain a region to be detected.

[0095] It can be understood that since the radar scanning window is fixedly connected to the radar and the temperature is relatively high during operation, the region growing algorithm is used to connect the binarized black area as the area to be detected.

[0096] Step S104, based on the three-dimensional point cloud data of the radar scanning window, obtain the three-dimensional point cloud data of the area to be detected; and calculate the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected.

[0097] That is to say, the three-dimensional point cloud data collected by the structured camera is obtained, and the data falling on the part of the area to be detected is obtained, and the curvature and roughness of each point of the part of the data are calculated. The calculation method of the curvature and roughness of each point is the same as step S101, which will not be repeated.

[0098] Step S105 , determining whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and prior information.

[0099] In some implementations, a method for determining whether a radar lens is contaminated by dirt based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and prior information includes: determining the prior three points closest to each point in the area to be detected from the prior information, and obtaining the curvature and roughness of each prior three points; calculating the average curvature and the average roughness of the area to be detected based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected; calculating the average curvature and the average roughness of the prior area based on the curvature and roughness of each prior three points; calculating a first difference between the average curvature of the area to be detected and the average curvature of the prior area, and a second difference between the average roughness of the area to be detected and the average roughness of the prior area; and determining whether the radar lens is contaminated by dirt based on the first difference and the second difference.

[0100] In some implementations, a method for determining whether a radar lens is contaminated by dirt based on a first difference and a second difference includes: determining whether the first difference is greater than a curvature change threshold and whether the second difference is greater than a roughness change threshold; and determining that the radar lens is contaminated by dirt when the first difference is greater than the curvature change threshold and the second difference is greater than the roughness change threshold.

[0101] Exemplarily, the nearest neighbor point of each point in the area to be detected in the prior KD tree is obtained and recorded as the prior three points of the corresponding prior area, and the curvature and roughness of each prior three points are obtained. The average curvature and roughness of the area to be detected are calculated, and the average curvature and roughness of the corresponding prior area are calculated at the same time. If the average curvature and roughness of the area to be detected under normal working conditions are significantly greater than the average curvature and roughness of the corresponding prior area, it is considered that the radar scanning surface has been contaminated by stains, otherwise it is considered that it is not contaminated temporarily.

[0102] The radar lens stain detection method applicable to a mine environment in the embodiment of the present application uses a structured camera and an infrared camera to obtain in advance the curvature and roughness of the point cloud data of the radar lens area in a clean state and the curvature and roughness of the point cloud data of the radar lens area during the current detection, and compares the point cloud data features in a clean state with the point cloud data features during the current detection, and determines whether the radar lens is contaminated by stains based on the data changes. The radar lens stain detection method of the present invention can be used for stain detection of radar lenses in a mine environment, is applicable to any type of radar equipment, and has universal applicability; once stains are detected, the stains can be removed immediately by manual or automatic cleaning devices.

[0103] In order to clearly explain the above embodiment, a specific example is now used for illustration. The embodiment of the present application proposes a radar lens stain detection device suitable for a mine environment, including a detection camera, a rotatable bracket and a processor. The detection camera includes a fixedly connected infrared camera and a structured light camera. A lens protection cover is provided on the outside of the lens of the detection camera. The detection camera is communicatively connected to the processor, and one end of the rotatable bracket is fixedly connected to the detection camera. During detection, the lens of the detection camera faces the radar to be detected. After the detection is completed, the rotatable bracket rotates to move the detection camera outside the field of view of the radar to be detected.

[0104] The structured light camera and the infrared camera are used to obtain the three-dimensional point cloud data and the infrared image of the radar scanning window of the radar to be detected respectively;

[0105] The processor is used to implement the radar lens stain detection method applicable to a mine environment of the above embodiment based on the three-dimensional point cloud data and infrared images collected by the detection camera.

[0106] In order to implement the above embodiment, the present application also proposes a radar lens stain detection device suitable for a mine environment. Figure 3 This is a block diagram of a radar lens stain detection device suitable for a mine environment provided in an embodiment of the present application. Figure 3 As shown, the radar lens stain detection device suitable for a mine environment may include: a data processing module 301 , a data acquisition module 302 and an image processing module 303 .

[0107] The data processing module 301 is used to obtain the prior information of the radar scanning window, and the prior information includes the curvature and roughness of each point of the radar scanning window in a clean state;

[0108] A data acquisition module 302 is used to acquire three-dimensional point cloud data and infrared images of the radar scanning window collected by a structured light camera and an infrared camera respectively;

[0109] The image processing module 303 is used to convert the infrared image into a binary image and connect the black area of ​​the binary image to obtain the area to be detected;

[0110] The data processing module 301 is also used to obtain the three-dimensional point cloud data of the area to be detected based on the three-dimensional point cloud data of the radar scanning window; and calculate the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected;

[0111] The data processing module 301 is also used to determine whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the prior information.

[0112] Furthermore, in a possible implementation of the embodiment of the present application, when acquiring the prior information of the radar scanning window, the data processing module 301 is used to:

[0113] The structured light camera is used to collect data of the radar scanning window in a clean state, and the three-dimensional point cloud data of the clean radar scanning window is obtained;

[0114] Based on the three-dimensional point cloud data of the clean radar scanning window, the curvature and roughness of each point of the radar scanning window in the clean state are calculated.

[0115] Further, in a possible implementation of the embodiment of the present application, when the data processing module 301 calculates the curvature and roughness of each point of the radar scanning window in the clean state based on the three-dimensional point cloud data of the clean radar scanning window, it is used to:

[0116] Based on the 3D point cloud data of the clean radar scanning window, the curvature of each point is obtained by using the principal component analysis method on the neighborhood of each point;

[0117] Based on the three-dimensional point cloud data of the clean radar scanning window, the least squares fitting is performed on the neighborhood of each point to obtain the fitting error, and the fitting error is used as the roughness of the corresponding point to obtain the roughness of each point.

[0118] Furthermore, in a possible implementation of the embodiment of the present application, the data processing module 301 is further configured to:

[0119] Each point of the radar scanning window in a clean state and the curvature and roughness of each point are stored in a multidimensional search tree.

[0120] Furthermore, in a possible implementation of the embodiment of the present application, the image processing module 303 is specifically configured to:

[0121] Convert infrared images into temperature maps;

[0122] Map the temperature map to a grayscale image;

[0123] The grayscale image is processed by adaptive binarization to obtain a binary image;

[0124] The black areas in the binary image are connected by the region growing algorithm to obtain the area to be detected.

[0125] Further, in a possible implementation of the embodiment of the present application, when the data processing module 301 determines whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the prior information; it is used to:

[0126] Determine the prior three points closest to each point in the area to be detected from the prior information, and obtain the curvature and roughness of each prior three points;

[0127] Based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected, the average curvature and the average roughness of the area to be detected are calculated;

[0128] Based on the curvature and roughness of each prior three points, the average curvature and the average roughness of the prior area are calculated;

[0129] Calculate a first difference between the average curvature of the region to be detected and the average curvature of the prior region, and a second difference between the average roughness of the region to be detected and the average roughness of the prior region;

[0130] Based on the first difference and the second difference, it is determined whether the radar lens is contaminated by dirt.

[0131] Further, in a possible implementation of the embodiment of the present application, when the data processing module 301 determines whether the radar lens is contaminated by stains based on the first difference and the second difference, it is configured to:

[0132] Determining whether the first difference is greater than a curvature change threshold and whether the second difference is greater than a roughness change threshold;

[0133] When the first difference is greater than the curvature change threshold and the second difference is greater than the roughness change threshold, it is determined that the radar lens is contaminated by dirt.

[0134] It should be noted that the above explanation of the embodiment of the radar lens stain detection method applicable to a mine environment is also applicable to the radar lens stain detection device applicable to a mine environment of this embodiment, and will not be repeated here.

[0135] In order to implement the above embodiment, the present application also proposes an electronic device. Figure 4 , Figure 4 is a block diagram of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device 400 includes: a processor 401, and a memory 402 communicatively connected to the processor 401; the memory 402 stores computer-executable instructions; the processor 401 executes the computer-executable instructions stored in the memory to implement the method provided by the aforementioned embodiment.

[0136] In order to implement the above embodiments, the present application also proposes a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the methods provided by the above embodiments.

[0137] In order to implement the above embodiments, the present application also proposes a computer program product, including a computer program, which implements the methods provided by the above embodiments when executed by a processor.

[0138] The collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in this application are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0139] It should be noted that personal information from users should be collected for legitimate and reasonable purposes and should not be shared or sold outside of these legitimate uses. In addition, such collection / sharing should be carried out after receiving the user's informed consent, including but not limited to notifying the user to read the user agreement / user notice and sign the agreement / authorization including authorization of relevant user information before the user uses the function. In addition, any necessary steps should be taken to protect and safeguard access to such personal information data and ensure that others who have access to personal information data comply with its privacy policy and procedures.

[0140] The present application is expected to provide an implementation scheme for users to selectively block the use or access of personal information data. That is, the present disclosure is expected to provide hardware and / or software to prevent or block access to such personal information data. Once the personal information data is no longer needed, the risk can be minimized by limiting data collection and deleting the data. In addition, when applicable, such personal information is de-identified to protect the privacy of the user.

[0141] In the description of the aforementioned embodiments, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0142] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0143] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0144] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0145] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0146] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0147] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0148] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A radar lens stain detection method suitable for a mine environment, characterized in that: The following steps are involved: Acquiring prior information of the radar scanning window, wherein the prior information includes the curvature and roughness of each point of the radar scanning window in a clean state; Acquire three-dimensional point cloud data and infrared images of the radar scanning window collected by a structured light camera and an infrared camera respectively; Converting the infrared image into a binary image, and connecting the black area of ​​the binary image to obtain the area to be detected; Based on the three-dimensional point cloud data of the radar scanning window, the three-dimensional point cloud data of the area to be detected is obtained; and the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected are calculated; Based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the priori information, it is determined whether the radar lens is contaminated by stains.

2. The method according to claim 1, characterized in that The obtaining of the prior information of the radar scanning window includes: Collecting data of the radar scanning window in a clean state by using the structured light camera to obtain three-dimensional point cloud data of the clean radar scanning window; Based on the three-dimensional point cloud data of the clean radar scanning window, the curvature and roughness of each point of the radar scanning window in a clean state are calculated.

3. The method according to claim 2, characterized in that The method of calculating the curvature and roughness of each point of the radar scanning window in a clean state based on the three-dimensional point cloud data of the clean radar scanning window comprises: Based on the three-dimensional point cloud data of the cleaning radar scanning window, the curvature of each point is obtained by using a principal component analysis method on the neighborhood of each point; Based on the three-dimensional point cloud data of the cleaning radar scanning window, the neighborhood of each point is fitted by least squares fitting to obtain a fitting error, and the fitting error is used as the roughness of the corresponding point to obtain the roughness of each point.

4. The method according to claim 2, characterized in that: After calculating the curvature and roughness of each point of the radar scanning window in a clean state based on the three-dimensional point cloud data of the clean radar scanning window, the method includes: Each point of the radar scanning window in a clean state and the curvature and roughness of each point are stored in a multidimensional search tree.

5. The method according to claim 1, characterized in that The step of converting the infrared image into a binary image and connecting the black area of ​​the binary image to obtain the area to be detected comprises: converting the infrared image into a temperature map; Mapping the temperature map into a grayscale map; Processing the grayscale image by adaptive binarization to obtain a binary image; The black areas in the binary image are connected by a region growing algorithm to obtain the area to be detected.

6. The method according to claim 1, characterized in that The step of determining whether the radar lens is contaminated by dirt based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the prior information comprises: Determine the a priori three points closest to each point of the area to be detected from the a priori information, and obtain the curvature and roughness of each of the a priori three points; Calculating the average curvature and the average roughness of the area to be detected based on the curvature and the roughness of each point in the three-dimensional point cloud data of the area to be detected; Calculate the average curvature and the average roughness of the prior area based on the curvature and the roughness of each of the three prior points; Calculating a first difference between an average value of curvature of the area to be detected and an average value of curvature of the prior area, and a second difference between an average value of roughness of the area to be detected and an average value of roughness of the prior area; Based on the first difference and the second difference, it is determined whether the radar lens is contaminated by dirt.

7. The method according to claim 6, characterized in that The determining whether the radar lens is contaminated by dirt based on the first difference and the second difference includes: Determining whether the first difference is greater than a curvature change threshold and whether the second difference is greater than a roughness change threshold; When the first difference is greater than a curvature change threshold and the second difference is greater than a roughness change threshold, it is determined that the radar lens is contaminated by dirt.

8. A radar lens stain detection device suitable for a mine environment, characterized in that: include: A data processing module, used for acquiring prior information of the radar scanning window, wherein the prior information includes the curvature and roughness of each point of the radar scanning window in a clean state; A data acquisition module, used to acquire the three-dimensional point cloud data and infrared image of the radar scanning window collected by a structured light camera and an infrared camera respectively; An image processing module, used for converting the infrared image into a binary image, and connecting the black area of ​​the binary image to obtain an area to be detected; The data processing module is further used to obtain the three-dimensional point cloud data of the area to be detected based on the three-dimensional point cloud data of the radar scanning window; and calculate the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected; The data processing module is further used to determine whether the radar lens is contaminated by stains based on the curvature and roughness of each point in the three-dimensional point cloud data of the area to be detected and the priori information.

9. A radar lens stain detection device suitable for a mine environment, characterized in that: It comprises a detection camera, a rotatable bracket and a processor, wherein the detection camera comprises a fixedly connected infrared camera and a structured light camera, a lens protection cover is provided on the outer side of the lens of the detection camera, the detection camera is communicatively connected with the processor, and one end of the rotatable bracket is fixedly connected with the detection camera; during detection, the lens of the detection camera faces the radar to be detected, and after the detection is completed, the rotatable bracket moves the detection camera out of the field of view of the radar to be detected by rotating; The structured light camera and the infrared camera are used to respectively acquire the three-dimensional point cloud data and the infrared image of the radar scanning window of the radar to be detected; The processor is used to implement the method according to any one of claims 1 to 7 based on the three-dimensional point cloud data and infrared images collected by the detection camera.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 7 when executed by a processor.

Images