A method for monitoring the transport status of a mining transport vehicle
By adaptively adjusting the search window size of the NLM algorithm, based on the noise coefficient of each pixel and the grayscale complexity of the surrounding area, the problem of smoothing pothole features in existing technologies is solved, achieving more accurate road condition image denoising and pothole recognition, and reducing the risk of transport vehicle accidents.
Patent Information
- Application Number
- CN202411281790.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-13
- Publication Date
- 2024-11-19
- Estimated Expiration
- 2044-09-13
AI Technical Summary
In existing technologies, the NLM algorithm, which uses a search window of uniform size, cannot effectively preserve the features of potholes in images of mining transport vehicles, resulting in potholes not being detected in the denoised images and increasing the risk of accidents.
By acquiring the noise coefficient of each pixel and the grayscale complexity of the surrounding area, the search window size is adaptively adjusted, and the NLM algorithm is used for denoising to ensure that the features of the pitted area are preserved.
It improves the accuracy of identifying potholes, reduces the risk of accidents involving transport vehicles, and enables more precise road condition monitoring.
Smart Images

Figure CN118799838B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of image processing technology, and more specifically, to a method for monitoring the transport status of a mining transport vehicle. Background Art
[0002] Competition in the global mining market is becoming increasingly fierce. Mining companies need to find advantages in safety, efficiency and cost control. Therefore, an effective monitoring solution for the transportation status of mining transport vehicles can help mining companies stand out in this competition and improve their market competitiveness. With the rapid development of image processing technology, artificial intelligence and sensor technology, the monitoring technology of mining transport vehicles is constantly improving. Image processing technology can monitor the status of transport vehicles in real time, identify potential dangerous situations (such as potholes, abnormal vehicle conditions, etc.), and reduce the risk of accidents. This solution detects potholes in the road by collecting road condition images of the road in front of the transport vehicle in real time. However, in the actual image data transmission process, data loss or damage may occur, resulting in a small number of noise pixels in the collected road condition images, which will affect the subsequent identification of potholes in the road. Therefore, the road condition images must be denoised first.
[0003] The patent application document with the publication number CN115049614A currently discloses an RFID tag image denoising method based on an improved NLM algorithm. The optimal NLM kernel function coefficient is obtained by using the golden section method; the optimal NLM kernel function coefficient is substituted into the NLM kernel function; the RFID tag noisy image is input and Gaussian convolution is performed on the RFID tag noisy image; finally, the NLM algorithm is used to process the RFID tag noisy image after Gaussian convolution to complete the RFID tag image denoising process.
[0004] When using the NLM algorithm in the above method to denoise road images, it is necessary to use a search window of the same size for all pixels in the road image. However, since the pothole areas in the road image usually have high details and changes, using a search window of the same size may not be able to effectively retain these details, resulting in the features of the pothole area being smoothed out, resulting in the pothole area in the denoised road image may not be detected, increasing the risk of accidents for transport vehicles. Summary of the invention
[0005] In order to solve the problem that the NLM algorithm may not be able to effectively retain the features of the pothole area using a search window of the same size, resulting in the inability to detect the pothole area in the denoised road condition image, the present invention proposes a method for monitoring the transportation status of a mining transport vehicle, the method comprising the following steps:
[0006] Collect road condition images; obtain the noise coefficient of each pixel in the road condition image:
[0007] ; Represents the noise coefficient of the i-th pixel; Represents the gray value of the i-th pixel; Represents the grayscale mean of the eight neighboring pixels of the i-th pixel; Represents the preset hyperparameters; represents the number of pixels in the road condition image that have the same grayscale value as the i-th pixel; R represents the number of pixels in the road condition image; || represents the absolute value symbol; exp() represents an exponential function with a natural constant as the base;
[0008] Get the surrounding area of each pixel in the road condition image; get the size of the search window for each pixel in the road condition image , Represents the size of the search window for the i-th pixel; Represents the grayscale information entropy of all pixels in the surrounding area of the i-th pixel; norm() represents the normalization function; Represents the Euclidean distance between the i-th pixel and the last pixel in its column;
[0009] Based on the size of the search window, the side length of the initial search window of each pixel in the road condition image is obtained; based on the side length of the initial search window, the side length of the final search window of each pixel in the road condition image is obtained; the road condition image is denoised according to the side length of the final search window to obtain the denoised road condition image; and the pothole area is identified according to the denoised road condition image.
[0010] The innovation of the present invention lies in that the size of the search window for each pixel in the road condition image is obtained according to the noise coefficient of each pixel, the grayscale complexity of its surrounding area and its position, and the size of the search window is adapted to the search window size. Different denoising intensities can be applied in different areas. Using a small search window in the detail area can effectively reduce the noise and retain more original information. In the area with strong noise, using a search window of appropriate size can better remove the noise and reduce the impact of noise on the road condition image. Finally, the pothole area in the denoised road condition image can be detected, reducing the risk of accidents for transport vehicles.
[0011] Preferably, the step of obtaining the side length of the initial search window for each pixel in the road condition image includes:
[0012] ;
[0013] In the formula, Represents the size of the search window for the i-th pixel in the road condition image; Represents the side length of the preset search window; Represents the side length of the initial search window for the i-th pixel in the road condition image; Represents the floor symbol.
[0014] The adaptive search window size can retain the detailed information of the pothole area and avoid smoothing the pothole area later.
[0015] Preferably, the step of obtaining the side length of the final search window of each pixel in the road condition image based on the side length of the initial search window includes:
[0016] If the side length of the initial search window of the i-th pixel in the road condition image is an odd number, the side length of the initial search window of the i-th pixel in the road condition image is used as the side length of the final search window of the i-th pixel in the road condition image; if the side length of the initial search window of the i-th pixel in the road condition image is an even number, the side length of the final search window of the i-th pixel in the road condition image is the side length of the initial search window of the i-th pixel in the road condition image + 1.
[0017] Preferably, the step of obtaining the denoised road condition image includes:
[0018] According to the side length of the final search window of each pixel in the road condition image, the search window of each pixel in the road condition image is obtained, and then the road condition image is denoised in combination with the NLM algorithm to obtain a denoised road condition image.
[0019] The filtering effect is improved and the information of pothole areas is preserved.
[0020] Preferably, identifying pothole areas according to the denoised road condition image includes:
[0021] A low threshold a and a high threshold b are preset, and the canny edge detection algorithm is used to detect the pothole area in the denoised road image.
[0022] Pothole areas are identified more accurately.
[0023] Preferably, the obtaining of the surrounding area of each pixel in the road condition image includes:
[0024] The default surrounding area size is , with each pixel in the road condition image as the center, construct a as the surrounding area of each pixel in the road condition image.
[0025] Preferably, the collecting of road condition images includes:
[0026] A high-definition camera is installed at the front of the mining transport vehicle. When the mining transport vehicle is driving, the road condition is photographed to obtain an RGB image of the road condition, and the RGB image of the road condition is gray-processed to obtain a road condition image.
[0027] Pothole areas are identified more accurately.
[0028] The present invention has the following beneficial effects: the present invention first obtains the noise coefficient of each pixel point according to the difference between the grayscale value of each pixel point and the grayscale mean value of its eight neighboring pixels, and then obtains the size of the search window of each pixel point in the road condition image according to the noise coefficient of each pixel point, the grayscale complexity of its surrounding area and its position. The size of the search window is adapted to the size of the search window, and different denoising strengths can be applied in different areas. Using a small search window in a detail area can effectively reduce noise and retain more original information. In an area with strong noise, using a search window of an appropriate size can better remove noise and reduce the impact of noise on the road condition image. Finally, the pothole area in the denoised road condition image can be detected, reducing the risk of accidents for transport vehicles. BRIEF DESCRIPTION OF THE DRAWINGS
[0029] By reading the following detailed description with reference to the accompanying drawings, the above and other objects, features and advantages of the exemplary embodiments of the present invention will become readily understood. In the accompanying drawings, several embodiments of the present invention are shown in an exemplary and non-limiting manner, and the same or corresponding reference numerals represent the same or corresponding parts, wherein:
[0030] Figure 1 The present invention is a flowchart of the steps of a method for monitoring the transport status of a mining transport vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION
[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.
[0032] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.
[0033] See also Figure 1 , which shows a flow chart of a method for monitoring the transport status of a mining transport vehicle provided by an embodiment of the present invention, the method comprising the following steps:
[0034] S001. Collect road condition images.
[0035] In an embodiment of the present invention, a high-definition camera is installed at the front of the mining transport vehicle. When the mining transport vehicle is driving, the road condition is photographed to obtain an RGB image of the road condition. In order to facilitate subsequent processing, the RGB image of the road condition is grayed to obtain a road condition image.
[0036] S002. Obtain a noise coefficient for each pixel in the road condition image, and obtain a size of a search window for each pixel in the road condition image according to the noise coefficient for each pixel in the road condition image.
[0037] It should be noted that when the NLM algorithm denoises an image, a search window of the same size is used for all pixels in the road condition image. Since the pothole area in the road condition image usually has high details and changes, the use of a search window of the same size may not be able to effectively retain these details, resulting in the features of the pothole area being smoothed out, and the pothole area in the denoised road condition image may not be detected, increasing the risk of accidents for transport vehicles. Therefore, when the present invention uses the NLM algorithm to denoise the road condition image, it is necessary to adaptively obtain the search window of each pixel in the road condition image for denoising, so as to avoid smoothing the pothole area in the road condition image, which is helpful for the subsequent identification of the pothole area.
[0038] It should be further explained that the grayscale characteristics of each pixel in the road condition image are analyzed to obtain the noise coefficient of each pixel. When analyzing this indicator, due to the obvious difference in the grayscale value of the noise pixel and its surrounding pixels, it may appear as a prominent bright or dim area; and the noise pixel may exist in isolation; therefore, the greater the difference between the grayscale value of any pixel in the road condition image and the grayscale mean of the pixels in its eight-domain area, the greater the possibility that the pixel is a noise pixel, and the fewer the number of pixels in the road condition image that have the same grayscale value as the pixel, the more isolated the pixel is, and the greater the possibility that the pixel is a noise pixel.
[0039] In an embodiment of the present invention, the noise coefficient of each pixel in the road condition image is obtained:
[0040] ;
[0041] In the formula, Represents the noise coefficient of the i-th pixel in the road condition image; Represents the gray value of the i-th pixel in the road condition image; Represents the grayscale mean of the eight neighborhood pixels of the i-th pixel in the road condition image; represents a preset hyperparameter. In the embodiment of the present invention, the preset hyperparameter , which exists to prevent The value of is 0; R represents the number of pixels in the road condition image; Represents the number of pixels in the road condition image that have the same grayscale value as the i-th pixel; The larger the value of is, the greater the difference between the gray value of the i-th pixel and the gray value mean of the pixels in its eight neighborhoods is, indicating that the i-th pixel is more likely to be noise and the greater the noise coefficient of the i-th pixel is. The smaller the value is, the more isolated the i-th pixel is in the road condition image, which means the noise coefficient of the i-th pixel is larger.
[0042] It should be noted that it is known that the noise coefficient of each pixel has been obtained. This indicator is obtained by analyzing the grayscale value characteristics of each pixel. The larger the noise coefficient of any pixel, the greater the possibility that the pixel is a noise pixel, which means that the pixel needs a larger search window for greater denoising. However, this indicator can only reflect the degree of demand for the search window size of a pixel during filtering from the aspect of grayscale value characteristics. If the size of the search window is determined only by the noise coefficient of a pixel, the following problems may occur: the grayscale value characteristics of some edge pixels in the pothole area are similar to the grayscale value characteristics of noise pixels. Determining the size of the search window by the noise coefficient of the pixel may over-smooth the edge pixels of some pothole areas, making it difficult to detect the complete pothole area later.
[0043] It should be further explained that, compared with the noise pixels, the pixels in the pothole area have richer texture changes in the surrounding areas. Therefore, the present invention analyzes the grayscale change characteristics of each pixel and the pixels in the surrounding area, and further optimizes the noise coefficient of each pixel. If the grayscale value in the area surrounded by any pixel and its surrounding pixels is more chaotic, it can be explained that the pixel and the area surrounded by the surrounding pixels contain richer detail information, and the pixel has a smaller demand for the search window size. In addition, the pixels in different areas of the road condition image collected under the corresponding scene often have different importance. The pixel points closer to the lower boundary of the road condition image will be located in the area where the transport vehicle is about to travel, and the pixel points closer to the upper boundary of the road condition image will be located in the area where the transport vehicle may need some time to travel. Therefore, the pixel points closer to the lower boundary of the road condition image will have a smaller demand for the search window size, which can ensure that the road surface where the transport vehicle is about to travel can be more clearly identified whether there are potholes or other potential dangerous situations.
[0044] In the embodiment of the present invention, the preset surrounding area size is , with each pixel in the road condition image as the center, construct a As the surrounding area of each pixel in the road condition image; in the embodiment of the present invention, the size of the surrounding area is preset to be In other embodiments, the implementer may preset The value of .
[0045] Get the size of the search window for each pixel in the traffic image:
[0046] ;
[0047] In the formula, Represents the size of the search window for the i-th pixel in the road condition image; Represents the noise coefficient of the i-th pixel in the road condition image; Represents the grayscale information entropy of all pixels in the surrounding area of the i-th pixel in the road condition image; norm() represents the normalization function; Represents the Euclidean distance between the i-th pixel in the road condition image and the last pixel in its column; The larger the value of , the more likely the pixel is a noise pixel, and the larger the search window is needed to achieve large-scale filtering. The larger the value of is, the denser the texture of the area around the pixel is and the richer the detail information it contains. The more likely it is that the pixel belongs to a pothole area, so the pixel needs a smaller search window to achieve a small filtering. The larger the value, the greater the possibility that the area where the pixel point is located in the road condition image is the road where the transport vehicle is about to travel. Therefore, a smaller search window is required to perform a smaller filtering to ensure that the road where the transport vehicle is about to travel is clearer to facilitate identification of potholes or other potential dangerous conditions.
[0048] S003. According to the size of the search window of each pixel in the road condition image, the side length of the initial search window of each pixel in the road condition image is obtained, and based on the side length of the initial search window, the side length of the final search window of each pixel in the road condition image is obtained.
[0049] It should be noted that, through the above analysis, the size of the search window for each pixel in the road condition image is obtained. When the size of the search window of any pixel in the road condition image is larger, it means that the pixel needs a larger search window to achieve a larger degree of denoising. When the size of the search window of any pixel in the road condition image is smaller, it means that the pixel needs a smaller search window to achieve a smaller degree of denoising. That is, according to the size of the search window of each pixel in the road condition image, the side length of the preset search window is adjusted to obtain the side length of the initial search window for each pixel in the road condition image.
[0050] In this embodiment of the present invention, the side length of the initial search window of each pixel in the road condition image is obtained:
[0051] ;
[0052] In the formula, Represents the size of the search window for the i-th pixel in the road condition image; Represents the side length of the preset search window; Represents the side length of the initial search window for the i-th pixel in the road condition image; represents a round-down symbol; in the embodiment of the present invention, the side length of the preset search window In other embodiments, implementers may preset the side length of the search window according to specific implementation conditions.
[0053] If the side length of the initial search window of the i-th pixel in the road condition image is an odd number, the side length of the initial search window of the i-th pixel in the road condition image is used as the side length of the final search window of the i-th pixel in the road condition image; if the side length of the initial search window of the i-th pixel in the road condition image is an even number, the side length of the final search window of the i-th pixel in the road condition image is the side length of the initial search window of the i-th pixel in the road condition image + 1.
[0054] S004. According to the side length of the final search window of each pixel in the road condition image, the road condition image is denoised in combination with the NLM algorithm.
[0055] In an embodiment of the present invention, the search window of each pixel in the road condition image is obtained according to the side length of the final search window of each pixel in the road condition image, and then the road condition image is denoised in combination with the NLM algorithm to obtain a denoised road condition image.
[0056] A low threshold a and a high threshold b are preset, and the canny edge detection algorithm is used to detect potholes in the denoised road image. If there are potholes, the driver of the mining transport vehicle is reminded to slow down. In an embodiment of the present invention, the low threshold a=70 and the high threshold b=150 are preset. In other embodiments, the implementer can preset the low threshold and the high threshold according to the specific implementation situation.
[0057] It should be noted that the use of the NLM algorithm to perform denoising on road condition images is an existing technology, and in the embodiments of the present invention, it will not be described in detail; Canny edge detection is an existing well-known technology, and its specific steps will not be described in detail here.
[0058] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A method for monitoring the transport status of a mining transport vehicle, It is characterized in that include: Collect road condition images; Get the noise coefficient of each pixel in the road condition image: ; Represents the noise coefficient of the i-th pixel; Represents the gray value of the i-th pixel; Represents the grayscale mean of the eight neighboring pixels of the i-th pixel; represents the preset hyperparameters; Represents the number of pixels in the road condition image that have the same grayscale value as the i-th pixel; R represents the number of pixels in the road condition image; || represents the absolute value symbol; exp() represents an exponential function with a natural constant as the base; Get the surrounding area of each pixel in the road condition image; get the size of the search window for each pixel in the road condition image , Represents the size of the search window for the i-th pixel; Represents the grayscale information entropy of all pixels in the surrounding area of the i-th pixel; norm() represents the normalization function; Represents the Euclidean distance between the i-th pixel and the last pixel in its column; Get the side length of the initial search window for each pixel in the road condition image , Represents the side length of the initial search window for the i-th pixel; Represents the side length of the preset search window; represents the floor symbol; if is an odd number, and it is used as the side length of the final search window of the i-th pixel; if For an even number, +1 as the side length of the final search window of the i-th pixel point; denoising the road condition image according to the side length of the final search window to obtain the denoised road condition image; and identifying the pothole area according to the denoised road condition image.
2. A method for monitoring the transport status of a mining transport vehicle according to claim 1, It is characterized in that The step of obtaining a denoised road condition image includes: According to the side length of the final search window of each pixel in the road condition image, the search window of each pixel in the road condition image is obtained, and then the road condition image is denoised in combination with the NLM algorithm to obtain a denoised road condition image.
3. A method for monitoring the transport status of a mining transport vehicle according to claim 1, It is characterized in that The step of identifying pothole areas according to the denoised road condition image includes: A low threshold a and a high threshold b are preset, and the canny edge detection algorithm is used to detect the pothole area in the denoised road image.
4. A method for monitoring the transport status of a mining transport vehicle according to claim 1, It is characterized in that The step of obtaining the surrounding area of each pixel in the road condition image includes: The default surrounding area size is , with each pixel in the road condition image as the center, construct a as the surrounding area of each pixel in the road condition image.
5. A method for monitoring the transport status of a mining transport vehicle according to claim 1, It is characterized in that The collecting of road condition images comprises: A high-definition camera is installed at the front of the mining transport vehicle. When the mining transport vehicle is driving, the road condition is photographed to obtain an RGB image of the road condition, and the RGB image of the road condition is gray-processed to obtain a road condition image.
