Demisting method and demisting device

Abstract

The invention provides a demisting method and a demisting device. The method comprises the following steps: acquiring a historical image and corresponding historical dense fog information of an operation area of the strip mine area; a historical image and corresponding historical dense fog information are adopted to train an improved YOLOv5 model to obtain an identification model, and the improved YOLOv5 model is obtained by replacing a neck structure in the YOLOv5 model with a PANet structure; acquiring a current image of the working area of the strip mine area, and recognizing the current image by adopting the recognition model to obtain current dense fog information; under the condition that the current dense fog information represents that the dense fog area exists, the area of the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, whether defogging is carried out or not is determined at least according to the size of the area, and the problem that in the prior art, dense fog area recognition accuracy is low is solved.

Description

Technical Field

[0001] The present application relates to the technical field of open-pit mine operation safety, and in particular to a demisting method and a demisting device. Background Art

[0002] In winter, at the shoveling operation site of the open-pit mine, the internal temperature of the coal rock layer is higher than the atmospheric temperature, and the coal rock layer contains more water. When the electric shovel is shoveling, the high-temperature water vapor in the coal rock encounters the airflow of the external cold air and liquefies into water mist. Continuous shoveling operations cause dense fog at the operation site, which seriously affects the efficiency and safety of the shoveling operation. In the existing technology, the identification of dense fog areas in open-pit mines mainly relies on manual observation, which has the problem of low accuracy.

[0003] Therefore, there is an urgent need for a method to identify dense fog areas with high accuracy. Summary of the invention

[0004] The main purpose of the present application is to provide a defogging method and a defogging device, so as to at least solve the problem of low accuracy of the dense fog area identification method in the prior art.

[0005] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a defogging method is provided, comprising obtaining historical images of an open-pit mine working area and corresponding historical dense fog information, the historical dense fog information including whether the historical image has a dense fog area and, in the case of having the dense fog area, the location information of the dense fog area is the coordinates of the dense fog area; using the historical images and the corresponding historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, the improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with a PANet structure; obtaining a current image of the open-pit mine working area, and using the recognition model to recognize the current image to obtain current dense fog information; in the case where the current dense fog information indicates the existence of the dense fog area, calculating the area of ​​the dense fog area at least according to the location information of the dense fog area in the current dense fog information, and determining whether to perform defogging at least according to the size of the area of ​​the dense fog area.

[0006] Optionally, obtaining historical images of an open-pit mining area includes: collecting image data of the open-pit mining area; at least adjusting the ratio of the noise power spectral density of the image data to the image power spectral density of the image data to obtain an adjusted ratio; according to the adjusted ratio, eliminating motion blur of the image data to obtain a first image; at least setting parameters of adaptive histogram averaging to obtain first parameters; according to the first parameters, performing image enhancement on the first image to obtain a second image; at least setting parameters of a bilateral filter to obtain second parameters; according to the second parameters, performing denoising on the second image to obtain historical images of the open-pit mining area.

[0007] Optionally, the historical image and the corresponding historical dense fog information are used to train an improved YOLOv5 model to obtain a recognition model, including: inputting the historical image and the corresponding historical dense fog information into the improved YOLOv5 model; using the CSPDarknet53 structure of the improved YOLOv5 model to extract features of the historical image and the corresponding historical dense fog information to obtain a first feature layer; inputting the first feature layer into the PANet structure of the improved YOLOv5 model for feature fusion to obtain a second feature layer; setting bounding boxes with different aspect ratios according to feature maps of different resolutions in the second feature layer; performing category classification and bounding box regression on at least each of the bounding boxes to determine bounding box information, the bounding box information including the confidence difference of the category and the position of the first prediction box of the category; using a non-maximum suppression method to filter the bounding box information to obtain final bounding box information; and using at least the final bounding box information to update the initial bounding box information in the improved YOLOv5 model to obtain the recognition model.

[0008] Optionally, the first feature layer is input into the PANet structure of the improved YOLOv5 model for feature fusion to obtain the second feature layer, including: inputting the first feature layer into the PANet structure of the improved YOLOv5 model; the PANet structure extracts the first feature layer to obtain feature maps of different resolutions, the feature maps include high-level feature maps and low-level feature maps, the high-level feature maps are images formed by high-level features, and the low-level feature maps are images formed by low-level features; the PANet structure upsamples the high-level feature maps to obtain multiple first feature maps; the PANet structure adds multiple first feature maps to the corresponding low-level feature maps to obtain multiple third feature layers; the PANet structure horizontally connects the feature maps of different layers of the multiple third feature layers, and connects two adjacent feature maps of the same layer of the third feature layer to obtain the second feature layer.

[0009] Optionally, each of the bounding boxes is classified into categories to determine the confidence difference of the categories in the bounding box list, including: adding a classification head to the bounding box of the same aspect ratio in the second feature layer to trigger classification of the feature maps of different resolutions in the second feature layer to obtain multiple categories; using a Sigmoid function to activate the classification head to obtain the existence probability of each category, the categories including a true category and a first predicted category, the true category being a category with dense fog areas represented by the historical image in the historical dense fog information or a category without dense fog areas in the historical image, and the first predicted category being a category with the dense fog areas obtained when training the model or a category without the dense fog areas; determining the category with the highest existence probability in the first predicted category as the second predicted category; and calculating the difference between the existence probability of the second predicted category and the existence probability of the true category based on the existence probability to obtain the confidence difference.

[0010] Optionally, for each of the bounding boxes, a bounding box regression is performed to determine the position of the first predicted bounding box of the category in the bounding box list, including: adding a regression head to the bounding box of the same aspect ratio in the second feature layer to trigger the bounding box regression for the bounding box; using a Sigmoid function to process the regression head to obtain parameters of the regression head, the parameters of the regression head including the center coordinates of the bounding box and the size of the bounding box; according to the parameters of the regression head, using a CIOU loss function, at least calculating the overlapping area ratio of the second predicted bounding box and the true bounding box and the distance of the center coordinates to obtain a loss signal. information, the second predicted bounding box is the bounding box of the second predicted category, the true bounding box is the bounding box of the true category, and the overlapping area ratio is the ratio of the area of ​​the overlapping part of the second predicted bounding box and the true bounding box to the area of ​​the true bounding box; according to the loss information, an offset is calculated using a linear function, and the offset maximizes the overlapping area ratio and minimizes the distance from the center coordinates; the second predicted bounding box is moved by the offset to obtain a first bounding box; the first bounding box is scaled to obtain a second bounding box; and the position of the first predicted bounding box is determined according to the scaled second bounding box.

[0011] Optionally, after using the final bounding box information to update the initial bounding box information in the improved YOLOv5 model, the improved YOLOv5 model is trained using the historical image and the corresponding historical dense fog information to obtain the recognition model, including: acquiring mAP information obtained by training the improved YOLOv5 model with the historical image, the mAP information being an indicator generated in each iterative process when training the model; and updating the initial model parameters of the improved YOLOv5 model using the model parameters generated by the iteration corresponding to the highest mAP value in the mAP information to obtain the recognition model.

[0012] Optionally, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least based on the position information of the dense fog area in the current dense fog information, including: when the recognition model recognizes that the dense fog area exists in the current image, the dense fog area is marked by using a deep learning algorithm to obtain a first area; grayscale value extraction is performed on the first area to obtain extraction information, the extraction information at least including the position information of the first area; and the area of ​​the first area is calculated based on the extraction information.

[0013] Optionally, whether to perform defog is determined at least based on the size of the area, including: calculating the ratio of the area of ​​the first region to the area of ​​the current image based on the size of the area to obtain an area ratio; and determining to perform defog at least when the area ratio is greater than or equal to a predetermined threshold, wherein the predetermined threshold is set according to the degree of influence of dense fog on visibility.

[0014] According to another aspect of the present application, a demisting device is provided, comprising:

[0015] An acquisition unit is configured to acquire a historical image of an open-pit mine operation area and corresponding historical dense fog information, wherein the historical dense fog information includes whether the historical image has a dense fog area and, if the dense fog area exists, the location information of the dense fog area is the coordinates of the dense fog area;

[0016] A training unit, using the historical image and the corresponding historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with a PANet structure;

[0017] The recognition unit acquires a current image of the open-pit mine operation area, and uses the recognition model to recognize the current image to obtain current dense fog information;

[0018] The calculation unit calculates the area of ​​the dense fog area at least according to the position information of the dense fog area in the current dense fog information when the current dense fog information indicates the existence of the dense fog area, and determines whether to perform defogger at least according to the size of the area of ​​the dense fog area.

[0019] By applying the technical solution of the present application, first, historical images of the open-pit mine working area and corresponding historical dense fog information are obtained; secondly, the improved YOLOv5 model is trained using the above historical images and the corresponding historical dense fog information to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure; then, a current image of the open-pit mine working area is obtained, and the above recognition model is used to recognize the above current image to obtain current dense fog information; finally, in the case that the above current dense fog information indicates the existence of a dense fog area, the area of ​​the above dense fog area is calculated at least according to the position information of the above dense fog area in the above current dense fog information, and whether to perform defogger is determined at least according to the area of ​​the above dense fog area. Compared with the prior art, the solution of the present application can integrate feature information of different scales and semantic levels by utilizing the PANet structure of the improved YOLOv5 model for feature fusion, thereby improving the network's detection robustness for targets and making the network more generalizable. At the same time, the PANet structure can effectively reduce information loss, retain more useful information, and improve target detection performance and generalization ability, thereby solving the problem of low recognition accuracy in dense fog areas in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0021] Figure 1 A hardware structure block diagram of a mobile terminal for performing a defogging method provided in an embodiment of the present application is shown;

[0022] Figure 2 A schematic diagram of a demisting method according to an embodiment of the present application is shown;

[0023] Figure 3 A schematic diagram of a flow chart of determining whether to perform demisting according to an area ratio in a demisting method provided in an embodiment of the present application is shown;

[0024] Figure 4 A schematic structural diagram of a demisting system provided according to an embodiment of the present application is shown;

[0025] Figure 5A schematic flow chart of the main steps of a demisting method provided according to an embodiment of the present application is shown;

[0026] Figure 6 A structural block diagram of a demisting device provided according to an embodiment of the present application is shown;

[0027] Figure 7 A schematic diagram of a dense fog identification module of a defogging method provided according to an embodiment of the present application is shown.

[0028] The above drawings include the following reference numerals:

[0029] 50. switch; 60. defogger; 70. processor; 80. image acquisition device. DETAILED DESCRIPTION

[0030] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0031] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] As introduced in the background technology, the prior art has the problem of low accuracy in identifying dense fog areas. To solve the above problem, the embodiments of the present application provide a defogging method and a defogging device.

[0034] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0035] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a defogging method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0036] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0037] In this embodiment, a defogging method running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] Figure 2 is a flow chart of a defogging method according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0039] Step S201, obtaining historical images of the open-pit mine operation area and corresponding historical dense fog information, the historical dense fog information including whether the historical image has a dense fog area and, if so, the location information of the dense fog area is the coordinates of the dense fog area.

[0040] Specifically, high-definition industrial cameras are set up in the open-pit mine operation area to monitor the operation area in real time and collect image data. The above-mentioned industrial cameras should have a high-speed shooting function to adapt to the dynamic environment of open-pit mine operations, and use cold light sources to provide auxiliary light sources to ensure that the captured images are clear. After the collected pictures are pre-processed, the dense fog areas in the historical images are marked with image annotation software, and the file format generated by the marked dense fog areas is converted into a file format suitable for the YOLOv5 model algorithm. The content of the file includes whether there is a dense fog area and, if there is a dense fog area, the coordinates of the dense fog area.

[0041] Step S202, using the above historical images and the corresponding above historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure.

[0042] Specifically, the YOLOv5 model adopts a single-stage target detection method, that is, target detection and positioning are completed simultaneously in one network. It divides the input image into multiple grid cells, each of which is responsible for detecting the target in the image. The model uses a lightweight backbone network structure and adopts multi-scale training and data enhancement techniques to improve detection accuracy and generalization ability.

[0043] Specifically, PANet (Pyramid Attention Network) is a neural network structure for target detection tasks. Its main function is to fuse and weight features at different scales to improve the performance of target detection. Feature fusion refers to the integration and combination of features from different levels or scales to obtain richer information and more comprehensive context. PANet can effectively fuse multi-scale features through the pyramid attention mechanism, so that the model can better adapt to targets of different scales.

[0044] Step S203, obtaining a current image of the open-pit mine operation area, and using the above recognition model to recognize the current image to obtain current dense fog information.

[0045] Specifically, after obtaining the current image of the open-pit mine operation area, it is input into the pre-trained recognition model for analysis to obtain relevant information about the current dense fog situation. Through the analysis of the recognition model, it is possible to accurately determine whether the current operation area is affected by dense fog, and take corresponding measures in time to ensure the safety of the operators. This intelligent recognition technology can effectively improve the safety and production efficiency of mining operations, and provide strong support for mining management and monitoring.

[0046] Step S204, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, and whether to perform defogger is determined at least according to the size of the area of ​​the dense fog area.

[0047] Specifically, the signal processing module is responsible for calculating the area ratio of the dense fog area to the entire image, and determining whether to perform defog at least based on the size of the ratio.

[0048] In the above embodiment, first, historical images of the open-pit mine working area and corresponding historical dense fog information are obtained; secondly, the improved YOLOv5 model is trained using the above historical images and the corresponding historical dense fog information to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure; then, the current image of the open-pit mine working area is obtained, and the above recognition model is used to identify the above current image to obtain the current dense fog information; finally, in the case that the above current dense fog information indicates the existence of a dense fog area, the area of ​​the above dense fog area is calculated at least based on the position information of the above dense fog area in the above current dense fog information, and it is determined whether to perform defogger at least based on the area of ​​the above dense fog area. Compared with the prior art, the solution of the present application can integrate feature information of different scales and semantic levels by utilizing the PANet structure of the improved YOLOv5 model for feature fusion, thereby improving the network's detection robustness for targets and making the network more generalizable. At the same time, the PANet structure can effectively reduce information loss, retain more useful information, and improve target detection performance and generalization ability, thereby solving the problem of low recognition accuracy in dense fog areas in the prior art.

[0049] In one embodiment of the present application, obtaining historical images of an open-pit mine operation area includes: collecting image data of the open-pit mine operation area; at least adjusting the ratio of the noise power spectrum density of the image data to the image power spectrum density of the image data to obtain an adjustment ratio; according to the adjustment ratio, removing motion blur from the image data to obtain a first image; at least setting a parameter of adaptive histogram averaging to obtain a first parameter; according to the first parameter, performing image enhancement on the first image to obtain a second image; at least setting a parameter of a bilateral filter to obtain a second parameter; according to the second parameter, performing noise reduction on the second image to obtain a historical image of the open-pit mine operation area. In this method, by preprocessing the collected image data, the image quality is further improved and the noise is reduced.

[0050] Specifically, eliminating motion blur in image data usually involves a frequency domain filtering method. In this process, it is first necessary to adjust the ratio of the noise power spectrum density of the image data to the image power spectrum density, which can be achieved by performing noise preprocessing on the image. Then, according to the adjusted ratio, the image data is subjected to motion blur elimination, wherein the specific process includes: first, calculating the power spectrum density of the image data and the noise power spectrum density, and adjusting the ratio of the noise power spectrum density to the image power spectrum density; second, according to the adjusted ratio, using the Wiener filter to perform frequency domain filtering on the image data to eliminate motion blur in the image; finally, the image after motion blur elimination is obtained by inverse transformation, and these parameters are optimized through experimental data to adapt to the image characteristics in the open-pit mine environment, so that the motion blur in the image data in the dense fog area can be more effectively eliminated, and the clarity and quality of the image can be improved.

[0051] Specifically, the adaptive histogram equalization (CLAHE) in the image enhancement method is used to enhance the image contrast by limiting the contrast to avoid excessively amplifying the noise in certain areas during the image equalization process. The key parameters of CLAHE include contrast limit and block size, which are usually set to a contrast limit of 2 and a block size of 8x8 to avoid over-enhancement, thereby further improving the image quality.

[0052] Specifically, a bilateral filter is applied for denoising, which can retain image edges while removing noise. The parameters of the bilateral filter include the filter σ color in the color space and the filter σ space in the coordinate space, which are usually set to σ color = 50 and σ space = 15 to balance the denoising effect and edge retention, thereby making the image data clearer and further improving the accuracy of subsequent identification of dense fog areas.

[0053] In order to further improve the recognition accuracy of dense fog areas, in one embodiment of the present application, the above-mentioned historical images and the corresponding historical dense fog information are used to train an improved YOLOv5 model to obtain a recognition model, including: inputting the above-mentioned historical images and the corresponding historical dense fog information into the above-mentioned improved YOLOv5 model; using the CSPDarknet53 structure of the above-mentioned improved YOLOv5 model to extract features of the above-mentioned historical images and the corresponding historical dense fog information to obtain a first feature layer; inputting the above-mentioned first feature layer into the PANet structure of the above-mentioned improved YOLOv5 model for feature fusion to obtain a second feature layer; setting bounding boxes with different aspect ratios according to feature maps of different resolutions in the above-mentioned second feature layer; performing category classification and bounding box regression on at least each of the above-mentioned bounding boxes to determine bounding box information, the above-mentioned bounding box information including the confidence difference of the category and the position of the first prediction box of the above-mentioned category; using the non-maximum suppression method to filter the above-mentioned bounding box information to obtain final bounding box information; at least using the above-mentioned final bounding box information to update the initial bounding box information in the above-mentioned improved YOLOv5 model to obtain the above-mentioned recognition model.

[0054] Specifically, the input end of the improved YOLOv5 model is scaled to 640x640 pixels to adapt to the network input. Then, the Backbone structure of the YOLOv5 model uses CSPDarknet53 as a feature extractor, which contains multiple residual blocks. The parameters of each residual block are optimized through cross-validation. Secondly, the extracted features are transferred to the PANet structure in the Neck structure of the YOLOv5 model, which is used to build a feature pyramid network to enhance feature transfer. Finally, the fused features are transferred to the Prediction structure, in which preset functions are used for classification and bounding box regression, so that the identification of dense fog areas is more accurate.

[0055] In order to generate richer and more accurate feature representation, in one embodiment of the present application, the first feature layer is input into the PANet structure of the improved YOLOv5 model for feature fusion to obtain a second feature layer, including: inputting the first feature layer into the PANet structure of the improved YOLOv5 model; the PANet structure extracts the first feature layer to obtain the feature maps of different resolutions, the feature maps include high-level feature maps and low-level feature maps, the high-level feature maps are images formed by high-level features, and the low-level feature maps are images formed by low-level features; the PANet structure upsamples the high-level feature maps to obtain multiple first feature maps; the PANet structure adds the multiple first feature maps to the corresponding low-level feature maps to obtain multiple third feature layers; the PANet structure horizontally connects the feature maps of different layers of the multiple third feature layers, and connects two adjacent feature maps of the same layer of the third feature layer to obtain the second feature layer.

[0056] In another embodiment, each of the above-mentioned bounding boxes is classified into categories, and the confidence difference of the categories in the bounding box list is determined, including: adding a classification head to the above-mentioned bounding box with the same aspect ratio of the above-mentioned second feature layer to trigger the classification of the above-mentioned feature maps with different resolutions in the above-mentioned second feature layer to obtain multiple categories; using a Sigmoid function, activating the above-mentioned classification head to obtain the existence probability of each of the above-mentioned categories, the above-mentioned categories include a real category and a first predicted category, the above-mentioned real category is a category with dense fog areas represented by the above-mentioned historical image in the above-mentioned historical dense fog information or a category without dense fog areas in the above-mentioned historical image, and the above-mentioned first predicted category is a category with the above-mentioned dense fog areas obtained when training the model or a category without the above-mentioned dense fog areas; determining that the category with the largest existence probability in the above-mentioned first predicted category is the second predicted category; according to the above-mentioned existence probability, calculating the difference between the above-mentioned existence probability of the above-mentioned second predicted category and the above-mentioned existence probability of the above-mentioned real category to obtain the above-mentioned confidence difference.

[0057] Specifically, by classifying each bounding box and calculating the confidence difference, the most likely category of each bounding box can be determined, thereby further improving the accuracy of the target detection model for object categories.

[0058] Specifically, by adding and activating the classification head, feature maps of different resolutions can be classified to obtain more category information, further enhancing the classification ability of the model.

[0059] Specifically, by calculating the difference between the probability of existence of the predicted category and the probability of existence of the true category, we can obtain the confidence difference, which can more accurately evaluate the classification results of the model for each bounding box, further improving the robustness and generalization ability of the model.

[0060] In yet another embodiment, for each of the above-mentioned bounding boxes, a bounding box regression is performed to determine the position of the first predicted bounding box of the above-mentioned category in the bounding box list, including: adding a regression head to the above-mentioned bounding box of the same aspect ratio of the above-mentioned second feature layer to trigger the above-mentioned bounding box regression for the above-mentioned bounding box; using a Sigmoid function to process the above-mentioned regression head to obtain parameters of the above-mentioned regression head, and the parameters of the above-mentioned regression head include the center coordinates of the bounding box and the size of the bounding box; according to the parameters of the above-mentioned regression head, using a CIOU loss function, at least the overlapping area ratio of the second predicted bounding box and the true bounding box and the distance of the above-mentioned center coordinates are calculated to obtain the loss function. loss information, the second predicted bounding box is the bounding box of the second predicted category, the true bounding box is the bounding box of the true category, and the overlapping area ratio is the ratio of the area of ​​the overlapping part of the second predicted bounding box and the true bounding box to the area of ​​the true bounding box; according to the loss information, an offset is calculated using a linear function, and the offset makes the overlapping area ratio maximum and the distance from the center coordinates minimum; the second predicted bounding box is moved by the offset to obtain a first bounding box; the first bounding box is scaled to obtain a second bounding box; and the position of the first predicted bounding box is determined according to the scaled second bounding box.

[0061] Specifically, the predicted bounding box position is finely adjusted and optimized by performing bounding box regression on each bounding box, so that the predicted bounding box can more accurately frame the position of the target object. By adding a regression head and using the Sigmoid function, the center coordinates and size parameters of the bounding box can be obtained, and then the loss information is calculated through the CIOU loss function, and then the offset is calculated to maximize the overlap area ratio between the predicted bounding box and the true bounding box and minimize the distance between the center coordinates. Finally, by moving and scaling the predicted bounding box, a more accurate bounding box position is obtained, which further improves the accuracy and stability of identifying dense fog areas, so that the position of the bounding box can be more effectively optimized, making the prediction results of the model more accurate and reliable.

[0062] In order to further ensure that the model achieves the best results during the training process, after using the above-mentioned final bounding box information to update the initial bounding box information in the above-mentioned improved YOLOv5 model, the above-mentioned historical image and the corresponding historical fog information are used to train the improved YOLOv5 model to obtain the above-mentioned recognition model, including: obtaining the mAP information obtained by training the above-mentioned improved YOLOv5 model with the above-mentioned historical images, and the above-mentioned mAP information is an indicator generated in each iteration process when training the model; using the model parameters generated by the iteration corresponding to the highest mAP value in the above-mentioned mAP information to update the initial model parameters of the improved YOLOv5 model to obtain the above-mentioned recognition model.

[0063] In order to further determine the size of the identified dense fog area, in one embodiment of the present application, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, including: when the recognition model identifies that the current image contains the dense fog area, the dense fog area is marked by a deep learning algorithm to obtain a first area; grayscale value extraction is performed on the first area to obtain extraction information, and the extraction information at least includes the position information of the first area; and the area of ​​the first area is calculated according to the extraction information.

[0064] Specifically, through the recognition model and deep learning algorithm, the location of the dense fog area in the current image can be accurately marked, and the area can be marked to obtain the first area. By using the single-chip microcomputer to extract the grayscale value information of the first area, the industrial router receives the grayscale value signal processed by the single-chip microcomputer and calculates its area. The size and location of the dense fog area can be better understood, providing a more accurate reference for whether to perform defog operations later.

[0065] In yet another embodiment, whether to perform defog is determined at least based on the size of the above-mentioned area, including: calculating the ratio of the area of ​​the above-mentioned first region to the area of ​​the above-mentioned current image based on the size of the above-mentioned area to obtain the area ratio; at least when the above-mentioned area ratio is greater than or equal to a predetermined threshold, determining to perform defog is performed, and the above-mentioned predetermined threshold is set according to the degree of influence of dense fog on visibility.

[0066] Specifically, Figure 3 As shown, whether it is necessary to transmit the signal to the intelligent control module is determined according to the area size of the image and the degree of influence of dense fog on visibility. By calculating the ratio of the area of ​​the first area to the area of ​​the current image, a relative comparison index is obtained to determine whether there is a large dense fog area in the image. If the area ratio is greater than or equal to a predetermined threshold, the area ratio signal is transmitted to the intelligent control module.

[0067] Specifically, Figure 4 As shown, the intelligent control module receives the area ratio signal, and determines whether the area ratio of the dense fog area exceeds the standard according to the set threshold value. The switch 50 in the intelligent control module determines whether to activate the defogger. When the defogger 60 needs to be activated, a start signal is sent. After the defogger 60 receives the start signal, the fan of the defogger 60 starts to defog the working area. The defogger 60 is designed to automatically adjust the speed and wind direction of the fan according to the distribution of dense fog to achieve the most effective defogger effect. In this method, by comparing the area ratio, the defogger operation is further made more efficient and accurate.

[0068] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the demisting method of the present application will be described in detail below in conjunction with specific embodiments.

[0069] This embodiment relates to a specific defogging method, such as Figure 5 As shown, the following steps are included:

[0070] Image acquisition: using industrial cameras to obtain current images of the open-pit mine operation area;

[0071] Image preprocessing: that is, preprocessing the current image;

[0072] Dense fog recognition: The recognition model is used to recognize the current image to obtain dense fog information. The recognition model is obtained by training the improved YOLOv5 model using historical images and corresponding historical dense fog information. The improved YOLOv5 model is obtained by replacing the neck structure in the original YOLOv5 model with the PANet structure.

[0073] Gray value extraction: when the current dense fog information indicates that there is a dense fog area, the gray value of the dense fog area in the current image is extracted;

[0074] Gray value analysis: Analyze the gray value according to the extracted gray value to obtain the area of ​​dense fog area;

[0075] Decision-making: Calculate the area ratio of the dense fog area to the current image. When the area ratio of the dense fog area exceeds the set threshold, the instruction to start defog will be sent to the defog system;

[0076] Demisting system control: The demisting system starts the switch after receiving the demisting command;

[0077] Demisting operation: The switch controls the demisting equipment to perform demisting operation. Figure 4 Demisting blower shown.

[0078] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0079] The embodiment of the present application also provides a demisting device. It should be noted that the demisting device of the embodiment of the present application can be used to execute the demisting method provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred embodiments, and the descriptions have been made and will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware is also possible and conceived.

[0080] The following is an introduction to the demisting device provided in the embodiments of the present application.

[0081] Figure 6 Schematic diagram of a demisting device according to an embodiment of the present application. Figure 6 As shown, the device comprises:

[0082] The acquisition unit 10 acquires historical images of the open-pit mine operation area and corresponding historical dense fog information, wherein the historical dense fog information includes whether the historical image has a dense fog area and, if so, the location information of the dense fog area is the coordinates of the dense fog area.

[0083] Specifically, high-definition industrial cameras are set up in the open-pit mine operation area to monitor the operation area in real time and collect image data. The above-mentioned industrial cameras should have a high-speed shooting function to adapt to the dynamic environment of open-pit mine operations, and use cold light sources to provide auxiliary light sources to ensure that the captured images are clear. After the collected pictures are pre-processed, the dense fog areas in the historical images are marked with image annotation software, and the file format generated by the marked dense fog areas is converted into a file format suitable for the YOLOv5 model algorithm. The content of the file includes whether there is a dense fog area and, if there is a dense fog area, the coordinates of the dense fog area.

[0084] The training unit 20 uses the above historical images and the corresponding above historical dense fog information to train the improved YOLOv5 model to obtain a recognition model. The above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure.

[0085] Specifically, the YOLOv5 model adopts a single-stage target detection scheme, that is, target detection and localization are completed simultaneously in one network. It divides the input image into multiple grid cells, each of which is responsible for detecting the target in the image. The model uses a lightweight backbone network structure and adopts multi-scale training and data enhancement techniques to improve detection accuracy and generalization ability.

[0086] Specifically, PANet (Pyramid Attention Network) is a neural network structure for target detection tasks. Its main function is to fuse and weight features at different scales to improve the performance of target detection. Feature fusion refers to the integration and combination of features from different levels or scales to obtain richer information and more comprehensive context. PANet can effectively fuse multi-scale features through the pyramid attention mechanism, so that the model can better adapt to targets of different scales.

[0087] The recognition unit 30 obtains a current image of the open-pit mine operation area, and uses the above recognition model to recognize the above current image to obtain current dense fog information.

[0088] Specifically, after obtaining the current image of the open-pit mine operation area, it is input into the pre-trained recognition model for analysis to obtain relevant information about the current dense fog situation. Through the analysis of the recognition model, it is possible to accurately determine whether the current operation area is affected by dense fog, and take corresponding measures in time to ensure the safety of the operators. This intelligent recognition technology can effectively improve the safety and production efficiency of mining operations, and provide strong support for mining management and monitoring.

[0089] The calculation unit 40 calculates the area of ​​the dense fog area at least according to the position information of the dense fog area in the current dense fog information, and determines whether to perform defogger at least according to the size of the area of ​​the dense fog area, when the current dense fog information indicates the existence of the dense fog area.

[0090] Specifically, the signal processing module is responsible for calculating the area ratio of the dense fog area to the entire image, and determining whether to perform defog at least based on the size of the ratio.

[0091] In the above embodiment, first, the historical images of the open-pit mine operation area and the corresponding historical dense fog information are obtained; secondly, the improved YOLOv5 model is trained using the above historical images and the corresponding historical dense fog information to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure; then, the current image of the open-pit mine operation area is obtained, and the above recognition model is used to recognize the above current image to obtain the current dense fog information; finally, when the above current dense fog information indicates the existence of a dense fog area, the area of ​​the above dense fog area is calculated at least according to the position information of the above dense fog area in the above current dense fog information, and whether to perform defogging is determined at least according to the area of ​​the above dense fog area. Compared with the prior art, the solution of the present application fuses features by using the PANet structure of the improved YOLOv5 model, so that the model can understand the image information more comprehensively, improve the performance and generalization ability of target detection, and thus solve the problem of low recognition accuracy of dense fog areas in the prior art. Compared with the prior art, the solution of the present application can integrate feature information of different scales and semantic levels by utilizing the PANet structure of the improved YOLOv5 model for feature fusion, thereby improving the network's detection robustness for targets and making the network more generalizable. At the same time, the PANet structure can effectively reduce information loss, retain more useful information, and improve target detection performance and generalization ability, thereby solving the problem of low recognition accuracy in dense fog areas in the prior art.

[0092] In one embodiment of the present application, the acquisition unit includes an acquisition module, an adjustment module, an elimination module, a first setting module, an enhancement module, a second setting module and a noise reduction module, wherein the acquisition module is used to acquire image data of the above-mentioned open-pit mine operation area; the adjustment module is used to at least adjust the ratio of the noise power spectrum density of the above-mentioned image data to the image power spectrum density of the above-mentioned image data to obtain an adjustment ratio; the elimination module is used to eliminate motion blur of the above-mentioned image data according to the above-mentioned adjustment ratio to obtain a first image; the first setting module is used to at least set the parameters of the adaptive histogram averaging to obtain the first parameters; the enhancement module is used to enhance the above-mentioned first image according to the above-mentioned first parameters to obtain a second image; the second setting module is used to at least set the parameters of the bilateral filter to obtain the second parameters; the noise reduction module is used to reduce the noise of the above-mentioned second image according to the above-mentioned second parameters to obtain a historical image of the above-mentioned open-pit mine operation area. In this solution, by preprocessing the collected image data, the image quality is further improved and the noise is reduced.

[0093] Specifically, eliminating motion blur in image data usually involves a frequency domain filtering scheme. In this process, it is first necessary to adjust the ratio of the noise power spectrum density of the image data to the image power spectrum density, which can be achieved by performing noise preprocessing on the image. Then, according to the adjusted ratio, the image data is subjected to motion blur elimination, wherein the specific process includes: first, calculating the power spectrum density of the image data and the noise power spectrum density, and adjusting the ratio of the noise power spectrum density to the image power spectrum density; second, according to the adjusted ratio, using the Wiener filter to perform frequency domain filtering on the image data to eliminate motion blur in the image; finally, the image after motion blur elimination is obtained by inverse transformation, and these parameters are optimized through experimental data to adapt to the image characteristics in the open-pit mine environment, so that the motion blur in the image data in the dense fog area can be eliminated more effectively, and the clarity and quality of the image can be improved.

[0094] Specifically, the adaptive histogram equalization (CLAHE) in the image enhancement scheme is used to enhance the image contrast by limiting the contrast to avoid over-amplifying the noise in certain areas during the image equalization process. The key parameters of CLAHE include contrast limit and block size, which are usually set to a contrast limit of 2 and a block size of 8x8 to avoid over-enhancement, thereby further improving the image quality.

[0095] Specifically, a bilateral filter is applied for denoising, which can retain image edges while removing noise. The parameters of the bilateral filter include the filter σ color in the color space and the filter σ space in the coordinate space, which are usually set to σ color = 50 and σ space = 15 to balance the denoising effect and edge retention, thereby making the image data clearer and further improving the accuracy of subsequent identification of dense fog areas.

[0096] In order to further improve the recognition accuracy of dense fog areas, in one embodiment of the present application, the training unit includes an input module, a first extraction module, a fusion module, a third setting module, a determination module, a screening module and an update module, wherein the input module is used to input the above-mentioned historical image and the corresponding historical dense fog information into the above-mentioned improved YOLOv5 model; the first extraction module is used to use the CSPDarknet53 structure of the above-mentioned improved YOLOv5 model to perform feature extraction on the above-mentioned historical image and the corresponding historical dense fog information to obtain a first feature layer; the fusion module is used to input the above-mentioned first feature layer into the above-mentioned improved YOLOv5 model The PANet structure is used for feature fusion to obtain a second feature layer; the third setting module is used to set bounding boxes with different aspect ratios according to feature maps of different resolutions in the above second feature layer; the determination module is used to perform category classification and bounding box regression on at least each of the above bounding boxes to determine the bounding box information, where the above bounding box information includes the confidence difference of the category and the position of the first prediction box of the above category; the screening module is used to screen the above bounding box information by using the non-maximum suppression method to obtain the final bounding box information; the updating module is used to update the initial bounding box information in the above improved YOLOv5 model by at least using the above final bounding box information to obtain the above recognition model.

[0097] Specifically, in the dense fog area recognition module, the input end of the improved YOLOv5 model is scaled to 640x640 pixels to adapt to the network input. Then, the Backbone structure of the YOLOv5 model uses CSPDarknet53 as a feature extractor, which contains multiple residual blocks. The parameters of each residual block are optimized through cross-validation. Secondly, the extracted features are transferred to the PANet structure in the Neck structure of the YOLOv5 model, which is used to build a feature pyramid network to enhance feature transfer. Finally, the fused features are transferred to the Prediction structure, in which preset functions are used for classification and bounding box regression, so that the recognition of dense fog areas is more accurate.

[0098] In order to generate richer and more accurate feature representation, in one embodiment of the present application, the feature fusion module includes an input submodule, an extraction submodule, an upsampling submodule, an addition submodule and a connection submodule, wherein the input submodule is used to input the above-mentioned first feature layer into the above-mentioned PANet structure of the above-mentioned improved YOLOv5 model; the extraction submodule is used for the above-mentioned PANet structure to extract the above-mentioned first feature layer to obtain the above-mentioned feature maps of different resolutions, and the above-mentioned feature maps include high-level feature maps and low-level feature maps, the above-mentioned high-level feature maps are images formed by high-level features, and the above-mentioned low-level feature maps are images formed by low-level features; the upsampling submodule is used for the above-mentioned PANet structure to upsample the above-mentioned high-level feature maps to obtain multiple first feature maps; the addition submodule is used for the above-mentioned PANet structure to add multiple first feature maps with the corresponding low-level feature maps to obtain multiple third feature layers; the connection submodule is used for the above-mentioned PANet structure to laterally connect the above-mentioned feature maps of different layers of the above-mentioned third feature layers, and connect two adjacent above-mentioned feature maps of the same layer of the above-mentioned third feature layer to obtain the above-mentioned second feature layer.

[0099] In another embodiment, the determination module includes an adding submodule, an activation submodule, a first determining submodule and a first calculating submodule, wherein the adding submodule is used to add a classification head to the above-mentioned bounding box of the same aspect ratio of the above-mentioned second feature layer to trigger the classification of the above-mentioned feature maps of different resolutions in the above-mentioned second feature layer to obtain multiple categories; the activation submodule is used to activate the above-mentioned classification head using a Sigmoid function to obtain the existence probability of each of the above-mentioned categories, the above-mentioned categories include real categories and first predicted categories, the above-mentioned real categories are categories with dense fog areas represented by the above-mentioned historical images in the above-mentioned historical dense fog information or categories without dense fog areas in the above-mentioned historical images, and the above-mentioned first predicted categories are categories with the above-mentioned dense fog areas obtained when training the model or categories without the above-mentioned dense fog areas; the first determining submodule is used to determine that the category with the largest existence probability in the above-mentioned first predicted category is the second predicted category; the first calculating submodule is used to calculate the difference between the above-mentioned existence probability of the above-mentioned second predicted category and the above-mentioned existence probability of the above-mentioned real category according to the above-mentioned existence probability, and obtain the above-mentioned confidence difference value.

[0100] Specifically, by classifying each bounding box and calculating the confidence difference, the most likely category of each bounding box can be determined, thereby further improving the accuracy of the target detection model for object categories.

[0101] Specifically, by adding and activating the classification head, feature maps of different resolutions can be classified to obtain more category information, further enhancing the classification ability of the model.

[0102] Specifically, by calculating the difference between the probability of existence of the predicted category and the probability of existence of the true category, we can obtain the confidence difference, which can more accurately evaluate the classification results of the model for each bounding box, further improving the robustness and generalization ability of the model.

[0103] In another embodiment, the determination module also includes a second adding submodule, a processing submodule, a second calculating submodule, a third calculating submodule, a moving submodule, a scaling submodule and a second determining submodule, wherein the second adding submodule is used to add a regression head to the above-mentioned bounding box of the same aspect ratio of the above-mentioned second feature layer to trigger the above-mentioned bounding box regression of the above-mentioned bounding box; the processing submodule is used to process the above-mentioned regression head using the Sigmoid function to obtain the parameters of the above-mentioned regression head, and the parameters of the above-mentioned regression head include the center coordinates of the bounding box and the size of the bounding box; the second calculating submodule is used to calculate at least the overlapping area ratio of the second predicted bounding box and the true bounding box and the distance of the above-mentioned center coordinates according to the parameters of the above-mentioned regression head using the CIOU loss function. , obtain loss information, the above-mentioned second predicted bounding box is the above-mentioned bounding box of the second predicted category, the above-mentioned true bounding box is the above-mentioned bounding box of the true category, and the above-mentioned overlapping area ratio is the ratio of the area of ​​the overlapping part of the above-mentioned second predicted bounding box and the above-mentioned true bounding box to the area of ​​the above-mentioned true bounding box; the third calculation submodule is used to calculate the offset using a linear function according to the above-mentioned loss information, and the above-mentioned offset makes the above-mentioned overlapping area ratio maximum and the distance from the above-mentioned center coordinates minimum; the moving submodule is used to move the above-mentioned second predicted bounding box by the above-mentioned offset to obtain the first bounding box; the scaling submodule is used to scale the above-mentioned first bounding box to obtain the second bounding box; the second determination submodule is used to determine the position of the above-mentioned first predicted bounding box according to the scaled second bounding box.

[0104] Specifically, the predicted bounding box position is finely adjusted and optimized by performing bounding box regression on each bounding box, so that the predicted bounding box can more accurately frame the position of the target object. By adding a regression head and using the Sigmoid function, the center coordinates and size parameters of the bounding box can be obtained, and then the loss information is calculated through the CIOU loss function, and then the offset is calculated to maximize the overlap area ratio between the predicted bounding box and the true bounding box and minimize the distance between the center coordinates. Finally, by moving and scaling the predicted bounding box, a more accurate bounding box position is obtained, which further improves the accuracy and stability of identifying dense fog areas, so that the position of the bounding box can be more effectively optimized, making the prediction results of the model more accurate and reliable.

[0105] In order to further ensure that the model achieves the best results during the training process, the update module also includes an acquisition submodule and an update submodule, wherein the acquisition submodule is used to obtain the mAP information obtained by training the above-mentioned improved YOLOv5 model with the above-mentioned historical images, and the above-mentioned mAP information is an indicator generated in each iteration process when training the model; the update submodule is used to update the initial model parameters of the improved YOLOv5 model using the model parameters generated by the iteration corresponding to the highest mAP value in the above-mentioned mAP information to obtain the above-mentioned recognition model.

[0106] In order to further determine the size of the identified dense fog area, in one embodiment of the present application, the calculation unit includes a marking module and a second extraction module, wherein the marking module is used to mark the dense fog area by using a deep learning algorithm to obtain a first area when the above-mentioned recognition model identifies that the above-mentioned dense fog area exists in the above-mentioned current image; the second extraction module is used to extract the grayscale value of the above-mentioned first area to obtain extraction information, and the above-mentioned extraction information at least includes the location information of the above-mentioned first area; according to the above-mentioned extraction information, the area of ​​the above-mentioned first area is calculated.

[0107] Specifically, through the recognition model and deep learning algorithm, the location of the dense fog area in the current image can be accurately marked, and the area can be marked to obtain the first area. By using the single-chip microcomputer to extract the grayscale value information of the first area, the industrial router receives the grayscale value signal processed by the single-chip microcomputer and calculates its area. The size and location of the dense fog area can be better understood, providing a more accurate reference for whether to perform defog operations later.

[0108] In another embodiment, the calculation unit also includes a fourth calculation submodule and a third determination submodule, wherein the fourth calculation submodule is used to calculate the ratio of the area of ​​the above-mentioned first region to the area of ​​the above-mentioned current image according to the size of the above-mentioned area to obtain the area ratio; the third determination submodule is used to determine to perform defog at least when the above-mentioned area ratio is greater than or equal to a predetermined threshold value, and the above-mentioned predetermined threshold value is set according to the degree of influence of dense fog on visibility.

[0109] Specifically, in the signal processing module and the intelligent control module, the signal processing module determines whether defogging is required based on the area size of the image and the degree of influence of dense fog on visibility. By calculating the ratio of the area of ​​the first area to the area of ​​the current image, a relative comparison index is obtained to determine whether there is a large dense fog area in the image. If the area ratio is greater than or equal to a predetermined threshold, the signal is transmitted to the intelligent control module. The intelligent control module receives the area ratio signal from the signal processing module, and determines whether the area ratio of the dense fog area exceeds the standard based on the set threshold. The intelligent start-stop decision module in the intelligent control module determines whether to activate the defogging device. When the intelligent start-stop decision module activates the defogging system, it sends a start signal. After the defogging system receives the start signal, the fan starts to perform defogging operations on the operating area. The defogging system is designed to automatically adjust the speed and wind direction of the fan according to the distribution of dense fog to achieve the most effective defogging effect. In this scheme, by comparing the area ratio, the defogging operation is further made more efficient and accurate.

[0110] The above-mentioned demisting device includes a processor and a memory, and the above-mentioned acquisition unit, training unit, recognition unit and calculation unit are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize corresponding functions. The above-mentioned modules are all located in the same processor; or, the above-mentioned modules are located in different processors in any combination.

[0111] The processor includes a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set to solve the problem of low recognition accuracy in dense fog areas by adjusting kernel parameters.

[0112] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0113] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the defogging method, the method comprising:

[0114] Step S201, obtaining historical images of the open-pit mine operation area and corresponding historical dense fog information, the historical dense fog information including whether the historical image has a dense fog area and, if so, the location information of the dense fog area is the coordinates of the dense fog area.

[0115] Step S202, using the above historical images and the corresponding above historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure.

[0116] Step S203, obtaining a current image of the open-pit mine operation area, and using the above recognition model to recognize the current image to obtain current dense fog information.

[0117] Step S204, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, and whether to perform defogger is determined at least according to the size of the area of ​​the dense fog area.

[0118] An embodiment of the present invention provides a processor, such as Figure 7 As shown, the processor 70 is used to run a program, wherein the program executes a dense fog information identification method when it is run, and the method includes:

[0119] Step S201, obtaining historical images of the open-pit mine operation area and corresponding historical dense fog information, the historical dense fog information including whether the historical image has a dense fog area and, if so, the location information of the dense fog area is the coordinates of the dense fog area.

[0120] Step S202, using the above historical images and the corresponding above historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure.

[0121] Step S203, obtaining a current image of the open-pit mine operation area, and using the above recognition model to recognize the current image to obtain current dense fog information.

[0122] Step S204, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, and whether to perform defogger is determined at least according to the size of the area of ​​the dense fog area.

[0123] Figure 7 Also shown is an image acquisition device 80, which sends the acquired image to the processor 70 so that the processor 70 can identify dense fog information.

[0124] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0125] Step S201, obtaining historical images of the open-pit mine operation area and corresponding historical dense fog information, the historical dense fog information including whether the historical image has a dense fog area and, if so, the location information of the dense fog area is the coordinates of the dense fog area.

[0126] Step S202, using the above historical images and the corresponding above historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure.

[0127] Step S203, obtaining a current image of the open-pit mine operation area, and using the above recognition model to recognize the current image to obtain current dense fog information.

[0128] Step S204, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, and whether to perform defogger is determined at least according to the size of the area of ​​the dense fog area.

[0129] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0130] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0131] Step S201, obtaining historical images of the open-pit mine operation area and corresponding historical dense fog information, the historical dense fog information including whether the historical image has a dense fog area and, if so, the location information of the dense fog area is the coordinates of the dense fog area.

[0132] Step S202, using the above historical images and the corresponding above historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure.

[0133] Step S203, obtaining a current image of the open-pit mine operation area, and using the above recognition model to recognize the current image to obtain current dense fog information.

[0134] Step S204, when the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least according to the position information of the dense fog area in the current dense fog information, and whether to perform defogger is determined at least according to the size of the area of ​​the dense fog area.

[0135] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0136] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0137] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0138] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0139] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0140] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0141] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0142] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes and other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0143] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0144] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0145] In the above embodiment, first, historical images of the open-pit mine working area and corresponding historical dense fog information are obtained; secondly, the improved YOLOv5 model is trained using the above historical images and the corresponding historical dense fog information to obtain a recognition model, wherein the above improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure; then, the current image of the open-pit mine working area is obtained, and the above recognition model is used to identify the above current image to obtain the current dense fog information; finally, in the case that the above current dense fog information indicates the existence of a dense fog area, the area of ​​the above dense fog area is calculated at least based on the position information of the above dense fog area in the above current dense fog information, and it is determined whether to perform defogger at least based on the area of ​​the above dense fog area. Compared with the prior art, the solution of the present application can integrate feature information of different scales and semantic levels by utilizing the PANet structure of the improved YOLOv5 model for feature fusion, thereby improving the network's detection robustness for targets and making the network more generalizable. At the same time, the PANet structure can effectively reduce information loss, retain more useful information, and improve target detection performance and generalization ability, thereby solving the problem of low recognition accuracy in dense fog areas in the prior art.

[0146] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A demisting method, characterized in that: include: Acquire historical images of the open-pit mine operation area and corresponding historical dense fog information, wherein the historical dense fog information includes whether the historical image has a dense fog area and, if the dense fog area exists, the location information of the dense fog area is the coordinates of the dense fog area; The improved YOLOv5 model is trained using the historical image and the corresponding historical dense fog information to obtain a recognition model, wherein the improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with the PANet structure; Acquire a current image of an open-pit mine operation area, and use the recognition model to recognize the current image to obtain current dense fog information; In the case where the current dense fog information indicates the existence of the dense fog area, the area of ​​the dense fog area is calculated at least based on the position information of the dense fog area in the current dense fog information, and whether to perform defogger is determined at least based on the size of the area of ​​the dense fog area.

2. The demisting method according to claim 1, characterized in that: Access historical imagery of open pit mine operating areas, including: Collecting image data of the open-pit mine operation area; At least adjusting a ratio of a noise power spectrum density of the image data to an image power spectrum density of the image data to obtain an adjusted ratio; According to the adjustment ratio, the image data is subjected to motion blur removal to obtain a first image; At least setting a parameter of adaptive histogram averaging to obtain a first parameter; performing image enhancement on the first image according to the first parameter to obtain a second image; At least setting a parameter of the bilateral filter to obtain a second parameter; The second image is denoised according to the second parameter to obtain a historical image of the operating area of ​​the open-pit mine.

3. The demisting method according to claim 1, characterized in that: The improved YOLOv5 model is trained using the historical image and the corresponding historical dense fog information to obtain a recognition model, including: Inputting the historical image and the corresponding historical dense fog information into the improved YOLOv5 model; Using the CSPDarknet53 structure of the improved YOLOv5 model, feature extraction is performed on the historical image and the corresponding historical dense fog information to obtain a first feature layer; Inputting the first feature layer into the PANet structure of the improved YOLOv5 model for feature fusion to obtain a second feature layer; According to feature maps of different resolutions in the second feature layer, setting bounding boxes of different aspect ratios; Performing category classification and bounding box regression on at least each of the bounding boxes to determine bounding box information, wherein the bounding box information includes a confidence difference of the category and a position of a first predicted box of the category; The bounding box information is screened using a non-maximum suppression method to obtain final bounding box information; At least the final bounding box information is used to update the initial bounding box information in the improved YOLOv5 model to obtain the recognition model.

4. The demisting method according to claim 3, characterized in that: Inputting the first feature layer into the PANet structure of the improved YOLOv5 model for feature fusion to obtain a second feature layer, including: Inputting the first feature layer into the PANet structure of the improved YOLOv5 model; The PANet structure extracts the first feature layer to obtain the feature maps of different resolutions, wherein the feature maps include high-level feature maps and low-level feature maps, wherein the high-level feature maps are images formed by high-level features, and the low-level feature maps are images formed by low-level features; The PANet structure upsamples the high-level feature map to obtain a plurality of first feature maps; The PANet structure adds the plurality of the first feature maps to the corresponding low-level feature maps to obtain a plurality of third feature layers; The PANet structure laterally connects the feature maps of different layers of the plurality of third feature layers, and connects two adjacent feature maps of the same layer of the third feature layer to obtain the second feature layer.

5. The demisting method according to claim 3, characterized in that: For each of the bounding boxes, classify the categories and determine the confidence difference of the categories in the bounding box list, including: Adding a classification head to the bounding box of the same aspect ratio in the second feature layer to trigger classification of the feature maps of different resolutions in the second feature layer to obtain multiple categories; The classification head is activated by using a Sigmoid function to obtain the existence probability of each of the categories, wherein the categories include a true category and a first predicted category, wherein the true category is a category with a dense fog area represented by the historical image in the historical dense fog information or a category without a dense fog area in the historical image, and the first predicted category is a category with a dense fog area or a category without a dense fog area obtained when training the model; Determine the category with the highest existence probability in the first prediction categories as the second prediction category; According to the existence probability, a difference between the existence probability of the second predicted category and the existence probability of the true category is calculated to obtain the confidence difference value.

6. The demisting method according to claim 3, characterized in that: For each of the bounding boxes, performing bounding box regression to determine the position of the first predicted bounding box of the category in the bounding box list includes: Adding a regression head to the bounding box of the same aspect ratio of the second feature layer to trigger the bounding box regression for the bounding box; Using a Sigmoid function, the regression head is processed to obtain parameters of the regression head, where the parameters of the regression head include the center coordinates of the bounding box and the size of the bounding box; According to the parameters of the regression head, at least the overlapping area ratio of the second predicted bounding box and the true bounding box and the distance of the center coordinates are calculated using the CIOU loss function to obtain loss information, wherein the second predicted bounding box is the bounding box of the second predicted category, the true bounding box is the bounding box of the true category, and the overlapping area ratio is the ratio of the area of ​​the overlapping part of the second predicted bounding box and the true bounding box to the area of ​​the true bounding box; Calculate an offset using a linear function according to the loss information, wherein the offset maximizes the overlap area ratio and minimizes the distance between the center coordinates; Moving the second predicted bounding box by the offset to obtain a first bounding box; Scaling the first bounding box to obtain a second bounding box; The position of the first predicted bounding box is determined according to the scaled second bounding box.

7. The demisting method according to claim 3, characterized in that: After the final bounding box information is used to update the initial bounding box information in the improved YOLOv5 model, the improved YOLOv5 model is trained using the historical image and the corresponding historical dense fog information to obtain the recognition model, including: Obtain mAP information obtained by training the improved YOLOv5 model with the historical image, where the mAP information is an indicator generated in each iteration process when training the model; The model parameters generated by the iteration corresponding to the highest mAP value in the mAP information are used to update the initial model parameters of the improved YOLOv5 model to obtain the recognition model.

8. The demisting method according to claim 1, characterized in that: In the case where the current dense fog information indicates the existence of the dense fog area, calculating the area of ​​the dense fog area at least according to the position information of the dense fog area in the current dense fog information includes: When the recognition model recognizes that the current image has the dense fog area, the dense fog area is marked by using a deep learning algorithm to obtain a first area; Extracting grayscale values ​​of the first region to obtain extraction information, wherein the extraction information at least includes location information of the first region; The area of ​​the first region is calculated according to the extracted information.

9. The demisting method according to claim 8, characterized in that: At least according to the size of the area, determining whether to perform demisting includes: According to the size of the area, calculating the ratio of the area of ​​the first region to the area of ​​the current image to obtain an area ratio; At least when the area ratio is greater than or equal to a predetermined threshold, it is determined to perform defogger, and the predetermined threshold is set according to the degree of influence of dense fog on visibility.

10. A demisting device, characterized in that: include: An acquisition unit is configured to acquire a historical image of an open-pit mine operation area and corresponding historical dense fog information, wherein the historical dense fog information includes whether the historical image has a dense fog area and, if the dense fog area exists, the location information of the dense fog area is the coordinates of the dense fog area; A training unit, using the historical image and the corresponding historical dense fog information to train an improved YOLOv5 model to obtain a recognition model, wherein the improved YOLOv5 model is obtained by replacing the neck structure in the YOLOv5 model with a PANet structure; The recognition unit acquires a current image of the open-pit mine operation area, and uses the recognition model to recognize the current image to obtain current dense fog information; The calculation unit calculates the area of ​​the dense fog area at least according to the position information of the dense fog area in the current dense fog information when the current dense fog information indicates the existence of the dense fog area, and determines whether to perform defogger at least according to the size of the area of ​​the dense fog area.

Images