A multi-source data fusion unexploded bomb detection method based on deep learning

Abstract

This invention proposes a method for detecting unexploded ordnance using multi-source data fusion based on deep learning. This method primarily addresses the problems of low accuracy and excessive false alarms and missed alarms in existing unexploded ordnance detection technologies. The method comprises: a data processing module processes received magnetic detection data and image data and transmits them to a deep learning model, which then identifies unexploded ordnance; a data storage module records the input and output of the deep learning model; and a display module displays the acquired magnetic field, image, and location of the unexploded ordnance. This invention utilizes both the magnetic detection module and a camera to acquire information, and obtains unexploded ordnance detection results based on multi-source data fusion and deep learning. This effectively avoids the shortcomings of existing detection methods using a single magnetic detection module, resulting in more comprehensive and diverse information, reducing false alarms and missed alarms, further ensuring personal safety, and reducing resource consumption.

Description

Technical Field

[0001] The present invention belongs to the technical field of unexploded bomb detection, and in particular relates to an unexploded bomb detection method based on multi-source data fusion based on deep learning. Background Art

[0002] Unexploded ordnance (UXO) generally refers to various unexploded or abandoned munitions left in an area after armed conflict, military training, or exercises. In modern high-tech warfare, both sides use a variety of ammunition, leaving significant quantities of UXO both above and below the ground. During routine military training and exercises, UXO also frequently appears at test sites, firing ranges, or training locations, posing significant risks to subsequent testing and training, as well as the livelihoods and production of surrounding residents. Over the past 30 years, these UXO have resulted in approximately 30,000 serious injuries annually and extensive land contamination. Consequently, the removal of these UXO is urgently needed.

[0003] Accurate detection and positioning of unexploded bombs is one of the important links in ensuring personnel safety. Traditional detection methods mostly use manual methods, which require people or vehicles to arrive near the target, which is inefficient and cannot guarantee safety. In order to solve this problem, predecessors have proposed the solution of aerial detection by drones. The current aerial detection technology mainly relies on geophysical exploration technology, mainly electromagnetic detection and magnetic detection. However, the detection environment is often very complex. The large number of scattered shrapnel and exploded bombs in the detection environment cause great interference to the magnetic detection method, and there are many types of unexploded ammunition. Therefore, a single detection technology often has the problem of missed reports and high false alarm rates. Missed reports will result in casualties and property losses. [1] False alarms will cause a waste of resources and manpower. A single detection method has a false alarm rate of more than 40% in extremely complex environments, causing great casualties and property losses. The false alarm rate is several times as low as several times as high as dozens of times. [1,2] , which greatly increases the workload of subsequent troubleshooting operations.

[0004] References

[0005] [1]Xu Jianguo, Ding Kai, Li Yangming. Current status and thinking of unexploded ordnance detection technology[J]. China Public Security, 2020(04):176-178.

[0006] [2] Wang Haofeng. Research on portable transient electromagnetic unexploded bomb detection system[D]. Jilin University, 2019. Summary of the Invention

[0007] Technical Problem: To address the shortcomings of existing technologies, this paper proposes a multi-source data fusion method for unexploded ordnance detection based on deep learning. This method effectively overcomes the shortcomings of existing detection methods using only a magnetic detection module, providing more comprehensive and diverse information and reducing false alarms and missed detections. By simultaneously utilizing the magnetic detection module and a camera to acquire information, the paper uses data fusion and deep learning to determine the location and type of unexploded ordnance, achieving more accurate and efficient detection results, further ensuring personal safety and reducing resource consumption.

[0008] Technical Solution: The purpose of the present invention is to provide a method for detecting unexploded bombs using multi-source data fusion based on deep learning. The method is based on a magnetic detection module, a camera, a data processing module, a data storage module, a deep learning module, and a display module. The method comprises the following steps:

[0009] Step 1: Processing the captured information through the magnetic detection module, camera and data processing module. The data processing module receives the magnetic sensor data captured by the magnetic detection module and the image data captured by the camera, performs denoising and reconstruction operations on them to obtain the required vector data, and transmits it to the deep learning model;

[0010] Step 2: The deep learning module performs unexploded bomb identification processing based on the data input by the data processing module, outputs information about whether an unexploded bomb exists, the type of unexploded bomb, and the location, and transmits the output result to the data storage module;

[0011] Step 3: The input and output information of the deep learning module is stored in the database through the data storage module for subsequent use;

[0012] Step 4: The display module processes the received data and displays the magnetic field distribution, image, location and type of unexploded bombs.

[0013] in,

[0014] In step one, the magnetic detection module obtains the magnetic field data of the current detection area, and the camera obtains global or local image data of the detection area; the data processing module processes the magnetic detection data, filters out background noise and the interference magnetic field of the detection equipment to obtain a high-quality magnetic field distribution of the detection area, and transmits it together with the image data to the deep learning model.

[0015] In step 2, the deep learning model is trained using existing magnetic field data, image data, unexploded bomb location identifiers, and known unexploded bomb types to obtain an unexploded bomb identification model. The unexploded bomb identification model uses the magnetic detection data and image data to obtain the location and type of the unexploded bomb. The unexploded bomb location identification result of the deep learning model is a rectangular box, and the location identifier includes the center coordinates (x, y) of the rectangular box and the width w and height h of the rectangular box.

[0016] When training the deep learning model, samples in a training data set are input into the deep learning network for unexploded bomb recognition training. The training data set includes magnetic field distribution data, image data, unexploded bomb location identification, and unexploded bomb type information of the sample area.

[0017] In step three, the data storage module stores the magnetic field distribution and image data of the detection area, and the location and type information of the unexploded bombs, to facilitate the subsequent destruction of unexploded bombs and the training of deep learning models.

[0018] In step 4, the display module processes the received data and displays the magnetic field distribution, the local image of the detection area captured by the camera, and whether there is an unexploded bomb in the current detection area. If there is an unexploded bomb, a rectangular frame is used to mark its location in the magnetic field distribution and the detection area image, and the type of unexploded bomb is indicated.

[0019] Beneficial Effects: Compared to existing technologies, this present invention offers the following advantages: This deep learning-based multi-source data fusion unexploded bomb detection method effectively avoids the shortcomings of existing single-magnetic detection methods, generating more comprehensive and diverse information and reducing false alarms and missed detections. By simultaneously utilizing the magnetic detection module and camera to acquire information, and using data fusion and deep learning to generate unexploded bomb detection results, this method is more accurate and efficient, further ensuring personal safety and reducing resource consumption. The deep learning model can update the training set based on collected data, facilitating training and improvement of the learning model, resulting in even better detection performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 Schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0021] The following will provide a clear and complete description of the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. However, the described embodiments are only part of, and not all of, the embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort shall fall within the scope of protection of the present invention.

[0022] like Figure 1 As shown, the present invention discloses a method for detecting unexploded bombs using multi-source data fusion based on deep learning, comprising the following steps:

[0023] Step 1: The data processing module receives the magnetic sensing data captured by the magnetic detection module and the image data captured by the camera, and performs denoising, reconstruction and other operations on them to obtain the required vector data, and transmits it to the deep learning model.

[0024] Step 2: The deep learning model identifies unexploded ordnance based on the input data, outputs the presence of unexploded ordnance, and outputs its type and location, and transmits the output to a data storage module. Step 3: The data storage module stores the input and output of the deep learning model and the type of unexploded ordnance in a database to facilitate subsequent related work and model training.

[0025] Step 4: The display module processes the received data and displays information such as magnetic field distribution, images, location and type of unexploded bombs.

[0026] In step one, the magnetic detection module obtains the magnetic field data of the current detection area, and the camera obtains the global or local image data of the detection area. The data processing module uses the relevant signal processing method to process the magnetic detection data to eliminate the influence of the background magnetic field. The purpose of data processing is to reduce the noise signal, extract the magnetic anomaly signal and thus improve the signal-to-noise ratio. The main signal processing methods include low-pass filtering and detrending method. The sampled magnetic field intensity signal e(n) is Fourier transformed to obtain the spectrum E(f). The high-frequency component is filtered out using a low-pass filter to obtain the filtered spectrum.

[0027]

[0028] where f g is the cutoff frequency of the low-pass filter. Perform inverse Fourier transform to obtain the filtered signal On this basis, the least square method is used to perform detrending processing on the filtered signal, and the optimal (least square) fitting function is subtracted to finally obtain the high-quality magnetic field distribution vector e=(e1, e2,…, e mn ). Convert the RGB three-channel two-dimensional image data into a vector λ=(r1, r2,…, r pq ,g1,g2,…,g pq ,b1,b2,…,b pq ), where p and q are the width and height of the image respectively, r is the R channel data, g is the G channel data, and b is the B channel data; finally, the image data vector and the magnetic field distribution vector are transmitted to the deep learning model together.

[0029] In step 2, a deep learning model is trained using existing magnetic field data, image data, and unexploded ordnance location markers. This model can identify unexploded ordnance using magnetic detection data and image data to determine its location. The deep learning model generates an unexploded ordnance identification result as a rectangular box. The location marker includes the box's center coordinates (x, y) as well as its width w and height h. The deep learning model inputs include the magnetic field distribution vector e and the vector λ obtained by converting the image's RGB three-channel two-dimensional data. The model outputs (x, y), w, h, and the unexploded ordnance type serial number s through a convolutional neural network. The relationship between the input and output can be represented by the function (x, y, w, h, s) = f(e, λ).

[0030] In addition, when training the deep learning model, the samples in the training dataset are input into the deep learning network for unexploded bomb recognition training. The training dataset includes magnetic field distribution data, image data, and unexploded bomb location markers of the sample area. In the process of generating the training dataset, it is necessary to convert between the actual pixel position of the location marker and the marker format required for deep learning model training. Taking image data as an example, w img ,h img The width and height of the image need to be normalized to obtain the normalized center coordinates of the rectangular frame as well as the width and height. Normalized marker box width Normalized marker height The normalized data is the data identification form required for deep learning model training.

[0031] In step three, the data storage module stores magnetic field distribution data, detection area image data, and unexploded ordnance location and type information. This data facilitates the subsequent generation of an unexploded ordnance distribution map and the destruction of unexploded ordnance based on its location information. Furthermore, the stored data can be used to further train deep learning models for more accurate identification results.

[0032] In step 4, the display module processes the received data and displays the magnetic field distribution, the local image of the detection area captured by the camera, and whether there are any unexploded bombs in the current detection area. If there are any unexploded bombs, their location in the magnetic field and the image of the current detection area is marked with a rectangular frame, and the type of unexploded bomb is indicated.

[0033] When using this deep learning-based multi-source data fusion unexploded bomb detection method, the magnetic field distribution and image are recognized through the deep learning model, the recognition results and magnetic field information and image information are sorted through the data sorting module, the magnetic field distribution, image and unexploded bomb location information are stored through the data storage module to facilitate subsequent work and the training and improvement of the deep learning model, and the magnetic field, image information and unexploded bomb location of the detection area are displayed through the display module to facilitate relevant personnel to observe and carry out related work.

[0034] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes based on the technical solutions and inventive concepts of the present invention within the scope disclosed by the present invention, which fall within the scope of protection of the present invention.

Claims

1. A multi-source data fusion unexploded bomb detection method based on deep learning, characterized by: The method is based on a magnetic detection module, a camera, a data processing module, a data storage module, a deep learning module, and a display module; and includes the following steps: Step 1: Processing the captured information through the magnetic detection module, camera and data processing module. The data processing module receives the magnetic sensor data captured by the magnetic detection module and the image data captured by the camera, performs denoising and reconstruction operations on them to obtain the required vector data, and transmits it to the deep learning model; Step 2: The deep learning module performs unexploded bomb identification processing based on the data input by the data processing module, outputs information about whether an unexploded bomb exists, the type of unexploded bomb, and the location, and transmits the output result to the data storage module; The deep learning model is trained using existing magnetic field data, image data, unexploded bomb location identifiers, and known unexploded bomb types to obtain an unexploded bomb identification model. The unexploded bomb identification model uses magnetic detection data and image data to obtain the location and type information of the unexploded bomb. The unexploded bomb location identification result of the deep learning model is a rectangular box, and the location identifier includes the center coordinates of the rectangular box ( x,y ) and the width of the rectangular frame w and high h； Step 3: The input and output information of the deep learning module is stored in the database through the data storage module for subsequent use; Step 4: The display module processes the received data and displays the magnetic field distribution, image, location and type of unexploded bombs.

2. The method for detecting unexploded bombs using multi-source data fusion based on deep learning according to claim 1, characterized in that: In step one, the magnetic detection module obtains the magnetic field data of the current detection area, and the camera obtains global or local image data of the detection area; the data processing module processes the magnetic detection data, filters out background noise and the interference magnetic field of the detection equipment to obtain a high-quality magnetic field distribution of the detection area, and transmits it together with the image data to the deep learning model.

3. The method for detecting unexploded bombs using multi-source data fusion based on deep learning according to claim 1, characterized in that: When training the deep learning model, samples in a training data set are input into the deep learning network for unexploded bomb recognition training. The training data set includes magnetic field distribution data, image data, unexploded bomb location identification, and unexploded bomb type information of the sample area.

4. The method for detecting unexploded bombs using multi-source data fusion based on deep learning according to claim 1, characterized in that: In step three, the data storage module stores the magnetic field distribution and image data of the detection area, and the location and type information of the unexploded bombs, to facilitate the subsequent destruction of unexploded bombs and the training of deep learning models.

5. The method for detecting unexploded bombs using multi-source data fusion based on deep learning according to claim 1, characterized in that: In step 4, the display module processes the received data and displays the magnetic field distribution, the local image of the detection area captured by the camera, and whether there is an unexploded bomb in the current detection area. If there is an unexploded bomb, a rectangular frame is used to mark its location in the magnetic field distribution and the detection area image, and the type of unexploded bomb is indicated.

Images