Underwater target tracking system based on DBN

Abstract

The invention relates to the technical field of ocean detection, in particular to a DBN-based underwater target tracking system, which comprises a multi-source data processing module, an environment monitoring module, a feature fusion module, a target feature extraction module and a dynamic prediction recognition module. According to the invention, through real-time signal-to-noise ratio evaluation of sonar and visual data and dynamic adjustment of modal weight, data reliability under the condition of insufficient illumination or complex water quality is ensured, and a data fusion strategy is optimized in combination with illumination intensity, turbid concentration and visible distance, so that the system has good environmental adaptability. Time sequence features and image edge gradients are utilized, key geometric differences are extracted, the feature recognition distinction degree is improved, significant edge features are screened and optimized, a target area is highlighted, the recognition precision is effectively improved, and target tracking conditions can be judged in real time based on migration analysis of DBN node response; and the underwater detection efficiency and the applicability in a complex environment are improved.

Description

Technical Field

[0001] The present invention relates to the field of ocean detection technology, and in particular to an underwater target tracking system based on DBN. Background Art

[0002] The field of ocean exploration involves the use of various scientific methods and tools to study and monitor the marine environment and its phenomena, including physical, chemical, biological and geological exploration, with the aim of understanding the dynamic processes of the ocean, its ecosystem and its impact on the global environment. It includes acoustic positioning, satellite remote sensing, deep-sea submersibles, and various sensors and instruments for measuring ocean temperature, salinity, current velocity and other parameters. Ocean exploration is not only a part of scientific research, but also crucial to navigation safety, marine resource development, environmental protection and climate change research.

[0003] Among them, DBN's underwater target tracking system involves the use of deep belief networks (DBNs) to achieve automatic tracking of underwater targets. DBN is a deep learning algorithm that can learn features and patterns from large amounts of data. In underwater target tracking systems, DBNs are used to process and parse data collected from sonar, radar or other sensors to identify and track underwater objects such as submarines, schools of fish or other marine life. Its main uses include marine scientific research, military reconnaissance, fishery management, and search and rescue missions, which improves the efficiency and accuracy of underwater detection and surveillance.

[0004] Although there are underwater target recognition methods based on sonar or images in the existing technology, they have obvious limitations in complex ocean environments. The recognition accuracy of single-modal data drops significantly when there is insufficient lighting, high turbidity or severe acoustic interference. Existing data fusion mostly uses static weights or simple weighting, lacks adaptive adjustment to environmental changes, resulting in unstable results. Feature extraction mostly relies on edge or spectral features, which makes it difficult to reflect the dynamic changes of targets in time series and multimodal environments. In terms of target tracking, there is a lack of support for deep learning algorithms, insufficient discrimination, and it is easy to cause misidentification or loss of targets. Therefore, it is difficult to meet the needs of real-time and accurate detection and tracking of underwater targets. Summary of the Invention

[0005] The purpose of the present invention is to solve the shortcomings of the prior art and propose an underwater target tracking system based on DBN.

[0006] In order to achieve the above object, the present invention adopts the following technical solution: a DBN-based underwater target tracking system, the system comprising: The multi-source data processing module obtains the amplitude value and frequency value in the sonar signal and the pixel brightness value and color saturation value in the visual image, performs preliminary signal-to-noise ratio evaluation, compares with the signal-to-noise ratio benchmark in the current marine environment, judges the validity of the modal data, dynamically adjusts the weight of the acoustic and visual data, and generates a modal data weight coefficient; The environmental monitoring module monitors the illumination intensity, turbidity particle concentration and visibility distance of the underwater area based on the modal data weight coefficient, analyzes the influence degree of the environmental parameters on the sonar and visual data quality, adjusts the data fusion process, and obtains a data fusion strategy ratio; The feature fusion module processes the time series data of the sonar signal according to the data fusion strategy ratio, analyzes the frequency and amplitude change of the short-time data segment, determines the key geometric center distribution difference, and obtains a feature fusion density index; The target feature extraction module calls the feature fusion density index, performs spatial and noise distribution analysis on the target region in the sonar image, calculates the interference ratio and reflection feature of the region, screens the region with obvious edge features, optimizes the target feature recognition degree, and obtains an edge recognition intensity value.

[0007] The present application improves that the modal data weight coefficient includes acoustic data weight and visual data weight, the data fusion strategy ratio includes illumination adaptation ratio, turbidity adaptation ratio and visibility adaptation ratio, the feature fusion density index includes frequency change density, amplitude change density and visual edge density, and the edge recognition intensity value includes spatial interference ratio, noise reflection ratio and edge recognition accuracy.

[0008] The present application improves that the multi-source data processing module comprises: The sonar data extraction submodule obtains the amplitude value and frequency value in the sonar signal, calculates the signal-to-noise ratio under each frequency according to the environmental influence factors and the amplitude and frequency data change range, and obtains a sonar signal-to-noise ratio result; The visual image extraction submodule obtains the pixel brightness value and color saturation value in the visual image, calculates the color stability of each image region according to the different illumination conditions and color distribution, and obtains an image stability result; The data weight calculation submodule assigns weights to the sonar and visual image data based on the sonar signal-to-noise ratio result and the image stability result, and uses the formula: obtains a modal data weight coefficient WS, wherein SNR i represents the sonar signal-to-noise ratio result under the i-th frequency, W snr represents the weight coefficient of the sonar data, STB j represents the image stability result in the j-th region, W stb ​The weight coefficient represents visual image data, n represents the number of frequencies of the sonar signal, and m represents the number of regions of the visual image.

[0009] The application improves that the environment monitoring module comprises: The light monitoring submodule monitors the light intensity of the underwater region based on the modal data weight coefficient, analyzes the light intensity fluctuation within the time sequence according to the data collected by the light sensor, identifies the average light intensity and the standard deviation thereof, and obtains a light intensity analysis result; The particle concentration analysis submodule analyzes the turbidity particle data recorded by the turbidity sensor according to the light intensity analysis result, and adopts the formula: , The adjusted turbidity particle concentration is obtained , and the turbidity data greatly affected by light is optimized, wherein, is the average light intensity, is the turbidity particle concentration of the bth measurement point, is the total number of measurement points, is the maximum value of the turbidity particle concentration during the measurement; The data fusion adjustment submodule integrates the water depth and the water temperature based on the adjusted turbidity particle concentration, analyzes the influence of the environmental parameters, optimizes the fusion process of the sonar and visual data, matches the current underwater environment, and obtains a data fusion strategy ratio.

[0010] The application improves that the feature fusion module comprises: The sonar feature extraction submodule monitors the time sequence data of the sonar signal according to the data fusion strategy ratio, analyzes the short-time changes of the frequency and amplitude in each data segment, calculates the feature fluctuation index of the sonar signal, and obtains a sonar feature fluctuation value; The visual feature analysis submodule collects the edge gradient changes in the visual data, analyzes the gradient value of each image unit, calculates the average of the gradient value, and is associated with the geometric center position of each image unit, determines the geometric center distribution of the image unit, and obtains a visual geometric distribution difference; The fusion density calculation submodule calculates the feature fusion density based on the sonar feature fluctuation value and the visual geometric distribution difference by adopting the formula: , The feature fusion density index DR is obtained, wherein FR u represents the uth sonar feature fluctuation value, GR u represents the uth visual geometric distribution difference value, n dr represents the total sample amount of the data.

[0011] The application improves that the target feature extraction module comprises: The feature density analysis submodule calls the feature fusion density index, extracts the reflection intensity and position information of the pixel points based on the target area data in the sonar image, analyzes the spatial distribution, and marks the difference density partitions to generate density distribution features; The noise distribution calculation submodule calls the density distribution feature, analyzes the noise signal strength, identifies the key areas of noise interference by comparing the ratio of the reflected signal to the noise signal, and calculates the correlation between noise and density to obtain the noise interference ratio; The edge feature optimization submodule selects regions with obvious edge features according to the noise interference ratio, analyzes the edges of the regions, and uses the formula: ; Optimize the target feature recognition and obtain the edge recognition strength value SQ, where GQ p is the gradient intensity value of the p-th pixel, CQ p is the edge curvature, EQ p is the reflection intensity value, RQ p is the reflection characteristic of the area, NQ p is the noise signal strength, n sq is the total number of pixels analyzed.

[0012] The present invention is improved in that the system further comprises: The dynamic prediction and recognition module performs a difference analysis on the node response values ​​output by the DBN model based on the edge recognition strength value, calculates the offset amplitude of the node response, performs dynamic prediction based on the offset amplitude, determines whether the current response meets the target tracking conditions, and obtains the target tracking analysis result; The target tracking analysis results include node response offset, tracking prediction accuracy, and condition compliance.

[0013] The present invention is improved in that the dynamic prediction and recognition module includes: The edge recognition analysis submodule analyzes the node response value output by the DBN model based on the edge recognition strength value, detects the edge recognition strength of the node, identifies the strength difference of the node response value, and obtains edge recognition deviation data; The node response offset calculation submodule adopts the formula based on the edge recognition deviation data: ; Calculate the offset PF of the zth node z, Get the offset amplitude data, where R z,y represents the yth response value of the zth node, E y Represents the preset edge value of the y-th response, n pf Represents the total number of response events, WF k Represents the weight coefficient of the kth node, m pfThe total number of representative nodes is determined according to the following formula: The target tracking judgment submodule judges whether the node meets the target tracking condition based on the offset amplitude data, identifies the node meeting the condition, and obtains a target tracking analysis result.

[0014] Compared with the prior art, the application has the advantages and positive effects that: In the application, the real-time signal-to-noise ratio of the sonar and visual data is evaluated, the modal weight is dynamically adjusted, the data reliability in the case of insufficient light or complex water quality is ensured, the data fusion strategy is optimized in combination with the light intensity, turbidity concentration and visibility distance, the system has good environmental adaptability, the key geometric differences are extracted by using the time sequence characteristics and image edge gradient, the feature recognition distinction is improved, the significant edge features are screened and optimized, the target area is highlighted, the recognition accuracy is effectively improved, the target tracking condition can be judged in real time based on the offset analysis of the DBN node response, the system response speed and tracking accuracy are improved, and the efficiency of underwater detection and the applicability in complex environments are enhanced. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 The system flowchart of the application is shown in the figure; Figure 2 The flowchart of the multi-source data processing module in the application is shown in the figure; Figure 3 The flowchart of the environmental monitoring module in the application is shown in the figure; Figure 4 The flowchart of the feature fusion module in the application is shown in the figure; Figure 5 The flowchart of the target feature extraction module in the application is shown in the figure; Figure 6 The flowchart of the dynamic prediction and recognition module in the application is shown in the figure. DETAILED DESCRIPTION

[0016] In order to make the purpose, technical scheme and advantages of the application clearer, the application will be further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the application and do not limit the application.

[0017] In the description of the application, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the application. In addition, in the description of the application, the meaning of "a plurality of" is two or more, unless otherwise specifically limited. Embodiment

[0018] Please refer to Figure 1 The application provides a technical solution: a DBN-based underwater target tracking system comprises: The multi-source data processing module obtains the amplitude value and frequency value in the sonar signal and the pixel brightness value and color saturation value in the visual image, performs preliminary signal-to-noise ratio evaluation according to the variation range of the data and the environmental influence factors, compares the signal-to-noise ratio with the signal-to-noise ratio benchmark in the current marine environment, judges the effectiveness of the modal data, dynamically adjusts the weights of the acoustic and visual data, and generates a modal data weight coefficient; The environmental monitoring module monitors the illumination intensity, turbidity particle concentration and visibility distance of the underwater area based on the modal data weight coefficient, analyzes the influence degree of the environmental parameters on the quality of the sonar and visual data, adjusts the data fusion process, matches the current underwater environment, and obtains a data fusion strategy ratio; The feature fusion module processes the time series data of the sonar signal according to the data fusion strategy ratio, analyzes the frequency and amplitude variation of the short-time data segment, combines the edge gradient variation of the visual data, determines the key geometric center distribution difference, and obtains a feature fusion density index; The target feature extraction module calls the feature fusion density index, analyzes the spatial and noise distribution of the target area in the sonar image, calculates the interference ratio and reflection feature of the area, selects the area with obvious edge features, optimizes the target feature recognition degree, and obtains an edge recognition intensity value; The dynamic prediction and recognition module performs difference analysis on the node response value output by the DBN model based on the edge recognition intensity value, calculates the offset amplitude of the node response, performs dynamic prediction according to the offset amplitude, judges whether the current response meets the target tracking condition, and obtains a target tracking analysis result.

[0019] The modal data weight coefficient includes acoustic data weight and visual data weight, the data fusion strategy ratio includes illumination adaptation ratio, turbidity adaptation ratio and visibility adaptation ratio, the feature fusion density index includes frequency variation density, amplitude variation density and visual edge density, the edge recognition intensity value includes spatial interference ratio, noise reflection ratio and edge recognition accuracy, and the target tracking analysis result includes node response offset, tracking prediction accuracy and condition compliance degree.

[0020] The current signal-to-noise ratio benchmark in the marine environment refers to the "typical environmental noise level" or "empirical signal-to-noise ratio standard" of sonar or underwater acoustic signals in the actual application area or monitored sea area. Specifically, it is a reference value or threshold for judging and comparing the quality of collected sonar signals. It is set based on historical monitoring of the target sea area and data from marine acoustic standards. It is the "reference signal-to-noise ratio level" of sonar signals at a specific frequency under the condition of no strong interference and no obvious target in the target water area, providing a benchmark for multi-source data effectiveness evaluation and weight adaptive allocation. As a judgment benchmark for "whether the newly collected data is effective and can be used for target identification", for example, if the real-time signal-to-noise ratio is lower than the benchmark, it indicates that the current data quality is poor and the weight needs to be reduced. Compared with the real-time signal-to-noise ratio, it is used to adjust the weight distribution of sonar data and visual data, and improve the adaptive ability of the system to complex environments. It can cope with the fluctuation of background noise caused by seasonal changes, weather changes (wind, waves, rain and snow), and human interference in the marine environment. For example, after long-term monitoring of a certain sea area, the "background noise signal-to-noise ratio benchmark" of this area is 20 dB. If the sonar signal-to-noise ratio of the system at a certain frequency is 15 dB, it is determined to be lower than the benchmark, and the data weight of this frequency is reduced. If it is higher than 20 dB, it means that the current signal quality is excellent, and the weight is increased.

[0021] Please refer to Figure 2 , the multi-source data processing module includes: The sonar data extraction submodule obtains the amplitude value and frequency value of the sonar signal, calculates the signal-to-noise ratio at each frequency according to the environmental impact factors and the amplitude and frequency data variation range, and obtains the sonar signal-to-noise ratio result. In the underwater sensor or sonar detection device, the sonar signal is monitored, and the sonar signal is converted into an electrical signal form through a numerical conversion module. The amplitude value can be extracted by a wave amplitude measuring instrument that uses a sensor to record the fluctuation amplitude of the sonar signal, obtains the difference between the peak and valley values of each waveform, and records the difference as the amplitude value. The frequency value can be extracted by a frequency analysis module that performs a fast Fourier transform (FFT) on the periodic waveform of the sonar signal, extracts the frequency corresponding to the position with the largest amplitude in the FFT result as the main frequency value of the sonar signal, and realizes the extraction of the frequency value. In the identification of environmental impact factors, water temperature, salinity, and sea current speed parameters need to be monitored. The water temperature data is collected by a temperature sensor, the salinity data is measured by a salinometer, and the sea current speed data is monitored by a flowmeter. After converting the parameters into numerical form, they are introduced into the calculation model. According to the amplitude value, frequency value, and environmental impact factors at different frequencies, the signal-to-noise ratio at each frequency is calculated. For the calculation of the signal-to-noise ratio, the amplitude value at each frequency is divided by the corresponding environmental noise amplitude Ni, Taking logarithm again, the calculation formula is: In the formula, SNR i is the signal-to-noise ratio at the i-th frequency, A i is the sonar amplitude value at the i-th frequency, N i is the ambient noise amplitude at the i-th frequency, and the signal-to-noise ratio at each frequency point is taken as a single data point. The signal-to-noise ratio results of all frequency points are calculated and recorded to obtain the sonar signal-to-noise ratio results.

[0022] The visual image extraction submodule obtains the pixel brightness value and color saturation value in the visual image, calculates the color stability of each image region according to the difference in light conditions and color distribution, and obtains the image stability result; For the acquisition process of the visual image, an underwater camera is needed to collect visual data in the water in real time, and the image data is converted into RGB format. In this process, for the extraction of the pixel brightness value, the YUV color space conversion method can be used to extract the Y channel data in the image as the brightness value. By traversing each image pixel point, the Y value of each pixel point is recorded to realize the extraction of the pixel brightness value. For the extraction of the color saturation value, the RGB image can be converted into the HSV color space. In the HSV space, the S channel of the image represents the color saturation value. Using the image data traversal algorithm, the S channel value of each pixel point is extracted to realize the extraction of the pixel color saturation value. After the above data extraction is completed, in order to calculate the color stability of each image region, the image needs to be divided into several regions. According to the horizontal and vertical coordinate axes of the image, the image region is divided into fixed width and height. The standard deviation of the pixel brightness value and the color saturation value in each region is calculated as the color stability of the region. The calculation formula is: In the formula, STB j is the color stability of the j-th region, is the brightness value or color saturation value of the k-th pixel point, is the average value of the brightness value or color saturation value of all pixel points in the region, and p is the total number of pixel points in the region. The color stability of each image region is calculated to obtain the image stability result.

[0023] The data weight calculation submodule assigns weights to the data of the sonar and the visual image based on the sonar signal-to-noise ratio result and the image stability result, using the formula: ; The modal data weight coefficient WS is obtained, wherein SNR i represents the sonar signal-to-noise ratio result at the i-th frequency, W snr represents the weight coefficient of the sonar data, STB j represents the image stability result in the j-th region, and W stbweight coefficient of visual image data, n represents frequency number of sonar signal, m represents area number of visual image; The relative influence of each parameter is analyzed, and a reference coefficient is introduced. The weight parameter is set with reference to the reference value of the signal-to-noise ratio of marine acoustics and the reference value of the stability of marine optical images. The average value of the signal-to-noise ratio of the sonar data at different frequencies is taken as the weight parameter W snr , and the calculation formula is: The weight parameter W stb of the visual image data is taken as the average value of the color stability of each area of the image, and the formula is: In the formula, n is the frequency number of the sonar signal, and m is the area number of the visual image. Then, according to the above weight parameter, the weight coefficient WS of the modal data is calculated. If the frequency number of the sonar signal is 4, the area number of the visual image is 3, the sonar signal-to-noise ratio results are 20, 18, 15 and 22 in turn, and the image stability results are 0.12, 0.15 and 0.10 in turn, the calculation process is as follows: ; ; The numerical result of the weight coefficient of the modal data is 17.3, which indicates that the data fusion weight of the current sonar signal and the visual image is in the upper middle interval, which can be further adjusted according to the specific application scenario.

[0024] Please refer to Figure 3 The environmental monitoring module comprises: The light monitoring sub-module monitors the light intensity of the underwater area based on the weight coefficient of the modal data, analyzes the light intensity fluctuation in the time sequence according to the data collected by the light sensor, identifies the average light intensity and its standard deviation, and obtains the light intensity analysis result; First, a plurality of light sensors are arranged in the underwater area. The sensors are distributed at different depths and horizontal positions to ensure that the overall range of the monitoring area is covered. For example, 5 light sensors are arranged at depths of 3 meters, 5 meters and 7 meters, forming a grid arrangement. The sensors record light intensity values at fixed time intervals. Assuming that the sampling interval is 10 seconds, each sensor can record 6 data points in 1 minute. All data points are arranged in time sequence and labeled with time tags. According to the recorded data, the light intensity curve of each sensor is extracted. For the curve, a sliding window average method is used. For example, with a window width of 30 seconds, the sliding average of each sensor's light data is calculated to smooth the light intensity curve. The extreme points (i.e. the highest and lowest points of light intensity) in the data are filtered out. Based on the exclusion of extreme abnormal points, the average light intensity of each sensor is calculated. According to the average light intensity of all sensors, the average light intensity of the overall area is calculated, and the standard deviation of the light intensity in the area is further calculated. The standard deviation can be calculated by the formula: wherein, is the standard deviation of light intensity, N is the total number of measurement points, x i is the light intensity value of the i-th measurement point, is the mean value of light intensity of the measurement points, assuming that in a certain measurement, the light intensity mean values of the 5 sensors are 120, 110, 115, 118 and 125 lx respectively, the standard deviation of light intensity is calculated to be 5.74 lx, and the light intensity analysis result is obtained.

[0025] The particle concentration analysis submodule analyzes the turbidity particle data recorded by the turbidity sensor based on the light intensity analysis result, and uses the formula: ; The adjusted turbidity particle concentration CH t, The turbidity data with large light influence is optimized, wherein, is the average light intensity, is the turbidity particle concentration of the b-th measurement point, is the total number of measurement points, is the maximum value of turbidity particle concentration during the measurement; The turbidity sensor is arranged in a position similar to the light sensor, records the turbidity particle concentration value with the same time interval, and labels the time tag. According to the turbidity particle data of each sensor, the time period with significant decrease in light intensity is selected as the key analysis target. The turbidity particle concentration data in this time period is analyzed. For the selected data segment, the turbidity particle concentration mean value of each measurement point is extracted. In a certain period, the average light intensity of the 4 sensors is 112, and the corresponding turbidity particle concentration data is 6.5, 7.0, 8.0 and 7.5 mg / L, and the maximum turbidity particle concentration value is 8.0 mg / L. Then, the formula is calculated as: ; ; The adjusted turbidity particle concentration mg / L, the corrected turbidity particle concentration value is obtained, which indicates that after the turbidity particle concentration is adjusted, it can better reflect the actual turbidity degree in the period with large light intensity fluctuation.

[0026] The data fusion adjustment submodule integrates the water depth and water temperature based on the adjusted turbidity particle concentration, analyzes the influence of environmental parameters, optimizes the fusion process of sonar and visual data, matches the current underwater environment, and obtains the data fusion strategy ratio. The water depth and water temperature data are integrated, the water depth data is obtained by an ultrasonic water depth sensor, the sensor is arranged at three measuring points in the water depth direction, and the three measuring points are located at 2 meters, 4 meters and 6 meters below the water surface respectively to record the water depth fluctuation at different depths, the water temperature data is obtained by a temperature sensor, the temperature sensor is arranged at the same three measuring points to record the water temperature fluctuation of each measuring point, on the basis of the obtained water depth and water temperature data, the water temperature and water depth variation coefficients at each sensor position are calculated, the variation coefficient formula is: , wherein CV is the variation coefficient, is the standard deviation of water temperature or water depth, is the mean value of water temperature or water depth, if the mean values of water temperature data at the three measuring points are 12.5℃, 14.0℃ and 15.5℃ respectively, and the standard deviations are 0.8℃, 0.5℃ and 0.6℃ respectively, then the variation coefficients are respectively: , , The data of the measuring point with the smallest variation coefficient is selected as the reference to optimize the fusion process of the sonar and visual data, the corrected turbidity particle concentration value, the water temperature and water depth data of the reference measuring point and the current underwater environment are combined to adjust the weight coefficients of the light, turbidity particle concentration and visibility distance data, and a data fusion strategy ratio is obtained.

[0027] Please refer to Figure 4 The feature fusion module comprises: The sonar feature extraction submodule monitors the time series data of the sonar signal according to the data fusion strategy ratio, analyzes the short-time changes of the frequency and amplitude in each data segment, calculates the characteristic fluctuation index of the sonar signal, and obtains the sonar characteristic fluctuation value; The underwater sonar array device is used for signal monitoring of the target water area. The sonar array is laid in a water area with stable acoustic characteristics, and the sonar sensors are properly spaced to cover the acoustic signals in the monitoring range. For example, 8 sonar sensors are laid in a water area with a diameter of 100 meters, and the working frequency of each sensor is set to a medium-low frequency acoustic wave range of 10-200 Hz. The sonar signal data received by each sensor is continuously monitored at an interval of 1 second. After the data monitoring is completed, the time series data of the sonar signal is divided into multiple short time data segments according to the time axis. The time division interval can be set to 1-5 seconds, and the appropriate interval is selected according to different signal strengths to ensure the effectiveness of the data. The divided data segments contain all the sonar signal data points in the time range. The frequency and amplitude data in each data segment need to be processed respectively. First, the frequency data is processed by point-by-point difference to calculate the change value. For example, the frequency data in a data segment is 12 Hz, 13 Hz, 14 Hz, 15 Hz and 16 Hz. The frequency change of each adjacent data point is calculated, and the change is 1 Hz, 1 Hz, 1 Hz and 1 Hz. Further, the average value of the frequency change in the data segment is taken as the frequency fluctuation index of the data segment. The calculation result of the above example data is 1 Hz. The amplitude data is processed in a similar manner. The amplitude change is calculated by point-by-point difference. For example, the amplitude data in a data segment is 3 dB, 3.5 dB, 4 dB, 4.5 dB and 5 dB. The amplitude change of adjacent data points is 0.5 dB, 0.5 dB, 0.5 dB and 0.5 dB. Further, the average value of the amplitude change in the data segment is calculated to obtain the amplitude fluctuation index of the data segment. Then, the frequency fluctuation index and the amplitude fluctuation index of each short time data segment are summarized to form the characteristic fluctuation index of the sonar signal as a whole. Further, the characteristic fluctuation index is summarized according to the interval to obtain the characteristic fluctuation trend of the sonar signal as a whole, and the sonar characteristic fluctuation value is obtained.

[0028] The visual feature analysis submodule collects the edge gradient change in the visual data, analyzes the gradient value of each image unit, calculates the average of the gradient value, and associates it with the geometric center position of each image unit to determine the geometric center distribution of the image unit and obtain the visual geometric distribution difference. The image data collected needs to be preprocessed, including image graying and noise reduction processing. The graying can be achieved by converting the original color image into a gray image to eliminate color interference. The noise reduction processing can be achieved by mean filtering or median filtering to remove random noise points in the image to improve the image quality. After the image preprocessing is completed, the edge gradient change in the image data is analyzed. Each image unit (such as a 3*3 pixel area) is taken as an analysis unit, and the gradient value in each unit is calculated. The gradient value can be calculated by using the Sobel operator method. The gradient values in the horizontal direction and the vertical direction of the image unit are calculated respectively, and the square root of the sum of squares of the two is taken as the gradient value of each image unit. For example, the gradient value of a certain image unit in the horizontal direction is 8, and the gradient value in the vertical direction is 6. The total gradient value of the image unit is calculated as follows: Next, the gradient mean value of each image unit is calculated, and the gradient mean value of each image unit is associated with its geometric center position to obtain the geometric center distribution feature. The geometric center position can be determined by the image coordinate system. Assuming that the image size is 640*480 pixels, the coordinates of the center position of the image are (320, 240). The center coordinates of each image unit are calculated by the row and column numbers of its position. For example, a certain image unit is located at the 150th row and the 200th column, and the center coordinates are (200, 150). After obtaining the geometric center position and the gradient mean value of each image unit, the offset degree of the geometric center position is further calculated. Specifically, the difference between the center coordinates of each image unit and the center coordinates of the whole image is calculated to reflect the geometric center deviation of the unit. The geometric center deviation of each image unit is averaged by region to obtain the visual geometric distribution difference.

[0029] The fusion density calculation sub-module calculates the feature fusion density based on the sonar feature fluctuation value and the visual geometric distribution difference by using the formula: The feature fusion density index DR is obtained, wherein FR u represents the u-th sonar feature fluctuation value, GR u represents the u-th visual geometric distribution difference value, n dr represents the total sample size of the data. If the sonar feature fluctuation values are 1.2, 1.5, 1.3 and 1.4, and the visual geometric distribution difference values are 1.1, 1.6, 1.2 and 1.3, the calculation is as follows: ​​The result shows that the feature fusion density index is small, indicating that the feature fluctuation of the sonar signal and the visual geometric distribution are small, and further it can be judged that the feature correlation of the two is high, and there is strong correlation between the data, and the feature fusion density index is obtained.

[0030] Please refer to Figure 5 , the target feature extraction module comprises: The feature density analysis submodule calls the feature fusion density index, extracts the reflection intensity and position information of the pixel points according to the target region data in the sonar image, analyzes the spatial distribution, marks the difference density partition, and generates the density distribution feature; According to the target region data in the sonar image, first, the target region data in the sonar image is obtained, and the data acquisition of the sonar image can be based on the change of the pulse echo signal intensity, and the position coordinates of each point in the sonar data are determined through the angle and position coordinates of the scanning device. Each coordinate point should contain its reflection signal intensity value as a key parameter. Assuming that the coordinate system of the sonar image is defined by X-Y axis, the coordinates of each pixel point can be expressed as , wherein represents the i-th pixel point, and the reflection signal intensity is , in the case of known sonar image resolution of 300x300 pixels, after obtaining the data of each point, the reflection intensity value of each point and the spatial distribution density between adjacent points are calculated. The density index calculation can be completed by using the local point reflection intensity value mean value method. For each pixel point, a 5x5 neighborhood window (i.e. 25 pixel points including the current point) is set, and the reflection signal intensity mean value of each point in the neighborhood window is calculated , and the ratio of the reflection signal intensity value of each point to the reflection signal intensity mean value in the neighborhood is taken as the density index of the point, that is, the density index can be calculated as: In this calculation, if the density index of a point is higher than the set threshold, the point is identified as a high-density point, otherwise it is a low-density point. After calculating each point in the sonar image in this way, the adjacent high-density points are clustered into a density interval using clustering method. Assuming that the clustering standard is that the Euclidean distance between points does not exceed 2 pixel points, the adjacent high-density point set will be classified as a group of density partitions. Further, the average density index value of each density partition region is taken as the feature density index of the partition. Assuming that the density partition region is the k-th partition, the feature density index of the partition can be expressed as: , wherein, n k is the number of pixel points in the density partition, and the density distribution index of each partition is obtained as the density distribution feature.

[0031] The noise distribution calculation submodule calls the density distribution feature, analyzes the noise signal intensity, identifies the key area of noise interference by comparing the ratio of the reflected signal and the noise signal, and calculates the correlation between the noise and the density to obtain the noise interference ratio; Based on the density distribution feature, the noise signal intensity in each density partition area is obtained. The determination of the noise signal intensity can be completed by a frequency spectrum energy analysis method. For each density partition, the total energy of the frequency spectrum signal of each pixel point in the partition is taken as the noise signal intensity. The specific calculation method is as follows: , wherein N k is the total noise signal intensity in the kth partition, is the frequency spectrum energy value of each pixel point. Further, in the calculation process, the ratio of the total reflected signal energy Q of each density partition to the total noise signal energy N is taken as the noise interference ratio, and the calculation method is as follows: , wherein Q k is the noise interference ratio of the density partition. Further, the correlation between the noise signal and the characteristic density of the density partition is calculated. The total reflected signal energy is taken as the correlation reference. The distribution proportion of the noise signal intensity in each density partition area is calculated. It is judged whether there is a significant noise interference area in the area. If the noise interference ratio of a certain density partition is less than 0.5 (in experimental tests, the ratio of the sonar image signal reflection intensity to the noise signal intensity is usually about 0.5 as the noise interference reference value), it is considered that there is obvious noise interference in the area. In this way, the key area of noise interference is identified, and the noise interference ratio is obtained.

[0032] The edge feature optimization submodule selects the area with obvious edge features according to the noise interference ratio, analyzes the area edge, and adopts the formula: ; the target feature recognition degree is optimized to obtain the edge recognition intensity value SQ. The area with a value higher than the threshold value is selected as the target feature, wherein GQ p is the gradient intensity value of the pth pixel point, CQ p is the edge curvature, EQ p is the reflection intensity value, RQ P is the reflection feature of the area, NQ P is the noise signal intensity, n sq is the total number of analyzed pixel points; If the area contains 2 pixel points, the parameters take the following values: 1st pixel point: , , , , ; 2nd pixel point: , , , , ; Substitute the above parameters into the formula and calculate item by item: ; ; The result shows that the edge recognition strength value is about 0.829. The area with a value greater than the set threshold (assuming the threshold is 0.5) is further screened as the target feature to obtain the edge recognition strength value.

[0033] See also Figure 6 , the dynamic prediction and recognition module includes: The edge recognition analysis submodule analyzes the node response values ​​output by the DBN model based on the edge recognition strength value, detects the edge recognition strength of the node, identifies the strength difference of the node response value, and obtains the edge recognition deviation data; Obtain the response value of each node output by the DBN model at different time points, extract the edge recognition strength value of each node, and when obtaining the edge recognition strength value, use the historical response data of each node as a reference to calculate the response amplitude change of each node at different time points. Use the maximum and minimum values ​​of the response amplitude change to determine the response strength interval value of the node, and further divide the strength threshold range of the node according to the upper and lower limits of the interval value. For example, if the historical response data of node 1 fluctuates in the interval of [0.2, 0.8], then the upper limit 0.8 of the interval is used as the upper limit of the strength threshold, and the lower limit 0.2 is used as the lower limit of the strength threshold. Then, compare the current response value of each node with the corresponding strength threshold, and mark the response value that exceeds the upper threshold or is lower than the lower threshold as a deviation point. Record the number of deviation points and the deviation amplitude of each deviation point for each node at different time points. Use the amplitude value of each deviation point as the edge recognition deviation value, and summarize the values ​​to form the edge recognition deviation data of the node.

[0034] The node response offset calculation submodule is based on edge recognition deviation data and uses the formula: ; Calculate the offset PF of the zth node z , get the offset amplitude data, where R z,y Represents the yth response value of the zth node, which is used to identify the activity level of the node at the target time point. The preset edge value representing the yth response is a benchmark value used to evaluate whether the node response meets a specific edge recognition standard. Represents the total number of response events, represents the weight coefficient of the kth node, total number of representative nodes; If the response value of the first node is [0.25, 0.55, 0.45], the preset edge value is [0.30, 0.50, 0.40], the total number of response events is 3, the total number of nodes is 4, and the weight value of each node is [0.2, 0.3, 0.25, 0.25], the offset amplitude is calculated as follows: ; ; The calculation result of the offset amplitude value of the first node is 0.0075, which indicates that the response value of the node at each time point deviates from the preset edge value by a small amplitude, and the node is stable in fluctuation. The response data is closer to the set reference range, and the result is further included in the node response offset amplitude data.

[0035] The target tracking judgment sub-module judges whether the node meets the target tracking condition based on the offset amplitude data, identifies the nodes meeting the condition, and obtains the target tracking analysis result; The offset amplitude value of each node is extracted, the target tracking threshold range is set, and the offset amplitude values of the nodes are further compared with the target tracking threshold to determine whether each node meets the target tracking condition. When setting the target tracking threshold range, the median of the node offset amplitude value is taken as the reference, and the upper quartile and the lower quartile are combined to calculate the target tracking threshold interval. For example, in the sample data, the node offset amplitude values are [0.0075, 0.012, 0.015, 0.020], the median is 0.0135, the upper quartile is 0.018, and the lower quartile is 0.010. Nodes lower than the lower quartile or higher than the upper quartile are marked as offset abnormal nodes. Further, according to the distribution of the abnormal nodes, the nodes meeting or not meeting the target tracking condition are marked, and the target tracking analysis result is obtained.

[0036] The above is only a preferred embodiment of the present application, and does not limit the form of the present application. Any skilled person in the art can modify or change the above disclosed technology content to equivalent embodiments applied to other fields, but any simple modification, equivalent change and modification of the above embodiments within the technical solution content of the present application, according to the technical essence of the present application, still belongs to the protection scope of the technical solution of the present application.

Claims

1. A DBN-based underwater target tracking system, characterized in that, The system comprises: The multi-source data processing module acquires the amplitude value and frequency value in the sonar signal and the pixel brightness value and color saturation value in the visual image, performs preliminary signal-to-noise ratio evaluation, compares with the signal-to-noise ratio benchmark in the current marine environment, judges the validity of the modal data, dynamically adjusts the weight of the acoustic and visual data, and generates a modal data weight coefficient; The environmental monitoring module monitors the underwater area according to the modal data weight coefficient, the light intensity, the turbidity particle concentration and the visibility, analyzes the influence degree of the environmental parameters on the sonar and visual data quality, adjusts the data fusion process, and obtains a data fusion strategy ratio; The feature fusion module processes the time series data of the sonar signal according to the data fusion strategy ratio, analyzes the frequency and amplitude changes of the short-time data segment, determines the key geometric center distribution difference, and obtains a feature fusion density index; The target feature extraction module calls the feature fusion density index, performs spatial and noise distribution analysis on the target area in the sonar image, calculates the interference ratio and reflection feature of the area, filters the area with obvious edge features, optimizes the target feature recognition degree, and obtains an edge recognition intensity value; The dynamic prediction and recognition module performs difference analysis on the node response value output by the DBN model based on the edge recognition intensity value, calculates the offset amplitude of the node response, dynamically predicts according to the offset amplitude, judges whether the current response meets the target tracking condition, and obtains a target tracking analysis result. The target tracking analysis result includes the node response offset, the tracking prediction accuracy and the condition compliance degree.

2. The DBN-based underwater target tracking system of claim 1, wherein, The modal data weight coefficient includes the acoustic data weight and the visual data weight, the data fusion strategy ratio includes the light adaptation ratio, the turbidity adaptation ratio and the visibility adaptation ratio, the feature fusion density index includes the frequency change density, the amplitude change density and the visual edge density, and the edge recognition intensity value includes the spatial interference ratio, the noise reflection ratio and the edge recognition accuracy.

3. The DBN-based underwater target tracking system of claim 1, wherein, The multi-source data processing module comprises: The sonar data extraction submodule acquires the amplitude value and frequency value in the sonar signal, calculates the signal-to-noise ratio at each frequency according to the environmental influence factors and the amplitude and frequency data change range, and obtains a sonar signal-to-noise ratio result; The visual image extraction submodule acquires the pixel brightness value and color saturation value in the visual image, calculates the color stability of each image area according to the different light conditions and color distribution, and obtains an image stability result; The data weight calculation sub-module assigns weights to the data of the sonar and the visual image based on the sonar signal-to-noise ratio result and the image stability result, and adopts the formula: ; obtaining a modal data weight coefficient WS, wherein SNR1 represents a sonar signal-to-noise ratio result at the i th frequency, W snr a weight coefficient representing sonar data, STB j a weight coefficient representing sonar data, STB stb a weight coefficient representing sonar data, STB a weight coefficient representing sonar data, STB 4. The DBN-based underwater target tracking system of claim 1, wherein, The environmental monitoring module comprises: The light monitoring submodule monitors the light intensity of the underwater area based on the modal data weight coefficient, analyzes the light intensity fluctuation in the time series according to the data collected by the light sensor, identifies the average light intensity and its standard deviation, and obtains a light intensity analysis result; The particle concentration analysis sub-module analyzes the turbidity particle data recorded by the turbidity sensor according to the light intensity analysis result, and adopts the formula: ; The adjusted haze particle concentration CH t, Optimizing the haze data that is greatly affected by the light, wherein LH avg is the average light intensity, PH b is the haze particle concentration of the bth measurement point, n ch is the total number of measurement points, is the maximum value of the haze particle concentration during the measurement; The data fusion adjustment submodule integrates the water depth and water temperature based on the adjusted turbidity particle concentration, analyzes the influence of the environmental parameters, optimizes the fusion process of the sonar and visual data, matches the current underwater environment, and obtains a data fusion strategy ratio.

5. The DBN-based underwater target tracking system of claim 1, wherein, The feature fusion module comprises: The sonar feature extraction submodule monitors time series data of the sonar signal according to the data fusion strategy, analyzes short-time changes of the frequency and amplitude in each data segment, calculates a characteristic fluctuation index of the sonar signal, and obtains a sonar characteristic fluctuation value; The visual feature analysis submodule collects edge gradient changes in the visual data, analyzes gradient values of each image unit, calculates an average of the gradient values, associates the average with a geometric center position of each image unit, determines a geometric center distribution of the image unit, and obtains a visual geometric distribution difference; The fusion density calculation submodule adopts a formula based on the sonar characteristic wave value and the visual geometric distribution difference: ; The feature fusion density is calculated to obtain a feature fusion density index DR, wherein FR u CR represents the u-th sonar feature fluctuation value u n represents the u-th visual geometry distribution difference value dr N represents the total sample size of the data.

6. The DBN-based underwater target tracking system of claim 1, wherein, The target feature extraction module includes: A feature density analysis submodule calls the feature fusion density index, extracts reflection intensity and position information of a pixel point according to target region data in the sonar image, analyzes spatial distribution, marks a difference density partition, and generates a density distribution feature; A noise distribution calculation submodule calls the density distribution feature, analyzes noise signal intensity, identifies a key region of noise interference by comparing a ratio of the reflection signal and the noise signal, calculates a correlation between the noise and the density, and obtains a noise interference ratio; The edge feature optimization submodule screens the region with obvious edge features according to the noise interference ratio, analyzes the region edge, and adopts the formula: ; Optimize the target feature recognition and obtain the edge recognition strength value SQ, where GQ p is the gradient intensity value of the p-th pixel, CQ p is the edge curvature, EQ p is the reflection intensity value, RQ p is the reflection characteristic of the area, NQ p is the noise signal strength, n sq is the total number of pixels analyzed.

7. The DBN-based underwater target tracking system of claim 1, wherein, The dynamic prediction recognition module includes: An edge recognition analysis submodule analyzes node response values output by the DBN model based on the edge recognition intensity value, detects edge recognition intensity of the node, recognizes intensity differences of the node response values, and obtains edge recognition deviation data; A node response offset calculation submodule calculates an offset of the node response value based on the edge recognition deviation data, and uses a formula: ； calculating the offset amplitude PF of the zth node z, obtaining offset amplitude data, wherein R z,y representing the yth response value of the zth node, E y representing the preset edge value of the yth response, n pf representing the total number of response events, WF k representing the weight coefficient of the kth node, m pf representing the total number of nodes; A target tracking judgment submodule judges whether the node meets a target tracking condition based on the offset amplitude data, identifies the node meeting the condition, and obtains a target tracking analysis result.

Images