Underwater robot target detection method based on multi-sensor information fusion

Abstract

The invention relates to an underwater robot target detection method based on multi-sensor information fusion, and the method comprises the steps: obtaining the sonar echo, visual image, surrounding water temperature and chemical information of a target through the sonar, visual, temperature and chemical sensors of an underwater robot, adding a timestamp to each piece of information, aligning the information to a unified time axis through an interpolation method, and carrying out the detection of the target. Extracting sensor feature information of the target, fusing the sensor information based on a credibility model, adjusting the credibility of each sensor in real time, performing similarity matching on the fused feature information and a target feature library to identify a potential target object, and within a confirmed target potential range, identifying the target in the target potential range. A probability density function detection algorithm is adopted to judge the existence of a target, temperature and chemical sensor feature information are combined, and a support vector machine algorithm is used to determine the type of the target, so that the problems that the underwater target is relatively high in similarity with the surrounding environment during detection, the features are not obvious enough and accurate detection is difficult are solved.

Description

Technical Field

[0001] The present application relates to the technical field of underwater target detection, and in particular to an underwater robot target detection method based on multi-sensor information fusion. Background Art

[0002] With the continuous deepening of marine ecological research, underwater robots are being used more and more widely in underwater operations. However, due to the complexity and uncertainty of the underwater environment, a single sensor is often difficult to accurately and comprehensively obtain target information, resulting in the accuracy and reliability of target detection being limited.

[0003] In underwater target detection, for some targets whose shapes, sizes or materials are highly similar to the surrounding environment, their salient features are not obvious enough and are difficult to be accurately detected. In this case, detecting the target through a single sensor information may result in detection errors or even failure to detect.

[0004] The technical solution proposed in this application to solve the above-mentioned shortcomings of the existing technology is to obtain the sonar echo of the target, the visual image of the target, the water temperature and chemical information around the target through the sonar sensor, visual sensor, temperature sensor and chemical sensor of the underwater robot, and add a timestamp to each collected information, align it to a unified time axis using the interpolation method, extract features of the target information collected by the sensor, use a trust model to fuse the multi-sensor information of the target, and adjust the trust of each sensor in real time, match the fused feature information with the target feature library for similarity to identify potential target objects, and use a probability density function detection algorithm to determine whether the target exists within the potential range of the determined target. If the target exists, the support vector machine algorithm is used to determine the type of target in combination with the feature information of the temperature sensor and the chemical sensor. This solves the problem that when a single sensor is used to detect an underwater target, the similarity between the underwater target and the surrounding environment is high, resulting in its features not being obvious enough and difficult to detect accurately. Summary of the Invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the purpose of this application is to provide an underwater robot target detection method based on multi-sensor information fusion.

[0006] The present application discloses an underwater robot target detection method based on multi-sensor information fusion, comprising:

[0007] S1, using the underwater robot's sonar sensor, visual sensor, temperature sensor, and chemical sensor to obtain the target's sonar echo, the target's visual image, and the water temperature and chemical information around the target;

[0008] S2. Add a timestamp to the information collected by each sensor, align it to a unified time axis using interpolation, and filter and normalize each sensor information;

[0009] S3, extracting target feature information by collecting target sonar echoes, target visual images, and water temperature and chemical information around the target;

[0010] S4. Use the trust model to fuse the multi-sensor information of the target, dynamically adjust the trust of each sensor, establish a target feature library, and perform similarity matching between the fused feature information and the target feature library. When the similarity match exceeds a threshold, the target is identified as a potential target object and the potential range of the target is confirmed.

[0011] S5. Within the potential range of the target, a probability density function detection algorithm is used. According to the prior probability distribution of the target in the marine ecosystem, a detection threshold is set. When the probability density of the target exceeds the threshold, the target is judged to exist. Combined with the characteristic information of the temperature sensor and the chemical sensor, the support vector machine algorithm is used to determine the type of the target.

[0012] Preferably, step S1 specifically includes deploying an underwater robot in an underwater detection area, wherein the underwater robot's sonar sensor calculates the distance, direction, and size of the target relative to the underwater robot based on the time difference and frequency change information between the emission and reception of its reflected echo signal; the visual sensor uses a high-definition optical camera to acquire images underwater, wherein light is focused on the image sensor through the lens, and the image sensor converts the optical signal into an electrical signal, which is then converted into a digital image through analog-to-digital conversion to obtain visual image information of the target; the temperature sensor uses a thermistor sensor to measure the water temperature around the target; the electrochemical sensor detects the dissolved oxygen content in the water; when oxygen molecules undergo a reduction reaction on the electrode surface to generate a current signal, the concentration of the dissolved oxygen is determined by measuring the magnitude of the current; the pH sensor uses a pH sensor to measure the pH value of the water body; and before the chemical sensor is actually used, an accurate standard curve or measurement model is established based on the composition and concentration range of the chemical substance, and the chemical sensor is calibrated and installed. In the underwater target area, the chemical sensor collects and analyzes water samples, and calculates the concentration of the chemical substance by comparison with the standard curve or based on a pre-established measurement model.

[0013] Preferably, step S2 specifically includes recording the time information when each sensor collects information, and recording and saving the information collected by each sensor and the corresponding time information together. When aligning the collected information to a unified time axis, the time axis of the sonar sensor collecting information is selected as the reference time axis, and the information collected by the visual sensor, temperature sensor and chemical sensor is unified to the reference time axis through interpolation.

[0014] When unifying the information onto the reference time axis through interpolation, each sensor information is filtered using the median filter method. The filtering process includes defining the window length of each information sequence and selecting the middle value in each window after sorting in ascending or descending order as the filtered value.

[0015] Preferably, in step S3, the extracting characteristic information of the target includes extracting the target's echo signal strength, distance, azimuth and spectral characteristics through the target's sonar echo, extracting the target's shape, color and texture characteristics through the target's visual image, extracting the average temperature and temperature gradient around the target through the water temperature around the target, and extracting the dissolved oxygen concentration, pH value and chemical substance concentration around the target through chemical information.

[0016] Preferably, in step S4, the multi-sensor information of the target is fused using the trust model, including fusing characteristic information of the sonar sensor, the visual image of the target, and the water temperature and chemistry around the target, and dynamically adjusting the trust of each sensor includes defining an initial trust value for each sensor and adjusting the trust of the sensor according to underwater environmental factors.

[0017] The initial trust of each sensor is defined as follows: T for sonar sensor sonar0 , the visual sensor is T vision0 , the temperature sensor is T temp0 , the chemical sensor is T chem0 , and T sonar0 >T vision0 >T temp0 >T chem0 .

[0018] Dynamically adjusting the trust of the sonar sensor includes linearly adjusting the trust of the sonar sensor according to the water temperature and salinity in the underwater environment. The adjusted trust of the sonar sensor T sonar1 The calculation formula is:

[0019] T sonar1 =T sonar0 -ω T ×f T (ΔT)-ω S ×f S (ΔS)

[0020] Among them, T sonar1 represents the adjusted sonar sensor trust, T sonar0 represents the initial trust of the sonar sensor, f T and f S Represent the functions of water temperature change and salinity change, ΔT and ΔS represent the amount of water temperature change and salinity change, respectively. T and ωS Represent the weights of water temperature change and salinity change respectively.

[0021] Dynamically adjusting the trust of the visual sensor includes using fuzzy logic to adjust the trust of the visual sensor. The underwater light intensity is divided into strong light area, medium light area and weak light area. If the light intensity is reduced by one level, the trust of the visual sensor will be adjusted from the initial trust level T vision0 The trust level of the visual sensor is lowered by one level based on the initial trust level T and the water turbidity is divided into heavy turbidity, moderate turbidity and light turbidity. If the water turbidity rises by one level, the trust level of the visual sensor is lowered by one level based on the initial trust level T vision0 On the basis of the lower half of the level.

[0022] Dynamically adjusting the trust of the temperature sensor includes adjusting the trust of the temperature sensor using a linear weighted method based on the turbulence of the water flow and the thermal stratification index. The adjusted trust of the temperature sensor is T temp1 The calculation formula is:

[0023] T temp1 =T temp0 -ω V ×f V (V)-ω H ×f H (H)

[0024] Among them, T temp1 Represents the adjusted temperature sensor trust, T temp0 represents the initial trust of the temperature sensor, f V and f H V and H represent the functions of water turbulence and thermal stratification index, respectively. T and ω S They represent the weights of water turbulence and thermal stratification index respectively.

[0025] Dynamically adjusting the trust of chemical sensors includes adjusting the trust of chemical sensors based on the turbulence of water flow and the salinity of water body using fuzzy logic method. The turbulence of water flow is divided into low turbulence, medium turbulence and high turbulence. If the turbulence of water flow increases by one level, the trust of chemical sensors increases from the initial trust level T chem0 Based on the initial trust level T, the water salinity is divided into low salinity, medium salinity and high salinity. If the water salinity rises by one level, the trust level of the chemical sensor is chem0 Downgrade one level based on the

[0026] After dynamically adjusting the trust level of each sensor, a trust level interval is defined for each sensor. If the trust level of a sensor does not meet the trust level interval, the sensor information that does not meet the trust level interval is removed during multi-sensor information fusion for that period of time. The sensor information that meets the trust level interval is constructed as a joint feature vector to perform multi-sensor information fusion.

[0027] If the trust level of the sensor meets its trust level range, the multi-sensor information is fused by constructing a joint feature vector from the feature vectors of the sensor information. The joint feature vector construction includes constructing the echo spectrum feature vector of the sonar sensor. Texture feature vector of visual image Gradient eigenvector of temperature and chemical concentration feature vector The order of each sub-feature vector is determined in the order of sonar-vision-temperature-chemistry, and a joint feature vector is constructed.

[0028] Collect characteristic information of various targets in advance, including sonar echo spectrum, visual image texture, temperature gradient and characteristic vectors of chemical concentration, and build target feature library and characteristic vectors And build a matching model based on cosine similarity, and transform the joint feature vector of sensor information into Input the feature vector of its matching model and the target feature information library Perform similarity matching and set the similarity matching threshold. When the similarity matching value exceeds the threshold, the target is identified as a potential target object. Otherwise, the feature vector of the target is and the feature vector in the target feature library Rematch until a potential target object with a similarity matching value exceeding the threshold is found.

[0029] Confirming the potential range of the target includes taking each sensor as the center and combining the information of sonar sensor, visual sensor, temperature sensor and chemical sensor to determine the potential range of the target as R sonar 、R vision 、R temp and R chem Based on the spatial intersection method, the final target potential range is confirmed to be R = R sonar ∩R vision ∩R temp ∩R chem .

[0030] Preferably, in step S5, the determination of target existence includes: using a probability density function detection algorithm, determining the prior probability distribution of the target based on historical data information on the frequency, location, temperature and chemical concentration of the target in historical marine ecological research, using a multivariate normal distribution model to construct a prior probability distribution model of the target, and setting a detection threshold T within the potential range of the target. d , the vector consisting of the target position coordinates (x, y, z), temperature and chemical concentration collected in real time Bring the target's prior probability density distribution model into the target to calculate the target's probability density. When calculating the target's probability density , the target is judged to exist; otherwise, the target is judged not to exist.

[0031] After determining the existence of the target, the temperature sensor information features and chemical sensor information features are combined to assist in determining the type of the target, including constructing the feature vector [w avg , DO, pH], w avg is the average water temperature around the target, DO is the dissolved oxygen content around the target, pH is the acidity and alkalinity around the target, and the real-time feature vector [w avg , DO, pH] is input into the trained support vector machine algorithm and the determined target species is output.

[0032] The advantage of the underwater robot target detection method based on multi-sensor information fusion described in the present application is that the sonar echo of the target, the visual image of the target, the water temperature and chemical information around the target are respectively obtained through the sonar sensor, visual sensor, temperature sensor and chemical sensor of the underwater robot, and a timestamp is added to the information collected by each sensor. The sensor information is aligned to a unified reference time axis using the interpolation method, and each sensor information is filtered and normalized. Then, the features of the sonar echo of the target, the visual image of the target, the water temperature and chemical information around the target are extracted, and the multi-sensor information is fused using a trust model. According to the real-time underwater environment, the trust of each sensor is dynamically adjusted before its information is fused. The system combines the limitations of a single sensor and improves the accuracy of target detection in underwater environments. By establishing a target feature library and performing similarity matching calculation on the feature information of multi-sensor information fusion and the target feature library, when the similarity match exceeds the preset threshold, it is identified as a potential target object and the potential range of the target is confirmed. The probability density function detection algorithm is used to judge whether the target exists within the potential range of the target based on the prior probability density distribution of the target in the marine ecology. After determining that the target exists, the support vector machine algorithm is used to determine the type of target in combination with the feature information of the temperature sensor and the chemical sensor. This solves the problem that when a single sensor is used to detect an underwater target, the similarity between the target and the surrounding environment is high, resulting in its features not being obvious enough and difficult to detect accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of the structure of an underwater robot target detection method based on multi-sensor information fusion described in this application;

[0034] Figure 2 This is a schematic flow chart of the steps of an underwater robot target detection method based on multi-sensor information fusion described in this application. DETAILED DESCRIPTION

[0035] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are merely used to explain the present invention and are not intended to limit the present invention.

[0036] The preferred implementation methods of the present invention are described below with reference to the accompanying drawings. Those skilled in the art should understand that these implementation methods are only used to explain the technical principles of the present invention and are not intended to limit the scope of protection of the present invention.

[0037] like Figure 1-Figure 2 As shown, the underwater robot target detection method based on multi-sensor information fusion described in this application includes the following steps:

[0038] S1, using the underwater robot's sonar sensor, visual sensor, temperature sensor, and chemical sensor to obtain the target's sonar echo, the target's visual image, and the water temperature and chemical information around the target;

[0039] S2. Add a timestamp to the information collected by each sensor, align it to a unified time axis using interpolation, and filter and normalize each sensor information;

[0040] S3, extracting target feature information by collecting target sonar echoes, target visual images, and water temperature and chemical information around the target;

[0041] S4. Use the trust model to fuse the multi-sensor information of the target, dynamically adjust the trust of each sensor, establish a target feature library, and perform similarity matching between the fused feature information and the target feature library. When the similarity match exceeds a threshold, the target is identified as a potential target object and the potential range of the target is confirmed.

[0042] S5. Within the potential range of the target, a probability density function detection algorithm is used. According to the prior probability distribution of the target in the marine ecosystem, a detection threshold is set. When the probability density of the target exceeds the threshold, the target is judged to exist. Combined with the characteristic information of the temperature sensor and the chemical sensor, the support vector machine algorithm is used to determine the type of the target.

[0043] like Figure 1-Figure 2 As shown, in step S1, the sonar sensor, visual sensor, temperature sensor and chemical sensor of the underwater robot are used to obtain the sonar echo of the target, the visual image of the target, the water temperature and chemical information around the target respectively;

[0044] Specifically, an underwater robot is deployed in the underwater detection area. The underwater robot's sonar sensor calculates the target's distance, direction, and size relative to the underwater robot based on the time difference and frequency change between transmitting and receiving its reflected echo signal. The visual sensor uses a high-definition optical camera to capture images underwater. Light passes through the lens and is focused on the image sensor, which converts the optical signal into an electrical signal. This signal is then converted into a digital image through analog-to-digital conversion, thereby acquiring visual image information of the target. The temperature sensor uses a thermistor sensor to measure the water temperature around the target. The electrochemical sensor detects the dissolved oxygen content in the water. When oxygen molecules undergo a reduction reaction on the electrode surface, a current signal is generated, and the concentration of the dissolved oxygen is determined by measuring the current. The pH sensor measures the pH value of the water. Before the chemical sensor is actually used, an accurate standard curve or measurement model is established based on the composition and concentration range of the chemical substance, and the chemical sensor is calibrated and installed. In the underwater target area, the chemical sensor collects and analyzes water samples, and calculates the concentration of the chemical substance by comparing it with the standard curve or using a pre-established measurement model.

[0045] like Figure 1-Figure 2 As shown, in step S2, a timestamp is added to the information collected by each sensor, the information is aligned to a unified time axis using interpolation, and each sensor information is filtered and normalized;

[0046] Specifically, when each sensor collects information, the time information of the collection is recorded, and the information collected by each sensor and the corresponding time information are recorded and saved together. When aligning the collected information to a unified time axis, the time axis of the sonar sensor collecting information is selected as the reference time axis, and the information collected by the visual sensor, temperature sensor and chemical sensor is unified to the reference time axis through interpolation.

[0047] When unifying the information onto the reference time axis through interpolation, the median filtering method is used to filter each sensor information. Specifically, the window length of each information sequence is defined as 5 information points. In each window, the middle value after ascending or descending sorting is selected as the filtered value.

[0048] The sensor information normalization process is to calculate the z-score value of the sensor information parameter. The z-score value is the difference between the sensor information and the average value divided by the standard deviation. The formula for calculating the normalized z-score is:

[0049]

[0050] Among them, x i is the original value of the sensor information parameter, μ is the mean value of the sensor information parameter, and σ is the standard deviation of the sensor information parameter.

[0051] like Figure 1-Figure 2 As shown, in step S3, characteristic information of the target is extracted respectively by collecting the target sonar echo, target visual image, water temperature and chemical information around the target;

[0052] Specifically, extracting the sonar echo information characteristics of the target includes obtaining the target's echo signal strength, distance, and azimuth, measuring the echo signal strength of the target sound wave, and calculating the target's distance by measuring the time delay between the transmitted sound wave and the echo, combined with the propagation speed of the sound wave in the water. The target's azimuth is calculated by measuring the echo intensity and time delay of different beams, and using the spectrum analysis method to obtain the spectral characteristics of the echo.

[0053] Image processing technology is used to extract the visual image information features of the target, and the shape, color and texture of the target in the image information are obtained. For targets with shapes and colors similar to the surrounding environment, the extraction of texture features is strengthened, including the use of wavelet transform analysis algorithms to extract the differences between the target surface texture and the surrounding similar environment, and obtain the contrast, entropy and angular second-order moment of the visual image.

[0054] The features of the temperature sensor information are extracted by extracting the average temperature and temperature gradient around the target, wherein the temperature gradient is obtained by taking the target as the reference origin to obtain the temperature gradient of the target in the x, y and z directions.

[0055] The information features of the chemical sensor are extracted, including the dissolved oxygen concentration and pH value around the target, and the spatial distribution information of the chemical concentration around the target is obtained.

[0056] like Figure 1-Figure 2 As shown, in step S4, the multi-sensor information of the target is fused using the trust model, and the trust of each sensor is dynamically adjusted to establish a target feature library. The fused feature information is matched with the target feature library for similarity. When the similarity match exceeds a threshold, the target is identified as a potential target object and the potential range of the target is confirmed.

[0057] Specifically, a trust model is used to fuse the multi-sensor information of the target, including the sonar sensor, the visual image of the target, and the characteristic information of the water temperature and chemistry around the target. The trust of each sensor is dynamically adjusted according to the real-time underwater environment, including defining the initial trust value of each sensor and dynamically adjusting the trust of the sensor according to underwater environmental factors.

[0058] The initial trust of each sensor is defined as follows: T for sonar sensor sonar0 , the visual sensor is T vision0 , the temperature sensor is T temp0 , the chemical sensor is T chem0 , and T sonar0 >T vision0 >T temp0 >T chem0 .

[0059] The trust adjustment of the sonar sensor includes adjusting the propagation speed, attenuation and scattering of sound waves according to the water temperature and salinity, which in turn affect the detection accuracy and range of the sonar sensor. The change of water temperature has a greater impact on the performance of the sonar sensor than the change of salinity. Based on the degree of change of the water temperature and salinity environmental factors, the trust of the sonar sensor is linearly adjusted. The adjusted trust of the sonar sensor is T sonar1 The calculation formula is:

[0060] T sonar1 =T sonar0 -ω T ×f T (ΔT)-ω S ×f S (ΔS)

[0061] Among them, T sonar1 represents the adjusted sonar sensor trust, T sonar0 represents the initial trust of the sonar sensor, f T and f S Represent the functions of water temperature change and salinity change, ΔT and ΔS represent the amount of water temperature change and salinity change, respectively. T and ω S Represent the weights of water temperature change and salinity change respectively.

[0062] The water temperature variation function f T Expressed as:

[0063]

[0064] Among them, f T represents the function of water temperature change, ΔT represents the amount of water temperature change, k T represents the proportionality coefficient;

[0065] The salinity variation function f S Expressed as:

[0066]

[0067] Among them, f Srepresents the function of water temperature change, ΔS represents the amount of water temperature change, k S represents the proportionality coefficient;

[0068] The trust level adjustment of the visual sensor using fuzzy logic includes: because the underwater light intensity and water turbidity affect the image quality, when the underwater light intensity decreases, the contrast of the acquired image decreases, resulting in the loss of image details. Moreover, the more turbid the water is, the smaller the effective range of the visual sensor is, and the more difficult it is to obtain images from a distance. Therefore, the trust level of the visual sensor is adjusted using fuzzy logic. The underwater light intensity is divided into strong light area, medium light area and weak light area. If the light intensity is reduced by one level, the trust level of the visual sensor is within the initial trust level T. vision0 The trust level of the visual sensor is lowered by one level based on the initial trust level T and the water turbidity is divided into heavy turbidity, moderate turbidity and light turbidity. If the water turbidity rises by one level, the trust level of the visual sensor is lowered by one level based on the initial trust level T vision0 On the basis of the lower half of the level.

[0069] In a feasible embodiment, the trust level of the visual sensor is defined as the initial trust level T vision0 The value is reduced by one level, and the reduction can be 0.1.

[0070] Dynamically adjust the trust of the temperature sensor. In turbulent water, the movement of water is violent and irregular, resulting in uneven distribution of water temperature around the temperature sensor. The temperature measured by the sensor is local and unstable, and cannot accurately reflect the true temperature of the water. Thermal stratification causes significant temperature differences at different depths of the water. If the temperature sensor is placed near the interface of thermal stratification, the temperature measured by the sensor is affected by the two water layers with different temperatures due to the large temperature difference between the upper and lower water layers, resulting in inaccurate measurement results. Based on the turbulence of the water flow and the thermal stratification index, the trust of the temperature sensor is adjusted using a linear weighted method. The adjusted trust of the temperature sensor is T temp1 The calculation formula is:

[0071] T temp1 =T temp0 -ω V ×f V (V)-ω H ×f H (H)

[0072] Among them, T temp1 Represents the adjusted temperature sensor trust, T temp0 represents the initial trust of the temperature sensor, f V and f H V and H represent the functions of water turbulence and thermal stratification index, respectively.T and ω S They represent the weights of water turbulence and thermal stratification index respectively.

[0073] The water turbulence function f V Expressed as:

[0074]

[0075] Among them, f V represents the turbulence function of the water flow, V represents the turbulence of the water flow, k v represents the proportionality coefficient;

[0076] The thermal stratification index function f H Expressed as:

[0077]

[0078] Among them, f S The function of thermal stratification index, H represents the change of water temperature, k H represents the proportionality coefficient;

[0079] Dynamic adjustment of the trust of chemical sensors includes: the turbulence of water flow affects the contact time between the substance to be measured and the electrode, affects the composition and concentration of the substance detected by the electrode, and affects the accuracy of the chemical sensor measurement results; changes in water salinity affect the ionic strength and conductivity of the water, thereby interfering with the electron transfer process between the electrode and the substance to be measured; high salinity water causes ion adsorption and precipitation on the electrode surface, forming an insulating layer, which hinders the transfer of electrons, reduces the sensitivity and accuracy of the electrode, and affects the measurement accuracy of the chemical sensor. Based on the turbulence of water flow and the salinity of water body, the trust of the chemical sensor is adjusted using fuzzy logic method, and the turbulence of water flow is divided into low turbulence, medium turbulence and high turbulence. If the turbulence of water flow increases by one level, the trust of the chemical sensor is increased from the initial trust level T chem0 Based on the initial trust level T, the water salinity is divided into low salinity, medium salinity and high salinity. If the water salinity rises by one level, the trust level of the chemical sensor is chem0 Downgrade one level based on the

[0080] After dynamically adjusting the trust level of each sensor using a trust model based on the real-time environment, a trust interval is defined for each sensor. If the trust level of a sensor does not meet the trust interval, the sensor information that does not meet the trust interval is removed during multi-sensor information fusion for that period. The sensor information that meets the trust interval is constructed into a joint feature vector to perform multi-sensor information fusion.

[0081] If the trust level of the sensor meets its trust level range, the multi-sensor information is fused by constructing a joint feature vector from the feature vectors of the sensor information. The joint feature vector construction includes constructing the echo spectrum feature vector of the sonar sensor. Texture feature vector of visual image Gradient eigenvector of temperature and chemical concentration feature vector The order of each sub-feature vector is determined in the order of sonar-vision-temperature-chemistry, and a joint feature vector is constructed.

[0082] Collect characteristic information of various targets in advance, including sonar echo spectrum, visual image texture, temperature gradient and characteristic vectors of chemical concentration, and build target feature library and characteristic vectors And build a matching model based on cosine similarity, and transform the joint feature vector of sensor information into Input the feature vector of its matching model and the target feature information library Perform similarity matching and set the similarity matching threshold. When the similarity matching value exceeds the threshold, the target is identified as a potential target object. Otherwise, the feature vector of the target is and the feature vector in the target feature library Rematch until a potential target object with a similarity matching value exceeding the threshold is found.

[0083] Confirming the potential range of the target includes taking each sensor as the center and combining the information of sonar sensor, visual sensor, temperature sensor and chemical sensor to determine the potential range of the target as R sonar 、R vision 、R temp and R chem Based on the spatial intersection method, the final target potential range is confirmed to be R = R sonar ∩R vision ∩R temp ∩R chem .

[0084] like Figure 1-Figure 2 As shown, in step S5, within the potential range of the target, a probability density function detection algorithm is used. According to the prior probability distribution of the target in the marine ecosystem, a detection threshold is set. When the probability density of the target exceeds the threshold, the target is judged to exist. The support vector machine algorithm is used to determine the type of the target by combining the characteristic information of the temperature sensor and the chemical sensor.

[0085] Specifically, the determination of target presence includes using a probability density function detection algorithm to determine the prior probability distribution of the target based on historical data information on the frequency, location, temperature, and chemical concentration of the target in historical marine ecological studies, and using a multivariate normal distribution model to construct a prior probability distribution model of the target. The calculation formula of the prior probability distribution model of the target is expressed as:

[0086]

[0087] in, is the probability density, is the mean vector, Σ is the covariance matrix, and d is the vector Dimension, is a vector consisting of the target's position coordinates (x, y, z), temperature, and chemical concentration.

[0088] Within the potential range of the target, set the detection threshold to T d , the vector consisting of the target position coordinates (x, y, z), temperature and chemical concentration collected in real time Bring the target's prior probability density distribution model into the target to calculate the target's probability density. When calculating the target's probability density , the target is judged to exist; otherwise, the target is judged not to exist.

[0089] After determining the presence of a target, the temperature sensor information features and chemical sensor information features are combined to assist in determining the type of target. The support vector machine algorithm is used to determine the type of target, including:

[0090] Construct the feature vector [w avg , DO, pH], where w avg is the average water temperature around the target, DO is the dissolved oxygen content around the target, and pH is the acidity and alkalinity around the target. For each detected target, a binary classification algorithm is constructed with n categories preset. For each category i, a binary classification SVM model is constructed, using the radial basis kernel function as the kernel function. The samples of category i are taken as positive classes, and the samples of the remaining n-1 categories are taken as negative classes. The training set in the collected feature data is used to train the n binary classification SVM models, and the test set is used to evaluate the accuracy of the trained models.

[0091] The real-time feature vector [w avg , DO, pH] is input into the trained support vector machine algorithm, and based on the trained support vector machine algorithm, the real-time feature vector [w avg , DO, pH] to classify its target types and output the determined target types.

[0092] Those skilled in the art can make various other corresponding changes and deformations based on the technical solutions and concepts described above, and all of these changes and deformations should fall within the scope of protection of the claims of this application.

Claims

1. A target detection method for underwater robots based on multi-sensor information fusion, characterized in that: include: S1, using the underwater robot's sonar sensor, visual sensor, temperature sensor, and chemical sensor to obtain the target's sonar echo, the target's visual image, and the water temperature and chemical information around the target; S2. Add a timestamp to the information collected by each sensor, align it to a unified time axis using interpolation, and filter and normalize each sensor information; S3, extracting target feature information by collecting target sonar echoes, target visual images, and water temperature and chemical information around the target; S4. Use the trust model to fuse the multi-sensor information of the target, dynamically adjust the trust of each sensor, establish a target feature library, and perform similarity matching between the fused feature information and the target feature library. When the similarity match exceeds a threshold, the target is identified as a potential target object and the potential range of the target is confirmed. S5. Within the potential range of the target, a probability density function detection algorithm is used. According to the prior probability distribution of the target in the marine ecosystem, a detection threshold is set. When the probability density of the target exceeds the threshold, the target is judged to exist. Combined with the characteristic information of the temperature sensor and the chemical sensor, the support vector machine algorithm is used to determine the type of the target.

2. The underwater robot target detection method based on multi-sensor information fusion according to claim 1, characterized in that: In step S3, the target characteristic information is extracted by extracting the target's echo signal strength, distance, azimuth and spectral characteristics through the target's sonar echo, extracting the target's shape, color and texture characteristics through the target's visual image, extracting the average temperature and temperature gradient around the target through the water temperature around the target, and extracting the dissolved oxygen concentration, pH value and chemical substance concentration around the target through chemical information.

3. The underwater robot target detection method based on multi-sensor information fusion according to claim 1, characterized in that: In step S4, the dynamic adjustment of the trustworthiness of each sensor includes defining an initial trustworthiness of the sonar sensor, the visual sensor, the temperature sensor, and the chemical sensor, and dynamically adjusting the trustworthiness of each sensor according to the real-time underwater environment; The initial trust of each sensor is defined as follows: T for sonar sensor sonar0 , the visual sensor is T vision0 , the temperature sensor is T temp0 , the chemical sensor is T chem0 , and T sonar0 >T vision0 >T temp0 >T chem0 .

4. The underwater robot target detection method based on multi-sensor information fusion according to claim 3 is characterized in that: Dynamically adjusting the trust of the sonar sensor includes linearly adjusting the trust of the sonar sensor according to the water temperature and salinity in the underwater environment. The adjusted trust of the sonar sensor T sonar1 The calculation formula is: T sonar1 =T sonar0 -oh T ×f T (ΔT)-ω S ×f S (ΔS) Among them, T sonar1 represents the adjusted sonar sensor trust, T sonar0 represents the initial trust of the sonar sensor, f T and f S Represent the functions of water temperature change and salinity change, ΔT and ΔS represent the amount of water temperature change and salinity change, respectively. T and ω S Represent the weights of water temperature change and salinity change respectively.

5. The underwater robot target detection method based on multi-sensor information fusion according to claim 3, characterized in that: Dynamically adjusting the trust of the visual sensor includes using fuzzy logic to adjust the trust of the visual sensor. The underwater light intensity is divided into strong light area, medium light area and weak light area. If the light intensity is reduced by one level, the trust of the visual sensor will be adjusted from the initial trust level T vision0 The trust level of the visual sensor is lowered by one level based on the initial trust level T and the water turbidity is divided into heavy turbidity, moderate turbidity and light turbidity. If the water turbidity rises by one level, the trust level of the visual sensor is lowered by one level based on the initial trust level T vision0 On the basis of the lower half of the level.

6. The underwater robot target detection method based on multi-sensor information fusion according to claim 3, characterized in that: Dynamically adjusting the trust of the temperature sensor includes adjusting the trust of the temperature sensor using a linear weighted method based on the turbulence of the water flow and the thermal stratification index. The adjusted trust of the temperature sensor is T temp1 The calculation formula is: T temp1 =T temp0 -ω V ×f V (V)-ω H ×f H (H) Among them, T temp1 Represents the adjusted temperature sensor trust, T temp0 represents the initial trust of the temperature sensor, f V and f H V and H represent the functions of water turbulence and thermal stratification index, respectively. T and ω S They represent the weights of water turbulence and thermal stratification index respectively.

7. The underwater robot target detection method based on multi-sensor information fusion according to claim 3, characterized in that: Dynamically adjusting the trust of chemical sensors includes adjusting the trust of chemical sensors based on the turbulence of water flow and the salinity of water body using fuzzy logic method. The turbulence of water flow is divided into low turbulence, medium turbulence and high turbulence. If the turbulence of water flow increases by one level, the trust of chemical sensors increases from the initial trust level T chem0 Based on the initial trust level T, the water salinity is divided into low salinity, medium salinity and high salinity. If the water salinity rises by one level, the trust level of the chemical sensor is chem0 Downgrade one level based on the 8. The underwater robot target detection method based on multi-sensor information fusion according to claim 1, characterized in that: Step S4 specifically includes, after dynamically adjusting the trust level of each sensor, defining a trust level interval for each sensor. If the trust level of a sensor does not meet the trust level interval, then when performing multi-sensor information fusion for that period, the sensor information that does not meet the trust level interval is removed. The multi-sensor information is fused by constructing the sensor information that meets the trust level interval into a joint feature vector. If the trust level of the sensor meets its trust level range, the multi-sensor information is fused by constructing a joint feature vector from the feature vectors of the sensor information. The joint feature vector construction includes constructing the echo spectrum feature vector of the sonar sensor. Texture feature vector of visual image Gradient eigenvector of temperature and chemical concentration feature vector The order of each sub-feature vector is determined in the order of sonar-vision-temperature-chemistry, and a joint feature vector is constructed. Collect characteristic information of various targets in advance, including sonar echo spectrum, visual image texture, temperature gradient and characteristic vectors of chemical concentration, and build target feature library and characteristic vectors And build a matching model based on cosine similarity, and transform the joint feature vector of sensor information into Input the feature vector of its matching model and the target feature information library Perform similarity matching and set the similarity matching threshold. When the similarity matching value exceeds the threshold, the target is identified as a potential target object. Otherwise, the feature vector of the target is and the feature vector in the target feature library Rematch until a potential target object with a similarity matching value exceeding the threshold is found; Confirming the potential range of the target includes taking each sensor as the center and combining the information of sonar sensor, visual sensor, temperature sensor and chemical sensor to determine the potential range of the target as R sonar 、R vision 、R temp and R chem Based on the spatial intersection method, the final target potential range is confirmed to be R = R sonar ∩R vision ∩R temp ∩R chem .

9. The underwater robot target detection method based on multi-sensor information fusion according to claim 1, characterized in that: In step S5, the determination of target existence includes using a probability density function detection algorithm to determine the prior probability distribution of the target based on historical data information on the frequency, location, temperature and chemical concentration of the target in historical marine ecological research, and using a multivariate normal distribution model to construct a prior probability distribution model of the target. Within the potential range of the target, the detection threshold is set to T d , the vector consisting of the target position coordinates x, y, z, temperature and chemical concentration collected in real time Bring the target's prior probability density distribution model into the target to calculate the target's probability density. When calculating the target's probability density When , it is judged that the target exists, otherwise, it is judged that the target does not exist; After determining the existence of the target, the temperature sensor information features and chemical sensor information features are combined to assist in determining the type of the target, including constructing the feature vector [w avg , DO, pH], w avg is the average water temperature around the target, DO is the dissolved oxygen content around the target, pH is the acidity and alkalinity around the target, and the real-time feature vector [w avg , DO, pH] is input into the trained support vector machine algorithm and the determined target species is output.

10. The underwater robot target detection method based on multi-sensor information fusion according to claim 1, characterized in that: In step S2, the following steps are specifically included: When each sensor collects information, the time information of the information collection is recorded, and the information collected by each sensor and the corresponding time information are recorded and saved together. When aligning the collected information to a unified time axis, the time axis of the sonar sensor information collection is selected as the reference time axis, and the information collected by the visual sensor, temperature sensor, and chemical sensor is unified to the reference time axis through interpolation. When unifying the information onto the reference time axis through interpolation, each sensor information is filtered using the median filter method. The filtering process includes defining the window length of each information sequence and selecting the middle value in each window after sorting in ascending or descending order as the filtered value.

Images