Intelligent identification method for river floating objects based on multi-device image fusion

Abstract

The present application relates to the technical field of image recognition, and discloses a river channel floating object intelligent recognition method based on multi-device image fusion, comprising: acquiring the visible light polarization signal and the long-wave infrared absolute temperature of a target water area, mapping into a polarization radiation intensity field and a thermal radiation intensity field and extracting corresponding gradient vector fields; calculating the divergence of the polarization gradient vector field to obtain a polarization anomaly divergence field; combining the time partial derivative of the thermal radiation intensity field and the thermal force gradient vector field to deduce a thermal force phase flow velocity field; calculating the spatial first-order gradient vector of the polarization anomaly divergence field and performing an inner product operation with the thermal force phase flow velocity field to obtain a cross-damping tensor field; substituting into a Poisson equation to reconstruct a global closure topological potential energy field; solving an intrinsic real phase Gaussian curvature field of the potential energy field; calculating the global energy intrinsic expectation ground state, and performing a phase change collapse judgment on the curvature field according to the same to output a binary mask.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and more specifically, to a method for intelligent identification of floating objects in rivers based on multi-device image fusion. Background Technology

[0002] Most existing machine vision recognition methods rely primarily on empirically preset parameters, grayscale contrast adjustment, and morphological thresholding for image segmentation. With the development of multimodal acquisition systems, dual-device joint observation, including both visible light polarization sensing units and long-wave infrared thermal imaging units, is becoming increasingly common. In existing multimodal fusion paradigms, it is typically enforced that the edge boundaries of the same object on different detectors must achieve absolute static overlap and registration in three-dimensional spacetime.

[0003] However, real floating objects in natural waterways, governed by their own density and buoyancy, are inevitably in a partially submerged, semi-submerged state, forming a closed boundary with the continuously flowing water surface. When water waves impact the object's boundary, energy dissipation occurs, leading to a reduction in surface roughness in local areas. This instantaneously alters the polarization distribution of water surface reflection. Simultaneously, surface smoothing shrinks the microscopic evaporation area and changes the latent heat exchange divergence, thus affecting the long-wave infrared absolute temperature. Due to the differences in specific heat capacity and heat conduction laws of different phase media, changes in the temperature field must undergo a time integration process of energy transfer, resulting in a natural lag in phase between thermal evolution and polarization optical abrupt changes. Therefore, the real polarization optical distortion boundary and the infrared heat flux boundary are asymmetrically misaligned in time. If existing technologies forcibly employ image weighted registration or pixel-level absolute alignment, not only will they fail to accurately map the boundary, but they will also force the aforementioned real phase shift misalignment to cause destructive interference, erasing the only delayed evolution information that can identify the semi-submerged state, ultimately causing fusion failure and a severe decrease in recognition accuracy. Summary of the Invention

[0004] This invention provides an intelligent identification method for floating objects in rivers based on multi-device image fusion, which solves the technical problems mentioned in the background art.

[0005] This invention provides an intelligent identification method for floating objects in rivers based on multi-device image fusion, applicable to a multimodal acquisition system, including: The visible light polarization signal and long-wave infrared absolute temperature of the target water area are obtained in the two-dimensional spatial computational domain and mapped into polarization radiation intensity field and thermal radiation intensity field, respectively. Extract the polarization gradient vector field and the thermodynamic gradient vector field corresponding to the polarization radiation intensity field and the thermal radiation intensity field, respectively. Calculate the first-order partial derivative of the thermal radiation intensity field with respect to time; The two-dimensional spatial divergence of the polarization gradient vector field is calculated to obtain the polarization anomaly divergence field; By combining the first-order partial derivatives with the thermodynamic gradient vector field, the thermodynamic phase velocity field is deduced; The spatial first-order gradient vector of the polarization anomaly divergence field is obtained, and a two-dimensional vector inner product operation is performed with the thermodynamic phase velocity field to obtain the multiphase cross-damping tensor field. Substituting the multiphase cross-damping tensor field into the two-dimensional Poisson equation for reconstruction yields the global closure topological potential field. The curvature of the global closure topological potential field is calculated, and the intrinsic real Gaussian curvature field is extracted. Calculate the global energy eigenvalue expected ground state of the intrinsic real Gaussian curvature field within the two-dimensional computational domain; The intrinsic real Gaussian curvature field is evaluated using the global energy intrinsic expectation ground state, and a phase transition binary mask is output.

[0006] Furthermore, the step of acquiring the visible light polarization signal and long-wave infrared absolute temperature of the target water area within the two-dimensional spatial computational domain, and mapping them respectively into a polarization radiation intensity field and a thermal radiation intensity field; and extracting the polarization gradient vector field and thermodynamic gradient vector field corresponding to the polarization radiation intensity field and the thermal radiation intensity field respectively, includes: The Stokes first polarization component and the Stokes second polarization component are obtained as the visible light polarization signal; The square root of the square of the first Stokes polarization component and the square of the second Stokes polarization component is obtained by mapping the polarization radiation intensity field. The thermal radiation intensity field is obtained by multiplying the intrinsic emissivity constant of the water body by the Stefan-Boltzmann constant, dividing the product by pi, and then multiplying it by the fourth power of the long-wave infrared absolute temperature. The polarization radiation intensity field and the thermal radiation intensity field are respectively calculated by taking the first-order partial differential in space, and the polarization gradient vector field and the thermal gradient vector field are respectively extracted.

[0007] Furthermore, the step of calculating the two-dimensional spatial divergence of the polarization gradient vector field to obtain the polarization anomaly divergence field includes: By calculating the two-dimensional spatial Laplace partial derivative of the polarization radiation intensity field, the equivalent physical calculation of the two-dimensional spatial divergence of the polarization gradient vector field is realized, and the polarization anomaly divergence field reflecting the intensity of local polarization energy accumulation sources and sinks is obtained by inversion.

[0008] Furthermore, the step of combining the first-order partial derivative with the thermodynamic gradient vector field to deduce the thermodynamic phase velocity field includes: Calculate the square of the vector magnitude of the thermodynamic gradient vector field; Divide the thermodynamic gradient vector field by the square of the vector magnitude, and multiply the resulting quotient by the negative value of the first-order partial derivative to obtain the thermodynamic phase velocity field used to characterize the evolution and diffusion of latent heat flux.

[0009] Furthermore, the step of obtaining the spatial first-order gradient vector of the polarization anomaly divergence field and performing a two-dimensional vector inner product operation with the thermodynamic phase velocity field to obtain the multiphase cross-damping tensor field includes: The first-order partial differential of the polarization anomaly divergence field is obtained to yield the divergence spatial gradient vector field as the first-order spatial gradient vector. The multiphase cross-damping tensor field is obtained by performing a two-dimensional vector dot product operation between the divergence space gradient vector field and the thermodynamic phase velocity field.

[0010] Furthermore, the step of substituting the multiphase cross-damping tensor field into the two-dimensional Poisson equation to reconstruct the global closure topological potential field includes: The multiphase cross-damping tensor field is converted into a source charge density field by taking the inverse negative value; A two-dimensional spatial potential energy convergence equation with the source charge density field as the equation correlation term is established as the two-dimensional Poisson equation; Solving the two-dimensional potential energy convergence equation forces the discrete distribution breakpoints to spontaneously fill and close within the two-dimensional computational domain, resulting in the reconstructed global closure topological potential energy field.

[0011] Furthermore, the step of calculating the curvature of the global closure topological potential energy field and extracting the eigenvalue Gaussian curvature field includes: Calculate the second-order pure partial derivatives of the global closure topological potential field along the first spatial dimension and along the second spatial dimension in the two-dimensional computational domain, and multiply the two to obtain the first product term. Calculate the second-order mixed partial derivatives of the global closure topological potential field along the first spatial dimension and the second spatial dimension, and then square the second-order mixed partial derivatives to obtain the second product term; Subtracting the second product term from the first product term, the abrupt peak envelope of the absolutely isolated topology is extracted to obtain the intrinsic real Gaussian curvature field.

[0012] Furthermore, the calculation of the global energy eigenvalue expected ground state of the intrinsic real Gaussian curvature field within the two-dimensional computational domain; and the use of the global energy eigenvalue expected ground state to perform a determination on the intrinsic real Gaussian curvature field, outputting a phase transition binary mask, includes: Multiply the absolute value of the intrinsic real Gaussian curvature field by the intrinsic real Gaussian curvature field itself, and perform a double surface integral in the two-dimensional computational domain to obtain the total real topological energy of the entire domain. The absolute value of the intrinsic real Gaussian curvature field is subjected to a double surface integral within the two-dimensional computational domain to obtain the global reference area envelope. Divide the total global real topological energy by the global reference area envelope to obtain the global energy intrinsic expected ground state. Subtracting the global energy eigenvalue ground state from the eigenreal Gaussian curvature field yields the eigenenergy difference. The sign function polarity of the intrinsic energy difference is extracted to obtain the extracted polarity value; Add one to the extracted polarity value and divide by two to output the phase transition binary mask containing only zero and one.

[0013] The beneficial effects of this invention include: constructing a radiation intensity field based on visible light polarization signals and long-wave infrared absolute temperature; deeply deriving the dynamic evolution relationship between polarization divergence and thermodynamic phase velocity; reconstructing the global topological potential energy field using vector inner product and the Poisson equation; then calculating the intrinsic real phase Gaussian curvature; and finally spontaneously outputting a binary mask based on the natural energy distribution law. The entire process comprehensively restores the true optical abrupt changes and thermodynamic response hysteresis evolution process of a semi-submersible object under water wave damping, significantly improving the accuracy of identifying real floating objects in complex natural waters and the system's anti-interference capability. Attached Figure Description

[0014] Figure 1 This is a flowchart of the intelligent identification method for floating objects in rivers based on multi-device image fusion according to the present invention. Detailed Implementation

[0015] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0016] This embodiment applies to a multimodal acquisition system, which includes a visible light polarization imaging unit, a long-wave infrared thermal imaging unit, a rigid mounting bracket, a data processing unit, and a clock synchronization module. The visible light polarization imaging unit uses a planar polarization camera with a wavelength range of 400nm to 700nm. It incorporates a micro-polarizer array in four directions: 0°, 45°, 90°, and 135°. The pixel size is no larger than 3.45μm, the resolution is no less than 1920×1080, and the frame rate ranges from 10fps to 60fps. The long-wave infrared thermal imaging unit uses a cooled or uncooled long-wave infrared camera with a wavelength range of 8μm to 14μm. The temperature measurement accuracy is no less than ±0.5℃, the pixel size is no larger than 17μm, and its resolution and frame rate match those of the visible light polarization imaging unit. The rigid mounting bracket is used to fix the two types of imaging units. The optical axis parallelism deviation between the two types of imaging units is no more than 0.1°, the field of view overlap rate is no less than 95%, the installation height is 3m to 15m, and the pitch angle range is 15° to 75°, ensuring that the field of view completely covers the target water area.

[0017] like Figure 1 As shown, a method for intelligent identification of floating objects in rivers based on multi-device image fusion is applied to a multimodal acquisition system, including: The visible light polarization signal and long-wave infrared absolute temperature of the target water area are obtained in the two-dimensional spatial computational domain and mapped into polarization radiation intensity field and thermal radiation intensity field, respectively. Extract the polarization gradient vector field and the thermodynamic gradient vector field corresponding to the polarization radiation intensity field and the thermal radiation intensity field, respectively. Calculate the first-order partial derivative of the thermal radiation intensity field with respect to time; The two-dimensional spatial divergence of the polarization gradient vector field is calculated to obtain the polarization anomaly divergence field; By combining the first-order partial derivatives with the thermodynamic gradient vector field, the thermodynamic phase velocity field is deduced; The spatial first-order gradient vector of the polarization anomaly divergence field is obtained, and a two-dimensional vector inner product operation is performed with the thermodynamic phase velocity field to obtain the multiphase cross-damping tensor field. Substituting the multiphase cross-damping tensor field into the two-dimensional Poisson equation for reconstruction yields the global closure topological potential field. The curvature of the global closure topological potential field is calculated, and the intrinsic real Gaussian curvature field is extracted. Calculate the global energy eigenvalue expected ground state of the intrinsic real Gaussian curvature field within the two-dimensional computational domain; The intrinsic real Gaussian curvature field is evaluated using the global energy intrinsic expectation ground state, and a phase transition binary mask is output.

[0018] A checkerboard calibration board was used to jointly calibrate two types of imaging units. The checkerboard size was no less than 20mm × 20mm, and the number of corner points was no less than 12 × 9. The calibration process was as follows: the calibration board was placed at different distances and angles from the target water area, and images of the calibration board for both types of imaging units were acquired simultaneously; the coordinates of the checkerboard corner points in the two types of images were extracted, the homography matrix and lens distortion coefficients were calculated, and radial and tangential distortion corrections were completed; based on the homography matrix, a pixel mapping relationship between the visible light polarization imaging plane and the long-wave infrared imaging plane was established to achieve pixel-level registration of the two types of images, with a registration error of no more than 1 pixel. For non-coplanar parallax, a planar homography mapping combined with bilinear interpolation was used to complete pixel alignment, ensuring that each pixel in the imaging plane corresponds one-to-one with the physical spatial position of the target water area.

[0019] The two-dimensional spatial computational domain is the corrected imaging plane corresponding to the target water area. A Cartesian coordinate system is constructed with the horizontal spatial dimension x and the vertical spatial dimension y. The x-axis corresponds to the horizontal direction of the imaging plane, and the y-axis corresponds to the vertical direction of the imaging plane. The physical spatial step size corresponding to a single pixel is denoted as Δs, with the unit being meters. The formula for calculating Δs is: Where H is the installation height of the imaging unit, in meters (m); The horizontal field of view of the imaging unit is expressed in rad. This represents the total number of pixels in the horizontal direction of the imaging plane.

[0020] The clock synchronization module adopts a hardware-triggered synchronization method, simultaneously outputting frame trigger signals to both types of imaging units through a synchronization signal generator. The time synchronization error of the trigger signal is no greater than 10μs. Simultaneously, a network time protocol is used to align the timestamps of the acquired data, with a timestamp synchronization error not exceeding 10% of the sampling period. The sampling period Δt is the time interval between the acquisition of two adjacent frames, in seconds, with Δt taking the value of 1 / fps. fps is the frame rate of the imaging unit; in still water scenarios, fps ranges from 10fps to 20fps; in slow-flowing scenarios, fps ranges from 20fps to 30fps; and in fast-flowing scenarios, fps ranges from 30fps to 60fps (water flow scenarios are classified according to the average surface velocity: still water <0.1m / s, slow-flowing 0.1-1.0m / s, fast-flowing >1.0m / s).

[0021] The hysteresis time was calibrated to address the natural hysteresis phase caused by polarization optical abrupt changes and thermodynamic evolution.

[0022] The standard semi-submersible polyethylene calibration sphere is manufactured by using a 5mm thick hollow polyethylene sphere, with an internal mass equal to the mass of the sphere displaced. A cylindrical lead block is used, which is glued to the center of the bottom of the sphere with epoxy resin to ensure that the sphere's draft is equal to its diameter when it is at rest. The calibration object is secured using a flexible anti-drift traction scheme, employing a 3mm diameter nylon traction rope to attach the calibration ball to a steel anchor point on the seabed. The traction rope is longer than [missing information - likely a length]. The water depth is doubled, allowing the calibration ball to fluctuate slightly with the water surface without horizontal drift, ensuring the stability and traceability of the calibration area within consecutive frames.

[0023] A standard semi-submersible calibration object (specifically, a polyethylene standard sphere with a surface coated with matte black paint with an emissivity of 0.95, an outer diameter of 0.5 meters, and internal weights to keep it in a semi-submersible state; the semi-submersible state is defined as a draft of 40%-60% of the sphere's diameter) was placed in the target water area. Polarization and infrared images were simultaneously acquired at different times; the abrupt changes in polarization radiation intensity in the calibration object area were calculated. (Defined as the moment when the absolute value of the first-order partial derivative of the mean polarized radiation intensity in the region with respect to time reaches its global maximum), and the moment of abrupt change in the corresponding region's thermal radiation intensity response. (Defined as the moment when the absolute value of the first-order partial derivative of the mean thermal radiation intensity of the region with respect to time reaches its global maximum); the formula for calculating the hysteresis time is: The value ranges from 0.1s to 2s, and the values ​​are obtained using corresponding calibrations under different water flow scenarios. This is used for time alignment of the subsequent polarization field with the thermal field.

[0024] The acquired visible light polarization images and long-wave infrared images were first smoothed and denoised using a 5×5 Gaussian filter kernel, and the standard deviation of the Gaussian filter was... The value is set to 1.0 to 1.5 to filter out shot noise and thermal noise in the image; the filtered image is then subjected to single-image linear gray-level normalization to map the pixel values ​​to the [0,1] interval, thereby eliminating the influence of differences in imaging unit gain.

[0025] The visible light polarization signal and long-wave infrared absolute temperature of the target water body are obtained in a two-dimensional spatial computational domain and mapped into polarization radiation intensity field and thermal radiation intensity field, respectively.

[0026] The Stokes first polarization component and the Stokes second polarization component are obtained as visible light polarization signals. The Stokes first polarization component is denoted as... Dimensionless, representing the intensity difference of linearly polarized light at 0° and 90° directions at corresponding pixels within the imaging plane; the Stokes second polarization component is denoted as... Dimensionless, representing the intensity difference of linearly polarized light at 45° and 135° directions at the corresponding pixel. Both polarization components are directly acquired through a micro-polarizer array in the visible light polarization imaging unit, with a sampling period consistent with that of the long-wave infrared thermal imaging unit.

[0027] The polarized radiation intensity field is obtained by adding the squares of the first and second Stokes polarization components and taking the square root. The formula for calculating the polarized radiation intensity field is as follows: in Let t be the polarized radiation intensity field corresponding to the (x,y) coordinate point in the imaging plane at time t. It is dimensionless and represents the total linearly polarized radiation intensity at that spatial location. t is the time of image acquisition, in seconds.

[0028] Multiplying the intrinsic emissivity constant of water by the Stefan-Boltzmann constant, dividing the product by pi, and then multiplying by the fourth power of the long-wave infrared absolute temperature, yields the thermal radiation intensity field. The formula for calculating the thermal radiation intensity field is: in The intrinsic emissivity constant of the water body is dimensionless, with a value of 0.96 for clear water, 0.94 for turbid water containing sediment, and 0.92 for water containing algae. Water scenes are automatically classified using the RGB color histogram features of visible light images. The threshold rule for automatic classification of water scenes based on RGB histograms is as follows: extract the RGB three-channel histogram of the pure water surface area without floating objects in the visible light image, and calculate the mean and variance of each channel; for clear water: the difference between the means of any two channels of R, G, and B. And the variance of each channel Muddy water containing sediment: The mean of channel R is greater than the mean of channel G, and the mean of channel R is greater than the mean of channel B, and the difference between the mean of channel R and the means of channels G and B is also greater than the mean of channel B. Water bodies containing algae: the mean of channel G > the mean of channel R and the mean of channel G > the mean of channel B, and the difference between the mean of channel G and the means of channels R and B is... .

[0029] The Stefan Boltzmann constant is a physical constant with values ​​of 10 ... ; Pi is a dimensionless mathematical constant, with a value of 3.1415926535. The absolute temperature is measured in Kelvin and is acquired by a long-wave infrared thermal imager. It represents the absolute temperature value at time t of the (x,y) coordinate point in the imaging plane. Let be the thermal radiation intensity field at time t corresponding to the (x,y) coordinate point in the imaging plane, in units of . , which represents the thermal radiation flux density at that spatial location.

[0030] First-order partial differential equations in space are performed for the polarization radiation intensity field and the thermal radiation intensity field, respectively, to extract the polarization gradient vector field and the thermodynamic gradient vector field. Two-dimensional first-order partial differential operator. Defined in a two-dimensional Cartesian coordinate system ,in , These are the unit orthogonal basis vectors in the x and y directions, respectively.

[0031] The formula for calculating the polarization gradient vector field is: in The polarization gradient vector field characterizes the direction and rate of change of polarization radiation intensity in two-dimensional space.

[0032] The formula for calculating the thermal gradient vector field is: in The thermal gradient vector field represents the direction and rate of change of thermal radiation intensity in two-dimensional space.

[0033] For discrete imaging pixel data, the first-order partial derivative is calculated using the central difference method. The discrete calculation formula for the partial derivative in the x-direction is as follows: The discrete formula for calculating the partial derivative in the y-direction is: Where i is the pixel row index, j is the pixel column index, and L is the pixel value of the corresponding radiation intensity field. For the boundary pixels of the imaging plane, a copy-fill method is used to expand the boundary, with a fill width of 1 pixel, to ensure that the partial differential calculation of the boundary pixels can be performed normally.

[0034] The two-dimensional spatial divergence of the polarization gradient vector field is calculated to obtain the polarization anomaly divergence field. By taking the two-dimensional spatial Laplace partial derivative of the polarization radiation intensity field, an equivalent physical calculation of the two-dimensional spatial divergence of the polarization gradient vector field is achieved, resulting in the inversion of the polarization anomaly divergence field reflecting the intensity of local polarization energy accumulation sources and sinks. Inversion refers to the calculation process of reconstructing the spatial source and sink distribution characteristics of the vector field through second-order differential operations of the scalar field. The two-dimensional spatial divergence operator is denoted as... Used to calculate the source and sink strengths of a vector field in space; two-dimensional Laplace operator Defined in a two-dimensional Cartesian coordinate system .

[0035] The formula for calculating the polarization anomaly divergence field is: in This represents the polarization anomaly divergence field, with units of . , which represents the intensity of local polarization energy accumulation or diffusion at the (x,y) coordinate point at time t. Positive values ​​correspond to polarization energy accumulation regions, negative values ​​correspond to polarization energy diffusion regions, and zero values ​​correspond to uniform polarization regions.

[0036] For discrete pixel data, the Laplacian operation is implemented using a 3×3 Laplacian convolution kernel with values ​​of [[0,1,0],[1,-4,1],[0,1,0]]. After performing the discrete convolution operation, the result is divided by the square of the physical space stride. To unify the physical spatial scale and dimensions, the image boundary is expanded using a copy-padding method before convolution operations, with a padding width of 1 pixel, to ensure that convolution calculations at boundary pixels can be performed correctly. For the result after convolution operations, a 3×3 median filter is used to remove extreme noise. When the imaging resolution is greater than or equal to 3840×2160, the filter window size can be adjusted to 5×5.

[0037] The first-order partial derivative of the thermal radiation intensity field with respect to time is calculated. Combined with the first-order partial derivative and the thermal gradient vector field, the thermal phase velocity field is deduced. The deduction refers to the calculation process of solving the propagation velocity field of thermal disturbances on the water surface based on the physical equations of thermal transport and the known spatiotemporal variation characteristics of the thermal radiation field.

[0038] Calculate the first partial derivative of the thermal radiation intensity field with respect to time. The first partial derivative of the thermal radiation intensity field with respect to time is denoted as... Characterizes the rate of change of thermal radiation intensity with time, with units of . For discrete-time series imaging data, the first-order time partial derivative is calculated using the adjacent frame difference method, and the calculation formula is as follows: For scenarios involving frame loss or data anomalies, linear interpolation is used to complete the thermal radiation intensity data of missing frames, ensuring the continuity of time partial derivative calculation.

[0039] Calculate the square of the magnitude of the thermal gradient vector field. The formula for calculating the square of the magnitude of the thermal gradient vector field is: in The vector magnitude of the thermal gradient field is the square of the vector length, in units of . .

[0040] Dividing the thermal gradient vector field by the square of the vector magnitude and multiplying the quotient by the negative of the first-order partial derivative yields the thermal phase velocity field characterizing the evolution and diffusion of latent heat flux. This formula is derived based on the characteristic line solution of the thermal convection-diffusion equation and is applicable to the thermal transport scenario of thin-layer fluids at the water surface, characterizing the propagation speed and direction of thermal disturbances at the water surface. The formula for calculating the thermal phase velocity field is: in This represents the thermodynamic phase velocity field, in units of... This characterizes the evolution and diffusion velocity and direction of the latent heat flux at the (x,y) coordinate point at time t, corresponding to the propagation phase characteristics of the water surface thermal disturbance. During the calculation, when... When the corresponding position of the thermal phase velocity field is zero, the zero vector is directly taken to avoid the error of dividing by 0. When the calculated velocity modulus exceeds the preset maximum velocity threshold, the direction angle of the vector is kept unchanged, and the x and y components of the thermal phase velocity field are scaled proportionally so that the scaled velocity modulus is equal to the maximum velocity threshold (5m / s), which corresponds to the maximum surface velocity of the natural river.

[0041] The spatial first-order gradient vector of the polarization anomaly divergence field is obtained, and a two-dimensional vector inner product operation is performed with the thermodynamic phase velocity field to derive the multiphase cross-damping tensor field. During the calculation, the hysteresis time obtained from calibration is used. The time alignment of the polarization field and the thermal field is completed. Not the sampling period When the value is an integer multiple of the time dimension, a linear interpolation method is used to obtain the time dimension of two adjacent physical frames. The polarization anomaly divergence field at time t is calculated with the thermodynamic phase velocity field at time t to match the polarization abrupt change and the natural hysteresis phase of thermodynamic evolution.

[0042] Taking the first-order partial differential in space of the polarization anomaly divergence field yields the divergence space gradient vector field as the first-order gradient vector in space. The divergence space gradient vector field is... The polarization anomaly divergence field at time t is the result of performing a first-order partial differential operation in two-dimensional space, calculated by the following formula: The calculation method is the same as the aforementioned gradient vector field calculation method. The resulting vector field represents the direction and rate of change of polarization anomaly divergence in two-dimensional space.

[0043] By performing a two-dimensional vector dot product between the divergence spatial gradient vector field and the thermodynamic phase velocity field, a multiphase cross-damping tensor field is obtained. This multiphase cross-damping tensor field characterizes the cross-damping effect between abrupt polarization abrupt changes and thermodynamic phase evolution, corresponding to the energy dissipation intensity generated by the interaction between the semi-submersible floating object and water surface ripples. Positive values ​​correspond to energy dissipation regions, negative values ​​to energy release regions, and zero values ​​to uniform water surface regions. The formula for calculating the multiphase cross-damping tensor field is as follows: in This is a multiphase cross-damped tensor field (the scalar result of the two-dimensional vector inner product is represented as a zero-order tensor), with units of... The two-dimensional vector dot product operation is calculated by multiplying the corresponding x-axis components and y-axis components of the two vectors, and then adding the two products to obtain the scalar result. The result is then smoothed using a 5×5 Gaussian filter to eliminate high-frequency noise introduced by the discrete calculation.

[0044] By substituting the multiphase cross-damping tensor field into the two-dimensional Poisson equation, the global closure topological potential field is obtained.

[0045] The source charge density field is constructed by taking the inverse negative values ​​of the multiphase cross-damped tensor field. The source charge density field is... , which corresponds to the source term of the two-dimensional Poisson equation, characterizes the spatial convergence intensity of the potential energy field. Positive values ​​correspond to potential energy convergence regions, while negative values ​​correspond to potential energy diffusion regions.

[0046] A two-dimensional spatial potential energy accumulation equation, with the source charge density field as the related term, is established as the two-dimensional Poisson equation. The expression of the two-dimensional Poisson equation is: in The global closed topological potential field, in units of , represents the topological potential energy value at the (x,y) coordinate point at time t, and reflects the topological boundary potential energy distribution corresponding to the floating object in the target water area.

[0047] Solving the two-dimensional potential energy convergence equation forces the discretely distributed discontinuities to spontaneously fill and close within the two-dimensional computational domain, yielding the reconstructed global closure topological potential energy field. The solution employs the Fast Fourier Transform method, with the following specific steps: For source charge density field Perform a two-dimensional fast Fourier transform to obtain the frequency domain source terms. , where u and v are the wavenumbers in the horizontal and vertical directions of the frequency domain; Construct the frequency domain differential kernel. The formula for calculating the frequency domain differential kernel is as follows: in To represent the total number of pixels in the vertical direction of the imaging plane, we introduce... This maps the frequency domain to the actual physical wavenumber scale. To eliminate numerical singularities in the frequency domain solution, a minimum regularization term is added to the frequency domain differential kernel. , Values The regularized frequency domain differential kernel is obtained. ; The formula for calculating the frequency domain potential field is: Performing a two-dimensional inverse fast Fourier transform on the frequency domain potential field yields the global closure topological potential field in the spatial domain. ; To ensure that the periodic implicit condition of the Fast Fourier Transform satisfies the homogeneous Dirichlet boundary condition, the spatial domain source charge density field is first oddly extended before the transform. After solving, the effective region is extracted to avoid boundary mathematical artifacts caused by forced zeroing. A radially symmetric Hanning window is used to window the frequency domain source terms. The formula for calculating the Hanning window is as follows: ;in, For the Hanning window function, The highest spatial frequency is used to eliminate the Gibbs effect caused by FFT solving.

[0048] Curvature is calculated for the global closure topological potential energy field, and the intrinsic real Gaussian curvature field is extracted. The intrinsic real Gaussian curvature field is the second-order differential determinant of the potential energy surface, used to characterize the local topological abrupt change features of the potential energy surface, and to extract the peak envelope of the floating object boundary. This calculation method removes the first-order gradient normalization denominator term in the known Gaussian curvature formula, enhances the abrupt change features of the topological boundary, and is suitable for edge extraction scenarios in the image domain.

[0049] Calculate the second-order pure partial derivatives of the global closure topological potential field along the first spatial dimension and the second spatial dimension in the two-dimensional computational domain, respectively, and multiply them to obtain the first product term. The second-order pure partial derivative along the x-direction is denoted as... The second-order pure partial derivative along the y-direction is denoted as The first product term is .

[0050] Calculate the second-order mixed partial derivatives of the global closure topological potential field along the first and second spatial dimensions, and then square these second-order mixed partial derivatives to obtain the second product term. The second-order mixed partial derivatives are denoted as... The second product term is .

[0051] Subtracting the second product term from the first product term extracts the abrupt peak envelope of the absolutely isolated topology, yielding the intrinsic real Gaussian curvature field. The formula for calculating the intrinsic real Gaussian curvature field is as follows: in This is the intrinsic real Gaussian curvature field, in units of , represents the local topological curvature of the potential energy surface at the (x,y) coordinate point at time t. Positive curvature values ​​correspond to convex surface regions, negative values ​​correspond to concave surface regions, zero values ​​correspond to planar regions, and the peak curvature corresponds to the boundary contour of the floating object.

[0052] For discrete pixel data, the second-order pure partial derivative is calculated using the second-order central difference method. The discrete calculation formula for the second-order pure partial derivative in the x-direction is as follows: The discrete formula for calculating the second-order pure partial derivative in the y-direction is: The discrete calculation formula for the second-order mixed partial derivative is as follows: For the boundary pixels of the imaging plane, a copy-fill method is used to expand the boundary, with a fill width of 1 pixel, to ensure that the second-order partial derivative calculation of the boundary pixels can be performed normally.

[0053] The system calculates the global energy expected ground state of the intrinsic real Gaussian curvature field within a two-dimensional computational domain. Using this global energy expected ground state, it performs phase transition collapse determination on the intrinsic real Gaussian curvature field, outputting a phase transition binary mask. Phase transition collapse determination refers to the binarization process of separating the curvature field into background and target regions based on the global energy expected ground state. Based on the critical threshold theory of second-order phase transitions, it achieves automatic segmentation of the floating object region.

[0054] Multiplying the absolute value of the intrinsic real Gaussian curvature field by the intrinsic real Gaussian curvature field itself, and then performing a double surface integral within the two-dimensional computational domain, yields the global real topological total energy. The expression for the global real topological total energy is as follows: ,in It is a two-dimensional spatial computational domain, corresponding to the complete imaging plane region of the target water area; Let be the area differential element in two-dimensional space, corresponding to the physical area of ​​a single pixel of a discrete pixel, with a value of . The unit is .

[0055] The absolute value of the intrinsic real Gaussian curvature field is subjected to a double surface integral in the two-dimensional computational domain to obtain the global reference area envelope. The expression for the global reference area envelope is as follows: .

[0056] Dividing the total real topological energy of the entire domain by the global reference area envelope yields the expected global energy ground state. The expected global energy ground state is the energy-weighted average of the curvature field, serving as the critical phase transition threshold distinguishing background water from floating debris. Its calculation formula is as follows: in Let be the global energy intrinsic expectation ground state at time t, in units of . For discrete pixel data, the dual area integral is implemented by summing pixels. The discrete calculation formula for the total energy of the global real topology is as follows: The discrete calculation formula for the global reference area envelope is as follows: During the calculation, when the value of the global reference area envelope is less than When the global energy intrinsic expectation ground state is set to 0, the error of dividing by 0 is avoided.

[0057] The intrinsic energy difference is obtained by subtracting the global energy of the desired ground state from the intrinsic real Gaussian curvature field. The expression for calculating the intrinsic energy difference is as follows: It represents the degree of deviation of the Gaussian curvature of each pixel from the global eigenstate.

[0058] The sign function polarity is extracted from the intrinsic energy difference to obtain the extracted polarity value. (Sign function) The definition is as follows: when the input value is greater than 0, the output value is 1; when the input value is equal to 0, the output value is 0; and when the input value is less than 0, the output value is -1. The expression for calculating the extracted polarity value is: .

[0059] Add one to the extracted polarity value and divide by two to output a phase transition binary mask containing only zeros and ones. The formula for calculating the phase transition binary mask is: in The phase transition binary mask corresponding to the (x,y) coordinate point at time t is dimensionless and takes only 0 or 1 values; the region with a value of 1 corresponds to the identified floating objects in the river channel, and the region with a value of 0 corresponds to the background water body region.

[0060] Post-processing is performed on the output phase transition binary mask. A 3×3 structuring element is used to perform a closing operation to fill the holes in the mask. Then, a 3×3 structuring element is used to perform an opening operation to remove isolated noise points. For the processed mask, connected components with an area of ​​less than 10 pixels are removed to obtain the final floating object recognition mask.

[0061] Binary mask morphological operations are uniformly adopted. A square structuring element is used for both closing and opening operations to ensure consistent results in hole filling and isolated noise removal. The connected component removal threshold is adaptively adjusted based on resolution: at 1920×1080 resolution, the removal area is less than... The connected components of a pixel, with a culling area of ​​less than 3840×2160 resolution and above. The connected components of pixels can effectively eliminate small-area false detection noise at different resolutions, while preserving the complete outline of real floating objects.

[0062] For the recognition results of consecutive frames, an inter-frame logical AND operation of two consecutive frames is used to eliminate false detections in a single frame and improve the stability of the recognition results.

[0063] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A method for intelligent identification of floating objects in rivers based on multi-device image fusion, applied to a multimodal acquisition system, characterized in that, include: The visible light polarization signal and long-wave infrared absolute temperature of the target water area are obtained in the two-dimensional spatial computational domain and mapped into polarization radiation intensity field and thermal radiation intensity field, respectively. Extract the polarization gradient vector field and the thermodynamic gradient vector field corresponding to the polarization radiation intensity field and the thermal radiation intensity field, respectively. Calculate the first-order partial derivative of the thermal radiation intensity field with respect to time; The two-dimensional spatial divergence of the polarization gradient vector field is calculated to obtain the polarization anomaly divergence field; By combining the first-order partial derivatives with the thermodynamic gradient vector field, the thermodynamic phase velocity field is deduced; The spatial first-order gradient vector of the polarization anomaly divergence field is obtained, and a two-dimensional vector inner product operation is performed with the thermodynamic phase velocity field to obtain the multiphase cross-damping tensor field. Substituting the multiphase cross-damping tensor field into the two-dimensional Poisson equation for reconstruction yields the global closure topological potential field. The curvature of the global closure topological potential field is calculated, and the intrinsic real Gaussian curvature field is extracted. Calculate the global energy eigenvalue expected ground state of the intrinsic real Gaussian curvature field within the two-dimensional computational domain; The intrinsic real Gaussian curvature field is evaluated using the global energy intrinsic expectation ground state, and a phase transition binary mask is output.

2. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The process of acquiring the visible light polarization signal and long-wave infrared absolute temperature of the target water body within a two-dimensional spatial computational domain, and mapping them respectively into a polarization radiation intensity field and a thermal radiation intensity field; and extracting the polarization gradient vector field and thermal gradient vector field corresponding to the polarization radiation intensity field and the thermal radiation intensity field respectively, includes: The Stokes first polarization component and the Stokes second polarization component are obtained as the visible light polarization signal; The square root of the square of the first Stokes polarization component and the square of the second Stokes polarization component is obtained by mapping the polarization radiation intensity field. The thermal radiation intensity field is obtained by multiplying the intrinsic emissivity constant of the water body by the Stefan-Boltzmann constant, dividing the product by pi, and then multiplying it by the fourth power of the long-wave infrared absolute temperature. The polarization radiation intensity field and the thermal radiation intensity field are respectively calculated by taking the first-order partial differential in space, and the polarization gradient vector field and the thermal gradient vector field are respectively extracted.

3. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The step of calculating the two-dimensional spatial divergence of the polarization gradient vector field to obtain the polarization anomaly divergence field includes: By calculating the two-dimensional spatial Laplace partial derivative of the polarization radiation intensity field, the equivalent physical calculation of the two-dimensional spatial divergence of the polarization gradient vector field is realized, and the polarization anomaly divergence field reflecting the intensity of local polarization energy accumulation sources and sinks is obtained by inversion.

4. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The derivation of the thermal phase velocity field by combining the first-order partial derivative with the thermal gradient vector field includes: Calculate the square of the vector magnitude of the thermodynamic gradient vector field; Divide the thermodynamic gradient vector field by the square of the vector magnitude, and multiply the resulting quotient by the negative value of the first-order partial derivative to obtain the thermodynamic phase velocity field used to characterize the evolution and diffusion of latent heat flux.

5. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The process of obtaining the spatial first-order gradient vector of the polarization anomaly divergence field and performing a two-dimensional vector inner product operation with the thermodynamic phase velocity field to obtain the multiphase cross-damping tensor field includes: The first-order partial differential of the polarization anomaly divergence field is obtained to yield the divergence spatial gradient vector field as the first-order spatial gradient vector. The multiphase cross-damping tensor field is obtained by performing a two-dimensional vector dot product operation between the divergence space gradient vector field and the thermodynamic phase velocity field.

6. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The step of substituting the multiphase cross-damping tensor field into the two-dimensional Poisson equation to reconstruct the global closure topological potential field includes: The multiphase cross-damping tensor field is converted into a source charge density field by taking the inverse negative value; A two-dimensional spatial potential energy convergence equation with the source charge density field as the equation correlation term is established as the two-dimensional Poisson equation; Solving the two-dimensional potential energy convergence equation forces the discrete distribution breakpoints to spontaneously fill and close within the two-dimensional computational domain, resulting in the reconstructed global closure topological potential energy field.

7. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The step of solving for the curvature of the global closure topological potential energy field and extracting the eigenvalue Gaussian curvature field includes: Calculate the second-order pure partial derivatives of the global closure topological potential field along the first spatial dimension and along the second spatial dimension in the two-dimensional computational domain, and multiply the two to obtain the first product term. Calculate the second-order mixed partial derivatives of the global closure topological potential field along the first spatial dimension and the second spatial dimension, and then square the second-order mixed partial derivatives to obtain the second product term; Subtracting the second product term from the first product term, the abrupt peak envelope of the absolutely isolated topology is extracted to obtain the intrinsic real Gaussian curvature field.

8. The intelligent identification method for floating objects in rivers based on multi-device image fusion according to claim 1, characterized in that, The calculation of the global energy eigenvalue expected ground state of the intrinsic real Gaussian curvature field within the two-dimensional computational domain; the use of the global energy eigenvalue expected ground state to perform a determination on the intrinsic real Gaussian curvature field, and outputting a phase transition binary mask, includes: Multiply the absolute value of the intrinsic real Gaussian curvature field by the intrinsic real Gaussian curvature field itself, and perform a double surface integral in the two-dimensional computational domain to obtain the total real topological energy of the entire domain. The absolute value of the intrinsic real Gaussian curvature field is subjected to a double surface integral within the two-dimensional computational domain to obtain the global reference area envelope. Divide the total global real topological energy by the global reference area envelope to obtain the global energy intrinsic expected ground state. Subtracting the global energy eigenvalue ground state from the eigenreal Gaussian curvature field yields the eigenenergy difference. The sign function polarity of the intrinsic energy difference is extracted to obtain the extracted polarity value; Add one to the extracted polarity value and divide by two to output the phase transition binary mask containing only zero and one.

Images