Device for tracking and telemetering ship pollution gas emission

Abstract

The invention discloses a device for tracking and telemetering ship pollution gas emission, which is characterized in that a measuring instrument is fixed on a cradle head after being mounted in a casing, the cradle head can rotate, the measuring device and the cradle head are connected to an upper computer, measuring modules are mounted in double layers in the casing, and a moving target multi-feature fusion rapid tracking algorithm runs in the upper computer. The algorithm comprises the following steps: S1, starting tracking, initializing a camera, and obtaining a camera video stream; s2, selecting a tracking target; s3, establishing an appearance model of the tracking target; s4, judging whether the tracking target is lost or not, if not, entering the step S5 to predict the motion trail, and otherwise, entering the step S6 to re-identify the target; s5, motion trail prediction; and S6, target re-identification. According to the invention, a three-layer compact structure design and a moving target multi-feature fusion rapid tracking algorithm are adopted, and the problem that the real-time emission of pollution gas in the ship navigation process is difficult to accurately measure is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of optical monitoring, in particular to a passive remote sensing device based on a Fourier spectrometer. Background Art

[0002] With the advancement of vehicle emission improvement technologies, the number of motor vehicles has increased annually, but pollutant emissions have decreased. However, emissions from ships have increased year by year. Accurately measuring ship emissions has become a key factor in improving air quality. Currently, the focus of ship emissions research is on determining the sulfur content of fuel oil, estimating SO₂ and CO₂ concentrations by monitoring fuel oil. Research at home and abroad primarily focuses on two approaches: drone-based sniffing and optical methods. Optical methods are categorized by application scenario, such as underbridges, shore-based, shipborne, and airborne. Underbridges and shore-based are fixed, passive, and suitable for narrow waterways. Shipborne and airborne are mobile, active, and suitable for open waters. Infrared spectroscopy telemetry systems, with their advantages of strong infrared signatures for most pollutants and long-range, non-contact measurement, have become one of the most promising technologies for measuring ship emissions. However, conventional infrared spectroscopy telemetry can only measure a specific target point or area. In real-world conditions, ships typically move at a constant speed, making accurate measurement of ship emissions under typical operating conditions a challenge.

[0003] Patent application CN109342350A discloses an infrared spectral scanning imaging remote sensing system for pollutant distribution. The system primarily comprises an infrared radiation receiving telescope, a Fourier transform infrared spectrometer, a color zoom camera, a heavy-duty variable-speed pan / tilt head, and a portable computer. While this remote sensing system can be used for emergency monitoring of sudden atmospheric pollution incidents such as leaks, combustion, and explosions, it cannot accurately measure ship emissions under typical operating conditions. Summary of the Invention

[0004] The technical problem to be solved by the present invention is how to achieve accurate measurement of ship exhaust gas under typical working conditions.

[0005] The present invention solves the above-mentioned technical problems through the following technical means: a device for tracking and remote sensing of ship pollutant gas emissions, wherein a measuring instrument is installed in a housing and then fixed on a pan-tilt platform, the pan-tilt platform can rotate, a measuring module and a pan-tilt platform are connected to a host computer, the measuring module is installed in two layers in the housing, and a moving target multi-feature fusion fast tracking algorithm is run in the host computer, the algorithm includes: S1: start tracking, initialize the camera, and obtain the camera video stream; S2: track target selection; S3: establish an appearance model of the tracked target; S4: determine whether the tracked target is lost, if not lost, enter step S5 to predict the motion trajectory, otherwise enter step S6 to re-identify the target; S5: motion trajectory prediction, the predicted motion trajectory x of the ship during uniform speedk is: x k =x k|k-1 +K k ·(z k -H·x k|k-1 ), x k|k-1 is the predicted state, K k is the Kalman gain, z k is the current observation value, S6: Object re-identification.

[0006] As a further optimized technical solution, the measuring instruments include an interferometer, a telescope, a camera, and a high-temperature real-time radiation calibration module. The interferometer, camera, high-temperature blackbody and pan-tilt table are all connected to the host computer. The device for tracking and remote sensing of ship pollutant emissions is designed as a three-layer structure, including two layers inside the casing and a pan-tilt table on the bottom layer. The interferometer, camera and some reflectors are placed on the upper layer of the casing, the telescope and the remaining reflectors are placed in the middle part of the lower layer of the casing, and the electronic control module, acquisition module and power supply units for the interferometer drive control are placed in other positions of the casing.

[0007] As a further optimized technical solution, the camera includes a visible camera and an infrared camera, which are used for image acquisition during the day and at night respectively, and the two automatically switch according to the set brightness threshold.

[0008] As a further optimized technical solution,

[0009] In order to correct the nonlinear effect of the spectrometer response, a high-temperature blackbody is required for spectrum calibration. The specific steps of calibration include:

[0010] The measured spectrum is expressed as:

[0011] S measured (v) = α(v)(L source (v)+γ(v)) 2 +β(v)((L source (v)+γ(v)))(1)

[0012] Where v is the spectral wave number, S measured (v) is the actual measured spectrum of the system, α(v), β(v) represent the nonlinear and linear response functions of the spectrometer, γ(v) is the radiation bias, L source (v) is the target radiance spectrum, which is the target emissivity multiplied by the Planck blackbody radiance spectrum, that is:

[0013] L source (v)=τ·B(T,v) (2)

[0014] Where τ is the target emissivity. For a standard black body, its emissivity τ≈1, and T is the target temperature. Therefore, Equation (1) can be rewritten as:

[0015] S measured (v)=α(v)(B(T,v)+γ(v)) 2 +β(v)((B(T,v)+γ(v))) (3)

[0016] The host computer software queries the high-temperature blackbody temperature in real time. When the temperature reaches the set temperature, the spectrum is triggered to collect the high-temperature blackbody spectrum. After the three temperature gradient spectra (B(T1,v), B(T2,v), and B(T3,v) are collected, the high-temperature blackbody spectrum is read by the calibration software built into the host computer to establish a nonlinear calibration model, which includes:

[0017] S measures (T1,v)=α(v)(B(T1,v)+γ(v)) 2 +β(v)((B(T1,v)+γ(v))) (4)

[0018] S meaaures (T2,v)=α(v)(B(T2,v)+γ(v)) 2 +β(v)((B(T2,v)+γ(v))) (5)

[0019] S measures (T3,v)=α(v)(B(T3,v)+γ(v)) 2 +β(v)((B(T3,v)+γ(v))) (6) Simplify the above formula by algebraic elimination, let z i =B(T i ,v)+γ(v), then:

[0020]

[0021] The numerical optimization method is used to solve the problem and construct the total residual square sum function:

[0022]

[0023] The process of solving α(v), β(v), and γ(v) is the process of minimizing the parameter function. The nonlinear least squares method is used to solve the equation. The first step is to select the initial value, and use the linear scaling coefficients as the initial values ​​of β(v) and γ(v).

[0024]

[0025] Substitute Equation (10) into Equation (7) to obtain the initial value of α(v) and calculate the current residual:

[0026] r1=[S measures (T1,v)-α(v)(B(T1,v)+γ(v)) 2 -β(v)(B(T1,v)+γ(v)) 2 (11) r2=[s measures (T2,v)-α(v)(B(T2,v)+γ(v)) 2 -β(v)(B(T2,v)+γ(v)) 2 ] (12)r3=[S measures (T3,v)-α(v)(B(T3,v)+γ(v)) 2 -β(v)(B(T3,v)+γ(v)) 2 ] (13)

[0027] Then the residual vector is:

[0028] r=[r1,r2,r3] T (14)

[0029] For the convenience of expression, let θ = (α(v), β(v), γ(v)) and calculate the Jacobian matrix, which is:

[0030]

[0031] The parameter update amount is calculated by the Levenberg-Marquardt method:

[0032] Δθ=-(J T J+λI) -1 J T r (16)

[0033] Where λ is the damping factor, and the update parameters are:

[0034] θ new =θ old +Δθ (17)

[0035] Substitute equation (17) into equation (8), if:

[0036] R new <R old

[0037] Then accept the update and appropriately reduce λ, otherwise increase λ. When the maximum number of iterations is reached or the objective function changes <0.1, the α(v), β(v), and γ(v) obtained at this time are the final values. The calibrated target spectral radiance is:

[0038]

[0039] As a further optimized technical solution, the host computer queries the temperature of the high-temperature blackbody in real time. When the temperature reaches the set temperature, the spectrum is triggered to collect the high-temperature blackbody spectrum. The specific steps are: the high-temperature blackbody is connected to the casing through an electric worm. Under normal circumstances, the worm contracts and the high-temperature blackbody collects the target plume spectrum. When the device needs to be calibrated, the user manually inputs or uses the default calibration temperature gradient to start calibration. The electric worm is extended to the infrared light path between the telescope and the reflector, and the high-temperature blackbody begins to heat up. The host computer queries the temperature of the high-temperature blackbody in real time. When the temperature reaches the set temperature, the spectrum is triggered to collect the high-temperature blackbody spectrum. After the temperature gradient spectrum is completed, the calibration software built into the host computer reads the high-temperature blackbody spectrum to establish a nonlinear calibration model.

[0040] As a further optimized technical solution, S3: Establishing an apparent model of the tracking target specifically includes:

[0041] Taking into account the characteristics of the ship, the target directional gradient histogram and edge contour features are extracted as the appearance model. The canny edge detection operator is used to process it to enhance the boundary between the target and the background. Then the target intensity map is converted into a binary image. The Sobel operator is used to extract the target color histogram as one of the features. A weighted fusion of the edge contour features and color features is established with the peak sidelobe ratio as the weight. The two are fused through the channel layer to establish an appearance model of adaptive feature fusion.

[0042] As a further optimized technical solution, S5: Motion trajectory prediction specifically includes:

[0043] S51. Target motion trajectory prediction: Kalman filter is used for motion trajectory prediction. When the state space model is established, considering that the ship is in a uniform speed state in actual conditions, the motion trajectory prediction model is:

[0044]

[0045] The observation models are:

[0046]

[0047] where v k is the observation noise, H is the observation matrix;

[0048] Assuming that the target initial state x0 and the initial covariance matrix P0 are used to represent the detection position, velocity and initial estimation uncertainty of the target initial state, the predicted state and covariance are:

[0049] x k|k-1 =F·x k-1

[0050] P k|k-1=F·p k-1 ·F T +Q

[0051]

[0052] Where Q is the tracking process noise covariance matrix, T represents the transpose, Δt represents the time step between adjacent moments, and x k|k-1 is the predicted state, P k|k-1 is the prediction covariance;

[0053] The predicted trajectory of the ship during uniform speed is x k is: x k =x k|k-1 +K k ·(z k -H·x k|k-1 );

[0054] S52, calculating the pan / tilt motion angle;

[0055] S53, issuing movement instructions;

[0056] S54: Trigger spectrum acquisition, and then return to step S3.

[0057] As a further optimized technical solution, K k Calculated by the following formula:

[0058] K k =P k|k-1 ·H T ·(H·P k|k-1 ·H T +R) -1

[0059] R is the covariance matrix of observation noise;

[0060] Calculating the gimbal motion angle specifically includes: The gimbal motion angle information is calculated using the following formula:

[0061]

[0062] Among them, (x0, y0) and (x1, y1) are the pixel coordinates of the target center of mass in the previous and next frames respectively, α and β are the pitch angle and altitude angle of the gimbal respectively, and are the upper left and lower right pixel coordinates of the target in each frame respectively. When Δα>0, the gimbal moves to the right, otherwise it moves to the left; when Δβ>0, the gimbal rotates upward, otherwise it rotates downward.

[0063] As a further optimized technical solution, when the target cannot be found in the video stream due to occlusion or other reasons, it is necessary to re-identify the object appearing in the video stream. In the target re-identification, a strategy of separating detection and re-identification is adopted.

[0064] As a further optimized technical solution, S6: Target Re-ID specifically includes:

[0065] S61, first use the object detection network to obtain the position and bounding box of the object of interest in the video;

[0066] S62, cropping the detection result according to the bounding box and inputting it into the re-identification feature extraction network to obtain the re-identification features of the target;

[0067] S63, calculating the similarity between the appearance model and the re-identification feature model;

[0068] S64, using data association matching to calculate the current result and the existing trajectory IOU;

[0069] S65: When the IOU values ​​of the two are close to 1, they are considered to be the same target and the process returns to step S3; otherwise, the process returns to step S61 and the target re-identification step is repeated.

[0070] The advantages of the present invention are that, in response to the complex environment of typical ship operating conditions, susceptibility to obstruction and interference from other targets, and the difficulty of measuring low concentrations caused by the easy diffusion of exhaust gases, this device for tracking and remotely measuring ship pollutant emissions adopts a three-layer compact structure design and a mobile target multi-feature fusion rapid tracking algorithm, solving the problem of the difficulty in accurately measuring real-time pollutant emissions during ship navigation. The three-layer structure design not only effectively reduces the weight and volume of the device, but also reduces tracking errors caused by heavy loads and uneven device shapes. The fusion of multiple features and a re-identification mechanism enables rapid and accurate tracking of target ships in complex interference environments on the sea surface, realizing real-time monitoring of characteristic emission gases under typical ship operating conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 This is an overall structural diagram of a device for tracking and remotely measuring ship pollutant emissions according to an embodiment of the present invention;

[0072] Figure 2 4 is a flow chart of a multi-feature fusion fast tracking algorithm for a moving target in an embodiment of the present invention. DETAILED DESCRIPTION

[0073] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0074] To accurately measure the various polluting gases emitted by ships under typical operating conditions, it is necessary to achieve targeted tracking of target ships. Based on this, the present invention provides a device for tracking and remotely measuring ship-borne pollutant emissions. This device features real-time tracking capabilities. It uses a built-in camera to select a target vessel and then uses a tracking algorithm to track it. It then uses an infrared system to acquire the infrared radiation spectrum of the target gas. Furthermore, it uses a built-in high-temperature calibration function model and a quantitative analytical algorithm to achieve remote measurement of ship emissions under typical operating conditions.

[0075] This device for tracking and remotely monitoring ship emissions utilizes a three-layer compact design and a fast tracking algorithm that fusions multiple features for moving targets. This solves the challenge of accurately measuring real-time emissions during ship navigation. The following describes these two approaches.

[0076] (1) Three-layer compact hardware structure design

[0077] See Figure 1 In terms of hardware, the ship's gas emissions tracking and remote sensing system primarily consists of a vibration-resistant interferometer, a high-throughput telescope, a camera, a high-temperature blackbody, a 360° high-precision pan-tilt head (PTZ), and a host computer. The interferometer, telescope, camera, and high-temperature blackbody are housed in a housing, the bottom of which is secured to the PTZ. The interferometer, camera, high-temperature blackbody, and PTZ are all connected to the host computer. The cameras include a starlight visible camera and an infrared camera. The starlight visible camera captures visible images of the target measurement area, while the infrared camera captures infrared images of the target measurement area. The starlight visible camera and the infrared camera are used for daytime and nighttime image acquisition, respectively, and automatically switch between them based on a set brightness threshold. The telescope receives infrared radiation signals, which are then reflected three times by a reflector before entering the interferometer to perform infrared spectral measurements. The PTZ rotates the entire telemetry system. The host computer processes infrared spectral data, stores visible and infrared images, calibrates the spectrum, and controls the PTZ.

[0078] To meet the requirements of long-term, continuous measurement of ship tracking in complex scenarios, the housing of the device used for tracking and remotely measuring ship pollutant emissions features a compact, two-layer structure. The interferometer, visible camera, and some reflectors are placed in the upper layer of the housing, while the telescope and the remaining reflectors are placed in the center of the lower layer. The telescope utilizes a Cassegrain structure. Elsewhere in the housing are located the electronic control module for driving the interferometer, the acquisition module, and the power supply units for each module. The acquisition module includes an infrared detector, preamplifier, and digital acquisition card. The real-time tracking module for moving targets is comprised of a starlight visible camera, infrared camera, pan / tilt, and the motion tracking algorithm built into the host computer.

[0079] In terms of the system hardware structure, the overall design of the device for tracking and remote sensing of ship pollutant emissions is a three-layer compact structure, as shown in Figure (1). The top layer is the core interferometer and reflector and other optical components, the middle layer is the Cassegrain telescope for receiving the infrared red line of the object, and the two sides are electronic modules. The bottom layer is the adjustment structure and the connected mechanical components, namely the pan-tilt platform. Its main structure and advantages are: by adopting the optical path folding design to reflect the telescope receiving optics to the upper interferometer, the optical path length is minimized, and the space layout is compressed while ensuring optical stability. In addition, the electronic control unit, power supply unit and acquisition imaging of the circuit part adopt a modular design, and the modules are connected through standard interfaces, which effectively improves space utilization and maintainability. Finally, by integrating the optical and mechanical base, the coupling layout of the optical path and structural space and the integrated adjustment structure design, the component stacking and interface space are reduced, and the system optical, mechanical, electronic and other multifunctional components are compactly integrated.

[0080] The compact structural design effectively reduces the size and weight of the system, reduces the impact of end-positioning deviation amplification and tracking drift caused by tracking large-load equipment, and reduces the operating power consumption of the equipment. Secondly, the compact three-layer structural design basically conforms to the structure of a symmetrical object, with its center of mass and center of gravity coinciding, and the moment of inertia evenly distributed. During long-term continuous tracking of uniform motion, the motor can maintain a stable speed without frequent adjustment of the drive current, reducing energy loss caused by dynamic torque fluctuations, and effectively overcoming motor loss of step, stalling, and bearing wear caused by asymmetric objects. In addition, the compact design reduces the exposure of spectral components to a certain extent, avoids corrosion in harsh environments during long-term telemetry of ships, improves the robustness of the equipment, and achieves long-term stable measurement of equipment and accuracy of ship tracking in complex environments.

[0081] (2) Nonlinear radiation calibration

[0082] An infrared spectrometer consists of an interferometer, an infrared detector, and electronic components. The nonlinear effects of the interferometer, the detector, and the instrument's linear function are all responsible for these effects. To correct the nonlinear effects of the spectrometer's response, a high-temperature blackbody is required for spectral calibration. A nonlinear calibration model based on a multi-point temperature gradient blackbody radiation spectrum is proposed. The specific steps include:

[0083] S11: By establishing a quadratic function model for correction, the measured spectrum can be expressed as:

[0084] S measured (v) = α(v)(L source (v)+γ(v)) 2 +β(v)((L source (v)+γ(v)))(1)where v is the spectral wave number, S measured(v) is the actual measured spectrum of the system, α(v), β(v) represent the nonlinear and linear response functions of the spectrometer, γ(v) is the radiation bias, L source (v) is the target radiance spectrum, which is the target emissivity multiplied by the Planck blackbody radiance spectrum, that is:

[0085]

[0086] Where τ is the target emissivity, h is Planck's constant 6.626*10 -34 J·s, c is the speed of light 3*10 8 m / s,K B is the Boltzmann constant 1.381*10 -23 J / K, T is the absolute temperature (K). For a standard black body (complete radiation, complete absorption, radiation characteristics are only related to temperature), its emissivity τ≈1, T is the target temperature, so Equation (1) can be rewritten as:

[0087] S measured (v)=α(v)(B(T,v)+γ(v)) 2 +β(v)((B(T,v)+γ(v))) (3)

[0088] The high-temperature blackbody is connected to the housing through an electric worm. Under normal circumstances, the worm is retracted and the high-temperature blackbody collects the target plume spectrum. When the device needs to be calibrated, the user manually enters or uses the default low, medium, and high calibration temperature gradients (30°C, 80°C, and 130°C) to start calibration. The electric worm is extended to the infrared light path between the telescope and the reflector, and the high-temperature blackbody begins to heat up. The host computer software queries the high-temperature blackbody temperature in real time. When the temperature reaches the set temperature, the acquisition card is triggered to collect the high-temperature blackbody spectrum. After the three temperature gradient spectra (B(T1,v), B(T2,v), and B(T3,v)) are collected, the high-temperature blackbody spectrum is read by the calibration software built into the host computer to establish a nonlinear calibration model, which is:

[0089] S measures (T1,v)=α(v)(B(T1,v)+γ(v)) 2 +β(v)((B(T1,v)+γ(v))) (4)

[0090] S measures (T2,v)=α(v)(B(T2,v)+γ(v)) 2 +β(v)((B(T2,v)+γ(v))) (5)

[0091] S measures (T3,v)=α(v)(B(T3,v)+γ(v)) 2+β(v)((B(T3,v)+γ(v))) (6)

[0092] Simplify the above formula by algebraic elimination, let z i =B(T i ,v)+γ(v), then:

[0093]

[0094] Because z i Dependence on γ(v), the analytical solution is still highly nonlinear, so a numerical optimization method is used to solve it and construct the total residual square sum function:

[0095]

[0096] The process of solving α(v), β(v), and γ(v) is the process of minimizing the parameter function. The nonlinear least squares method is used to solve the equation. The first step is to select the initial value, and use the linear scaling coefficients as the initial values ​​of β(v) and γ(v).

[0097]

[0098] Substitute Equation (10) into Equation (7) to obtain the initial value of α(v) and calculate the current residual:

[0099] r1=[S measures (T1,v)-α(v)(B(T1,v)+γ(v)) 2 -β(v)(B(T1,v)+γ(v)) 2 (11) r2=[S measures (T2,v)-α(v)(B(T2,v)+γ(v)) 2 -β(v)(B(T2,v)+γ(v)) 2 ] (12)r3=[S measures (T3,v)-α(v)(B(T3,v)+γ(v)) 2 -β(v)(B(T3,v)+γ(v)) 2 ] (13)

[0100] Then the residual vector is:

[0101] r=[r1,r2,r3] T (14)

[0102] For the convenience of expression, let θ = (α(v), β(v), γ(v)) and calculate the Jacobian matrix, which is:

[0103]

[0104] The parameter update amount is calculated by the Levenberg-Marquardt method:

[0105] Δθ=-(J T J+λI) -1 J T r (16)

[0106] where λ is the damping factor.

[0107] The updated parameters are:

[0108] θ new =θ old +Δθ (17)

[0109] Substitute equation (17) into equation (8), if:

[0110] R n3w <R old

[0111] Then accept the update and appropriately reduce λ, otherwise increase λ. When the maximum number of iterations is reached or the objective function changes <0.1, the α(v), β(v), and γ(v) obtained at this time are the final values. The calibrated target spectral radiance is:

[0112]

[0113] The measured spectrum is radiometrically calibrated using formula (18) to reduce the influence of spectrometer fluctuations on the measured spectrum.

[0114] To ensure the authenticity and scientific validity of acquired spectral data, radiometric calibration, a key step in remote sensing system data processing, is essential. Radiometric calibration accurately converts the raw electrical signals received by the detector (such as voltage or digital counts) into physically meaningful radiant brightness or spectral radiance, enabling quantitative analysis of remote sensing data.

[0115] The calibrated data can not only accurately invert various gas components in the atmosphere (such as CO2, CH4, NO x , SO2, etc.) can also be used to infer the temperature distribution, thermal anomaly characteristics, or pollutant diffusion in the target area, providing strong support for environmental monitoring and pollution source identification and monitoring. Furthermore, calibration processing ensures the consistency and comparability of telemetry data across time, equipment, and observation conditions, laying the foundation for establishing a standardized data analysis system and conducting long-term trend analysis.

[0116] Furthermore, accurate radiometric calibration can significantly improve the sensitivity and resolution of Fourier transform infrared remote sensing systems, enhance the ability to identify the infrared signatures of target materials, and reduce the likelihood of misidentification and missed detection. In engineering applications, calibration data can also be used to train, validate, and optimize inversion models, driving the transition from empirical algorithms to physical models and enhancing the overall intelligence of remote sensing systems. In short, radiometric calibration is not only fundamental for ensuring data quality but also a prerequisite for achieving high-precision, highly reliable remote sensing analysis. It plays a significant role in promoting the in-depth application of passive Fourier transform infrared spectroscopy in various fields, including scientific research, environmental governance, and national defense security.

[0117] (3) Fast tracking algorithm for moving targets with multi-feature fusion

[0118] The above-mentioned visible camera, infrared camera, gimbal and the built-in mobile tracking algorithm of the host computer constitute a moving target multi-feature fusion fast tracking module.

[0119] In the target tracking of ships, the main process is:

[0120] S1: Start tracking. Initialize the visible or infrared camera and obtain the camera video stream.

[0121] S2: Tracking target selection: The user manually selects the tracking target and extracts a sample block x of size M*N with the target as the center;

[0122] S3: Establish an appearance model for tracking targets: Considering the characteristics of the ship, the target directional gradient histogram (HOG) and edge contour features are extracted as the appearance model. The canny edge detection operator is used to process the target to strengthen the boundary between the target and the background. The target intensity map is then converted into a binary image to make the contrast between the ship target and the background more obvious. To address the impact of target deformation on tracking filter drift during motion, the Sobel operator is used to extract the target's color histogram as one of the features. A weighted fusion of edge contour features and color features is established with the peak sidelobe ratio as the weight. The two are fused through the channel layer to establish an appearance model with adaptive feature fusion.

[0123] S4: Determine whether the tracked target is lost. If not, proceed to step S5 to perform motion trajectory prediction. Otherwise, proceed to step S6 to perform target re-identification.

[0124] S5: Motion trajectory prediction, including:

[0125] S51. Target motion trajectory prediction: Kalman filter is used for motion trajectory prediction. When the state space model is established, considering that the ship is in a uniform speed state in actual conditions, the motion trajectory prediction model is:

[0126]

[0127] where x k is the predicted trajectory of the ship during uniform speed, p x ,v x is the position and velocity of the ship in the x direction, p y ,v y In terms of position and velocity in the y direction, the observation model is:

[0128]

[0129] where z k is the current observation value, v k is the observation noise, and H is the observation matrix.

[0130] Assuming that the target initial state x0 and the initial covariance matrix P0 are used to represent the detection position, velocity and initial estimation uncertainty of the target initial state, the state and covariance predicted at time k are:

[0131] x k|k-1 =F·x k-1

[0132] P k|k-1 =F·p k-1 ·F T +Q (21)

[0133]

[0134] Where Q is the tracking process noise covariance matrix, F is the state transfer matrix, x k-11 is the state vector at time k-1, P k-1 is the covariance matrix at time k-1. T represents the transpose, Δt represents the time step between adjacent moments, x k|k-1 is the predicted state at time k, P k|k-1 is the predicted covariance at time k. The predicted trajectory x of the ship during the uniform speed process k for:

[0135] x k =x k|k-1 +K k ·(z k -H·x k|k-1 ) (twenty three)

[0136] K k is the Kalman gain, which is calculated as follows:

[0137] K k =P k|k-1 ·H T ·(H·P k|k-1 ·HT +R) -1 (twenty four)

[0138] R is the covariance matrix of observation noise;

[0139] S52: Calculate the pan-tilt motion angle. The pan-tilt motion angle information is calculated using the following formula (25).

[0140]

[0141] Among them, (x0, y0) and (x1, y1) are the pixel coordinates of the target center of mass in the previous and next frames respectively, α and β are the pitch angle and altitude angle of the gimbal respectively, and are the upper left and lower right pixel coordinates of the target in each frame respectively. When Δα>0, the gimbal moves to the right, otherwise it moves to the left; when Δβ>0, the gimbal rotates upward, otherwise it rotates downward.

[0142] S53, issuing movement instructions;

[0143] S54, triggering spectrum acquisition, and then returning to step S3;

[0144] S6: Target Re-identification: When the target cannot be found in the video stream due to occlusion or other reasons, the object in the video stream needs to be re-identified. In target re-identification, a detection and re-identification separation strategy is adopted, including:

[0145] S61, first use the object detection network to obtain the position and bounding box of the object of interest in the video;

[0146] S62, cropping the detection result according to the bounding box and inputting it into the re-identification feature extraction network to obtain the re-identification features of the target;

[0147] S63, calculating the similarity between the appearance model and the re-identification feature model;

[0148] S64, using data association matching to calculate the current result and the existing trajectory IOU;

[0149] S65: When the IOU values ​​of the two are close to 1, they are considered to be the same target and the process returns to step S3; otherwise, the process returns to step S61 and the target re-identification step is repeated.

[0150] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A device for tracking and remotely measuring ship pollutant emissions, characterized by: The measuring instrument is installed in a housing and then fixed on a pan-tilt platform. The pan-tilt platform can rotate. The measuring device and the pan-tilt platform are connected to the host computer. The measuring module is installed in two layers in the housing. The host computer runs a moving target multi-feature fusion fast tracking algorithm, which includes: S1: start tracking, initialize the camera, and obtain the camera video stream; S2: track target selection; S3: establish an appearance model of the tracked target; S4: determine whether the tracked target is lost. If not, enter step S5 to predict the motion trajectory. Otherwise, enter step S6 to re-identify the target; S5: motion trajectory prediction, the predicted motion trajectory x of the ship during the uniform speed process k is: x k =x k|k-1 +K k ·(z k -H·x k|k-1 ), x k|k-1 is the predicted state, K k is the Kalman gain, z k is the current observation value, S6: Object re-identification.

2. The device for tracking and remotely measuring ship pollutant emissions according to claim 1, characterized in that: The measuring instruments include an interferometer, a telescope, a camera, and a high-temperature real-time radiation calibration module. The interferometer, camera, high-temperature blackbody and pan-tilt table are all connected to the host computer. The device for tracking and remote sensing of ship pollutant emissions is designed as a three-layer structure, including two layers inside the casing and a pan-tilt table at the bottom. The interferometer, camera and some reflectors are placed in the upper layer of the casing, the telescope and the remaining reflectors are placed in the middle part of the lower layer of the casing, and the electronic control module, acquisition module and power supply units for the interferometer drive control are placed in other positions of the casing.

3. The device for tracking and remotely measuring ship pollutant emissions according to claim 1, characterized in that: The camera includes a visible camera and an infrared camera. The visible camera and the infrared camera are used for image acquisition during the day and at night respectively. The two cameras are automatically switched according to the set brightness threshold.

4. The device for tracking and remotely measuring ship pollutant emissions according to claim 1, characterized in that: In order to correct the nonlinear effect of the spectrometer response, a high-temperature blackbody is required for spectrum calibration. The specific steps of calibration include: The measured spectrum is expressed as: S measured (v)=α(v)(L source (v)+γ(v)) 2 +β(v)((L source (v)+γ(v)))(1) Where v is the spectral wave number, S measured (v) is the actual measured spectrum of the system, α(v), β(v) represent the nonlinear and linear response functions of the spectrometer, γ(v) is the radiation bias, L source (v) is the target radiance spectrum, which is the target emissivity multiplied by the Planck blackbody radiance spectrum, that is: L source (v)=τ·B(T,v) (2) Where τ is the target emissivity. For a standard black body, its emissivity τ≈1, and T is the target temperature. Therefore, Equation (1) can be rewritten as: S measured (v)=α(v)(B(T,v)+γ(v)) 2 +β(v)((B(T,v)+γ(v))) (3) The host computer software queries the high-temperature blackbody temperature in real time. When the temperature reaches the set temperature, the spectrum is triggered to collect the high-temperature blackbody spectrum. After the three temperature gradient spectra (B(T1,v), B(T2,v), B(T3,v)) are collected, the high-temperature blackbody spectrum is read by the calibration software built into the host computer to establish a nonlinear calibration model, which includes: S measures (T1,v)=α(v)(B(T1,v)+γ(v)) 2 +β(v)((B(T1,v)+γ(v))) (4) S measures (T2,v)=α(v)(B(T2,v)+γ(v)) 2 +β(v)((B(T2,v)+γ(v))) (5) S measures (T3,v)=α(v)(B(T3,v)+γ(v)) 2 +β(v)((B(T3,v)+γ(v))) (6) Simplify the above formula by algebraic elimination, let z i =B(T i ,v)+γ(v), then: The numerical optimization method is used to solve the problem and construct the total residual square sum function: The process of solving α(v), β(v), and γ(v) is the process of minimizing the parameter function. The nonlinear least squares method is used to solve the equation. The first step is to select the initial value, and use the linear scaling coefficients as the initial values ​​of β(v) and γ(v). Substitute Equation (10) into Equation (7) to obtain the initial value of α(v) and calculate the current residual: r1=[S measures (T1,v)-α(v)(B(T1,v)+γ(v)) 2 -β(v)(B(T1,v)+γ(v)) 2 (11) r2=[S measures (T2,v)-α(v)(B(T2,v)+γ(v)) 2 -β(v)(B(T2,v)+γ(v)) 2 ](12) r3=[S measures (T3,v)-α(v)(B(T3,v)+γ(v)) 2 -β(v)(B(T3,v)+γ(v)) 2 ] (13) Then the residual vector is: r＝[r1,r2,r3] T (14) For the convenience of expression, let θ = (α(v), β(v), γ(v)) and calculate the Jacobian matrix, which is: The parameter update amount is calculated by the Levenberg-Marquardt method: Δθ=-(J T J+λI) -1 J T r (16) Where λ is the damping factor, and the update parameters are: i new =θ old +Δθ (17) Substitute equation (17) into equation (8), if: R new <R old Then accept the update and appropriately reduce λ, otherwise increase λ. When the maximum number of iterations is reached or the objective function changes <0.1, the α(v), β(v), and γ(v) obtained at this time are the final values. The calibrated target spectral radiance is:

5. The device for tracking and remotely measuring ship pollutant emissions according to claim 4, characterized in that: The host computer queries the temperature of the high-temperature blackbody in real time. When the temperature reaches the set temperature, the spectrum is triggered to collect the high-temperature blackbody spectrum. The specific steps are as follows: the high-temperature blackbody is connected to the casing through an electric worm. Under normal circumstances, the worm contracts and the high-temperature blackbody collects the target plume spectrum. When the device needs to be calibrated, the user manually inputs or uses the default calibration temperature gradient to start calibration. The electric worm is extended to the infrared light path between the telescope and the reflector, and the high-temperature blackbody begins to heat up. The host computer queries the temperature of the high-temperature blackbody in real time. When the temperature reaches the set temperature, the spectrum is triggered to collect the high-temperature blackbody spectrum. After the temperature gradient spectrum is completed, the calibration software built into the host computer reads the high-temperature blackbody spectrum to establish a nonlinear calibration model.

6. The device for tracking and remotely measuring ship pollutant emissions according to claim 1, wherein: S3: establishing an apparent model of the tracking target specifically comprises: Taking into account the characteristics of the ship, the target directional gradient histogram and edge contour features are extracted as the appearance model. The canny edge detection operator is used to process it to enhance the boundary between the target and the background. Then the target intensity map is converted into a binary image. The Sobel operator is used to extract the target color histogram as one of the features. A weighted fusion of the edge contour features and color features is established with the peak sidelobe ratio as the weight. The two are fused through the channel layer to establish an appearance model of adaptive feature fusion.

7. The device for tracking and remotely measuring ship pollutant emissions according to claim 1, characterized in that: S5: Motion trajectory prediction specifically includes: S51. Target motion trajectory prediction: Kalman filter is used for motion trajectory prediction. When the state space model is established, considering that the ship is in a uniform speed state in actual conditions, the motion trajectory prediction model is: The observation models are: z k =Hx k +v k , where v k is the observation noise, H is the observation matrix; Assuming that the target initial state x0 and the initial covariance matrix P0 are used to represent the detection position, velocity and initial estimation uncertainty of the target initial state, the state and covariance predicted at time k are: x k|k-1 =F·x k-1 p k|k-1 =F·p k-1 ·F T +Q Where Q is the tracking process noise covariance matrix, T represents the transpose, Δt represents the time step between adjacent moments, and x k|k-1 is the predicted state, P k|k-1 is the prediction covariance; The predicted trajectory of the ship during uniform speed is x k is: x k =x k|k-1 +K k ·(z k -H·x k|k-1 ); S52, calculating the pan / tilt motion angle; S53, issuing movement instructions; S54: Trigger spectrum acquisition, and then return to step S3.

8. The device for tracking and remotely measuring ship pollutant emissions according to claim 7, characterized in that: K k Calculated by the following formula: K k =Pk |k-1 ·H T ·(H·P k|k-1 ·H T +R) -1 R is the covariance matrix of observation noise; Calculating the gimbal motion angle specifically includes: The gimbal motion angle information is calculated using the following formula: Among them, (x0, y0) and (x1, y1) are the pixel coordinates of the target center of mass in the previous and next frames respectively, α and β are the pitch angle and altitude angle of the gimbal respectively, and are the upper left and lower right pixel coordinates of the target in each frame respectively. When Δα>0, the gimbal moves to the right, otherwise it moves to the left; when Δβ>0, the gimbal rotates upward, otherwise it rotates downward.

9. The device for tracking and remotely measuring ship pollutant emissions according to claim 1, characterized in that: When the target cannot be found in the video stream due to occlusion or other reasons, it is necessary to re-identify the object appearing in the video stream. In the target re-identification, a detection and re-identification separation strategy is adopted.

10. The device for tracking and remotely measuring ship pollutant emissions according to claim 9, It is characterized in that: S6: target re-identification specifically includes: S61, first use the object detection network to obtain the position and bounding box of the object of interest in the video; S62, cropping the detection result according to the bounding box and inputting it into the re-identification feature extraction network to obtain the re-identification features of the target; S63, calculating the similarity between the appearance model and the re-identification feature model; S64, using data association matching to calculate the current result and the existing trajectory IOU; S65: When the IOU values ​​of the two are close to 1, they are considered to be the same target and the process returns to step S3; otherwise, the process returns to step S61 and the target re-identification step is repeated.

Images