Self-adaptive environment image processing method for industrial gas detection

Abstract

The invention discloses a self-adaptive environment image processing method for industrial gas detection, which relates to the field of industrial gas detection and comprises the following steps: extracting feature points of a current frame by utilizing a feature point detection algorithm according to image frames of an industrial monitoring video stream, and acquiring coordinates of feature points between frames by combining an optical stream tracking technology; generating affine transformation parameters, performing transformation decomposition on the coordinates of the feature points by using the affine transformation parameters, and extracting pose change parameters to obtain an image motion track; predicting an image motion track by using an image track processing algorithm, and comparing the predicted image motion track with a real-time image motion track generated by the current frame; and optimizing the affine transformation parameters, and carrying out frame-by-frame processing on the video stream by using the optimized affine transformation parameters. Through feature point detection and an optical flow tracking algorithm, image motion information in an industrial monitoring video can be extracted, and industrial application requirements of high real-time performance and high stability are met.

Description

Technical Field

[0001] The present invention relates to the field of industrial gas detection. Specifically, it relates to an adaptive environment image processing method for industrial gas detection. Background Art

[0002] With the continuous improvement of industrial automation and intelligence levels, industrial gas detection plays an increasingly important role in key industries such as petrochemical, electric power, and emergency safety. Traditional gas detection methods mainly rely on point sensors, laser rangefinders, or infrared spectroscopy analysis equipment. Although these methods can provide high-precision data in some static and constant environments, in complex and dynamic industrial sites, their adaptability, coverage, and real-time performance still have many limitations.

[0003] In recent years, to improve detection efficiency and intelligence levels, video surveillance and image recognition technologies based on image perception have gradually become a trend. By using advanced devices such as infrared thermal imaging and spectral cameras, non-contact recognition and early warning functions for gas leakage and its diffusion state have been achieved. However, in unattended and continuously operating industrial scenarios, such systems need to withstand long-term operating pressures and cope with a series of complex environmental interference factors, such as periodic rotational inspections of equipment, switching between cooling / non-cooling modes, image blurring caused by structural vibrations, contrast changes, etc. These problems affect the stability and reliability of the original image data, and thus weaken the accuracy of subsequent image processing and gas recognition.

[0004] Currently, most image processing methods usually rely on fixed parameter settings and lack the ability to self-adjust according to changes in the working environment. This results in their difficulty in effectively suppressing interference, correcting motion offsets, or improving image quality, and thus prone to false alarms and missed detections in long-term continuous detection tasks, which is not conducive to the long-term stable operation of the system.

[0005] Regarding the problems in the related art, no effective solutions have been proposed yet. Summary of the Invention

[0006] Regarding the problems in the related art, the present invention proposes an adaptive environment image processing method for industrial gas detection to overcome the above-mentioned technical problems existing in the existing related technologies.

[0007] To this end, the specific technical solution adopted by the present invention is as follows: An adaptive environment image processing method for industrial gas detection, the method includes: S1. According to the image frames of the industrial monitoring video stream, use the feature point detection algorithm to extract the feature points of the current frame, and combine the optical flow tracking technology to obtain the feature point coordinates between frames; S2. Generate affine transformation parameters based on the obtained feature point coordinates, and decompose the transformation of the feature point coordinates using the affine transformation parameters to extract pose change parameters, so as to obtain the image motion trajectory; S3. Use the image trajectory processing algorithm to predict the image motion trajectory, and compare the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame; S4. Optimize the affine transformation parameters based on the comparison result, and use the optimized affine transformation parameters to process the video stream frame by frame.

[0008] Optionally, according to the image frames of the industrial monitoring video stream, use the feature point detection algorithm to extract the feature points of the current frame, and combine the optical flow tracking technology to obtain the feature point coordinates between frames, including: S11. In the image frames of the industrial monitoring video stream, use the feature point detection algorithm based on blob detection to extract features from the current frame and generate a set of candidate feature points; S12. Use the feature point response threshold mechanism to screen the set of candidate feature points, and combine the preset inertia filtering ratio threshold to establish a set of feature points; S13. Use the optical flow tracking technology to track the feature points in the set of feature points frame by frame, and combine the adaptive window size adjustment mechanism to optimize the tracking accuracy to obtain the corresponding feature point coordinates between the current frame and the previous frame.

[0009] Optionally, use the feature point response threshold mechanism to screen the set of candidate feature points, and combine the preset inertia filtering ratio threshold to establish a set of feature points, including: S121. Based on the generated set of candidate feature points, use the multi-scale corner detection algorithm to perform response analysis in the image spatial domain, and calculate the response values of each feature point in the set of candidate feature points in different scale spaces based on the analysis results; S122. Divide the image into several sub-grid regions, set the response threshold according to the mean and standard deviation of the response values in each sub-grid region, and use the response threshold to perform a preliminary screening of the set of candidate feature points to establish an initial set of feature points; S123. Calculate the ratio of the response value of each feature point in the initial set of feature points to the maximum response value in the corresponding neighborhood, and combine the preset inertia filtering ratio threshold to perform a secondary screening of the initial set of feature points to establish a set of feature points.

[0010] Optionally, calculate the ratio of the response value of each feature point in the initial set of feature points to the maximum response value in the corresponding neighborhood, and combine the preset inertia filtering ratio threshold to perform a secondary screening of the initial set of feature points to establish a set of feature points, including: S1231. Use the persistent homology algorithm to perform neighborhood response analysis on each feature point in the initial set of feature points, and calculate the response ratio of the response value of each feature point to the maximum response value in the corresponding neighborhood; S1232. Construct a double-threshold screening condition based on the calculated response ratio and the preset inertia filtering ratio threshold, and use the double-threshold screening condition as a constraint condition to perform a secondary screening on the initial feature point set to construct a feature point set.

[0011] Optionally, using the persistent homology algorithm, perform neighborhood response analysis on each feature point in the initial feature point set, and calculate the response ratio of the response value of each feature point to the maximum response value in the corresponding neighborhood, including: S12311. Take the response value of each feature point in the initial feature point set as a scalar function, and sequentially construct a sub-level set filtering sequence in the feature response space, gradually fill the response space and record the evolution process of the topological structure; S12312. In the local neighborhood of each feature point, extract the persistent features of each feature point by analyzing the evolution process of the topological invariant in the feature response subspace with respect to scale change; S12313. Calculate the ratio of the response value of each feature point to the maximum response value with the longest persistent feature in the corresponding neighborhood, and combine normalization processing to obtain the response ratio of each feature point.

[0012] Optionally, based on the obtained feature point coordinates, generate affine transformation parameters, and use the affine transformation parameters to decompose the transformation of the feature point coordinates to extract pose change parameters to obtain the image motion trajectory, including: S21. Based on the obtained feature point coordinates, use the feature matching algorithm to establish feature point matching pairs between the current frame and the preset reference frame, and generate affine transformation parameters by least squares fitting to obtain an affine transformation matrix; S22. Use the obtained affine transformation matrix to transform the coordinates of each feature point in the current frame and decompose the pose change parameters of each feature point; S23. Perform kinematic integration on the pose change parameters of the current frame and the motion information of the historical frame, and stack the affine transformation matrices at each moment to obtain the image motion trajectory.

[0013] Optionally, the pose change parameters include: translation change parameters in the horizontal and vertical directions, and rotation angle change parameters around the image center.

[0014] Optionally, perform kinematic integration on the pose change parameters of the current frame and the motion information of the historical frame, and stack the affine transformation matrices at each moment to obtain the image motion trajectory, including: S231. According to the pose change parameters of each feature point, use the kinematic integration algorithm to recursively fuse the pose change parameters of the current frame and the motion information of the historical frame to establish an inter-frame kinematic constraint model; S232. Use the sliding window mechanism to perform sequential multiplicative accumulation on the affine transformation matrices of consecutive multiple frames, and perform calibration processing in combination with spatio-temporal constraints to obtain the pose transformation matrix; S233. Analyze the obtained pose transformation matrix, extract the motion parameters of each frame of image in the spatio-temporal sequence, and generate the image motion trajectory.

[0015] Optionally, using the sliding window mechanism to perform sequential multiplicative accumulation on the affine transformation matrices of consecutive multiple frames, and perform calibration processing in combination with spatio-temporal constraints to obtain the pose transformation matrix includes: S2321. Based on the established kinematic constraint model, select a sequence of affine transformation matrices of consecutive several frames through the sliding window mechanism, and perform cumulative calculation through sequential matrix multiplication operations to obtain the global pose transformation corresponding to the current frame; S2322. Combine spatial constraints and time constraints to perform joint error correction on the calculated global pose transformation, and construct a non-linear optimization objective function including the geometric and time relationships between image frames; S2323. Use the Lie algebra optimization algorithm to optimize and solve the pose sequence on the constructed optimization objective function, and perform post-processing on the optimization result in combination with smoothing processing to obtain the pose transformation matrix.

[0016] Optionally, use the image trajectory processing algorithm to predict the image motion trajectory, and compare the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame, including: S31. Based on the obtained image motion trajectory, use the Kalman filter algorithm to establish a non-linear motion model including the image position and attitude states, and use the non-linear motion model to predict the image motion trajectory of subsequent frames to obtain the prediction result of the current frame image motion trajectory; S32. Compare the predicted image motion trajectory frame by frame with the real-time image motion trajectory generated by the current frame, and construct an error vector including the joint deviation of position and attitude by calculating the spatial Euclidean distance and the attitude angle difference; S33. Use the weighted assignment algorithm to adjust the weights of the position deviation and the attitude deviation in the error vector to obtain a quantified trajectory comparison result.

[0017] The beneficial effects of the present invention are: 1. Through the feature point detection and optical flow tracking algorithms, the present invention can extract the image motion information in industrial monitoring videos, achieve accurate tracking of inter-frame feature points, thus having good adaptability to environmental changes, and further enhancing the system's perception ability of dynamic change targets such as the operating state of equipment and the gas diffusion trajectory, meeting the industrial application requirements of high real-time and high stability.

[0018] 2. The present invention predicts the motion trend through an image trajectory processing algorithm, and compares the prediction result with the image trajectory of the current frame, thereby effectively identifying the motion estimation error; by introducing a trajectory comparison mechanism, the prediction deviation can be corrected in real time, and further the accuracy and robustness of motion modeling are improved.

[0019] 3. By using the optimized affine transformation parameters to process the video stream frame by frame, the present invention can accurately model the image pose changes in different industrial scenarios, and can adaptively adjust the image processing strategy according to the dynamic changes of the environment, thereby improving the accuracy, reliability and application range of the gas detection system. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0021] Figure 1 FIG. is a flowchart of an adaptive environment image processing method for industrial gas detection according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0022] To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementation manners and the advantages of the present invention.

[0023] According to an embodiment of the present invention, an adaptive environment image processing method for industrial gas detection is provided.

[0024] Now, the present invention will be further described in conjunction with the drawings and specific implementation manners. As Figure 1 shown, the adaptive environment image processing method for industrial gas detection according to an embodiment of the present invention includes: S1. According to the image frames of the industrial monitoring video stream, use a feature point detection algorithm to extract the feature points of the current frame, and combine with the optical flow tracking technology to obtain the feature point coordinates between frames.

[0025] In this alternative embodiment, according to the image frames of the industrial monitoring video stream, using a feature point detection algorithm to extract the feature points of the current frame, and combining with the optical flow tracking technology to obtain the feature point coordinates between frames includes: S11. In the image frame of the industrial monitoring video stream, use a feature point detection algorithm based on blob detection to extract features from the current frame and generate a set of candidate feature points.

[0026] It should be added that the specific embodiments of using a feature point detection algorithm based on blob detection to extract features from the current frame and generate a set of candidate feature points in the image frame of the industrial monitoring video stream are as follows: First, perform Gaussian blur processing on the image to reduce the interference of noise on subsequent feature extraction; then use the Harris corner detection algorithm or the FAST feature detection algorithm to identify the blob regions in the image as potential feature points; next, by analyzing the gray-scale changes in the image, calculate the response values of each candidate feature point, and further screen out the most distinguishable feature points; this process can extract stable and distinguishable feature points under different lighting conditions, thereby providing reliable data support for subsequent image matching, object tracking, motion analysis, etc. For example, in a typical industrial environment, changes such as the movement of the robotic arm and the operating state of the equipment in the image may cause obvious displacement or deformation of the feature points in different regions. The feature point detection method based on blob detection can effectively capture these changes and ensure that clear feature points can still be extracted in a complex background and dynamic environment; in this way, not only the accuracy of feature extraction is improved, but also the processing speed of the image frames in the monitoring video stream can be greatly enhanced, meeting the high requirements for real-time performance and robustness in the industrial production process, thereby providing strong support for accurate monitoring and fault detection and reducing the need for manual intervention.

[0027] S12. Use the feature point response threshold mechanism to screen the set of candidate feature points, and combine the preset inertia filtering ratio threshold to establish a set of feature points.

[0028] In this alternative embodiment, using the feature point response threshold mechanism to screen the set of candidate feature points and combining the preset inertia filtering ratio threshold to establish a set of feature points includes: S121. Based on the generated set of candidate feature points, use a multi-scale corner detection algorithm to perform response analysis in the image spatial domain, and calculate the response values of each feature point in the set of candidate feature points in different scale spaces based on the analysis results.

[0029] S122. Divide the image into several sub-grid regions, set the response threshold according to the mean and standard deviation of the response values in each sub-grid region, and use the response threshold to perform a preliminary screening on the set of candidate feature points to establish an initial set of feature points.

[0030] S123. Calculate the ratio of the response value of each feature point in the initial set of feature points to the maximum response value in the corresponding neighborhood, and combine the preset inertia filtering ratio threshold to perform a secondary screening on the initial set of feature points to establish a set of feature points.

[0031] It should be noted that the following is a specific embodiment of establishing a feature point set by screening a candidate feature point set using a feature point response threshold mechanism and combining a preset inertia filtering ratio threshold: First, use a hyperspectral infrared imaging system to collect infrared images of the gas leakage area, and construct a Gaussian pyramid ( σ = 1.0, 1.5, 2.0) for multi-scale spatial analysis; in the scenario of micro-leakage of pipeline welds, the candidate feature points correspond to the points with the maximum curvature of the gas diffusion edge; for example, at a small scale ( σ = 1.0), the response value of a methane leakage point detected at 0.1 ppm is 0.92, and at a large scale ( σ = 2.0), the response value of the same leakage point decays to 0.75 (the decay rate is about 18.5%), which conforms to the smooth characteristics of gas diffusion. Stable corner points are screened through a multi-scale response decay rate threshold (such as ≤ 20%) to reduce false detections caused by changes in the gas concentration gradient. Secondly, divide the hyperspectral image into sub-grids of 16×16 pixels, and dynamically calculate the mean and standard deviation of the response values within each grid; for example, if the mean of a leakage area in a certain sub-grid is 0.68 and the standard deviation is 0.15, then the response threshold is set to 0.53 (mean - standard deviation). Finally, for the leakage points in the initial feature point set, calculate the ratio of their response values to the maximum response value in the neighborhood (5×5 pixel window); for example, if the response value of a leakage point is 0.65 and the maximum value in the neighborhood is 0.82, the ratio is 0.79, which is higher than the preset inertia filtering ratio threshold of 0.7, so this point is retained; combined with the prior knowledge of the conical shape of gas diffusion, further exclude interference objects such as humans and equipment (the false alarm rate is reduced by 42%); finally, in a complex scenario with metal reflection and steam interference, the accuracy of gas leakage source localization can reach 98.73%.

[0032] In this alternative embodiment, the ratio of the response value of each feature point in the initial feature point set to the maximum response value in the corresponding neighborhood is calculated, and combined with the preset inertia filtering ratio threshold, the initial feature point set is secondarily screened. The establishment of the feature point set includes: S1231. Use the persistent homology algorithm to perform neighborhood response analysis on each feature point in the initial feature point set, and calculate the response ratio of the response value of each feature point to the maximum response value in the corresponding neighborhood.

[0033] In this alternative embodiment, using the persistent homology algorithm to perform neighborhood response analysis on each feature point in the initial feature point set and calculate the response ratio of the response value of each feature point to the maximum response value in the corresponding neighborhood includes: S12311. Take the response value of each feature point in the initial feature point set as a scalar function, and sequentially construct a sub-level set filtering sequence in the feature response space, gradually fill the response space and record the evolution process of the topological structure; S12312. In the local neighborhood of each feature point, by analyzing the evolution process of the topological invariants in the feature response subspace with respect to scale changes, extract the persistence features of each feature point; S12313. Calculate the ratio of the response value of each feature point to the maximum response value with the longest persistence feature in the corresponding neighborhood, and combine with normalization processing to obtain the response ratio of each feature point.

[0034] S1232. According to the calculated response ratio and the preset inertial filtering ratio threshold, construct a double-threshold screening condition, and use the double-threshold screening condition as a constraint condition to perform a secondary screening on the initial feature point set to construct a feature point set.

[0035] It should be noted that the specific embodiments of calculating the ratio of the response value of each feature point in the initial feature point set to the maximum response value in the corresponding neighborhood, and combining with the preset inertial filtering ratio threshold to perform a secondary screening on the initial feature point set to establish a feature point set are as follows: Perform persistent homology analysis on the initial feature point set (about 5000 candidate points); by constructing a neighborhood topological structure (such as an annular neighborhood with a radius of 3 pixels), calculate the response ratio of each feature point. For example, the response value of a certain leakage point is 0.85, the maximum response value in its neighborhood is 0.95, and the response ratio is 0.89, while the response ratio of a metal reflection interference point is 0.45 (its own response value is 0.3, and the maximum response value in the neighborhood is 0.67); the persistent homology algorithm distinguishes real leakage points from transient noise by identifying the topological "hole" features (such as continuous high-ratio regions) in the response ratio distribution. Set the inertial filtering ratio threshold to 0.7 (a preset empirical value), and construct a double-threshold condition in combination with the response ratio: Condition 1: Response ratio ≥ 0.7 (retain feature points with stable topological structures); Condition 2: Response value decay rate ≤ 20% (exclude instantaneous high-response points caused by gas concentration fluctuations).

[0036] For example, the response ratio of a certain leakage point is 0.89 (meeting Condition 1), and the response value decay rate in three consecutive frames of images is 18% ( σ 0.85 → when = 1.0 σ 0.72 when = 2.0), meeting Condition 2; while the response ratio of the reflection interference point, 0.45, and the decay rate of 35% are both filtered.

[0037] S13. Use the optical flow tracking technology to track the feature points in the feature point set frame by frame, and combine with the adaptive window size adjustment mechanism to optimize the tracking accuracy to obtain the corresponding feature point coordinates between the current frame and the previous frame.

[0038] It should be noted that the specific embodiment of using the optical flow tracking technology to track the feature points in the feature point set frame by frame and combining the adaptive window size adjustment mechanism to optimize the tracking accuracy to obtain the corresponding feature point coordinates between the current frame and the previous frame is as follows: The sparse optical flow algorithm is used to track the feature points frame by frame, and the displacement vector between adjacent frames is calculated. For example, the coordinates of a leakage point in the t th frame are (120, 80), and it moves to (122, 83) in the t th + 1 frame, and the optical flow vector (Δ x = 2, Δ y = 3). The gas diffusion model is combined to verify the rationality of its movement (such as the diffusion speed conforming to the methane leakage characteristics of 0.5 - 3 m / s). A 5×5 pixel tracking window is set centered on the feature point, covering the area with significant edge gradient of the gas plume; the change rate of the window size is predicted through the Kalman filter. For example, when the leakage diffusion range expands, the window size gradually expands from 5×5 to 9×9 pixels; the Bhattacharyya coefficient within the window is calculated. If the similarity between adjacent frames is lower than 0.8, the window size adjustment is triggered. Trajectory clustering analysis is performed on the continuously tracked feature points, and the three-dimensional coordinates of the leakage source are reconstructed in combination with the optical flow vector field. For example, in a 30-second video sequence, 50 stable feature points are tracked, and their motion vectors converge at the pipeline weld coordinates (3.2 m, 5.7 m, 1.5 m), and the positioning error is less than 0.3 m.

[0039] S2. Based on the obtained feature point coordinates, affine transformation parameters are generated, and the feature point coordinates are transformed and decomposed using the affine transformation parameters to extract the pose change parameters to obtain the image motion trajectory.

[0040] In this alternative embodiment, based on the obtained feature point coordinates, generating affine transformation parameters, and using the affine transformation parameters to transform and decompose the feature point coordinates to extract the pose change parameters to obtain the image motion trajectory includes: S21. Based on the obtained feature point coordinates, a feature point matching pair between the current frame and the preset reference frame is established using the feature matching algorithm, and the affine transformation parameters are generated by least squares fitting to obtain the affine transformation matrix.

[0041] It should be noted that the specific embodiment of establishing a feature point matching pair between the current frame and the preset reference frame using the feature matching algorithm based on the obtained feature point coordinates and generating the affine transformation parameters by least squares fitting to obtain the affine transformation matrix is as follows: The preset reference frame is a standard template image in a leak-free state, and the feature point set contains approximately 1,200 key points; through feature descriptor matching, 800 groups of candidate matching pairs are initially screened out, and then the RANSAC algorithm is used to eliminate the mismatched point pairs. Based on the coordinates of the matching point pairs, the least squares method is used to fit the affine transformation parameters. Assume the reference frame coordinate matrix is P r , and the current frame coordinate matrix is P c , by solving the linear equation system P c = M*P r + T (where M is a 2×2 rotation and scaling matrix, and T is a translation vector). For example, in a certain fitting, the obtained parameters are: rotation angle θ = 1.2°, scaling factor s = 0.98 (due to local deformation caused by gas diffusion), translation amount T = (5.3 - 2.7) pixels; the root mean square of the fitting residuals is 0.8 pixels, meeting the industrial inspection accuracy requirements (error ≤ 1.5 pixels). After applying the affine transformation matrix to the real-time image and aligning the reference frame coordinate system, the three-dimensional space position is calculated through the centroid coordinates of the leakage area, such as the coordinates (3.5m, 2.1m, 0m) at the pipeline weld; combined with the time series data (frame rate 30fps), the optical flow vector of the feature points is tracked (such as the displacement of a leakage point Δ x = 2.1 pixels / frame, Δ y = 1.8 pixels / frame), and the gas diffusion speed (0.6m / s) and direction (15° east of northeast) are inverted.

[0042] S22. Use the obtained affine transformation matrix to transform the coordinates of each feature point in the current frame and decompose the pose change parameters of each feature point.

[0043] In this alternative embodiment, the pose change parameters include: the translation change parameters along the horizontal and vertical directions, and the rotation angle change parameter around the image center.

[0044] It should be added that the specific embodiments of using the obtained affine transformation matrix to transform the coordinates of each feature point in the current frame and decompose the pose change parameters of each feature point are as follows: Based on hyperspectral infrared imaging (resolution 640×480, frame rate 30fps), apply the affine transformation matrix to the feature point coordinates in the leakage area and decompose: the translation parameter, the horizontal displacement of a leakage point Δ x = 2.3 pixels / frame (about 0.15m / s), the vertical displacement Δ y= 1.8 pixels / frame (about 0.12 m / s), reflecting the gas diffusion direction (12° east of northeast); rotation parameter, rotation angle around the image center θ = 1.5° / frame, characterizing the morphological deflection of the leakage cloud affected by the wind speed. The three-dimensional coordinates of the leakage source are inverted through the translation parameter, such as the pipeline weld (3.2 m, 5.7 m, 1.5 m). Combining the rotation angle to correct the diffusion model (Gaussian plume correction rate +18%), the positioning error is reduced from 1.2 m of the traditional method to 0.3 m.

[0045] S23. Perform kinematic integration on the pose change parameters of the current frame and the motion information of the historical frames, superimpose the affine transformation matrices at each moment, and obtain the image motion trajectory.

[0046] In this alternative embodiment, performing kinematic integration on the pose change parameters of the current frame and the motion information of the historical frames, superimposing the affine transformation matrices at each moment, and obtaining the image motion trajectory includes: S231. According to the pose change parameters of each feature point, recursively fuse the pose change parameters of the current frame and the motion information of the historical frames through the kinematic integration algorithm, and establish a kinematic constraint model between frames.

[0047] S232. Utilize the sliding window mechanism to perform sequential multiplication and accumulation on the affine transformation matrices of multiple consecutive frames, and perform correction processing in combination with spatio-temporal constraints to obtain the pose transformation matrix.

[0048] In this alternative embodiment, utilizing the sliding window mechanism to perform sequential multiplication and accumulation on the affine transformation matrices of multiple consecutive frames, and perform correction processing in combination with spatio-temporal constraints to obtain the pose transformation matrix includes: S2321. Based on the established kinematic constraint model, select a sequence of affine transformation matrices of several consecutive frames through the sliding window mechanism, and perform sequential multiplication operations on the matrices to cumulatively calculate the global pose transformation corresponding to the current frame; S2322. Combine spatial constraints and time constraints to perform joint error correction on the calculated global pose transformation, and construct a non-linear optimization objective function containing the geometric and time relationships between image frames; S2323. Utilize the Lie algebra optimization algorithm to optimize and solve the pose sequence on the constructed optimization objective function, and perform post-processing on the optimization result in combination with smoothing processing to obtain the pose transformation matrix.

[0049] S233. Analyze the obtained pose transformation matrix, extract the motion parameters of each frame of image in the spatio-temporal sequence, and generate the image motion trajectory.

[0050] It should be noted that the specific embodiments of obtaining the image motion trajectory by performing kinematic integration on the pose change parameters of the current frame and the motion information of the historical frames, and superimposing the affine transformation matrices at each moment are as follows: Based on hyperspectral infrared imaging (640×480 pixels, 30 fps), characteristic points in the leakage area are collected. By using the kinematic integration algorithm to fuse the pose parameters of the historical frames (such as horizontal translation Δ x = 2.3 pixels / frame, vertical translation Δ y = 1.8 pixels / frame), a kinematic constraint model of the leakage cloud is constructed. A 5-frame sliding window is used to cumulatively multiply the affine transformation matrices in time series, and the Kalman filter is combined to correct the spatio-temporal constraints (such as the influence of wind speed on the gas diffusion pattern). For example, after a certain correction, the rotation angle error is reduced from ±1.5° to ±0.3°, the volatility of the translation parameters is reduced by 42%, and the leakage source positioning error is optimized from 1.2 m of the traditional method to 0.3 m. The spatio-temporal motion parameters are extracted by analyzing the pose transformation matrix (such as the diffusion direction deflecting 12° / frame), and a leakage trajectory heat map is generated by combining the UAV inspection data (covering 10 square kilometers per time).

[0051] S3. Use the image trajectory processing algorithm to predict the image motion trajectory, and compare the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame.

[0052] In this optional embodiment, using the image trajectory processing algorithm to predict the image motion trajectory and comparing the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame includes: S31. Based on the obtained image motion trajectory, use the Kalman filter algorithm to establish a non-linear motion model including the image position and pose states, and use the non-linear motion model to predict the image motion trajectory of the subsequent frames to obtain the prediction result of the current frame image motion trajectory; S32. Compare the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame frame by frame, and construct an error vector including the combined deviation of position and pose by calculating the spatial Euclidean distance and the pose angle difference; S33. Use the weighted assignment algorithm to adjust the weights of the position deviation and the pose deviation in the error vector to obtain a quantified trajectory comparison result.

[0053] It should be noted that the specific embodiments of using the image trajectory processing algorithm to predict the image motion trajectory and comparing the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame are as follows: A dynamic image sequence of the methane leakage area is collected using a hyperspectral infrared imager (FLIR G306, frame rate 30fps). A non-linear motion model is constructed through the Kalman filtering algorithm, and the image position (pixel coordinate error ±0.5) and attitude parameters (rotation angle error ±0.3°) are fused. For example, at a certain leakage point, the predicted trajectory at the t th frame is a horizontal displacement of Δ x = 2.1 pixels / frame (corresponding to an actual diffusion speed of 0.6 m / s), the attitude angle θ = 1.2° / frame, and the root mean square of the model prediction residuals is 0.8 pixels, which is better than the traditional linear model (RMSE = 1.5 pixels). The predicted trajectory is compared frame by frame with the real-time optical flow tracking results (such as the horizontal displacement Δ x = 2.3 pixels / frame and the vertical displacement Δ y = 1.8 pixels / frame) generated by the Lucas-Kanade algorithm, and the spatial Euclidean distance error (mean 0.6 pixels, standard deviation ±0.2) and attitude angle deviation (mean 0.5°, standard deviation ±0.1°) are calculated. Through the joint deviation formula: ; Based on the leakage diffusion characteristics (such as the upward trend caused by the lower density of methane than air), the weighted assignment algorithm is used to assign a weight of 0.7 to the position deviation and a weight of 0.3 to the attitude deviation. Through adaptive adjustment, in the scenario of sudden wind speed change (such as 4 m / s → 8 m / s), the volatility of the trajectory comparison error drops from 12% to 5%, and the leakage source positioning accuracy reaches 0.3 m (1.2 m for the traditional method).

[0054] S4. Based on the comparison results, optimize the affine transformation parameters, and use the optimized affine transformation parameters to process the video stream frame by frame.

[0055] It should be noted that, based on the comparison results, the specific embodiments of optimizing the affine transformation parameters and using the optimized affine transformation parameters to process the video stream frame by frame are as follows: Based on the comparison results, the SIFT feature matching inliers are screened through the RANSAC algorithm (error threshold 10 pixels, iteration 1000 times) to optimize the affine transformation parameters; the root mean square of the initial affine matrix residuals is 1.8 pixels. After weighted optimization of the sliding window (window size 5 frames) and dynamic compensation by the Kalman filter, the residuals are reduced to 0.8 pixels, and the horizontal translation parameter Δ x = 5.3 pixels / frame (corresponding to an actual diffusion speed of 0.6 m / s), and the rotation angle θ=1.2° / frame. The optimized affine parameters are used to perform sub-pixel deformation correction on the video stream. Combining with the kinematic integration algorithm, historical frame data is fused to reconstruct the three-dimensional trajectory of the leakage cloud (coverage area: 10 square kilometers). Through the error vector weight allocation mechanism (position deviation weight 0.7, attitude deviation weight 0.3), the gas diffusion trajectory is distinguished from the steam interference, and the false alarm rate is reduced from 15% to 3%. Combining with the acoustic sensor data fusion, the inversion error of methane concentration ≤ 3 ppm, and the sensitivity reaches 0.05 ppm. After the affine parameters are updated through error feedback, the pose estimation accuracy can be improved, and the cumulative error caused by interference factors such as vibration and light change can be effectively suppressed, so as to realize a more stable and continuous image motion modeling, and enhance the image processing robustness and detection reliability of the system in complex industrial environments.

[0056] In summary, by means of the above technical solutions of the present invention, through the feature point detection and optical flow tracking algorithms, the image motion information in the industrial monitoring video can be extracted, and the accurate tracking of the feature points between frames can be realized, so as to have good adaptability to environmental changes, and then improve the system's perception ability of dynamic change targets such as the operating state of equipment and the gas diffusion trajectory, meeting the industrial application requirements of high real-time and high stability. The motion trend is predicted through the image trajectory processing algorithm, and the prediction result is compared with the image trajectory of the current frame, so as to effectively identify the motion estimation error; by introducing the trajectory comparison mechanism, the prediction deviation can be corrected in real time, and then the accuracy and robustness of the motion modeling can be improved. By using the optimized affine transformation parameters to process the video stream frame by frame, accurate modeling of the image pose change in different industrial scenarios can be realized, and the image processing strategy can be adaptively adjusted according to the dynamic changes of the environment, thereby improving the accuracy, reliability and applicable range of the gas detection system.

[0057] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An adaptive environmental image processing method for industrial gas detection, characterized in that, The method includes: S1. According to the image frames of the industrial monitoring video stream, use the feature point detection algorithm to extract the feature points of the current frame, and combine the optical flow tracking technology to obtain the feature point coordinates between frames; S2. Based on the obtained feature point coordinates, generate affine transformation parameters, and use the affine transformation parameters to decompose the transformation of the feature point coordinates, and extract the pose change parameters to obtain the image motion trajectory; S3. Use the image trajectory processing algorithm to predict the image motion trajectory, and compare the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame; S4. Based on the comparison result, optimize the affine transformation parameters, and use the optimized affine transformation parameters to process the video stream frame by frame.

2. The adaptive environmental image processing method for industrial gas detection according to claim 1, wherein The step of according to the image frames of the industrial monitoring video stream, using the feature point detection algorithm to extract the feature points of the current frame, and combining the optical flow tracking technology to obtain the feature point coordinates between frames includes: S11. In the image frames of the industrial monitoring video stream, use the feature point detection algorithm based on blob detection to extract the features of the current frame and generate a candidate feature point set; S12. Use the feature point response threshold mechanism to screen the candidate feature point set, and combine the preset inertia filtering ratio threshold to establish a feature point set; S13. Use the optical flow tracking technology to track the feature points in the feature point set frame by frame, and combine the adaptive window size adjustment mechanism to optimize the tracking accuracy to obtain the corresponding feature point coordinates between the current frame and the previous frame.

3. An adaptive environmental image processing method for industrial gas detection according to claim 2, characterized in that, The step of using the feature point response threshold mechanism to screen the candidate feature point set, and combining the preset inertia filtering ratio threshold to establish a feature point set includes: S121. Based on the generated candidate feature point set, use the multi-scale corner detection algorithm to perform response analysis in the image spatial domain, and calculate the response values of each feature point in the candidate feature point set in different scale spaces based on the analysis results; S122. Divide the image into several sub-grid regions, set the response threshold according to the mean and standard deviation of the response values in each sub-grid region, and use the response threshold to perform a preliminary screening of the candidate feature point set to establish an initial feature point set; S123. Calculate the ratio of the response value of each feature point in the initial feature point set to the maximum response value in the corresponding neighborhood, and combine the preset inertia filtering ratio threshold to perform a secondary screening of the initial feature point set to establish a feature point set.

4. An adaptive environmental image processing method for industrial gas detection according to claim 3, wherein, The step of calculating the ratio of the response value of each feature point in the initial feature point set to the maximum response value in the corresponding neighborhood, and combining the preset inertia filtering ratio threshold to perform a secondary screening of the initial feature point set to establish a feature point set includes: S1231. Use the persistent homology algorithm to perform neighborhood response analysis on each feature point in the initial feature point set, and calculate the response ratio of each feature point's response value to the maximum response value in the corresponding neighborhood; S1232. According to the calculated response ratio and the preset inertia filtering ratio threshold, construct a double-threshold screening condition, and use the double-threshold screening condition as a constraint condition to perform a secondary screening of the initial feature point set to construct a feature point set.

5. An adaptive environment image processing method for industrial gas detection according to claim 4, characterized in that The step of using the persistent homology algorithm to perform neighborhood response analysis on each feature point in the initial feature point set, and calculate the response ratio of each feature point's response value to the maximum response value in the corresponding neighborhood includes: S12311. Take the response value of each feature point in the initial feature point set as a scalar function, successively construct a sub-level set filtering sequence in the feature response space, gradually fill the response space, and record the evolution process of the topological structure; S12312. In the local neighborhood of each feature point, by analyzing the evolution process of the topological invariant in the feature response subspace with respect to scale changes, extract the persistence features of each feature point; S12313. Calculate the ratio of the response value of each feature point to the maximum response value with the longest persistence feature in the corresponding neighborhood, and combine normalization processing to obtain the response ratio of each feature point.

6. The adaptive environment image processing method for industrial gas detection according to claim 1, wherein The method of generating affine transformation parameters based on the obtained feature point coordinates, and using the affine transformation parameters to decompose the transformation of the feature point coordinates to extract pose change parameters to obtain the image motion trajectory includes: S21. Based on the obtained feature point coordinates, use the feature matching algorithm to establish feature point matching pairs between the current frame and the preset reference frame, and generate affine transformation parameters by least squares fitting to obtain the affine transformation matrix; S22. Use the obtained affine transformation matrix to transform the coordinates of each feature point in the current frame, and decompose the pose change parameters of each feature point; S23. Perform kinematic integration on the pose change parameters of the current frame and the motion information of the historical frame, stack the affine transformation matrices at each moment, and obtain the image motion trajectory.

7. An adaptive environment image processing method for industrial gas detection according to claim 6, characterized in that, The pose change parameters include: translation change parameters along the horizontal and vertical directions, and rotation angle change parameters around the image center.

8. An adaptive environment image processing method for industrial gas detection according to claim 7, characterized in that, The method of performing kinematic integration on the pose change parameters of the current frame and the motion information of the historical frame, stacking the affine transformation matrices at each moment, and obtaining the image motion trajectory includes: S231. According to the pose change parameters of each feature point, use the kinematic integration algorithm to recursively fuse the pose change parameters of the current frame and the motion information of the historical frame, and establish a kinematic constraint model between frames; S232. Use the sliding window mechanism to perform sequential multiplication accumulation on the affine transformation matrices of multiple consecutive frames, and perform correction processing in combination with spatio-temporal constraints to obtain the pose transformation matrix; S233. Analyze the obtained pose transformation matrix, extract the motion parameters of each frame image in the spatio-temporal sequence, and generate the image motion trajectory.

9. An adaptive environment image processing method for industrial gas detection according to claim 8, characterized in that, The method of using the sliding window mechanism to perform sequential multiplication accumulation on the affine transformation matrices of multiple consecutive frames, and perform correction processing in combination with spatio-temporal constraints to obtain the pose transformation matrix includes: S2321. Based on the established kinematic constraint model, select a sequence of affine transformation matrices of several consecutive frames through the sliding window mechanism, and perform sequential multiplication operation of the matrices to cumulatively calculate the global pose transformation corresponding to the current frame; S2322. Combine spatial constraints and time constraints to perform joint error correction on the calculated global pose transformation, and construct a non-linear optimization objective function including the geometric and time relationships between image frames; S2323. Use the Lie algebra optimization algorithm to optimize and solve the pose sequence on the constructed optimization objective function, and perform post-processing on the optimization result in combination with smoothing processing to obtain the pose transformation matrix.

10. An adaptive environment image processing method for industrial gas detection according to claim 1, characterized in that, Predicting the image motion trajectory using the image trajectory processing algorithm and comparing the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame includes: S31. Based on the obtained image motion trajectory, a nonlinear motion model including the image position and attitude state is established using the Kalman filtering algorithm, and the nonlinear motion model is used to predict the image motion trajectory of subsequent frames to obtain the prediction result of the current frame image motion trajectory; S32. Comparing the predicted image motion trajectory with the real-time image motion trajectory generated by the current frame frame by frame, and constructing an error vector including the joint deviation of position and attitude by calculating the spatial Euclidean distance and the attitude angle difference; S33. Using the weighted assignment algorithm to adjust the weights of the position deviation and the attitude deviation in the error vector to obtain a quantified trajectory comparison result.

Images