A visual detection system for dust emission in a silicate cement plant

Abstract

The present application relates to the technical field of visual detection, in particular to a kind of silicate cement plant area dust's visual detection system, system includes: space spherical background construction and topological library module, according to the panorama background map of silicate plant area stockyard continuous scanning frame generation;Extract the macroscopic rigid structure profile in the panorama background map to obtain global long straight line segment set;The angle and length ratio attribute of the global long straight line segment set are calculated, and the global rigid line segment topological graph is constructed.In the present application, the false difference caused by the overall rise and fall of background brightness is suppressed, and the abnormal pixel coordinate set is retained by the preset dynamic limit limit, the clustering of adjacent pixel coordinate set in the connected state is performed, and the spatial dust scatter point is formed, so as to improve the detection stability, spatial positioning accuracy and continuous monitoring consistency of dust target in complex industrial background.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and in particular to a visual inspection system for dust in silicate cement plant areas. Background Technology

[0002] Visual inspection technology is a comprehensive technology based on digital image acquisition, image processing, spatial geometric analysis, target recognition, and scene information analysis, which enables automatic perception, state analysis, and result determination of target objects.

[0003] Current visual inspection typically relies on grayscale, color, texture, and edge variations in single-frame images or local continuous images to complete target recognition. In actual operation, changes in camera viewpoint, gimbal rotation, lens zoom, and equipment vibration can easily be mistaken for target changes, leading to large-area differences in background response. When the detection scene is located in a silicate cement plant area, the repetitive textures on the material pile surface, the large span of the building structure, and the interwoven distribution of conveying equipment make it easy to form incorrect correspondences in areas of similar texture when simply matching pixels, resulting in positional discrepancies between the working frame and the background image. Furthermore, dust has characteristics such as irregular boundaries, continuous changes in transparency, and significant influence of wind direction on its diffusion direction. If traditional target segmentation uses fixed limits, if the limits are too small, shadows, reflections, and changes in illumination are easily identified as dust; if the limits are too large, areas with low concentration or sparse edges of dust are easily missed. Therefore, improvements are needed. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and propose a visual detection system for dust in silicate cement plant areas.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a visual detection system for dust in a silicate cement plant area includes: The spatial spherical background construction and topology library module generates a panoramic background image based on continuous scan frames of the silicate plant's stockpile area; extracts the macroscopic rigid structure contours from the panoramic background image to obtain a global set of long straight line segments; calculates the angle-to-length ratio attribute of the global set of long straight line segments, and constructs a global rigid line segment topology map. The hardware prior cross-scale coarse positioning module obtains the absolute position metadata corresponding to the current working frame; maps the absolute position metadata to the panoramic background image, calculates the theoretical spatial boundary, and performs mapping and clipping based on the theoretical spatial boundary to generate a cropping reference block; The rigid topology and local frequency domain fusion registration module extracts the set of long straight line segments of the current working frame and the truncated reference block to construct a local geometry map; matches the local geometry map with the global rigid line segment topology map to extract isomorphic rigid line segment pairs; establishes a local structure mask with the isomorphic rigid line segment pairs; performs frequency domain transformation and parameter calculation on the image region within the local structure mask to generate a sub-pixel level spatial alignment matrix; The differential dust removal module performs an inverse perspective transformation on the current working frame based on the sub-pixel level spatial alignment matrix to obtain the perspective-transformed working frame; extracts the image gradient features of the perspective-transformed working frame and the panoramic background image and performs a difference operation to generate a feature difference map; retains the set of pixel coordinates in the feature difference map that exceeds the preset dynamic limit, and performs clustering on the adjacent pixel coordinate sets in the connected state to generate spatial dust scatter points.

[0006] Preferably, the steps for obtaining the global set of long line segments are as follows: Read the frame number, acquisition time, and pixel coordinates of continuous scan frames in the silicate plant's stockpile area. Select adjacent scan frames according to the acquisition time. Locate gray-level gradient inflection points, edge intersection points, and texture corner points in each group of adjacent scan frames. Record the pixel horizontal coordinates, pixel vertical coordinates, and local gray-level arrangement of the two-dimensional image feature points of adjacent scan frames. Establish the correspondence of feature points based on the consistency of the local gray-level arrangement. Delete the correspondence of feature points whose coordinate mapping residuals exceed the preset matching error limit. Use the remaining feature point correspondence to obtain the two-dimensional homography matrix from the previous scan frame to the next scan frame. Unify the observation direction of continuous scan frames according to the two-dimensional homography matrix. Convert the unified pixel coordinates into spherical meridional coordinates and cylindrical longitudinal coordinates. Perform coordinate alignment, brightness difference compensation, and boundary transition fusion on the same-named pixels in the overlapping area. Stitch all pixel areas according to the scan frame arrangement order to generate a panoramic background image. Based on the panoramic background image, the grayscale change amplitude and gradient direction of adjacent pixels are calculated row by row and column by column. Edge pixels with continuous gradient directions and spatial spacing not exceeding the contour connection limit are connected to form continuous edge trajectories. According to the direction change rate, endpoint spacing and the distance of the middle pixel from the straight line reference, continuous edge trajectories with direction change rate exceeding the trajectory curvature limit, continuous edge trajectories with endpoint spacing not reaching the line segment length limit, and continuous edge trajectories with middle pixel deviation distance exceeding the straight line deviation limit are filtered out. Candidate straight line trajectories with endpoint spacing reaching the line segment length limit and middle pixel deviation distance not exceeding the straight line deviation limit are retained. The straight line direction and line segment length are determined according to the pixel coordinates at both ends of the candidate straight line trajectory. Collinear candidate straight line trajectories with straight line direction difference not exceeding the direction merging limit and endpoint projection gap not exceeding the gap limit are merged. Long straight line segments corresponding to the macroscopic rigid structure contour are extracted to obtain a global long straight line segment set.

[0007] Preferably, the steps for obtaining the global rigid line segment topology graph are as follows: Based on the global set of long straight line segments, the starting coordinates, ending coordinates, direction angle, line length, and two side strip pixel regions of each long straight line segment are read. The grayscale mean, gradient direction distribution, edge pixel quantity ratio, and texture grayscale change order of the two side strip pixel regions are statistically analyzed. The statistical results are written into the feature code of the corresponding long straight line segment according to a unified field order. Adjacent long straight line segments whose endpoint distance does not exceed the adjacency distance limit are selected according to the spatial adjacency position in the panoramic background image. The angle attribute is determined by the difference in direction angle of adjacent long straight line segments, and the length ratio attribute is determined by the quotient of line length of adjacent long straight line segments. Line segment nodes are established with the feature code of each long straight line segment. Node connection relationships are established with the angle attribute, length ratio attribute, and endpoint distance of adjacent long straight line segments. Node connection relationships with angle attributes exceeding the angle consistency limit or length ratio attributes exceeding the proportion consistency limit are deleted. All line segment nodes are organized according to the retained node connection relationships to construct a global rigid line segment topology graph.

[0008] Preferably, the steps for obtaining the absolute location metadata are as follows: Read the acquisition time and frame number of the current working frame, retrieve the PTZ PTZ control record according to the acquisition time, extract the absolute position of the horizontal deflection angle corresponding to the horizontal rotation axis, extract the absolute position of the pitch angle corresponding to the pitch rotation axis, read the optical zoom ratio corresponding to the lens zoom drive position, verify the time correspondence between the frame number and the PTZ PTZ control record, delete the control record with a time difference exceeding the synchronization tolerance, retain the horizontal deflection angle, pitch angle and optical zoom ratio with the smallest time difference, and write the acquisition time, horizontal deflection angle, pitch angle and optical zoom ratio according to the current working frame identifier to obtain the absolute position metadata.

[0009] Preferably, the step of obtaining the truncated reference block is as follows: Based on the absolute position metadata, the focal length, principal point coordinates, pixel size, and lens distortion parameters in the pre-calibrated camera imaging model are called. The effective focal length corresponding to the current working frame is determined according to the optical zoom ratio. The camera optical axis direction is adjusted according to the horizontal deflection angle and the pitch angle. The imaging rays at the upper left, upper right, lower left, and lower right corners of the current working frame are transformed to the physical coordinate system of the panoramic background image using the coordinate transformation matrix. The intersection coordinates of the four imaging rays on the panoramic background image are calculated. The boundary endpoints are connected according to the horizontal and vertical arrangement of the intersection coordinates. It is verified whether the boundary endpoints fall within the effective range of the panoramic background image. Boundary endpoints that exceed the effective range are corrected to form the theoretical spatial boundary. Read the horizontal and vertical physical coordinates of the four boundary endpoints of the theoretical spatial boundary in the panoramic background image. Convert the horizontal physical coordinates into pixel column coordinates of the panoramic background image and the vertical physical coordinates into pixel row coordinates of the panoramic background image. Determine the horizontal cropping range according to the minimum and maximum values ​​of the pixel column coordinates and the vertical cropping range according to the minimum and maximum values ​​of the pixel row coordinates. Read the panoramic background image pixels within the horizontal cropping range row by row. Remove pixels outside the boundary according to the theoretical spatial boundary. Perform pixel coordinate mapping and grayscale interpolation on the boundary overlap positions to maintain the size ratio and spatial orientation of the cropping area and generate a cropping reference block.

[0010] Preferably, the step of obtaining the local geometry is as follows: Read the grayscale pixel coordinates of the current working frame and the extracted reference block. Determine edge pixels according to the grayscale change of adjacent pixels. Connect edge pixels whose gradient direction difference does not exceed the direction continuity limit and whose pixel spacing does not exceed the connection distance limit to form candidate line segment trajectories. Delete candidate line segment trajectories whose trajectory length does not reach the length filtering limit or whose middle pixel deviates from the endpoint connection line by more than the straight line deviation limit. Merge candidate line segment trajectories whose direction angle difference does not exceed the collinearity limit and whose endpoint gap does not exceed the endpoint distance limit. Obtain the set of long straight line segments in the current working frame and the set of long straight line segments in the extracted reference block respectively. Read the start coordinates, end coordinates, direction angle, and line segment length of each long straight line segment. Filter spatially adjacent long straight line segments according to the endpoint distance. Calculate the ratio of the direction angle difference to the line segment length of spatially adjacent long straight line segments. Establish nodes with long straight line segments. Establish node connection relationships with the direction angle difference, the line segment length ratio, and the endpoint distance to construct a local geometry.

[0011] Preferably, the step of obtaining the sub-pixel level spatial alignment matrix is ​​as follows: Read the start coordinates, end coordinates, direction angle, line segment length, and number of node connections of each node in the local geometry graph. According to the node direction angle difference tolerance, line segment length ratio tolerance, and node connection number difference tolerance, filter candidate nodes corresponding to the attributes from the global rigid line segment topology graph. Compare the angle arrangement, length ratio arrangement, and endpoint distance arrangement of the adjacent connection relationships of the candidate nodes layer by layer. Retain node combinations with consistent node attributes and continuous adjacent connection relationships. Delete node combinations where the number of matching line segments does not reach the rigid structure quantity limit or the number of connection relationship conflicts exceeds the conflict tolerance. Extract the current working frame long straight line segment and the reference block long straight line segment according to the corresponding node in the retained node combination to form isomorphic rigid line segment pairs. Using the line segment center coordinates of each pair of isomorphic rigid line segments as the reference, expand the structure coverage width along the line segment direction and expand the endpoint coverage distance along both ends of the line segment. Merge the overlapping coverage area into a continuous mask area. Delete dust interference pixels not covered by the continuous mask area to establish a local structure mask. Image regions located within the coverage area of ​​the local structure mask in the current working frame and the cropping reference block are extracted respectively. The structural center coordinates of each image region are set as the origin of polar coordinates. The radial coordinates are transformed according to the distance from the pixel to the structural center coordinates, and the angular coordinates are transformed according to the direction of the pixel connection line relative to the horizontal axis. Logarithmic interval rearrangement is performed on the radial coordinates to form a logarithmic polar coordinate image region. The grayscale changes of the logarithmic polar coordinate image region are decomposed into radial frequency coordinates and angular frequency coordinates. The conjugate product of the frequency component corresponding to the current working frame and the frequency component corresponding to the cropping reference block is calculated, and the frequency components are used... The amplitude product is normalized to obtain the Fourier-Merlin cross-power spectrum. The coordinates of the Fourier-Merlin cross-power spectrum with the highest correlation value exceeding the peak screening limit and the largest sum of neighborhood correlation values ​​are located. The frequency domain phase correlation peak data is obtained. The scale factor is calculated based on the radial peak coordinates, and the rotation parameter is calculated based on the angular peak coordinates. The current working frame is corrected based on the scale factor and rotation parameter. Then, the frequency domain translation parameter is determined by the horizontal and vertical displacements of the corresponding pixels after correction. Sub-pixel interpolation correction is performed on the frequency domain translation parameter, rotation parameter, and scale factor to generate a sub-pixel level spatial alignment matrix.

[0012] Preferably, the step of obtaining the spatial dust scattering points is as follows: Read the horizontal translation, vertical translation, rotation component, scale component and perspective component in the sub-pixel level spatial alignment matrix. According to the reverse coordinate relationship of each target pixel in the current working frame and its corresponding position in the panoramic background image, calculate the source coordinates of the current working frame pixel by pixel. For source coordinates falling between adjacent pixels, reconstruct the gray value according to the horizontal distance weight and the vertical distance weight. Mark the source coordinates that exceed the effective imaging range of the current working frame as empty pixels. Reconstruct the pixels according to the pixel row and column reference of the panoramic background image. Anchor the spatial position of the current working frame to the corresponding position in the panoramic background image to obtain the working frame after perspective transformation. Read the grayscale values ​​of the same pixel coordinates in the working frame after perspective transformation and the panoramic background image, calculate the grayscale change along the horizontal direction and the grayscale change along the vertical direction for each pixel coordinate, determine the gradient magnitude based on the grayscale change in the two directions, determine the gradient direction based on the deflection relationship between the horizontal grayscale change and the vertical grayscale change, calculate the absolute difference between the gradient magnitude corresponding to the working frame after perspective transformation and the gradient magnitude corresponding to the panoramic background image pixel by pixel, calculate the minimum circumferential difference between the two gradient directions, record the gradient magnitude difference, gradient direction difference and the continuity of spatial neighborhood change according to the pixel coordinates, and generate a feature difference map; Based on the feature difference map, the gradient magnitude difference, gradient direction difference, and spatial neighborhood change continuity are read according to pixel coordinates. A preset dynamic limit is determined based on the local gradient fluctuation range of the corresponding position in the panoramic background image. Pixel coordinates with gradient magnitude difference exceeding the magnitude limit and gradient direction difference exceeding the direction limit are retained. Isolated pixel coordinates and pixel coordinates with a continuous number not reaching the connectivity limit are deleted. Adjacent pixel coordinate sets in the connected state are divided according to the eight-neighbor contact relationship. The horizontal coordinate, vertical coordinate, difference intensity, and diffusion range of each adjacent pixel coordinate set are read. Cluster centers and covariance ranges are set based on the mean of horizontal coordinates, the mean of vertical coordinates, and the degree of coordinate dispersion. The probability of each pixel coordinate belonging to different cluster centers is calculated iteratively until the displacement of the cluster centers does not exceed the convergence limit. Clustering results where the center spacing does not exceed the convergence distance limit and the diffusion range overlaps are merged to generate spatial dust scatter points.

[0013] Compared with the prior art, the advantages and positive effects of the present invention are as follows: In this invention, a panoramic background image is generated based on continuous scanning frames of a silicate cement plant's stockpile. Macroscopic rigid structure outlines are extracted from this panoramic background image. Furthermore, a global rigid line segment topology map is constructed using the angular relationships and length ratios of a global set of long straight line segments. This allows the plant's building edges, material shed supports, conveying corridors, and equipment frames to form a spatial reference base with stable geometric relationships, reducing interference from light fluctuations, local occlusion, and dust diffusion on the reliability of the background reference. By reading the absolute position metadata corresponding to the current working frame, the camera's orientation, pitch, and zoom states are mapped to the panoramic background image. The theoretical spatial boundary of the current working frame is calculated, and a truncation reference block is generated. This compresses the search process within the panoramic range to a locally corresponding area, reducing errors and computational burden caused by irrelevant backgrounds participating in the matching. A local geometry map is constructed using the long straight line segments in the current working frame and the truncation reference block. Then, isomorphic rigid lines are extracted based on the structural correspondence between the local geometry map and the global rigid line segment topology map. The segment pairing expands the registration basis from single pixel similarity to constraints based on line segment direction, length ratio, and adjacent structure, maintaining stable matching even under dust occlusion, texture weakening, or local brightness changes. A local structural mask is established around the isomorphic rigid line segment pair, limiting the frequency domain analysis area and reducing the interference of non-rigid dust areas on rotation, scaling, and translation parameter calculations. A sub-pixel-level spatial alignment matrix is ​​generated from the frequency domain transformation result, compensating for subtle displacements between the current working frame and the panoramic background image. After completing the inverse perspective transformation, the gradient magnitude and gradient direction of the image at the same spatial location are differentially analyzed, highlighting contour blurring, edge attenuation, and local diffusion changes caused by dust, while suppressing false differences caused by overall background brightness fluctuations. Abnormal pixel coordinate sets are retained through preset dynamic limit boundaries, and clustering of adjacent pixel coordinate sets in a connected state is performed to form spatial dust scatter points, thereby improving the detection stability, spatial positioning accuracy, and continuous monitoring consistency of dust targets in complex industrial backgrounds. Attached Figure Description

[0014] Figure 1 Filter the distribution map for candidate straight line trajectories; Figure 2 This is the peak value diagram of the Fourier-Merlin cross power spectrum. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0016] Please see Figure 1-2 This invention provides a technical solution: a visual detection system for dust in a silicate cement plant area, comprising: The spatial spherical background construction and topology library module generates a panoramic background image based on continuous scan frames of the silicate plant's stockpile area; extracts the macroscopic rigid structure outline from the panoramic background image to obtain a global set of long straight line segments; calculates the angle and length ratio attributes of the global set of long straight line segments, and constructs a global rigid line segment topology map. The hardware prior cross-scale coarse localization module obtains the absolute position metadata corresponding to the current working frame; maps the absolute position metadata to the panoramic background image, calculates the theoretical spatial boundary, and performs mapping and clipping based on the theoretical spatial boundary to generate a cropping reference block; The rigid topology and local frequency domain fusion registration module extracts the set of long straight line segments from the current working frame and the truncated reference block to construct a local geometry map; it matches the local geometry map with the global rigid line segment topology map to extract isomorphic rigid line segment pairs; it establishes a local structure mask using the isomorphic rigid line segment pairs; and it performs frequency domain transformation and parameter calculation on the image region within the local structure mask to generate a sub-pixel level spatial alignment matrix. The differential dust removal module performs an inverse perspective transformation on the current working frame based on the sub-pixel level spatial alignment matrix to obtain the perspective-transformed working frame; it extracts the image gradient features of the perspective-transformed working frame and the panoramic background image and performs differential operations to generate a feature difference map; it retains the set of pixel coordinates in the feature difference map that exceeds the preset dynamic limit, and performs clustering on the adjacent pixel coordinate sets in the connected state to generate spatial dust scatter points.

[0017] The steps to obtain the global set of long line segments are as follows: Read the frame number, acquisition time, and pixel coordinates of continuous scan frames in the silicate plant's stockpile area. Select adjacent scan frames according to the acquisition time. Locate gray-level gradient inflection points, edge intersection points, and texture corner points in each group of adjacent scan frames. Record the pixel horizontal coordinates, pixel vertical coordinates, and local gray-level arrangement of the two-dimensional image feature points of adjacent scan frames. Establish the correspondence of feature points based on the consistency of the local gray-level arrangement. Delete the correspondence of feature points whose coordinate mapping residuals exceed the preset matching error limit. Use the remaining feature point correspondence to obtain the two-dimensional homography matrix from the previous scan frame to the next scan frame. Unify the observation direction of continuous scan frames according to the two-dimensional homography matrix. Convert the unified pixel coordinates into spherical meridional coordinates and cylindrical longitudinal coordinates. Perform coordinate alignment, brightness difference compensation, and boundary transition fusion on the same-named pixels in the overlapping area. Stitch all pixel areas according to the scan frame arrangement order to generate a panoramic background image. Based on the panoramic background image, the grayscale change amplitude and gradient direction of adjacent pixels are calculated row by row and column by column. Edge pixels with continuous gradient directions and spatial spacing not exceeding the contour connection limit are connected to form continuous edge trajectories. According to the direction change rate of the continuous edge trajectory, the endpoint spacing and the distance of the middle pixel from the straight line reference, continuous edge trajectories with a direction change rate exceeding the trajectory curvature limit, continuous edge trajectories with an endpoint spacing not reaching the line segment length limit, and continuous edge trajectories with a middle pixel deviation distance exceeding the straight line deviation limit are filtered out. Candidate straight line trajectories with an endpoint spacing reaching the line segment length limit and a middle pixel deviation distance not exceeding the straight line deviation limit are retained. The straight line direction and line segment length are determined according to the pixel coordinates at both ends of the candidate straight line trajectory. Collinear candidate straight line trajectories with a straight line direction difference not exceeding the direction merging limit and an endpoint projection gap not exceeding the gap limit are merged. Long straight line segments corresponding to the macroscopic rigid structure contour are extracted to obtain a global set of long straight line segments.

[0018] Specifically, the frame number, acquisition time, and pixel coordinates of continuous scan frames from the silicate plant's stockpile are read. Adjacent scan frames are selected based on their acquisition times. Within each group of adjacent scan frames, the Scale Invariant Feature Transform (SIFT) algorithm is used to locate gray-level gradient inflection points, edge intersections, and texture corners. A 128-dimensional descriptor is generated for each feature point. The pixel x-coordinate, pixel y-coordinate, and corresponding SIFT descriptor of the 2D image feature points in adjacent scan frames are recorded. The Euclidean distance between the SIFT descriptors of feature points in two frames is calculated. Feature point pairs with a distance less than a preset descriptor distance limit (e.g., 0.7, an empirical value used to distinguish between good and fuzzy matches) are used as initial matches. Subsequently, the Random Sample Consensus (RANSAC) algorithm is used to refine the correspondence between feature points. During the iteration process, four pairs of feature points are randomly selected to calculate a candidate homography matrix. This matrix is ​​then used to map all feature points, and the coordinate mapping residual is calculated. Points with a residual less than a preset matching error limit (e.g., 3 pixels) are defined as inliers. After repeated iterations (e.g., 1000 times), the homography matrix with the most inliers is selected as the final 2D homography matrix. Its transformation relationship is expressed as ,in , where is the homogeneous pixel coordinate of the previous scan frame. The corresponding homogeneous pixel coordinates of the next scan frame are used to unify the viewing direction of all consecutive scan frames according to this two-dimensional homography matrix, and the unified pixel coordinates are then used as the basis for this process. Through camera intrinsic parameter matrix The inverse matrix and rotation parameters are converted into spherical meridional coordinates and cylindrical longitudinal coordinates. For pixels with the same name in the overlapping area, the gradient domain fusion algorithm is used to perform coordinate alignment and brightness difference compensation, and the seam is fused to the boundary transition. Finally, according to the original arrangement order of the scan frame, all the transformed and fused pixel areas are stitched together to generate a seamless panoramic background image.

[0019] Based on the panoramic background image, the Sobel operator is applied to each pixel of the image to calculate the gradients in the horizontal and vertical directions, respectively. and This allows us to obtain the gradient magnitude of each pixel. and gradient direction Then, a dual-limit method is used to connect the edges. Pixels with gradient magnitudes higher than the high limit (e.g., 120) are marked as strong edge points, and pixels with gradient magnitudes between the low limit (e.g., 50) and the high limit are marked as weak edge points. By connecting weak edge points that start from strong edge points and continue along the gradient direction, a continuous edge trajectory is formed. The spatial spacing contour connection limit is set to 5 pixels, meaning that trajectories with a break distance exceeding 5 pixels are not connected. For each formed continuous edge trajectory, its direction change rate is calculated, that is, the change rate of the tangent direction angle at each point on the trajectory. If the average change rate exceeds the trajectory curvature limit (e.g., deflection per pixel exceeding 0.5 degrees), it is judged as a curve and filtered out. At the same time, the endpoint spacing of the trajectory is calculated. If it is less than the line segment length limit (e.g., 2% of the panoramic image width, if the panoramic image is 8000 pixels wide), it is considered a curve. If the limit is 160 pixels, the line segment is considered too short and is filtered out. For the remaining trajectories, a straight line reference is fitted using the least squares method, and the maximum vertical distance from all intermediate pixels on the trajectory to the straight line reference is calculated. If the maximum value exceeds the straight line deviation limit (e.g., 2.5 pixels), the trajectory is determined to be non-straight and is filtered out. The candidate straight line trajectories that pass all the above filtering are retained. The straight line direction and line segment length are determined according to the pixel coordinates of their two ends. Finally, all candidate straight line trajectories are merged. If the directional angle difference between two trajectories does not exceed the directional merging limit (e.g., 5 degrees) and the gap between their endpoint projections in one direction does not exceed the gap limit (e.g., 20 pixels), they are merged into a longer straight line segment. All long straight line segments corresponding to the merged macroscopic rigid structure contour are extracted to obtain a global long straight line segment set.

[0020] The steps to obtain the global rigid line segment topology graph are as follows: Based on the global set of long straight line segments, the starting coordinates, ending coordinates, direction angle, line length, and two side strip pixel regions of each long straight line segment are read. The grayscale mean, gradient direction distribution, edge pixel quantity ratio, and texture grayscale change order of the two side strip pixel regions are statistically analyzed. The statistical results are written into the feature code of the corresponding long straight line segment according to a unified field order. Adjacent long straight line segments whose endpoint distance does not exceed the adjacency distance limit are selected according to the spatial adjacency position in the panoramic background image. The angle attribute is determined by the difference in direction angle of adjacent long straight line segments, and the length ratio attribute is determined by the quotient of line length of adjacent long straight line segments. Line segment nodes are established with the feature code of each long straight line segment. Node connection relationships are established with the angle attribute, length ratio attribute, and endpoint distance of adjacent long straight line segments. Node connection relationships with angle attributes exceeding the angle consistency limit or length ratio attributes exceeding the proportion consistency limit are deleted. All line segment nodes are organized according to the retained node connection relationships to construct a global rigid line segment topology graph.

[0021] Specifically, based on the global set of long straight line segments, the starting point coordinates, ending point coordinates, direction angle, and length of each long straight line segment are read. A 15-pixel-wide strip region is defined on each side of the line segment as its context pixel region. The mean grayscale value, 8-bin histogram distribution of the gradient direction, and the proportion of edge pixels (gradient magnitude greater than 50) to the total number of pixels are statistically analyzed for these two strip regions. The texture grayscale change order feature vector calculated using the Local Binary Pattern (LBP) operator is also analyzed. These statistical results are concatenated according to a unified field order (e.g., [mean grayscale left, mean grayscale right, gradient distribution left, gradient distribution right, edge proportion left, edge proportion right, LBP feature left, LBP feature right]) and written into the feature encoding of the corresponding long straight line segment. Then, spatially adjacent line segment pairs are searched in the panoramic background image, and the Euclidean distance between the nearest endpoints of two line segments that does not exceed the adjacency distance limit (e.g., ...). For each adjacent long straight line segment (50 pixels), the included angle attribute is calculated using the direction angle of these two adjacent long straight line segments, and the length ratio attribute is calculated using their segment lengths. The feature code of each long straight line segment is used as a node of the graph, and the included angle attribute, length ratio attribute, and endpoint distance of adjacent long straight line segments are used as edges connecting two nodes to establish an initial graph structure. Then, the graph is pruned, deleting those connection relationships whose edge attributes do not conform to the a priori of conventional building structures. For example, non-collinear or non-perpendicular connections whose included angle attributes do not fall within the typical angle range (such as 90°±5°, 180°±5°), or connections with a disproportionate length ratio attribute of less than 0.1 or greater than 10. That is, node connection relationships whose included angle attributes exceed the included angle consistency limit or whose length ratio attributes exceed the proportional consistency limit are deleted. All line segment nodes are organized according to all retained node connection relationships, and finally a global rigid line segment topology graph describing the spatial relationship of the rigid structure of the factory area is constructed.

[0022] The steps to obtain absolute location metadata are as follows: Read the acquisition time and frame number of the current working frame, retrieve the PTZ PTZ control record according to the acquisition time, extract the absolute position of the horizontal deflection angle corresponding to the horizontal rotation axis, extract the absolute position of the pitch angle corresponding to the pitch rotation axis, read the optical zoom ratio corresponding to the lens zoom drive position, verify the time correspondence between the frame number and the PTZ PTZ control record, delete the control record with a time difference exceeding the synchronization tolerance, retain the horizontal deflection angle, pitch angle and optical zoom ratio with the smallest time difference, and write the acquisition time, horizontal deflection angle, pitch angle and optical zoom ratio according to the current working frame identifier to obtain the absolute position metadata.

[0023] Specifically, the acquisition time and frame number of the current working frame are read. Based on the acquisition time, a timestamp is retrieved from the PTZ PTZ control log. This log records the PTZ commands associated with the timestamps, including the absolute position of the horizontal yaw angle corresponding to the horizontal rotation axis, the absolute position of the pitch angle corresponding to the pitch rotation axis, and the optical zoom ratio corresponding to the lens zoom driver position. For each of the retrieved control records, the frame number and the timestamp correspondence in the PTZ PTZ control record are checked one by one. The absolute time difference between the frame acquisition time and the control record timestamp is calculated, and a synchronization tolerance is set. This synchronization tolerance is set based on the camera frame rate (e.g., 30fps, i.e., 33.3ms interval per frame) and the PTZ motor response delay (e.g., 20ms). The calculation method is: Synchronization tolerance = (1 / frame rate) + response delay, i.e., 33.3ms + 20ms = At 53.3ms, control records with a time difference exceeding 53.3ms are considered invalid synchronizations and deleted. Among the remaining valid synchronization records, the record with the smallest time difference is selected, and its corresponding horizontal yaw angle, pitch angle, and optical zoom are extracted. These three parameters, along with the identifier (frame number) of the current working frame and the acquisition time, are recorded to form a data structure, such as {frame_id: 1024,timestamp: '2026-07-21T10:00:05.123Z', pan: 125.3 degrees, tilt: -15.8 degrees, zoom:3.5x}, to obtain absolute position metadata.

[0024] The steps for obtaining the reference block are as follows: Based on the absolute position metadata, the system calls the focal length, principal point coordinates, pixel size, and lens distortion parameters from the pre-calibrated camera imaging model. It determines the effective focal length corresponding to the current working frame based on the optical zoom ratio, adjusts the camera optical axis direction according to the horizontal yaw angle and pitch angle, and uses the coordinate transformation matrix to transform the imaging rays at the top left, top right, bottom left, and bottom right corners of the current working frame to the physical coordinate system of the panoramic background image. It calculates the intersection coordinates of the four imaging rays on the panoramic background image, connects the boundary endpoints according to the horizontal and vertical arrangement of the intersection coordinates, verifies whether the boundary endpoints fall within the effective range of the panoramic background image, corrects the boundary endpoints that exceed the effective range, and forms the theoretical spatial boundary. Read the horizontal and vertical physical coordinates of the four boundary endpoints of the theoretical spatial boundary in the panoramic background image. Convert the horizontal physical coordinates into pixel column coordinates of the panoramic background image and the vertical physical coordinates into pixel row coordinates of the panoramic background image. Determine the horizontal clipping range according to the minimum and maximum values ​​of the pixel column coordinates and the vertical clipping range according to the minimum and maximum values ​​of the pixel row coordinates. Read the panoramic background image pixels within the horizontal clipping range row by row. Remove pixels outside the boundary according to the theoretical spatial boundary. Perform pixel coordinate mapping and grayscale interpolation at the boundary overlap positions to maintain the size ratio and spatial orientation of the clipping area and generate a cropping reference block.

[0025] Specifically, based on the absolute position metadata, the camera imaging model parameters, including focal length, are retrieved in advance using the Zhang Zhengyou calibration method. Principal point coordinates Pixel size and radial and tangential distortion parameters Read the optical zoom ratio from the absolute position metadata. And based on the camera lens's preset base focal length Calculate the effective focal length corresponding to the current working frame. Simultaneously read the horizontal deflection angle and pitch angle Rotation matrix used to construct the rotation from the camera coordinate system to the world coordinate system (panoramic image coordinate system). By utilizing coordinate transformation relationships, the pixel coordinates of the four corner points of the current working frame are... First, distortion correction formulas are applied to remove distortion, and then the distortion is converted into a 3D ray vector in the camera coordinate system. ,in This is the intrinsic parameter matrix corresponding to the current effective focal length. Then, the ray vector is processed through a rotation matrix. Transform to world coordinate system The intersection points of these four imaging rays with the predefined panoramic spherical model (e.g., a sphere of unit radius) are calculated to obtain the three-dimensional coordinates of the four corner points in the physical coordinate system of the panoramic image. These four three-dimensional coordinate points are then connected in a spatial quadrilateral according to their horizontal (longitude) and vertical (latitude) arrangement, which serves as the theoretical spatial boundary. The vertices of this quadrilateral are checked to see if they fall within the effective latitude and longitude range of the panoramic background image. For boundary vertices that exceed the range, they are clipped to the boundary of the effective range to form the final theoretical spatial boundary.

[0026] Read the lateral physical coordinates (longitude) of the four boundary endpoints of the theoretical space boundary in the panoramic background image. ) and vertical physical coordinates (latitude) Based on the mapping relationship during panoramic image generation, these physical coordinates are converted into pixel coordinates of the panoramic background image. For example, for a cylindrical projection panoramic image, the pixel column coordinates are... pixel row coordinates ,in and These are the width and height of the panoramic background image, respectively, which gives the pixel coordinates of the four corner points on the panoramic image. Based on the minimum and maximum values ​​of these four pixel coordinates, a rectangular clipping region is determined that encloses the theoretical space boundary, with a horizontal range of [missing value]. The vertical range is Then iterate through every pixel within this rectangular cropping area. The cross product or area method is used to determine whether a pixel is located inside the quadrilateral formed by its four corner points (i.e., the theoretical space boundary). If it is outside the boundary, it is discarded. For pixels inside the boundary, in order to generate a reference block with the same size and scale as the current working frame, a reverse perspective transformation is required, which means that each pixel in the reference block is cropped for the generated target. Calculate the non-integer coordinates that map back to the quadrilateral region in the panoramic image. Then, using bilinear interpolation, The grayscale value of a pixel is calculated by using the grayscale values ​​of the four nearest panoramic pixels, thus maintaining the size ratio and spatial orientation of the cropped area and generating the final cropping reference block.

[0027] The steps to obtain the local geometry diagram are as follows: Read the grayscale pixel coordinates of the current working frame and the extracted reference block. Determine edge pixels according to the grayscale change of adjacent pixels. Connect edge pixels whose gradient direction difference does not exceed the direction continuity limit and whose pixel spacing does not exceed the connection distance limit to form candidate line segment trajectories. Delete candidate line segment trajectories whose trajectory length does not reach the length filtering limit or whose middle pixel deviates from the endpoint connection line by more than the straight line deviation limit. Merge candidate line segment trajectories whose direction angle difference does not exceed the collinearity limit and whose endpoint gap does not exceed the endpoint distance limit. Obtain the set of long straight line segments in the current working frame and the set of long straight line segments in the extracted reference block respectively. Read the start coordinates, end coordinates, direction angle, and line segment length of each long straight line segment. Filter spatially adjacent long straight line segments according to the endpoint distance. Calculate the ratio of the direction angle difference to the line segment length of spatially adjacent long straight line segments. Establish nodes with long straight line segments. Establish node connection relationships with the direction angle difference, the line segment length ratio, and the endpoint distance to construct a local geometry.

[0028] Specifically, the grayscale pixel matrix of the current working frame and the extracted reference block are read, and the same line detection process is performed on both images. First, the gradient magnitude and direction of each pixel are calculated using a 3x3 Sobel operator. Then, edge pixels whose gradient direction difference does not exceed the direction continuity limit (e.g., 22.5 degrees) and whose pixel spacing does not exceed the connection distance limit (e.g., 3 pixels) are connected to form candidate line segment trajectories. Subsequently, these trajectories are filtered, and trajectories whose number of pixels does not reach the local length filtering limit (e.g., 30 pixels) and non-linear trajectories whose maximum vertical distance from the middle pixel to its two ends exceeds the line deviation limit (e.g., 2 pixels) are deleted. For the trajectories that pass the filtering, spatially collinear trajectory segments are further merged. If the direction angle difference between two trajectories is less than 2 pixels, the trajectories are considered as collinear. If the local collinearity exceeds the limit (e.g., 5 degrees) and the gap between their endpoints does not exceed the endpoint distance limit (e.g., 10 pixels), they are merged into a single straight line. This process obtains the set of long straight line segments for the current working frame and the set of long straight line segments for the truncated reference block. For each set, the start coordinates, end coordinates, direction angle, and line length of each long straight line segment are read. Spatially adjacent pairs of long straight line segments are selected based on the Euclidean distance between their endpoints (e.g., the endpoint distance is less than 15% of the short side length of the image). The direction angle difference and line length ratio between these adjacent long straight line segments are calculated. Then, each long straight line segment is used as a graph node, and the direction angle difference, line length ratio, and endpoint distance between adjacent segments are used as edge attributes of the graph to construct a local geometry graph for the current working frame and the truncated reference block, respectively.

[0029] The steps to obtain the sub-pixel level spatial alignment matrix are as follows: Read the start coordinates, end coordinates, direction angle, line segment length, and number of node connections of each node in the local geometry graph. According to the node direction angle difference tolerance, line segment length ratio tolerance, and node connection number difference tolerance, filter candidate nodes corresponding to the attributes from the global rigid line segment topology graph. Compare the angle arrangement, length ratio arrangement, and endpoint distance arrangement of the adjacent connection relationships of the candidate nodes layer by layer. Retain node combinations with consistent node attributes and continuous adjacent connection relationships. Delete node combinations where the number of matching line segments does not reach the rigid structure quantity limit or the number of connection relationship conflicts exceeds the conflict tolerance. Extract the current working frame long line segment and the reference block long line segment according to the corresponding node in the retained node combination to form isomorphic rigid line segment pairs. Using the line segment center coordinates of each isomorphic rigid line segment pair as the reference, expand the structure coverage width along the line segment direction and expand the endpoint coverage distance along both ends of the line segment. Merge the overlapping coverage area into a continuous mask area. Delete dust interference pixels not covered by the continuous mask area to establish a local structure mask. Image regions located within the local structure mask coverage area are extracted from the current working frame and the truncated reference block, respectively. The structure center coordinates of each image region are set as the origin of polar coordinates. The radial coordinates are transformed according to the distance from the pixel to the structure center coordinates, and the angular coordinates are transformed according to the direction of the pixel connection line relative to the horizontal axis. Logarithmic interval rearrangement is performed on the radial coordinates to form a logarithmic polar coordinate image region. The grayscale changes of the logarithmic polar coordinate image region are decomposed into radial frequency coordinates and angular frequency coordinates. The conjugate product of the frequency component corresponding to the current working frame and the frequency component corresponding to the truncated reference block is calculated, and the frequency component amplitude is used... The product is normalized to obtain the Fourier-Merlin cross-power spectrum. The coordinates of the Fourier-Merlin cross-power spectrum with the highest correlation value exceeding the peak screening limit and the largest sum of neighborhood correlation values ​​are located. The frequency domain phase correlation peak data are obtained. The scale factor is calculated based on the radial peak coordinates, and the rotation parameter is calculated based on the angular peak coordinates. The current working frame is corrected based on the scale factor and rotation parameter. Then, the frequency domain translation parameter is determined by the horizontal and vertical displacements of the corresponding pixels after correction. Sub-pixel interpolation correction is performed on the frequency domain translation parameter, rotation parameter, and scale factor to generate a sub-pixel level spatial alignment matrix.

[0030] Specifically, the system reads the attributes of each node in the local geometry of the current working frame and the captured reference block, including start-point coordinates, end-point coordinates, orientation angle, line segment length, and number of node connections (degrees). Starting from a node in the local geometry, it performs a matching search in the global rigid line segment topology based on its attributes. The filtering criteria are: the node orientation angle difference is within ±5 degrees, the line segment length ratio is within ±15%, and the difference in the number of node connections does not exceed 1. Candidate nodes that meet the criteria are selected from the global topology. Starting from a candidate node, the system recursively and layer by layer compares the connection relationships of its adjacent nodes with those in the local geometry, including the order of the included angle, the order of the length ratio, and the order of the endpoint distance. It retains the node combinations with consistent node attributes and continuous adjacent connections, forming a matching subgraph. The matching results are then verified. Node combinations that do not meet the rigid structure limit (e.g., at least 3 matching segments) or have more than one conflict in their connection relationships are deleted. The corresponding segments of all the retained node combinations that pass verification are extracted to form isomorphic rigid line segment pairs between the current working frame long straight line segment and the cut reference block long straight line segment. Using the center coordinates of each isomorphic rigid line segment pair as a reference, a fixed structural coverage width (e.g., 20 pixels) is extended bidirectionally along the vertical direction of the line segment, and an endpoint coverage distance (e.g., 25 pixels) is extended outward from both ends along the line segment direction to form multiple rectangular regions. The overlapping rectangular regions generated by all isomorphic rigid line segment pairs are merged into a continuous mask region to create a local structural mask that only contains stable rigid structure regions.

[0031] Extract the image regions covered by local structure masks from the current working frame and the cropped reference block respectively. For each connected region covered by the mask, calculate its geometric center. Use it as the origin of polar coordinates, and then use each Cartesian coordinate within the region. Convert to logarithmic polar coordinates The conversion formula is: and By using radial coordinates A logarithmic interval rearrangement is performed to form a logarithmic polar coordinate image region. This transformation converts rotations and scaling in the original image into translations in the logarithmic polar coordinate domain. Subsequently, a two-dimensional Fourier transform is performed on the logarithmic polar coordinate image regions of the current working frame and the extracted reference block to obtain the complex frequency spectrum. and The normalized cross-power spectrum of the two is calculated using the following formula: ,in yes The complex conjugate, for Perform an inverse Fourier transform to obtain a phase correlation matrix. Locate the coordinates of the point in this matrix where the correlation value exceeds the peak screening limit (e.g., 0.8 times the maximum value of the matrix) and the sum of the correlation values ​​in its 3x3 neighborhood is the largest. Calculate the scaling factor based on the peak coordinates. and rotation parameters After correcting the current working frame based on the calculated scaling factor and rotation parameters, phase correlation calculations are performed again with the truncated reference block in Cartesian coordinates to determine the translation parameters. Then, by fitting a two-dimensional Gaussian surface around the peak of the correlation matrix, the peak position is solved to the sub-pixel level. Sub-pixel interpolation corrections are then performed on the frequency domain translation parameters, rotation parameters, and scale factor to generate a sub-pixel level spatial alignment matrix.

[0032] The steps for obtaining spatial dust scatter points are as follows: Read the horizontal translation, vertical translation, rotation component, scale component, and perspective component from the subpixel-level spatial alignment matrix. Calculate the source coordinates of the current working frame pixel by pixel according to the inverse coordinate relationship of each target pixel in the corresponding position in the panoramic background image. For source coordinates falling between adjacent pixels, reconstruct the grayscale value based on the horizontal and vertical distance weights. Mark source coordinates that exceed the effective imaging range of the current working frame as vacant pixels. Reconstruct pixels according to the pixel row and column reference of the panoramic background image. Anchor the spatial position of the current working frame to the corresponding position in the panoramic background image to obtain the perspective-transformed working frame. Read the grayscale values ​​of the same pixel coordinates in the working frame after perspective transformation and the panoramic background image. Calculate the grayscale change along the horizontal direction and the grayscale change along the vertical direction for each pixel coordinate. Determine the gradient magnitude based on the grayscale changes in the two directions. Determine the gradient direction based on the deflection relationship between the horizontal grayscale change and the vertical grayscale change. Calculate the absolute difference between the gradient magnitude corresponding to the working frame after perspective transformation and the gradient magnitude corresponding to the panoramic background image pixel by pixel. Calculate the minimum circumferential difference between the two gradient directions. Record the gradient magnitude difference, gradient direction difference, and the degree of continuity of spatial neighborhood changes according to the pixel coordinates to generate a feature difference map. Based on the feature difference map, the gradient magnitude difference, gradient direction difference, and spatial neighborhood change continuity are read according to pixel coordinates. The preset dynamic limit is determined based on the local gradient fluctuation range of the corresponding position in the panoramic background image. Pixel coordinates with gradient magnitude difference exceeding the magnitude limit and gradient direction difference exceeding the direction limit are retained. Isolated pixel coordinates and pixel coordinates with a continuous number not reaching the connectivity limit are deleted. Adjacent pixel coordinate sets in the connected state are divided according to the eight-neighbor contact relationship. The horizontal coordinate, vertical coordinate, difference intensity, and diffusion range of each adjacent pixel coordinate set are read. The cluster center and covariance range are set according to the mean of the horizontal coordinate, the mean of the vertical coordinate, and the degree of coordinate dispersion. The probability of each pixel coordinate belonging to different cluster centers is calculated iteratively until the displacement of the cluster center does not exceed the convergence limit. The clustering results with the center spacing not exceeding the convergence distance limit and the diffusion range overlapping are merged to generate spatial dust scatter points.

[0033] Specifically, read the subpixel-level spatial alignment matrix, which is typically a 3x3 affine or perspective transformation matrix. This includes horizontal translation, vertical translation, rotation components, scale components, and perspective components. To accurately align the current working frame to the coordinate system of the panoramic background image, a reverse mapping method is used to generate the perspective-transformed working frame. Specifically, a blank image with the same size as the cropped reference block is created as the target image, and the coordinates of each pixel in the target image are traversed. By left-multiplying the inverse of the space alignment matrix Calculate its corresponding source coordinates in the current working frame. Obtain source coordinates Non-integer coordinates are obtained by homogeneous coordinate normalization. Check whether the source coordinates fall within the effective imaging range of the current working frame (e.g., and If it exceeds the range, then in the target image The location is marked as an empty cell (e.g., gray value of 0). If it is within the range, because... If the coordinates are floating-point, then bilinear interpolation is used to calculate the grayscale value, that is, based on the grayscale values ​​of the four nearest integer pixels and their interpolation with the coordinates of the surrounding pixels. The distance weights are used to reconstruct the grayscale values, and all the reconstructed pixels are arranged according to the pixel row and column reference of the panoramic background image to obtain a perspective-transformed working frame that is completely aligned in space with the cropped reference block.

[0034] Read the grayscale values ​​within the same pixel coordinate range in the perspective-transformed working frame and the panoramic background image, and for each corresponding pixel position in the two images... Applying the same 3x3 Sobel operator, calculate the horizontal and vertical gradients of the working frame after perspective transformation. , and the corresponding gradient of the panoramic background image. , Based on these gradient components, the gradient magnitude of the two images at each pixel is calculated. and and gradient direction and Then, the difference in gradient features is calculated pixel by pixel, and the difference in gradient magnitude is calculated as the absolute difference. The gradient direction difference is then considered in terms of the periodicity of the angle, and its minimum circumferential difference is calculated. In order to measure the continuity of differences in regions, a spatial neighborhood change continuity index is calculated. This metric is defined as the difference in gradient magnitude within a 5x5 neighborhood window. The standard deviation of the standard deviation indicates that the difference in the area is smooth and continuous, and is more likely to belong to translucent objects such as dust, rather than hard edge noise. Then, the coordinates of each pixel are... and the corresponding gradient magnitude difference gradient direction difference and the degree of continuity of spatial neighborhood changes Record them together to generate a multi-channel feature difference map.

[0035] Based on the feature difference map, the gradient magnitude difference, gradient direction difference, and spatial neighborhood continuity are read pixel by pixel coordinates. A dynamic limit is set for each pixel, which is adaptively adjusted according to the local texture complexity of the panoramic background image at the corresponding location. Specifically, for each pixel in the panoramic background image... Calculate the mean gradient magnitude within a 15x15 neighborhood. and standard deviation The dynamic amplitude limit is set to ,in The sensitivity coefficient is empirically set to 2.5, and the directional limit is set to a fixed value, such as 30 degrees, to preserve gradient magnitude differences in the feature difference map exceeding [a certain value]. Furthermore, for the pixel coordinates with a gradient direction difference exceeding 30 degrees, noise is filtered out from these retained pixel coordinates, any isolated pixels are deleted, and a connected component labeling algorithm is used to delete small sets of pixels whose number of connected pixels does not reach the connection limit (e.g., 25 pixels). For the remaining set of adjacent pixel coordinates that have passed the screening of connected states, a Gaussian Mixture Model (GMM) is used for clustering. The pixel coordinates in each connected component are used as data points, and the Expectation-Maximization (EM) algorithm is used to fit the GMM until the displacement of the cluster center is less than the convergence limit (e.g., 0.5 pixels). Thus, each dust area is fitted as one or more Gaussian components, and each component represents a cluster. The obtained clustering results are then merged. If the Gaussian distributions of two cluster centers overlap in the range of 3-sigma, or the Mahalanobis distance between the centers does not exceed a preset convergence distance limit, then the two clusters are merged. Finally, the center (mean) and shape (covariance matrix) of each cluster (Gaussian component) represent a spatial dust scattering point.

Claims

1. A visual detection system for dust in a silicate cement plant area, characterized in that, The system includes: The spatial spherical background construction and topology library module generates a panoramic background image based on continuous scan frames of the silicate plant's stockpile area; extracts the macroscopic rigid structure contours from the panoramic background image to obtain a global set of long straight line segments; calculates the angle-to-length ratio attribute of the global set of long straight line segments, and constructs a global rigid line segment topology map. The hardware prior cross-scale coarse positioning module obtains the absolute position metadata corresponding to the current working frame; maps the absolute position metadata to the panoramic background image, calculates the theoretical spatial boundary, and performs mapping and clipping based on the theoretical spatial boundary to generate a cropping reference block; The rigid topology and local frequency domain fusion registration module extracts the set of long straight line segments of the current working frame and the truncated reference block to construct a local geometry map; matches the local geometry map with the global rigid line segment topology map to extract isomorphic rigid line segment pairs; establishes a local structure mask with the isomorphic rigid line segment pairs; performs frequency domain transformation and parameter calculation on the image region within the local structure mask to generate a sub-pixel level spatial alignment matrix; The differential dust removal module performs an inverse perspective transformation on the current working frame based on the sub-pixel level spatial alignment matrix to obtain the perspective-transformed working frame; extracts the image gradient features of the perspective-transformed working frame and the panoramic background image and performs a difference operation to generate a feature difference map; retains the set of pixel coordinates in the feature difference map that exceeds the preset dynamic limit, and performs clustering on the adjacent pixel coordinate sets in the connected state to generate spatial dust scatter points.

2. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the global set of long line segments are as follows: Read the frame number, acquisition time, and pixel coordinates of continuous scan frames in the silicate plant's stockpile area. Select adjacent scan frames according to the acquisition time. Locate gray-level gradient inflection points, edge intersection points, and texture corner points in each group of adjacent scan frames. Record the pixel horizontal coordinates, pixel vertical coordinates, and local gray-level arrangement of the two-dimensional image feature points of adjacent scan frames. Establish the correspondence of feature points based on the consistency of the local gray-level arrangement. Delete the correspondence of feature points whose coordinate mapping residuals exceed the preset matching error limit. Use the remaining feature point correspondence to obtain the two-dimensional homography matrix from the previous scan frame to the next scan frame. Unify the observation direction of continuous scan frames according to the two-dimensional homography matrix. Convert the unified pixel coordinates into spherical meridional coordinates and cylindrical longitudinal coordinates. Perform coordinate alignment, brightness difference compensation, and boundary transition fusion on the same-named pixels in the overlapping area. Stitch all pixel areas according to the scan frame arrangement order to generate a panoramic background image. Based on the panoramic background image, the grayscale change amplitude and gradient direction of adjacent pixels are calculated row by row and column by column. Edge pixels with continuous gradient directions and spatial spacing not exceeding the contour connection limit are connected to form continuous edge trajectories. According to the direction change rate, endpoint spacing and the distance of the middle pixel from the straight line reference, continuous edge trajectories with direction change rate exceeding the trajectory curvature limit, continuous edge trajectories with endpoint spacing not reaching the line segment length limit, and continuous edge trajectories with middle pixel deviation distance exceeding the straight line deviation limit are filtered out. Candidate straight line trajectories with endpoint spacing reaching the line segment length limit and middle pixel deviation distance not exceeding the straight line deviation limit are retained. The straight line direction and line segment length are determined according to the pixel coordinates at both ends of the candidate straight line trajectory. Collinear candidate straight line trajectories with straight line direction difference not exceeding the direction merging limit and endpoint projection gap not exceeding the gap limit are merged. Long straight line segments corresponding to the macroscopic rigid structure contour are extracted to obtain a global long straight line segment set.

3. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the global rigid line segment topology graph are as follows: Based on the global set of long straight line segments, the starting coordinates, ending coordinates, direction angle, line length, and two side strip pixel regions of each long straight line segment are read. The grayscale mean, gradient direction distribution, edge pixel quantity ratio, and texture grayscale change order of the two side strip pixel regions are statistically analyzed. The statistical results are written into the feature code of the corresponding long straight line segment according to a unified field order. Adjacent long straight line segments whose endpoint distance does not exceed the adjacency distance limit are selected according to the spatial adjacency position in the panoramic background image. The angle attribute is determined by the difference in direction angle of adjacent long straight line segments, and the length ratio attribute is determined by the quotient of line length of adjacent long straight line segments. Line segment nodes are established with the feature code of each long straight line segment. Node connection relationships are established with the angle attribute, length ratio attribute, and endpoint distance of adjacent long straight line segments. Node connection relationships with angle attributes exceeding the angle consistency limit or length ratio attributes exceeding the proportion consistency limit are deleted. All line segment nodes are organized according to the retained node connection relationships to construct a global rigid line segment topology graph.

4. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the absolute location metadata are as follows: Read the acquisition time and frame number of the current working frame, retrieve the PTZ PTZ control record according to the acquisition time, extract the absolute position of the horizontal deflection angle corresponding to the horizontal rotation axis, extract the absolute position of the pitch angle corresponding to the pitch rotation axis, read the optical zoom ratio corresponding to the lens zoom drive position, verify the time correspondence between the frame number and the PTZ PTZ control record, delete the control record with a time difference exceeding the synchronization tolerance, retain the horizontal deflection angle, pitch angle and optical zoom ratio with the smallest time difference, and write the acquisition time, horizontal deflection angle, pitch angle and optical zoom ratio according to the current working frame identifier to obtain the absolute position metadata.

5. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the reference block are as follows: Based on the absolute position metadata, the focal length, principal point coordinates, pixel size, and lens distortion parameters in the pre-calibrated camera imaging model are called. The effective focal length corresponding to the current working frame is determined according to the optical zoom ratio. The camera optical axis direction is adjusted according to the horizontal deflection angle and the pitch angle. The imaging rays at the upper left, upper right, lower left, and lower right corners of the current working frame are transformed to the physical coordinate system of the panoramic background image using the coordinate transformation matrix. The intersection coordinates of the four imaging rays on the panoramic background image are calculated. The boundary endpoints are connected according to the horizontal and vertical arrangement of the intersection coordinates. It is verified whether the boundary endpoints fall within the effective range of the panoramic background image. Boundary endpoints that exceed the effective range are corrected to form the theoretical spatial boundary. Read the horizontal and vertical physical coordinates of the four boundary endpoints of the theoretical spatial boundary in the panoramic background image. Convert the horizontal physical coordinates into pixel column coordinates of the panoramic background image and the vertical physical coordinates into pixel row coordinates of the panoramic background image. Determine the horizontal cropping range according to the minimum and maximum values ​​of the pixel column coordinates and the vertical cropping range according to the minimum and maximum values ​​of the pixel row coordinates. Read the panoramic background image pixels within the horizontal cropping range row by row. Remove pixels outside the boundary according to the theoretical spatial boundary. Perform pixel coordinate mapping and grayscale interpolation on the boundary overlap positions to maintain the size ratio and spatial orientation of the cropping area and generate a cropping reference block.

6. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the local geometry are as follows: Read the grayscale pixel coordinates of the current working frame and the extracted reference block. Determine edge pixels according to the grayscale change of adjacent pixels. Connect edge pixels whose gradient direction difference does not exceed the direction continuity limit and whose pixel spacing does not exceed the connection distance limit to form candidate line segment trajectories. Delete candidate line segment trajectories whose trajectory length does not reach the length filtering limit or whose middle pixel deviates from the endpoint connection line by more than the straight line deviation limit. Merge candidate line segment trajectories whose direction angle difference does not exceed the collinearity limit and whose endpoint gap does not exceed the endpoint distance limit. Obtain the set of long straight line segments in the current working frame and the set of long straight line segments in the extracted reference block respectively. Read the start coordinates, end coordinates, direction angle, and line segment length of each long straight line segment. Filter spatially adjacent long straight line segments according to the endpoint distance. Calculate the ratio of the direction angle difference to the line segment length of spatially adjacent long straight line segments. Establish nodes with long straight line segments. Establish node connection relationships with the direction angle difference, the line segment length ratio, and the endpoint distance to construct a local geometry.

7. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the sub-pixel level spatial alignment matrix are as follows: Read the start coordinates, end coordinates, direction angle, line segment length, and number of node connections of each node in the local geometry graph. According to the node direction angle difference tolerance, line segment length ratio tolerance, and node connection number difference tolerance, filter candidate nodes corresponding to the attributes from the global rigid line segment topology graph. Compare the angle arrangement, length ratio arrangement, and endpoint distance arrangement of the adjacent connection relationships of the candidate nodes layer by layer. Retain node combinations with consistent node attributes and continuous adjacent connection relationships. Delete node combinations where the number of matching line segments does not reach the rigid structure quantity limit or the number of connection relationship conflicts exceeds the conflict tolerance. Extract the current working frame long straight line segment and the reference block long straight line segment according to the corresponding node in the retained node combination to form isomorphic rigid line segment pairs. Using the line segment center coordinates of each pair of isomorphic rigid line segments as the reference, expand the structure coverage width along the line segment direction and expand the endpoint coverage distance along both ends of the line segment. Merge the overlapping coverage area into a continuous mask area. Delete dust interference pixels not covered by the continuous mask area to establish a local structure mask. Image regions located within the coverage area of ​​the local structure mask in the current working frame and the cropping reference block are extracted respectively. The structural center coordinates of each image region are set as the origin of polar coordinates. The radial coordinates are transformed according to the distance from the pixel to the structural center coordinates, and the angular coordinates are transformed according to the direction of the pixel connection line relative to the horizontal axis. Logarithmic interval rearrangement is performed on the radial coordinates to form a logarithmic polar coordinate image region. The grayscale changes of the logarithmic polar coordinate image region are decomposed into radial frequency coordinates and angular frequency coordinates. The conjugate product of the frequency component corresponding to the current working frame and the frequency component corresponding to the cropping reference block is calculated, and the frequency components are used... The amplitude product is normalized to obtain the Fourier-Merlin cross-power spectrum. The coordinates of the Fourier-Merlin cross-power spectrum with the highest correlation value exceeding the peak screening limit and the largest sum of neighborhood correlation values ​​are located. The frequency domain phase correlation peak data is obtained. The scale factor is calculated based on the radial peak coordinates, and the rotation parameter is calculated based on the angular peak coordinates. The current working frame is corrected based on the scale factor and rotation parameter. Then, the frequency domain translation parameter is determined by the horizontal and vertical displacements of the corresponding pixels after correction. Sub-pixel interpolation correction is performed on the frequency domain translation parameter, rotation parameter, and scale factor to generate a sub-pixel level spatial alignment matrix.

8. The visual inspection system for dust control in silicate cement plant areas according to claim 1, characterized in that, The steps for obtaining the spatial dust scatter points are as follows: Read the horizontal translation, vertical translation, rotation component, scale component and perspective component in the sub-pixel level spatial alignment matrix. According to the reverse coordinate relationship of each target pixel in the current working frame and its corresponding position in the panoramic background image, calculate the source coordinates of the current working frame pixel by pixel. For source coordinates falling between adjacent pixels, reconstruct the gray value according to the horizontal distance weight and the vertical distance weight. Mark the source coordinates that exceed the effective imaging range of the current working frame as empty pixels. Reconstruct the pixels according to the pixel row and column reference of the panoramic background image. Anchor the spatial position of the current working frame to the corresponding position in the panoramic background image to obtain the working frame after perspective transformation. Read the grayscale values ​​of the same pixel coordinates in the working frame after perspective transformation and the panoramic background image, calculate the grayscale change along the horizontal direction and the grayscale change along the vertical direction for each pixel coordinate, determine the gradient magnitude based on the grayscale change in the two directions, determine the gradient direction based on the deflection relationship between the horizontal grayscale change and the vertical grayscale change, calculate the absolute difference between the gradient magnitude corresponding to the working frame after perspective transformation and the gradient magnitude corresponding to the panoramic background image pixel by pixel, calculate the minimum circumferential difference between the two gradient directions, record the gradient magnitude difference, gradient direction difference and the continuity of spatial neighborhood change according to the pixel coordinates, and generate a feature difference map; Based on the feature difference map, the gradient magnitude difference, gradient direction difference, and spatial neighborhood change continuity are read according to pixel coordinates. A preset dynamic limit is determined based on the local gradient fluctuation range of the corresponding position in the panoramic background image. Pixel coordinates with gradient magnitude difference exceeding the magnitude limit and gradient direction difference exceeding the direction limit are retained. Isolated pixel coordinates and pixel coordinates with a continuous number not reaching the connectivity limit are deleted. Adjacent pixel coordinate sets in the connected state are divided according to the eight-neighbor contact relationship. The horizontal coordinate, vertical coordinate, difference intensity, and diffusion range of each adjacent pixel coordinate set are read. Cluster centers and covariance ranges are set based on the mean of horizontal coordinates, the mean of vertical coordinates, and the degree of coordinate dispersion. The probability of each pixel coordinate belonging to different cluster centers is calculated iteratively until the displacement of the cluster centers does not exceed the convergence limit. Clustering results where the center spacing does not exceed the convergence distance limit and the diffusion range overlaps are merged to generate spatial dust scatter points.

Images