Real-time detection method for uniformity of glycoside-containing compound fertilizer based on machine vision

Abstract

This invention relates to the field of material mixing monitoring technology. It discloses a machine vision-based real-time detection method for the mixing uniformity of glycoside-containing compound fertilizers. The method includes acquiring spray segment image frames and aligning them to obtain an aligned frame set; filtering clear segment sets and merging occluded segment sets; constructing a fogging layer from the occluded segment set and performing occlusion compensation to obtain a compensated frame set; combining the clear segment set and the compensated frame set into a sequence to be inspected; locating segregated patch sets based on a mixing reference template and generating a boundary constraint set to complete boundary integration and obtain a coherent patch set; generating a patch tracking chain from the coherent patch set and determining the cross-frame patch status to obtain a tracking state set; determining the mixing uniformity detection result based on the tracking state set and outputting the detection result set. This achieves sustainable patch tracking while reducing fogging intensity and enhancing patch boundary coherence.

Description

Technical Field

[0001] This invention relates to the field of material mixing monitoring technology, and more specifically, to a machine vision-based method for real-time detection of the mixing uniformity of glycoside-containing compound fertilizers. Background Technology

[0002] In the spray coating section of continuous production of glycoside compound fertilizer, it is usually necessary to detect the coating status of the material flow online and output the process judgment results under the constraints of limited installation space and production line time delay. Existing technologies mostly use grayscale and color difference judgment, local texture feature extraction, transform domain characterization, regional connectivity constraint, correlation measurement, time smoothing and screening to achieve patch recognition and status output. The above methods usually rely on the premise of stable imaging contrast, weak field of view occlusion, and relatively smooth material flow appearance changes.

[0003] In the spray coating process, the vapor fog formed by atomized spraying will cause the image contrast to decrease and local occlusion. The mirror contamination caused by the adhesion of glycoside-containing materials will cause local spot drift and boundary breakage. The above factors will jointly weaken the continuous support of gray-scale difference judgment and regional connectivity constraints for the patch boundary, making it difficult for temporal smoothing and screening to maintain the stable continuation of patch identification between consecutive frames. This will lead to the interruption of the patch tracking chain and the instability of the endpoint determination. Therefore, the technical problem that needs to be solved is how to ensure the continuous tracking of patches while reducing the intensity of fogging occlusion and enhancing the coherence of patch boundaries.

[0004] In view of this, the present invention proposes a machine vision-based real-time detection method for the mixing uniformity of glycoside-containing compound fertilizers to solve the above problems. Summary of the Invention

[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a real-time detection method for the mixing uniformity of glycoside-containing compound fertilizers based on machine vision.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, a machine vision-based method for real-time detection of the mixing uniformity of glycoside-containing compound fertilizers is provided, including: Acquire image frames of the spraying section, and perform displacement alignment on the image frames of the spraying section according to the conveying direction to obtain an aligned frame set. Filter the clear segment set from the aligned frame set, and merge the remaining image frames into an occluded segment set. Select a hybrid reference segment from the set of clear segments and stitch the hybrid reference segments together to obtain a hybrid reference template. Construct a fogging occlusion layer from the set of occluded segments and perform occlusion compensation on the set of occluded segments based on the fogging occlusion layer to obtain a compensation frame set. The set of clear fragments and the set of compensated frames are combined into a sequence to be inspected. Based on the hybrid reference template, the segregated region is located in the sequence to be inspected to obtain a set of segregated patches. A set of boundary constraints is generated in the set of segregated patches, and boundary integration is performed on the set of segregated patches according to the set of boundary constraints to obtain a set of coherent patches. A patch tracking chain is generated based on a coherent patch set, and the cross-frame patch state is determined according to the patch tracking chain to obtain a tracking state set. The mixed uniform detection result is determined based on the tracking state set, and the detection result set is output.

[0007] In some embodiments, a fogging occlusion layer is constructed from the set of occluded segments, and occlusion compensation is performed on the set of occluded segments based on the fogging occlusion layer to obtain a compensated frame set, including: The image frame sequence of each occluded segment is extracted from the occluded segment set, and the pixel position correspondence of the image frame sequence is determined according to the displacement alignment relationship corresponding to the alignment frame set, thus obtaining the occluded alignment frame set; Fog texture elements are extracted from the occlusion-aligned frame set and then grouped according to the image frame number to form a fog texture element set. Extract the fog coverage boundary from the fog texture metadata set, and fuse the fog coverage boundary according to the image frame number to form a fog masking layer; The fogging occlusion layer is mapped to the image frame coordinates of the occlusion alignment frame set. Available texture fragments within the same image frame are extracted outside the fogging occlusion layer coverage area. Regional texture statistical features are calculated based on the available texture fragments. Restoration enhancement parameters for the fogging occlusion layer coverage area are generated based on the regional texture statistical features, forming a compensation parameter set. Based on the compensation parameter set, the image content corresponding to the area covered by the fogging occlusion layer in the occlusion alignment frame set is subjected to dehazing restoration and enhancement processing, and the processed image frames are collected according to the image frame number to obtain the compensation frame set.

[0008] In some embodiments, the fog coverage boundary is extracted from the fog texture metadata set, and the fog coverage boundary is fused according to the image frame number to form a fog occlusion layer, including: Extract the texture direction features of the atomized texture elements from the atomized texture element set, and filter the atomized texture elements whose texture direction features are consistent with the delivery direction to form a candidate atomized texture element set; The center point coordinates of the candidate fog texture elements are extracted from the candidate fog texture element set, and spatially adjacent center point pairs are retrieved between adjacent image frames to obtain a set of spatially adjacent center point pairs. Based on the set of spatially adjacent center point pairs, construct the spatial adjacency association between candidate fog texture elements, and generate candidate connection paths along the spatial adjacency association to obtain the candidate skeleton path set; Project the fog coverage boundary of the candidate fog texture set onto the candidate skeleton path set, and fuse the projected fog coverage boundary according to the image frame number to obtain the fog occlusion layer.

[0009] In some embodiments, spatial adjacency associations between candidate fog texture elements are constructed based on a set of spatially adjacent center point pairs, and candidate connection paths are generated along these spatial adjacency associations to obtain a set of candidate skeleton paths, including: Convert the set of spatially adjacent center point pairs into a set of center point connection edges, and construct a center point association graph using the set of center point connection edges; Retrieve connected centroid chains from the centroid association graph and sort the connected centroid chains by image frame number to form a connection path sequence; In the connected path sequence, determine the path fork point and the path end point, and perform path segmentation on the connected path sequence based on the path fork point and the path end point to form a candidate connected path set; The candidate connection path set is aggregated according to the path connectivity relationship to obtain the candidate skeleton path set.

[0010] In some embodiments, a boundary constraint set is generated in the segregated patch set, and boundary integration is performed on the segregated patch set according to the boundary constraint set to obtain a coherent patch set, including: The outer contours of the segregated patches are extracted one by one from the set of segregated patches, and the boundary transition zones are located along the outer contours of the segregated patches to obtain the set of boundary transition zones; Boundary directional segments are extracted from the boundary transition zone and merged according to the projection relationship of the transport direction to obtain the projection boundary segment set. Based on the projection boundary segment set, candidate boundary segments that conform to the projection relationship are extracted from the segregated patch set, and the boundary constraint set is obtained by aggregating the candidate boundary segments. Based on the boundary constraint set, determine the splicability of the boundaries in the segregated patch set, and merge the segregated patches corresponding to the splicability of the boundaries to form a spliced ​​patch set; Boundary stitching and boundary resampling are performed on the patch set, and the patch set after boundary stitching is aggregated to obtain a coherent patch set.

[0011] In some embodiments, candidate boundary segments conforming to projection relationships are extracted from the segregated patch set based on the projection boundary segment set, and a boundary constraint set is obtained based on the set of candidate boundary segments, including: Obtain the outer contour of the segregated patches from the segregated patch set, and perform contour segmentation on the outer contour of the segregated patches according to the transport direction to obtain the boundary segmentation set; Extract the direction sequence of the boundary segment from the boundary segment set, and filter the boundary candidate segments according to the adjacent continuation relationship of the direction sequence to obtain the boundary candidate segment set; Extract endpoint pairs of the boundary candidate segments from the boundary candidate segment set, and construct an endpoint association graph based on the spatial adjacency relationship of the endpoint pairs; Retrieve connected endpoint chains in the endpoint association graph and map the connected endpoint chains to the boundary candidate segment set to form a boundary chain set; The boundary chain sets are aggregated according to the segregated patch identifiers, and the aggregated boundary chain sets are then combined to obtain the boundary constraint set.

[0012] In some embodiments, a patch tracking chain is generated based on a coherent patch set, and the cross-frame patch state is determined according to the patch tracking chain to obtain a tracking state set, including: Extract the outer contour and centroid of the continuous patch set, and organize the outer contour and centroid of the continuous patch according to the image frame order to obtain the frame-order patch table. Enumerate the consecutive patch pairs of adjacent image frames in the frame order patch table, and filter candidate consecutive patch pairs according to the coverage relationship of the outer contour of the consecutive patches to obtain the candidate association pair set. The unique set of association pairs is determined from the candidate association pairs, and the patch tracking chain is obtained by concatenating the unique set of association pairs. Based on the patch tracking chain, state transition segments of coherent patches across frames are extracted, and the state identifiers of patches across frames are determined according to the state transition segments to obtain the tracking state set.

[0013] In some embodiments, determining a unique set of association pairs from the candidate association pair set and concatenating the unique association pairs to obtain a patch tracing chain includes: Extract source coherent patch identifiers and target coherent patch identifiers from the candidate association pair set, and merge the candidate association pairs according to the source coherent patch identifiers to obtain the source merge candidate set; Extract the outer contours of source coherent patches and target coherent patches from the source merging candidate set, and filter the coverage-consistent candidate pairs according to the coverage relationship of the outer contours to obtain the coverage-consistent candidate set. Extract the source coherent patch centroid and the target coherent patch centroid from the consistent candidate set, determine the unique target coherent patch identifier based on the transport direction, and obtain a unique set of associated pairs. By concatenating the unique association pairs in the order of image frames and connecting the concatenated results according to the source coherent patch identifier, a patch tracing chain is obtained.

[0014] In some embodiments, source coherent patch centroids and target coherent patch centroids are extracted from the coverage consistency candidate set, and a unique target coherent patch identifier is determined based on the transport direction to obtain a unique set of associated pairs, including: Extract the centroids of source coherent patches and target coherent patches from the consistent candidate set, and generate a centroid connection set based on the source coherent patch centroids and target coherent patch centroids. Extract the direction vectors of the centroids from the set of centroids and merge the centroids according to the direction vectors to obtain the set of merged centroids. In the directional merging connection, the directional merging connection with the same direction as the conveying direction is selected, and the target continuous patch identifier corresponding to the directional merging connection with the same direction is locked to obtain a unique continuous patch identifier. The candidate association pairs corresponding to the unique coherent patch identifiers are recorded as unique association pairs, and the unique association pairs are aggregated to obtain a unique association pair set.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention acquires image frames of the spraying section and aligns them according to the conveying direction to obtain an aligned frame set. The clear segment set and the occluded segment set are separated, and the influence range of fogging occlusion and specular contamination on the patch boundary is first limited. A mixed reference segment is selected from the clear segment set and stitched together to obtain a mixed reference template. At the same time, a fogging occlusion layer is constructed from the occluded segment set, and occlusion compensation is performed on the occluded segment set to form a compensation frame set. The fogging occlusion intensity is weakened and the alignment error is absorbed. The clear segment set and the compensation frame set are combined into a sequence to be inspected. Based on the mixed reference template, the segregated region is located to obtain a segregated patch set. A boundary constraint set is generated in the segregated patch set, and boundary integration is performed to obtain a coherent patch set. The patch boundary remains coherent between consecutive frames. A patch tracking chain is generated from the coherent patch set, and the cross-frame patch state is determined accordingly to obtain a tracking state set. Finally, the mixed uniform detection result is determined according to the tracking state set, and the detection result set is output. Thus, sustainable patch tracking is achieved while reducing fogging occlusion intensity and enhancing patch boundary coherence. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the real-time detection method for the mixing uniformity of glycoside-containing compound fertilizer based on machine vision in this invention. Figure 2 This is a schematic diagram of the structure of the machine vision-based real-time detection system for the mixing uniformity of glycoside-containing compound fertilizers in this invention. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are some embodiments of the present invention, but not all embodiments. Based on the described embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] Unless otherwise defined, the technical or scientific terms used in this invention shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention pertains. The terms “first,” “second,” and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as “comprising” or “including” mean that the element or object preceding the term covers the element or object listed after the term and its equivalents, without excluding other elements or objects. Terms such as “connection” or “linked” are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect.

[0019] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data are all carried out in accordance with relevant laws, regulations, and standards, and necessary confidentiality measures have been taken. They do not violate public order and good morals, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0020] Example 1 Figure 1 This disclosure illustrates a machine vision-based real-time detection method for the mixing uniformity of glycoside-containing compound fertilizers, provided in at least one embodiment, including: S10: Acquire image frames of the spraying section, and perform displacement alignment on the image frames of the spraying section according to the conveying direction to obtain an aligned frame set. Filter the clear segment set in the aligned frame set and merge the remaining image frames into an occluded segment set. In this embodiment, image frames of the spraying section are acquired to form a continuous frame sequence input, making the appearance of the material flow in the spraying and coating section traceable on the time axis. This avoids accidental misjudgments caused by judging based on a single frame. Displacement alignment is performed on the image frames of the spraying section according to the conveying direction to eliminate the overall translation of the material flow caused by conveying. This ensures that the same material area in adjacent image frames falls on corresponding pixel positions, thus obtaining an aligned frame set to support subsequent cross-frame segment filtering and occlusion recognition. This alignment process is not a simple geometric translation and superposition, but rather establishes a correspondence between pixel positions between frames based on the conveying direction. This ensures that the texture and boundaries will not break due to inter-frame drift after alignment, resulting in clearer images in the aligned frame set. The fragment set is used to distinguish frames with discernible imaging contrast and texture from those caused by fogging or mirror contamination, providing a stable basis for subsequent selection of hybrid reference fragments. The remaining image frames are grouped into the occlusion fragment set for centralized management of occluded attenuated frames, facilitating the unified construction of fogging layers and the execution of occlusion compensation. Combined with the spray coating process, clear fragment selection after displacement alignment can reduce the interference of local occlusion caused by fogging on the consistency between frames, avoiding misclassification of the same material area as different clarity categories on different frames. This ensures that the occlusion fragment set can continuously cover the fogging period and provide complete input for the generation of subsequent compensation frame sets.

[0021] For example, five spraying segment image frames are continuously acquired along the conveying direction and numbered 1, 2, 3, 4, 5 in the acquisition order. After displacement alignment, an aligned frame set {1, 2, 3, 4, 5} is obtained. After alignment, a clear segment set {1, 2, 5} is obtained by filtering from the aligned frame set. The remaining image frames are merged to obtain an occluded segment set {3, 4}. This segment division is used in subsequent steps to support the construction of the hybrid reference template with the clear segment set and to support the construction of the atomized occlusion layer and occlusion compensation with the occlusion segment set, thereby generating a compensation frame set.

[0022] S20: Select a mixed reference segment from the set of clear segments, and stitch the mixed reference segments together to obtain a mixed reference template. Construct a fogging occlusion layer from the set of occlusion segments, and perform occlusion compensation on the set of occlusion segments according to the fogging occlusion layer to obtain a compensation frame set. A fogging occlusion layer is constructed from the set of occluded segments, and occlusion compensation is performed on the set of occluded segments based on the fogging occlusion layer to obtain a set of compensated frames, including: The image frame sequence of each occluded segment is extracted from the occluded segment set, and the pixel position correspondence of the image frame sequence is determined according to the displacement alignment relationship corresponding to the alignment frame set, thus obtaining the occluded alignment frame set; Fog texture elements are extracted from the occlusion-aligned frame set and then grouped according to the image frame number to form a fog texture element set. Extract the fog coverage boundary from the fog texture metadata set, and fuse the fog coverage boundary according to the image frame number to form a fog masking layer; The fogging occlusion layer is mapped to the image frame coordinates of the occlusion alignment frame set. Available texture fragments within the same image frame are extracted outside the fogging occlusion layer coverage area. Regional texture statistical features are calculated based on the available texture fragments. Restoration enhancement parameters for the fogging occlusion layer coverage area are generated based on the regional texture statistical features, forming a compensation parameter set. Based on the compensation parameter set, the image content corresponding to the area covered by the fogging occlusion layer in the occlusion alignment frame set is subjected to dehazing restoration and enhancement processing, and the processed image frames are collected according to the image frame number to obtain the compensation frame set.

[0023] In this embodiment, the hybrid reference segment is a frame segment selected from the set of clear segments that can stably present the apparent texture and tonal distribution of the spray coating section material flow. The hybrid reference template is an intra-frame consistent reference image formed by stitching the hybrid reference segments. It is used to provide a comparable reference texture and boundary shape for the localization of segregated areas in subsequent steps. Stitching is a common image combination operation in the art. Here, it is emphasized that the stitched template maintains texture continuity and complete field of view coverage in the transport direction. The fogging masking layer is a structured representation of the fogging fog coverage range of the masking segment set. The fogging masking layer is not the same as a simple masking rectangle, but a set of coverage boundaries obtained by fusing according to the image frame number. This allows the occlusion compensation to perform directional restoration and enhancement on the area covered by the fogging masking layer without interfering with the usable texture area. By constructing the fogging masking layer first and then performing occlusion compensation, the contrast attenuation and local occlusion caused by the fogging fog can be transformed from "uncontrollable noise" into "localizable and recoverable areas". This provides continuous and stable input for the compensation frame set and reduces the risk of subsequent patch tracking chain interruption.

[0024] Determining pixel position correspondences based on the displacement alignment relationships corresponding to the aligned frame set ensures that the image frame sequence of the occluded segment is consistent with the reference coordinates of the clear segment. This allows fog texture elements, fog coverage boundaries, and restoration enhancement parameters to be generated and applied within a unified coordinate system. Fog texture elements are the smallest texture units extracted from the occluded aligned frame set that characterize the direction of the fog texture and local graininess. Fog texture elements are grouped according to image frame numbers to form a fog texture element set, maintaining a continuous record of the fog morphology on the time axis. The fog coverage boundary is the outer contour of the fog coverage area extracted from the fog texture element set and fused according to image frame numbers. A fogging occlusion layer is used to stably describe the continuity and drift of the fog coverage area. The restoration and enhancement parameters are a set of parameters generated for the area covered by the fogging occlusion layer. The parameter generation does not rely on subjective threshold adjustment, but uses available texture fragments outside the fogging occlusion layer coverage area as a reference to calculate the texture statistical features of the area, so that the occlusion compensation has a comparable texture benchmark within the same frame. The defogging restoration and enhancement processing restores and enhances the image content in the fogging occlusion layer coverage area according to the compensation parameter set, and the processed image frames are collected according to the image frame sequence to obtain the compensation frame set, so that the occluded fragments have recognizable texture and boundary continuity similar to the clear fragments on the output side.

[0025] For example, continuing with the aforementioned aligned frame set {1,2,3,4,5}, clear fragment set {1,2,5}, and occluded fragment set {3,4}, a blending reference fragment {1,2} is selected from the clear fragment set and spliced ​​to obtain a blending reference template. Image frame sequences 3 and 4 are extracted one by one from the occluded fragment set {3,4}. Based on the displacement alignment relationship of the aligned frame set, the pixel position correspondence between frame 3 and frame 4 in the transport direction is determined to obtain the occluded aligned frame set {3,4}. Fog texture elements are extracted from the occluded aligned frame set {3,4} and grouped according to the image frame number to form a fog texture element set {(3,u31,u32), (4,u41,u42)}, where u31 and u32 represent two fog texture elements in image frame 3, and u41 and u42 represent two fog texture elements in image frame 4. A set of fog texture elements is used to extract fog coverage boundaries and fuse them according to image frame numbers to form a fog occlusion layer. The fog occlusion layer is then mapped to the coordinates of image frames 3 and 4. Usable texture fragments are extracted outside the fog occlusion layer coverage area, and regional texture statistical features are calculated to generate a compensation parameter set {(3,p3), (4,p4)}, where p3 represents the restoration and enhancement parameters corresponding to image frame 3, and p4 represents the restoration and enhancement parameters corresponding to image frame 4. Finally, based on the compensation parameter set, defogging restoration and enhancement processing is performed on the fog occlusion layer coverage area of ​​image frames 3 and 4, and the compensation frame set {3′,4′} is obtained by merging them according to image frame numbers. This compensation frame set, together with the clear fragment set, supports the construction of subsequent test sequences and maintains the continuous extraction of segregated patch boundaries.

[0026] The fog coverage boundary is extracted from the fog texture metadata set, and the fog coverage boundary is fused according to the image frame number to form a fog occlusion layer, including: Extract the texture direction features of the atomized texture elements from the atomized texture element set, and filter the atomized texture elements whose texture direction features are consistent with the delivery direction to form a candidate atomized texture element set; The center point coordinates of the candidate fog texture elements are extracted from the candidate fog texture element set, and spatially adjacent center point pairs are retrieved between adjacent image frames to obtain a set of spatially adjacent center point pairs. Based on the set of spatially adjacent center point pairs, construct the spatial adjacency association between candidate fog texture elements, and generate candidate connection paths along the spatial adjacency association to obtain the candidate skeleton path set; Project the fog coverage boundary of the candidate fog texture set onto the candidate skeleton path set, and fuse the projected fog coverage boundary according to the image frame number to obtain the fog occlusion layer.

[0027] Understandably, the fog coverage boundary describes the boundary of the fog coverage area corresponding to the fog texture element. The fog masking layer is a cross-frame coverage layer obtained by fusing the fog coverage boundary according to the image frame number. It is used to stably indicate the coverage area and drift trajectory of the fog fog in the masking alignment frame set. The texture direction feature is used to characterize the main direction arrangement trend of the fog texture elements. The transport direction is the main movement direction of the spray coating section material flow in the field of view. The candidate fog texture element set is the screening result of the fog texture element set. Only fog texture elements with texture direction features consistent with the transport direction are retained. This excludes the texture interference of non-fog factors such as specular pollution spots and local reflection stripes from the fog masking layer construction path, making the fog masking layer closer to the spatial continuity of the fog curtain body.

[0028] In this embodiment, retrieving spatially adjacent center point pairs between adjacent image frames is to transform the correspondence of candidate fog texture elements on the time axis into a set of spatially adjacent center point pairs. This allows subsequent spatial adjacency associations to represent the cross-frame continuity of the fog coverage area with a connectable chain of center points. The set of spatially adjacent center point pairs is not a simple nearest neighbor matching, but rather establishes pairs of spatially adjacent center points in adjacent frames based on the unified coordinates of the occlusion-aligned frame set. This forms a stable connected structure at the center point level. Constructing spatial adjacency associations and generating candidate connection paths based on the set of spatially adjacent center point pairs is equivalent to extracting a chain of center points that can continuously run through multiple frames in the center point association graph. This ensures that the main skeleton trend of the fog fog maintains a continuous expression between frames. The set of candidate skeleton paths serves as the carrying path for subsequent projections. It can merge the fog coverage boundary of the candidate fog texture element set along the skeleton path, thereby transforming the cross-frame boundary break into a boundary fusion along the skeleton path.

[0029] For example, continuing with the aforementioned fog texture set {(3,u31,u32), (4,u41,u42)}, texture direction features are extracted from the fog texture set and compared with the transport direction to obtain candidate fog texture sets {(3,u31), (4,u41,u42)}. Among them, u32 is eliminated because its texture direction is inconsistent with the transport direction. The center point coordinates are extracted from the candidate fog texture set to obtain {(3,c31), (4,c41,c42)}. Spatially adjacent center point pairs are retrieved between adjacent image frames 3 and 4 to obtain a set of spatially adjacent center point pairs {(c31,c41), (c31,c42)}. This set of center point pairs is used to construct the candidate... Spatial adjacency associations are established between fog texture elements, and candidate connection paths are generated along these associations to obtain a candidate skeleton path set {(c31→c41), (c31→c42)}. Subsequently, the fog coverage boundaries corresponding to the candidate fog texture element set are projected onto the candidate skeleton path set and fused according to the image frame number to obtain a fog occlusion layer {(3,b3), (4,b4)}, where b3 represents the fog coverage boundary fusion result of image frame 3 and b4 represents the fog coverage boundary fusion result of image frame 4. This fog occlusion layer is used to locate the fog occlusion layer coverage area in subsequent occlusion compensation and generate a compensation parameter set {(3,p3), (4,p4)} to obtain the compensation frame set {3′,4′}.

[0030] Based on the set of spatially adjacent center point pairs, spatial adjacency relationships are constructed between candidate fog texture elements, and candidate connection paths are generated along these spatial adjacency relationships to obtain a set of candidate skeleton paths, including: Convert the set of spatially adjacent center point pairs into a set of center point connection edges, and construct a center point association graph using the set of center point connection edges; Retrieve connected centroid chains from the centroid association graph and sort the connected centroid chains by image frame number to form a connection path sequence; In the connected path sequence, determine the path fork point and the path end point, and perform path segmentation on the connected path sequence based on the path fork point and the path end point to form a candidate connected path set; The candidate connection path set is aggregated according to the path connectivity relationship to obtain the candidate skeleton path set.

[0031] It should be noted that the center point connection edge set is the edge set directly transformed from the set of spatially adjacent center point pairs, used to convert center point pairs into connectable edges. The center point association graph is a graph structure with center points as nodes and center point connection edges as edges. A connected center point chain refers to a sequence of center points that can be continuously traversed along the edges in the center point association graph. The connection path sequence is a serialized expression of the connected center point chain after sorting it by image frame number. The path branching point is the center point in the connected center point chain that corresponds to multiple subsequent edges. The path termination point is the center point in the connected center point chain that no longer has any subsequent edges. The candidate connection path set is the path set after the connection path sequence is divided according to the branching and termination rules. The candidate skeleton path set is the skeleton-level path set after the candidate connection paths are gathered according to the path connectivity relationship, used to carry the projection and fusion of the subsequent fog coverage boundary.

[0032] Converting the set of spatially adjacent center point pairs into a set of center point connecting edges and constructing a center point association graph is to transform the spatial adjacency relationship between adjacent image frames from a paired record into a traversable structure. This facilitates the direct retrieval of continuous connected center point chains across frames in the center point association graph, thereby solidifying the cross-frame continuation relationship of fog texture elements into a stable connected chain. The connected center point chains are sorted by image frame number to form a connection path sequence. This is to avoid disordered concatenation of center points within the same image frame in the path, which would cause path jumps. It ensures that the connection path sequence maintains a consistent time progression with the image frame number, thus enabling subsequent candidate connection paths to correspond to the continuous trend of the fog in the transport direction.

[0033] In this embodiment, determining the path bifurcation point and the path termination point in the connection path sequence and performing path segmentation is to split different branches into mutually exclusive candidate connection paths when there are multiple continuous branches in the fog texture element. This avoids misconnecting the local light spot drift caused by mirror contamination into the main path of the fog skeleton. At the same time, the path at the termination point is truncated to avoid mixing the short chain noise caused by the sudden change in occlusion intensity with the long chain main path. After forming a set of candidate connection paths, they are then aggregated according to the path connectivity to obtain a set of candidate skeleton paths. This allows multiple candidate connection paths under the same fog coverage direction to be organized into a skeleton-level set, providing a consistent bearing framework for the subsequent projection of the fog coverage boundary on the candidate skeleton path set.

[0034] For example, continuing with the aforementioned set of spatially adjacent center point pairs {(c31,c41),(c31,c42)}, the set of spatially adjacent center point pairs is transformed into a set of center point connecting edges to obtain {e311,e312}, where e311 connects c31 and c41, and e312 connects c31 and c42. A center point association graph is constructed using this set of center point connecting edges to obtain G={V,E}, where V={c31,c41,c42} and E={e311,e312}. Connecting center point chains are retrieved from the center point association graph to obtain {(c31→c41),(c31→c42)}, and then sorted by image frame number to form a connection path sequence P={(3:c31→4: In the connected path sequence, the path bifurcation point is determined to be c31, and the path termination points are c41 and c42. Based on the path bifurcation point and the path termination point, the connected path sequence is segmented to form a candidate connected path set {p1:(c31→c41),p2:(c31→c42)}. The candidate connected path set is then aggregated according to the path connectivity relationship to obtain a candidate skeleton path set {p1,p2}. This candidate skeleton path set is used to merge the fog coverage boundaries b3 and b4 along p1 or p2 respectively in the subsequent projection to obtain the fog masking layer {(3,b3),(4,b4)} and maintain the continuous expression of cross-frame coverage.

[0035] S30: The set of clear fragments and the set of compensated frames are combined into a sequence to be inspected. Based on the hybrid reference template, the segregated region is located in the sequence to be inspected to obtain a set of segregated patches. A set of boundary constraints is generated in the set of segregated patches, and boundary integration is performed on the set of segregated patches according to the set of boundary constraints to obtain a set of coherent patches. A set of boundary constraints is generated from the segregated patch set, and boundary integration is performed on the segregated patch set based on the set of boundary constraints to obtain a coherent patch set, including: The outer contours of the segregated patches are extracted one by one from the set of segregated patches, and the boundary transition zones are located along the outer contours of the segregated patches to obtain the set of boundary transition zones; Boundary directional segments are extracted from the boundary transition zone and merged according to the projection relationship of the transport direction to obtain the projection boundary segment set. Based on the projection boundary segment set, candidate boundary segments that conform to the projection relationship are extracted from the segregated patch set, and the boundary constraint set is obtained by aggregating the candidate boundary segments. Based on the boundary constraint set, determine the splicability of the boundaries in the segregated patch set, and merge the segregated patches corresponding to the splicability of the boundaries to form a spliced ​​patch set; Boundary stitching and boundary resampling are performed on the patch set, and the patch set after boundary stitching is aggregated to obtain a coherent patch set.

[0036] The sequence to be inspected is sequence data obtained by collecting the clear fragment set and the compensation frame set according to the image frame number. The mixed reference template is reference texture and reference tone reference data obtained by stitching mixed reference fragments. The segregated region localization is to locate local regions in the sequence to be inspected that are inconsistent with the reference appearance and output the region boundary by using the mixed reference template as a reference. The segregated patch set is the set of patch identifiers and patch boundaries output by the segregated region localization. The boundary constraint set is the set of boundary chains extracted from the segregated patch set that can be used for boundary stitching judgment. The coherent patch set is the patch set after merging the segregated patches that can be stitched according to the boundary constraint set and completing the boundary continuity.

[0037] It should be noted that the extraction of the outer contour of the segregated patch is used to convert the segregated patch from the pixel region into a processable contour point sequence. The boundary transition zone is used to characterize the narrow band region on the outer contour of the segregated patch where grayscale and texture transition from the patch side to the background side. This facilitates locking the stable boundary segment even under conditions of local spot drift and boundary breakage caused by specular contamination. The boundary direction segment is used to divide the contour point sequence in the boundary transition zone into short segments according to the direction continuity. The projection relationship of the transport direction is used to map the short segments to the projection axis of the transport direction and perform reduction to form a consistent projection boundary segment set across frames. For example, continuing the aforementioned fogging layer boundaries b3 and b4 of image frames 3 and 4, the outer contour point sequence of the segregated patch in image frame 3 is (10,5), (14,5), (14,9), (10,9), and the outer contour point sequence of the segregated patch in image frame 4 is (12,5), (16,5), (16,9), (12,9). The boundary direction segment is extracted from the boundary transition zone to obtain: {((10,5),(10,9)),((14,5),(14,9))}; {((12,5),(12,9)),((16,5),(16,9))}; The boundary segments are projected and merged according to the transport direction to obtain the projected boundary segment set: {(x10,y5-9),(x12,y5-9),(x14,y5-9),(x16,y5-9)}.

[0038] In this embodiment, the purpose of extracting candidate boundary segments based on the projection boundary segment set is to reflect the consistent projection segments across frames back onto the outer contour of the segregated patch, thereby supplementing the boundary breaks with continuous candidate boundary segments. The purpose of the boundary constraint set processing of the candidate boundary segments is to organize the candidate boundary segments under the same segregated patch identifier into boundary chains. The determination of the boundary splicing relationship is used to determine which segregated patches have continuous splicing boundary chain pairs. The splicing patch group set is used to merge segregated patches with boundary splicing relationships into a splicing group, for example, using the projection boundary segment set {(x10, y5-9)}. (x12,y5-9)}, backtracking in image frame 3 and image frame 4 respectively to obtain the boundary candidate segment set {((10,5),(10,9)),((12,5),(12,9))}, the boundary candidate segment set is grouped according to the segregated patch identifier to obtain the boundary constraint set {A:{((10,5),(10,9)),((12,5),(12,9))}}, the boundary splicing relationship is determined according to the boundary constraint set to obtain {(A frame 3,A frame 4)}, and the segregated patches corresponding to (A frame 3,A frame 4) are merged to form the spliced ​​patch group set {G1:{A frame 3,A frame 4}}.

[0039] Understandably, boundary stitching is used to align the endpoints of boundary chains within the same patch set according to their splicability and connect them into a continuous boundary chain. Boundary resampling is used to resample the point sequence of the continuous boundary chain at a fixed point spacing to avoid jagged boundaries caused by local noise. The aggregation of coherent patch sets is used to output the patch set that has completed boundary stitching and boundary resampling as a traceable coherent patch set. Thus, under the conditions of contrast attenuation caused by the fogging layer and spot drift caused by specular contamination, the continuous support of the patch boundary is still maintained and the impact of boundary breakage on the subsequent patch tracking chain is reduced.

[0040] For example, using the patch set {G1:{Frame A3,Frame A4}}, the continuous boundary chain after boundary stitching is (10,5), (12,5), (16,5), (16,9), (12,9), (10,9); Boundary resampling was performed on the continuous boundary chain to obtain: (10,5),(13,5),(16,5),(16,7),(16,9),(13,9),(10,9),(10,7); The spliced ​​patch sets after boundary resampling are aggregated to obtain a coherent patch set {Acoherent: {(10,5),(13,5),(16,5),(16,7),(16,9),(13,9),(10,9),(10,7)}}.

[0041] Based on the projection boundary segment set, candidate boundary segments conforming to the projection relationship are extracted from the segregated patch set, and a boundary constraint set is obtained based on the set of candidate boundary segments, including: Obtain the outer contour of the segregated patches from the segregated patch set, and perform contour segmentation on the outer contour of the segregated patches according to the transport direction to obtain the boundary segmentation set; Extract the direction sequence of the boundary segment from the boundary segment set, and filter the boundary candidate segments according to the adjacent continuation relationship of the direction sequence to obtain the boundary candidate segment set; Extract endpoint pairs of the boundary candidate segments from the boundary candidate segment set, and construct an endpoint association graph based on the spatial adjacency relationship of the endpoint pairs; Retrieve connected endpoint chains in the endpoint association graph and map the connected endpoint chains to the boundary candidate segment set to form a boundary chain set; The boundary chain sets are aggregated according to the segregated patch identifiers, and the aggregated boundary chain sets are then combined to obtain the boundary constraint set.

[0042] In this embodiment, the boundary segment set is a set of contour segments obtained by segmenting the outer contour of the segregated patch according to the projection axis of the transport direction; the orientation sequence is a sequence of local orientation changes extracted by point order for each boundary segment; the endpoint pair is an ordered coordinate pair consisting of the starting coordinates and the ending coordinates of the candidate boundary segment; the endpoint association graph is a graph structure formed by using the endpoint coordinates as nodes and the spatial adjacency relationship as connecting edges; the connected endpoint chain is a sequence of endpoints that satisfy the connectivity relationship in the endpoint association graph; the boundary chain set is the concatenation result of the candidate segments after mapping the connected endpoint chains back to the candidate boundary segment set; and the boundary constraint set is a set of boundary chains obtained by aggregating the boundary chain set with the segregated patch identifier as the aggregator key.

[0043] Understandably, the purpose of performing contour segmentation on the outer contour of the segregated patch according to the transport direction is to transform the long outer contour into a short segment structure that can be aligned with the projected boundary segment set, so that usable boundary segments can still be retained when local breaks or local spot drift occur. The purpose of extracting the direction sequence in the boundary segment set and filtering the boundary candidate segments according to the adjacent continuity relationship is to eliminate short and abrupt turn-back segments and isolated noise segments, and only retain boundary candidate segments with continuous direction and consistent with the projection relationship of the transport direction, so that subsequent endpoint association can be performed for connectivity retrieval under a unified directional caliber and maintain feasibility.

[0044] The purpose of extracting endpoint pairs from the boundary candidate segment set and constructing an endpoint association graph based on the spatial adjacency of the endpoint pairs is to transform the boundary candidate segments from a set of line segments into a graph structure. This allows for the recovery of connectable boundary chains even when there are gaps between boundary candidate segments, based on endpoint adjacency. The purpose of retrieving connected endpoint chains in the endpoint association graph and mapping them to the boundary candidate segment set to form a boundary chain set is to obtain candidate connection paths that can be used to determine the boundary splicability relationship. The boundary chain set is then aggregated according to the partial patch identifier to obtain a boundary constraint set to support the subsequent boundary integration processing of coherent patch sets. Data examples are shown in Table 1 below: Table 1 Data Items content Segregation plaque identification A Boundary candidate segment endpoint pairs C1:(10,5)-(10,9), C2:(13,5)-(13,9), C3:(16,5)-(16,9) Spatial Adjacency Connectivity Edge Set (10,5)-(13,5),(13,5)-（16,5），(10,9)-（13,9），(13,9)-（16,9） Connected endpoint chain set L1:(10,5)-(13,5)-(16,5), L2:(10,9)-(13,9)-(16,9) Boundary constraint set A: {L1 corresponds to the boundary chain, L2 corresponds to the boundary chain}. S40: Generate a patch tracking chain based on a coherent patch set, determine the cross-frame patch state based on the patch tracking chain, obtain a tracking state set, determine the mixed uniform detection result based on the tracking state set, and output the detection result set.

[0045] A patch tracking chain is generated based on a coherent patch set, and the cross-frame patch state is determined according to the patch tracking chain to obtain a tracking state set, including: Extract the outer contour and centroid of the continuous patch set, and organize the outer contour and centroid of the continuous patch according to the image frame order to obtain the frame-order patch table. Enumerate the consecutive patch pairs of adjacent image frames in the frame order patch table, and filter candidate consecutive patch pairs according to the coverage relationship of the outer contour of the consecutive patches to obtain the candidate association pair set. The unique set of association pairs is determined from the candidate association pairs, and the patch tracking chain is obtained by concatenating the unique set of association pairs. Based on the patch tracking chain, state transition segments of coherent patches across frames are extracted, and the state identifiers of patches across frames are determined according to the state transition segments to obtain the tracking state set.

[0046] In this embodiment, the frame-order patch table is a structured record table indexed by the image frame number and containing entries of the outer contour and centroid of the continuous patches. The candidate association pair set is a set of continuous patch pairs that satisfy the outer contour coverage relationship within adjacent image frames. The unique association pair set is a set of association pairs after retaining only one target continuous patch identifier for the same source continuous patch identifier. The patch tracking chain is a chain record formed by concatenating the unique association pair sets according to the image frame number. The state switching segment is a continuous frame segment extracted from the patch tracking chain and includes the observation entries required for the state change of the continuous patches within the segment. The cross-frame patch state identifier is the labeling result of the continuous patches in the state switching segment in the three states of entry state, continuation state, and exit state. The tracking state set is a set obtained by aggregating the cross-frame patch state identifiers of each patch tracking chain according to the image frame number.

[0047] Understandably, the purpose of extracting the outer contours and centroids of coherent patches from the coherent patch set and organizing them according to the image frame order to obtain the frame-order patch table is to transform the coherent patch set from a frame-by-frame image representation into an enumerable temporal record, so that subsequent association decisions can be made under the same coordinate caliber in adjacent image frames. The outer contours of coherent patches are used to provide geometric boundary input for determining coverage relationships, and the centroids of coherent patches are used to provide a basis for determining directional consistency when there are multiple solutions to coverage relationships, thereby avoiding the interruption of the tracking chain caused by local boundary breaks due to fogging and specular contamination.

[0048] The purpose of enumerating consecutive patch pairs in the frame order patch table and filtering them based on the coverage relationship of the outer contours of consecutive patches to obtain a set of candidate association pairs is to first establish a conservative candidate set within the spatial overlap range of the outer contours, so that association judgment is performed only within a small number of candidate pairs without introducing large-scale matching overhead, and to ensure that the boundary caliber of the test sequence formed by the compensated frame set after occlusion compensation and the clear fragment set remains consistent when entering the tracking stage. Data examples are shown in Table 2 below: Table 2 Image frame number Continuous patch identification coherent plaque centroid Continuous patch outer contour bounding box 21 A (12,7) (10,5)-(16,9) 22 B (13,7) (11,5)-(17,9) 22 C (30,18) (28,16)-(32,20) In this embodiment, when screening candidate continuous patch pairs based on the outer contour coverage relationship, continuous patch identifier A with image frame number 21 and continuous patch identifier B with image frame number 22 are used to form candidate association pair A to B. Continuous patch identifier A and continuous patch identifier C are excluded to avoid mistakenly connecting patches far from the coverage area to the tracking chain. This allows the candidate association pair set to focus on the cross-frame continuity relationship of the same material flow appearance area and to provide a data basis for the determination of unique association pairs.

[0049] It should be noted that the purpose of determining a unique set of association pairs in the candidate association pair set and concatenating them to obtain the patch tracking chain is to eliminate the ambiguity of one-to-many and many-to-one associations, so that each patch tracking chain maintains a single continuous path between adjacent image frames. This allows the extraction of subsequent state transition segments to have a definite chain index and reduces temporal smoothing dependencies. In the table example above, candidate association pairs A to B are recorded as unique association pairs and concatenated according to the image frame number to form tracking chain entries. A corresponds to B in frame 22, so that the patch tracking chain has a verifiable correspondence between consecutive frames.

[0050] The purpose of extracting state transition segments of continuous patches across frames based on the patch tracking chain and determining the state identifiers of patches across frames based on the state transition segments to obtain the tracking state set is to transform the chain-like geometric continuity relationship into state data that can be used to determine the mixed uniform detection results. This allows the three types of states—entry state, continuation state, and exit state—to be consistently recorded in time and directly used for the generation of subsequent detection result sets. Compared with the existing method that only relies on grayscale difference determination and time smoothing, this processing ensures that the local contrast attenuation caused by fogging and the boundary breakage caused by specular contamination no longer directly lead to the interruption of the tracking chain, thereby improving the stability of cross-frame state determination and reducing the occurrence of instability in endpoint determination.

[0051] The unique set of association pairs is determined from the candidate association pair set, and the patch tracing chain is obtained by concatenating the unique association pair sets, including: Extract source coherent patch identifiers and target coherent patch identifiers from the candidate association pair set, and merge the candidate association pairs according to the source coherent patch identifiers to obtain the source merge candidate set; Extract the outer contours of source coherent patches and target coherent patches from the source merging candidate set, and filter the coverage-consistent candidate pairs according to the coverage relationship of the outer contours to obtain the coverage-consistent candidate set. Extract the source coherent patch centroid and the target coherent patch centroid from the consistent candidate set, determine the unique target coherent patch identifier based on the transport direction, and obtain a unique set of associated pairs. By concatenating the unique association pairs in the order of image frames and connecting the concatenated results according to the source coherent patch identifier, a patch tracing chain is obtained.

[0052] The source coherent patch centroid and the target coherent patch centroid are extracted from the consistent candidate set. A unique target coherent patch identifier is determined based on the transport direction, resulting in a unique set of associated pairs, including: Extract the centroids of source coherent patches and target coherent patches from the consistent candidate set, and generate a centroid connection set based on the source coherent patch centroids and target coherent patch centroids. Extract the direction vectors of the centroids from the set of centroids and merge the centroids according to the direction vectors to obtain the set of merged centroids. In the directional merging connection, the directional merging connection with the same direction as the conveying direction is selected, and the target continuous patch identifier corresponding to the directional merging connection with the same direction is locked to obtain a unique continuous patch identifier. The candidate association pairs corresponding to the unique coherent patch identifiers are recorded as unique association pairs, and the unique association pairs are aggregated to obtain a unique association pair set.

[0053] In this embodiment, the source merged candidate set is the grouping result after grouping the candidate association pair set according to the source coherent patch identifier; the coverage consistent candidate set is the set of candidate pairs that satisfy the outer contour coverage relationship in each group; the centroid connection set is the set of line segments determined by the centroid of the source coherent patch and the centroid of the target coherent patch; the connection direction vector is the direction representation extracted from each connection in the centroid connection set; the direction merged connection set is the set of connections merged according to the connection direction vector; the unique coherent patch identifier is a single target identifier determined from multiple target coherent patch identifiers corresponding to the same source coherent patch based on the transport direction; and the unique association pair is an association pair entry composed of the source coherent patch identifier and the unique coherent patch identifier.

[0054] The purpose of extracting source coherent patch identifiers and target coherent patch identifiers from the candidate association pair set and merging candidate association pairs according to the source coherent patch identifiers to obtain the source merged candidate set is to limit the ambiguity of cross-frame association to a local set of the same source coherent patch identifiers, so that subsequent coverage relationship screening and transport direction determination are completed within the candidate range of the same source coherent patch identifiers, thereby avoiding mutual interference between candidate pairs of different source coherent patch identifiers and reducing unnecessary matching calculations.

[0055] It should be noted that the purpose of extracting the outer contours of source coherent patches and target coherent patches from the source merged candidate set and filtering candidate pairs with consistent coverage based on the coverage relationship is to first exclude obviously discontinuous candidate associations based on geometric coverage relationship, so that the subsequent centroid direction determination only faces candidate pairs with consistent outer contour coverage relationship, thereby reducing the impact of local texture loss caused by fogging on centroid estimation and maintaining the spatial coherence of the patch tracking chain.

[0056] Using the entries for image frames 21 and 22 from the previous text, when the source coherent patch identifier is A and the candidate target coherent patch identifier formed in image frame 22 includes B and C, the structured results of the source merge candidate set and the coverage consistent candidate set are shown in Table 3 below: Table 3 Source coherent patch identification Candidate target coherent patch identification Outer contour bounding box coverage relationship Where does the merge result go? A B Coverage Consistency Coverage Consistent Candidate Set A C Not covered Source merge candidate set is retained but not included in the coverage consistency candidate set. It should be noted that the purpose of extracting the centroids of source coherent patches and target coherent patches from the coverage consistency candidate set and determining the unique target coherent patch identifier based on the transport direction to obtain a unique association pair set is to solve the situation where the same source coherent patch identifier may correspond to multiple coverage consistency candidate pairs in adjacent image frames. This ensures that the unique association pair set satisfies that each source coherent patch identifier corresponds to only one target coherent patch identifier, thereby guaranteeing that the unique association pair set can be stably concatenated in the order of image frames to form a sustainable patch tracking chain.

[0057] The processing action of generating a centroid connection set based on the centroids of source coherent patches and target coherent patches is used to convert each coverage consistency candidate pair into a comparable direction object, so that the transport direction determination does not directly depend on the local details of the outer contour but on the overall displacement trend between centroids. In the example above, the centroid of source coherent patch identifier A is (12,7), the centroid of target coherent patch identifier B is (13,7), and the centroid of target coherent patch identifier C is (30,18). Then the centroid connection set can be recorded as (12,7)-(13,7) from A to B, and (12,7)-(30,18) from A to C. This centroid connection set represents the cross-frame displacement direction and displacement trend of the same source coherent patch identifier pointing to different target coherent patch identifiers.

[0058] Understandably, the purpose of extracting the connection direction vectors from the centroid connection set and merging the centroid connections according to the connection direction vectors to obtain the direction-merged connection set is to transform the centroid connections from coordinate pairs to direction categories, so that candidate pairs in the same direction can be uniformly identified and used as input for direction determination. In the example above, the connection direction vector from A to B can be represented as (1,0), and the connection direction vector from A to C can be represented as (18,11). The direction-merged connection set is thus formed into two merged entries and associated with the corresponding target coherent patch identifier set. The purpose of selecting directional merging lines that are in the same direction as the conveying direction and locking the target continuous patch identifier corresponding to the directional merging line is to introduce the process conveying direction as an objective constraint to determine uniqueness, so that the patch tracking chain is more in line with the movement law of the material flow along the conveying direction and reduces the probability of reverse jump connection. In the example above, when the direction representation corresponding to the conveying direction is consistent with (1,0), the target continuous patch identifier corresponding to the directional merging line is locked as B, and B is determined as the unique continuous patch identifier.

[0059] In this embodiment, the purpose of recording the candidate association pairs corresponding to the unique coherent patch identifiers as unique association pairs and aggregating the unique association pairs to obtain a unique association pair set is to ensure that the uniqueness determination result falls into a concatenable data structure. In the example above, the unique association pair can be recorded as (A, B) and incorporated into the unique association pair set, so that the unique association pair set maintains a one-to-one correspondence between the source coherent patch identifier and the target coherent patch identifier and can be directly used for subsequent concatenation processing. The purpose of concatenating the unique association pair set according to the image frame order and connecting the concatenation results according to the source coherent patch identifier to obtain the patch tracking chain is to organize the discrete unique association pairs into a chain record, so that the patch tracking chain forms a searchable continuous path on continuous image frames and provides a stable index for the extraction of state switching segments. Compared with the method of maintaining the continuity of patch identifiers only by time smoothing, this process constrains the unique association pair set by the outer contour coverage relationship and the transport direction determination, so as to maintain the continuity of the patch tracking chain and reduce the occurrence of instability in the endpoint determination when fogging and boundary breakage coexist.

[0060] Example 2 Please see Figure 2 As shown, based on the same inventive concept, this embodiment discloses a machine vision-based real-time detection system for the mixing uniformity of glycoside-containing compound fertilizers. For details not covered in this embodiment, please refer to the relevant sections of Embodiment 1. The system includes: Alignment and filtering module: used to acquire image frames of the spraying section, and perform displacement alignment on the image frames of the spraying section according to the conveying direction to obtain an aligned frame set. Clear fragment set is filtered in the aligned frame set, and the remaining image frames are merged into an occluded fragment set. The baseline compensation module is used to select mixed baseline segments from the set of clear segments, stitch the mixed baseline segments together to obtain a mixed baseline template, construct a fogging layer from the set of occluded segments, and perform occlusion compensation on the set of occluded segments according to the fogging layer to obtain a set of compensated frames. The localization and integration module is used to combine the clear fragment set and the compensation frame set into the sequence to be inspected, perform segregation region localization on the sequence to be inspected based on the hybrid reference template to obtain the segregation patch set, generate the boundary constraint set in the segregation patch set, and perform boundary integration on the segregation patch set according to the boundary constraint set to obtain the coherent patch set. Tracking and determination module: It is used to generate a patch tracking chain based on a coherent patch set, determine the cross-frame patch state according to the patch tracking chain, obtain a tracking state set, determine the mixed uniform detection result based on the tracking state set, and output the detection result set.

[0061] The accompanying drawings of the embodiments of this invention only involve the structures involved in the embodiments of this invention. Other structures can refer to the general design. In the absence of conflict, the features of the same embodiment and different embodiments of this invention can be combined with each other. The above are only specific implementations of this invention, but the protection scope of this invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this invention should be included within the protection scope of this invention. Therefore, the protection scope of this invention should be determined by the protection scope of the claims.

Claims

1. A machine vision-based method for real-time detection of mixing uniformity of glycoside-containing compound fertilizers, characterized in that, include: Acquire image frames of the spraying section, and perform displacement alignment on the image frames of the spraying section according to the conveying direction to obtain an aligned frame set. Filter the clear segment set from the aligned frame set, and merge the remaining image frames into an occluded segment set. Select a hybrid reference segment from the set of clear segments and stitch the hybrid reference segments together to obtain a hybrid reference template. Construct a fogging occlusion layer from the set of occluded segments and perform occlusion compensation on the set of occluded segments based on the fogging occlusion layer to obtain a compensation frame set. The set of clear fragments and the set of compensated frames are combined into a sequence to be inspected. Based on the hybrid reference template, the segregated region is located in the sequence to be inspected to obtain a set of segregated patches. A set of boundary constraints is generated in the set of segregated patches, and boundary integration is performed on the set of segregated patches according to the set of boundary constraints to obtain a set of coherent patches. A patch tracking chain is generated based on a coherent patch set, and the cross-frame patch state is determined according to the patch tracking chain to obtain a tracking state set. The mixed uniform detection result is determined based on the tracking state set, and the detection result set is output.

2. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 1, characterized in that, A fogging occlusion layer is constructed from the set of occluded segments, and occlusion compensation is performed on the set of occluded segments based on the fogging occlusion layer to obtain a set of compensated frames, including: The image frame sequence of each occluded segment is extracted from the occluded segment set, and the pixel position correspondence of the image frame sequence is determined according to the displacement alignment relationship corresponding to the alignment frame set, thus obtaining the occluded alignment frame set; Fog texture elements are extracted from the occlusion-aligned frame set and then grouped according to the image frame number to form a fog texture element set. Extract the fog coverage boundary from the fog texture metadata set, and fuse the fog coverage boundary according to the image frame number to form a fog masking layer; The fogging occlusion layer is mapped to the image frame coordinates of the occlusion alignment frame set. Available texture fragments within the same image frame are extracted outside the fogging occlusion layer coverage area. Regional texture statistical features are calculated based on the available texture fragments. Restoration enhancement parameters for the fogging occlusion layer coverage area are generated based on the regional texture statistical features, forming a compensation parameter set. Based on the compensation parameter set, the image content corresponding to the area covered by the fogging occlusion layer in the occlusion alignment frame set is subjected to dehazing restoration and enhancement processing, and the processed image frames are collected according to the image frame number to obtain the compensation frame set.

3. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 2, characterized in that, The fog coverage boundary is extracted from the fog texture metadata set, and the fog coverage boundary is fused according to the image frame number to form a fog occlusion layer, including: Extract the texture direction features of the atomized texture elements from the atomized texture element set, and filter the atomized texture elements whose texture direction features are consistent with the delivery direction to form a candidate atomized texture element set; The center point coordinates of the candidate fog texture elements are extracted from the candidate fog texture element set, and spatially adjacent center point pairs are retrieved between adjacent image frames to obtain a set of spatially adjacent center point pairs. Based on the set of spatially adjacent center point pairs, construct the spatial adjacency association between candidate fog texture elements, and generate candidate connection paths along the spatial adjacency association to obtain the candidate skeleton path set; Project the fog coverage boundary of the candidate fog texture set onto the candidate skeleton path set, and fuse the projected fog coverage boundary according to the image frame number to obtain the fog occlusion layer.

4. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 3, characterized in that, Based on the set of spatially adjacent center point pairs, spatial adjacency relationships are constructed between candidate fog texture elements, and candidate connection paths are generated along these spatial adjacency relationships to obtain a set of candidate skeleton paths, including: Convert the set of spatially adjacent center point pairs into a set of center point connection edges, and construct a center point association graph using the set of center point connection edges; Retrieve connected centroid chains from the centroid association graph and sort the connected centroid chains by image frame number to form a connection path sequence; In the connected path sequence, determine the path fork point and the path end point, and perform path segmentation on the connected path sequence based on the path fork point and the path end point to form a candidate connected path set; The candidate connection path set is aggregated according to the path connectivity relationship to obtain the candidate skeleton path set.

5. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 1, characterized in that, A set of boundary constraints is generated from the segregated patch set, and boundary integration is performed on the segregated patch set based on the set of boundary constraints to obtain a coherent patch set, including: The outer contours of the segregated patches are extracted one by one from the set of segregated patches, and the boundary transition zones are located along the outer contours of the segregated patches to obtain the set of boundary transition zones; Boundary directional segments are extracted from the boundary transition zone and merged according to the projection relationship of the transport direction to obtain the projection boundary segment set. Based on the projection boundary segment set, candidate boundary segments that conform to the projection relationship are extracted from the segregated patch set, and the boundary constraint set is obtained by aggregating the candidate boundary segments. Based on the boundary constraint set, determine the splicability of the boundaries in the segregated patch set, and merge the segregated patches corresponding to the splicability of the boundaries to form a spliced ​​patch set; Boundary stitching and boundary resampling are performed on the patch set, and the patch set after boundary stitching is aggregated to obtain a coherent patch set.

6. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 5, characterized in that, Based on the projection boundary segment set, candidate boundary segments conforming to the projection relationship are extracted from the segregated patch set, and a boundary constraint set is obtained based on the set of candidate boundary segments, including: Obtain the outer contour of the segregated patches from the segregated patch set, and perform contour segmentation on the outer contour of the segregated patches according to the transport direction to obtain the boundary segmentation set; Extract the direction sequence of the boundary segment from the boundary segment set, and filter the boundary candidate segments according to the adjacent continuation relationship of the direction sequence to obtain the boundary candidate segment set; Extract endpoint pairs of the boundary candidate segments from the boundary candidate segment set, and construct an endpoint association graph based on the spatial adjacency relationship of the endpoint pairs; Retrieve connected endpoint chains in the endpoint association graph and map the connected endpoint chains to the boundary candidate segment set to form a boundary chain set; The boundary chain sets are aggregated according to the segregated patch identifiers, and the aggregated boundary chain sets are then combined to obtain the boundary constraint set.

7. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 1, characterized in that, A patch tracking chain is generated based on a coherent patch set, and the cross-frame patch state is determined according to the patch tracking chain to obtain a tracking state set, including: Extract the outer contour and centroid of the continuous patch set, and organize the outer contour and centroid of the continuous patch according to the image frame order to obtain the frame-order patch table. Enumerate the consecutive patch pairs of adjacent image frames in the frame order patch table, and filter candidate consecutive patch pairs according to the coverage relationship of the outer contour of the consecutive patches to obtain the candidate association pair set. The unique set of association pairs is determined from the candidate association pairs, and the patch tracking chain is obtained by concatenating the unique set of association pairs. Based on the patch tracking chain, state transition segments of coherent patches across frames are extracted, and the state identifiers of patches across frames are determined according to the state transition segments to obtain the tracking state set.

8. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 7, characterized in that, The unique set of association pairs is determined from the candidate association pair set, and the patch tracing chain is obtained by concatenating the unique association pair sets, including: Extract source coherent patch identifiers and target coherent patch identifiers from the candidate association pair set, and merge the candidate association pairs according to the source coherent patch identifiers to obtain the source merge candidate set; Extract the outer contours of source coherent patches and target coherent patches from the source merging candidate set, and filter the coverage-consistent candidate pairs according to the coverage relationship of the outer contours to obtain the coverage-consistent candidate set. Extract the source coherent patch centroid and the target coherent patch centroid from the consistent candidate set, determine the unique target coherent patch identifier based on the transport direction, and obtain a unique set of associated pairs. By concatenating the unique association pairs in the order of image frames and connecting the concatenated results according to the source coherent patch identifier, a patch tracing chain is obtained.

9. The method for real-time detection of mixing uniformity of glycoside-containing compound fertilizer based on machine vision according to claim 8, characterized in that, The source coherent patch centroid and the target coherent patch centroid are extracted from the consistent candidate set. A unique target coherent patch identifier is determined based on the transport direction, resulting in a unique set of associated pairs, including: Extract the centroids of source coherent patches and target coherent patches from the consistent candidate set, and generate a centroid connection set based on the source coherent patch centroids and target coherent patch centroids. Extract the direction vectors of the centroids from the set of centroids and merge the centroids according to the direction vectors to obtain the set of merged centroids. In the directional merging connection, the directional merging connection with the same direction as the conveying direction is selected, and the target continuous patch identifier corresponding to the directional merging connection with the same direction is locked to obtain a unique continuous patch identifier. The candidate association pairs corresponding to the unique coherent patch identifiers are recorded as unique association pairs, and the unique association pairs are aggregated to obtain a unique association pair set.

Images