Self-supervised motion estimation method for fiber laying thermal imaging
By spatially registering and estimating the motion of thermal imaging frames during fiber layup, the problem of inconsistent thermal imaging caused by equipment and material switching was solved, and stable layup displacement reconstruction was achieved.
Patent Information
- Application Number
- CN202610365161.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-03-24
- Publication Date
- 2026-05-15
AI Technical Summary
During automated fiber placement, equipment platform switching and material system switching cause changes in thermal imaging response distribution, affecting the consistency of characterization between thermal imaging frames and resulting in inconsistent ply displacement reconstruction results.
By acquiring continuous thermal imaging frames and main motion directions during fiber laying, spatial registration is performed to form thermal imaging frame pairs. These pairs are then input into a preset motion estimation network to extract the directional components and region aggregation of the inter-frame displacement field. A velocity statistical representation is constructed, triggering physical constraint solving, correcting the velocity estimates, forming a stable velocity estimate, and accumulating displacement reconstruction results.
Under equipment and material switching conditions, displacement drift is suppressed, ensuring the coherence and consistency of ply displacement reconstruction, thus improving the stability and accuracy of thermal imaging motion estimation.
Smart Images
Figure CN122049007A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of industrial visual inspection technology, and more specifically, to a self-supervised motion estimation method for fiber laying thermal imaging. Background Technology
[0002] In automated fiber placement, thermal imaging motion estimation is typically used to characterize the motion state of the placement head relative to the workpiece surface and to provide basic information for layup displacement reconstruction and subsequent quality monitoring. Related systems are generally deployed in online manufacturing carriers and need to meet constraints such as continuous acquisition, real-time processing, and stable output. Existing technologies typically employ methods such as thermal image sequence preprocessing, correlation measurement, local feature determination, time smoothing, and result screening to extract motion information from continuous frames and form displacement estimation results. Such methods are usually applicable under the premise that the thermal image response relationship is relatively stable, the imaging feature changes are gradual, and the boundaries of the equipment and working conditions are basically consistent.
[0003] In automated fiber placement scenarios, equipment platform switching can cause changes in thermal imaging response distribution, and material system switching can cause changes in thermal radiation characteristics. These unstable factors can alter the consistency of characterization between consecutive thermal image frames, causing cumulative shifts in motion estimation results based on existing thermal image correlation and time smoothing, and further affecting the consistency of ply displacement reconstruction results. Therefore, the technical problem to be solved is how to ensure the consistency of ply displacement reconstruction while maintaining the continuity of thermal imaging velocity estimation and suppressing cross-platform displacement drift.
[0004] In view of this, the present invention proposes a self-supervised motion estimation method for fiber laying thermal imaging to solve the above problems. Summary of the Invention
[0005] To overcome the above-mentioned deficiencies of the prior art, the present invention provides a self-supervised motion estimation method for fiber laying thermal imaging.
[0006] To achieve the above objectives, the present invention provides the following technical solution: Firstly, a self-supervised motion estimation method for fiber layup thermal imaging is provided, including: Acquire continuous thermal imaging frames during the fiber laying process, as well as the main motion direction of the fiber laying area, and perform spatial registration on the continuous thermal imaging frames to form thermal imaging frame pairs; The thermal imaging frames are input into a preset motion estimation network to obtain an inter-frame displacement field that represents the change in the position of corresponding pixels between the previous and next frames. The directional components of the inter-frame displacement field are extracted and the region is aggregated in the fiber laying area to obtain the displacement field representation. The displacement field representation is directionally merged according to the main motion direction to form the original velocity estimate. A velocity statistical representation is constructed based on multiple consecutive original velocity estimates to obtain a velocity statistical table. A domain offset result is formed based on the velocity statistical table. The physical constraint solution is triggered on the domain offset result to form the effective parameters of the target. The original velocity estimate is corrected based on the target effective parameters to obtain a stable velocity estimate. Displacement accumulation is then performed on the stable velocity estimate to form the ply displacement reconstruction result.
[0007] In some embodiments, spatial registration is performed on consecutive thermal imaging frames to form thermal imaging frame pairs, including: Read adjacent previous frame images and adjacent subsequent frame images from consecutive thermal imaging frames, and extract the boundary maps of the previous frame and the subsequent frame respectively; Establish a registration constraint map based on the boundary map of the previous frame and the boundary map of the next frame, and determine the mapping region map in the adjacent next frame map; Perform coordinate mapping on the mapping region map according to the registration constraint map, and write the mapping result into the adjacent subsequent frame map to obtain the registered frame map; The adjacent previous frame image and the registered frame image are paired according to the acquisition time sequence to obtain thermal imaging frame pairs.
[0008] In some embodiments, establishing a registration constraint map based on the previous frame boundary map and the subsequent frame boundary map, and determining a mapping region map in adjacent subsequent frame maps, includes: Perform boundary sampling on the previous frame boundary map and write the sampling results into a set of contour points in the previous frame; Perform corresponding sampling on the boundary map of the next frame, and write the sampling results into a set of contour points for the next frame; A point-pair association diagram is established based on the contour point set of the previous frame and the contour point set of the next frame, and a registration constraint diagram is generated based on the point-pair association diagram. Based on the registration constraint map, the contour-enclosed region is delineated in the adjacent subsequent frame map, and the contour-enclosed region is determined as the mapping region map.
[0009] In some embodiments, the directional components of the inter-frame displacement field within the fiber laying area are extracted and the region is aggregated to obtain a displacement field characterization, including: Extract the laying area map from the inter-frame displacement field, and establish a projection direction map in the laying area map according to the main motion direction; Based on the projection orientation map, the orientation component map is extracted from the tiling area map to obtain the component buffer map; The component buffer map is divided into regions based on the spatial distribution of the fiber laying area, and an aggregation region map is generated. Based on the aggregated region map, perform region aggregation on the directional component map to obtain the displacement field characterization.
[0010] In some embodiments, extracting the directional component map from the laying area map based on the projection orientation map to obtain the component buffer map includes: Based on the projection orientation map, a sampling trajectory map is generated in the tiling area map, and the sampling position map is marked in the sampling trajectory map; Read the displacement components in the laying area map based on the sampling location map, and write the reading results into a candidate component map; The candidate component map is subjected to orientation screening based on the main motion direction to obtain the component cache map.
[0011] In some embodiments, orientation screening is performed on the candidate component map according to the main motion direction to obtain a component cache map, including: Generate an orientation reference map based on the main motion direction, and write the orientation reference map to the index position of the candidate component map; Based on the orientation reference map, retrieve the component orientations in the candidate component maps and write the retrieval results into an orientation label map; The reverse component is removed based on the orientation marker map, and the retained result is used as a component cache map.
[0012] In some embodiments, the domain offset result triggers physical constraint solving to form the target effective parameters, including: Based on the domain offset results, retrieve the original velocity estimates for the corresponding time period and write the retrieved results into a velocity sequence table; Generate the direction constraint diagram, continuous constraint diagram, and range constraint diagram according to the velocity sequence table, and write the direction constraint diagram, continuous constraint diagram, and range constraint diagram into the constraint combination diagram; Based on the constraint combination diagram, the preset correction parameters are constrained and solved to obtain the parameter candidate diagram; The original velocity estimates are constrained and validated based on the candidate parameter maps, and the candidate parameter maps that pass the validation are determined as the target valid parameters.
[0013] In some embodiments, displacement accumulation is performed on the steady-state velocity estimate to form a ply displacement reconstruction result, including: Read the continuous and stable velocity estimates according to the processing time sequence, and generate time interval values based on adjacent processing times; The displacement increment is calculated based on the steady velocity estimate and the time interval, and the calculation results are written into an increment sequence list. Based on the incremental sequence list, perform displacement accumulation according to the processing time sequence, and write the accumulation results into a cumulative displacement table; The ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result.
[0014] In some embodiments, generating a ply displacement trajectory based on a cumulative displacement table and determining the ply displacement trajectory as the ply displacement reconstruction result includes: The displacement node values are read according to the processing time sequence based on the cumulative displacement table, and the displacement node values are written into a node sequence table. Establish a node connection table based on the node sequence table, and generate trajectory connection segments according to the adjacent processing time sequence; Synthesize the ply displacement trajectory based on the trajectory connection segment, and write the synthesis result as a trajectory result diagram; By associating the trajectory result map with the corresponding processing time sequence, the ply displacement reconstruction result is obtained.
[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention acquires continuous thermal imaging frames and the main motion direction of the fiber laying area during the fiber laying process, and performs spatial registration on the continuous thermal imaging frames to form thermal imaging frame pairs, so that the frames before and after each other are established on a unified spatial basis. The thermal imaging frame pairs are input into a preset motion estimation network to obtain the inter-frame displacement field, and the directional components are extracted in the fiber laying area and the region is aggregated to form a displacement field characterization. Then, the direction is merged according to the main motion direction to obtain the original velocity estimate, so that the thermal imaging velocity estimate remains consistent in the main motion direction. Based on multiple consecutive original velocity estimates, a velocity statistical characterization is constructed to obtain a velocity statistical table, and the velocity statistical table forms a domain offset result, which further triggers the physical constraint solution to form the target effective parameters. Then, the original velocity estimate is corrected according to the target effective parameters to obtain a stable velocity estimate, and the stable velocity estimate is subjected to displacement accumulation to form the layup displacement reconstruction result. Thus, displacement drift is suppressed under the conditions of equipment platform switching and material system switching, and the consistency of layup displacement reconstruction is guaranteed. Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating a self-supervised motion estimation method for fiber layup thermal imaging according to the present invention. Figure 2 This is a schematic diagram of a self-supervised motion estimation system for fiber laying thermal imaging according to the present invention. Detailed Implementation
[0017] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. In the following detailed description, many specific details are set forth to provide a thorough understanding of the exemplary embodiments described. However, it will be apparent to those skilled in the art that the described embodiments may be practiced without some or all of these specific details. In other exemplary embodiments, well-known structures have not been described in detail to avoid unnecessarily obscuring the concepts of this disclosure. It should be understood that the specific embodiments described herein are merely illustrative of the present invention and are not intended to limit the present invention. Furthermore, the various aspects described in the embodiments may be combined arbitrarily without conflict.
[0018] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.
[0019] Example 1 Figure 1 This disclosure illustrates a self-supervised motion estimation method for fiber layup thermal imaging, provided in at least one embodiment, comprising: S10: Acquire continuous thermal imaging frames during the fiber laying process, as well as the main motion direction of the fiber laying area, and perform spatial registration on the continuous thermal imaging frames to form thermal imaging frame pairs. Spatial registration is performed on consecutive thermal imaging frames to form thermal imaging frame pairs, including: Read adjacent previous frame images and adjacent subsequent frame images from consecutive thermal imaging frames, and extract the boundary maps of the previous frame and the subsequent frame respectively; Establish a registration constraint map based on the boundary map of the previous frame and the boundary map of the next frame, and determine the mapping region map in the adjacent next frame map; Perform coordinate mapping on the mapping region map according to the registration constraint map, and write the mapping result into the adjacent subsequent frame map to obtain the registered frame map; The adjacent previous frame image and the registered frame image are paired according to the acquisition time sequence to obtain thermal imaging frame pairs.
[0020] In this embodiment, spatial registration refers to using the boundary correspondence between the fiber laying area and the adjacent previous and subsequent frames as constraints to perform position mapping on the local area corresponding to the fiber laying area in the adjacent subsequent frame, so that the adjacent previous and subsequent frames satisfy a unified coordinate reference within the fiber laying area, rather than performing a unified translation process on the entire thermal imaging frame. This is because during automated fiber laying, the effective motion information in the thermal imaging frame is mainly concentrated in the fiber laying area. If the entire thermal image is directly used as the registration object, background hot spots, edge reflections, or non-laying areas of the mold are easily introduced into the subsequent inter-frame displacement field calculation, which will cause the subsequent original velocity estimate to be offset.
[0021] It should be noted that the previous frame boundary map and the next frame boundary map can be understood as data representing the outer edge position of the fiber laying area in the adjacent previous frame map and the adjacent next frame map, respectively. The registration constraint map can be understood as position constraint data generated by the boundary correspondence between the two. The mapping area map can be understood as the area to be adjusted in the adjacent next frame map constrained by the position constraint data. For example, the fiber laying area boundary in the adjacent previous frame map can be represented as 120,80, 420,82, 418,160, 118,158, and the corresponding boundary in the adjacent next frame map can be represented as 124,81, 424,83, 422,161, 122,159. Then, the registration constraint map represents the corresponding positional relationship between the two sets of boundary points, and the mapping area map represents the local area in the adjacent next frame map that needs to be aligned with the coordinate reference of the previous frame.
[0022] Furthermore, adjacent previous frames are paired with registered frames according to the acquisition sequence to obtain thermal imaging frame pairs. The purpose is to provide consistent input data for subsequent inter-frame displacement field calculations. Based on this, the pixel position changes between adjacent frames reflect more the thermal motion of the fiber laying area itself and less the pseudo-displacement caused by changes in the imaging viewpoint. Therefore, compared with the method of directly estimating motion using unregistered thermal imaging frames, this embodiment, through local spatial registration under boundary constraints, is beneficial to maintaining the consistency of thermal imaging velocity estimation and provides a more stable data foundation for subsequently suppressing cross-platform displacement drift and ensuring the consistency of ply displacement reconstruction.
[0023] A registration constraint map is established based on the boundary map of the previous frame and the boundary map of the next frame, and the mapping region map is determined in the adjacent next frame map, including: Perform boundary sampling on the previous frame boundary map and write the sampling results into a set of contour points in the previous frame; Perform corresponding sampling on the boundary map of the next frame, and write the sampling results into a set of contour points for the next frame; A point-pair association diagram is established based on the contour point set of the previous frame and the contour point set of the next frame, and a registration constraint diagram is generated based on the point-pair association diagram. Based on the registration constraint map, the contour-enclosed region is delineated in the adjacent subsequent frame map, and the contour-enclosed region is determined as the mapping region map.
[0024] It should be noted that the contour point set of the previous frame and the contour point set of the next frame are not simply saved as boundary maps of the previous frame and the next frame. Instead, the continuous boundary is discretized into corresponding and associative position point data. This is because when generating the registration constraint map later, directly processing the continuous boundary is easily affected by local hot spot diffusion and edge blurring. The contour point set can transform the boundary correspondence into a more stable position constraint relationship, which makes it easier to determine the constrained local mapping range in the adjacent next frame map.
[0025] Understandably, in the aforementioned example, the fiber laying area boundaries in adjacent previous frame images are represented as 120,80, 420,82, 418,160, 118,158, and the corresponding boundaries in adjacent subsequent frame images are represented as 124,81, 424,83, 422,161, 122,159. Based on this, the contour point set {120,80, 220,81, 320,81, 420,82} can be obtained along the boundary map of the previous frame, and the contour point set {124,81, 224,82, 324,82, 424,83} can be obtained along the boundary map of the subsequent frame. These two sets of contour point sets represent the boundary sampling positions of the same fiber laying area at two acquisition times, which facilitates those skilled in the art to understand the objects for establishing the subsequent point-pair association map.
[0026] Furthermore, a point-to-point association map is established based on the contour point sets of the previous frame and the contour point sets of the subsequent frame. Essentially, this establishes a positional correspondence between the sampling points in the contour point set of the previous frame and the corresponding sampling points in the contour point set of the subsequent frame. For example, a point-to-point association map such as {120,80→124,81,220,81→224,82,320,81→324,82,420,82→424,83} can be formed. This point-to-point association map represents the offset relationship between the boundaries of the same fiber laying area in adjacent previous frame images and adjacent subsequent frame images. Based on this, the registration constraint map no longer depends on the thermal image distribution of the entire frame, but is constrained by the position of the fiber laying area boundary, thus making it more suitable for local thermal image alignment in automated fiber laying scenarios.
[0027] For example, after the point-to-point association map is generated, the contour enclosing area can be delineated in the adjacent subsequent frame map according to the positional enclosing relationship of each corresponding point. For example, the delineated area can be enclosed by 124,81, 424,83, 422,161, and 122,159. This contour enclosing area is the mapping area map, which represents the local thermal image area in the adjacent subsequent frame map that needs to be mapped. The advantage of this approach is that the mapping area map only covers the thermal image content related to the fiber laying area, which can reduce the thermal response of the mold background and non-laying area from entering the registration process. Compared with the method of directly establishing the registration relationship for the entire frame of the adjacent subsequent frame map, it is more conducive to maintaining the local positional correspondence of subsequent thermal imaging frames.
[0028] S20: Input the thermal imaging frame into the preset motion estimation network to obtain the inter-frame displacement field that represents the change of the corresponding pixel position between the previous frame and the next frame. Extract the directional components and aggregate the regions of the inter-frame displacement field in the fiber laying area to obtain the displacement field representation. Perform directional merging on the displacement field representation according to the main motion direction to form the original velocity estimate. In this embodiment, the preset motion estimation network can be understood as a processing unit that performs corresponding analysis on the two frames before and after the thermal imaging frame alignment and outputs the inter-frame displacement field. The inter-frame displacement field represents the displacement change of each pixel position between the previous frame and the next frame, rather than directly giving the fiber laying speed value. Therefore, after obtaining the inter-frame displacement field, it is necessary to combine the fiber laying area and the main motion direction to continue to extract the direction component, aggregate the region, and merge the direction. This is because the effective motion in the automated fiber laying scenario mainly unfolds along the main motion direction of the fiber laying area. If the entire displacement field is used directly as the speed result, it is easy to introduce local heat diffusion, edge disturbance, and displacement response of non-laying areas, thereby affecting the stability of the original speed estimate.
[0029] It is understood that, in the aforementioned example, thermal imaging frame pairs have already been obtained, and the boundaries of the fiber laying area in adjacent previous frames are represented as 120,80, 420,82, 418,160, 118,158, and the corresponding boundaries in adjacent subsequent frames are represented as 124,81, 424,83, 422,161, 122,159. Based on this, the inter-frame displacement field output by the preset motion estimation network can be written as a set of directional displacements within the fiber laying area as {4,1, 4,1, 4,1, 4,1}, where each set of data represents the displacement change of the corresponding sampling position in the horizontal and vertical directions, respectively. This set of directional displacements is used to characterize the pixel movement of the same fiber laying area in the thermal imaging frame pair, making it easier for those skilled in the art to understand that the inter-frame displacement field is not an abstract concept, but rather position change data that corresponds one-to-one with the aforementioned thermal imaging frame pairs and fiber laying areas.
[0030] Furthermore, extracting the directional components of the inter-frame displacement field within the fiber laying area refers to extracting the displacement components consistent with the fiber laying motion from the directional displacement set based on the main motion direction. For example, if the main motion direction extends along the length of the fiber laying area, then {4,4,4,4} can be extracted from {4,1,4,1,4,1,4,1} as the directional component set. This directional component set is then aggregated according to the spatial distribution of the fiber laying area to obtain the displacement field representation 4,4,4,4. This displacement field representation reflects the overall displacement response of multiple positions within the fiber laying area in the main motion direction. Compared to directly using the original inter-frame displacement field, this processing method can better preserve information directly related to the fiber laying motion.
[0031] The directional components of the inter-frame displacement field are extracted and the region is aggregated within the fiber placement area to obtain a displacement field characterization, including: Extract the laying area map from the inter-frame displacement field, and establish a projection direction map in the laying area map according to the main motion direction; Based on the projection orientation map, the orientation component map is extracted from the tiling area map to obtain the component buffer map; The component buffer map is divided into regions based on the spatial distribution of the fiber laying area, and an aggregation region map is generated. Based on the aggregated region map, perform region aggregation on the directional component map to obtain the displacement field characterization.
[0032] In this embodiment, the purpose of extracting the laying area map is to extract the local displacement data directly related to the fiber laying motion in the inter-frame displacement field separately, and to establish a projection direction map consistent with the main motion direction within the local displacement data. This is because although the aforementioned thermal imaging frame pairs have been spatially registered, the inter-frame displacement field still contains thermal response changes outside the fiber laying area. If the laying area map is not defined first, the subsequent extraction of direction components is prone to mixing in the displacement information corresponding to the mold edge and background hot spots, thereby weakening the indicative role of the displacement field characterization on the fiber laying motion itself.
[0033] It is understandable that the boundaries corresponding to the fiber laying area in the aforementioned example are still 120,80, 420,82, 418,160, 118,158. The inter-frame displacement field output by the preset motion estimation network in this area can be written as {4,1, 4,1, 4,1, 4,1}. The laying area map obtained by cropping on this basis still corresponds to this set of local displacement data. If the main motion direction extends along the length direction of the fiber laying area, the established projection pattern represents that the displacement components are read only along the length direction. This projection pattern is used to limit the reading direction of the direction component extraction, so that those skilled in the art can understand that the subsequent extraction is not an arbitrary direction component, but a displacement response consistent with the main motion of fiber laying.
[0034] Furthermore, based on the projection orientation map, directional component maps are extracted from the layup area map to obtain a component cache map. Essentially, this transforms the bidirectional displacement data in the layup area map into single-direction displacement data. Based on the aforementioned {4,1,4,1,4,1,4,1}, directional component maps such as {4,4,4,4} can be extracted and written into the component cache map. Subsequently, the component cache map is divided into regions according to the spatial distribution of the fiber layup area. For example, it can be divided into two local regions, {4,4} and {4,4}, according to the front and rear sections of the layup area. The significance of this local region division is that the region aggregation is not a one-time merging of all directional components, but rather the spatial distribution relationship is preserved before aggregation.
[0035] It should be noted that performing region aggregation on the directional component map based on the aggregation region map to obtain the displacement field representation means compressing multiple local directional components into a representation result that can participate in subsequent directional merging while maintaining the spatial distribution relationship of the fiber laying area. For example, the aforementioned two local regions can still form a displacement field representation like 4,4 after aggregation. This displacement field representation not only retains the main directional motion response at different positions within the fiber laying area, but also avoids the redundant displacement information brought by directly using the original inter-frame displacement field. Therefore, compared with the existing technology that directly generates motion results based on the entire displacement field, it is more conducive to improving the comparability of subsequent original velocity estimates.
[0036] Based on the projection pattern, the directional component map is extracted from the laying area map to obtain the component buffer map, including: Based on the projection orientation map, a sampling trajectory map is generated in the tiling area map, and the sampling position map is marked in the sampling trajectory map; Read the displacement components in the laying area map based on the sampling location map, and write the reading results into a candidate component map; The candidate component map is subjected to orientation screening based on the main motion direction to obtain the component cache map.
[0037] In this embodiment, the directional component map is extracted from the laying area map based on the projection orientation map to obtain the component cache map. This mainly involves further converging the displacement information in the laying area map to the directional components consistent with the main motion direction, based on the already defined fiber laying area and main motion direction. The sampling trajectory map can be understood as the reading path laid out along the projection orientation map inside the laying area map, the sampling position map can be understood as the displacement reading position determined along each reading path, the candidate component map can be understood as the set of main direction displacements read from the laying area map based on the sampling position map, and the component cache map can be understood as the directional component results retained after orientation screening. The focus of this process is not ordinary sampling, but rather limiting the subsequent reading action to the main direction of fiber laying motion through the projection orientation map.
[0038] It is understandable that the purpose of generating the sampling trajectory map and calibrating the sampling position map is to ensure that the extraction of directional components has a stable spatial correspondence. This is because the displacement data in the laying area map itself still has two-dimensional distribution characteristics. If the displacement components are read directly from all positions, it is easy to mix local thermal disturbances and non-uniform changes at the edges into the subsequent directional component map. The sampling trajectory map limits the reading range to the direction indicated by the projection direction map, and the sampling position map limits the reading action to the discrete positions on these trajectories. Therefore, the data source in the subsequent candidate component map is more concentrated. Based on this, those skilled in the art can understand that this processing is not a complete traversal of the laying area map, but a directional reading for the main motion direction.
[0039] Furthermore, after reading the displacement components from the layup area map based on the sampling location map and writing them into a candidate component map, orientation screening is performed on the candidate component map according to the main motion direction. The reason for this is that although some displacement components in the layup area map fall on the sampling location, their orientation may not be consistent with the main motion direction. These components usually correspond to local heat diffusion, boundary disturbance, or non-main direction response inside the layup area. If orientation screening is not performed, the component cache map will be mixed with directional information unrelated to fiber layup movement. Therefore, in this embodiment, orientation screening is used to retain only the candidate components consistent with the main motion direction, so that the component cache map is more suitable as input data for subsequent region aggregation.
[0040] The candidate component map is subjected to orientation screening based on the main motion direction to obtain a component buffer map, including: Generate an orientation reference map based on the main motion direction, and write the orientation reference map to the index position of the candidate component map; Based on the orientation reference map, retrieve the component orientations in the candidate component maps and write the retrieval results into an orientation label map; The reverse component is removed based on the orientation marker map, and the retained result is used as a component cache map.
[0041] In this embodiment, orientation screening is performed on the candidate component map according to the main motion direction to obtain the component cache map. This is mainly to retain the directional components in the candidate component map that are consistent with the main motion of fiber laying, and to exclude the directional components that are opposite to or deviate from the main motion direction. The orientation reference map can be understood as the orientation reference data generated based on the main motion direction, the orientation mark map can be understood as the mark result obtained after searching the orientation of each directional component in the candidate component map, and the component cache map is the directional component data left after screening based on the orientation mark map. The purpose of this processing is to make the displacement information entering the subsequent region aggregation more concentrated in the main direction motion of fiber laying itself.
[0042] Understandably, in the aforementioned example, the candidate component map can be formed from the displacement readings of the main direction in the layup area map, for example, it can be represented as 4, 4, -1, 4, where the first three and the fourth positions still correspond to the continuous sampling positions in the aforementioned sampling position map. Based on this, if the main motion direction points to the fiber layup forward direction, the generated orientation reference map represents the orientation reference that should be retained at each sampling position. Subsequently, the orientation of the components in the candidate component map can be retrieved based on the orientation reference map to obtain the orientation marker. Figure 1 The values 1, 0, 1 indicate that the direction is consistent with the main motion direction, and the value 0 indicates that the direction is opposite to the main motion direction. This orientation marker is used to visually represent whether each candidate component satisfies the main direction retention condition.
[0043] Furthermore, by eliminating reverse components based on the orientation marker map and using the retained results as a component cache map, the essence is to remove directional components that are inconsistent with the main fiber placement motion from the candidate component map. In the example above, candidate component maps 4, 4, -1, 4 are processed by the orientation marker. Figure 1 After 1, 0, 1 sieving, component buffer map 4, 4, 4 can be obtained. This component buffer map represents the effective directional components that can participate in subsequent region aggregation under the current laying area and the current main motion direction. Compared with the method of directly using the candidate component map without orientation sieving, this processing can reduce the interference of reverse thermal disturbance on the displacement field characterization, thus helping to maintain the comparability of the subsequent original velocity estimates.
[0044] S30: Construct a velocity statistical representation based on multiple consecutive original velocity estimates to obtain a velocity statistical table. Generate a domain offset result based on the velocity statistical table. Trigger physical constraint solution on the domain offset result to form the target effective parameters. Velocity statistical characterization can be understood as the statistical description result formed after time-series aggregation of multiple consecutive raw velocity estimates. Velocity statistics table can be understood as the structured data carrying the statistical description result. Domain offset result can be understood as the result data that determines whether the current thermal imaging motion estimation state deviates from the existing statistical distribution based on the velocity statistics table. Target effective parameters are the parameter results retained after physical constraint solving triggered by the domain offset result and constraint verification. The focus of this process is not simply to statistically analyze the raw velocity estimates, but to use multiple consecutive raw velocity estimates to reflect whether the current thermal imaging motion estimation has shown a distribution offset under cross-device, cross-material, or continuous operation conditions.
[0045] It is understandable that the aforementioned steps have already obtained the original velocity estimates corresponding to the current thermal imaging frame pair. For example, during continuous processing, a sequence of original velocity estimates such as 4, 4, 4, 3, 2 can be obtained sequentially. Based on this, a velocity statistical representation is constructed, and a velocity statistics table is obtained. The purpose is to transform the original velocity estimates from single-moment results into statistical results that can reflect the state changes within a processing time sequence. This is because displacement drift in automated fiber laying scenarios is usually not determined by a single original velocity estimate, but rather manifests as an overall shift in time sequence of multiple consecutive original velocity estimates. Therefore, a velocity statistics table is more suitable for supporting the formation of subsequent domain shift results.
[0046] Furthermore, the domain offset result generated based on the velocity statistics table is essentially determined by the distribution of multiple consecutive original velocity estimates within the current processing period to determine whether the current thermal imaging motion estimation link has deviated from the existing stable state. Triggering physical constraint solving for the domain offset result means that after determining the existence of the domain offset result, the existing correction relationship is not directly used. Instead, the preset correction parameters are re-solved by combining the direction constraints, continuity constraints, and range constraints corresponding to the original velocity estimates, so that the resulting effective target parameters can match the current thermal imaging motion estimation state.
[0047] The domain offset results trigger physical constraint solving to form the effective parameters of the target, including: Based on the domain offset results, retrieve the original velocity estimates for the corresponding time period and write the retrieved results into a velocity sequence table; Generate the direction constraint diagram, continuous constraint diagram, and range constraint diagram according to the velocity sequence table, and write the direction constraint diagram, continuous constraint diagram, and range constraint diagram into the constraint combination diagram; Based on the constraint combination diagram, the preset correction parameters are constrained and solved to obtain the parameter candidate diagram; The original velocity estimates are constrained and validated based on the candidate parameter maps, and the candidate parameter maps that pass the validation are determined as the target valid parameters.
[0048] In this embodiment, the domain offset result triggers physical constraint solving to form target effective parameters. This mainly involves further transforming the offset state reflected in the aforementioned velocity statistics table into parameter results that can participate in correction. The velocity sequence table can be understood as a set of continuous original velocity estimates extracted from the corresponding time period when the domain offset result occurs. The direction constraint diagram is used to characterize the variation constraint of the original velocity estimate in the main motion direction. The continuity constraint diagram is used to characterize the change connection relationship between adjacent original velocity estimates. The range constraint diagram is used to characterize the acceptable variation range of the original velocity estimate under the current fiber laying condition. The constraint combination diagram is used to uniformly write the above three types of constraints into the same parameter solving basis, so that the target effective parameters formed subsequently are not fixed parameters detached from the scene, but parameter results constrained by the temporal state of the current original velocity estimate.
[0049] It is understandable that the previous steps have already yielded continuous raw velocity estimates 4, 4, 4, 3, 2. Based on this, after the velocity statistics table forms the corresponding domain offset result, the raw velocity estimates for this period can be written as a velocity sequence table 4, 4, 4, 3, 2. Among them, the direction constraint diagram can represent that the sequence as a whole still changes along the main direction of fiber laying motion, the continuity constraint diagram can represent that there is no abrupt relationship between adjacent values that is inconsistent with the current laying process, and the range constraint diagram can represent that the sequence is still within the velocity variation range allowed by the current laying conditions. This velocity sequence table and the three types of constraint diagrams together constitute a constraint combination diagram, which is used to represent that the correction parameters under the current domain offset result cannot be generated independently without deviating from the variation law of the fiber laying motion itself.
[0050] Furthermore, constraint solving is performed on the preset correction parameters based on the constraint combination diagram to obtain the parameter candidate diagram. In essence, this means that, given the current domain offset result has already appeared, the preset correction parameters are re-matched to ensure that they are consistent with the state of the continuous original velocity estimates represented by the velocity sequence table. The constraint verification of the original velocity estimates is performed based on the parameter candidate diagram, and the parameter candidate diagram that passes the verification is determined as the target valid parameters. This is to ensure that the parameter results participating in the subsequent correction of the original velocity estimates not only satisfy the direction constraint diagram, the continuity constraint diagram, and the range constraint diagram, but also maintain a consistent data interpretation relationship with the original velocity estimates in the current time period. Therefore, this process is not a simple calculation of parameters, but a validity screening of the parameter candidate diagram.
[0051] S40: Correct the original velocity estimate based on the target effective parameters to obtain a stable velocity estimate. Accumulate the displacement of the stable velocity estimate to form the ply displacement reconstruction result.
[0052] Displacement accumulation is performed on the steady-state velocity estimates to generate ply displacement reconstruction results, including: Read the continuous and stable velocity estimates according to the processing time sequence, and generate time interval values based on adjacent processing times; The displacement increment is calculated based on the steady velocity estimate and the time interval, and the calculation results are written into an increment sequence list. Based on the incremental sequence list, perform displacement accumulation according to the processing time sequence, and write the accumulation results into a cumulative displacement table; The ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result.
[0053] In this embodiment, the stable velocity estimate can be understood as the velocity result after the original velocity estimate has been corrected under the constraints of the target effective parameters. The displacement increment value can be understood as the displacement change obtained from the stable velocity estimate within a single processing period. The cumulative displacement table can be understood as a table of data that continuously records the cumulative displacement results at each moment according to the processing sequence. The layup displacement trajectory is the continuous displacement path data generated based on the cumulative displacement table. The focus of this processing is to further transform the corrected velocity result into a reconstruction result that can directly characterize the spatial propulsion state of the fiber layup process, rather than remaining at the level of a single moment velocity value.
[0054] It is understood that the aforementioned steps have already yielded continuous raw velocity estimates of 4, 4, 4, 3, 2, and after the domain offset results trigger the physical constraint solution, the target effective parameters are formed. Based on this, after correction by the target effective parameters, continuous stable velocity estimates of 4, 4, 3, 3, 2 can be obtained. If the time interval value corresponding to the processing time sequence is written as 1, 1, 1, 1, 1, then the displacement increment values of 4, 4, 3, 3, 2 can be further obtained and written as the increment sequence table 4, 4, 3, 3, 2. This increment sequence table represents the amount of fiber laying area advancing in the main motion direction between adjacent processing times, which is helpful for those skilled in the art to understand that the subsequent displacement accumulation is not simply a mathematical superposition, but rather a conversion of the stable velocity estimates into displacement advancement results with clear temporal significance.
[0055] Furthermore, displacement accumulation is performed according to the processing time sequence based on the incremental sequence list, and the accumulated results are written into a cumulative displacement table. In essence, the displacement advance within each processing time period is continuously superimposed to form the cumulative displacement state of the fiber laying area throughout the entire processing interval. In the example above, the incremental sequence list 4, 4, 3, 3, 2 can form cumulative displacement tables 4, 8, 11, 14, 16. Subsequently, the ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result. This transforms the discrete cumulative displacement table into a continuous ply displacement path, enabling the thermal imaging motion estimation results to be further used to characterize the spatial position changes during the ply advancement process.
[0056] The ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result, including: The displacement node values are read according to the processing time sequence based on the cumulative displacement table, and the displacement node values are written into a node sequence table. Establish a node connection table based on the node sequence table, and generate trajectory connection segments according to the adjacent processing time sequence; Synthesize the ply displacement trajectory based on the trajectory connection segment, and write the synthesis result as a trajectory result diagram; By associating the trajectory result map with the corresponding processing time sequence, the ply displacement reconstruction result is obtained.
[0057] In this embodiment, a ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result. This mainly involves further transforming the discrete displacement results in the aforementioned cumulative displacement table into trajectory results that are continuously unfolded according to the processing time sequence. Here, the displacement node value can be understood as the displacement record corresponding to each processing time in the cumulative displacement table, the node sequence table can be understood as the set of displacement nodes arranged according to the processing time sequence, the node connection table can be understood as the connection relationship data between adjacent displacement nodes, the trajectory connection segment represents the local trajectory fragment formed sequentially by adjacent displacement nodes, and the trajectory result diagram represents the overall ply displacement trajectory synthesized from all trajectory connection segments. Therefore, the focus of this process is not to recalculate the displacement, but to organize the existing cumulative displacement results into a trajectory structure that can directly represent the ply advancement path.
[0058] Furthermore, the ply displacement trajectory is synthesized based on the trajectory connection segments, and the synthesis result is written as a trajectory result diagram. In essence, this transforms the discrete node connection relationship into a continuous path result. Then, the trajectory result diagram is associated with the corresponding processing time sequence to obtain the ply displacement reconstruction result. This makes the ply displacement trajectory not only have a spatial progression relationship, but also a clear temporal correspondence relationship. The advantage of this processing is that, compared with the method of only outputting the cumulative displacement table, the ply displacement reconstruction result generated in this embodiment is more convenient for subsequent continuous characterization of the positional changes during fiber placement.
[0059] Example 2 Please see Figure 2 As shown, based on the same inventive concept, this embodiment discloses a self-supervised motion estimation system for fiber laying thermal imaging. For details not covered in this embodiment, please refer to the relevant sections of Embodiment 1. The system includes: Registration and pairing module: used to acquire continuous thermal imaging frames during the fiber laying process, as well as the main motion direction of the fiber laying area, and to perform spatial registration on the continuous thermal imaging frames to form thermal imaging frame pairs; The velocity estimation module is used to input thermal imaging frames into a preset motion estimation network to obtain an inter-frame displacement field that represents the change in the position of corresponding pixels between the previous and next frames. The module extracts the directional components and aggregates the regions of the inter-frame displacement field within the fiber laying area to obtain a displacement field representation. The module then performs directional merging of the displacement field representation according to the main motion direction to form the original velocity estimate. Constraint Solving Module: Used to construct a velocity statistical representation based on multiple consecutive original velocity estimates, obtain a velocity statistics table, generate a domain offset result based on the velocity statistics table, trigger physical constraint solving on the domain offset result, and generate effective target parameters; Correction and Reconstruction Module: This module is used to correct the original velocity estimate based on the target effective parameters, obtain a stable velocity estimate, accumulate displacements from the stable velocity estimate, and form a ply displacement reconstruction result.
[0060] The detailed description above, in conjunction with the accompanying drawings, describes examples but does not represent all examples that can be implemented or fall within the scope of the claims. The terms “example” and “exemplary” are used in this specification to mean “serving as an example, instance or illustration” and do not mean “superior to or better than other examples”.
[0061] Throughout this specification, the phrase "an embodiment" or "an embodiment" means that a particular feature, structure, or characteristic described in connection with that embodiment is included in at least one embodiment of the invention. Therefore, the use of these phrases may refer to more than one embodiment. Furthermore, the described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
[0062] It should also be noted that these embodiments may be described as processes depicted as flowcharts, structural diagrams, or block diagrams. Although a flowchart may describe the operations as sequential processes, many of these operations can be performed in parallel or concurrently, and the order of these operations may be rearranged.
Claims
1. A self-supervised motion estimation method for fiber layup thermal imaging, characterized in that, include: Acquire continuous thermal imaging frames during the fiber laying process, as well as the main motion direction of the fiber laying area, and perform spatial registration on the continuous thermal imaging frames to form thermal imaging frame pairs; The thermal imaging frames are input into a preset motion estimation network to obtain an inter-frame displacement field that represents the change in the position of corresponding pixels between the previous and next frames. The directional components of the inter-frame displacement field are extracted and the region is aggregated in the fiber laying area to obtain the displacement field representation. The displacement field representation is directionally merged according to the main motion direction to form the original velocity estimate. A velocity statistical representation is constructed based on multiple consecutive original velocity estimates to obtain a velocity statistical table. A domain offset result is formed based on the velocity statistical table. The physical constraint solution is triggered on the domain offset result to form the effective parameters of the target. The original velocity estimate is corrected based on the target effective parameters to obtain a stable velocity estimate. Displacement accumulation is then performed on the stable velocity estimate to form the ply displacement reconstruction result.
2. The self-supervised motion estimation method for fiber layup thermal imaging according to claim 1, characterized in that, Spatial registration is performed on consecutive thermal imaging frames to form thermal imaging frame pairs, including: Read adjacent previous frame images and adjacent subsequent frame images from consecutive thermal imaging frames, and extract the boundary maps of the previous frame and the subsequent frame respectively; Establish a registration constraint map based on the boundary map of the previous frame and the boundary map of the next frame, and determine the mapping region map in the adjacent next frame map; Perform coordinate mapping on the mapping region map according to the registration constraint map, and write the mapping result into the adjacent subsequent frame map to obtain the registered frame map; The adjacent previous frame image and the registered frame image are paired according to the acquisition time sequence to obtain thermal imaging frame pairs.
3. A self-supervised motion estimation method for fiber placement thermal imaging according to claim 2, characterized in that, A registration constraint map is established based on the boundary map of the previous frame and the boundary map of the next frame, and the mapping region map is determined in the adjacent next frame map, including: Perform boundary sampling on the previous frame boundary map and write the sampling results into a set of contour points in the previous frame; Perform corresponding sampling on the boundary map of the next frame, and write the sampling results into a set of contour points for the next frame; A point-pair association diagram is established based on the contour point set of the previous frame and the contour point set of the next frame, and a registration constraint diagram is generated based on the point-pair association diagram. Based on the registration constraint map, the contour-enclosed region is delineated in the adjacent subsequent frame map, and the contour-enclosed region is determined as the mapping region map.
4. The self-supervised motion estimation method for fiber layup thermal imaging according to claim 1, characterized in that, The directional components of the inter-frame displacement field are extracted and the region is aggregated within the fiber layup area to obtain a displacement field characterization, including: Extract the laying area map from the inter-frame displacement field, and establish a projection direction map in the laying area map according to the main motion direction; Based on the projection orientation map, the orientation component map is extracted from the tiling area map to obtain the component buffer map; The component buffer map is divided into regions based on the spatial distribution of the fiber laying area, and an aggregation region map is generated. Based on the aggregated region map, perform region aggregation on the directional component map to obtain the displacement field characterization.
5. The self-supervised motion estimation method for fiber layup thermal imaging according to claim 4, characterized in that, Based on the projection pattern, the directional component map is extracted from the tiling area map to obtain the component buffer map, including: Based on the projection orientation map, a sampling trajectory map is generated in the tiling area map, and the sampling position map is marked in the sampling trajectory map; Read the displacement components in the laying area map based on the sampling location map, and write the reading results into a candidate component map; The candidate component map is subjected to orientation screening based on the main motion direction to obtain the component cache map.
6. The self-supervised motion estimation method for fiber layup thermal imaging according to claim 5, characterized in that, Based on the main motion direction, orientation screening is performed on the candidate component map to obtain the component buffer map, including: Generate an orientation reference map based on the main motion direction, and write the orientation reference map to the index position of the candidate component map; Based on the orientation reference map, retrieve the component orientations in the candidate component maps and write the retrieval results into an orientation label map; The reverse component is removed based on the orientation marker map, and the retained result is used as a component cache map.
7. The self-supervised motion estimation method for fiber layup thermal imaging according to claim 1, characterized in that, The domain offset results trigger physical constraint solving to form the effective parameters of the target, including: Based on the domain offset results, retrieve the original velocity estimates for the corresponding time period and write the retrieved results into a velocity sequence table; Generate the direction constraint diagram, continuous constraint diagram, and range constraint diagram according to the velocity sequence table, and write the direction constraint diagram, continuous constraint diagram, and range constraint diagram into the constraint combination diagram; Based on the constraint combination diagram, the preset correction parameters are constrained and solved to obtain the parameter candidate diagram; The original velocity estimates are constrained and validated based on the candidate parameter maps, and the candidate parameter maps that pass the validation are determined as the target valid parameters.
8. The self-supervised motion estimation method for fiber layup thermal imaging according to claim 1, characterized in that, Displacement accumulation is performed on the steady-state velocity estimates to generate ply displacement reconstruction results, including: Read the continuous and stable velocity estimates according to the processing time sequence, and generate time interval values based on adjacent processing times; The displacement increment is calculated based on the steady velocity estimate and the time interval, and the calculation results are written into an increment sequence list. Based on the incremental sequence list, perform displacement accumulation according to the processing time sequence, and write the accumulation results into a cumulative displacement table; The ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result.
9. A self-supervised motion estimation method for fiber layup thermal imaging according to claim 8, characterized in that, The ply displacement trajectory is generated based on the cumulative displacement table, and the ply displacement trajectory is determined as the ply displacement reconstruction result, including: The displacement node values are read according to the processing time sequence based on the cumulative displacement table, and the displacement node values are written into a node sequence table. Establish a node connection table based on the node sequence table, and generate trajectory connection segments according to the adjacent processing time sequence; Synthesize the ply displacement trajectory based on the trajectory connection segment, and write the synthesis result as a trajectory result diagram; By associating the trajectory result map with the corresponding processing time sequence, the ply displacement reconstruction result is obtained.