Multi-mode sensing carbon fiber laying defect detection method, equipment and medium

Abstract

The invention discloses a multi-mode sensing carbon fiber laying defect detection method, equipment and a medium, and relates to the technical field of carbon fiber detection. The method comprises the steps that image data of a carbon fiber laying layer are collected through a hyperspectral camera, thickness distribution data of the carbon fiber laying layer are collected through a laser displacement sensor, and the carbon fiber laying layer comprises resin and carbon fibers; preprocessing and fusing the image data and the thickness distribution data to obtain spectral dimension data and spatial dimension data; inputting the spectral dimension data and the spatial dimension data into the trained Transform defect detection model to obtain a defect classification result; and generating a laying head pose adjustment instruction according to the defect classification result. According to the method, the defect detection precision is remarkably improved through multi-modal data fusion and a Transform cross attention mechanism, the defect classification result is directly mapped into the placement head pose adjustment instruction, and defect accumulation is avoided through real-time feedback.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field, and particularly relates to a multi-modal perception carbon fiber laying defect detection method, device and medium. BACKGROUND

[0002] In the current carbon fiber composite material automatic laying process, defect detection mainly relies on periodic manual sampling inspection. Traditional equipment uses a single-point contact sensor or a two-dimensional visual detection system, which has problems such as a large blind area in detection. In particular, when dealing with microscopic defects such as fiber wrinkles and uneven resin distribution, the existing technology cannot realize dynamic closed-loop control of the laying process, resulting in a long-term high defect omission rate.

[0003] The existing technology uses infrared thermal imaging for layer detection, but is limited by the temperature interference of the resin curing process, and the accuracy of identifying fiber arrangement abnormalities in uncured layers is insufficient. Another disclosed scheme proposes a defect classification method based on a convolutional neural network, but since a multi-physical field coupling analysis model is not established, there is a risk of misjudgment for gap defects under complex working conditions, and the calculation is delayed, which cannot meet the real-time control requirements.

[0004] Through the above analysis, the problems and defects of the prior art are:

[0005] The existing technology cannot accurately detect fiber arrangement defects in real time during the carbon fiber automatic laying process, and further cannot meet the real-time control requirements. SUMMARY

[0006] The embodiments of the present application provide a multi-modal perception carbon fiber laying defect detection method, device and medium, which can solve the problem that the existing technology cannot accurately detect fiber arrangement defects in real time during the carbon fiber automatic laying process, and further cannot meet the real-time control requirements.

[0007] In a first aspect, the embodiments of the present application provide a multi-modal perception carbon fiber laying defect detection method, characterized in that the method comprises: collecting image data of a carbon fiber laying layer by a hyperspectral camera, and collecting thickness distribution data of the carbon fiber laying layer by a laser displacement sensor, wherein the carbon fiber laying layer comprises resin and carbon fiber; pre-processing and fusing the image data and the thickness distribution data to obtain spectral dimension data and spatial dimension data; inputting the spectral dimension data and the spatial dimension data into a trained Transformer defect detection model to obtain a defect classification result; and generating a laying head pose adjustment instruction according to the defect classification result.

[0008] In one implementation of the present application, image data and thickness distribution data are preprocessed and fused to obtain spectral dimension data and spatial dimension data, specifically including: performing band fusion processing on the image data to extract spectral features, wherein the image data includes image pixel coordinates; separating the spectral contributions of resin and carbon fiber based on the spectral features, and enhancing the ply direction contrast through polarization-sensitive hyperspectral imaging; performing Gaussian filtering on the thickness distribution data, wherein the thickness distribution data includes thickness coordinates; obtaining a thickness gradient based on the thickness coordinates; establishing a spatial mapping relationship between the image pixel coordinates and the thickness coordinates to obtain a fused data matrix containing spectral contributions, ply direction contrast, and thickness gradient.

[0009] In one implementation of the present application, after establishing the spatial mapping relationship between image pixel coordinates and thickness coordinates, the method also includes: applying external force to the carbon fiber ply, obtaining the fiber orientation based on the fusion data matrix, and constructing a physical constraint equation; based on a contact force compensation algorithm of bionic tactile sense, correcting the errors in the preprocessing and fusion of image data and thickness distribution data.

[0010] In one implementation of the present application, before inputting the spectral dimension data and the spatial dimension data into the trained Transformer defect detection model, the method also includes: constructing a defect sample library of carbon fiber plies; designing a four-layer cross-attention mechanism for the defect sample library to synchronously process the spectral dimension data and the spatial dimension data; and compressing the model through knowledge distillation technology to obtain the trained Transformer defect detection model.

[0011] In one implementation of the present application, spectral dimension data and spatial dimension data are input into the trained Transformer defect detection model to obtain defect classification results, specifically including: linearly projecting the fused data matrix and the spatial mapping relationship respectively to generate an embedding vector with consistent dimensions; extracting multi-scale defect features layer by layer through the Transformer decoder, obtaining the long-range dependency of the defect features through the self-attention layer, using deep separable convolution to enhance local feature extraction in the feedforward neural network, and outputting a defect feature map; based on the defect feature map, generating a global classification feature vector through global average pooling; inputting the global classification feature vector into the fully connected layer, and outputting the existence probability of the carbon fiber ply defect category.

[0012] In one implementation of the present application, after the global classification feature vector is input into the fully connected layer and the existence probability of the carbon fiber ply defect category is output, the method further includes: applying upsampling and threshold segmentation to the defect feature map to generate a pixel-level defect mask; performing confidence screening on the existence probability, locating defects greater than the confidence threshold, and obtaining the defect spatial distribution based on the pixel-level defect mask; performing morphological closing operations on the positioning mask to fill the false detection areas smaller than the area threshold; and generating placement head posture adjustment instructions based on the defect category and spatial distribution.

[0013] In one implementation of the present application, placement head posture adjustment instructions are generated based on the defect category and spatial distribution, specifically including: matching a preset compensation strategy library according to the defect category, the defect category includes wrinkle defects and gap defects; starting the vacuum adsorption device to adjust the fiber tension for wrinkle defects, calculating the required feed amount for gap defects and adjusting the prepreg cutting length; updating the process parameter database and placement path planning.

[0014] In one implementation of the present application, after generating the placement head posture adjustment instruction based on the defect classification result, the method also includes: monitoring the influence coefficient of ambient temperature and humidity on resin viscosity to dynamically compensate the compensation strategy library; and calculating the adaptive adjustment formula for the optimal detection frequency.

[0015] In a second aspect, an embodiment of the present application further provides a device, comprising at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so as to enable the at least one processor to: collect image data of the carbon fiber ply through a hyperspectral camera, and collect thickness distribution data of the carbon fiber ply through a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; preprocess and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; input the spectral dimension data and spatial dimension data into a trained Transformer defect detection model to obtain a defect classification result; and generate a placement head posture adjustment instruction based on the defect classification result.

[0016] In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to: collect image data of a carbon fiber ply through a hyperspectral camera, and collect thickness distribution data of the carbon fiber ply through a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; pre-process and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; input the spectral dimension data and spatial dimension data into a trained Transformer defect detection model to obtain defect classification results; and generate placement head posture adjustment instructions based on the defect classification results.

[0017] The embodiments of the present application provide a multimodal perception carbon fiber placement defect detection method, equipment and medium. Through multimodal data fusion and Transformer cross-attention mechanism, the defect detection accuracy is significantly improved. Hyperspectral data provides material spectral characteristics, laser displacement data reflects the geometric morphology of the ply, and the self-attention mechanism can dynamically capture the cross-modal correlation between spectral characteristics and spatial and thickness characteristics. Transformer performs better in long sequence dependency modeling and is suitable for continuous detection of large-area defects in plies. The defect classification results are directly mapped to the placement head posture adjustment instructions, and the accumulation of defects is avoided through real-time feedback, thereby improving the overall quality of the ply. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0019] Figure 1 A flowchart of a multimodal sensing carbon fiber placement defect detection method provided in an embodiment of the present application;

[0020] Figure 2 A schematic diagram of the internal structure of a multimodal sensing carbon fiber placement defect detection device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] To make the purpose, technical solutions, and advantages of this application more clear, the technical solutions of this application will be clearly and completely described below in conjunction with the specific embodiments of this application and the corresponding drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0022] The embodiments of the present application provide a multimodal sensing carbon fiber placement defect detection method, device and medium, which solves the problem in the prior art that the carbon fiber automatic placement process cannot accurately detect fiber arrangement defects in real time, and further cannot meet the real-time control requirements.

[0023] The technical solutions proposed in the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0024] Figure 1 This is a flow chart of a multi-modal sensing carbon fiber placement defect detection method provided in an embodiment of the present application. Figure 1 As shown, the embodiment of the present application provides a multimodal sensing carbon fiber placement defect detection method, which specifically includes the following steps:

[0025] Step 10: Collect image data of the carbon fiber ply using a hyperspectral camera, and collect thickness distribution data of the carbon fiber ply using a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber.

[0026] In this step, for example, when laying prepreg tape with a width of 300 mm, the hyperspectral camera can be installed at a 30° angle, combined with a ring-shaped LED light source with a color temperature of 5500K to ensure shadow-free imaging. The laser displacement sensor is then linked to the placement head to track the placement position in real time.

[0027] Step 20: Preprocess and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data.

[0028] As an optional embodiment, preprocessing and fusing the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data may specifically include: Step 201: performing band fusion processing on the image data to extract spectral features, wherein the image data includes image pixel coordinates;

[0029] In this step, for example, the resin characteristic band (1420-1480nm) and the fiber characteristic band (920-980nm) are selected for principal component analysis to generate a fusion image, and the associated mapping of spectral dimension data: characteristic vector and spatial dimension data: pixel coordinate is obtained, which provides a basis for subsequent material separation.

[0030] Step 202: Separate the spectral contributions of the resin and carbon fiber based on the spectral characteristics, and enhance the ply direction contrast through polarization-sensitive hyperspectral imaging.

[0031] In this step, the spectral contributions of the resin and carbon fiber are separated based on non-negative matrix decomposition or linear spectral unmixing algorithms. Polarization-sensitive hyperspectral imaging is used to obtain image sequences at different polarization angles, calculate the degree of polarization (DoP) and angle of polarization (AoP), and enhance the contrast of the ply fiber directions, such as the difference between 0° and 90° plies.

[0032] Step 203: Perform Gaussian filtering on the thickness distribution data, which includes thickness coordinates. Step 204: Obtain a thickness gradient based on the thickness coordinates. Step 205: Establish a spatial mapping relationship between the image pixel coordinates and the thickness coordinates to obtain a fused data matrix containing spectral contribution, ply direction contrast, and thickness gradient.

[0033] In this step, the gradient amplitude and direction can be quantified, and areas of thickness mutation, such as delamination and resin accumulation, can be marked. Through spectral spatial feature extraction, material unmixing, thickness analysis and gradient calculation, multi-dimensional defect detection of carbon fiber ply material distribution, fiber direction and thickness uniformity can be achieved.

[0034] As an optional embodiment, after establishing the spatial mapping relationship between image pixel coordinates and thickness coordinates, the method may further include: step 206: applying external force to the carbon fiber ply, obtaining the fiber orientation based on the fused data matrix, and constructing a physical constraint equation; step 207: correcting errors in the preprocessing and fusion of image data and thickness distribution data based on a contact force compensation algorithm based on bionic tactile sensation.

[0035] In this step, controllable external forces, such as stretching and bending, are applied to the carbon fiber ply. Hyperspectral images and thickness distribution data after deformation are collected simultaneously. The spatial mapping relationship between image pixel coordinates and thickness coordinates is combined to establish a correlation between the deformation field and material distribution. This allows for a theoretical relationship between fiber orientation and thickness. For example, the fiber deflection angle Δθ results in a thickness change Δh. Through physical constraint equations, the spectral-thickness data is correlated with the mechanical behavior to achieve a predictive evaluation of the ply performance. The bionic tactile compensation algorithm simulates human touch and uses contact force data to correct optical detection errors. For example, when fiber wrinkles are detected, the tactile sensor detects a sudden increase in local pressure, such as an increase from the standard 10N to 25N, triggering the compensation algorithm to reduce the confidence level of the optical detection by 30%. Mechanical data is prioritized for decision-making to obtain a high-precision fusion data matrix after error correction.

[0036] Step 30: Input the spectral dimension data and spatial dimension data into the trained Transformer defect detection model to obtain the defect classification results.

[0037] As an optional embodiment, before inputting the spectral dimension data and the spatial dimension data into the trained Transformer defect detection model, the method may further include: constructing a defect sample library of carbon fiber plies; designing a four-layer cross-attention mechanism for the defect sample library to synchronously process the spectral dimension data and the spatial dimension data; and compressing the model through knowledge distillation technology to obtain the trained Transformer defect detection model.

[0038] In this step, for example, 100,000 images containing wrinkles, gaps, and contamination defects provide sufficient training data to cover all defect types, allowing the model to simultaneously focus on spectral features, such as resin distribution, and spatial features, such as fiber arrangement. Natural image pre-training feature extraction capabilities can also be leveraged to address the lack of annotations in near-infrared spectral data. The knowledge distillation compression model transfers the capabilities of large models to lightweight models, ensuring real-time operation on edge devices. Cross-attention is used to achieve dual-dimensional spectral and spatial analysis, while transfer learning and distillation are used to address the conflict between small sample sizes and computing power.

[0039] As an optional embodiment, the spectral dimension data and the spatial dimension data are input into the trained Transformer defect detection model to obtain the defect classification result, which may specifically include: step 301: linearly projecting the fused data matrix and the spatial mapping relationship respectively to generate an embedding vector with consistent dimension; step 302: extracting multi-scale defect features layer by layer through the Transformer decoder, obtaining the long-range dependency of the defect features through the self-attention layer, using deep separable convolution to enhance local feature extraction in the feedforward neural network, and outputting a defect feature map; step 303: based on the defect feature map, generating a global classification feature vector through global average pooling; step 304: inputting the global classification feature vector into the fully connected layer, and outputting the existence probability of the carbon fiber ply defect category.

[0040] In this step, for example, the spectral features (256 dimensions) and spatial coordinates (2 dimensions) are projected to 128-dimensional vectors through a fully connected layer respectively; a 4-layer Transformer decoder is used, each layer contains multi-head self-attention, with 8 heads; the depthwise separable convolution kernel is 3×3; layer normalization and residual connection, global average pooling followed by a fully connected layer, that is, the input is 128-dimensional and the output is 3-dimensional, and the Sigmoid activation function is used to output the probability. For example, for a sample containing a gap defect, the model output probability is [0.02, 0.97, 0.01], corresponding to a gap defect confidence level of 97% and a positioning error of <0.2mm.

[0041] As an optional embodiment, after inputting the global classification feature vector into the fully connected layer and outputting the existence probability of the carbon fiber ply defect category, the method may further include: step 305: applying upsampling and threshold segmentation to the defect feature map to generate a pixel-level defect mask; step 306: performing confidence screening on the existence probability, locating defects greater than the confidence threshold, and obtaining the defect spatial distribution based on the pixel-level defect mask; step 307: performing a morphological closing operation on the positioning mask to fill the false detection area smaller than the area threshold.

[0042] In this step, bilinear interpolation is used to restore the defect feature map from 32×32 to 512×512 resolution. The dynamic threshold T is set to 0.7×pmax, where pmax is the maximum class probability. A binary mask is generated and a 5×5 circular kernel is used for closing operation to eliminate areas < 0.1mm. 2 The original mask of a wrinkle defect contains three discrete regions with areas of 0.08 / 0.12 / 0.15mm2. After the closing operation, it is merged into a single region with an area of ​​0.35mm 2 .

[0043] Step 40: Generate placement head posture adjustment instructions based on the defect classification results.

[0044] As an optional embodiment, a placement head posture adjustment instruction is generated according to the defect category and spatial distribution, which may specifically include: Step 401: matching a preset compensation strategy library according to the defect category, the defect category includes wrinkle defects and gap defects; Step 402: starting the vacuum adsorption device to adjust the fiber tension for wrinkle defects, calculating the required feed amount for gap defects and adjusting the prepreg cutting length; Step 403: updating the process parameter database and placement path planning.

[0045] In this step, correction schemes for different defects are preset, and the defect classification results, such as fiber orientation deviation and insufficient resin impregnation, are directly mapped to the placement head posture adjustment instructions, angle correction, and pressure compensation. For example, wrinkles correspond to scheme A, gaps correspond to scheme B, and vacuum adsorption adjusts the tension to physically eliminate wrinkles, directly acting on the placement head actuator; the amount of feed is calculated, and the length of the prepreg tape to be increased is determined through a mathematical model; the path planning is updated, and the subsequent layup path is dynamically adjusted to avoid defect accumulation; closed-loop control of detection, decision-making, and execution is realized, replacing the traditional open-loop placement. Through real-time feedback, detection and adjustment are made after each layer is laid to avoid defect accumulation and improve the overall quality of the layup.

[0046] As an optional embodiment, after generating the placement head posture adjustment instruction based on the defect classification result, the method may also include: monitoring the influence coefficient of ambient temperature and humidity on resin viscosity to dynamically compensate the compensation strategy library; and calculating the adaptive adjustment formula for the optimal detection frequency.

[0047] In this step, the effects of temperature and humidity are monitored, for example, the resin viscosity η = η0·e^(kΔT), and the temperature change ΔT causes the fluidity to change; the detection frequency is adaptive, for example, when the resin flow rate increases, the sampling frequency is increased, such as adjusting it from 50ms to 30ms.

[0048] The above is an embodiment of the method proposed in this application. Based on the same inventive concept, this application embodiment also provides a multi-modal sensing carbon fiber placement defect detection device, the structure of which is as follows: Figure 2 shown.

[0049] Figure 2 This is a schematic diagram of the internal structure of a multi-modal sensing carbon fiber placement defect detection device provided in an embodiment of the present application. Figure 2 As shown, the equipment includes:

[0050] at least one processor 201;

[0051] and, a memory 202 communicatively coupled to the at least one processor;

[0052] The memory 202 stores instructions that can be executed by at least one processor, and the instructions are executed by the at least one processor 201 so that the at least one processor 201 can: collect image data of the carbon fiber ply through a hyperspectral camera, and collect thickness distribution data of the carbon fiber ply through a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; preprocess and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; input the spectral dimension data and spatial dimension data into the trained Transformer defect detection model to obtain defect classification results; and generate placement head posture adjustment instructions based on the defect classification results.

[0053] Some embodiments of the present application provide corresponding Figure 1 A non-volatile computer storage medium for multimodal sensing of carbon fiber placement defects stores computer-executable instructions, wherein the computer-executable instructions are configured to: collect image data of a carbon fiber ply through a hyperspectral camera, and collect thickness distribution data of the carbon fiber ply through a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; preprocess and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; input the spectral dimension data and spatial dimension data into a trained Transformer defect detection model to obtain defect classification results; and generate placement head posture adjustment instructions based on the defect classification results.

[0054] The various embodiments in this application are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from the other embodiments. In particular, the IoT device and media embodiments are generally similar to the method embodiments, so their description is relatively simple. For relevant portions, refer to the description of the method embodiments.

[0055] The system and medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the system and medium also have similar beneficial technical effects to their corresponding methods. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the system and medium will not be repeated here.

[0056] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0057] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0058] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0059] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0060] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0061] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0062] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0063] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0064] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A multimodal sensing carbon fiber placement defect detection method, characterized in that: The method comprises: collecting image data of a carbon fiber ply using a hyperspectral camera and collecting thickness distribution data of the carbon fiber ply using a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; Preprocessing and fusing the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; Inputting the spectral dimension data and the spatial dimension data into the trained Transformer defect detection model to obtain a defect classification result; Generate placement head posture adjustment instructions based on the defect classification results.

2. The multimodal sensing carbon fiber placement defect detection method according to claim 1, characterized in that: Preprocessing and fusing the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data specifically includes: Performing band fusion processing on the image data to extract spectral features, wherein the image data includes image pixel coordinates; Based on the spectral characteristics, the spectral contributions of the resin and carbon fiber are separated, and the ply direction contrast is enhanced by polarization-sensitive hyperspectral imaging; performing Gaussian filtering on the thickness distribution data, the thickness distribution data including thickness coordinates; Based on the thickness coordinates, a thickness gradient is obtained; A spatial mapping relationship between the image pixel coordinates and the thickness coordinates is established to obtain a fused data matrix including the spectral contribution, ply direction contrast and thickness gradient.

3. The multimodal sensing carbon fiber placement defect detection method according to claim 2, characterized in that: After establishing the spatial mapping relationship between the image pixel coordinates and the thickness coordinates, the method further includes: applying an external force to the carbon fiber ply, obtaining a fiber orientation according to the fused data matrix, and constructing a physical constraint equation; A contact force compensation algorithm based on bionic tactile sensation is used to correct errors in preprocessing and fusing the image data and thickness distribution data.

4. The multimodal sensing carbon fiber placement defect detection method according to claim 1, characterized in that: Before inputting the spectral dimension data and the spatial dimension data into the trained Transformer defect detection model, the method further includes: Constructing a defect sample library of the carbon fiber ply; A four-layer cross-attention mechanism is designed for the defect sample library to synchronously process the spectral dimension data and the spatial dimension data; The model is compressed using knowledge distillation technology to obtain the trained Transformer defect detection model.

5. The multimodal sensing carbon fiber placement defect detection method according to claim 2, characterized in that: The spectral dimension data and spatial dimension data are input into the trained Transformer defect detection model to obtain defect classification results, specifically including: Performing linear projection on the fused data matrix and the spatial mapping relationship to generate an embedding vector with consistent dimension; The Transformer decoder extracts multi-scale defect features layer by layer, and the self-attention layer obtains the long-range dependency of defect features. The feedforward neural network uses deep separable convolution to enhance local feature extraction and output a defect feature map. Based on the defect feature map, generating a global classification feature vector through global average pooling; The global classification feature vector is input into the fully connected layer, and the existence probability of the carbon fiber ply defect category is output.

6. The multimodal sensing carbon fiber placement defect detection method according to claim 5, characterized in that: After inputting the global classification feature vector into a fully connected layer and outputting the probability of existence of a carbon fiber ply defect category, the method further includes: Applying upsampling and threshold segmentation to the defect feature map to generate a pixel-level defect mask; Performing confidence screening on the existence probability, locating defects greater than a confidence threshold, and obtaining a defect spatial distribution based on the pixel-level defect mask; A morphological closing operation is performed on the positioning mask to fill the false detection area that is smaller than the area threshold.

7. The multimodal sensing carbon fiber placement defect detection method according to claim 6, characterized in that: Generate placement head posture adjustment instructions based on the defect classification results, specifically including: Matching a preset compensation strategy library according to defect categories, wherein the defect categories include wrinkle defects and gap defects; A vacuum adsorption device is activated for the wrinkle defect to adjust the fiber tension, and a required amount of material is calculated for the gap defect and the prepreg cutting length is adjusted; Update process parameter database and placement path planning.

8. The multimodal sensing carbon fiber placement defect detection method according to claim 7, characterized in that: After generating a placement head posture adjustment instruction according to the defect classification result, the method further includes: Monitoring the influence coefficient of ambient temperature and humidity on resin viscosity to dynamically compensate the compensation strategy library; Adaptive adjustment formula for calculating the optimal detection frequency.

9. A multi-modal sensing carbon fiber placement defect detection device, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to: collecting image data of a carbon fiber ply using a hyperspectral camera and collecting thickness distribution data of the carbon fiber ply using a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; Preprocess and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; Inputting the spectral dimension data and the spatial dimension data into the trained Transformer defect detection model to obtain a defect classification result; Generate placement head posture adjustment instructions based on the defect classification results.

10. A non-volatile computer storage medium for multimodal sensing of carbon fiber placement defects, storing computer-executable instructions, characterized in that: The computer executable instructions are configured to: collecting image data of a carbon fiber ply using a hyperspectral camera and collecting thickness distribution data of the carbon fiber ply using a laser displacement sensor, wherein the carbon fiber ply includes resin and carbon fiber; Preprocess and fuse the image data and thickness distribution data to obtain spectral dimension data and spatial dimension data; Inputting the spectral dimension data and the spatial dimension data into the trained Transformer defect detection model to obtain a defect classification result; Generate placement head posture adjustment instructions based on the defect classification results.

Images