METHOD AND DEVICE FOR DETECTING DEFECTS IN A FIBRE MATERIAL AND FIBRE LAYING SYSTEM THEREFOR

Description

[0001] The invention relates to a method and a device for detecting defects in fiber material of a fiber composite material deposited on a tool, which contains the fiber material and a matrix material embedding the fiber material. The invention also relates to a fiber laying system for depositing fiber material of a fiber composite material onto a tool for producing a fiber composite component therefor.

[0002] Due to the weight-specific strength and stiffness of fiber composite components made from fiber composite materials, such components have become indispensable in aerospace and many other applications, such as the automotive sector. During the production of a fiber composite component, a matrix material embedding the fiber material is usually cured under heat and pressure. After curing, it forms an integral unit with the fiber material. This forces the reinforcing fibers of the fiber material into their specified direction and allows them to transfer the loads that occur in the specified direction.

[0003] Fiber composite materials from which such fiber composite components are manufactured generally have two main components: a fiber material and a matrix material. In addition, other secondary components may be used, such as binder materials or additional functional elements to be integrated into the component. If dry fiber materials are provided for production, the matrix material of the fiber composite is infused into the fiber material during the manufacturing process using an infusion process, through which the dry fiber material is impregnated with the matrix material. This usually occurs due to a pressure difference between the matrix material and the fiber material, for example, by evacuating the fiber material using a vacuum pump.In contrast, fiber composite materials are also known in which the fiber material is already pre-impregnated with the matrix material (so-called prepregs).

[0004] Before the matrix material cures, the fiber material is usually introduced into a mold, which, with its shaping tool surface, replicates the final component shape. Both dry and pre-impregnated fiber materials can be deposited or introduced into the mold. For the production of large-scale structural components, such as the wing shells of commercial aircraft or rotor blades of wind turbines, automated fiber deposition processes are used to optimize the deposition process. In these processes, a production system and at least one fiber deposition head are used to deposit a quasi-continuous fiber material fed to the fiber deposition head onto the tool. In so-called fiber placement technology, for example, pre-impregnated fiber materials are deposited in webs on the mold using such a fiber deposition head.The fiber placement head is mounted on a robot and can be moved or moved relative to the forming tool. This allows the individual fiber webs to be deposited on the tool surface, first web by web and then layer by layer. With fiber placement technology, several, usually 8, 16, or 32 narrow strips of material, called tows, are deposited on the tool simultaneously. In contrast, with fiber tape laying technology, wide fiber webs, also called tapes (usually 150 mm, 300 mm, or 600 mm wide with a thickness of a few tenths of a millimeter), are deposited on the forming tool using the fiber placement head.

[0005] Such an automated fiber placement system is known, for example, from DE 10 2010 015 027 B4, in which several robots, each with a placement head as the end effector, are guided on a rotating rail system. A fiber feed continuously feeds fiber material from a fiber magazine to the placement heads, while the individual robots, with their placement heads, deposit the supplied fiber material onto a forming tool located in the center of the rotating rail system. With the help of the fiber placement heads, so-called fiber laminates, also called fiber preforms, are produced in the forming tool. The process of infusing the matrix material or curing the matrix material has not yet taken place. Such fiber laminates thus represent the component in its raw state before the matrix material cures.

[0006] Fiber laminates or fiber preforms for large-scale structural components, such as wings or rotor blades, often consist of many individual layers, which can include more than 100 layers depending on the component's application. To ensure high component quality, the number of process errors that occur during the deposition of the fiber material must be minimized or completely prevented. The occurrence of process errors or manufacturing deviations with a significant negative impact on the strength of the component structure must be fully detected during the process, if possible, and ideally prevented.

[0007] In practice, manual inspection of fiber laminates or individual layers has often been established for this purpose. Depending on the component size, staff availability, and defect frequency within an individual layer, an average of approximately 15 to 30 minutes is required to inspect the deposition of a single fiber layer. However, since the deposition of a single fiber layer can usually be completed much more quickly using automated fiber placement systems, manual inspection significantly reduces system efficiency. For example, the production of wing shells for short-haul aircraft requires up to 400 individual layers, resulting in a system downtime of 6,000 to 12,000 minutes per component due to manual inspection.

[0008] Likewise, inspection by a trained employee is highly dependent on their level of experience and, beyond that, does not provide the level of quality required by the manufacturing tolerances. As an example, a permissible gap of 1 mm between two material webs can only be qualitatively examined with the help of tools, and even then, only in a spot check.

[0009] For these reasons, an automated examination of deposited fiber layers to detect deposition defects is desirable in order to improve both qualitatively and quantitatively the detection of surface defects in deposited fiber materials.

[0010] For example, DE 10 2013 104 546 A1 discloses a method for detecting defects in semi-finished fiber products deposited on a tool surface. Using a light projection method, in which the surface is illuminated with light from a light source from a first direction and the light reflected from the surface is recorded with a camera from a different, second direction, a height profile of the fiber material surface is generated based on the image data captured by the camera. Based on this height profile, defects in the fiber material surface can then be detected through image analysis.

[0011] From DE 10 2013 112 260 A1, a method for detecting defects using a light projection method is also known, wherein the fiber material surface is illuminated by means of a lighting unit in addition to the light from the light source of the light projection method and then, in addition to the height profile, the defects are also determined as a function of an intensity distribution of the reflected light from the lighting unit.

[0012] Furthermore, DE 10 2013 104 545 A1 discloses a method for detecting defects using a light projection process. In addition to determining the height profile, the tool surface or the deposited fiber materials are tempered and then recorded with a thermal camera. Based on the thermographic images from the thermal camera and the underlying height profile from the light projection process, defects on the fiber material surface can now be detected.

[0013] Furthermore, the subsequently published DE 10 2018 124 079.1 discloses a method and device for detecting defects in fiber materials. In this method, the fiber surface of pre-impregnated fiber materials is imaged by at least one image sensor of an imaging device in a wavelength spectrum outside the visible spectrum. Defects in the deposited, pre-impregnated fiber materials are then detected based on digital image data generated on this basis.

[0014] From the subsequently published DE 10 2019 112 312.7, a method for the three-dimensional detection of a fiber material surface of a fiber material of a fiber composite material is known. In this method, the fiber material surface is recorded using at least two polarization-sensitive recording sensors, and three-dimensional surface data of the fiber material surface is then created from the resulting measurement data. This then results in, for example, a digital, three-dimensional surface model of the fiber material surface to be detected.

[0015] From the subsequently published DE 10 2019 112 317.8, a method for detecting geometric surface defects on a fiber material surface is known. In this method, the fiber material surface is irradiated with various electromagnetic radiations of different polarizations. Using an evaluation unit, geometric surface defects can then be detected from the polarization-dependent measurement data recorded from the fiber material surface. By varying the polarization of the emitted radiation and the different angles of irradiation, the reflected radiation or reflections from the material surface can be clearly assigned to a radiation source, making it possible to reliably detect geometric inhomogeneities on the material surface and, if necessary, also to characterize the defect.

[0016] EP 3 432 266 A1 discloses a method and device for detecting defects. A height profile is determined using a laser light section sensor, and defects within the fiber material are then detected based on the height profile. For this purpose, a preliminary analysis is first performed, in which individual sub-areas are statistically evaluated. Potential defect areas are then investigated using a detailed pattern recognition analysis.

[0017] EP 3 396 484 A1 discloses a method for manufacturing a fiber composite component in which fiber material is deposited onto a tool in an automated manner. The deposition process can be manually monitored using an optical control system.

[0018] DE 10 2012 220 923 A1 discloses the inspection of fiber directions in carbon fiber materials using polarization images.

[0019] Against this background, it is the object of the present invention to provide an improved method and device for determining defects in fiber material of a fiber composite material that can reliably detect any type of defect.

[0020] The object is achieved according to the invention with the method for detecting defects according to claim 1, with the detection device according to claim 11, and with the fiber laying system according to claim 12. Advantageous embodiments of the invention can be found in the corresponding subclaims.

[0021] According to claim 1, a method is proposed for detecting defects in fiber material of a fiber composite material deposited on a tool, wherein the fiber composite material contains at least one fiber material and a matrix material embedding the fiber material. Furthermore, the fiber composite material can contain further additional materials that are or are added to the fiber composite material as needed. Defects in the fiber material that has been deposited on the tool are to be detected or identified by means of a detection device. For this purpose, the invention provides that through the combined use of three different sensor types, namely a laser light section sensor, a thermographic camera, and a polarization camera, the fiber material surface of the fiber material is detected and the defects are then reliably detected based on the measurement data from all three sensors.

[0022] According to the invention, first image data of the fiber material surface of the deposited fiber material are recorded by means of at least one optical light section sensor, in which the fiber material surface is illuminated with light from a light source from a first direction and the light reflected from the fiber material surface is recorded from a different, second direction with a camera. The light source can preferably be a laser light source that projects a laser light line onto the fiber material surface, wherein the projection angle relative to the fiber material surface is different from 90°. This laser light line projected in this way is then recorded by a camera, wherein, due to the projection angle different from 90° (relative to the fiber material surface), height differences on the fiber material surface become visible.This is because the straight laser light line projected onto the fiber material surface is deformed due to height differences depending on the selected projection angle. This can be detected by the selected camera and recorded in the initial image data. This allows even the smallest height differences on the fiber material surface to be detected.

[0023] According to the invention, second image data of the fiber material surface of the deposited fiber material are further recorded using at least one thermographic camera (also called a thermal imaging camera), wherein the thermographic camera records electromagnetic radiation in the infrared spectrum. This allows the reflection behavior of the fiber material surface to be recorded in the infrared range of the electromagnetic spectrum.

[0024] Finally, according to the invention, third image data of the fiber material surface of the deposited fiber material are recorded using at least one polarization camera, whereby only reflected electromagnetic radiation (preferably in the visible light range) with a specific polarization is recorded. The polarization camera can be provided with different recording channels, with each recording channel being assigned a specific polarization (e.g., 0°, 45°, and 135°). This allows the polarization-dependent reflection behavior of the fiber material surface to be recorded in a targeted manner, particularly in the visible light range (e.g., white light).

[0025] In principle, the invention does not exclude the use of other types of sensors, such as normal cameras that only record the fiber material surface.

[0026] The first, second, and third image data thus acquired are then calibrated to a common reference system using a calibration unit, so that the pixels in the first, second, and third image data correspond to the respective measurement points on the fiber material surface. This means that one and the same reference (pixel coordinates) in the first, second, and third image data refer to one and the same measurement point. This allows each pixel in the image data to be assigned to one another, so that each pixel can be related to every other pixel. A fused database consisting of the first, second, and third image data can thus be used for further analysis, significantly minimizing the limitations of the individual sensors.

[0027] By means of an image evaluation of the first image data, the second image data and the third image data by an image evaluation unit, defects in the deposited fiber material are now detected and, if necessary, classified.

[0028] The inventor recognized that by combining the three sensor types—light section sensor, thermographic camera, and polarization camera—the defects in the fiber material that occur during fiber deposition can be detected with significantly greater process reliability and, moreover, with a classifiable classification than is possible with individual sensors. Surprisingly, it was discovered that the combination of light section sensor, thermographic camera, and polarization camera offers an excellent detection rate, which can also be carried out quickly and efficiently. This is particularly important for large-scale components.

[0029] According to one embodiment, a height profile of the fiber material surface is generated from the first image data, which then serves as the first image data for determining defects. In this case, the deformation of the essentially straight projection of the laser light line is converted into a height profile along the projected laser light line, taking into account the projection angle and, if applicable, the recording angle due to height differences of the fiber material surface.

[0030] According to a further embodiment, the image evaluation unit determines potential defect areas in the image data based on a statistical image analysis of the first, second, and / or third image data. It can be assumed with a certain probability that the potential defect areas each contain at least one defect. For example, so-called "adaptive thresholding" can be used here, which identifies conspicuous image areas by analyzing a histogram. All unmarked areas are then excluded from further analysis. This results in a significant reduction in the amount of data to be analyzed. Accordingly, only those areas in the image data that correspond to the previously identified potential defect areas are used to determine the defects based on the first, second, and third image data.

[0031] According to one embodiment, it is provided that only the first image data of the laser light section sensor are used to determine potential defect areas.

[0032] According to one embodiment, it is provided that for the first image data, the second image data and the third image data, a plurality of image features of one or more image feature types are determined by means of the image evaluation unit, wherein defects in the deposited fiber material are then determined depending on the determined image features of the first image data, the determined image features of the second image data and the determined image features of the third image data.

[0033] Image features can be elements that serve to describe specific forms of representation, such as different forms of histograms. Such image features often serve to identify and, if necessary, distinguish certain patterns and structures within images.

[0034] According to one embodiment, one or more of the following image feature types can be used to determine the image features: Histogram of Oriented Gradients (HOG), Rotation invariant Histogram of Oriented Gradients (RiHOG), one or more statistical moments, Local Binary Patterns (LBP).

[0035] It has been shown that, particularly when combining the above-mentioned types of image features, defects on the fiber material surface or on the fiber material can be reliably detected and, if necessary, classified.

[0036] According to one embodiment, it is provided that a trained machine learning system is provided which, by means of at least one machine-learned decision algorithm, contains a correlation between a plurality of image features as input data and defects in the fiber material as output data, wherein at least one defect in the fiber material is averaged by means of the provided machine learning system by generating at least one defect in the fiber material as output data of the machine learning system from the determined image features as input data of the machine learning system based on the learned decision algorithm.

[0037] It was surprisingly discovered that a machine-trained decision algorithm of a machine learning system can be trained in such a way that, using the image features determined from the three image data sets of the various sensor systems as input data, corresponding defects can be generated as output data. It was also surprisingly discovered that this not only allows defects to be detected, but also allows defects to be classified accordingly, thus determining the type of defect or defect based solely on the image features.

[0038] According to one embodiment, a support vector machine (SVM) can be used as the machine-trained decision algorithm of the machine learning system. This support vector machine (SVM) is trained based on the input data to classify the input data and thus detect and simultaneously classify the corresponding defect. The advantage of this is that it can be easily analyzed, making certification for, for example, production in the aviation industry significantly easier to implement than with other machine learning methods.

[0039] It is of course also conceivable that an artificial neural network is used as the machine-trained decision algorithm of the machine learning system, which contains a correspondingly trained assignment of the image features to defects.

[0040] According to one embodiment, when determining one or more potential defect areas, as already described above, for example, the corresponding image features of the image feature types are determined for each identified potential defect area and then used as input data for the trained decision algorithm. Especially with large-scale component structures, this allows the entire component to be examined very quickly and efficiently, and corresponding defects to be detected and classified.

[0041] According to one embodiment, it is provided that each potential error area is divided into a plurality of individual cells (for example, nine cells), wherein the image features are determined for each cell of a potential error area and the determined image features of all cells are then used as input data for the trained decision algorithm

[0042] The object is also achieved with the detection device for detecting defects in fiber material of a fiber composite material deposited on a tool according to claim 11, wherein the detection device comprises: at least one optical light section sensor, in which the fiber material surface is illuminated with light from a light source from a first direction, and the light reflected from the fiber material surface is recorded with a camera from another, second direction; at least one thermographic camera, at least one polarization camera, as well as a calibration unit and an image evaluation unit.

[0043] The detection device is now designed to carry out the method described above for detecting defects.

[0044] The object is also achieved according to the invention with the fiber laying system for laying fiber material of a fiber composite material according to claim 12, wherein the fiber laying system has a detection device and is designed to carry out the method as described above.

[0045] According to one embodiment, it is provided that the fiber laying system has at least one robot on which a fiber laying head for the continuous laying of fiber material is provided as an end effector, wherein the at least one optical light section sensor, the at least one thermographic camera and the at least one polarization camera of the detection device are arranged on the fiber laying head.

[0046] The invention is explained in more detail by way of example with reference to the accompanying figures. They show: Figure 1: Schematic representation of a fiber laying system with the detection device according to the invention; Figure 2: Schematic representation of a potential error area.

[0047] Figure 1 shows, in a highly simplified schematic representation, a fiber laying system 10 having a fiber laying head 11 arranged on a robot (not shown) as the end effector. The fiber laying head 11 has a compacting roller 12 for depositing the fiber material 13 fed to the fiber laying head 11 onto a forming tool surface 110 of a forming tool 100.

[0048] The fiber laying system 10 further comprises the detection device 20 according to the invention, which has a laser light section sensor 21, a thermographic camera 22, and a polarization camera 23. The laser light section sensor 21, the thermographic camera 22, and the polarization camera 23 are fixedly mounted on the fiber laying head 11 in the wake of the latter and are thus moved relative to the fiber material surface of the already deposited fiber materials 13 during the deposition of the fiber material 13 on the shaping tool surface 110. During the movement of the three sensors 21, 22, and 23 relative to the fiber material surface 24, the fiber material surface 24 is detected and examined accordingly for defects.

[0049] For this purpose, the three sensors 21, 22 and 23 are connected to a data processing unit 24, which receives the first image data from the laser light section sensor 21, the second image data from the thermographic camera 22 and the third image data from the polarization camera 23.

[0050] The three image data sets are first calibrated to a common reference system using a calibration unit 25, so that the pixels in the first, second, and third image data correspond to the respective measurement points on the fiber material surface. Subsequently, the calibrated and thus fused image data from the three sensors 21, 22, and 23 are analyzed using an image evaluation unit 26 to detect defects on the fiber material surface 14 or on the fiber material 13.

[0051] For this purpose, a preliminary analysis is carried out to identify potential error areas, such as those in Figure 2can be detected. For this purpose, the sensor data is first analyzed using a very fast statistical calculation method to determine potential error areas. This can be done, for example, using so-called "adaptive thresholding," which identifies conspicuous image areas by analyzing a histogram. All unmarked areas are excluded from further examination and image analysis.

[0052] Figure 2 shows such a potential defect area 30, which has been divided into exactly nine cells by the image evaluation unit 26. Within the potential defect area 30 is a defect 40, which extends partially into the individual cells of the potential defect area 30.

[0053] For each of these cells, image features of different image feature types are determined to enable a specific abstraction of the input data (image data). The following elements are used as image feature types: Histogram of Oriented Gradients (HOG), Rotation invariant Histogram of Oriented Gradients (RiHOG), one or more statistical moments, Local Binary Patterns (LBP).

[0054] In one embodiment, nine different histograms of oriented gradients (HOG) and nine rotation-invariant histograms of oriented gradients (RiHOG) are determined for each cell. Furthermore, the statistical moments determined for each cell include an expected value, a standard deviation, an inclined plane, a curvature, a center of gravity in the X direction, a center of gravity in the Y direction, an orientation, an eccentricity, a mean, a mean in the X direction, a mean in the Y direction, a variance, a variance in the X direction, a variance in the Y direction, the energy, the inertia, the homogeneity, the covariance, and a correlation. For each of the nine cells, a total of twelve different histograms of local binary patterns (LBP) are calculated, allowing a total of 49 image features to be determined per cell and per image data.

[0055] The complete image features determined for each image dataset and each cell are then fed as input data into a machine learning system, which is trained to generate corresponding output data based on the input data. This output data indicates whether or not a defect is present on the fiber material surface or in the fiber material. Such a machine learning system can be, for example, a support vector machine (SVM).

[0056] The present invention thus makes it possible to reliably and efficiently examine even large-scale structural components with regard to defects in the fiber material or the fiber material surface, thereby minimizing process times and increasing component quality. List of reference symbols

[0057] 10Fiber laying system 11Fiber laying head 12Compacting roll 13Fiber material 14Fiber material surface 20Detection device 21Laser light section sensor 22Thermography camera 23Polarization camera 24Data processing unit 25Calibration unit 26Image evaluation unit 30Potential error areas 40Defects

Claims

1. A method of detecting defects (40) of fiber material (13) of a fiber composite material deposited on a tool and containing the fiber material (13) and a matrix material embedding the fiber material (13) by means of a detection device (20), the method comprising the following steps: - Recording of first image data of the fiber material surface (14) of the deposited fiber material (13) by means of at least one optical light section sensor (21), in which the fiber material surface (14) is illuminated with light from a light source from a first direction and the light reflected by the fiber material surface (14) from another, second direction is recorded with a camera; - Recording second image data of the fiber material surface (14) of the deposited fiber material (13) by means of at least one thermographic camera (22); - Recording third image data of the fiber material surface (14) of the deposited fiber material (13) by means of at least one polarization camera (23); - Calibrating the image data by means of a calibration unit (25) to a common reference system in such a way that the image points in the first, second and third image data correspond to the respective measuring points on the fiber material surface (14); - Determination of defects (40) of the deposited fiber materials (13) by means of an image evaluation unit (26) by means of an image evaluation of the first, second and third image data.

2. Method according to claim 1, characterized in that a height profile of the fiber material surface (14) is generated from the first image data, which is then used as the basis for determining imperfections (40) as the first image data.

3. Method according to claim 1 or 2, characterized in that potential defect areas (30) in the image data are determined by means of the image evaluation unit (26) as a function of a statistical image evaluation of the first, second and / or third image data, in which it can be assumed with a certain probability that they each contain at least one defect (40).

4. Method according to claim 3, characterized in that only the first image data is used to determine potential defect areas (30).

5. Method according to one of the preceding claims, characterized in that for the first image data, the second image data and the third image data, in each case a plurality of image features of one or more image feature types is determined by means of the image evaluation unit (26), wherein defects (40) of the deposited fiber materials (13) are then determined as a function of the determined image features of the first image data, the determined image features of the second image data and the determined image features of the third image data.

6. Method according to claim 5, characterized in that a trained machine learning system is provided which contains a correlation between a plurality of image features as input data and defects (40) on the fiber material (13) as output data by means of at least one machine-learned decision algorithm, wherein at least one defect (40) on the fiber material (13) is determined by means of the provided machine learning system by generating at least one defect (40) on the fiber material (13) as output data of the machine learning system from the determined image features as input data of the machine learning system based on the trained decision algorithm.

7. Method according to claim 6, characterized in that a support vector machine (SVM) or an artificial neural network (ANN) is provided as the machine-learned decision algorithm of the machine learning system.

8. The method according to any one of claims 5 to 7, characterized in that image features are determined for one or more of the following image feature types: - Histogram of Oriented Gradients (HOG), - Rotation invariant Histogram of Oriented Gradients (RiHOG), - one or more statistical moments, - Local Binary Patterns (LBP).

9. The method according to any one of claims 5 to 8, characterized in that one or more potential error areas (30) are determined, wherein the image features for each determined potential defect area (30) are determined and used as input data for the learned decision algorithm.

10. The method according to claim 9, characterized in that each potential defect area (30) is divided into a plurality of cells, wherein the image features are determined for each cell of a potential defect area (30) and the determined image features of all cells are then used as input data for the learned decision algorithm.

11. A detection device (20) for detecting defects (40) of fiber material (13) of a fiber composite material deposited on a tool, the fiber composite material comprising the fiber material (13) and a matrix material embedding the fiber material (13), wherein the detection device (20) comprises: - at least one optical light section sensor (21), in which the fiber material surface (14) is illuminated with light from a light source from a first direction and the light reflected by the fiber material surface (14) from another, second direction is recorded with a camera; - at least one thermographic camera (22), and - at least one polarization camera (23), - as well as a calibration unit (25) and an image evaluation unit (26), wherein the recognition device (20) is designed to carry out the method according to one of the preceding claims.

12. Fiber laying system (10) for depositing fiber material (13) of a fiber composite material, which comprises a fiber material (13) and a matrix material embedding the fiber material (13), on a tool for producing a fiber composite component, characterized in that the fiber laying system (10) has a recognition device (20) according to claim 11 and is designed to carry out the method according to one of claims 1 to 10.

13. Fiber laying system (10) according to claim 12, characterized in that the fiber laying system (10) has at least one robot on which a fiber laying head (11) for continuously laying fiber material (13) is provided as an end effector, the at least one optical light section sensor (21), the at least one thermographic camera (22) and the at least one polarization camera (23) of the detection device (20) being arranged on the fiber laying head (11).