Magnetic Detection System and Method for Defects around Rivet Holes on Aircraft Skin Based on Machine Vision

Abstract

The present invention provides a magnetic detection system and method for defects around rivet holes of aircraft skin based on machine vision. The above method includes the following steps: taking pictures of each area of the aircraft skin rivets to obtain original images to be spliced; extracting feature information from the rivet images to obtain the feature information of the rivet images; matching, performing unified coordinate transformation on the images, and fusing the images for the feature information of the rivet images to obtain enhanced images; performing splicing processing on the images to obtain 3D point cloud images of each area of the aircraft skin rivets; based on the 3D point cloud images, through multi-scale feature processing, performing rivet target detection on the 3D point cloud images of each area of the spliced aircraft skin rivets, performing preliminary positioning on each rivet, and generating a photographing path for each rivet; according to the generated rivet photographing path, taking pictures of each rivet to obtain 3D point cloud images of the local areas of the rivets. The present invention can effectively improve the detection accuracy and detection efficiency of rivet defect detection.

Description

Technical Field

[0001] The present invention belongs to the technical field of defect detection, and particularly relates to a magnetic detection system for defects around rivet holes of aircraft skin based on machine vision. Background Art

[0002] Riveting is widely used in various industries, especially in the aviation field. Riveting is the main connection form on the aircraft skin, and the quality of riveting directly affects the overall aerodynamic performance and long fatigue life of the skin and even the aircraft. Under normal circumstances, the rivet head should be flush with the surface of the aircraft skin, but during the riveting process, problems such as protruding rivet heads and abnormal countersunk holes may occur. If the rivet head protrudes, it will inevitably affect the lap joint structure of the skin and reduce the aircraft performance. For a single rivet, if the diameter of the deformed end formed after riveting is large, the thickness of the deformed end will inevitably decrease, and at this time, the mechanical properties of the rivet are reduced and it is easy to be damaged; if the diameter of the deformed end after riveting is small, the connection at this time is unstable and the rivet is likely to fall off. In addition, the connection between aircraft skin parts is formed by riveting multiple rivets. If the distance between two certain rivets is large, the load of the parts cannot be evenly distributed, and the load borne by some rivets is greater than that of other rivets, and some rivets are likely to be damaged during service.

[0003] The detection of rivets mainly focuses on detecting small cracks, delamination and corrosion at their edges. Early researchers tried to use eddy current detection to detect such defects. With the development of various non-destructive testing technologies, technologies such as magneto-optics, ultrasound, and vision have gradually been applied to the detection of rivet hole positions. The detection during the production and assembly process of rivets and skins is relatively easy, but the detection of rivets in the intake area after the skin is assembled or during the service of the aircraft, as well as the improvement of the detection accuracy and efficiency of rivet defects, are the biggest difficulties in aircraft skin defect detection.

[0004] There are a large number of riveting structures in the aircraft skin area, and there is a great demand for the detection of riveting quality. Most traditional rivet defect detections are manually operated. When relying on the human eye to observe, problems such as reduced efficiency will occur due to the easy fatigue of the human eye and limited individual cognition. This detection method has a slow detection speed and may have problems of human error in the results. Summary of the Invention

[0005] The present invention provides a magnetic detection system for defects around rivet holes of aircraft skin based on machine vision, which can effectively solve the above problems.

[0006] The present invention is realized through the following technical solutions:

[0007] On the one hand, the present invention provides a magnetic detection method for defects around rivet holes of aircraft skin based on machine vision, including the following steps:

[0008] S10. Take pictures of each area of the aircraft skin rivets to obtain the original images to be stitched;

[0009] S20. After obtaining the original images to be stitched, extract the feature information of the rivet images to obtain the feature information of the rivet images;

[0010] S30. Match the feature information of the rivet images, perform unified coordinate transformation of the images, and image fusion to obtain enhanced images;

[0011] S40. Perform stitching processing on the images to obtain the 3D point cloud images of each area of the aircraft skin rivets;

[0012] S50. Based on the 3D point cloud images, through multi-scale feature processing, perform rivet target detection on the 3D point cloud images of each area of the stitched aircraft skin rivets, perform initial positioning on each rivet, and generate the photographing paths of each rivet;

[0013] S60. According to the generated rivet photographing paths, take pictures of each rivet to obtain the 3D point cloud images of the local areas of the rivets;

[0014] S70. Classify and segment the point cloud data in the 3D images in each rivet area, perform rivet area selection and feature extraction, and perform rivet edge detection on the 3D images in each local area of the rivets to obtain the rivet edge paths;

[0015] S80. Based on the rivet edge path information, detect around the rivet holes by a magnetic detector.

[0016] In some embodiments, performing stitching processing on the images to obtain the 3D point cloud images of each area of the aircraft skin rivets includes:

[0017] S401. Read the enhanced images to be stitched;

[0018] S402. Extract the feature points of each enhanced image;

[0019] S403. Use the k-nearest neighbor algorithm to match the feature points of each enhanced image, and stitch each enhanced image to obtain the 3D point cloud images of each area of the aircraft skin rivets.

[0020] On the other hand, the present invention provides a magnetic detection system for defects around rivet holes of aircraft skin based on machine vision, including: a large-field-of-view 3D camera module configured to take pictures of each area of the aircraft skin rivets to obtain the original images to be stitched; a small-field-of-view 3D camera module configured to take pictures of each rivet according to the generated rivet photographing paths; and a magnetic detector configured to detect around the rivet holes.

[0021] In some of these embodiments, the magnetic detector includes: an excitation device configured to generate a sinusoidal alternating current; a sensor device including two magneto-detecting sensors vertically distributed, with the sensors located at the exact center of the magnetic yoke. Among them, the Z-axis sensor collects the magnetic field signal perpendicular to the surface of the test piece, and the X-axis sensor collects the magnetic field signal parallel to the surface of the test piece; a signal processing device including a differential amplification module, a phase-sensitive demodulation module, and a digital-to-analog conversion module.

[0022] In some of these embodiments, the magnetic detection system for defects around aircraft skin rivet holes based on machine vision is integrated into a composite robot.

[0023] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0024] The present invention detects, identifies, and locates the rivets on the aircraft fuselage through 3D vision, and guides the collaborative robot to carry the detector to detect the surface of the rivet area. The large-field-of-view 3D camera module can capture the overall information and perform a preliminary positioning of the rivets. The small-field-of-view 3D camera module can locate the precise position of the rivets. By combining the large-field-of-view 3D camera module and the small-field-of-view 3D camera module, the success rate of rivet defect detection can be higher. The magnetic detector is used to detect the defects around the rivet holes. The magnetic field generated by the excitation coil is concentrated on the surface of the workpiece to be inspected through the cylindrical orthogonal magnetic yoke of the excitation device. In the excitation magnetic field environment, the workpiece to be inspected will generate an induced current. Due to the difference in conductivity between the defect area and the material itself, the induced current will accumulate on the surface of the workpiece, resulting in abnormal changes in the induced magnetic field. The magnetic induction intensity on the surface of the workpiece is collected by the magneto-detecting sensors of the sensor device to qualitatively detect and quantitatively evaluate the defects around the holes. A complete intelligent electromagnetic detection system for rivet defects is realized, improving the detection accuracy and detection efficiency of rivet defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative efforts.

[0026] Figure 1 It is a schematic flowchart of the magnetic detection method for defects around aircraft skin rivet holes based on machine vision provided by some embodiments of the present invention;

[0027] Figure 2 It is a schematic diagram of the effect of image stitching processing provided by some embodiments of the present invention;

[0028] Figure 3 Schematic diagram of the effect of obtaining the 3D point cloud image of the local area of the rivet provided by some embodiments of the present invention;

[0029] Figure 4 Schematic diagram of the effect of generating the rivet edge path to obtain the rivet edge path provided by some embodiments of the present invention;

[0030] Figure 5 Schematic diagram of the structure of the magnetic detection system for defects around the rivet holes on the aircraft skin based on machine vision provided by some embodiments of the present invention;

[0031] Figure 6 Schematic diagram of the structure of the magnetic detection system for defects around the rivet holes on the aircraft skin based on machine vision provided by other embodiments of the present invention;

[0032] Figure 7 Schematic diagram of the structure of the large - field - of - view 3D camera module provided by some embodiments of the present invention;

[0033] Figure 8 Schematic diagram of the structure of the small - field - of - view 3D camera module provided by some embodiments of the present invention;

[0034] Figure 9 Schematic diagram of the structure of the magnetic detector probe provided by some embodiments of the present invention. Detailed implementation manners

[0035] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention.

[0036] In the description of the present invention, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, or the orientation or positional relationship in which the product of the invention is usually placed during use. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be construed as a limitation to the present invention.

[0037] In addition, terms such as "horizontal" and "vertical" in the description of the present invention do not mean that the components are required to be absolutely horizontal or hanging vertically, but can be slightly inclined. For example, "horizontal" only means that its direction is more horizontal relative to "vertical", and does not mean that the structure must be completely horizontal, but can be slightly inclined.

[0038] In the description of the present invention, it should also be noted that unless otherwise clearly specified and limited, the terms "set", "installed", "connected", and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations.

[0039] The terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or modules is not limited to the listed steps or modules, but optionally further includes steps or modules not listed, or optionally further includes other steps or modules inherent to these processes, methods, products, or devices.

[0040] On the one hand, an embodiment of the present invention provides a magnetic detection method for defects around rivet holes in aircraft skins based on machine vision. Please refer to Figure 1 , which mainly includes the following steps:

[0041] S10. Take pictures of each area of the aircraft skin rivets to obtain the original images to be stitched. In some examples, a large-field-of-view 3D camera module can be used to take pictures of each area of the aircraft skin rivets to obtain two original images to be stitched.

[0042] S20. After obtaining the original images to be stitched, extract the feature information of the rivet images to obtain the feature information of the rivet images.

[0043] S30. Match the feature information of the rivet images, perform image unified coordinate transformation and image fusion to obtain an enhanced image.

[0044] S40. Perform stitching processing on the images to obtain a 3D point cloud image of each area of the aircraft skin rivets. In some examples, the left and right enhanced images to be stitched can be taken first, the feature points of the left and right enhanced images can be extracted, the feature points of the left and right enhanced images can be matched using the k-nearest neighbor algorithm, and the left and right enhanced images can be stitched to obtain the 3D point cloud of each area of the entire rivet, as Figure 2 shown.

[0045] S50. Based on the 3D point cloud image, through multi-scale feature processing, perform rivet target detection on the 3D point cloud image of each area of the stitched aircraft skin rivets, perform initial positioning on each rivet, and generate a photographing path for each rivet.

[0046] S60. According to the generated rivet photographing path, take pictures of each rivet to obtain a 3D point cloud image of the local area of the rivet, as Figure 3as shown

[0047] S70. Classify and segment the point cloud data in the 3D image within each rivet area, perform rivet area selection and feature extraction, directly detect rivets from the original 3D point cloud, perform rivet edge detection on the 3D image within each local rivet area, and accurately locate. And generate a rivet edge path based on the path planning intelligent algorithm to obtain the rivet edge path, as Figure 4 shown

[0048] S80. Based on the rivet edge path information, detect around the circumference of the rivet hole through a magnetic detector.

[0049] On the other hand, an embodiment of the present invention provides a magnetic detection system for detecting defects around the rivet holes of an aircraft skin based on machine vision. Please refer to Figure 5 and Figure 6 , which mainly includes a large-field-of-view 3D camera module, a small-field-of-view 3D camera module, and a magnetic detector.

[0050] Among them, the large-field-of-view 3D camera module adopts line-scanning laser structured light imaging, binocular structure, and optimal baseline ratio design to meet the requirements of high-precision data acquisition. The large-field-of-view 3D camera module is suitable for workpiece recognition and positioning at a long distance and with a large field of view, as Figure 7 shown. The small-field-of-view 3D camera module adopts line-scanning laser structured light imaging, is small in size, and meets the requirements of being mounted on a small-load robotic arm. With a binocular structure and optimal baseline ratio design, it meets the requirements of high-precision data acquisition. The small-field-of-view 3D camera module is suitable for workpiece recognition and positioning at a medium-close distance and with a medium-small field of view, as Figure 8 shown. Use a magnetic detector to detect defects around the rivet holes. The magnetic field generated by the excitation coil is concentrated on the surface of the workpiece to be inspected through the cylindrical orthogonal magnetic yoke of the excitation device. In the excitation magnetic field environment, the workpiece to be inspected will generate an induced current. Due to the difference in conductivity between the defect area and the material itself, there will be an accumulation of induced current on the surface of the workpiece, resulting in an abnormal change in the induced magnetic field. The magnetic induction intensity on the surface of the workpiece is collected by the magnetic detection sensor of the sensor device to qualitatively detect and quantitatively evaluate the defects around the hole. The excitation device and the sensor device (i.e., the probe) are as Figure 9 shown.

[0051] In some of the embodiments, the magnetic detection system for detecting defects around the rivet holes of an aircraft skin based on machine vision is integrated into a composite robot. In this embodiment, the composite robot is an advanced robot system integrating multiple robot technologies, functions, and applications. By simultaneously carrying a large-field-of-view 3D camera module, a small-field-of-view 3D camera module, and a magnetic detector, the efficiency of target recognition and target detection will be greatly improved.

[0052] In some of these embodiments, the magnetic detector used mainly includes an excitation device, a sensor device, and a signal processing device. The excitation device generates a sinusoidal alternating current with adjustable amplitude and frequency by a signal generator, amplifies the signal through a power amplifier and transmits it to the induction coil, and converges the induced magnetic field generated by the coil to the surface of the workpiece through a magnetic yoke; the sensor device consists of two magneto-detection sensors vertically distributed, and the sensors are located at the exact center of the magnetic yoke. Among them, the Z-axis sensor collects the magnetic field signal perpendicular to the surface of the test piece, and the X-axis sensor collects the magnetic field signal parallel to the surface of the test piece. The signals collected by the sensor device are further transmitted to the signal processing device; the signal processing device performs differential amplification, phase-sensitive demodulation, and analog-to-digital conversion processing on the signals collected by the sensors, and finally transmits the processed signals to the host computer. The magnetic detector uses a cylindrical orthogonal design to apply the excitation magnetic field. After planning the detection path, the magnetic detector detects around the rivet hole.

[0053] The objective of the present invention is to build an intelligent electromagnetic detection system for aircraft fuselage rivet detection based on robot vision guidance to complete the detection of rivet defects during the service of the aircraft. The entire 3D vision-guided automated detection system consists of multiple component links such as a 3D sensor, a collaborative robotic arm, an AGV chassis of a composite robot, a scheduling platform of the composite robot, an intelligent algorithm engine, a visual interaction system, and a magnetic detector. Multi-component system calibration is required to guide the robotic arm to perform precise detection tasks.

[0054] This embodiment also provides a computer storage medium, on which a computer program is stored. The computer program is loaded by a processing module to implement the magnetic detection method for defects around the rivet holes of the aircraft skin based on machine vision in any one of the above embodiments.

[0055] The above description is only a preferred embodiment of the present invention and does not impose any limitations on the progress of the present invention. Any simple modification or equivalent change made to the above embodiments based on the technical essence of the present invention falls within the protection scope of the present invention.

Claims

1. A magnetic detection method for defects around rivet holes on aircraft skin based on machine vision, characterized in that, It includes the following steps: S10. Take pictures of each area of the aircraft skin rivets to obtain the original images to be stitched, specifically including: taking pictures of each area of the aircraft skin rivets through a large-field-of-view 3D camera module to obtain the original images to be stitched; S20. After obtaining the original images to be stitched, extract the feature information of the rivet images to obtain the feature information of the rivet images; S30. Match the feature information of the rivet images, perform image unified coordinate transformation and image fusion to obtain enhanced images; S40. Perform stitching processing on the images to obtain the 3D point cloud images of each area of the aircraft skin rivets; S50. Based on the 3D point cloud images, through multi-scale feature processing, perform rivet target detection on the 3D point cloud images of each area of the stitched aircraft skin rivets, perform initial positioning on each rivet, and generate the photographing paths of each rivet; S60. According to the generated rivet photographing paths, take pictures of each rivet to obtain the 3D point cloud images of the local areas of the rivets, specifically including: taking pictures of each rivet through a small-field-of-view 3D camera module to obtain the 3D point cloud images of the local areas of the rivets; S70. Classify and segment the point cloud data in the 3D point cloud images in each rivet area, perform rivet area selection and feature extraction, and perform rivet edge detection on the 3D point cloud images in each local area of the rivets to obtain the rivet edge paths; S80. Based on the rivet edge path information, detect around the rivet hole by a magnetic detector, specifically including: planning the detection path based on the rivet edge path information to obtain the detection path, and detecting around the rivet hole by the magnetic detector; Performing stitching processing on the images to obtain the 3D point cloud images of each area of the aircraft skin rivets includes: S401. Read the enhanced images to be stitched; S402. Extract the feature points of each enhanced image; S403. Use the k-nearest neighbor algorithm to match the feature points of each enhanced image, and stitch each enhanced image to obtain the 3D point cloud images of each area of the aircraft skin rivets.

2. The magnetic detection system for defects around rivet holes on aircraft skin based on machine vision is used to execute the magnetic detection method for defects around rivet holes on aircraft skin based on machine vision according to claim 1, and is characterized in that, It includes: A large-field-of-view 3D camera module, which is configured to take pictures of each area of the aircraft skin rivets to obtain the original images to be stitched; A small-field-of-view 3D camera module, which is configured to take pictures of each rivet according to the generated rivet photographing paths; A magnetic detector, which is configured to detect around the rivet hole.

3. The magnetic detection system for defects around rivet holes in aircraft skin based on machine vision according to claim 2, wherein, The magnetic detector includes: An excitation device, which is configured to generate sinusoidal alternating current; A sensor device, the sensor device includes two magneto-detection sensors distributed perpendicular to each other, the magneto-detection sensors include a Z-axis sensor and an X-axis sensor, and the sensors are located at the center of the magnetic yoke. Among them, the Z-axis sensor collects the magnetic field signal in the direction perpendicular to the surface of the specimen, and the X-axis sensor collects the magnetic field signal in the direction parallel to the surface of the specimen; A signal processing device, the signal processing device includes a differential amplification module, a phase-sensitive demodulation module and a digital-to-analog conversion module.

4. The magnetic detection system for defects around rivet holes of aircraft skin based on machine vision according to claim 2 or 3, characterized in that, The magnetic detection system for defects around the aircraft skin rivet holes based on machine vision is integrated in a composite robot.

Images