Method and device for detecting wing protection tool, electronic equipment and storage medium

Abstract

This invention discloses a method, apparatus, electronic device, and storage medium for detecting wing protection fixtures. The method includes: extracting the wing region from an aircraft image, correcting the wing region's viewpoint, and binarizing the wing region to obtain a binary wing image; counting the number of non-zero pixels in the direction perpendicular to the installation direction of the protection fixture in the binary wing image to obtain a non-zero pixel count result; and performing compliance detection of the protection fixture installation based on the non-zero pixel count result. This invention enables high-precision automated compliance detection of wing protection fixtures.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and image processing technology, and in particular to a method, apparatus, electronic device, and storage medium for detecting wing protection fixtures. Background Technology

[0002] During aircraft assembly, protective fixtures are typically installed on the wings to protect them from damage caused by tools, personnel, and other factors. The correct installation and securing of these fixtures is crucial; any misalignment or improper installation can potentially damage the wings. Therefore, verifying the compliance of the wing protective fixture installation is an important part of the aircraft assembly process.

[0003] Existing compliance inspections of wing protection fixture installations are either performed manually, which is inefficient, costly, and has limited effectiveness, or rely on sensor-assisted inspections, which are expensive in terms of hardware. Therefore, introducing automated, intelligent, and low-cost compliance inspection methods for wing protection fixture installations is a necessary direction for improving safety management. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for detecting wing protection fixtures, so as to achieve high-precision automated compliance detection of wing protection fixtures.

[0005] In a first aspect, embodiments of the present invention provide a method for detecting wing protection fixtures, the method comprising:

[0006] The aircraft image is processed by extracting the wing region, correcting the wing region viewpoint, and binarizing the wing region to obtain a binary wing image.

[0007] The number of non-zero pixels in the vertical direction of the protective tooling paving direction is counted in the binary image of the wing, and the result of the non-zero pixel count is obtained.

[0008] Based on the statistical results of the number of non-zero pixels, compliance testing of the protective fixture installation is conducted.

[0009] Secondly, embodiments of the present invention also provide a detection device for wing protection fixtures, the device comprising:

[0010] The wing binary image determination module is used to extract the wing region from the aircraft image, correct the wing region viewpoint, and perform wing region binarization to obtain a wing binary image.

[0011] The module for determining the number of non-zero pixels is used to count the number of non-zero pixels in the vertical direction of the protective tooling paving direction in the binary image of the wing, and to obtain the count of non-zero pixels.

[0012] The protective fixture installation compliance detection module is used to perform installation compliance detection of protective fixtures based on the statistical results of the number of non-zero pixels.

[0013] Thirdly, embodiments of the present invention also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a detection method for wing protection tooling as described in any of the embodiments of the present invention.

[0014] Fourthly, embodiments of the present invention also provide a storage medium for storing computer-executable instructions, which, when executed by a computer processor, are used to perform a detection method for a wing protection fixture as described in any of the embodiments of the present invention.

[0015] The technical solution of this invention extracts the wing region from an aircraft image, corrects the wing region's perspective, and binarizes the wing region to obtain a binary wing image. This avoids the impact of perspective distortion on detection accuracy and improves the accuracy and robustness of protective fixture detection. The number of non-zero pixels in the vertical direction perpendicular to the protective fixture's installation direction is counted in the binary wing image, and the installation compliance of the protective fixture is checked based on this count. By determining whether the protective fixture meets installation compliance conditions based on the pixel distribution statistical characteristics of the image, high-precision, automated, and real-time detection of the protective fixture's installation status is achieved.

[0016] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart of a detection method for a wing protection fixture provided in Embodiment 1 of the present invention;

[0019] Figure 2 This is a schematic diagram of the semantic segmentation result of an aircraft image provided in Embodiment 1 of the present invention;

[0020] Figure 3 This is a schematic diagram of a wing region provided in Embodiment 1 of the present invention;

[0021] Figure 4 This is a schematic diagram of the wing region outline provided in Embodiment 1 of the present invention;

[0022] Figure 5 This is a schematic diagram of a modified wing region outline provided in Embodiment 1 of the present invention;

[0023] Figure 6 This is a wing image from a top-down view after perspective correction, provided in Embodiment 1 of the present invention;

[0024] Figure 7 This is a schematic diagram of a binary image of a wing rotated 90 degrees counterclockwise, provided in Embodiment 1 of the present invention;

[0025] Figure 8 This is a schematic diagram of marking non-compliant protective tooling installation in an aircraft image, provided by Embodiment 1 of the present invention.

[0026] Figure 9 This is a flowchart of a detection method for a wing protection fixture provided in Embodiment 2 of the present invention;

[0027] Figure 10 This is a schematic diagram of a histogram showing the number of non-zero pixels in each segment, provided in Embodiment 2 of the present invention;

[0028] Figure 11 This is a schematic diagram of a non-compliant area of ​​protective tooling paving in a binary image of an aircraft wing, provided in Embodiment 2 of the present invention;

[0029] Figure 12 This is a schematic diagram of the structure of a detection device for a wing protection fixture provided in Embodiment 3 of the present invention;

[0030] Figure 13 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or device that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices. In the embodiments of this application, certain software, components, models, and other existing industry solutions may be mentioned. These should be considered exemplary, intended only to illustrate the feasibility of implementing the technical solutions of this application, and do not imply that the applicant has already used or necessarily used such solutions.

[0033] The acquisition, transmission, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0034] Example 1

[0035] Figure 1 The flowchart of the detection method for wing protection fixtures provided in Embodiment 1 of the present invention is applicable to the situation of installation compliance detection of wing protection fixtures. The method can be executed by a detection device for wing protection fixtures. The detection device for wing protection fixtures can be implemented in hardware and / or software. The detection device for wing protection fixtures can be configured in a server or electronic device and used in conjunction with a shooting device.

[0036] like Figure 1 As shown, the method includes:

[0037] S110. Extract the wing region from the aircraft image, correct the wing region viewpoint, and perform binarization processing on the wing region to obtain a binary wing image.

[0038] The aircraft image refers to an image taken of the aircraft after the wing protective fixtures have been installed during the aircraft assembly process. It can be taken by a camera or a surveillance camera; this embodiment does not limit the type of capturing equipment. Because this embodiment performs compliance checks on the installation of the wing protective fixtures, the aircraft image in this embodiment must at least completely include the wing area of ​​the aircraft.

[0039] Before extracting the wing region from the aircraft image, the image quality can be evaluated. Image quality can be assessed from dimensions such as sharpness, brightness, and noise. This embodiment does not impose any limitations on this evaluation. Aircraft images that meet the image quality requirements will then undergo further processing. Image preprocessing can also be performed on the aircraft image, including noise reduction and image enhancement.

[0040] Wing region extraction is used to remove background areas from aircraft images, retaining only the wing region, thereby enabling more accurate detection of wing protection fixtures.

[0041] Wing region perspective correction is used to convert the wing region into a perspective-corrected image. Perspective correction can accurately capture the pixels of the wing protection fixture, thereby reducing the detection error caused by perspective distortion.

[0042] Wing region binarization is used to obtain a binary image of the wing through grayscale conversion and threshold segmentation, making the wing image clearer and facilitating the statistical analysis of pixel distribution features.

[0043] The binary image of the wing represents the wing region from a top-down view. In the binary image of the wing, a pixel value of 1 indicates the protective tooling area, and a pixel value of 0 indicates the background area.

[0044] Furthermore, S110 may include:

[0045] S111. Extract the wing region from the aircraft image and correct the outline of the extracted wing region.

[0046] S112. Perform perspective correction on the wing region after contour correction according to the perspective transformation matrix to obtain the wing image from the top view.

[0047] S113. Perform binarization processing on the wing image to obtain a binary wing image.

[0048] The wing region extraction can be achieved using semantic segmentation. Specifically, a semantic segmentation model is pre-trained, the aircraft image is input into the semantic segmentation model, the semantic segmentation result of the aircraft image is obtained, and the wing region is extracted from the semantic segmentation result.

[0049] In this embodiment, semantic-level region analysis is performed on the aircraft image to automatically identify and extract the wing region in the aircraft image, thereby achieving stable positioning of the wing region in the complex background of aircraft assembly, and providing a unified processing basis for subsequent detection.

[0050] For example, Figure 2 A schematic diagram of semantic segmentation results for an aircraft image is provided, such as... Figure 2As shown, in the results of semantic segmentation of the aircraft image, different colors correspond to different types of regions, with the pink part being the wing region. Figure 3 A schematic diagram of the wing region is provided, such as... Figure 3 As shown, the white area is the wing area.

[0051] The extracted wing region often has irregular edge contours. To facilitate subsequent statistical analysis of pixel distribution features, this embodiment corrects the contour of the wing region. Specifically, contour correction can be achieved using morphological operations or by utilizing the contour fitting and correction function of image processing software; this embodiment does not impose any restrictions. The corrected wing region has a regular quadrilateral contour, and the wing region itself is a regular quadrilateral region.

[0052] Figure 4 A schematic diagram of the wing region outline is provided, such as... Figure 4 As shown, the extracted wing region has an irregular edge contour. Figure 5 A schematic diagram of the corrected wing region profile is provided, such as... Figure 5 As shown, the wing area after contour correction is a regular quadrilateral.

[0053] The perspective transformation matrix is ​​a 3×3 matrix, which can be obtained through... The perspective transformation matrix can be calculated using the coordinates of the four corner points of the wing region after contour correction, as well as the coordinates of the four corner points of the defined wing image.

[0054] Viewpoint correction can be achieved using the following formula: Where (u, v) are the pixel coordinates of the wing region after contour correction, (x=x ' / w ' y=y ' / w ' ) represents the pixel coordinates in the wing image after viewpoint correction, w ' This represents the coordinate transformation coefficient.

[0055] In this embodiment, based on the geometric feature information of the wing region, an image perspective mapping relationship is constructed. The perspective of the wing region with perspective distortion under the original perspective is corrected to generate the wing image with the corresponding top-down perspective. This can reduce the scale inconsistency and deformation error caused by the change of shooting angle, accurately capture the pixels of the wing protection fixture, reduce the detection error caused by perspective distortion, make the installation position and coverage of the protection fixture more accurate, and ensure the detection accuracy.

[0056] For example, Figure 6 This provides a view-corrected image of the wing from a top-down perspective, such as... Figure 6 As shown, by correcting the viewing angle, the installation position of the protective tooling and its coverage of the wing are more accurate, which can ensure the accuracy of the inspection.

[0057] Furthermore, before extracting the wing region from the aircraft image, the process includes updating the perspective transformation matrix if it is determined that the current aircraft image does not match historical aircraft images. Correspondingly, the perspective correction is performed on the wing region after contour correction based on the perspective transformation matrix, including: performing perspective correction on the wing region after contour correction based on the updated perspective transformation matrix.

[0058] Understandably, compliance inspection of protective equipment installation can be an ongoing process. When the inspection scenario remains unchanged—that is, when the positions of the wing and the camera equipment remain unchanged—the perspective transformation matrix also remains unchanged. Therefore, once the perspective transformation matrix is ​​calculated, it can be directly used until the inspection scenario changes. However, if the inspection scenario changes, including changes in the wing position and / or the position of the camera equipment, the perspective transformation matrix needs to be recalculated and updated.

[0059] In this embodiment, detecting whether the scene has changed can be achieved by performing template matching on the aircraft images. Specifically, template matching is performed between the current aircraft image and historical aircraft images. The historical aircraft images can be either the previous frame of the current aircraft image or historical aircraft images that are spaced a preset number of frames away from the current aircraft image. That is, template matching can be performed on each aircraft image in real time or at interval frames.

[0060] This embodiment determines whether the detection scene has changed through template matching. When a change is detected, new semantic segmentation is performed, achieving adaptive updating of detection parameters based on scene change judgment. This enables timely updates to detection results, improving the system's flexibility and robustness. Simultaneously, it avoids the high computational load associated with frame-by-frame segmentation, significantly improving the system's real-time response capability and processing efficiency.

[0061] S120. Count the number of non-zero pixels in the vertical direction of the protective tooling paving direction on the binary image of the wing, and obtain the result of the non-zero pixel count.

[0062] The number of non-zero pixels in the vertical direction of the protective tooling paving direction in the binary image of the wing can be counted either pixel by pixel along the protective tooling paving direction of the binary image of the wing, or the binary image of the wing can be divided into segments along the protective tooling paving direction, and the number of non-zero pixels in each segment can be counted separately.

[0063] For example, Figure 6In the wing image, the protective fixture is laid along the image height. To facilitate the counting of non-zero pixels, this embodiment rotates the binary wing image counterclockwise by 90 degrees. Figure 7 A schematic diagram is provided showing a binary image of a wing rotated 90 degrees counterclockwise, as shown below. Figure 7 As shown, the protective fixture is laid in the direction along the width of the image.

[0064] In this embodiment, based on Figure 7 The image can be counted pixel by pixel along the width direction, either by counting the number of non-zero pixels in the vertical direction (i.e., counting the number of non-zero pixels in each column along the width direction) or by dividing the image into segments and counting the number of non-zero pixels in each segment separately.

[0065] In this embodiment, by measuring the number of non-zero pixels in the vertical direction of the protective fixture installation direction, pixel statistical characteristics, regional distribution characteristics, and continuity characteristics can be determined, thereby reflecting the coverage of the protective fixture installation and realizing automatic detection of the protective fixture installation status.

[0066] Furthermore, before counting the number of non-zero pixels in the vertical direction of the protective fixture installation direction in the binary image of the wing, morphological operations such as erosion and dilation can be used to remove noise from the binary image of the wing, enhance and connect the features of the protective fixture, and make the binary image of the wing clearer in subsequent analysis.

[0067] S130. Based on the statistical results of the number of non-zero pixels, conduct compliance testing on the installation of protective fixtures.

[0068] In this embodiment, since the non-zero pixel count is obtained by counting the number of non-zero pixels in the vertical direction along the paving direction of the protective fixture, the non-zero pixel count can reflect the paving coverage of the protective fixture in the paving direction.

[0069] Specifically, with Figure 7 Taking this example, the statistical results of the number of non-zero pixels, whether it is the number of non-zero pixels in each column corresponding to each pixel or the number of non-zero pixels in each segment, can be used to determine whether the installation of the protective fixture is compliant by comparing the statistical results of the number of non-zero pixels with a pre-set threshold.

[0070] Understandably, the non-zero pixel count can reflect the coverage of the protective fixture. When the non-zero pixel count is less than the preset threshold, it indicates that the coverage of the protective fixture does not meet the paving regulations.

[0071] In this embodiment, the statistical results of the number of non-zero pixels can be compared with the threshold one by one, or the statistical results of the number of non-zero pixels can be represented by histograms, line graphs, curves, etc., and then the regions where the statistical results of the number of non-zero pixels are less than or equal to the threshold can be determined by horizontal search of histograms, line graphs, curves, etc.

[0072] In this embodiment, after obtaining the statistical result of the number of non-zero pixels corresponding to the non-compliant installation of the protective tooling, an inverse transformation is performed based on the position of the statistical result of the number of non-zero pixels in the binary image of the wing to obtain the position of the non-compliant installation of the protective tooling in the original aircraft image.

[0073] Specifically, regarding the position coordinates of the non-zero pixel count results corresponding to non-compliant protective equipment installation in the binary wing image, if the binary wing image has been rotated 90 degrees counterclockwise, it should first be rotated 90 degrees clockwise. Specifically, the position coordinates (x, y) after a 90-degree clockwise rotation can be expressed by the following formula: x = H - 1 - Y, y = X. (X, Y) are the position coordinates in the binary wing image, and H represents the height of the binary wing image. Then, through inverse perspective transformation, the position coordinates (x, y) are converted to position coordinates (u, v) in the original aircraft image. Inverse perspective transformation can be expressed by the following formula: , , .in, Let represent the inverse perspective transformation matrix, which is the inverse of the perspective transformation matrix M in the above embodiment.

[0074] Furthermore, after obtaining the location of non-compliant protective tooling installation in the original aircraft image, alarms can be issued regarding the non-compliant location and coverage, thus preventing wing damage and safety hazards caused by improper installation of the protective tooling.

[0075] Figure 8 This document provides a schematic diagram for marking non-compliant protective tooling installations in aircraft images, such as... Figure 8 As shown, the detection results obtained from the corrected viewpoint image are transformed into the original image coordinate system through reverse mapping, which realizes intuitive labeling and prompting of non-compliant areas of protective equipment paving, making it easier for on-site personnel to understand and handle.

[0076] In this embodiment, compliance inspection of protective tooling installation is performed based on the statistical results of non-zero pixel counts. This improves the automation and accuracy of the compliance inspection, significantly reduces the need for manual intervention, enhances safety and management efficiency in the production process, and provides reliable image evidence chain support. This embodiment is applicable to different shooting equipment configurations, has strong scalability, and can be extended to other aerospace manufacturing or high-precision protection operating environments.

[0077] The technical solution of this invention extracts the wing region from an aircraft image, corrects the wing region's perspective, and binarizes the wing region to obtain a binary wing image. This avoids the impact of perspective distortion on detection accuracy and improves the accuracy and robustness of protective fixture detection. The number of non-zero pixels in the vertical direction perpendicular to the protective fixture's installation direction is counted in the binary wing image, and the installation compliance of the protective fixture is checked based on this count. By determining whether the protective fixture meets installation compliance conditions based on the pixel distribution statistical characteristics of the image, high-precision, automated, and real-time detection of the protective fixture's installation status is achieved.

[0078] Example 2

[0079] Figure 9 This is a flowchart of a detection method for a wing protection fixture provided in Embodiment 2 of the present invention. Based on the above embodiments, the present invention further specifies the process of counting the number of non-zero pixels and the process of detecting compliance of the protective fixture installation.

[0080] like Figure 9 As shown, the method includes:

[0081] S210. Extract the wing region from the aircraft image and correct the contour of the extracted wing region.

[0082] S220. Based on the perspective transformation matrix, the wing region after contour correction is subjected to perspective correction to obtain the wing image from the top view.

[0083] S230. Perform binarization processing on the wing image to obtain a binary wing image.

[0084] The processes of wing region extraction, wing region contour correction, viewpoint correction, and binarization have been described in the above embodiments and will not be repeated here.

[0085] S240. Count the number of non-zero pixels in the vertical direction of the protective tooling paving direction on the binary image of the wing, and obtain the result of the non-zero pixel count.

[0086] This embodiment provides two specific implementation methods for counting the number of non-zero pixels.

[0087] In an optional embodiment, S240 may include: segmenting the binary image of the wing along the direction of the protective fixture installation, and counting the number of non-zero pixels in each segment to obtain the count of the number of non-zero pixels in each segment.

[0088] In this embodiment, the binary image of the wing is divided into a preset number of segments, such as 20 segments, along the direction of the protective fixture installation. For each segment, a search and count of non-zero pixels is performed to obtain the statistical result of the number of non-zero pixels in each segment. In this embodiment, performing non-zero pixel counts on the segments of the wing binary image can reduce the consumption of computing resources and improve computational and detection efficiency while ensuring detection effectiveness.

[0089] In this embodiment, the statistical results of the number of non-zero pixels corresponding to each segment can be directly represented by the segment index of each segment and the statistical results of the number of non-zero pixels corresponding to it, or by the line chart or histogram corresponding to the index of each segment. This can provide more intuitive image evidence chain support for the compliance inspection of the protective tooling.

[0090] Figure 10 A schematic diagram of a histogram showing the number of non-zero pixels in each segment is provided, such as... Figure 10 As shown, the horizontal axis represents the segment index of each segment obtained by segmenting the binary image of the wing along the paving direction of the protective fixture. Since there are 20 segments in total, the segment index is set to 0-19. The vertical axis represents the statistical result of the number of non-zero pixels corresponding to each segment.

[0091] In another optional embodiment, S240 may further include: taking the binary image of the wing, with the direction of the protective tooling installation as the horizontal axis and the number of non-zero pixels in the direction perpendicular to the direction of the protective tooling installation as the vertical axis, to obtain a statistical curve of the number of non-zero pixels.

[0092] In this embodiment, the number of non-zero pixels in the vertical direction is counted pixel by pixel along the protective fixture installation direction of the binary image of the wing. At the same time, a non-zero pixel count curve is plotted with each pixel in the protective fixture installation direction as the horizontal axis and the number of non-zero pixels corresponding to each pixel in the vertical direction as the vertical axis.

[0093] In this embodiment, the number of non-zero pixels in the vertical direction of the protective tooling installation direction is counted pixel by pixel, which can improve the detection accuracy. Depending on the needs of the detection site for computing efficiency and detection accuracy, the binary image of the wing can be flexibly segmented or the number of non-zero pixels can be counted pixel by pixel.

[0094] Furthermore, to improve the accuracy of subsequent compliance testing of protective equipment installation and avoid interference from image noise, the non-zero pixel count curve can be smoothed, and subsequent compliance testing of protective equipment installation can be performed based on the smoothed non-zero pixel count curve.

[0095] S250: Based on the statistical results of the number of non-zero pixels, conduct compliance testing on the installation of protective fixtures.

[0096] When the non-zero pixel count result is represented by the non-zero pixel count result of each segment, S250 may include: if the non-zero pixel count result of the target segment is less than or equal to a preset non-zero pixel count threshold, then the protective fixture installation at the target segment is determined to be non-compliant; if the number of consecutive target segments with non-compliant protective fixture installation is greater than or equal to a preset segment number threshold, then the consecutive target segments are merged to obtain the non-compliant area of ​​the protective fixture installation.

[0097] Specifically, the process of setting the non-zero pixel count threshold may include: calculating the size ratio based on the actual size of the wing and the actual size of the protective fixture; calculating the theoretical value of the protective fixture pixel size based on the segmented pixel size and size ratio of the binary image of the wing; and finally calculating the non-zero pixel count threshold based on the theoretical value of the protective fixture pixel size and the dynamic coefficient.

[0098] After obtaining the non-zero pixel count statistics for each segment, the non-zero pixel count statistics for each segment are compared with the non-zero pixel count threshold. Segments with a non-zero pixel count less than or equal to the threshold are marked as target segments. The position coordinates of the target segment in the wing binary image are determined based on its segment index (represented by the coordinates of its four endpoints). Following the inverse transformation method described in the above embodiment, the position coordinates of the target segment in the wing binary image are converted to position coordinates in the original aircraft image. Based on these position coordinates, non-compliant protection tooling is marked in the aircraft image.

[0099] Furthermore, if the number of consecutive target segments is greater than or equal to the preset segment number threshold (e.g., 2), that is, there are at least two consecutive target segments, then each target segment is merged to obtain the non-compliant area of ​​the protective equipment paving.

[0100] Similarly, for the non-compliant area of ​​the protective tooling after merging, determine its vertex coordinates in the binary image of the wing. According to the inverse transformation method in the above embodiment, convert the vertex coordinates of the non-compliant area of ​​the protective tooling into vertex coordinates in the original aircraft image. Based on the vertex coordinates of the non-compliant area of ​​the protective tooling in the original aircraft image, mark the non-compliant area of ​​the protective tooling in the aircraft image.

[0101] For example, Figure 10 The number of non-zero pixels corresponding to the middle segment indices 14, 15, 16 and 17 are all less than the non-zero pixel number threshold, so the target segments 14, 15, 16 and 17 are merged. Figure 11 A schematic diagram of non-compliant areas of protective tooling installation in a binary image of an airfoil is provided, such as... Figure 11 As shown, the region after merging target segments 14, 15, 16, and 17 is... Figure 11 The area in the middle of the yellow box.

[0102] When the non-zero pixel count result is represented by a non-zero pixel count curve, S250 may include: if the horizontal coordinate interval on the non-zero pixel count curve is determined to be less than or equal to a preset non-zero pixel count threshold, and greater than or equal to a preset interval threshold, then the protective fixture installation at the horizontal coordinate interval is determined to be non-compliant.

[0103] Specifically, in this embodiment, the process of setting the non-zero pixel count threshold may include: calculating the size ratio based on the actual size of the wing in the vertical direction of the protective fixture installation direction and the actual size of the protective fixture in the vertical direction of the protective fixture installation direction; then calculating the theoretical value of the protective fixture pixel size based on the pixel size and size ratio of the binary image of the wing in the vertical direction of the protective fixture installation direction; and finally calculating the non-zero pixel count threshold based on the theoretical value of the protective fixture pixel size and the dynamic coefficient.

[0104] In this embodiment, the abscissa interval of the non-zero pixel count statistics curve that is less than or equal to the non-zero pixel count threshold is represented on the curve as the abscissa interval of the intersection point of the non-zero pixel count statistics curve and the straight line: y = non-zero pixel count threshold.

[0105] Setting interval thresholds (e.g., 10 pixels) and setting both the horizontal coordinate interval and the interval threshold serves to eliminate false positives caused by image noise. Only when the non-compliant interval exceeds a certain interval threshold is the protective fixture installation at that horizontal coordinate interval considered non-compliant. If the number of non-zero pixels before and after a certain horizontal coordinate interval is greater than the non-zero pixel number threshold, and the horizontal coordinate interval is small, then that horizontal coordinate interval can be considered a false positive.

[0106] The technical solution of this invention extracts the wing region from aircraft images, corrects the wing region perspective, and binarizes the wing region to obtain a binary wing image. This achieves semantic-level wing region analysis, automatically identifies and extracts the wing region, and enables stable positioning of the wing region in complex assembly environments. This provides a unified data foundation for subsequent protective tooling inspection. Simultaneously, perspective correction avoids the impact of perspective distortion on inspection accuracy, improving the accuracy and robustness of protective tooling inspection. The number of non-zero pixels in the vertical direction perpendicular to the protective tooling installation direction is counted in the binary wing image. Based on the non-zero pixel count, compliance inspection of the protective tooling installation is performed. Based on the pixel distribution statistical characteristics, regional distribution characteristics, and continuity characteristics of the image, it is determined whether the protective tooling meets the installation compliance conditions. This achieves high-precision, automated, real-time inspection of the protective tooling installation status and provides two selectable implementation methods: segmented statistics and pixel-by-pixel statistics. This greatly improves the flexibility and applicability of protective tooling installation compliance inspection in aircraft assembly sites.

[0107] Example 3

[0108] Figure 12 This is a schematic diagram of the detection device for a wing protection fixture provided in Embodiment 3 of the present invention. Figure 12 As shown, the device includes:

[0109] The wing binary image determination module 310 is used to extract the wing region, correct the wing region viewpoint, and perform binarization processing on the aircraft image to obtain a wing binary image.

[0110] The module 320 for determining the non-zero pixel count is used to count the number of non-zero pixels in the vertical direction of the protective tooling paving direction of the binary image of the wing, and to obtain the non-zero pixel count result.

[0111] The protective fixture installation compliance detection module 330 is used to perform installation compliance detection of protective fixtures based on the statistical results of the number of non-zero pixels.

[0112] The technical solution of this invention extracts the wing region from an aircraft image, corrects the wing region's perspective, and binarizes the wing region to obtain a binary wing image. This avoids the impact of perspective distortion on detection accuracy and improves the accuracy and robustness of protective fixture detection. The number of non-zero pixels in the vertical direction perpendicular to the protective fixture's installation direction is counted in the binary wing image, and the installation compliance of the protective fixture is checked based on this count. By determining whether the protective fixture meets installation compliance conditions based on the pixel distribution statistical characteristics of the image, high-precision, automated, and real-time detection of the protective fixture's installation status is achieved.

[0113] Based on the above embodiments, optionally, the wing binary image determination module 310 includes:

[0114] The wing region extraction unit is used to extract the wing region from the aircraft image and to perform contour correction on the extracted wing region.

[0115] The perspective correction unit is used to correct the perspective of the wing area after the contour correction according to the perspective transformation matrix, so as to obtain the wing image from the top view.

[0116] The binarization processing unit is used to perform binarization processing on the wing image to obtain a binary wing image.

[0117] Optionally, based on the above embodiments, the apparatus further includes:

[0118] The perspective transformation matrix update module is used to update the perspective transformation matrix if it is determined that the current aircraft image does not match the historical aircraft image.

[0119] The view correction unit is specifically used for:

[0120] Based on the updated perspective transformation matrix, the viewpoint of the wing area after contour correction is adjusted.

[0121] Based on the above embodiments, optionally, the non-zero pixel count determination module 320 includes:

[0122] The non-zero pixel count segmentation and statistics unit is used to segment the binary image of the wing along the paving direction of the protective fixture, and to count the number of non-zero pixels in each segment to obtain the non-zero pixel count results for each segment.

[0123] Based on the above embodiments, optionally, the protective tooling installation compliance detection module 330 includes:

[0124] The segmented threshold judgment unit is used to determine that the protective fixture installation at the target segment is non-compliant if the statistical result of the number of non-zero pixels in the target segment is less than or equal to the preset threshold for the number of non-zero pixels.

[0125] The target segment merging unit is used to merge each continuous target segment to obtain the non-compliant area of ​​the protective equipment paving if the number of continuous target segments with non-compliant paving is greater than or equal to a preset segment number threshold.

[0126] Based on the above embodiments, optionally, the non-zero pixel count determination module 320 includes:

[0127] The non-zero pixel count curve determination unit is used to obtain a non-zero pixel count curve from the binary image of the wing, with the direction of the protective tooling installation as the horizontal axis and the number of non-zero pixels in the direction perpendicular to the direction of the protective tooling installation as the vertical axis.

[0128] Based on the above embodiments, optionally, the protective tooling installation compliance detection module 330 includes:

[0129] The horizontal coordinate interval judgment unit is used to determine that the protective fixture installation at the horizontal coordinate interval is non-compliant if the horizontal coordinate interval on the non-zero pixel number statistics curve is less than or equal to a preset non-zero pixel number threshold, and greater than or equal to a preset interval threshold.

[0130] The detection device for wing protection fixtures provided in this embodiment of the invention can execute the detection method for wing protection fixtures provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0131] Example 4

[0132] Figure 13 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0133] like Figure 13 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0134] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0135] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the detection method for wing protection tooling.

[0136] In some embodiments, the method for detecting wing protection fixtures can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the wing protection fixture detection method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the wing protection fixture detection method by any other suitable means (e.g., by means of firmware).

[0137] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] Computer programs used to implement the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to the processor of a general-purpose computer, a special-purpose computer, or other programmable wing protection tooling detection device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0139] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0140] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0141] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0142] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0143] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0144] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting a wing protection fixture, characterized in that, include: The aircraft image is processed by extracting the wing region, correcting the wing region viewpoint, and binarizing the wing region to obtain a binary wing image. The number of non-zero pixels in the vertical direction of the protective tooling paving direction is counted in the binary image of the wing, and the result of the non-zero pixel count is obtained. Based on the statistical results of the number of non-zero pixels, compliance testing of the protective fixture installation is conducted.

2. The method according to claim 1, characterized in that, The aircraft image undergoes wing region extraction, wing region viewpoint correction, and wing region binarization to obtain a binary wing image, including: Extract the wing region from the aircraft image and then perform contour correction on the extracted wing region; The perspective of the wing region after contour correction is corrected by the perspective transformation matrix to obtain the wing image from the top view. The wing image is binarized to obtain a binary wing image.

3. The method according to claim 2, characterized in that, Before extracting the wing region from the aircraft image, the following steps are also included: If it is determined that the current aircraft image does not match the historical aircraft image, the perspective transformation matrix is ​​updated. The perspective of the wing region after contour correction is adjusted based on the perspective transformation matrix, including: Based on the updated perspective transformation matrix, the viewpoint of the wing area after contour correction is adjusted.

4. The method according to claim 1, characterized in that, The number of non-zero pixels in the vertical direction of the protective tooling paving direction is counted in the binary image of the wing. The results of the non-zero pixel count are as follows: The binary image of the wing is segmented along the direction of the protective fixture installation, and the number of non-zero pixels in each segment is counted to obtain the statistical results of the number of non-zero pixels in each segment.

5. The method according to claim 4, characterized in that, Based on the statistical results of the number of non-zero pixels, compliance testing of the protective fixture installation is conducted, including: If the statistical result of the number of non-zero pixels in the target segment is less than or equal to the pre-set threshold for the number of non-zero pixels, then the protective fixture installation at the target segment is determined to be non-compliant. If the number of continuous target segments with non-compliant protective equipment paving is determined to be greater than or equal to the preset segment number threshold, then the continuous target segments are merged to obtain the non-compliant protective equipment paving area.

6. The method according to claim 1, characterized in that, The number of non-zero pixels in the vertical direction of the protective tooling paving direction is counted in the binary image of the wing. The results of the non-zero pixel count are as follows: For the binary image of the wing, the non-zero pixel count curve is obtained by taking the direction of the protective tooling installation as the horizontal axis and the number of non-zero pixels in the direction perpendicular to the direction of the protective tooling installation as the vertical axis.

7. The method according to claim 6, characterized in that, Based on the statistical results of the number of non-zero pixels, compliance testing of the protective fixture installation is conducted, including: If the horizontal coordinate interval on the non-zero pixel count statistics curve is determined to be less than or equal to a preset non-zero pixel count threshold, and greater than or equal to a preset interval threshold, then the protective fixture installation at the horizontal coordinate interval is determined to be non-compliant.

8. A detection device for a wing protection fixture, characterized in that, include: The wing binary image determination module is used to extract the wing region from the aircraft image, correct the wing region viewpoint, and perform wing region binarization to obtain a wing binary image. The module for determining the number of non-zero pixels is used to count the number of non-zero pixels in the vertical direction of the protective tooling paving direction in the binary image of the wing, and to obtain the count of non-zero pixels. The protective fixture installation compliance detection module is used to perform installation compliance detection of protective fixtures based on the statistical results of the number of non-zero pixels.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the detection method for the wing protection fixture as described in any one of claims 1-7.

10. A storage medium for storing computer-executable instructions, characterized in that, The computer-executable instructions, when executed by a computer processor, are used to perform the detection method of the wing protection tooling as described in any one of claims 1-7.

Images