A visual error-proofing detection method and electronic device for engine locking screws

Abstract

This invention relates to a visual error-proofing detection method and electronic device for engine locking wires. The method includes: Step 1, performing illumination invariance and perspective correction processing on the acquired engine locking wire image to achieve multi-view image registration; Step 2, performing feature point detection and caliper fitting processing on the registered image to determine the two ends of the locking wire; Step 3, determining the installation direction of the locking wire based on the two ends in Step 2. Compared with the prior art, this invention has advantages such as improved detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a visual error-proofing detection method for engine locking screws. Background Technology

[0002] As the core power unit of a flight system, the performance and reliability of an aircraft engine directly affect its flight capability and mission execution effectiveness. In the field of equipment manufacturing, the assembly of aircraft engines is an extremely complex and precise process, involving the assembly, docking, and fixing of numerous components, with extremely stringent requirements for precision and quality. Any minor assembly error can lead to a decline in engine performance, or even affect the normal operation of the entire aircraft system, potentially causing irreparable economic losses and mission failure. Therefore, ensuring the precision and reliability of aircraft engine assembly has become a crucial issue that urgently needs to be addressed in the field of defense equipment manufacturing.

[0003] Traditional assembly quality inspection methods primarily rely on manual inspection. Research indicates that over 80% of aircraft engine inspections are conducted manually using visual inspection. While simple, visual inspection, as a traditional and commonly used method, is susceptible to various factors affecting its reliability and accuracy. Inspectors judge the assembly results visually or with simple tools to determine if the assembly meets quality requirements. Although this method played a crucial role in early equipment production, its limitations are becoming increasingly apparent in the high-precision assembly of modern aircraft engines. First, manual inspection is highly subjective, easily influenced by the inspector's experience, operational skills, and fatigue, leading to missed or incorrect inspections and failing to meet the high-precision, high-consistency quality requirements. Second, manual inspection is inefficient, especially under large-scale mass production conditions, where the time and labor costs increase significantly, contradicting the demands of efficient production in modern equipment manufacturing. Furthermore, traditional inspection methods rely heavily on manual recording, lacking real-time data feedback and end-to-end traceability capabilities, making it difficult to support the transformation of equipment manufacturing towards intelligence and digitalization.

[0004] With the rapid development of industrial automation and intelligent technologies, machine vision technology, as an important component of intelligent manufacturing, is gradually replacing traditional manual inspection methods. Machine vision utilizes industrial cameras, optical imaging equipment, and computer vision algorithms to perform real-time detection and evaluation of key components and processes during assembly. As the "eyes of industrial manufacturing," machine vision has naturally become a top priority in technological research and innovation.

[0005] A search revealed that Chinese invention patent application publication number CN116452546A discloses a visual inspection method for locking wires based on semantic segmentation, comprising the following steps: S1. Obtaining a training set for pipe fitting locking wires: Designing and fabricating a physical template of pipe fitting locking wires to simulate a real environment; manually creating positive and negative samples of locking wires on the template; setting up an image acquisition device and acquiring images of the positive and negative samples of pipe fitting locking wires; acquiring images under multiple lighting conditions and finally forming a sample set of pipe fitting locking wires; creating a mask label file required for training the semantic segmentation network, generating a mask image that corresponds one-to-one with the sample set, and forming a training set of pipe fitting locking wires together with the sample set of pipe fitting locking wires; S2. Semantic segmentation-based fusion determination of the existence of pipe fitting locking wires: inputting the training set of pipe fitting locking wires obtained in step S1 into a deep learning segmentation network. The process involves training the locking wire features; optimizing the framework of the deep learning training network model; learning features from locking wire images in the pipe fitting locking wire training set; ultimately generating model weights that incorporate locking wire features; performing locking wire segmentation prediction in a real-world environment and initially determining whether a locking wire exists at the pipe fitting; further processing and judging the segmented image to arrive at the final conclusion regarding the presence of a locking wire at the pipe fitting; S3, locking wire locking direction discrimination based on segmented images: cropping individual locking wire images from the original image; extracting the twisted feature points of the locking wire from the main locking wire segment; fitting the locking wire direction and arriving at the final conclusion: the locking direction is correct or incorrect; determining whether the circumferential range of the locking wire meets the assembly process requirements based on the hole position n1 and the direction of the locking wire feature points. This existing patent application suffers from insufficient practicality and fails to achieve lightweight deployment.

[0006] How to achieve automatic visual error prevention detection of engine locking screws has become a technical problem that needs to be solved. Summary of the Invention

[0007] The purpose of this invention is to overcome the defects of the prior art by providing a visual error-proofing detection method and electronic device for engine locking wires.

[0008] The objective of this invention can be achieved through the following technical solutions:

[0009] According to one aspect of the present invention, a visual error-proofing detection method for engine locking screws is provided, the method comprising:

[0010] Step 1: Perform illumination invariance and perspective correction processing on the acquired engine locking wire image to achieve multi-view image registration;

[0011] Step 2: Perform feature point detection and caliper fitting on the registered image to determine the two ends of the locking wire;

[0012] Step 3: Determine the installation direction of the locking screw based on the two endpoints of Step 2.

[0013] Preferably, the illumination invariance processing includes sequential grayscale conversion, global average brightness calculation, sub-block division and local brightness calculation, illumination difference calculation, difference matrix adjustment, illumination compensation and pixel value limitation.

[0014] More preferably, the illumination invariance treatment specifically includes:

[0015] Grayscale conversion: Converts the input motor locking wire image into a grayscale image. gray ;

[0016] Calculate grayscale image I gray The global average brightness μ is used as the benchmark for illumination compensation;

[0017] Grayscale image I gray Divide the data into several sub-blocks of a certain size and calculate the average brightness of each sub-block;

[0018] Calculate the difference between the brightness of each sub-block and the global average brightness to obtain the illumination difference matrix D;

[0019] Adjust the illumination difference matrix D to match the original image I. gray With the same dimensions, the adjusted difference matrix D is obtained. resized ;

[0020] For grayscale image I gray Illumination compensation is performed to obtain the corrected image I. corrected ;

[0021] For the corrected image I corrected Apply pixel constraints to make its size the same as the input image.

[0022] Preferably, the perspective correction process transforms the projection of the image from different viewpoints to the projection from the same viewpoint. After the transformation, the actual coordinates of the image are homogeneously normalized to restore the coordinates from homogeneous space to Euclidean space.

[0023] Preferably, the perspective correction process further includes matching feature point pairs using a feature point matching algorithm based on geometric constraints, wherein the feature point matching error formula is:

[0024]

[0025] Where, p i and p i ' represents the matched feature point pairs, H is the perspective transformation matrix, N is the number of matched feature point pairs, and H is the perspective transformation matrix.

[0026] Preferably, the feature point detection and caliper fitting process includes:

[0027] Adaptive binarization: Converts an input grayscale image into a binary image;

[0028] Edge detection: Extracting edges from a binary image to obtain an edge contour map;

[0029] Caliper placement and edge sampling: Multiple equally spaced caliper segments are arranged horizontally on the region of interest. Zero-crossing edge detection is performed on each caliper based on the edge contour map to extract multiple edge point sets.

[0030] Fitting edge points to a straight line: Use the least squares method to obtain the fitted straight line from the extracted set of edge points.

[0031] More preferably, determining the two ends of the locking wire specifically involves: projecting the edge contour along the fitted straight line direction, and extracting the two points with the farthest projection distance as the two ends of the locking wire.

[0032] More preferably, determining the two ends of the locking wire specifically involves: directly obtaining the first and last intersection points of the fitted straight line and the edge contour as the two ends of the locking wire.

[0033] Preferably, step 3 includes: comparing the direction vector formed by the two ends of the locking screw with a given reference direction; if the angle between the two is less than 90 degrees, the direction of the engine locking screw is determined to be correct; otherwise, it is determined to be incorrect.

[0034] According to another aspect of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described thereon.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1) This invention forms a complete visual error prevention detection method through multi-view image registration, feature point detection and caliper fitting, and locking screw installation direction determination. Compared with the existing technical solutions, this invention can effectively cope with the complex and ever-changing environmental conditions during engine assembly, significantly improve the accuracy and efficiency of locking screw direction detection, and provide strong technical support for improving engine assembly quality.

[0037] 2) The multi-view image registration technology of the present invention includes illumination invariance and perspective correction processing, which realizes image registration under different angles, translation transformations and scaling scales. This enables accurate alignment of images under different viewpoints in the complex and ever-changing engine assembly environment, providing high-quality input data for subsequent locking wire detection and solving the limitation of traditional difference detection algorithms that can only process images under fixed angles and the same illumination conditions.

[0038] 3) This invention performs edge extraction on the image to obtain an edge contour map, and arranges multiple caliper line segments to perform edge zero-crossing detection, extracts multiple edge point sets, and uses the least squares method to obtain a fitted straight line from the edge point sets. This can more accurately find the reference direction feature points of the locking wire, and improve the accuracy and robustness of locking wire direction detection.

[0039] 4) This invention achieves automatic determination of the installation direction of the locking screw through geometric analysis and angle estimation, solves the problem of detecting the difference in the relative position of the locking screw, and can automatically determine whether the installation direction of the locking screw is correct, which significantly improves the detection efficiency and accuracy. Attached Figure Description

[0040] Figure 1 This is a schematic flowchart of the detection method in this invention;

[0041] Figure 2 This is a schematic diagram of the locking screw;

[0042] Figure 3 This is a schematic diagram illustrating the principle of perspective transformation.

[0043] Figure 4 A schematic diagram of preprocessing for illumination invariance output results;

[0044] Figure 5 This is a schematic diagram of the adaptive binarization result;

[0045] Figure 6 This is a schematic diagram of the locking wire boundary point;

[0046] Figure 7 This is a schematic diagram of the locking wire direction vector;

[0047] Figure 8 A diagram showing the correct orientation of the locking screws;

[0048] Figure 9 A diagram illustrating an incorrect locking screw direction;

[0049] Figure 10 This is a schematic diagram of the detection device in this invention;

[0050] Among them, 1: bracket, 2: camera, 3: product, 4: workbench. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0052] This invention utilizes basic image processing methods, which are lightweight and deployed on equipment, to solve the detection problem under different viewing angles and lighting conditions. Combined with an image comparison system, it is applied to the production line to determine whether there are assembly errors.

[0053] This embodiment relates to a visual error-proofing detection method for engine locking wires, focusing primarily on the locking wires used in engine assembly. Firstly, addressing the limitation of traditional image difference detection algorithms, which can only identify differences in images taken from fixed angles and under similar ambient lighting conditions, this invention investigates image registration technology. This enables spatial alignment of images of the same object from different perspectives, achieving image registration under different angles, translational transformations, and scaling scales. Experiments are conducted to verify the proposed image feature point detection model and feature point matching algorithm. Suitable image feature points are selected for matching and image correction based on the requirements of this invention. The installation direction is determined based on the locking wires within the positioning area. An edge extraction algorithm based on image features and an algorithm based on calipers for extracting image feature points are analyzed and compared. Combining image binarization, measurement models, and angle judgment, geometric analysis and installation angle estimation of the target area are completed.

[0054] The flowchart of the detection method is attached. Figure 1 The specific details are as follows:

[0055] Step 1: Multi-view image registration.

[0056] Step 2: Feature point detection and caliper fitting.

[0057] Step 3: Determine the installation direction of the locking screw.

[0058] A diagram of the engine locking screw is attached. Figure 2 .

[0059] Step 1, Multi-view image registration: This invention solves the limitation of traditional difference detection algorithms that can only handle images with fixed angles and the same lighting conditions by introducing the principles of perspective correction and illumination invariance, and realizes image registration under different angles, translation transformations and scaling scales.

[0060] Illumination invariance refers to the ability of image feature points to remain consistent under different lighting conditions. This invention achieves illumination invariance through the following method: the steps of the non-uniform illumination compensation algorithm include grayscale conversion, global average brightness calculation, sub-block partitioning and local brightness calculation, illumination difference calculation, difference matrix adjustment, illumination compensation, and pixel value constraint.

[0061] The specific algorithm steps are as follows:

[0062] Image preprocessing: The input image I is an M×N matrix representing the pixel values ​​of the image. First, it is converted into a grayscale image I. gray .

[0063] Grayscale conversion formula:

[0064] I gray (i,j)=0.299·R(i,j)+0.587·G(i,j)+0.114·B(i,j)

[0065] Where R(i,j), G(i,j), and B(i,j) are the red, green, and blue channel values ​​of the color image at position (i,j), respectively, and I gray (i,j) is the gray value of the grayscale image at position (i,j).

[0066] Global average brightness calculation: Calculate the grayscale image I gray The global average brightness μ is used as the benchmark for illumination compensation.

[0067] Global average brightness formula:

[0068]

[0069] Where M and N are the number of rows and columns of the image, respectively, and μ is the global average brightness.

[0070] Sub-block partitioning and local brightness calculation: Sub-block partitioning of grayscale image I gray Divide the data into sub-blocks of size block_size × block_size, and calculate the average brightness of each sub-block.

[0071] Sub-block average brightness formula:

[0072] For the sub-block in row k and column l, its average brightness B(k,l) is calculated as follows:

[0073]

[0074] Where k and l are the row index and column index of the sub-block, respectively, and B(k,l) is the average brightness of the sub-block (k,l).

[0075] Illumination difference calculation: Calculate the difference between the brightness of each sub-block and the global average brightness to obtain the illumination difference matrix D.

[0076] Formula for difference in illumination:

[0077] D(k,l)=B(k,l)-μ

[0078] Where D(k,l) is the illumination difference value of sub-block (k,l).

[0079] Difference matrix adjustment: Adjust the illumination difference matrix D to match the original image I. gray With the same dimensions, the adjusted difference matrix D is obtained. resized .

[0080] Difference matrix adjustment formula: Use bicubic interpolation (cv2.INTER_CUBIC) to adjust D to D0. resized :

[0081] D resized =resize(D,(M,N),interpolation=cv2.INTERCUBIC)

[0082] Where: D resized It is the adjusted difference matrix with a size of M×N.

[0083] Illumination compensation: To balance the degree of shadow compensation and illumination attenuation, a compensation factor is set based on the original illumination invariance algorithm. According to the compensation factor α (i.e., compensation_factor), the grayscale image I... gray Illumination compensation is performed to obtain the corrected image I. corrected .

[0084] Light compensation formula:

[0085] I corrected (i, j) = I gray (i,j)-α·D resized (i, j)

[0086] Where α is the compensation factor, controlling the intensity of illumination compensation, ranging from 0 to 1; I corrected It is the gray value of the corrected image at position (i,j).

[0087] Pixel value limit:

[0088] To ensure that pixel values ​​remain within the valid range (0 to 255), the following formula is used for limitation:

[0089] I corrected (i, j) = clip(i corrected (i, j), 0, 255)

[0090] Where clip(a,b) restricts the image to the interval [a,b], and finally outputs the corrected image I. corrected， Its size is the same as the input image I. A schematic diagram of the preprocessing of the illumination-invariant output results is attached. Figure 4 As shown.

[0091] Multi-view image registration technology overcomes the limitations of traditional difference detection algorithms, which can only handle images at fixed angles and under the same lighting conditions. By employing multi-view image registration, image registration under different angles, translation transformations, and scaling scales is achieved. Optimized feature point detection and matching algorithms significantly improve the flexibility and applicability of image processing. This enables accurate alignment of images from different perspectives in the complex and ever-changing engine assembly environment, providing high-quality input data for subsequent locking wire inspection.

[0092] Perspective correction transforms the projection of an image from different viewpoints to the projection from the same viewpoint, thus eliminating the effects of perspective differences. Perspective transformation can be represented by a 3×3 transformation matrix H, mapping the image from the original coordinate system (x, y) to the target coordinate system (x', y'). A diagram illustrating the principle of perspective transformation is attached. Figure 3 .

[0093] Perspective transformation formula:

[0094] Where H is the perspective transformation matrix, containing 8 degrees of freedom (h 33 Typically set to 1); (x,y) are the coordinates of a point in the original image; (x',y') are the coordinates of a point in the transformed image.

[0095] After the transformation, the actual coordinates of the image need to be homogeneously normalized to restore them from homogeneous space to Euclidean space.

[0096]

[0097] Where (x”,y”) are the coordinates of a point in Euclidean space, and ω' is the scaling factor for homogeneous coordinates.

[0098] Calculating the perspective transformation matrix H requires knowing at least four pairs of non-collinear points, corresponding to the point (x, y) before the transformation and the point (x′, y′) after the transformation. The following are the steps to solve for the perspective transformation matrix:

[0099] Establish the perspective transformation equations: For each pair of points, the perspective transformation must satisfy the following equations:

[0100]

[0101] To remove the nonlinear relationships in the denominator, the equations are cross-multiplied, resulting in two linear equations:

[0102] x′(h 31 x+h 32 y+h 33 ) = h 11 x+h 12 y+h 13

[0103] y′(h 31 x+h 32 y+h 33 ) = h 21 x+h 22 y+h 23

[0104] Constructing matrix equations: The system of linear equations for all points is rearranged to form a matrix expression:

[0105] A·h=0

[0106] Where A is the coefficient matrix, and h is the vector form of the parameters of the perspective transformation matrix to be determined.

[0107] Solving for H: The matrix equation is solved using singular value decomposition to obtain matrix H. To ensure uniqueness, H is usually normalized so that h... 33 =1.

[0108] Perspective transformations possess important geometric properties. First, they preserve linearity, mapping one straight line in two-dimensional space to another—a property stemming from fundamental properties of projective geometry. Second, in perspective transformations, seemingly parallel lines may converge at a vanishing point, creating a typical perspective effect. Furthermore, perspective transformations can cause non-linear changes in the shape of image regions; for example, a quadrilateral can be deformed into another quadrilateral while maintaining its topological integrity.

[0109] To improve the accuracy of locking wire direction detection, this invention proposes an improved feature point matching algorithm.

[0110] Input: Grayscale image I gray .

[0111] Output: Matched feature point pairs.

[0112] The improved feature point matching process includes matching feature point pairs using the RANSAC algorithm, which is based on geometric constraints.

[0113] Feature point matching error formula:

[0114]

[0115] Where: p i and p i ' represents the matched feature point pair, H is the perspective transformation matrix, and N is the number of matched feature point pairs.

[0116] In step 2, this invention proposes feature point detection and caliper fitting. This method takes a grayscale image as input and combines adaptive binarization, edge extraction, and caliper fitting to extract the two endpoints of the locking wire and calculate its direction vector, which serves as the basis for determining the direction of the locking wire.

[0117] The process from acquiring the detection image to determining the direction of the locking wire includes the following steps:

[0118] Step 201, Adaptive Binarization;

[0119] Step 202, edge extraction and caliper fitting;

[0120] Step 203, endpoint determination.

[0121] Step 201, Adaptive Binarization: To enhance image contrast and suppress interference under different lighting conditions, an adaptive thresholding method is used to binarize the input grayscale image I. gray (x,y) is converted into a binary image I. binary (x,y), as shown in the appendix Figure 5 As shown.

[0122] The binarization formula is as follows:

[0123]

[0124] The local threshold T(x,y) is calculated by the Gaussian weighted average of the image patches:

[0125]

[0126] Where N(x,y) is the neighborhood centered at (x,y); w(i,j) is the weighting coefficient; and C is the empirical offset (used to enhance the edge).

[0127] Step 202, the process of edge extraction and caliper fitting includes:

[0128] Step 1: Canny Edge Detection

[0129] For binary image I binary Edge extraction is performed using the Canny algorithm to obtain the edge contour map:

[0130] E = Canny(I) binalry ,low_thresh,hih,thresh)

[0131] Here, low_thresh and high_thresh represent the minimum threshold and the maximum threshold, respectively.

[0132] Step 2: Caliper placement and edge sampling

[0133] Multiple equally spaced caliper segments are arranged along a general direction on the region of interest (ROI). Zero-crossing edge detection is performed on each caliper to extract multiple edge point sets.

[0134] M calipers are arranged horizontally, with the center point being:

[0135] x i =x0+i·Δ,y i =y fixed

[0136] On each caliper, a pair of upper and lower edge points were measured, and the point with the strongest edge response was retained as the locking wire boundary point. The results are attached. Figure 6 As shown.

[0137] Step 3: Fit the edge points as straight lines

[0138] The extracted edge point set {(x i ,y i Use the least squares method to fit a straight line:

[0139] y = kx + b

[0140] Wherein, the fitting slope k and intercept b are obtained by the following formulas.

[0141]

[0142] Where N is the number of edge points.

[0143] Step 203, Feature point (endpoint) detection

[0144] Based on the fitted straight line y = kx + b, the intersection point with the edge contour map E is taken as the two endpoints of the locking wire:

[0145] P1 = (x1, y1), P2 = (x2, y2)

[0146] Where (x1,y1) and (x2,y2) are the coordinates of the two intersection points, respectively.

[0147] The endpoint extraction strategy is as follows: project the edge contour along the direction of the fitted line and extract the two points with the farthest projection distance; or directly obtain the first and last intersection points of the fitted line and the edge region.

[0148] The direction vector between the two endpoints is:

[0149]

[0150] This vector is the locking wire direction vector, and the result is shown in the appendix. Figure 7 As shown.

[0151] Step 3, Determining the installation direction of the locking screw:

[0152] Determining the installation angle of the locking screw requires the direction vector formed by its two endpoints (x1, y1) and (x2, y2). With the given reference direction Compare. If and If the included angle is less than 90 degrees or the directions are opposite, it is judged as "OK" (correct); otherwise, it is "NG" (incorrect).

[0153] (1) Calculate the direction vector

[0154] The coordinates of the two ends of the locking screw are (x1, y1) and (x2, y2) respectively, and its direction vector is:

[0155]

[0156] (2) Calculate the reference direction

[0157] Reference direction It is a pre-defined unit vector used to define the standard for the "correct" direction.

[0158] (3) Calculate the included angle

[0159] calculate and The included angle:

[0160]

[0161] in:

[0162]

[0163] Judgment rules:

[0164] If cosθ>0, it means the directions are basically the same, and it can be judged as "OK".

[0165] Otherwise, it means the direction is reversed, and it is judged as "NG".

[0166] The locking screw installation orientation determination system utilizes geometric analysis and angle estimation to automatically determine the installation orientation of the locking screws. This solves the problem of detecting relative positional differences in the locking screws and automatically determines whether the installation orientation is correct. By combining image binarization, measurement models, and angle estimation, the system performs geometric analysis and installation angle estimation of the target area, significantly improving detection efficiency and accuracy.

[0167] Based on the directional angle θ, the detection results are plotted on the image, with cosθ>0 used as the condition for correct installation.

[0168] Input: Direction angle θ, endpoints (x1, y1) and (x2, y2).

[0169] Output: An image with labeled detection results.

[0170] If cosθ > 0, draw a straight line connecting (x1, y1) and (x2, y2), and mark "OK" on the graph. The result is shown in the attached figure. Figure 8 As shown.

[0171] Otherwise, draw a straight line connecting (x1, y1) and (x2, y2), and mark "NG" on the graph. The result is shown in the attached figure. Figure 9 As shown.

[0172] This detection method can be applied to detection equipment, such as... Figure 10 As shown, camera 2 is mounted on bracket 1, and product 3 is mounted on liftable and rotatable worktable 4 for visual inspection. By rotating the inspection equipment, camera 2 takes pictures of product 3 to determine whether the engine locking screw is installed in place.

[0173] Inspection Equipment Application - Image Comparison System: The image comparison system uses a high-pixel camera to photograph the surface of the engine. The worktable 4 can move up and down and rotate via a moving device, which can work with the camera 2 to detect the fixing status of the locking screws on the outer surface of the engine product 3.

[0174] (1) System Composition

[0175] ① Camera 2: Used to capture high-definition images of the engine surface, ensuring that details are clearly visible.

[0176] ② Worktable 4: The worktable can move up and down and rotate, making it convenient for camera 2 to take pictures from different angles.

[0177] ③ Locking screw detection module: used to detect the fixing status of the locking screws on the outer surface of the engine.

[0178] (2) Workflow

[0179] ①Location and shooting:

[0180] Product 3 (referring to the engine) is placed on the workbench, and the system automatically positions it.

[0181] A high-resolution camera captures images of the engine surface from multiple angles, ensuring coverage of all critical areas.

[0182] ②Image processing:

[0183] The captured images are transmitted to the image processing system for preprocessing.

[0184] Extract the locking wire area and perform feature analysis.

[0185] ③Result Output:

[0186] The system generates a detection report, marking the locations of anomalies.

[0187] The operator will then proceed with further processing based on the report.

[0188] (3) Technical parameters

[0189] ① Camera resolution: 6 megapixels.

[0190] ② Lens focal length: 8mm

[0191] ③ Workbench parameters: lifting height 1200mm, rotation angle 360°, flip angle 180°, moving speed 4m / min.

[0192] The electronic device of this invention includes a central processing unit (CPU), which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) or loaded from a storage unit into random access memory (RAM). The RAM may also store various programs and data required for device operation. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0193] Multiple components in the device are connected to the I / O interface, including: input units such as keyboards and mice; output units such as various types of displays and speakers; storage units such as disks and optical discs; and communication units such as network interface cards (NICs), modems, and wireless transceivers. The communication unit allows the device to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0194] The processing unit performs the various methods and processes described above. For example, in some embodiments, the methods may be implemented as computer software programs tangibly contained in a machine-readable medium, such as a storage unit. In some embodiments, part or all of the computer program may be loaded and / or installed on the device via ROM and / or a communication unit. When the computer program is loaded into RAM and executed by the CPU, one or more steps of the methods described above may be performed. Alternatively, in other embodiments, the CPU may be configured to execute the methods by any other suitable means (e.g., by means of firmware).

[0195] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0196] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0197] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0198] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A visual error-proofing detection method for engine locking screws, characterized in that, The method includes: Step 1: Perform illumination invariance and perspective correction processing on the acquired engine locking wire image to achieve multi-view image registration; Step 2: Perform feature point detection and caliper fitting on the registered image to determine the two ends of the locking wire; Step 3: Determine the installation direction of the locking screw based on the two endpoints of Step 2; The process of feature point detection and caliper fitting includes: Adaptive binarization: Converts an input grayscale image into a binary image; Edge detection: Extracting edges from a binary image to obtain an edge contour map; Caliper placement and edge sampling: Multiple equally spaced caliper segments are arranged horizontally on the region of interest. Zero-crossing edge detection is performed on each caliper based on the edge contour map to extract multiple edge point sets. Fitting edge points to a straight line: Use the least squares method to obtain the fitted straight line from the extracted set of edge points; The specific method for determining the two endpoints of the locking wire is as follows: directly obtain the first and last intersection points of the fitted straight line and the edge contour as the two endpoints of the locking wire.

2. The visual error-proofing detection method for engine locking screws according to claim 1, characterized in that, The illumination invariance processing includes sequential grayscale conversion, global average brightness calculation, sub-block division and local brightness calculation, illumination difference calculation, difference matrix adjustment, illumination compensation and pixel value limitation.

3. The visual error-proofing detection method for engine locking screws according to claim 2, characterized in that, The aforementioned illumination invariance processing specifically includes: Grayscale conversion: Converts the input motor locking wire image into a grayscale image. I gray ; Calculate grayscale image I gray global average brightness µ This serves as the benchmark for illumination compensation. grayscale image I gray Divide the data into several sub-blocks of a certain size, and calculate the average brightness of each sub-block; The difference between the brightness of each sub-block and the global average brightness is calculated to obtain the illumination difference matrix. D ; Illumination difference matrix D Adjust to match the original image I gray The same size yields the adjusted difference matrix. D resized ; For grayscale images I gray Perform illumination compensation to obtain the corrected image. I corrected ; For the corrected image I corrected Apply pixel constraints to make its size the same as the input image.

4. The visual error-proofing detection method for engine locking screws according to claim 1, characterized in that, The perspective correction process transforms the projection of an image from different viewpoints to the projection from the same viewpoint. After the transformation, the actual coordinates of the image are homogeneously normalized to restore the coordinates from homogeneous space to Euclidean space.

5. The visual error-proofing detection method for engine locking screws according to claim 1, characterized in that, The perspective correction process also includes matching feature point pairs using a feature point matching algorithm based on geometric constraints, with the following feature point matching error formula: in, p i and p i ’ These are the matched feature point pairs. H This is the perspective transformation matrix. N The number of feature point pairs matched. H This is the perspective transformation matrix.

6. The visual error-proofing detection method for engine locking screws according to claim 1, characterized in that, The specific method for determining the two ends of the locking wire is as follows: project the edge contour along the fitted straight line direction, and extract the two points with the farthest projection distance as the two ends of the locking wire.

7. The visual error-proofing detection method for engine locking screws according to claim 1, characterized in that, Step 3 includes: comparing the direction vector formed by the two ends of the locking screw with a given reference direction. If the angle between the two is less than 90 degrees, the direction of the engine locking screw is determined to be correct; otherwise, it is determined to be incorrect.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

Images