Large-format object unsupervised defect detection method and system based on two-dimensional galvanometer

Abstract

The invention belongs to the technical field of defect detection, and discloses a large-format object unsupervised defect detection method and system based on a two-dimensional galvanometer. Comprising an image acquisition module used for acquiring an image of a to-be-detected object; the two-dimensional galvanometer system is used for quickly projecting the light spot to a specified position on the surface of the to-be-measured object; and the processor is internally provided with a dynamic weight attention module and is used for processing the image acquired by the image acquisition module, planning a scanning path of the two-dimensional galvanometer system and identifying defects. According to the method, a brand new dynamic weight attention module is provided, spatial feature information under different receptive fields can be extracted through multi-scale expansion convolution on the premise of keeping calculation efficiency, and adaptive fusion is performed on the features in combination with a global weighting mechanism. The module not only effectively enhances the perception ability of the model to complex textures and structure anomalies, but also significantly improves the recognition precision and positioning accuracy of surface defects of large-format objects, and has the characteristics of high efficiency, robustness and expandability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fault detection, and in particular to a large-format object unsupervised defect detection method and system based on a two-dimensional galvanometer. BACKGROUND

[0002] With the acceleration of modern industry towards intelligent and digital transformation, the surface state detection of large industrial equipment (such as wind turbine blades, oil storage tanks, pressure vessels, etc.) has become a key link to ensure safety in production and prevent structural failure. Traditional detection methods mainly rely on manual visual inspection and handheld detection instruments. Although such methods have the characteristics of simple operation and low cost, they have inherent defects such as low detection efficiency, strong subjectivity, and high safety risks, making it difficult to meet the needs of modern industry for high-precision and automated detection.

[0003] The use of mobile platforms such as robots or drones has expanded the application boundaries of machine vision in large equipment detection, but in the context of high-precision surface defect detection, such devices still face the contradiction between imaging quality and detection efficiency. For example, in the context of ship, aircraft, wind power and other large equipment manufacturing and maintenance, when collecting images of objects within the field of view, a large focal length lens is often used to capture the object surface at a small field of view, resulting in a higher resolution of surface details, but this inevitably leads to low collection efficiency. If a wide-angle camera with a large field of view is used directly for image collection, although the efficiency is improved, the ratio of defect size to field of view size becomes more sensitive.

[0004] Deep learning-based anomaly detection methods have made significant progress in the industry. With the continuous improvement and optimization of modern production and manufacturing processes, it becomes difficult to obtain defect data in real industrial scenarios, and the collected data often comes with an unbalanced long-tail data problem, which can make the model have low detection rate or miss detection for defects with less data, and be helpless for undefined defect types that have never been encountered due to ineffective generalization. Unsupervised anomaly detection models use defect-free samples for training, model the positive samples, and then use reconstruction error to evaluate the location of anomalies when encountering negative samples with different distributions.

[0005] Unsupervised anomaly detection algorithms based on deep learning automatically learn the latent feature representation of data through neural networks, and identify abnormal points deviating from the distribution of normal samples by utilizing the distribution characteristics of normal samples. Typical structures include encoder-decoder frameworks (such as autoencoders AE or variational autoencoders VAE) and generative adversarial networks (GAN). The core principle is to compress the input data into low-dimensional latent features through the encoder, and then reconstruct the original data through the decoder. Normal samples can be accurately reconstructed because they conform to the distribution of training data, while abnormal samples will result in higher reconstruction error due to feature deviation, and thus be detected. Another approach is based on generative adversarial networks, which use a generator to learn the distribution of normal data, and a discriminator to distinguish between real and generated data. Abnormal samples are identified because they are difficult to simulate by the generator.

[0006] However, the existing technology still has several problems and shortcomings. First, the performance of the model is highly dependent on the purity of the training data. If abnormal points are mixed in the normal samples, the label is contaminated, which will cause the model to learn the wrong distribution characteristics, significantly reducing the detection accuracy. Second, deep learning models usually require a large amount of training data, but in actual industrial scenarios, normal samples may be scarce or difficult to obtain, leading to model overfitting or insufficient generalization ability. In addition, the feature extraction and calculation cost of high-dimensional complex data is high, which limits the efficiency of real-time detection.

[0007] These problems can be summarized into three aspects: first, the collection efficiency and clarity can never be well balanced. When the field of view increases, the detection time often increases geometrically. Second, in the maintenance of large equipment such as ships, aircraft, and wind power, it is difficult to collect images of objects in the field of view, resulting in a lack of defect sample data and an imbalance in the number of defect samples of various categories. Third, a large number of studies focus on geometric anomaly detection on the surface of objects (such as shape distortion, missing, misplacement), and there is a lack of attention to texture anomalies (such as scratches, stains). SUMMARY

[0008] In order to solve the problems that the acquisition efficiency and the definition cannot be balanced in the prior art, when the field of view is increased, the detection time is often increased in geometry, and in the working scene of ship, airplane, wind power and other large equipment manufacturing and maintenance, it is difficult to collect images of objects in the field of view, resulting in few defect sample data and unbalanced number of defect samples in each category, and a large number of studies are based on geometric anomaly detection of object surface, and the attention to texture anomaly is obviously insufficient. The present application provides a large-format object unsupervised defect detection method and system based on a two-dimensional galvanometer, which combines the current abnormal detection model in the industrial detection field and the two-dimensional scanning galvanometer originally used for laser indication or marking, and innovatively provides a system suitable for large-format object surface image acquisition and abnormal defect detection, which is suitable for surface detection of large industrial equipment such as ship outer surface, wind power blade surface and high-speed train body, so that defects can be found more efficiently, conveniently and accurately during the construction or operation of heavy industry, and the defect position can be accurately positioned, so as to solve the defect problem, increase the manufacturing efficiency and ensure the safety during the operation.

[0009] The present application is realized by the following technical solutions: comprising

[0010] An image acquisition module is configured to acquire images of the object to be measured.

[0011] A two-dimensional galvanometer system is configured to project a light spot quickly to a specified position on the surface of the object to be measured.

[0012] A processor is configured to process the images acquired by the image acquisition module, plan a scanning path of the two-dimensional galvanometer system, and identify defects.

[0013] As a further optimization, the two-dimensional galvanometer system is composed of two galvanometers rotating around X-axis and Y-axis, respectively, and the plane composed of X-axis and Y-axis is parallel to the focusing plane of the camera; the two galvanometers complete the pitching and nodding actions and the horizontal shaking actions around X-axis and Y-axis, respectively; the two-dimensional galvanometer system further comprises a galvanometer controller system for controlling the rotation of the two galvanometers; the galvanometer controller system comprises a galvanometer motion control card, an adapter card and a controller terminal board.

[0014] As a further optimization, the image acquisition module comprises a wide-angle camera and a zoom camera; the wide-angle camera is arranged in front of the object to be measured and is configured to acquire images of the entire field of view and implement target segmentation and detection algorithms to obtain image coordinates of the target and the defect position in the entire large field of view; the zoom camera is arranged on the side of the two-dimensional galvanometer system and is configured to acquire images transmitted by the galvanometer in the two-dimensional galvanometer system.

[0015] The present application further provides a detection method suitable for the two-dimensional galvanometer-based large-format object unsupervised defect detection system, comprising the following steps:

[0016] Step 1, constructing the mapping relationship between the mirror rotation angle and the mirror coordinate system;

[0017] Step 2, constructing the mapping relationship between the mirror coordinate system and the world coordinate system;

[0018] Step 3, the wide-angle camera and the zoom camera shoot the image of the object to be measured, the processor performs path planning and sends the planning result to the mirror motion control card, and the two-dimensional mirror system scans through the mirror motion control card, and the wide-angle camera and the zoom camera collect images;

[0019] Step 4, using a teacher-student network of double-space collaborative modeling to detect defects and abnormalities in the images collected in step 3 to obtain the detection results.

[0020] As a further optimization, the formula for constructing the mapping relationship between the mirror rotation angle and the mirror coordinate system in step 1 is as follows:

[0021] ;

[0022] ;

[0023] ;

[0024] In the formula, point (x, y, z) is a point in the mirror coordinate system; is the Y-axis mirror rotation angle; is the X-axis mirror rotation angle.

[0025] As a further optimization, the specific steps of step 2 are as follows:

[0026] Step 2.1, the mirror motion control card projects a set of point sets to the projection plane and records their coordinates in the mirror coordinate system, represents a three-dimensional Euclidean space with the wide-angle camera coordinate system as the reference; then use the spot extraction algorithm to extract the center coordinates of the point set to obtain the pixel coordinate point set of the pixel coordinate system, represents a three-dimensional Euclidean space with the pixel coordinate system as the reference; and forms a point set pair with the point set ; the homography matrix from the pixel coordinate system to the mirror coordinate system is obtained by the following formula , the formula is as follows:

[0027] ;

[0028] ;

[0029] ;

[0030] ;

[0031] wherein H1 is the mapping from the mirror angle coordinate to the pixel coordinate; H2 is the mapping from the mirror coordinate to the mirror angle coordinate; b1 represents the number of elements in the point set and the point set .

[0032] Step 2.2, using a circular calibration board with the same size as the model composed of the point set in step 2.1, selecting the lower left corner point of the circular calibration board as the origin of the world coordinate system, extracting the pixel coordinate system point set composed of the feature points on the circular calibration board by shooting with the wide-angle camera , and at the same time extracting the corresponding feature point coordinate point set in the world coordinate system by projecting the numerical model , , wherein H3 represents the three-dimensional Euclidean space with the world coordinate system as the reference; the homography matrix from the world coordinate system to the pixel coordinate system can be obtained after optimization :

[0033] ;

[0034] wherein b2 represents the number of elements in the point set and the point set .

[0035] Step 2.3, according to the homography matrix from the pixel coordinate system to the mirror coordinate system obtained in step 2.1 and the homography matrix from the world coordinate system to the pixel coordinate system obtained in step 2.2, the mapping transformation relationship from the world coordinate system to the mirror coordinate system is calculated, and the formula is as follows:

[0036] ;

[0037] wherein x g , y g represent the coordinates in the mirror coordinate system; x c , y c represent the coordinates in the world coordinate system.

[0038] As a further optimization, the specific steps of step 3 are as follows:

[0039] Step 3.1, adjusting the wide-angle camera view position to cover the main contour of the object to be measured, calibrating the wide-angle camera with a chessboard calibration board, and obtaining the wide-angle camera intrinsic matrix K and distortion coefficient d;

[0040] Step 3.2, collecting N calibration points in the working area of the wide-angle camera to cover the entire working area, and establishing the mapping relationship between the wide-angle camera pixel coordinate and the mirror deflection angle according to step 2.1;

[0041] Step 3.3, using the deep estimation segmentation network DIS to mask the segmentation network to extract the region of the object to be tested after the post-processing process;

[0042] Step 3.4, according to the field of view size of the zoom camera, the scanning step is calculated, and the formula is as follows:

[0043] ;

[0044] ;

[0045] In the formula, represents the horizontal scanning step; represents the vertical scanning step; is the field of view overlap rate; represents the field of view width of the zoom camera; represents the field of view height of the zoom camera;

[0046] Step 3.5, according to the scanning step obtained in step 3.4 and the object region extracted in step 3.3, the object region is calculated to obtain the region boundary box, and then the region boundary box is divided into a plurality of scanning windows and the center coordinates of the scanning windows are recorded according to the scanning step in the horizontal and vertical directions, and the scanning path is generated by connecting the center coordinates.

[0047] Step 3.6, according to the scanning path obtained in step 3.5, the scanning path pixel sequence P (P1, P2, P3,... P n ) of the two-dimensional galvanometer system is obtained, and finally the deflection angle data of the two-dimensional galvanometer system is obtained according to the calibration corresponding relationship obtained in step 2, and the deflection angle sequence Q (Q1, Q2, Q3,..., Q n ) of the galvanometer is formed, and is transmitted to the two-dimensional galvanometer system;

[0048] Step 3.7, the X-axis galvanometer lens and the Y-axis galvanometer lens of the two-dimensional galvanometer system are controlled to deflect by the corresponding angle according to the galvanometer scanning angle sequence Q obtained in step 3.6; at the same time, according to the current focal length parameter of the zoom camera, the exposure time and the imaging scale are automatically adjusted, and the collected image is input to the processor.

[0049] As a further preferred, the specific steps of step 4 are as follows:

[0050] Step 4.1, constructing a teacher-student network of dual-space collaborative modeling, including a teacher network, a student network and a data space modeling module; the teacher network adopts a frozen parameter convolutional neural network model Efficient Net, the structure of which includes multiple layers of convolution, batch normalization and nonlinear activation layer; the student network has the same structure as the teacher network, but the parameters are dynamically updated, and a DWA module is added between the middle feature layer and the high-level feature fusion layer of the student network;

[0051] Step 4.2, the teacher network extracts features from the images collected in step 3 to obtain the features of the teacher network , the formula is as follows:

[0052] ;

[0053] Among them, represents the feature extraction function of the teacher network; represents the sample image after the image collected in step 3 is preprocessed by the processor; is a frozen parameter set;

[0054] Step 4.3, the student network extracts features from the images collected in step 3 to obtain the features of the teacher network , the formula is as follows:

[0055] ;

[0056] ;

[0057] Among them, represents the feature extraction function of the student network; is the parameter set of the student network; Y is the intermediate feature output by the student network; is the Normalizing Flow transformation function; represents the output of the Normalizing Flow transformation function;

[0058] Step 4.4, constructing the loss function of the teacher-student network of dual-space collaborative modeling, obtaining the feature residual map of the teacher network and the student network, the formula is as follows:

[0059] ;

[0060] ;

[0061] ;

[0062] ;

[0063] Among them, represents the feature loss between the teacher network and the student network; represents the feature loss between the autoencoder and the student network; represents the data space distance loss; represents the output feature of the teacher network; represents the output feature of the corresponding channel of the student network; represents the output feature of the corresponding channel of the student network; represents the output feature of the student network after weighting by the DWA module;

[0064] Step 4.5, the data space modeling module synthesizes the feature loss between the teacher network and the student network obtained in step 4.4 , the feature loss between the autoencoder and the student network , and the data space distance loss ; generates an abnormal heat map and a final defect detection result.

[0065] As a further optimization, the method for obtaining the output feature of the student network after weighting by the DWA module in step 4.4 is:

[0066] Step 4.4.1, replace the sample image x after preprocessing of the image collected in step 3 by the sample image after weighting by the DWA module , to obtain the output feature of the student network after weighting by the DWA module , which is as follows:

[0067] ;

[0068] wherein, is the sample image after preprocessing of the image collected in step 3; is the sample image after weighting by the DWA module; is an activation function; is a weighted sum function; C1 and C2 are both small receptive fields; C3 is a large receptive field; w1, w2, w3, and w4 are all hyperparameters of the DWA module.

[0069] As a further optimization, the formula of the activation function is as follows , wherein represents the independent variable of the activation function.

[0070] Compared with the prior art, the present application has the following advantages:

[0071] 1.The application proposes a brand-new dynamic weighted attention (DWA) module that can extract spatial feature information under different receptive fields through multi-scale dilated convolution while maintaining computational efficiency, and adaptively fuse the features by combining a global weighting mechanism. This module not only effectively enhances the model's ability to perceive complex textures and structural abnormalities, but also significantly improves the recognition accuracy and positioning accuracy of large-area object surface defects, with the characteristics of high efficiency, robustness, and scalability.

[0072] 2.The application proposes a processing algorithm for optimizing the scanning trajectory of a galvanometer window, which generates a scanning path only in the target effective area based on the outer contour information of the measured object obtained by a wide-angle camera, through sliding window traversal and adaptive row-column planning strategy. This algorithm can significantly reduce the invalid scanning area, optimize the galvanometer motion trajectory, reduce system running time and energy consumption, while ensuring full coverage collection and high-precision positioning of complex surface shapes, improving overall detection efficiency and system stability.

[0073] 3.The application proposes a new unsupervised anomaly detection architecture of a teacher-student network based on dual-space collaborative modeling, which uses a teacher-student network and a normalization flow to jointly model the image space and data space of the data, thereby jointly improving detection efficiency and effectively avoiding the waste of data space constraints, significantly improving the sensitivity and generalization ability of anomaly detection.

[0074] 4.The application proposes a zoom camera field of view control mechanism based on focal length adaptive adjustment, which adjusts the scanning step and exposure parameters in real time according to the camera focal length during galvanometer scanning, thereby achieving adaptive balance between large field of view and high resolution. This mechanism ensures image acquisition quality under different spatial depths and surface curvatures, allowing the system to maintain uniform resolution and focus clarity on complex three-dimensional surfaces, effectively improving overall detection accuracy and stability.

[0075] 5.The application establishes a cooperative control mechanism for galvanometer scanning and defect recognition, realizing real-time feedback coupling of acquisition path and detection results. When the detection network identifies potential abnormal areas, the system can automatically call the galvanometer to perform local rescan and high-magnification review imaging, thereby significantly reducing the miss rate while ensuring detection efficiency. This cooperative mechanism forms a closed-loop feedback between the galvanometer scanning system and the depth detection model, improving the intelligent level and detection reliability in large-area detection tasks. BRIEF DESCRIPTION OF DRAWINGS

[0076] Figure 1 The figure is a schematic diagram of the detection system layout of the application.

[0077] Figure 2 The figure is a flowchart of the overall detection method of the application.

[0078] Figure 3 A mirror imaging system calibration model schematic diagram based on homography matrix in the detection method of the application.

[0079] Figure 4 A two-dimensional mirror window scanning motion path planning algorithm flow chart of the application.

[0080] Figure 5 A two-dimensional mirror window scanning path schematic diagram of the application.

[0081] Figure 6 A detection network architecture diagram of the application. DETAILED DESCRIPTION

[0082] The advantages and features of the application will be illustrated and explained by the following non-limiting description of preferred embodiments, which are given by way of example only, with reference to the accompanying drawings.

[0083] As Figure 1 shown, the application provides a large-format object unsupervised defect detection system based on two-dimensional mirror, which comprises an image acquisition module for acquiring images of the object to be measured; a two-dimensional mirror system for quickly projecting a light spot to a specified position on the surface of the object to be measured; and a processor, which can be a PC host, for processing the images acquired by the image acquisition module, planning the scanning path of the two-dimensional mirror system and identifying defects.

[0084] The two-dimensional mirror system is composed of two mirrors rotating around X-axis and Y-axis respectively, and the plane composed of X-axis and Y-axis is parallel to the plane to be scanned by the mirror, i.e. the camera focusing plane; the two mirrors complete the pitching nodding action and horizontal shaking action around X-axis and Y-axis respectively, for quickly projecting a light spot to a specified position on the surface of the object to be measured and expanding the scanning range. The two-dimensional mirror system further comprises a mirror controller system for controlling the rotation of the two mirrors; the mirror controller system comprises a mirror motion control card, an adapter card and a controller terminal board.

[0085] The image acquisition module comprises a wide-angle camera and a zoom camera; the wide-angle camera is a wide-angle CMOS (Complementary Metal Oxide Semiconductor, CMOS for short) camera, which is arranged in front of the object to be measured, for acquiring full-field images and implementing target segmentation and detection algorithms to obtain image coordinates of the target and defect positions in the entire large field of view; the zoom camera is arranged at the side of the two-dimensional mirror system, for acquiring images transmitted by the mirrors in the two-dimensional mirror system. The zoom camera is a high-magnification zoom CMOS camera, with an optical magnification of 10 to 40 times, preferably 20 times, to ensure high-resolution imaging and accurate positioning of local defects in a small field of view.

[0086] The construction method of the large-format object unsupervised defect detection system based on two-dimensional galvanometer according to the present application is as follows: first, the support frame of the system is fixedly installed on a high-precision optical experiment platform, the two galvanometers of the two-dimensional galvanometer system are arranged at right angles to each other and are respectively installed on two corresponding motors for separate control, then a wide-angle camera is installed in front of the object to be detected for implementing target segmentation and detection algorithm to obtain image coordinates (x, y) of the target and defect position in the entire large field of view, and a zoom camera is installed at the side of the two-dimensional galvanometer system, i.e. at the light source entrance of the two-dimensional galvanometer system, and is aligned with the X-axis galvanometer lens to be responsible for collecting the target image reflected twice by the Y-axis galvanometer and the X-axis galvanometer; then the galvanometer motion control board card in the two-dimensional galvanometer system is connected with the processor and the X-axis galvanometer and the Y-axis galvanometer, and the specific process is as follows: the galvanometer motion control board card is preferably a solid high GT400-Scan multi-axis motion control card to control the galvanometer motion, the galvanometer motion control card and the adapter card are respectively connected to the PCI-E bus of the processor, i.e. the PC host, and are connected with the CN17 and CN18 ports of the controller terminal board through the connecting lines, while the control lines of the galvanometer motors of the two galvanometers in the two-dimensional galvanometer system are respectively connected with the CN1 and CN2 ports in the controller terminal board, so as to finally realize the connection among the processor, the galvanometer motion control card and the galvanometer motor, and then the communication control between the processor and the galvanometer motion control card is established through the galvanometer motion control card driver program. The GT-400-Scan galvanometer motion control card can control the point positioning and interpolation control of up to 4 galvanometer motor axes at the same time, i.e. can control two two-dimensional scanning galvanometers at the same time, and through the internal dynamic link library and high-level language function library such as C language, a control system such as projection planning and data processing that meets the application requirements can be designed; after the entire large-format object unsupervised defect detection system based on two-dimensional galvanometer is constructed, the galvanometer imaging model and calibration need to be established before use, so as to ensure the system accuracy and imaging quality.

[0087] As shown in Figure 2 The present application also provides a detection method suitable for the large-format object unsupervised defect detection system based on two-dimensional galvanometer according to the present application, which comprises the following steps:

[0088] Step 1, the mapping relationship between the galvanometer rotation angle and the galvanometer coordinate system is constructed, and the formula is as follows:

[0089]

[0090]

[0091]

[0092] In the formula, the point (x, y, z) is a point in the galvanometer coordinate system; is the Y-axis galvanometer rotation angle; is the X-axis galvanometer rotation angle.

[0093] In the two-dimensional galvanometer system, whether an arbitrary point P (x, y, z) in the galvanometer coordinate system space can be captured by the camera mainly depends on the orientation of the camera optical axis, and the orientation of the camera optical axis depends on the rotation angles of the two galvanometer mirrors, so the rotation angles of the horizontal galvanometer mirror and the pitch galvanometer mirror can be obtained and The calculation formula is as follows:

[0094]

[0095]

[0096]

[0097] For a point P (x, y) on a plane, after angle reflection, the imaging point is:

[0098]

[0099] where R is a two-dimensional reflection transformation matrix. For a point P in space, the imaging point after reflection is:

[0100]

[0101] In the formula, (x', y', z') is the imaging point after reflection; n represents the normal vector of the reflection plane, which determines the direction of the reflection plane; represents a unit matrix; represents a three-dimensional reflection transformation matrix, which is used to describe the mirror image transformation relationship of an arbitrary point in space relative to the plane.

[0102] Step 2, construct the mapping relationship between the galvanometer coordinate system and the world coordinate system;

[0103] As Figure 3 shown, the method for constructing the galvanometer coordinate system and the physical space coordinate system (i.e. the world coordinate system) of the projected target adopted by the application is a joint calibration of a wide-angle camera and a two-dimensional galvanometer system based on a homography matrix. The galvanometer scans the projection target, and the wide-angle camera captures the target image and extracts the feature point pixel coordinates. By matching the coordinates of the galvanometer coordinate system and the coordinates of the pixel coordinate system and the coordinates of the pixel coordinate system and the coordinates of the world coordinate system, the homography matrix H is solved, and high-precision mapping of the two coordinate systems is realized. Assuming that there is a point P1 and P2 on each of the two planes in space, the homogeneous coordinates are respectively and , then there must be a homography matrix H between them for conversion, and the formula is as follows:

[0104]

[0105] wherein ~ denotes the dimensional equivalence of the homogeneous coordinates of both, the above equation can be expanded as:

[0106]

[0107] The specific steps are as follows:

[0108] Step 2.1, the galvanometer motion control card projects a set of point sets to the projection plane and records its coordinates in the galvanometer coordinate system, wherein represents the three-dimensional Euclidean space with the wide-angle camera coordinate system as the reference, that is, the coordinate domain under the wide-angle camera imaging system, which is used to describe the position distribution of the target points in the pixel coordinate system; then the spot extraction algorithm is used to extract the pixel coordinate point set of the center coordinates of the circle , represents the three-dimensional Euclidean space with the pixel coordinate system as the reference; and forms a point set pair with the point set . The following formula can be obtained: , the homography matrix of the wide-angle camera (i.e. the pixel coordinate system) to the galvanometer coordinate system is calculated, and the formula is as follows:

[0109]

[0110]

[0111]

[0112]

[0113] In the formula, H1 is the mapping of the galvanometer rotation angle coordinates to the pixel coordinates; H2 is the mapping of the galvanometer coordinates to the galvanometer rotation angle coordinates; b represents the number of elements in the point set and the point set ;

[0114] Step 2.2, a circular calibration board with a size equal to the model formed by the point set in step 2.1 is used, and the calibration points of the circular calibration board are circular. The origin of the galvanometer coordinate system can be aligned with the origin of the world coordinate system, and the lower left corner point of the circular calibration board is selected as the origin of the world coordinate system. At this time, the coordinates of the points on the circular calibration board are the same as their coordinates in the world coordinate system. The pixel coordinate point set of the feature points on the circular calibration board is extracted by the wide-angle camera, and at the same time, the corresponding feature point coordinate point set A corresponding feature point coordinate point set in the world coordinate system , A three-dimensional Euclidean space taking the world coordinate system as a reference is obtained

[0115]

[0116] In the formula, b1 represents the number of elements in the point set and .

[0117] Step 2.3, the homography matrix from the pixel coordinate system obtained in step 2.1 to the galvanometer coordinate system and the homography matrix from the world coordinate system obtained in step 2.2 to the pixel coordinate system , the mapping transformation relationship from the world coordinate system to the galvanometer coordinate system is calculated, and the formula is as follows:

[0118]

[0119] In the formula, x g and y g represent the coordinates in the galvanometer coordinate system; x c and y c represent the coordinates in the world coordinate system.

[0120] Step 3, the wide-angle camera and the zoom camera shoot the image of the object to be measured, the processor performs path planning and sends the planning result to the galvanometer motion control card, and the two-dimensional galvanometer system scans through the galvanometer motion control card, and the wide-angle camera and the zoom camera collect images;

[0121] After calibration, the defect image collection of the object to be measured needs to be path planned, and the path planning process is as shown in Figure 4 : first, the wide-angle camera collects a comprehensive image, then considering the coupling relationship between the abnormal detection model main body and the background information, if the background area is not removed, it will cause the model to misjudge, so the image stored in the buffer area also needs to be operated to remove the background, and then the two-dimensional galvanometer system can scan and collect more accurate defect positions according to the planned path.

[0122] The path planning is a processing algorithm for optimizing the window scanning track of the two-dimensional galvanometer system proposed by the application, which is based on the wide-angle camera to obtain the contour information of the measured main body for optimization, and the algorithm flow is as shown in Figure 4 : panoramic images are collected by the wide-angle camera, opencv functions are called to generate a gray image, then DIS depth segmentation is performed to remove the background, then the contour height H is calculated, and then the field of view size is calculated, and finally the scanning field of view is planned according to the field of view fov size. The specific process is as follows:

[0123] Step 3.1, adjust the wide-angle camera view position to cover the main body contour of the object to be measured, calibrate the wide-angle camera with a chessboard calibration plate, obtain the wide-angle camera intrinsic matrix K and distortion coefficient d;

[0124] Step 3.2, collect N (N > 200) calibration points in the working area of the wide-angle camera, cover the entire working area, and establish the mapping relationship between the wide-angle camera pixel coordinates (u, v) and the galvanometer deflection angle (a, b) according to step 2.1; , ) of the galvanometer deflection angle (a, b) according to step 2.1;

[0125] Step 3.3, use the deep estimation segmentation network DIS mask segmentation network to extract the object region in cooperation with the post-processing process;

[0126] Step 3.4, calculate the scanning step according to the field of view size of the zoom camera , the formula is as follows:

[0127]

[0128]

[0129] In the formula, represents the scanning step in the x-axis direction, that is, the horizontal direction; represents the scanning step in the y-axis direction, that is, the vertical direction; here is the field of view overlap rate, and by setting partial overlap, edge information loss can be avoided; represents the field of view width of the zoom camera; represents the field of view height of the zoom camera.

[0130] Step 3.5, according to the scanning step obtained in step 3.4 and the object region extracted in step 3.3, calculate the circumscribed rectangle envelope of the object region to obtain the region boundary box, then divide the region boundary box in the horizontal and vertical directions according to the scanning step to generate a plurality of scanning windows and record the center coordinate points of the scanning windows, and connect the center coordinate points to generate a scanning path; in order to ensure the continuity of the scanning path and the smoothness of the galvanometer movement, the present application adopts a snake-shaped traversal scanning strategy. When the row number is even, the windows are arranged from left to right in turn; when the row number is odd, the windows are arranged from right to left; adjacent rows are connected by the shortest line to form a continuous curved scanning path.

[0131] Step 3.6, according to the scanning path obtained in step 3.5, obtain the scanning path pixel sequence P (P1, P2, P3,... P n), so as to reduce the idle time, and finally, the deflection angle data of the two-dimensional galvanometer system is obtained according to the calibration corresponding relationship of the elements in the pixel sequence P obtained in step 2, so as to form the galvanometer scanning deflection angle sequence Q (Q1, Q2, Q3,..., Q n ) and is transmitted to the two-dimensional galvanometer system.

[0132] Step 3.7, the X-axis galvanometer lens and the Y-axis galvanometer lens of the two-dimensional galvanometer system are controlled to deflect by the corresponding angles in the galvanometer scanning deflection angle sequence Q obtained in step 3.6 in turn, and each angle is delayed for 15-30 ms, so as to ensure that the two-dimensional galvanometer system is stable and to avoid image jitter; meanwhile, according to the current focal length parameter of the zoom camera The exposure time and the imaging scale are automatically adjusted to keep the image resolution and the target area size matched. In each view position, the zoom camera is triggered to expose and shoot synchronously, and the collected image is input to the processor, and the subsequent defect detection processing is waited.

[0133] The change of the focal length influences the field of view angle and the resolution scale factor of the camera, the system acquires the focal length information in real time while the galvanometer deflects, and adjusts the exposure parameter and the galvanometer deflection step according to the focal length. When the focal length increases, the system automatically increases the exposure time and reduces the scanning step, so as to ensure that the local details are imaged clearly; when the focal length decreases, the exposure time is reduced and the scanning step is increased correspondingly, so as to improve the scanning efficiency.

[0134] Step 4, a teacher-student network of double-space collaborative modeling is used for defect anomaly detection on the images collected in step 3, and a detection result is obtained.

[0135] Step 4.1, a teacher-student network of double-space collaborative modeling is constructed, including a teacher network, a student network and a data space modeling module; the teacher network adopts a convolutional neural network model Efficient Net with frozen parameters, and the structure thereof includes multiple layers of convolution, batch normalization and a nonlinear activation layer; the student network has the same structure as the teacher network, but the parameters are dynamically updated, and a dynamic weighted attention module (Dynamic Weighted Attention, referred to as DWA) is added between the middle feature layer and the high layer feature fusion layer of the student network.

[0136] The present application provides a teacher-student network structure based on double-space collaborative modeling, which is used for high-precision defect detection on abnormal samples. The network architecture is as shown in Figure 6 The network mainly consists of three parts: a teacher network, a student network and a data space modeling module; the input of the teacher network is a sample image x preprocessed by a processor, and the output is a teacher feature mapping ; the input of the student network is also a sample image x preprocessed by a processor, and the output is a student feature mapping , the target is to approximate the output features of the teacher network; the intermediate layer output features of the student network mapped to the latent space after transformation by the Normalizing Flow model . Through feature alignment and flow model probability constraints, joint modeling of feature space and data space is realized. The parameters of the teacher network are obtained by pre-training on a large-scale normal data set and remain fixed in the detection task. The task of the student network is to learn the feature distribution of normal samples by minimizing the difference with the teacher features.

[0137] Step 4.2, the teacher network extracts features from the images collected in step 3 to obtain the features of the teacher network , the formula is as follows:

[0138] ;

[0139] wherein, represents the feature extraction function of the teacher network; represents the sample image after preprocessing of the image collected in step 3 by the processor; is a frozen parameter set.

[0140] Step 4.3, the student network extracts features from the images collected in step 3 to obtain the features of the teacher network , the formula is as follows:

[0141] ;

[0142] ;

[0143] wherein, represents the feature extraction function of the student network; is the parameter set of the student network, which is dynamically updated; Y is the intermediate feature output by the student network; is the Normalizing Flow transformation function; represents the output of the Normalizing Flow transformation function, which conforms to the standard Gaussian distribution.

[0144] Step 4.4, construct the loss function of the teacher-student network of double-space collaborative modeling, obtain the feature residual map of the teacher network and the student network, the formula is as follows:

[0145]

[0146] The loss function is composed of three parts, which are the feature loss between the teacher and student networks , the feature loss between the autoencoder and the student network , data space distance loss In the present application, the autoencoder is a subnetwork structure composed of the intermediate feature extraction part of the teacher network, and its function is to encode and reconstruct the features of the input image, so as to learn the feature distribution of the normal sample.

[0147]

[0148]

[0149]

[0150] wherein, represents the output feature of the teacher network; represents the output feature of the corresponding channel of the student network, and the student network learns the feature distribution of the teacher network by minimizing the Euclidean distance between the two. represents the output feature of the student network after the DWA module weighting.

[0151] The present application models the latent variable distribution of the student feature by the Normalizing Flow model, obtains the probability density value of the sample under the standard Gaussian distribution, and calculates the data space distance loss for evaluating the confidence of the input image belonging to the normal distribution.

[0152] Step 4.4.1, the method for obtaining the output feature of the student network after the DWA module weighting is to replace the input parameter of the feature extraction function of the student network in step 4.3, i.e. the sample image x after the image collected in step 3 is preprocessed by the processor, with the sample image after the DWA module weighting, to obtain the output feature of the student network after the DWA module weighting , and the formula is as follows:

[0153]

[0154] In the formula, is the sample image after the image collected in step 3 is preprocessed by the processor; is the sample image after the DWA module weighting; is an activation function, and the formula is , represents the independent variable of the activation function, which is used to limit the weight in the interval [0,1] to realize dynamic adjustment of the features of each channel; ​is a weighted summation function used to fuse the channel features of different scale convolution outputs; C1 and C2 are both small receptive fields; C3 is a large receptive field; w1, w2, w3 and w4 are all superparameters of the DWA module, which are 0.6, 0.4, 0.3 and 0.7 respectively.

[0155] DWA is a dynamic weight adjustment attention module. Through multi-scale convolution and weighting mechanism of different receptive fields, the importance of channel features is dynamically adjusted, thereby enhancing the response ability of the network to abnormal regions. In the network architecture of the present application, the DWA module is arranged between the intermediate feature layer and the high layer feature fusion layer inside the student network, specifically after the Efficient Net backbone feature extraction unit, for multi-scale feature weighting and channel attention adjustment of the intermediate features extracted by the student network. The internal processing procedure of the DWA module is that the input feature x passes through three parallel dilated convolutions, of which the dilated rate of 1 is a normal convolution with a kernel size of 1 (padding=0, stride=1). Global maximum pooling (GMP) is used for small receptive fields (C1 and C2), and global average pooling (GAP) is used for large receptive fields (C3). Finally, 1D-Conv is used for feature extraction and compression operation in the channel dimension. Then the results of C1, C2 and C3 channel outputs are weighted and summed to obtain the fusion feature vector. The fusion feature is sent to the Swish activation function to generate an attention weight vector in the channel dimension. This function limits the weight in the interval [0,1], which realizes the dynamic adjustment of each channel feature. Finally, the attention weight is multiplied with the original input feature to obtain the weighted output feature , that is, the output of the DWA module.

[0156] Step 4.5, the data space modeling module synthesizes the dual space feature difference and the latent space probability distribution of the dual space collaborative modeling of the teacher-student network obtained in step 4.4, that is, the feature loss between the teacher and student networks , the feature loss between the autoencoder and the student network and the data space distance loss ; generates an abnormal thermal map and a final defect detection result.

[0157] The higher the score of the abnormal region, the more significant the difference between the region and the normal sample feature.

[0158] The application proposes a new double-space collaborative modeling teacher-student network unsupervised anomaly detection architecture, which uses teacher-student network and normalized flow to jointly model the image space and data space of the data, thereby jointly improving the detection efficiency, effectively avoiding the waste of data space constraints, and significantly improving the sensitivity and generalization ability of anomaly detection. Specifically, in the aspect of image space construction, first, the output difference of the teacher-student network on the multi-scale level is used to model the image texture, then the output difference of the student and AE is used to model the image global logic, and finally the inconsistency of the two is used for prediction. In the aspect of data space modeling, the reversible normalization flow model is innovatively introduced, the probability density function of the data latent space is modeled through the staggered clip affine coupling layer, the accurate likelihood estimation score is used as a constraint of the loss function, and the two sub-networks realize collaborative evolution through joint optimization of the objective function.

[0159] In addition to the above embodiments, the application can have other implementation manners, and any technical solutions formed by equivalent replacement or equivalent transformation shall fall within the protection scope required by the application.

Claims

1. A large-format object unsupervised defect detection system based on a two-dimensional galvanometer, characterized in that: include The image acquisition module is used to acquire images of the object under test. A two-dimensional galvanometer system is used to rapidly project a light spot onto a specified location on the surface of the object under test; The processor is used to process the images acquired by the image acquisition module, plan the scanning path of the two-dimensional galvanometer system, and identify defects.

2. The unsupervised defect detection system for large-format objects based on a two-dimensional galvanometer according to claim 1, characterized in that: The two-dimensional galvanometer system consists of two galvanometers orbiting the X-axis and Y-axis respectively, with the plane formed by the X-axis and Y-axis parallel to the camera's focusing plane. The two galvanometers perform pitch and nod movements and horizontal tilting movements respectively around the X-axis and Y-axis. The two-dimensional galvanometer system also includes a galvanometer controller system for controlling the rotation of the two galvanometers. The galvanometer controller system includes a galvanometer motion control card, an adapter card, and a controller terminal board.

3. The unsupervised defect detection system for large-format objects based on a two-dimensional galvanometer according to claim 2, characterized in that: The image acquisition module includes a wide-angle camera and a zoom camera; the wide-angle camera is set directly in front of the object under test and is used to acquire full-field images and implement target segmentation and detection algorithms to obtain the image coordinates of the target of interest and the defect location in the entire large field of view; the zoom camera is set on the side of the two-dimensional galvanometer system and is used to acquire images transmitted from the galvanometer in the two-dimensional galvanometer system.

4. A detection method applicable to the unsupervised defect detection system for large-format objects based on a two-dimensional galvanometer as described in claim 3, characterized in that: Includes the following steps: Step 1: Establish the mapping relationship between the galvanometer rotation angle and the galvanometer coordinate system; Step 2: Establish the mapping relationship between the galvanometer coordinate system and the world coordinate system; Step 3: The wide-angle camera and zoom camera capture images of the object under test. The processor performs path planning and sends the planning results to the galvanometer motion control card. The two-dimensional galvanometer system scans through the galvanometer motion control card, and the wide-angle camera and zoom camera acquire images. Step 4: Use a dual-space collaborative modeling teacher-student network to perform defect and anomaly detection on the images acquired in Step 3, and obtain the detection results.

5. The detection method of the large-format object unsupervised defect detection system based on a two-dimensional galvanometer as described in claim 4, characterized in that: The formula for establishing the mapping relationship between the galvanometer rotation angle and the galvanometer coordinate system in step 1 is as follows: ； ； ； In the formula, point (x, y, z) is a point in the galvanometer coordinate system; The rotation angle of the Y-axis galvanometer; The rotation angle of the X-axis galvanometer.

6. The detection method of the large-format object unsupervised defect detection system based on a two-dimensional galvanometer according to claim 5, characterized in that: The specific steps of step 2 are as follows: Step 2.1: The galvanometer motion control card projects a set of points. Project onto the plane and record its coordinates in the galvanometer coordinate system. This represents a three-dimensional Euclidean space referenced to the wide-angle camera coordinate system; then, a spot extraction algorithm is used to extract the point set. Obtain the pixel coordinate set of the pixel coordinate system from the center coordinates of the circle. , Represents a three-dimensional Euclidean space referenced to a pixel coordinate system; and is related to the point set. Form point set pairs; The homography matrix from the pixel coordinate system to the galvanometer coordinate system is obtained from the following formula. The formula is as follows: ； ； ； ； In the formula, H1 is the mapping from galvanometer rotation coordinates to pixel coordinates; H2 is the mapping from galvanometer coordinates to galvanometer rotation coordinates; b represents the point set. Hedianji The number of elements in; Step 2.2: Using a circular calibration board of the same size as the model composed of the point set in Step 2.1, select the lower left corner of the circular calibration board as the origin of the world coordinate system, and extract the pixel coordinate system point set composed of the feature points on the circular calibration board by taking pictures with a wide-angle camera. Simultaneously, the coordinate set of the corresponding feature points in the world coordinate system is extracted through the projection digital model. , Represents three-dimensional Euclidean space with reference to the world coordinate system; After optimization, the homography matrix from the world coordinate system to the pixel coordinate system can be obtained. for: ； In the formula, b1 represents the point set. and The number of elements in; Step 2.3: Obtain the homography matrix from the pixel coordinate system to the galvanometer coordinate system obtained in Step 2.

1. The homography matrix from world coordinates to pixel coordinates obtained in step 2.2 The mapping transformation relationship from the world coordinate system to the galvanometer coordinate system is calculated, and the formula is as follows: ； In the formula, x g ,y g Represents the coordinates in the galvanometer coordinate system; x c ,y c Represents coordinates in the world coordinate system.

7. The detection method of the large-format object unsupervised defect detection system based on a two-dimensional galvanometer as described in claim 6, characterized in that: The specific steps of step 3 are as follows: Step 3.1: Adjust the wide-angle camera's viewing angle to cover the main outline of the object to be measured, calibrate the wide-angle camera using a checkerboard calibration board, and obtain the wide-angle camera's intrinsic parameter matrix K and distortion coefficient d; Step 3.2: Collect N calibration points within the working area of ​​the wide-angle camera to cover the entire working area, and establish the mapping relationship between the pixel coordinates of the wide-angle camera and the deflection angle of the galvanometer according to Step 2.1; Step 3.3: Use the Depth Estimation Segmentation Network (DIS) mask segmentation network in conjunction with the post-processing to extract the region of the object to be tested; Step 3.4: Calculate the scanning step size based on the zoom camera's field of view, using the following formula: ； ； In the formula, Indicates the horizontal scan step size; Indicates the vertical scan step size; This represents the field-of-view overlap rate. Indicates the field of view width of a zoom camera; Indicates the field of view height of a zoom camera; Step 3.5: Based on the scanning step length obtained in Step 3.4 and the area to be measured extracted in Step 3.3, calculate the bounding rectangle of the area to be measured to obtain the area bounding box. Then, according to the scanning step length, divide the area bounding box into grids in the horizontal and vertical directions to generate several scanning windows and record the center coordinates of the scanning windows. Connect the center coordinates to generate the scanning path. Step 3.6: Based on the scanning path obtained in Step 3.5, obtain the scanning path pixel sequence P (P1, P2, P3, ... P) of the two-dimensional galvanometer system. n Finally, the deflection angle data of the two-dimensional galvanometer system are obtained one by one for each element in the pixel sequence P according to the calibration correspondence obtained in step 2, thereby constructing the galvanometer scanning rotation angle sequence Q (Q1, Q2, Q3, ..., Q n ), and then transmit it to the two-dimensional galvanometer system; Step 3.7: Control the X-axis and Y-axis galvanometer lenses of the two-dimensional galvanometer system to deflect sequentially by the corresponding angles according to the galvanometer scanning rotation angle sequence Q obtained in Step 3.6; simultaneously, based on the current focal length parameters of the zoom camera... The system automatically adjusts the imaging scale and compensates for changes in exposure time to accommodate the impact of focal length variations on imaging illumination and sharpness. The acquired images are then input into the processor for subsequent defect detection.

8. The detection method of the large-format object unsupervised defect detection system based on a two-dimensional galvanometer according to claim 7, characterized in that: The specific steps of step 4 are as follows: Step 4.1: Construct a dual-space collaborative modeling teacher-student network, including a teacher network, a student network, and a data space modeling module; the teacher network adopts the Efficient Net convolutional neural network model with frozen parameters, whose structure includes multiple convolutional layers, batch normalization, and nonlinear activation layers; the student network has the same structure as the teacher network, but the parameters are dynamically updated, and a DWA module is added between the intermediate feature layer and the high-level feature fusion layer of the student network. Step 4.2: The teacher network extracts features from the images acquired in Step 3 to obtain the features of the teacher network. The formula is as follows: ； in, This represents the feature extraction function of the teacher network; This represents the sample image after preprocessing by the processor from the image acquired in step 3. For the frozen parameter set; Step 4.3: The student network extracts features from the images acquired in Step 3 to obtain the features of the teacher network. The formula is as follows: ； ； In the formula, This represents the feature extraction function of the student network; Y represents the parameter set of the student network; Y represents the intermediate features output by the student network. This is the Normalizing Flow transformation function; This represents the output of the Normalizing Flow transformation function; Step 4.4: Construct the loss function for the teacher-student network in dual-space collaborative modeling, and obtain the feature residual maps of the teacher network and the student network, as shown in the following formula: ； ； ； ； In the formula, This represents the feature loss between teacher-student networks; This represents the feature loss between the autoencoder and the student network; Indicates spatial distance loss in the data; Indicates the teacher network Output characteristics of each channel; This represents the output characteristics of the corresponding channel in the student network; This represents the output characteristics of the student network after weighting by the DWA module; Step 4.5: The data space modeling module integrates the feature loss between the teacher and student networks obtained in Step 4.

4. Feature loss between autoencoder and student network and data space distance loss Generate anomaly heatmaps and final defect detection results.

9. The detection method of the large-format object unsupervised defect detection system based on a two-dimensional galvanometer as described in claim 8, characterized in that: The method for obtaining the weighted output features of the student network after DWA module in step 4.4 is as follows: Step 4.4.1: Replace the sample image x obtained in Step 3 after preprocessing by the processor with the sample image weighted by the DWA module. The output features of the student network after weighting by the DWA module are obtained. The formula is as follows: ； In the formula, It is a sample image obtained from the image acquired in step 3 after preprocessing by the processor; These are sample images weighted by the DWA module; It is an activation function; It is a weighted summation function; C1 and C2 are both small receptive fields; C3 is a large receptive field; w1, w2, w3 and w4 are all hyperparameters of the DWA module.

10. The detection method of the large-format object unsupervised defect detection system based on a two-dimensional galvanometer according to claim 9, characterized in that: The activation function The formula is as follows ,in This represents the independent variable of the activation function.

Images