Surface defect detection method and system based on curved surface coding and multi-view fusion

Abstract

The application relates to the technical field of machine vision detection, in particular to a surface defect detection method and system based on curved surface coding and multi-view fusion, which comprises the following steps: a curved surface coding illumination system is constructed, an encoding light field is generated and projected onto a target surface; through a multi-view imaging device, reflection images of the encoding light field are synchronously collected, and a multi-dimensional view image dataset is acquired; light components are separated from the multi-dimensional view image dataset, and a defect feature set is extracted; multi-view information integration is performed on the defect feature set, and comprehensive defect features are obtained; based on the comprehensive defect features, defect judgment and parameter calculation are performed on the target surface, and a defect detection result is output. Through the adoption of curved surface coding illumination and multi-view fusion, the encoding light field carries gradient information, and multi-view imaging supplements spatial details, so that accurate detection of surface and subsurface defects is realized, the adaptability to high-curvature and multi-reflection characteristic targets is improved, and the problem of insufficient defect detection precision of irregular light-reflecting surfaces is solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine vision detection, in particular to a surface defect detection method and system based on curved surface coding and multi-view fusion. BACKGROUND

[0002] In the fields of precision manufacturing, aerospace, etc., the surface defects of workpieces are directly related to product performance and service safety. However, complex workpieces often have irregular curved surface shapes, and defects exist in multiple types and fine features on the surface and subsurface, which brings great challenges to accurate detection of surface defects.

[0003] In the existing surface defect detection methods, single-view imaging is easy to miss defects due to visual angle blind area; traditional coded light field detection relies on fixed mode and is difficult to dynamically adapt to curved surface shape, resulting in coding information fault or redundancy; detection based on single light component cannot effectively distinguish the feature differences between surface and subsurface defects, and lacks accurate quantification of three-dimensional geometric parameters of defects, so the detection accuracy and robustness cannot meet the requirements when facing complex workpieces with high curvature and multiple defect types.

[0004] Moreover, these methods fail to dynamically modulate the coded light field according to the irregular shape of the surface to be detected, and cannot achieve full-type and three-dimensional accurate detection of surface and subsurface defects through multi-view fusion and light component separation. In the detection of defects of complex curved surface workpieces, it is easy to miss detection and misjudgment, and cannot adapt to the strict requirements of defect detection in high-precision manufacturing scenarios, so there is an urgent need for a detection method that can dynamically adapt to curved surfaces, multi-view fusion and accurately extract the features of irregular reflective surface defects. SUMMARY

[0005] In order to solve the problems of traditional surface defect detection methods, the present application provides a surface defect detection method and system based on curved surface coding and multi-view fusion.

[0006] In a first aspect, the present application provides a surface defect detection method based on curved surface coding and multi-view fusion, comprising:

[0007] constructing a curved surface coding lighting system to generate a coded light field and project it to a target surface;

[0008] synchronously acquiring reflection images of the coded light field through a multi-view imaging device to obtain a multi-dimensional view image data set;

[0009] separating light components from the multi-dimensional view image data set to extract a defect feature set;

[0010] integrating multi-view information of the defect feature set to obtain comprehensive defect features;

[0011] Based on the comprehensive defect features, defect determination and parameter calculation are performed on the target surface, and a defect detection result is output.

[0012] By adopting the technical scheme, the curved surface coded light and multi-view fusion are adopted to break through the limitation of traditional methods for detecting irregular reflective surfaces. The coded light field carries gradient information, and multi-view imaging supplements spatial details to realize accurate detection of surface and subsurface defects, improve the adaptability to high-curvature and multi-reflection characteristic targets, and solve the problem of insufficient defect detection precision of irregular reflective surfaces.

[0013] In a specific implementable scheme, the synchronous acquisition of the reflection image of the coded light field by the multi-view imaging device to obtain a multi-dimensional view image dataset includes:

[0014] The internal and external parameters of the multi-view imaging device are calibrated by a calibration device to generate calibration parameters.

[0015] Based on the calibration parameters, the projection of the coded light field and the timing alignment of the acquisition of the reflection image are realized by a synchronous triggering module.

[0016] By adopting the technical scheme, the calibration device and the synchronous triggering are adopted to eliminate the spatial and temporal deviations of multi-view imaging by accurately solving the camera parameters and timing alignment. The spatial and temporal consistency of the reflection image is ensured to lay a data foundation for subsequent feature extraction and improve the reliability of defect detection in complex scenes.

[0017] In a specific implementable scheme, the above method further includes:

[0018] Based on the calibration parameters, geometric registration is performed on the multi-dimensional view image dataset.

[0019] The registered multi-dimensional view image dataset is subjected to gray calibration and noise filtering to eliminate the differences between the views.

[0020] By adopting the technical scheme, the coordinate system is unified by geometric registration, and the differences between the views and the interference are eliminated by combining gray calibration and noise filtering. The different view images are comparable, the errors caused by light and equipment differences are reduced, standardized data is provided for defect feature extraction, and the robustness of the detection algorithm is improved.

[0021] In a specific implementable scheme, the separation of light components from the multi-dimensional view image dataset and the extraction of a defect feature set include:

[0022] The multi-dimensional view image dataset is subjected to high-order curved surface block planning to determine a sub-aperture detection region.

[0023] The three-dimensional geometric correction is performed on each of the sub-aperture detection areas, and the corrected sub-area images are fused to extract multi-resolution defect features.

[0024] By adopting the technical solutions, the high-order curved surface is divided into blocks, the three-dimensional correction is performed, the detection areas are divided according to the curvature, and targeted processing is performed. Through the geometric correction and fusion of the sub-area, the influence of the curved surface deformation is overcome, multi-resolution defect feature extraction is realized, and the capture ability of small defects in the high-curvature area is particularly enhanced.

[0025] In a specific implementable scheme, the construction curved surface encoding light system generates an encoding light field and projects it to a target surface, and includes:

[0026] The phase distribution of the encoding light field is modulated by controlling the number of phase shift steps, so that the encoding light field carries the pre-encoding information of the gradient of the target surface, and the number of phase shift steps is adapted to the curvature of the target surface.

[0027] By adopting the technical solutions, the encoding light field accurately carries the surface gradient information, balances the detection accuracy and efficiency, and solves the problem of information redundancy or deficiency in the detection of irregular reflective surfaces by traditional fixed phase shift.

[0028] In a specific implementable scheme, the separation of light components from the multi-dimensional perspective image data set and the extraction of a defect feature set further include:

[0029] Based on the phase characteristics of the encoding light field, the diffuse reflection and specular reflection light components are separated;

[0030] Based on the gradient of the target surface, the normal vector of the target surface is calculated and derived, and a defect feature containing three-dimensional geometric information is extracted.

[0031] By adopting the technical solutions, the diffuse reflection and specular reflection light components are separated based on the phase characteristics, and the normal vector is derived in combination with the gradient. The surface and subsurface defect are distinguished, the defect parameters are quantified through the three-dimensional geometric information, and the recognition accuracy is improved.

[0032] In a specific implementable scheme, the multi-perspective information integration of the defect feature set to obtain a comprehensive defect feature includes:

[0033] The defect feature set is reduced in dimension and spliced to construct a unified feature vector;

[0034] A multi-perspective feature correlation model is established, and based on the unified feature vector, a comprehensive defect feature is output.

[0035] By adopting the technical scheme, a unified vector is constructed through feature dimension reduction splicing, and information is fused in combination with a multi-view correlation model. High-dimensional feature redundancy is reduced, complementary information is mined through cross-view feature correlation, defect judgment robustness is improved, and the inconsistency problem of multi-view data fusion is solved.

[0036] In one specific implementation scheme, the curved surface coding illumination system generates a coded light field and projects it to the target surface, including:

[0037] A one-dimensional pseudo-random sequence is generated, and a two-dimensional pseudo-random matrix is constructed;

[0038] The two-dimensional pseudo-random matrix is checked for completeness of field of view coverage of the target surface, and the two-dimensional pseudo-random matrix that passes the check is converted into a coded light field that adapts to the target surface.

[0039] By adopting the technical scheme, a pseudo-random matrix generation and field of view checking mechanism is adopted, and optical simulation is used to ensure full coverage of the light field. The curved surface morphology is adjusted and adapted through partition splicing, blind areas are avoided, the adaptability of the coded light field to irregular reflective surfaces is improved, and the integrity of the defect features is ensured to be captured.

[0040] In a second aspect, the application also provides a surface defect detection system based on curved surface coding and multi-view fusion, including:

[0041] A curved surface coding illumination system is used to generate and project a coded light field, and through curvature adaptation and field of view adaptation, it carries pre-encoded information of the height gradient of the target surface;

[0042] A multi-view imaging module includes an industrial camera, a calibration device, and a synchronous triggering unit. The calibration device is used to solve the internal and external parameters of the camera, and the synchronous triggering unit is used to realize timing alignment. The industrial camera is used to collect reflected images and generate multi-dimensional view data sets;

[0043] An image preprocessing module is used to perform geometric registration, gray calibration, and noise filtering based on calibration parameters, and output standardized images;

[0044] A defect feature extraction module is used to extract and generate a defect feature set;

[0045] A multi-view integration module is used to splice the defect feature set into a unified feature vector, and output comprehensive defect features through multi-view feature correlation fusion;

[0046] A result output module is used to filter effective defects based on the comprehensive defect features, convert pixel parameters into physical parameters, divide defect levels, and generate defect detection results.

[0047] By adopting the technical scheme, the curved surface coding and multi-view imaging work cooperatively. The pre-processing module standardizes the image, and the multi-view integration module optimizes feature fusion, realizes precise control of the whole process from image acquisition to result output, and greatly improves the defect detection efficiency and precision of the irregular reflective surface.

[0048] In one specific implementation scheme, the curved surface coding light system comprises:

[0049] A phase shift coding unit is configured to adapt the number of phase shift steps to the target surface curvature, and generate a sinusoidal phase shift light field carrying the height gradient information of the target surface.

[0050] A pseudo-random coding unit is configured to construct a pseudo-random coding matrix and check whether the matrix meets the field of view requirement.

[0051] A light field adaptation unit is configured to perform partition splicing adjustment on the pseudo-random coding matrix, and then convert the adjusted matrix into the coding light field and project it onto the target surface.

[0052] A field of view checking unit is configured to simulate the projection effect of the coding light field on the target surface through optical simulation.

[0053] By adopting the technical scheme, the field of view checking and light field adaptation dynamic adjustment are realized. The number of phase shift steps is adapted to the curvature, the matrix partition optimizes the projection effect, the adaptability of the light field to the curved surface is enhanced, and the contrast between the defects and the background is improved, thereby providing a high-quality light field basis for accurate detection.

[0054] In summary, the present application includes at least one of the following beneficial effects:

[0055] 1. The present application cooperates with multi-view imaging through dynamic adaptation of coding light field, combines light field projection angle optimization and field of view checking, realizes accurate coverage of the whole field of view in high and low curvature areas, avoids coding blind area and feature covering, and adapts to the detection form of irregular reflective surface.

[0056] 2. The present application separates light components, extracts three-dimensional geometric features by gradient derivation method, and then outputs quantized results containing comprehensive confidence through multi-view feature correlation fusion, thereby improving the defect recognition type and parameter accuracy.

[0057] 3. The present application scheme is designed by modularization, combined with time sequence alignment and image standardization processing, eliminates the differences between equipment and viewing angles, ensures that the single target detection time is controllable, and the detection result contains traceable parameters and visualized annotation, and takes into account the efficiency and traceability requirements of industrial scenes. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 is a flowchart of a surface defect detection method based on curved surface coding and multi-view fusion provided by an embodiment of the present application.

[0059] Figure 2 is a structural schematic diagram of a multi-view imaging device provided by an embodiment of the present application;

[0060] Figure 3 is a module schematic diagram of a surface defect detection system based on curved surface coding and multi-view fusion provided by an embodiment of the present application. DETAILED DESCRIPTION

[0061] The terms used in the following embodiments of the present application are only for the purpose of describing specific embodiments and are not intended to be limiting on the present application. As used in the specification of the present application, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or" used in the present application, means any or all possible combinations of one or more of the listed items.

[0062] Hereinafter, the terms "first", "second" are only for the purpose of description, and cannot be understood as implying or suggesting relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise specified.

[0063] The technical solutions in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all.

[0064] Please refer to Figure 1 , Figure 1 A flowchart of a surface defect detection method based on curved surface coding and multi-view fusion provided by an embodiment of the present application is provided. The method can be implemented by relying on a computer program, can be implemented by relying on a single-chip microcomputer, and can run on a surface defect detection system based on curved surface coding and multi-view fusion. The computer program can be integrated in a computer device or run as an independent tool application. Specifically, the method includes S100 to S500, and the steps are as follows:

[0065] S100, a curved surface coding lighting system is constructed to generate a coded light field and project it to a target surface;

[0066] In some embodiments, the coding mode of the curved surface coding lighting system can include two core modes of phase shift coding and pseudo-random coding. In the detection process, the curved surface coding lighting system realizes the application of the two types of coding modes through dynamic modulation.

[0067] Specifically, for phase shift encoding, the system dynamically modulates the phase distribution of the light field, for example, adjusts the number of phase shift steps, the phase change frequency; for pseudo-random encoding, the system dynamically modulates the spatial mode of the light field, for example, adjusts the period of the pseudo-random sequence, the matrix filling method.

[0068] The core goal of dynamic modulation is to enable the generated coded light field to adapt to the irregular morphology of the target surface to be detected (such as curved protrusions, recesses, irregular edges), and thus to provide a high-resolution defect feature for the subsequent multi-view imaging device, avoiding the defect feature being covered or unable to be effectively captured due to the mismatch between the coded light field and the target morphology.

[0069] In some embodiments, the way to adapt to the irregular morphology of the target surface to be detected includes curved surface curvature adaptation, which means that the spatial morphology of the coded light field is adjusted in response to the curvature change of the target surface to be detected.

[0070] Specifically, for high-curvature areas of the target surface (such as protrusions and recesses with small curvature radii), the system optimizes the light field projection angle and the coded light field density to ensure that the light field can uniformly cover the curved area and the coded information is complete; for low-curvature areas of the target surface (such as flat areas), the system maintains the basic coding parameters of the light field to avoid coding redundancy caused by excessive modulation. Through curved surface curvature adaptation, information gaps in the curved area of the target surface can be effectively avoided, and the defect features in different curvature areas can be accurately marked by the light field.

[0071] In some embodiments, the way to adapt to the irregular morphology of the target surface to be detected also includes field of view range adaptation. Field of view range adaptation includes that the projection range of the coded light field completely covers all areas of the target surface to be detected without blind area, and the coding resolution of the coded light field meets the defect detection requirements.

[0072] Specifically, the coding resolution is reflected by the stripe density of the light field or the pixel density of the pseudo-random matrix, and the number of coding pixels corresponding to a single defect in the coded light field is required to be no less than 2 pixels. Through field of view range adaptation, the multi-view imaging device can obtain the full-area image of the target surface, ensuring the detail clarity of the defect features in the imaging result, and providing a reliable data basis for subsequent defect feature extraction and recognition.

[0073] On the basis of the above embodiments, as another optional embodiment, a curved surface coded light illumination system is constructed to generate and project the coded light field to the target surface, which includes:

[0074] S101, modulate the phase distribution of the coded light field by controlling the number of phase shift steps, so that the coded light field carries the pre-encoding information of the target surface gradient, and the number of phase shift steps is adapted to the curvature of the target surface.

[0075] The phase shift step number refers to the number of steps of controlling the phase shift of the coded light field in the embodiments of the present application. The step number is adapted to the target surface curvature, and the step number is larger in the high curvature area.

[0076] Specifically, the curvature distribution data of the target surface is collected through edge curvature analysis of the preprocessed image, the first step size phase shift is used for the high curvature area, and the second step size phase shift is used for the low curvature area, so that the phase modulation density is adapted to the surface morphology. The curvature radius corresponding to the high curvature area is less than or equal to 5 mm, and the first step size can be 6 steps. The curvature radius corresponding to the low curvature area is greater than or equal to 50 mm, and the second step size can be 4 steps.

[0077] In some embodiments, the formula of the phase shift step number N can include:

[0078]

[0079] Wherein, N is the phase shift step number calculated for the current surface area; Nbase is the basic phase shift step number, for example, Nbase = 4 for a plane or a low curvature area; a is a scaling factor, which is an empirical constant greater than 0, used to adjust the sensitivity of the curvature to the step number; |k| is the absolute average curvature of the current area, and the greater the curvature, the more phase shift steps are required to obtain higher phase measurement accuracy; is the upward rounding symbol, which ensures that the step number is an integer.

[0080] Based on the adapted phase shift step number, a sinusoidal phase shift fringe light field is generated, so that the light field carries the pre-encoding information of the height gradient of the target surface. The core characteristic of the sinusoidal phase shift fringe light field is that the phase change is linearly positively correlated with the height difference of the target surface. The pre-encoding information specifically embodies that the concave area of the target surface corresponds to the phase delay of the light field, and the convex area corresponds to the phase advance of the light field, so as to realize the accurate conversion of the height gradient information to the phase signal of the light field.

[0081] In some embodiments, the direction of the light field is adjusted to ensure that the light field is focused in the concave area and diffused in the convex area to compensate for the light field, so as to realize the geometric matching of the coded light field and the target surface.

[0082] On the basis of the above-mentioned embodiments, as another optional embodiment, a curved surface coding light system is constructed to generate and project the coded light field to the target surface, which includes:

[0083] S102, generating a one-dimensional pseudo-random sequence and constructing a two-dimensional pseudo-random matrix;

[0084] The one-dimensional pseudo-random sequence in the embodiments of the present application refers to a coding element with local randomness and global controllability. The sequence length and period parameters of the one-dimensional pseudo-random sequence are adapted to the resolution of multi-view imaging and the detection window size of the target surface.

[0085] Specifically, a detection window size parameter of the target surface is acquired. The detection window size parameter needs to match the field of view range and imaging resolution of the multi-view imaging device.

[0086] Based on the detection window size, a period of the one-dimensional pseudo-random sequence is determined. In a single period, the number of pixels corresponding to the encoding mode of the defect is greater than or equal to a defect pixel threshold.

[0087] In some embodiments, the defect pixel threshold can be 2 pixels.

[0088] Based on the detection window size, the length of the one-dimensional pseudo-random sequence is determined. The sequence length matches the imaging resolution of the multi-view camera. For example, when the multi-view camera resolution is 1024 pixels x 1024 pixels, the sequence length is set to 1024 pixels. The finally generated one-dimensional pseudo-random sequence satisfies the characteristics of local non-repetition mode and global cyclic calling, which provides uniform and recognizable encoding primitives for subsequent two-dimensional matrix filling.

[0089] Based on the one-dimensional pseudo-random sequence generated above, a two-dimensional pseudo-random matrix is constructed. Specifically, according to the field of view range and resolution of multi-view imaging, the two-dimensional pseudo-random matrix is constructed, and the number of pixels in the horizontal and vertical directions of the matrix is consistent with the imaging resolution of the multi-view camera, ensuring that the encoding light field generated by the matrix can completely cover the detection field of view of the target surface without blind area.

[0090] The diagonal line cyclic assignment method is used to fill the two-dimensional matrix. Starting from the top left corner of the matrix, the elements of the one-dimensional pseudo-random sequence are filled along the main diagonal direction. When filling to the row boundary or column boundary of the matrix, automatically jump to the starting column of the next row or the starting row of the next column, and continue to fill cyclically along the diagonal direction until all pixels of the entire two-dimensional matrix are assigned.

[0091] In the two-dimensional pseudo-random matrix, the encoding mode of any sub-window is not repeated, and the encoding resolution meets the defect detection requirement, ensuring that the subsequent projected encoding light field can accurately capture the subtle defect features of the target surface.

[0092] S103, verify the field of view coverage integrity of the two-dimensional pseudo-random matrix on the target surface, and convert the two-dimensional pseudo-random matrix that passes the verification into an encoding light field that adapts to the target surface.

[0093] In some embodiments, by converting the two-dimensional pseudo-random matrix into an encoding light field that accurately matches the irregular morphology of the target surface, it is ensured that subsequent multi-view imaging can obtain full field of view and high resolution defect features, avoiding defect omission or feature blurring caused by insufficient adaptability of the encoding light field.

[0094] In some embodiments, the projection effect of the encoded light field generated by the two-dimensional pseudo-random matrix on the target surface is simulated by an optical simulation tool. The simulation process of the optical simulation tool is substituted with parameters such as the actual curvature distribution, size range, etc. of the target surface, and the projection state of the light field in irregular areas such as recesses, protrusions, edges, etc. is restored.

[0095] In some embodiments, the verification standard includes that the encoded light field has no encoding blind area on the target surface, and that the resolution of the encoded light field meets the standard.

[0096] Specifically, the light field projection area needs to cover the entire detection range of the target surface, and the encoding stripe overlap rate of any area is greater than or equal to the stripe overlap threshold value; the number of encoding pixels corresponding to a single minimum defect in the light field is greater than or equal to the encoding pixel threshold value. If an area with a stripe overlap rate less than the stripe overlap threshold value is detected, it is determined to be an encoding blind area; if the number of encoding pixels corresponding to a single defect in a certain area is less than the encoding pixel threshold value, it is determined to be insufficient resolution. In some embodiments, the stripe overlap threshold value can be 30%, and the encoding pixel threshold value can be 2 pixels.

[0097] If the verification does not meet the standard, feedback is given to the one-dimensional pseudo-random sequence generation link, and the period parameter of the one-dimensional sequence is adjusted. After adjustment, the two-dimensional pseudo-random matrix is regenerated, and the above verification operation is performed again until the matrix passes the field of view coverage integrity verification.

[0098] In some embodiments, for the two-dimensional pseudo-random matrix that passes the field of view coverage integrity verification, an encoding array dynamic adaptation operation is performed to match the irregular morphology of the target surface.

[0099] Specifically, according to the curvature distribution data of the target surface, the two-dimensional pseudo-random matrix is regionally spliced and adjusted. For a concave area of the curved surface, a top-down splicing method is used to superimpose local units of multiple two-dimensional pseudo-random matrices, realizing local encryption of the encoding array in the concave area, and ensuring that the subtle defects inside the concave area can be marked by the encoded light field; for a convex area of the curved surface, a diagonal splicing method is used to trim the redundant units of the two-dimensional pseudo-random matrix, realizing sparse compensation of the encoding array in the convex area, and avoiding feature interference caused by excessive density of the convex surface.

[0100] Through splicing and adjustment, an adaptive encoding array corresponding to the morphology of each area of the target surface is generated, which not only retains the window characteristics of the two-dimensional pseudo-random matrix, but also fits the irregular geometric structure of the target surface.

[0101] In some embodiments, the encoding array is converted into an encoded light field by a curved surface encoding lighting system. Specifically, the dynamically adapted encoding array is input into the light field modulation module of the curved surface encoding lighting system, and the digital encoding information of the encoding array is converted into a physical encoded light field with a corresponding spatial mode through micro-mirror flipping or liquid crystal molecule orientation adjustment inside the module.

[0102] In some embodiments, the physical coded light field is projected to the target surface by the optical projection component of the curved coded lighting system. Specifically, the incident angle and focusing range of the light field are adjusted during projection to ensure that the encrypted coded array is accurately focused in the recessed area and the sparse coded array is uniformly diffused in the raised area, thereby achieving full-field accurate coverage of the irregular morphology of the target surface by the pseudo-random coded light field, and providing a light field basis for subsequent multi-view imaging devices to collect high-resolution defect features.

[0103] S200, synchronously collecting, by a multi-view imaging device, a reflected image of the coded light field to obtain a multi-dimensional view image data set;

[0104] Reference Figure 2 The multi-view imaging device in the embodiments of the present application refers to an imaging system composed of at least two industrial cameras. The cameras are fixedly arranged at preset angles. The core function is to synchronously collect the reflected image of the coded light field on the target surface to obtain defect feature information at different views, thereby providing multi-dimensional data support for subsequent multi-view information integration and avoiding the detection blind area of single-view imaging.

[0105] In some embodiments, the reflected image of the coded light field on the target surface is collected by the multi-view imaging device arranged at preset angles under the control of a synchronous triggering module to form a multi-dimensional view image data set containing multi-view and space-time alignment information.

[0106] Specifically, the arrangement position of the multi-view imaging device is confirmed. The industrial cameras are arranged in a ring around the target surface to ensure that the full-field of the target surface is covered without blind area. Then, the synchronous triggering module is started to output a synchronous signal to the curved coded lighting system and each camera, so that the coded light field projection and camera exposure are started synchronously, and each camera synchronously collects a reflected image. Finally, the collected images are classified according to the view angle, and the corresponding camera pose, imaging time and other parameters are associated to generate a multi-dimensional view image data set.

[0107] The multi-dimensional view image data set in the embodiments of the present application includes a set of reflected images of the target surface at different views. Each image in the data set is associated with corresponding view angle parameters, and the image resolution and gray level are adapted to the defect detection requirement, thereby providing spatially complementary defect feature data to support subsequent feature extraction and integration. The view angle parameters can include camera pose and imaging time.

[0108] In some embodiments, the multi-view imaging device can include four industrial cameras.

[0109] On the basis of the above-mentioned embodiments, as another optional embodiment, the multi-dimensional view image data set is obtained by synchronously collecting, by a multi-view imaging device, a reflected image of the coded light field, which includes:

[0110] S201, calibrate the internal and external parameters of the multi-view imaging device by a calibration device to generate calibration parameters;

[0111] The calibration device in the embodiments of the present application refers to a special device for determining the parameters of the multi-view imaging device, including a standard calibration board, a coordinate measurement module, and a parameter calculation software. The internal and external parameters of the camera are calculated by collecting multi-view images of the calibration board, which provides reference data for subsequent image geometric registration and time sequence synchronization, and ensures imaging accuracy. The calibration parameters are calculated by the calibration device and are used to describe the characteristics of the multi-view imaging device, including internal and external parameters, and are used to correct image distortion and unify the coordinate system of multi-view images to ensure the accuracy of subsequent geometric registration. The internal parameters can include camera focal length, principal point coordinates, and distortion coefficients, and the external parameters can include the relative pose relationship between cameras and the conversion matrix between the world coordinate system and the camera coordinate system.

[0112] In some embodiments, a calibration device containing a standard calibration board is used to collect multi-view images of the calibration board, and the internal and external parameters of the multi-view imaging device are obtained by parameter calculation software to generate calibration parameters for subsequent processing.

[0113] Specifically, a chessboard calibration board is selected as a calibration reference, and the calibration board is fixed at a predetermined position on the target surface. The multi-view imaging device is controlled to collect multi-view images of the calibration board. The collected calibration board images are input into the calibration parameter calculation software to calculate the internal and external parameters of each camera, including distortion coefficients and external parameters. The calculated internal and external parameters are integrated into a calibration parameter file and stored in a system database for subsequent geometric registration and time sequence synchronization. The predetermined position on the target surface includes the center of the detection area.

[0114] S202, based on the calibration parameters, the time sequence alignment of the projection of the coded light field and the collection of the reflected images is realized by a synchronous triggering module.

[0115] The synchronous triggering module in the embodiments of the present application refers to a control unit that realizes the time sequence cooperation of the projection of the coded light field and the collection of the images, including a signal generator, a time sequence control chip, and a communication interface, which is used to output a synchronous triggering signal to the curved coded light illumination system and the multi-view imaging device, reduce the time deviation of the light field projection and the image collection, and eliminate the image distortion caused by motion blur or light field misplacement.

[0116] In some embodiments, based on the camera imaging delay and the light field projection response time in the calibration parameters, a synchronous triggering module outputs a cooperative signal to ensure that the time sequence deviation of the projection of the coded light field and the collection of the reflected images is less than or equal to the time sequence deviation threshold. The time sequence deviation threshold can be 1 μs.

[0117] Specifically, based on the imaging characteristics of the multi-view imaging device analyzed based on the calibration parameters, the exposure delay of each camera (such as 20 μs) and the light field projection response time of the curved coded lighting system (such as 15 μs) are determined based on the calibration data; the above timing parameters are input into the synchronous trigger module, and the output logic of the trigger signal is set: after outputting the light field projection trigger signal, the camera exposure trigger signal is output to ensure that the camera starts exposure when the light field completely covers the target surface; the synchronous trigger module is started, and the trigger signal is sent to the curved coded lighting system and the multi-view imaging device respectively, and the timing deviation is monitored in real time.

[0118] On the basis of the above-mentioned embodiments, as another optional embodiment, the surface defect detection method further comprises:

[0119] S203, performing geometric registration on the multi-dimensional view image data set based on the calibration parameters;

[0120] The geometric registration in the embodiments of the present application refers to a process of correcting the spatial deviation of the multi-dimensional view image data set based on the calibration parameters, which is used to convert images of different views to a unified world coordinate system, correct the image offset caused by the difference in camera pose or lens distortion, and make the spatial positions of the same defect in the multi-view image consistent, thereby providing spatially aligned data for subsequent feature extraction.

[0121] In some embodiments, based on the intrinsic and extrinsic parameters in the calibration parameters, the images of different views in the multi-dimensional view image data set are converted to a unified world coordinate system through coordinate conversion and distortion correction, and the spatial deviation is eliminated.

[0122] Specifically, based on the distortion coefficients in the camera intrinsic parameters, the radial and tangential distortions of the lens are corrected through polynomial correction, so that the straight lines in the image are restored to the true geometric shape; based on the coordinate system conversion matrix in the extrinsic parameters, the corrected images of different views are converted to a preset world coordinate system (such as a coordinate system with the center of the target surface as the origin and the Z-axis perpendicular to the surface), and the world coordinates of the pixels in each image are calculated; finally, the image size is adjusted through image resampling, so that the resolution and coordinate system of the multi-view image are completely unified, and the spatial position deviation of the same defect in different view images is less than or equal to the encoding pixel threshold.

[0123] S204, performing gray scale calibration and noise filtering on the registered multi-dimensional view image data set to eliminate the difference between views.

[0124] The gray scale calibration in the embodiments of the present application is a processing operation for eliminating the gray scale difference of the multi-dimensional view image data set, which establishes the gray scale mapping relationship of each view by acquiring the multi-view image of the standard gray scale plate, dynamically compensates the image gray scale unevenness caused by the difference in illumination angle and camera sensitivity, makes the gray scale features of the same defect in the multi-view image consistent, and avoids the interference of gray scale difference with defect recognition.

[0125] The noise filtering is a processing operation for suppressing interference signals of the multi-dimensional perspective image dataset in the embodiments of the present application. The noise types include speckle noise generated by coded light field reflection and motion noise generated by slight shaking of the target. The image noise is smoothed by filtering while the defect edge features are preserved, so that the defect features extracted subsequently are not disturbed by the noise and the feature accuracy is improved.

[0126] In some embodiments, the gray scale difference of the registered image is corrected by establishing a gray scale mapping relationship, and then the image noise is suppressed by filtering to output a standardized image dataset without perspective difference and low noise.

[0127] Specifically, a standard gray scale plate is placed in a target surface detection area, multi-perspective registered images of the gray scale plate are collected, and the gray scale response curve of each camera is calculated. A gray scale mapping matrix of each perspective is established based on the gray scale response curve, and the gray scale of the registered multi-dimensional perspective image dataset is corrected pixel by pixel, so that the gray scale value deviation of the same gray scale plate area in different perspective images is less than or equal to a gray scale deviation threshold. For speckle noise generated by coded light field, Gaussian filtering is used to smooth the noise. For motion noise generated by slight shaking of the target, median filtering is used to preserve the defect edge. After filtering, the defect edge integrity is verified by edge detection to ensure that the defect features are not lost due to noise filtering. Finally, a standardized multi-dimensional perspective image dataset is output as the input for subsequent light component separation and defect feature extraction. The gray scale deviation threshold can be 3 gray scale levels.

[0128] S300, separating light components from the multi-dimensional perspective image dataset and extracting a defect feature set;

[0129] The light component separation in the embodiments of the present application refers to a process of distinguishing and extracting diffuse reflection light components and specular reflection light components from the multi-dimensional perspective image dataset based on the phase characteristics of the coded light field. Through the light component separation, accurate extraction of different types of defect features is realized, and defect feature confusion caused by light component superposition is avoided.

[0130] The defect feature set in the embodiments of the present application is obtained by extracting from the multi-dimensional perspective image dataset and is a feature combination including defect multi-dimensional information. The defect multi-dimensional information can include spatial morphological features such as defect contour and area, three-dimensional geometric features such as depth, slope and normal vector, and gray scale texture features such as gray scale difference between the defect and the background.

[0131] On the basis of the above embodiments, as another optional embodiment, separating light components from the multi-dimensional perspective image dataset and extracting a defect feature set include:

[0132] S301, performing high-order curved surface block planning on the multi-dimensional perspective image dataset to determine a sub-aperture detection area;

[0133] The high-order curved surface block planning in the embodiments of the present application refers to dividing the target surface into a plurality of sub-regions based on the curvature distribution data of the target surface (such as the curvature distribution data collected in S101). By fitting the target surface morphology with a high-order polynomial, the sub-blocks are divided according to the curvature similarity, ensuring that the curved surface morphology in each sub-block is approximately uniform, and avoiding the extraction deviation of defect features caused by the sudden change of the curved surface curvature. The sub-aperture detection region is obtained by high-order curved surface block planning and is used for local defect detection, including a plurality of sub-regions. Among them, the aperture size of each sub-region is adapted to the local curvature, the sub-aperture of the high-curvature region is small, the sub-aperture of the low-curvature region is large, and each sub-region is covered without overlapping. The sub-aperture detection region is used to convert the defect detection of a large area and irregular curved surface into the detection of a small range and approximately regular curved surface, reducing the difficulty of three-dimensional geometric correction and improving the extraction accuracy of local defect features.

[0134] In some embodiments, based on the curvature distribution data, the target surface morphology is fitted with a high-order polynomial, and the target surface is divided into a plurality of non-overlapping sub-aperture detection regions according to the curvature similarity.

[0135] Specifically, the curvature distribution data of the target surface is called, and a quadratic surface equation is used to fit the overall morphology of the target surface; a block rule is set, the high-curvature region is divided into a sub-aperture detection region according to a first size, ensuring that the local curved surface is approximately uniform; the low-curvature region is divided according to a second size, improving the detection efficiency; and the block rule is mapped to a standardized multi-dimensional perspective image data set. The first size can be 5mm×5mm; and the second size can be 20mm×20mm.

[0136] S302, performing three-dimensional geometric correction on each sub-aperture detection region, and fusing the corrected sub-region images to extract multi-resolution defect features.

[0137] The three-dimensional geometric correction in the embodiments of the present application refers to restoring the sub-region image to an approximately planar image by coordinate conversion and morphology correction for the curved surface morphology of each sub-aperture detection region. Through the curvature step data and the calibration parameters, the image stretching and compression distortion caused by the curved surface depression or protrusion is eliminated, and the geometric morphology and actual size of the defects in the corrected sub-region image are ensured to be consistent.

[0138] The multi-resolution defect features in the embodiments of the present application refer to the defect features extracted from the images of the corrected different sub-aperture detection regions according to low resolution and high resolution respectively. The low-resolution features (such as the approximate outline of the defect, the position) are used to quickly lock the defect region, and the high-resolution features (such as the defect edge details, the micro-protrusion height) are used to accurately describe the defect morphology. The combination of the two can guarantee the detection efficiency and ensure that the subtle defect features are not lost, and adapt to the detection needs of defects of different sizes.

[0139] In some embodiments, based on the curvature step data and the calibration parameters, three-dimensional geometric correction is performed on each sub-region image, the images after eliminating the curved surface distortion are fused, and multi-resolution defect features are extracted according to low resolution and high resolution respectively.

[0140] Specifically, for each sub-aperture detection region, based on the calibration parameters and the curvature step data, a conversion model of the curved surface image coordinates and the world coordinates is established, and the curved surface image of the sub-region is restored to an approximately planar image through inverse mapping; all the corrected sub-region images are fused, the sub-region images are spliced according to the actual position of the target surface to form a complete and distortion-free panoramic image of the target surface; multi-resolution defect features are extracted, the panoramic image is multi-scale decomposed, the contour and position features of the defects, and the edge details and gray gradient features of the defects are extracted, the features of different scales are associated and integrated to form multi-resolution defect features, and the multi-resolution defect features are merged into a defect feature set.

[0141] Based on the above embodiments, as another optional embodiment, the defect feature set is extracted by separating the light components from the multi-dimensional perspective image data set, which further includes:

[0142] S303, based on the phase characteristics of the coded light field, separating the diffuse reflection and specular reflection light components;

[0143] The diffuse reflection light component in the embodiments of the present application refers to the light signal formed by scattering of the coded light field projected on the target surface through the rough surface region or the subsurface structure. The diffuse reflection light component has uniform light intensity distribution and gentle phase change, and corresponds to the characteristic information of the subsurface defects (such as internal cracks and impurities) of the target surface; the specular reflection light component refers to the light signal formed by direct reflection of the coded light field through the smooth region of the target surface. The specular reflection light component has concentrated light intensity and significant phase change, and corresponds to the characteristic information of the surface defects (such as scratches and pits).

[0144] In some embodiments, based on the phase change law of the coded light field, the diffuse reflection light component corresponding to the subsurface defects and the specular reflection light component corresponding to the surface defects are separated from the standardized multi-dimensional perspective image data set through phase threshold judgment and light intensity distribution analysis.

[0145] Specifically, based on the phase characteristic parameter of the encoded light field, for each image in the standardized multi-dimensional view image dataset, the phase value of each pixel in the image is extracted; based on the characteristics that the phase change of the diffuse reflection light component is gentle and the phase change of the specular reflection light component is significant, the region with a pixel phase difference less than or equal to a phase difference threshold is determined as a diffuse reflection region, and the light intensity signal of the region is extracted as the diffuse reflection light component; the region with a pixel phase difference greater than the phase difference threshold is determined as a specular reflection region, and the light intensity signal of the region is extracted as the specular reflection light component; the two separated light components are subjected to gray scale normalization processing respectively to eliminate light intensity fluctuation interference; the processed light component data is associated with the corresponding view and sub-aperture region and stored in the defect feature set. The phase difference threshold can be 0.1 rad.

[0146] S304, based on the target surface gradient, a target surface normal vector is calculated and derived, and a defect feature containing three-dimensional geometric information is extracted.

[0147] The target surface normal vector in the embodiments of the present application refers to a vector perpendicular to the tangent plane of a point on the target surface, which is calculated and derived through target surface gradient data (such as the height gradient pre-encoding information carried by the encoded light field in S101). The surface morphology mutation region is identified through the direction change of the normal vector, and the slope and steepness of the defect are quantified in combination with the modulus value of the normal vector, thereby providing key parameters for the extraction of three-dimensional geometric features of the defect.

[0148] In some embodiments, based on the target surface gradient pre-encoding information carried by the encoded light field, the normal vector of each point on the target surface is calculated through gradient vector cross multiplication, and the three-dimensional geometric features such as the depth and slope of the defect are extracted in combination with the direction and modulus value changes of the normal vector, thereby supplementing the three-dimensional information dimension of the defect feature set.

[0149] Specifically, the gradient information of the target surface is parsed from the encoded light field data; the target surface normal vector is calculated, for each pixel point on the target surface, a gradient vector (Gx, Gy, -1) is constructed, the unit normal vector of the point is obtained through vector normalization processing, and the direction parameter and modulus value change of the normal vector are recorded; the normal vector anomaly of the defect region is identified, and the normal vector anomaly region is determined as the defect region; the three-dimensional geometric features of the defect are quantified, the slope of the defect is calculated based on the direction difference of the normal vector, the depth of the defect is derived in combination with the calibration parameter (depth = gradient value x pixel pitch), the defect depth, slope, and normal vector direction information are integrated as the defect feature containing three-dimensional geometric information, and are associated with the corresponding defect region and view and merged into the defect feature set.

[0150] S400, multi-view information integration is performed on the defect feature set to obtain comprehensive defect features;

[0151] The multi-view information integration in the embodiments of the present application refers to integrating different defect features into unified comprehensive defect features based on the defect feature set and corresponding view parameters. By eliminating the spatial redundancy and view deviation of multi-view features, the complementary correlation of cross-view features is mined, and the robustness and recognition of defect features are improved.

[0152] On the basis of the above-mentioned embodiments, as another optional embodiment, the multi-view information integration is performed on the defect feature set to obtain comprehensive defect features, which includes:

[0153] S401, dimension reduction and splicing are performed on the defect feature set to construct a unified feature vector;

[0154] The unified feature vector in the embodiments of the present application refers to a standardized feature vector formed by compressing the defect feature set and splicing according to a preset rule. The unified feature vector includes multi-dimensional information of defects, such as spatial morphological features, three-dimensional geometric features, and gray texture features, and the vector dimension needs to adapt to the input requirements of the subsequent correlation model. By converting the multi-view and multi-type discrete defect features into vector data with unified format and controllable dimension, the model input interference caused by data format difference is eliminated.

[0155] In some embodiments, the defect features of each view in the defect feature set are compressed to eliminate data redundancy, and then the features of each view after dimension reduction are spliced according to a preset rule to generate a unified feature vector with unified format and dimension adaptation.

[0156] Specifically, the defect features of each view are uniformly converted into multi-dimensional original feature vectors (such as spatial morphological features 307 dimensions, three-dimensional geometric features 410 dimensions, and gray texture features 307 dimensions) to ensure that the feature dimensions of different views are consistent; the multi-dimensional original features are reduced in dimension, and principal components are selected from high to low contribution rate to a preset minimum dimension (such as 256 dimensions); the preset minimum dimension feature vectors of each view after dimension reduction are spliced in turn according to the deployment order of the multi-view imaging device to form a unified feature vector, and a view parameter identifier such as the number of views and the camera pose index of each view is attached to the head of the vector.

[0157] S402, a multi-view feature correlation model is established, and based on the unified feature vector, a comprehensive defect feature is output.

[0158] The multi-view feature correlation model in the embodiments of the present application is used to mine the cross-view feature correlation in the unified feature vector and output a comprehensive defect feature, and can adopt a double-layer architecture of feature-level fusion and decision-level fusion. The feature-level fusion learns the spatial mapping relationship of different view features, and the decision-level fusion performs probability synthesis on the defect judgment results of each view. Through training of labeled defect samples, it is ensured that the output comprehensive defect feature has spatial consistency and reliable judgment.

[0159] In some embodiments, a double-layer multi-view feature correlation model of feature-level fusion and decision-level fusion is constructed to take a unified feature vector as input, learn cross-view feature correlation and synthesize defect judgment results through the model, and output comprehensive defect features including defect type, three-dimensional parameters and comprehensive confidence.

[0160] Specifically, based on the unified feature vector, the vector is split into multiple single-view feature streams according to view partition, each single-view feature stream extracts single-view core features through an independent convolution branch, and then the correlation between different feature streams is mined through a cross-stream attention mechanism to output feature-level fusion features; the feature-level fusion features are subjected to decision-level fusion, mapped to judgment probabilities of each defect type through a fully connected layer, and the probabilities are weighted and synthesized in combination with weight coefficients of each view (such as high-resolution view weight 0.3 and low-resolution view weight 0.2) to output comprehensive defect features.

[0161] S500, based on the comprehensive defect features, performing defect judgment and parameter calculation on the target surface to output defect detection results.

[0162] The defect detection results in the embodiments of the present application can include defect identification, detection time, type level, accurate physical parameters and associated multi-view image index, and the defect position can be marked through a visual interface.

[0163] Specifically, based on the comprehensive defect features, the effective defects are screened through a preset defect judgment threshold, and the misjudgment features with insufficient confidence are removed; the calibration parameters are called to convert the pixel parameters of the effective defects into actual physical parameters; the defect levels are divided according to the defect types and the physical parameters to generate the defect detection results.

[0164] The preset defect judgment threshold can include comprehensive confidence ≥ 0.8; the actual physical parameters include actual depth and actual area; and the defect levels include slight, moderate and severe.

[0165] Reference Figure 3 The embodiments of the present application also provide a surface defect detection system based on curved surface coding and multi-view fusion, based on the surface defect detection method based on curved surface coding and multi-view fusion provided in the embodiments, wherein the system comprises:

[0166] A curved surface coding illumination system is configured to generate and project a coded light field, and carry target surface height gradient pre-encoding information through curvature adaptation and field of view adaptation.

[0167] A multi-view imaging module comprises an industrial camera, a calibration device and a synchronous triggering unit, the calibration device is configured to solve camera internal and external parameters, the synchronous triggering unit is configured to realize timing alignment, and the industrial camera is configured to acquire reflection images to generate multi-dimensional view data sets.

[0168] An image preprocessing module is configured to perform geometric registration, grayscale calibration, and noise filtering based on calibration parameters, and output a standardized image.

[0169] A defect feature extraction module is configured to extract and generate a defect feature set.

[0170] A multi-view integration module is configured to splice the defect feature set into a unified feature vector, and output comprehensive defect features through multi-view feature correlation fusion.

[0171] A result output module is configured to filter effective defects based on the comprehensive defect features, convert pixel parameters into physical parameters, divide defect levels, and generate a defect detection result.

[0172] In some embodiments, the image preprocessing module includes a geometric registration unit, a grayscale calibration unit, and a noise filtering unit.

[0173] In some embodiments, the defect feature extraction module includes a block planning unit, a three-dimensional correction unit, a light component separation unit, and a three-dimensional feature calculation unit.

[0174] In some embodiments, the multi-view integration module includes a dimension reduction splicing unit and a feature correlation unit.

[0175] Based on the above embodiments, as another optional embodiment, the curved surface encoding illumination system includes:

[0176] A phase shift encoding unit is configured to adapt the number of phase shift steps to the target surface curvature, and generate a sinusoidal phase shift light field carrying target surface height gradient information.

[0177] A pseudo-random encoding unit is configured to construct a pseudo-random encoding matrix, and verify whether the matrix meets the field of view requirements.

[0178] A light field adaptation unit is configured to perform partition splicing adjustment on the pseudo-random encoding matrix, and then convert the adjusted matrix into an encoding light field and project it onto the target surface.

[0179] A field of view verification unit is configured to simulate the projection effect of the encoding light field on the target surface through optical simulation.

[0180] Specifically, the curvature distribution data of the target surface to be detected is collected through edge curvature analysis of the preprocessed image, and is synchronously transmitted to each unit of the curved surface encoding illumination system.

[0181] In the curved surface encoding light system, the phase shift encoding unit adapts the number of phase shift steps to generate a sinusoidal phase shift stripe light field carrying high gradient pre-encoding information, and adjusts the light field direction to match the target shape; the pseudo-random encoding unit generates a one-dimensional sequence adapted to the camera resolution and defect resolution, and constructs a two-dimensional pseudo-random matrix through the diagonal line cyclic assignment method; the field of view checking unit checks through optical simulation, and if it does not pass, it feeds back the adjustment sequence period until it passes the check; the light field adaptation unit performs regional splicing on the checked matrix to convert it into a physical pseudo-random encoding light field; the phase shift and pseudo-random encoding light field are projected onto the target surface through the optical projection component.

[0182] In the multi-view imaging module, the calibration device fixes the checkerboard calibration plate at the center of the target detection area, controls the industrial camera arranged in a ring to collect calibration images, solves the camera internal and external parameters and stores them; the synchronous triggering unit sets the timing deviation according to the camera exposure delay in the calibration parameters and the light field projection response time, triggers the light field projection and the camera to synchronously collect the reflection image, and generates a multi-dimensional view image data set according to the view angle associated camera pose and imaging time.

[0183] The image preprocessing module receives the multi-dimensional view image data set. Among them, the geometric registration unit corrects the lens distortion based on the calibration parameters, unifies the images to the world coordinate system and reduces the defect space deviation through resampling; the gray scale calibration unit establishes a gray scale mapping matrix for each view angle through a standard gray scale plate, and corrects and reduces the gray scale deviation pixel by pixel; the noise filtering unit uses Gaussian filtering to suppress speckle noise and uses median filtering to eliminate motion noise, and outputs standardized images.

[0184] In the defect feature extraction module, the block planning unit divides the detection area into sub-apertures; the three-dimensional correction unit establishes a coordinate conversion model for each sub-area, restores the planar image and fuses it into a panoramic image; the light component separation unit separates the diffuse reflection component and the specular reflection component according to the phase difference threshold, and performs gray scale normalization; the three-dimensional feature calculation unit analyzes the gradient information to construct the gradient vector (Gx, Gy, -1), deduces the unit normal vector and identifies the abnormal area of the normal vector, quantizes the defect slope and the actual depth, and generates a defect feature set.

[0185] The multi-view integration module receives the defect feature set, wherein the dimension reduction splicing unit reduces the high-dimensional features of each view angle, splices them in the order of camera deployment, and adds a view angle identifier to form a unified feature vector; the feature correlation unit outputs a comprehensive defect feature containing defect type, three-dimensional parameters and comprehensive confidence through pre-training model, feature level fusion and decision level fusion.

[0186] The result output module filters effective defects based on the comprehensive confidence, converts pixel parameters to actual physical parameters based on the calibration parameters, divides defect levels, generates defect detection results, and simultaneously labels defect positions in the panoramic image and stores them in the database.

[0187] It should be noted that the system provided by the above embodiment is only exemplified by the above division of functional modules when realizing its functions. In actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the above described functions. In addition, the system and method embodiments provided by the above embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be described here.

[0188] The embodiment of the present application further provides a computer storage medium, which can store a plurality of instructions, the instructions being suitable for being loaded by a processor and executing the above-mentioned embodiment of a surface defect detection method based on curved surface coding and multi-view fusion. The specific execution process can be referred to the specific description of the above-mentioned embodiment, and will not be described here.

[0189] The present application also discloses an electronic device, which can include at least one processor, at least one communication bus, a user interface, at least one network interface, and a memory.

[0190] The communication bus is used to realize the connection and communication between the components.

[0191] The user interface can include a display screen (Display) and a camera (Camera). Optionally, the user interface can further include a standard wired interface and a wireless interface.

[0192] The network interface can optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0193] The processor can include one or more processing cores. The processor connects various parts within the server by various interfaces and lines, executes various functions of the server and processes data by running or executing instructions, programs, code sets or instruction sets stored in the memory, and calling data stored in the memory. Optionally, the processor can be implemented in at least one of a hardware form of a digital signal processing (DSP), a field-programmable gate array (FPGA), and a programmable logic array (PLA). The processor can be integrated with a combination of one or more of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing the content required to be displayed on the display screen; and the modem is used for processing wireless communication. It can be understood that the above-mentioned modem can also not be integrated into the processor, but can be realized by a separate chip.

[0194] The memory can include a random access memory (RAM) and a read-only memory (ROM). Optionally, the memory includes a non-transitory computer-readable storage medium. The memory can be used to store instructions, programs, codes, code sets or instruction sets. The memory can include a program storage area and a data storage area, wherein the program storage area can store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the above-mentioned various method embodiments, etc.; the data storage area can store data involved in the above-mentioned various method embodiments, etc. The memory can also be at least one storage device located away from the above-mentioned processor. As a computer storage medium, the memory can include an operating system, a network communication module, a user interface module, and an application program of a surface defect detection method based on curved surface coding and multi-view fusion.

[0195] In the electronic device, the user interface is mainly used to provide an interface for the user to input, and obtain data input by the user; and the processor can be used to call an application program of a surface defect detection method based on curved surface coding and multi-view fusion stored in the memory, which, when executed by one or more processors, causes the electronic device to perform the method of one or more of the above embodiments. It should be noted that, for the foregoing method embodiments, in order to simply describe, they are all described as a combination of a series of actions, but those skilled in the art should know that the present application is not limited to the order of the actions described, because according to the present application, certain steps can be performed in other order or at the same time. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0196] In the above embodiments, the description of each embodiment has its own focus, and the parts not described in detail in a certain embodiment can be referred to the related description of other embodiments.

[0197] In the several embodiments provided by the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic. The division of the units is only a logical function division. There can be another division during actual implementation. For example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between the units can be indirect couplings or communication connections through some interfaces, devices or units, and can be electrical or other forms.

[0198] The units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0199] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of software functional unit.

[0200] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable memory. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a memory and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server or a network device, etc.) to execute all or part of the steps of the embodiments of the present application. The aforementioned memory includes: a U disk, a mobile hard disk, a magnetic disk or an optical disk, and various media that can store program codes.

[0201] The above are only exemplary embodiments of the present disclosure, and cannot limit the scope of the present disclosure. That is, any equivalent changes and modifications made in accordance with the teachings of the present disclosure are still within the scope of the present disclosure. Other embodiments of the present disclosure will be readily apparent to those skilled in the art upon considering the specification and practicing the true principles of the present disclosure.

[0202] The present application is intended to cover any variations, uses or adaptive changes of the present disclosure that follow the general principles of the present disclosure and include common knowledge or conventional technical means in the technical field not recorded in the present disclosure. The specification and examples are only considered as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A surface defect detection method based on curved surface coding and multi-view fusion, characterized in that, The method comprises the following steps: Constructing a curved surface encoding light system to generate an encoding light field and project it to a target surface; Synchronously collecting reflection images of the encoding light field by a multi-view imaging device to obtain a multi-dimensional view image dataset; Separating light components from the multi-dimensional view image dataset to extract a defect feature set; Integrating multi-view information of the defect feature set to obtain comprehensive defect features; Based on the comprehensive defect features, defect determination and parameter calculation are performed on the target surface to output a defect detection result; The step of separating light components from the multi-dimensional view image dataset to extract a defect feature set comprises the following steps: performing high-order curved surface block planning on the multi-dimensional view image dataset to determine sub-aperture detection regions; performing three-dimensional geometric correction on each of the sub-aperture detection regions; and fusing corrected sub-region images to extract multi-resolution defect features; The step of constructing a curved surface encoding light system to generate an encoding light field and project it to a target surface comprises the following steps: by controlling the phase shift step number to modulate the phase distribution of the encoding light field, the encoding light field carries pre-encoded information of the gradient of the target surface, and the phase shift step number is adapted to the curvature of the target surface; The phase shift step number is dynamically determined according to the radius of curvature of the target surface, wherein a first step length phase shift is used for a high-curvature region with a curvature radius ≤ 5 mm, and a second step length phase shift is used for a low-curvature region with a curvature radius ≥ 50 mm; The step of separating light components from the multi-dimensional view image dataset to extract a defect feature set further comprises the following steps: based on the phase characteristics of the encoding light field, diffuse reflection and specular reflection light components are separated; based on the gradient of the target surface, the normal vector of the target surface is calculated and deduced to extract defect features containing three-dimensional geometric information; The defect features containing three-dimensional geometric information include defect slope and depth calculated based on the normal vector of the target surface; The step of integrating multi-view information of the defect feature set to obtain comprehensive defect features comprises the following steps: dimension reduction and splicing are performed on the defect feature set to construct a unified feature vector; a multi-view feature correlation model is established, and based on the unified feature vector, the comprehensive defect features are outputted; Wherein, based on the unified feature vector, the vector is split into multiple single-view feature streams according to view partitioning, each single-view feature stream extracts single-view core features through an independent convolution branch, then the correlation between different single-view feature streams is mined through a cross-stream attention mechanism to output feature-level fusion features; decision-level fusion is performed on the feature-level fusion features, the decision probability of each defect type is mapped through a fully connected layer, the probabilities are weighted and synthesized by combining the weight coefficients of each view to output the comprehensive defect features.

2. The surface defect detection method based on curved surface coding and multi-view fusion according to claim 1, characterized in that, The step of synchronously collecting reflection images of the encoding light field by a multi-view imaging device to obtain a multi-dimensional view image dataset comprises the following steps: By a calibration device, the internal and external parameters of the multi-view imaging device are calibrated to generate calibration parameters; Based on the calibration parameters, a synchronous triggering module is used to realize time sequence alignment of the projection of the encoding light field and the collection of the reflection images.

3. The surface defect detection method based on curved surface coding and multi-view fusion according to claim 2, characterized in that, The method further comprises the following steps: Based on the calibration parameters, geometric registration is performed on the multi-dimensional view image dataset; The multi-dimensional perspective image dataset after registration is subjected to gray calibration and noise filtering to eliminate the differences between perspectives.

4. A surface defect detection system based on curved surface coding and multi-view fusion, based on the surface defect detection method based on curved surface coding and multi-view fusion according to any one of claims 1-3, characterized in that, The method comprises the following steps: A curved surface encoding light system is used to generate and project an encoding light field, which carries pre-encoded information of the height gradient of the target surface through curvature adaptation and field of view adaptation. A multi-perspective imaging module comprises an industrial camera, a calibration device, and a synchronous triggering unit. The calibration device is used to solve the internal and external parameters of the camera. The synchronous triggering unit is used to realize timing alignment. The industrial camera is used to collect reflection images and generate multi-dimensional perspective data sets. An image preprocessing module is used to perform geometric registration, gray calibration, and noise filtering based on the calibration parameters and output standardized images.

5. The surface defect detection system based on curved surface coding and multi-view fusion according to claim 4, characterized in that, A defect feature extraction module is used to extract and generate a defect feature set. A multi-perspective integration module is used to splice the defect feature set into a unified feature vector and output comprehensive defect features through multi-perspective feature correlation fusion. A result output module is used to filter effective defects based on the comprehensive defect features, convert pixel parameters into physical parameters, divide defect levels, and generate defect detection results. The curved surface encoding light system comprises: A phase shift encoding unit is used to adapt the number of phase shift steps to the curvature of the target surface and generate a sinusoidal phase shift light field carrying the height gradient information of the target surface.

Images