Intelligent production inspection system based on big data

Abstract

The application discloses a production intelligent inspection system based on big data, and relates to the technical field of intelligent inspection, comprising: a data acquisition module, which is used for collecting visible orthogonal frame pairs, near-infrared orthogonal frame pairs and initial depth frames of a work area through a big data acquisition terminal; a polarization coefficient solving module, which is used for performing polarization coefficient mapping calculation on a first homologous coordinate polarization pixel pair of the visible orthogonal frame pairs to obtain a mirror surface polarization coefficient frame, and performing diffraction depolarization coefficient solving on a second homologous coordinate polarization pixel pair of the near-infrared orthogonal frame pairs to obtain a diffraction depolarization coefficient frame; a mask and difference value calculation module, which is used for performing coupling distortion region marking operation on the mirror surface polarization coefficient frame and the diffraction depolarization coefficient frame to obtain a coupling initial mask, and calculating mirror surface polarization coefficient difference values and diffraction depolarization coefficient difference values according to the mirror surface polarization coefficient frame and the diffraction depolarization coefficient frame, respectively. The application can remove coupling distortion pixels in real time in a heterogeneous data flood.

Description

Technical Field

[0001] This invention relates to the field of intelligent inspection technology, and in particular to a production intelligent inspection system based on big data. Background Technology

[0002] As high-speed arc welding production lines for automotive body-in-white and precision pressure vessels move towards 24-hour unmanned operation, welding robots simultaneously perform multiple tasks within a single workstation, including high-speed oscillating welding, short-cycle torch changing, and online self-inspection. In the working environment, a combination of factors such as molten spatter, high-frequency magnetic fields, and sudden changes in water vapor can instantaneously generate extremely low-probability coupling interference objects above the weld, such as a metal vapor mirror dome and a nano-water vapor diffraction network. These interference objects produce strong specular reflection and maintain polarization in the visible band, while exhibiting Bragg diffraction and polarization depolarization in the near-infrared band, leading to simultaneous overexposure and fringe distortion in visible-near-infrared dual-spectrum imaging. To ensure continuous production, the industry has widely deployed big data-driven inspection systems: extracting working condition features from hundreds of millions of cross-shift images and depth samples to support real-time assessment of weld centerline, torch posture, and trajectory deviation. However, existing systems mainly rely on single-band brightness thresholds, passband filtering, or static template comparisons, lacking a polarization and time-coordinated discrimination mechanism that matches ultra-low probability coupling interference. This makes it difficult to remove coupled distortion pixels in real time from heterogeneous data streams and maintain a geometric measurement accuracy of <0.05mm.

[0003] When a metal vapor mirror dome and a nano-water vapor diffraction mesh appear simultaneously in a high-speed welding scenario, the mirror polarization preservation index in the visible band and the diffraction depolarization index in the near-infrared band briefly overlap at the same pixel position. This causes the existing single-spectrum threshold and statistical filtering strategies in the big data inspection system to fail, making it impossible to locate the coupled distortion region in real time from tens of millions of frame streams and eliminate its interference with depth fitting. As a result, the three-dimensional geometric reconstruction error of the weld centerline is amplified cumulatively. Summary of the Invention

[0004] The purpose of this invention is to address the shortcomings of existing technologies in which it is difficult to remove coupled distorted pixels in real time from heterogeneous data deluge, and to propose a production intelligent inspection system based on big data.

[0005] To address the problems existing in the prior art, the present invention adopts the following technical solution:

[0006] A big data-based intelligent production inspection system includes:

[0007] The data acquisition module is used to collect visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames of the work area through a big data acquisition terminal.

[0008] The polarization coefficient calculation module is used to perform polarization-preserving coefficient mapping calculation on the first co-coordinate polarized pixel pair of the visible orthogonal frame pair to obtain the mirror polarization-preserving coefficient frame, and to perform diffraction depolarization coefficient calculation on the second co-coordinate polarized pixel pair of the near-infrared orthogonal frame pair to obtain the diffraction depolarization coefficient frame.

[0009] The mask and difference calculation module is used to perform coupling distortion region marking operations on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame to obtain the initial coupling mask, and calculate the mirror polarization preservation coefficient difference and the diffraction depolarization coefficient difference based on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, respectively.

[0010] The mask refinement module is used to perform dual threshold gating screening on the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient using the activated pixels of the coupled initial mask to obtain the refined mask;

[0011] The depth frame processing and point set construction module is used to selectively hole-fill the pixel depth values ​​of the initial depth frame according to the refined mask, generate the net depth frame, and generate the spatial point set matrix based on the net depth frame.

[0012] The centerline parameter solving module is used to construct a matrix equation based on the three-dimensional spatial coordinates of the spatial point set matrix, solve the matrix equation, and obtain the centerline parameter vector of the working area.

[0013] The inspection index determination module is used to generate inspection indexes for the work area based on the centerline parameter vector and net depth frame, and to determine whether the quality status of the work area is qualified based on the inspection indexes.

[0014] Preferably, when the data acquisition module performs the acquisition of visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames of the work area via the big data acquisition terminal, it includes:

[0015] Synchronization trigger signals are sent to the visible light subsystem, near-infrared subsystem, and structured light depth subsystem via the big data acquisition terminal;

[0016] The visible light system responds to the synchronous trigger signal and synchronously acquires two visible light band polarized light signals at a wavelength of 532nm with vibration directions of 0° and 90°. The two visible light band polarized light signals are converted into corresponding digital image frames. Orthogonal polarization correlation pairing is performed on the corresponding digital image frames to obtain visible orthogonal frame pairs.

[0017] The near-infrared subsystem responds to the synchronous trigger signal and synchronously acquires two near-infrared polarized light signals at a wavelength of 1064nm with vibration directions of 0° and 90°. The two near-infrared polarized light signals are converted into corresponding near-infrared digital image frames. Orthogonal polarization correlation pairing is performed on the corresponding near-infrared digital image frames to obtain near-infrared orthogonal frame pairs.

[0018] The structured light depth subsystem responds to the synchronization trigger signal, synchronously acquires depth information of the work area, and generates an initial depth frame.

[0019] The big data acquisition terminal is used to receive spectral line data, full frame image data and initial depth frame in real time.

[0020] Preferably, when the polarization coefficient calculation module performs polarization-preserving coefficient mapping calculation on the first co-coordinate polarization pixel pair of the visible orthogonal frame pair to obtain the mirror polarization-preserving coefficient frame, it includes:

[0021] For each pixel coordinate of a visible orthogonal frame pair, extract the pixel values ​​of frame V0 and frame V90, and construct the first co-polarized pixel pair based on the pixel values ​​of frame V0 and frame V90.

[0022] The degree of linear polarization of the first co-polarized pixel pair is calculated using the formula for calculating the degree of linear polarization.

[0023] Substitute the linear polarization degree into the preset polarization response model to obtain the mirror polarization preservation coefficient;

[0024] All the mirror polarization preservation coefficients are recombined according to their spatial positions to generate a mirror polarization preservation coefficient frame.

[0025] Preferably, when the polarization coefficient calculation module performs diffraction depolarization coefficient calculation on the second co-coordinate polarized pixel pair of the near-infrared orthogonal frame pair to obtain the diffraction depolarization coefficient frame, it includes:

[0026] For each pixel coordinate of a near-infrared orthogonal frame pair, extract the pixel values ​​of frame N0 and frame N90, and construct a second co-polarized pixel pair based on the pixel values ​​of frame N0 and frame N90.

[0027] The diffraction depolarization coefficient of the second co-polarized pixel pair is calculated using the diffraction depolarization coefficient calculation formula.

[0028] All diffraction depolarization coefficients are reorganized according to their spatial positions to generate a diffraction depolarization coefficient frame.

[0029] Preferably, when the mask and difference calculation module performs coupling distortion region marking operations on the mirror polarization-preserving coefficient frame and the diffraction depolarization coefficient frame to obtain the initial coupling mask, it includes:

[0030] Align the pixel coordinates of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame.

[0031] For the same coordinates of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, extract the mirror polarization preservation coefficient and the diffraction depolarization coefficient at the same coordinates.

[0032] When the mirror polarization preservation coefficient of the same coordinate is greater than the preset first threshold and the diffraction depolarization coefficient of the same coordinate is greater than the preset second threshold, the same coordinate is assigned a value of 1; otherwise, the same coordinate is assigned a value of 0.

[0033] All assignments with the same coordinates are integrated into a coupling initial mask according to their spatial location.

[0034] Preferably, when the mask and difference calculation module calculates the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient based on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, respectively, it includes:

[0035] Obtain the mirror polarization preservation coefficient frame and diffraction depolarization coefficient frame for the current acquisition cycle;

[0036] Obtain the mirror polarization preservation coefficient frame and diffraction depolarization coefficient frame for the next acquisition cycle;

[0037] The difference between the mirror polarization preservation coefficient frame of the current acquisition cycle and the mirror polarization preservation coefficient frame of the next acquisition cycle is calculated to obtain the mirror polarization preservation coefficient difference.

[0038] The difference between the diffraction depolarization coefficient frame of the current acquisition cycle and the diffraction depolarization coefficient frame of the next acquisition cycle is calculated to obtain the diffraction depolarization coefficient difference.

[0039] Preferably, when the mask refinement module performs dual-threshold gating screening on the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient using the activated pixels of the coupled initial mask to obtain the refined mask, it includes:

[0040] For an active pixel, if the difference in the mirror polarization preservation coefficient is greater than the first preset difference threshold and the difference in the diffraction depolarization coefficient is greater than the second preset difference threshold, the active pixel's activation mark will be retained; otherwise, the active pixel's activation mark will be canceled.

[0041] The active pixels with activation markers are reorganized into a refinement mask according to their spatial location.

[0042] Preferably, the depth frame processing and point set construction module, when performing selective hole-filling processing on the pixel depth values ​​of the initial depth frame based on the refined mask to generate a net depth frame, and generating a spatial point set matrix based on the net depth frame, includes:

[0043] Extract the set of all mask pixel coordinates from the refined mask;

[0044] If there are depth pixel coordinates in the initial depth frame that belong to the mask pixel coordinate set, then the pixel depth value of the depth pixel coordinate is set to invalid.

[0045] The pixel depth values ​​that were not marked as invalid in the initial depth frame are reorganized according to their spatial positions to generate a net depth frame;

[0046] Extract the two-dimensional coordinates and depth values ​​of the effective pixels in the net depth frame;

[0047] Substitute the two-dimensional coordinates and depth values ​​of the effective pixels into the inverse perspective projection transformation formula to calculate the three-dimensional spatial coordinates of the effective pixels.

[0048] Combine camera extrinsic parameters to transform 3D spatial coordinates to the world coordinate system;

[0049] Assemble the three-dimensional spatial coordinates in the world coordinate system into a spatial point set matrix.

[0050] Preferably, the centerline parameter solving module, when performing the operation of constructing a matrix equation based on the three-dimensional spatial coordinates of the spatial point set matrix, solving the matrix equation, and obtaining the centerline parameter vector, includes:

[0051] The three-dimensional coordinate points are projected onto the main plane of the work area, and the design matrix and observation vector are constructed based on the projection points of the main plane.

[0052] Construct matrix equations based on design matrices and observation vectors;

[0053] The matrix equation is solved using the least squares method to obtain the centerline parameter vector.

[0054] Preferably, when the inspection index determination module executes the inspection index for the work area based on the centerline parameter vector and net depth frame, and determines whether the quality status of the work area is qualified based on the inspection index, it includes:

[0055] The fitting residuals of effective pixels are calculated based on the centerline parameter vector and the net depth frame.

[0056] The fitting residuals were determined as the inspection indicators for the work area.

[0057] Determine whether the inspection indicators exceed the preset robot process tolerance:

[0058] If the inspection indicators exceed the preset robot process tolerance, the quality status of the work area will be marked as unqualified.

[0059] If the inspection indicators do not exceed the preset robot process tolerance, the quality status of the work area will be marked as qualified.

[0060] Compared with the prior art, the beneficial effects of the present invention are:

[0061] 1. In this invention, orthogonal polarization imaging at 532nm visible light and 1064nm near-infrared light is first triggered synchronously, and then combined with a structured light depth camera to form three time-consistent data streams. By calculating the linear polarization degree of pixels with the same coordinates and mapping it to a mirror polarization-preserving coefficient frame, and calculating the diffraction depolarization coefficient frame for near-infrared pixels, the system can quantify the polarization-preserving effect of the metal vapor mirror dome on the visible band and the depolarization-diffraction effect of the nano-water vapor diffraction network on the near-infrared band at the original frame level. This directly transforms the two extremely low-probability interferences into a two-dimensional numerical field that can be thresholded. This stage lays the foundation for subsequent accurate removal of distorted pixels.

[0062] 2. In this invention, a coupling initial mask is generated based on the overlap of the same coordinate thresholds of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame. Then, consistency gating is performed by comparing the coefficient differences between two adjacent frames, retaining only the active pixels that are stable in terms of lifetime, and finally obtaining the refined mask. The refined mask accurately marks the position where visible overexposure and near-infrared diffraction fringes truly overlap, significantly reducing the probability of noise and artifacts entering, and solving the defect of traditional single-band or static template methods that are difficult to identify coupling interference in real time.

[0063] 3. Selectively hollow out the initial depth frame using a refined mask to generate a net depth frame. Then, convert the effective pixels into a set of world coordinate points and solve the centerline parameter vector using the least squares method. Calculate the fitting residual based on this. If the residual exceeds the robot's process tolerance, it is automatically marked as unqualified. This not only restores the accuracy of the weld centerline that is masked by coupling distortion at the geometric level, but also completes the closed loop of self-inspection and alarm for the production robot at the business level, solving the problems of difficult distortion positioning and difficult eccentricity determination in existing technologies. Attached Figure Description

[0064] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0065] Figure 1 This is a functional module diagram of a big data-based intelligent production inspection system provided in an embodiment of the present invention. Detailed Implementation

[0066] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0067] Example: This example provides a big data-based intelligent production inspection system. See [link to example]. Figure 1 Specifically, including:

[0068] The data acquisition module is used to collect visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames of the work area through a big data acquisition terminal.

[0069] Specifically, in production operation scenarios, when high-end arc welding robots are working, molten spatter and ultra-fine water vapor in the shielding gas can form metal vapor mirror film domes and nano-water vapor diffraction networks at the lens, causing interference such as blurred visible band images and abnormal near-infrared radiation. Conventional single-mode detection is unable to capture minute defects. Acquiring visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames utilizes the difference in response of the visible and near-infrared bands to different interferences, combined with depth information to construct multi-dimensional data. Visible orthogonal frame pairs can capture macroscopic surface features, near-infrared orthogonal frame pairs can penetrate some interference to reflect the internal state, and initial depth frames provide spatial dimensional information. The three work together to provide basic data support for subsequent calculation of polarization preservation coefficients and diffraction depolarization coefficients, accurately restore the geometric details of the weld center, and identify quality defects, thus solving the technical problem that traditional methods cannot detect ultra-fine defects.

[0070] In detail, the metal vapor mirror film dome refers to the high-temperature vapor that evaporates from the nickel-silicon alloy molten pool due to the instantaneous overshoot of the welding arc. This vapor rapidly sublimates at the cooling turbulent interface 3-5 cm above the molten pool, forming a hemispherical metal film with a thickness of less than 1 μm and a mirror-like surface. It has extremely high visible light reflectivity and almost completely maintains the incident polarization, while radiating strongly in the mid-infrared band with its own temperature. The nano water vapor diffraction network is formed on the surface of the aforementioned mirror film dome. Under the influence of the high-frequency magnetic field and electrostatic polarization of the welding machine, the surrounding <50 nm water vapor droplets self-assemble along the electric field lines to form a two-dimensional liquid bridge grid with a period of about 400-450 nm, which constitutes a Bragg diffraction grating. It produces obvious depolarization diffraction fringes in the near-infrared band, while being almost transparent to visible light. When the two are superimposed, the visible light is overexposed due to the mirror surface and the near-infrared is distorted due to the diffraction fringes, forming an extremely low probability fusion interference scene.

[0071] In an embodiment of the present invention, when the data acquisition module performs the task of acquiring visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames of the work area through a big data acquisition terminal, it includes:

[0072] Synchronization trigger signals are sent to the visible light subsystem, near-infrared subsystem, and structured light depth subsystem via the big data acquisition terminal;

[0073] The visible light system responds to the synchronous trigger signal and synchronously acquires two visible light band polarized light signals at a wavelength of 532nm with vibration directions of 0° and 90°. The two visible light band polarized light signals are converted into corresponding digital image frames. Orthogonal polarization correlation pairing is performed on the corresponding digital image frames to obtain visible orthogonal frame pairs.

[0074] Specifically, after receiving the synchronization trigger signal sent by the big data acquisition terminal, the visible light system initiates the synchronous acquisition process. Its internal optical sensor components detect the working area in the visible light band at a wavelength of 532nm. Through the equipped polarization beam splitter, it collects two polarized light signals with vibration directions of 0° and 90°, respectively. These two polarized light signals carry optical information of the working area at different polarization states at that wavelength. Subsequently, the visible light system uses a photoelectric conversion module to convert the two acquired visible light band polarized light signals into electrical signals, and further generates corresponding visible light digital image frames through analog-to-digital conversion. Each polarized light signal corresponds to one digital image frame. After acquiring the visible light digital image frames corresponding to the two visible light band polarized light signals with vibration directions of 0° and 90°, the two image frames are first preprocessed, including grayscale correction and noise filtering, to improve image quality. Based on the physical characteristics of orthogonal polarization, the pixels of the two image frames are traversed, and for each pixel position, the pixel values ​​of the 0° polarized image frame and the 90° polarized image frame are extracted. By calculating the degree of orthogonal polarization, the 0° polarized image frame is used as the reference image frame. The brightness difference between the two images is obtained by subtracting the brightness value of the 90° polarized image frame from the brightness value of the first image frame. The brightness values ​​of the 0° polarized image frame and the 90° polarized image frame are then added together to obtain the sum of their brightness values. The brightness difference is then divided by the sum of their brightness values ​​to obtain the orthogonal polarization degree of that pixel location. This polarization degree is used as the new pixel value. In this way, the orthogonal polarization degree of all pixel locations in the image is calculated sequentially, and the orthogonal polarization degree of each location is used as the new pixel value to reconstruct a complete image, which is the visible orthogonal frame pair. This frame pair can accurately reflect the polarization characteristic distribution of the working area at a wavelength of 532nm, providing basic data for subsequent analysis.

[0075] The near-infrared subsystem responds to the synchronous trigger signal and synchronously acquires two near-infrared polarized light signals at a wavelength of 1064nm with vibration directions of 0° and 90°. The two near-infrared polarized light signals are converted into corresponding near-infrared digital image frames. Orthogonal polarization correlation pairing is performed on the corresponding near-infrared digital image frames to obtain near-infrared orthogonal frame pairs.

[0076] Specifically, the near-infrared subsystem responds to the big data acquisition terminal and initiates a synchronous acquisition process. Its internal near-infrared optical sensing components detect the working area in the 1064nm near-infrared band. Using a matching near-infrared polarization beam splitter, it acquires two near-infrared polarized light signals with vibration directions of 0° and 90° respectively. These signals carry optical information about the working area under different polarization states at this near-infrared wavelength. Next, the near-infrared subsystem uses a near-infrared photoelectric conversion module to convert the two acquired near-infrared polarized light signals into electrical signals. Then, through analog-to-digital conversion, it generates corresponding near-infrared digital image frames. Each polarized light signal corresponds to one near-infrared digital image frame. A specially adapted orthogonal polarization correlation pairing algorithm, such as a multi-scale phase-consistent polarization correlation algorithm, is called to process these two near-infrared digital image frames. Based on the orthogonal characteristics of near-infrared polarization, pixel values ​​of the 0° polarized near-infrared image frame and the 90° polarized near-infrared image frame are extracted pixel by pixel, establishing a correlation between them. The calculation result is used as new pixel values ​​to construct the correlated image matrix. After all pixel association operations and assignments are completed, a new image is generated that integrates near-infrared orthogonal polarization information. This is the near-infrared orthogonal frame pair, which can accurately present the near-infrared polarization feature distribution of the working area at a wavelength of 1064nm, providing basic data for subsequent working area analysis based on the near-infrared band.

[0077] The structured light depth subsystem responds to the synchronization trigger signal, synchronously acquires depth information of the work area, and generates an initial depth frame.

[0078] Specifically, upon receiving a synchronization trigger signal, the structured light depth subsystem immediately initiates the depth information acquisition process. The structured light transmitter projects an encoded structured light pattern onto the work area. This pattern contains encoded information such as specific stripes, grids, or random spots, and its spatial distribution is unique and identifiable. The depth camera captures the structured light pattern reflected from the surface of the work area at a frequency synchronized with the transmitter. Due to the undulating shape of the work area surface, the reflected structured light pattern will be distorted. This distortion is directly related to the actual depth of each point in the work area. The depth camera converts the captured light signal into an electrical signal, and then converts it into a digital image signal through an analog-to-digital converter. Using a decoding algorithm, the distortion of the structured light pattern in the digital image signal is analyzed. By calculating the offset of each feature point in the encoded pattern, combined with known calibration parameters such as the relative position and angle of the transmitter and camera, and based on the principle of triangulation, the depth value in the three-dimensional spatial coordinates corresponding to each pixel in the work area is accurately calculated. Finally, the depth values ​​of all pixels are integrated and arranged to generate an initial depth frame according to a specific image format. This initial depth frame completely records the surface depth information of the working area, providing basic data for subsequent depth data processing and analysis.

[0079] The system utilizes a big data acquisition terminal to receive visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames in real time.

[0080] Specifically, the big data acquisition terminal is equipped with a high-speed data receiving module and a distributed caching architecture. It establishes real-time data transmission channels with the visible light subsystem, near-infrared subsystem, and structured light depth subsystem via high-speed communication links such as gigabit Ethernet interfaces and fiber optic channels. Once each subsystem completes data acquisition, it immediately sends visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames to the acquisition terminal as data streams. The acquisition terminal employs multi-threaded parallel processing technology, simultaneously opening multiple independent data receiving threads. Each thread corresponds to one subsystem data transmission channel, ensuring that different types of data can be received synchronously without interference. To address bandwidth pressure during massive data transmission, the acquisition terminal uses data compression algorithms to compress the received data in real time. For example, it uses the JPEG2000 compression standard based on discrete cosine transform for image data and octree encoding compression for depth data, effectively reducing the amount of data transmitted. Simultaneously, the acquisition terminal internally constructs a distributed cache queue, temporarily storing the received data in multiple cache nodes to avoid frame loss due to data processing delays. In addition, the acquisition terminal is equipped with a data verification mechanism. It uses a cyclic redundancy check algorithm to verify the integrity of the received data. If an error is detected, a retransmission request is immediately triggered to ensure the accuracy and integrity of the data. Finally, the acquisition terminal integrates the compressed, verified, and cached visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames into a unified format big data packet, which is stored in a local storage array or uploaded to a cloud server, providing a reliable data foundation for subsequent big data analysis and processing.

[0081] The polarization coefficient calculation module is used to perform polarization-preserving coefficient mapping calculation on the first co-coordinate polarized pixel pair of the visible orthogonal frame pair to obtain the mirror polarization-preserving coefficient frame, and to perform diffraction depolarization coefficient calculation on the second co-coordinate polarized pixel pair of the near-infrared orthogonal frame pair to obtain the diffraction depolarization coefficient frame.

[0082] In an embodiment of the present invention, when the polarization coefficient calculation module performs polarization-preserving coefficient mapping calculation on the first co-coordinate polarization pixel pair of visible orthogonal frame pairs to obtain a mirror polarization-preserving coefficient frame, it includes:

[0083] For each pixel coordinate of a visible orthogonal frame pair, extract the pixel values ​​of frame V0 and frame V90, and construct the first co-polarized pixel pair based on the pixel values ​​of frame V0 and frame V90.

[0084] Specifically, all pixel coordinates in the visible orthogonal frame pair are traversed by scanning row by row and column by column. For each traversed pixel coordinate, the pixel value at the corresponding coordinate position is extracted from the V0 frame and V90 frame in the visible orthogonal frame pair. The V0 frame pixel value and V90 frame pixel value extracted under the same pixel coordinate are combined and stored according to a specific data structure to form the first same-coordinate polarization pixel pair. The first same-coordinate polarization pixel pair contains the light intensity information of two orthogonal polarization directions at the same position in the working area.

[0085] The linear polarization degree of the first co-polarized pixel pair is calculated using the linear polarization degree calculation formula, which is as follows:

[0086]

[0087] In the formula, Indicates pixel coordinates Linear polarization degree at the location, Indicates pixel coordinates Pixel value of frame V0 at the location. Indicates pixel coordinates V90 frame pixel value at the location;

[0088] Specifically, the linear polarization degree calculation formula is a mathematical tool used to quantify the linear polarization characteristics of light. In this invention, for polarized pixel pairs consisting of pixel values ​​at the same coordinate in orthogonal visible frames, the formula determines the degree of linear polarization of light at that coordinate location by calculating pixel values ​​obtained under two orthogonal polarization directions. First, pixel values ​​at the same pixel coordinate in different orthogonal polarization frames are obtained. The absolute value of the difference between these two values ​​is taken, then the sum of these two values ​​is calculated. Finally, the absolute value of the difference is divided by the sum, and the result is the linear polarization degree at that pixel coordinate location. This calculation method converts the polarization characteristics of light into quantifiable numerical values, facilitating the analysis and processing of polarization information in the work area. This formula, as prior art, is used in this invention to implement the linear polarization degree calculation function, providing basic data support for subsequent work area detection based on polarization features.

[0089] Substitute the linear polarization degree into the preset polarization response model to obtain the mirror polarization preservation coefficient;

[0090] Specifically, the pre-defined polarization response model is established based on the physical laws of light-matter interaction. Its construction process is as follows: First, theoretical analysis determines the main factors affecting the polarization-preserving characteristics of the mirror, including the incident angle, material refractive index, and surface roughness. Linear polarization is identified as a key input parameter. Then, for standard test samples of typical materials involved in the target operating scenario, such as metals and ceramics, high-precision polarization measurement equipment is used under controlled experimental conditions to collect the measured values ​​of the linear polarization and corresponding mirror polarization-preserving coefficients of samples under different incident angles and surface treatment processes. Using these experimental data, the least squares method is used to fit the functional relationship between the linear polarization and the mirror polarization-preserving coefficient. This functional relationship is the initial polarization response model. To improve the model's adaptability, a material correction coefficient and an environmental compensation factor are further introduced. The former is used to adjust for the differences in the influence of different materials on the polarization response, while the latter is used to compensate for the interference of environmental factors such as temperature and humidity. The final expression of the pre-defined polarization response model is: Mirror polarization-preserving coefficient = Basic response function (linear polarization) × Material correction coefficient × Environmental compensation factor. In practical applications, the calculated linear polarization degree is substituted into the model as input. First, the basic polarization-preserving coefficient is calculated through the basic response function. Then, the basic polarization-preserving coefficient is corrected by querying the corresponding material correction coefficient according to the material type of the work area. Next, the environmental compensation factor is determined by combining the work environment monitoring data for secondary correction. The final output result is the specular polarization-preserving coefficient that takes into account material properties and environmental factors. This coefficient reflects the ability of the work area surface to maintain the polarization state of incident light, providing key parameters for subsequent work quality assessment based on polarization characteristics.

[0091] All the mirror polarization preservation coefficients are recombined according to their spatial positions to generate a mirror polarization preservation coefficient frame.

[0092] Specifically, following the pixel coordinate order of the visible orthogonal frame pairs, the mirror polarization-preserving coefficients calculated for each coordinate position are sequentially stored in a blank matrix of the same size as the visible orthogonal frame pair, ensuring that the mirror polarization-preserving coefficients maintain a spatial correspondence with the original pixel coordinates. After all mirror polarization-preserving coefficients are filled, the matrix is ​​directly converted into image format data to generate a mirror polarization-preserving coefficient frame. This frame completely preserves the spatial distribution information of the mirror polarization-preserving coefficients at each location in the working area, providing an intuitive data carrier for subsequent analysis.

[0093] In an embodiment of the present invention, when the polarization coefficient calculation module performs diffraction depolarization coefficient calculation on the second co-coordinate polarized pixel pair of near-infrared orthogonal frame pairs to obtain a diffraction depolarization coefficient frame, it includes:

[0094] For each pixel coordinate of a near-infrared orthogonal frame pair, extract the pixel values ​​of frame N0 and frame N90, and construct a second co-polarized pixel pair based on the pixel values ​​of frame N0 and frame N90.

[0095] The diffraction depolarization coefficient of the second co-polarized pixel pair is calculated using the following formula:

[0096]

[0097] In the formula, Indicates pixel coordinates The diffraction depolarization coefficient at the location Indicates pixel coordinates The pixel value of frame N0 at the location. Indicates pixel coordinates N90 frame pixel value at the location, These are the coordinates of near-infrared orthogonal frame pairs;

[0098] All diffraction depolarization coefficients are reorganized according to their spatial positions to generate a diffraction depolarization coefficient frame.

[0099] Specifically, the formula for calculating the diffraction depolarization coefficient is used to measure the change in polarization characteristics of near-infrared light after diffraction on the surface of the working area. Taking the pixel values ​​of near-infrared frames N0 and N90 at the same pixel coordinates, first calculate the absolute value of the difference between the two, then calculate their sum. Dividing the absolute value of the difference by the sum yields the diffraction depolarization coefficient at that location, reflecting the strength of surface diffraction depolarization and providing data for subsequent analysis.

[0100] Specifically, when processing near-infrared orthogonal frame pairs, their pixel coordinates are traversed one by one. For each coordinate, the pixel values ​​of the corresponding near-infrared N0 and N90 frames are extracted to form a second in-coordinate polarization pixel pair. Next, the diffraction depolarization coefficient of this pixel pair is calculated. First, the absolute value of the difference between the pixel values ​​of the N0 and N90 frames is calculated, then the sum of the two is calculated, and the coefficient is obtained by dividing the absolute value of the difference by the sum. After traversing all coordinates, the diffraction depolarization coefficients of each coordinate are rearranged according to their original spatial positions and combined to generate a diffraction depolarization coefficient frame for subsequent near-infrared polarization characteristic analysis.

[0101] The mask and difference calculation module is used to perform coupling distortion region marking operations on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame to obtain the initial coupling mask, and calculate the mirror polarization preservation coefficient difference and the diffraction depolarization coefficient difference based on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, respectively.

[0102] Specifically, in the welding operation environment, the superposition of interference such as metal vapor mirror film dome and nano water vapor diffraction network will cause abnormal correlation deviations in the polarization characteristic detection data of visible and near-infrared bands, such as the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame. These areas where the data is abnormal due to the fusion of multiple interferences are called coupling distortion areas. The initial coupling mask is the result obtained by processing the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame.

[0103] Specifically, the mirror polarization preservation coefficient frame is the result of polarization preservation coefficient mapping calculation for the same-coordinate polarized pixel pairs in the visible orthogonal frame pair. Its pixel value reflects the quantitative data of polarization state preservation characteristics in the visible band of the working area under interference such as metal vapor mirror film domes, and is used to capture polarization anomalies caused by mirror-like interference in the visible band. The diffraction depolarization coefficient frame is generated by solving the diffraction depolarization coefficient for the same-coordinate polarized pixel pairs in the near-infrared orthogonal frame pair. Its pixel value reflects the quantitative information of the degree of polarization state deviation caused by interference such as nano-water vapor diffraction nets in the near-infrared band, and is used to identify polarization anomalies caused by diffraction-like interference in the near-infrared band. Together, they serve as the basic data carrier for analyzing multi-band polarization interference in the working area and detecting quality defects.

[0104] In an embodiment of the present invention, the mask and difference calculation module, when performing coupling distortion region marking operations on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame to obtain the initial coupling mask, includes:

[0105] Align the pixel coordinates of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame.

[0106] For the same coordinates of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, extract the mirror polarization preservation coefficient and the diffraction depolarization coefficient at the same coordinates.

[0107] When the mirror polarization preservation coefficient of the same coordinate is greater than the preset first threshold and the diffraction depolarization coefficient of the same coordinate is greater than the preset second threshold, the same coordinate is assigned a value of 1; otherwise, the same coordinate is assigned a value of 0.

[0108] All assignments with the same coordinates are integrated into a coupling initial mask according to their spatial location.

[0109] Specifically, since both the mirror polarization-preserving coefficient frame and the diffraction depolarization coefficient frame correspond to the spatial information of the working area, the pixel coordinates of the two frames are first strictly matched to ensure that pixels at the same spatial location have the same coordinates in both frames. For the same coordinates in the two frames, the mirror polarization-preserving coefficient in the mirror polarization-preserving coefficient frame and the diffraction depolarization coefficient in the diffraction depolarization coefficient frame are extracted sequentially. A preset first threshold and a second threshold are introduced. These two thresholds are determined based on a large amount of experimental data and the analysis of the normal polarization characteristics of the working area, and are used to determine whether the coordinate position exists. In the case of coupling distortion, when the mirror polarization preservation coefficient of a certain coordinate is greater than the first threshold and the diffraction depolarization coefficient is greater than the second threshold, it is determined that there is coupling distortion at that position, and the coordinate is assigned a value of 1; otherwise, if these two conditions are not met, it is determined that there is no coupling distortion, and the coordinate is assigned a value of 0; according to the original spatial position relationship between the two frames, the assignment results of all the same coordinates are integrated, and these 0 and 1 assignments are arranged into a matrix with the same size as the original frame according to the corresponding coordinates. This matrix is ​​the initial coupling mask, which can intuitively mark the position where there is coupling distortion in the working area.

[0110] In an embodiment of the present invention, the mask and difference calculation module, when performing the calculation of the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient based on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame respectively, includes:

[0111] Obtain the mirror polarization preservation coefficient frame and diffraction depolarization coefficient frame for the current acquisition cycle;

[0112] Obtain the mirror polarization preservation coefficient frame and diffraction depolarization coefficient frame for the next acquisition cycle;

[0113] The difference between the mirror polarization preservation coefficient frame of the current acquisition cycle and the mirror polarization preservation coefficient frame of the next acquisition cycle is calculated to obtain the mirror polarization preservation coefficient difference.

[0114] The difference between the diffraction depolarization coefficient frame of the current acquisition cycle and the diffraction depolarization coefficient frame of the next acquisition cycle is calculated to obtain the diffraction depolarization coefficient difference.

[0115] Specifically, the mirror polarization-preserving coefficient frame and the diffraction depolarization coefficient frame of the working area in the current acquisition cycle are obtained. These two frames record the spatial distribution information of the surface of the working area in terms of mirror polarization preservation and diffraction depolarization characteristics in the current cycle, respectively. The mirror polarization-preserving coefficient frame and the diffraction depolarization coefficient frame corresponding to the next acquisition cycle are then obtained. Based on the principle of one-to-one correspondence between pixel coordinates, the difference between the mirror polarization-preserving coefficient frame of the current acquisition cycle and the mirror polarization-preserving coefficient frame of the next acquisition cycle is calculated. That is, for each pixel coordinate position, the mirror polarization-preserving coefficient of that coordinate in the next acquisition cycle is subtracted from the current acquisition cycle. The mirror polarization preservation coefficients corresponding to the coordinates are calculated by arranging the results of all pixel coordinates according to their original spatial positions to form mirror polarization preservation coefficient difference data. This data reflects the changes in the mirror polarization preservation characteristics of the working area within two acquisition cycles. For the diffraction depolarization coefficient frames of the current acquisition cycle and the next acquisition cycle, the difference between the diffraction depolarization coefficients of the next acquisition cycle and the current acquisition cycle is calculated for each coordinate based on the pixel coordinates. These differences are then integrated according to their spatial positions to obtain the diffraction depolarization coefficient difference, which is used to reflect the changes in the diffraction depolarization characteristics of the working area between two acquisition cycles.

[0116] The mask refinement module is used to perform dual threshold gating screening on the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient using the activated pixels of the coupled initial mask to obtain the refined mask;

[0117] In an embodiment of the present invention, the mask refinement module, when performing dual-threshold gating screening of the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient using the activated pixels of the coupled initial mask to obtain the refined mask, includes:

[0118] For an active pixel, if the difference in the mirror polarization preservation coefficient is greater than the first preset difference threshold and the difference in the diffraction depolarization coefficient is greater than the second preset difference threshold, the active pixel's activation mark will be retained; otherwise, the active pixel's activation mark will be canceled.

[0119] The active pixels with activation markers are reorganized into a refinement mask according to their spatial location.

[0120] Specifically, the active pixels in the initial coupling mask are identified, i.e., pixels with a value of 1 in the initial coupling mask. These pixels correspond to locations in the working area initially determined to have coupling distortion. For each active pixel, its corresponding value in the difference between the mirror polarization preservation coefficients and the difference between the diffraction depolarization coefficients is extracted. A first preset difference threshold and a second preset difference threshold are introduced. These two thresholds are determined based on a reasonable analysis of the changes in the characteristics of the working area and the actual application requirements, and are used to further accurately screen out coupling distortion locations that truly have significant characteristic changes. When the difference between the mirror polarization preservation coefficients of a certain active pixel is greater than the first preset difference threshold, and its difference between the diffraction depolarization coefficients and the difference between the mirror polarization preservation coefficients is greater than the first preset difference threshold, the coupling distortion location is further identified. When the difference exceeds the second preset threshold, the location corresponding to the active pixel is determined to have not only coupling distortion but also significant characteristic changes within the preceding and following acquisition cycles, and the activation mark of the active pixel is retained. Conversely, if neither of these conditions is met, it indicates that although the location corresponding to the active pixel is initially determined to have coupling distortion, the characteristic changes are not obvious, possibly due to noise or other non-critical factors, and the activation mark of the active pixel is canceled. After completing the screening and judgment of all active pixels, the active pixels that still have activation marks are collected, and they are rearranged and combined according to their spatial position relationship in the working area. The coordinates and activation states of these pixels are integrated to generate a refined mask. Compared with the initial coupling mask, this refined mask removes interfering pixels with insignificant characteristic changes and more accurately marks the locations in the working area with significant coupling distortion and obvious characteristic changes.

[0121] The depth frame processing and point set construction module is used to selectively hole-fill the pixel depth values ​​of the initial depth frame according to the refined mask, generate the net depth frame, and generate the spatial point set matrix based on the net depth frame.

[0122] In an embodiment of the present invention, the depth frame processing and point set construction module, when performing selective hole-filling processing on the pixel depth values ​​of the initial depth frame according to the refined mask to generate a net depth frame, and generating a spatial point set matrix based on the net depth frame, includes:

[0123] Extract the set of all mask pixel coordinates from the refined mask;

[0124] If there are depth pixel coordinates in the initial depth frame that belong to the mask pixel coordinate set, then the pixel depth value of the depth pixel coordinate is set to invalid.

[0125] The pixel depth values ​​that were not marked as invalid in the initial depth frame are reorganized according to their spatial positions to generate a net depth frame;

[0126] Specifically, the refined mask is parsed, and all mask pixel coordinates marked as valid (i.e., active markers) are extracted. These coordinates are organized into a specific set, representing the locations in the work area that require hole-filling. Each depth pixel coordinate in the initial depth frame is checked one by one to determine if it belongs to the previously extracted set of mask pixel coordinates. If a depth pixel coordinate is in this set, it means that the location needs hole-filling, and its corresponding pixel depth value is marked as invalid (e.g., using a specific numerical value or identifier to indicate that the depth value is unusable). After checking and marking all pixel coordinates in the initial depth frame, the pixel depth values ​​that are not marked as invalid are filtered out. These valid depth values ​​are rearranged and combined according to their spatial relationships in the initial depth frame to generate the net depth frame. The net depth frame removes the depth information of the locations marked as needing hole-filling in the refined mask, retaining only the valid depth data.

[0127] Extract the two-dimensional coordinates and depth values ​​of the effective pixels in the net depth frame;

[0128] Substituting the two-dimensional coordinates and depth values ​​of the effective pixels into the inverse perspective projection transformation formula, the three-dimensional spatial coordinates of the effective pixels are calculated. The inverse perspective projection transformation formula is as follows:

[0129]

[0130] In the formula, These are the three-dimensional spatial coordinates of the effective pixels. These are the two-dimensional coordinates of the effective pixels. It is the depth value of the effective pixels. , It's the camera's focal length. , These are the coordinates of the camera's principal point;

[0131] Combine camera extrinsic parameters to transform 3D spatial coordinates to the world coordinate system;

[0132] Specifically, when generating the spatial point set matrix, the two-dimensional coordinates of the effective pixels are first extracted from the net depth frame, that is, the row and column positions of pixels that are not marked as invalid within the frame. The depth value is then extracted, which corresponds to the depth information of the working area. Next, using the inverse perspective projection transformation method, combined with the pre-calibrated focal length and principal point coordinates of the camera (the principal point coordinates are the reference point positions on the camera's imaging plane), the two-dimensional coordinates and depth values ​​of the effective pixels are converted into three-dimensional spatial coordinates. This process is based on the geometric principles of camera imaging, and the position of the pixel in three-dimensional space is restored through inverse calculation. Combined with the camera extrinsic parameters, that is, the parameters describing the position and attitude of the camera in the world coordinate system, the obtained three-dimensional spatial coordinates are transformed into a unified world coordinate system, so that the coordinates have absolute positional meaning in the actual scene.

[0133] Assemble the three-dimensional spatial coordinates in the world coordinate system into a spatial point set matrix.

[0134] Specifically, based on the spatial arrangement logic of the work area, and according to the original two-dimensional coordinate order of effective pixels in the net depth frame, the corresponding three-dimensional spatial coordinates in the world coordinate system are extracted sequentially to create a two-dimensional matrix structure. The rows and columns of the matrix correspond to the ordered arrangement dimensions of the spatial point set. Each three-dimensional spatial coordinate is used as an element of the matrix, and the matrix is ​​filled one by one in order from left to right, from top to bottom, or other orders that conform to the spatial distribution rules. During the filling process, it is ensured that the position of each three-dimensional spatial coordinate corresponds to the actual spatial position of the work area, so that the spatial point set matrix can accurately reflect the three-dimensional spatial distribution of effective pixels in the work area. In this way, all three-dimensional spatial coordinates in the world coordinate system are integrated and finally assembled into a spatial point set matrix.

[0135] The centerline parameter solving module is used to construct a matrix equation based on the three-dimensional spatial coordinates of the spatial point set matrix, solve the matrix equation, and obtain the centerline parameter vector of the working area.

[0136] In an embodiment of the present invention, the centerline parameter solving module, when performing the operation of constructing a matrix equation based on the three-dimensional spatial coordinates of the spatial point set matrix, solving the matrix equation, and obtaining the centerline parameter vector, includes:

[0137] The three-dimensional coordinate points are projected onto the main plane of the work area, and the design matrix and observation vector are constructed based on the projection points of the main plane.

[0138] Specifically, the process begins with projecting the 3D coordinate points. Based on the definition of the main plane of the work area (e.g., a pre-defined reference plane parallel or perpendicular to the work platform), a projection transformation algorithm is used to project each 3D spatial coordinate point in the spatial point set matrix perpendicularly onto this main plane, resulting in corresponding 2D projection points. These projection points retain the original 3D points' positional information along the main plane. If a straight line is used as the model requirement, the horizontal and vertical coordinates within the main plane are extracted for each projection point. The horizontal coordinates are then combined with a constant term used for the intercept of the fitted line, sequentially according to the order of the projection points, to generate a design matrix. Each row of the matrix corresponds to the coordinate transformation of a single projection point, thus establishing a linear relationship with the centerline parameters. For the observation vector, the coordinate values ​​of the corresponding dependent variables in the fitted model are selected from the projection points and arranged into column vectors strictly following the order of the projection points.

[0139] A matrix equation is constructed based on the design matrix and the observation vector, as follows:

[0140]

[0141] In the formula, It is a design matrix. It is the centerline parameter vector. It is the observation vector;

[0142] The matrix equation is solved using the least squares method to obtain the centerline parameter vector.

[0143] Specifically, after constructing the design matrix and observation vectors, a matrix equation is built based on their mathematical relationship. Since the design matrix describes the linear relationship between the projection points and the centerline parameters, and the observation vectors reflect the observation characteristics of the projection points, the centerline parameter vector is introduced to construct the matrix equation. The matrix equation embodies the intrinsic relationship between the design matrix, parameter vector, and observation vector. The least squares method determines the centerline parameter vector that best fits the equation by minimizing the sum of squared errors between the observed values ​​and the model predictions. In the solution process, linear algebra operations are used to process the matrix equation to calculate the parameter vector that satisfies the minimum error condition. This vector contains key parameters such as the direction and position of the centerline in the corresponding space.

[0144] The inspection index determination module is used to generate inspection indexes for the work area based on the centerline parameter vector and net depth frame, and to determine whether the quality status of the work area is qualified based on the inspection indexes.

[0145] In an embodiment of the present invention, the inspection index determination module, when executing the inspection index of the work area generated based on the centerline parameter vector and net depth frame, and determining whether the quality status of the work area is qualified according to the inspection index, includes:

[0146] Based on the centerline parameter vector and net depth frames Calculate the fitting residuals of the effective pixels, where the formula for calculating the fitting residuals is as follows:

[0147]

[0148] In the formula, It is the fitting residual. It is the number of valid pixels in the net depth frame. These are the two-dimensional coordinates of the effective pixels. It is the depth value of the effective pixels. It is a net depth frame. , , These are the components of the centerline parameter vector;

[0149] Specifically, calculating the effective pixel fitting residual is to quantify the degree of deviation between the effective pixel depth values ​​in the net depth frame and the theoretical model constructed based on the centerline parameter vector. The centerline parameter vector describes the spatial characteristics of the centerline of the work area. Based on this vector, a corresponding theoretical model can be constructed to derive the theoretical depth values ​​that effective pixels should have. By subtracting the actual depth values ​​of effective pixels in the net depth frame from the theoretical depth values ​​derived by the theoretical model, the fitting residual is obtained. This residual reflects the fitting effect of the theoretical model on the actual depth data, helps to determine whether the centerline parameter vector accurately fits the spatial characteristics of the work area, and can also be used to identify outliers in the depth data and evaluate data quality.

[0150] Specifically, in the fitting process, it is assumed that there exists a theoretical model determined by the centerline parameter vector, which is obtained through, for example... The form describes the relationship between the two-dimensional coordinates of effective pixels and the theoretical depth, while the actual depth value of effective pixels in the net depth frame is... Then, the deviation between the actual depth value of a single effective pixel and the theoretical value of the model can be expressed as: To comprehensively measure the overall deviation of all valid pixels, the deviation of each valid pixel needs to be processed. First, the deviation of each individual pixel is squared, which eliminates the mutual cancellation of positive and negative deviations and highlights the magnitude of the deviation. Then, the squared deviations of all valid pixels are summed to obtain the total sum of squared deviations, which is then divided by the number of valid pixels. The average value is then processed to obtain the squared average deviation. Finally, the square root of this square root is taken to make the dimensions of the result consistent with the depth value. This combination of calculations yields the result... It can reflect the root mean square deviation between the actual depth of effective pixels and the depth of the theoretical model, which meets the requirement of using fitting residuals to measure the model fitting effect.

[0151] The fitting residuals were determined as the inspection indicators for the work area.

[0152] Determine whether the inspection indicators exceed the preset robot process tolerance:

[0153] If the inspection indicators exceed the preset robot process tolerance, the quality status of the work area will be marked as unqualified.

[0154] If the inspection indicators do not exceed the preset robot process tolerance, the quality status of the work area will be marked as qualified.

[0155] Specifically, the calculated fitting residual is used as an inspection index reflecting the status of the work area. This index reflects the deviation between the theoretical model and the actual depth data, and is used to assess the quality consistency of the work area. Based on the process requirements of robot operation, a reasonable robot process tolerance is preset. This tolerance is the benchmark for judging whether the quality of the work area meets the standards. The inspection index corresponding to the fitting residual is compared with the preset robot process tolerance. If the inspection index exceeds the preset robot process tolerance, it indicates that the depth data of the work area deviates significantly from the theoretical model, and the quality does not meet the process requirements. The quality status of the work area is marked as unqualified. If the inspection index does not exceed the preset robot process tolerance, it indicates that the depth data of the work area fits the theoretical model well, and the quality meets the process standards. The quality status of the work area is marked as qualified. This completes the judgment of the quality status of the work area.

[0156] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A big data-based intelligent production inspection system, characterized in that, include: The data acquisition module is used to collect visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames of the work area through a big data acquisition terminal. The polarization coefficient calculation module is used to perform polarization-preserving coefficient mapping calculation on the first co-coordinate polarized pixel pair of the visible orthogonal frame pair to obtain the mirror polarization-preserving coefficient frame, and to perform diffraction depolarization coefficient calculation on the second co-coordinate polarized pixel pair of the near-infrared orthogonal frame pair to obtain the diffraction depolarization coefficient frame. The mask and difference calculation module is used to perform coupling distortion region marking operations on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame to obtain the initial coupling mask, and calculate the mirror polarization preservation coefficient difference and the diffraction depolarization coefficient difference based on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, respectively. The mask refinement module is used to perform dual threshold gating screening on the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient using the activated pixels of the coupled initial mask to obtain the refined mask; The depth frame processing and point set construction module is used to selectively hole-fill the pixel depth values ​​of the initial depth frame according to the refined mask, generate the net depth frame, and generate the spatial point set matrix based on the net depth frame. The centerline parameter solving module is used to construct a matrix equation based on the three-dimensional spatial coordinates of the spatial point set matrix, solve the matrix equation, and obtain the centerline parameter vector of the working area. The inspection index determination module is used to generate inspection indexes for the work area based on the centerline parameter vector and net depth frame, and to determine whether the quality status of the work area is qualified based on the inspection indexes.

2. The intelligent production inspection system based on big data according to claim 1, characterized in that, When the data acquisition module performs the acquisition of visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames of the work area via the big data acquisition terminal, it includes: Synchronization trigger signals are sent to the visible light subsystem, near-infrared subsystem, and structured light depth subsystem via the big data acquisition terminal; The visible light system responds to the synchronous trigger signal and synchronously acquires two visible light band polarized light signals at a wavelength of 532nm with vibration directions of 0° and 90°. The two visible light band polarized light signals are converted into corresponding digital image frames. Orthogonal polarization correlation pairing is performed on the corresponding digital image frames to obtain visible orthogonal frame pairs. The near-infrared subsystem responds to the synchronous trigger signal and synchronously acquires two near-infrared polarized light signals at a wavelength of 1064nm with vibration directions of 0° and 90°. The two near-infrared polarized light signals are converted into corresponding near-infrared digital image frames. Orthogonal polarization correlation pairing is performed on the corresponding near-infrared digital image frames to obtain near-infrared orthogonal frame pairs. The structured light depth subsystem responds to the synchronization trigger signal, synchronously acquires depth information of the work area, and generates an initial depth frame. The system utilizes a big data acquisition terminal to receive visible orthogonal frame pairs, near-infrared orthogonal frame pairs, and initial depth frames in real time.

3. The intelligent production inspection system based on big data according to claim 1, characterized in that, When the polarization coefficient calculation module performs polarization-preserving coefficient mapping calculation on the first co-coordinate polarization pixel pair of the visible orthogonal frame pair to obtain the mirror polarization-preserving coefficient frame, it includes: For each pixel coordinate of a visible orthogonal frame pair, extract the pixel values ​​of frame V0 and frame V90, and construct the first co-polarized pixel pair based on the pixel values ​​of frame V0 and frame V90. The degree of linear polarization of the first co-polarized pixel pair is calculated using the formula for calculating the degree of linear polarization. Substitute the linear polarization degree into the preset polarization response model to obtain the mirror polarization preservation coefficient; All the mirror polarization preservation coefficients are recombined according to their spatial positions to generate a mirror polarization preservation coefficient frame.

4. The intelligent production inspection system based on big data according to claim 1, characterized in that, When the polarization coefficient calculation module performs diffraction depolarization coefficient calculation on the second co-coordinate polarized pixel pair of near-infrared orthogonal frame pairs to obtain the diffraction depolarization coefficient frame, it includes: For each pixel coordinate of a near-infrared orthogonal frame pair, extract the pixel values ​​of frame N0 and frame N90, and construct a second co-polarized pixel pair based on the pixel values ​​of frame N0 and frame N90. The diffraction depolarization coefficient of the second co-polarized pixel pair is calculated using the diffraction depolarization coefficient calculation formula. All diffraction depolarization coefficients are reorganized according to their spatial positions to generate a diffraction depolarization coefficient frame.

5. The intelligent production inspection system based on big data according to claim 1, characterized in that, When the mask and difference calculation module performs coupling distortion region marking operations on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame to obtain the initial coupling mask, it includes: Align the pixel coordinates of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame. For the same coordinates of the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame, extract the mirror polarization preservation coefficient and the diffraction depolarization coefficient at the same coordinates. When the mirror polarization preservation coefficient of the same coordinate is greater than the preset first threshold and the diffraction depolarization coefficient of the same coordinate is greater than the preset second threshold, the same coordinate is assigned a value of 1; otherwise, the same coordinate is assigned a value of 0. All assignments with the same coordinates are integrated into a coupling initial mask according to their spatial location.

6. The intelligent production inspection system based on big data according to claim 1, characterized in that, The mask and difference calculation module, when performing the calculation of the difference between the mirror polarization preservation coefficient and the difference between the diffraction depolarization coefficient based on the mirror polarization preservation coefficient frame and the diffraction depolarization coefficient frame respectively, includes: Obtain the mirror polarization preservation coefficient frame and diffraction depolarization coefficient frame for the current acquisition cycle; Obtain the mirror polarization preservation coefficient frame and diffraction depolarization coefficient frame for the next acquisition cycle; The difference between the mirror polarization preservation coefficient frame of the current acquisition cycle and the mirror polarization preservation coefficient frame of the next acquisition cycle is calculated to obtain the mirror polarization preservation coefficient difference. The difference between the diffraction depolarization coefficient frame of the current acquisition cycle and the diffraction depolarization coefficient frame of the next acquisition cycle is calculated to obtain the diffraction depolarization coefficient difference.

7. The intelligent production inspection system based on big data according to claim 1, characterized in that, The mask refinement module, when performing dual-threshold gating filtering on the difference between the mirror polarization preservation coefficients and the difference between the diffraction depolarization coefficients using the activated pixels of the coupled initial mask to obtain the refined mask, includes: For an active pixel, if the difference in the mirror polarization preservation coefficient is greater than the first preset difference threshold and the difference in the diffraction depolarization coefficient is greater than the second preset difference threshold, the active pixel's activation mark will be retained; otherwise, the active pixel's activation mark will be canceled. The active pixels with activation markers are reorganized into a refinement mask according to their spatial location.

8. The intelligent production inspection system based on big data according to claim 7, characterized in that, The depth frame processing and point set construction module, when performing selective hole-filling processing on the pixel depth values ​​of the initial depth frame based on the refined mask to generate a net depth frame, and generating a spatial point set matrix based on the net depth frame, includes: Extract the set of all mask pixel coordinates from the refined mask; If there are depth pixel coordinates in the initial depth frame that belong to the mask pixel coordinate set, then the pixel depth value of the depth pixel coordinate is set to invalid. The pixel depth values ​​that were not marked as invalid in the initial depth frame are reorganized according to their spatial positions to generate a net depth frame; Extract the two-dimensional coordinates and depth values ​​of the effective pixels in the net depth frame; Substitute the two-dimensional coordinates and depth values ​​of the effective pixels into the inverse perspective projection transformation formula to calculate the three-dimensional spatial coordinates of the effective pixels. Combine camera extrinsic parameters to transform 3D spatial coordinates to the world coordinate system; Assemble the three-dimensional spatial coordinates in the world coordinate system into a spatial point set matrix.

9. The intelligent production inspection system based on big data according to claim 1, characterized in that, The centerline parameter solution module, when constructing a matrix equation based on the three-dimensional spatial coordinates of the spatial point set matrix, solving the matrix equation, and obtaining the centerline parameter vector, includes: The three-dimensional coordinate points are projected onto the main plane of the work area, and the design matrix and observation vector are constructed based on the projection points of the main plane; Construct matrix equations based on design matrices and observation vectors; The matrix equation is solved using the least squares method to obtain the centerline parameter vector.

10. A production intelligent inspection system based on big data according to claim 8, characterized in that, The inspection index determination module, when executing the inspection indexes for the work area generated based on the centerline parameter vector and net depth frame, and determining whether the quality status of the work area is qualified based on the inspection indexes, includes: The fitting residuals of effective pixels are calculated based on the centerline parameter vector and the net depth frame. The fitting residuals were determined as the inspection indicators for the work area. Determine whether the inspection indicators exceed the preset robot process tolerance: If the inspection indicators exceed the preset robot process tolerance, the quality status of the work area will be marked as unqualified. If the inspection indicators do not exceed the preset robot process tolerance, the quality status of the work area will be marked as qualified.

Images