Visual closed-loop detection method, system and device for automatic filling and sealing production line

Abstract

The invention relates to a visual closed-loop detection method, system and device for an automatic filling and sealing production line in the technical field of flexible filling and sealing automatic production lines, a hyperspectral camera and a 3D visual camera are adopted for data acquisition of a calibration information base, respective mathematical models are established, a coupling model is constructed, and finally whether the product is a qualified product or not is judged. Through multi-dimensional data fusion of the hyperspectral camera and the 3D camera, synchronous detection of spectral features and three-dimensional structure parameters of the potting layer is realized, single-dimensional judgment of traditional manual visual inspection is upgraded to multi-modal composite analysis, the detection precision and reliability are greatly improved, and particularly, whether a potting adhesive of a potting product is cured or not can be accurately judged.

Description

Technical Field

[0001] This application relates to the field of flexible potting automated production line technology, specifically to a visual closed-loop inspection method, system and device for automated potting production lines, used to solve the problem of effective quality closed-loop inspection and control of potted products after glue curing during production line operation. Background Technology

[0002] Currently, the main function of potting production lines in the market is to execute product manufacturing processes, with product inspection primarily conducted manually. While ultrasonic and X-ray inspections are used for high-end products, these involve significant cost and environmental risks. Therefore, a comprehensive online closed-loop inspection device for potting quality on production lines is currently a technological gap in the market.

[0003] Therefore, the potting compound and related equipment industry currently lacks a closed-loop online potting quality inspection system based on 3D vision camera modules and hyperspectral cameras. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide an automated visual closed-loop inspection method, system, and device for potting production lines. It employs an innovative application combining a 3D vision camera module and a hyperspectral camera. The 3D vision camera module's image capture function measures the height and volume of the product. Data is coupled with calibrated height and volume data through recognition software to identify abnormalities, thereby detecting appearance defects such as potting overflow and insufficient volume. Utilizing the differences in spectral characteristics of the colloid before and after potting curing, and the integrated nature of the AB potting polymerization reaction surface, the hyperspectral camera captures spectral characteristic data of the surface layer (non-transparent potting material) or the refractive layer (transparent potting material) of the potting liquid. This data is compared and coupled with calibrated spectral characteristic data in the recognition software to detect physical property defects such as uncured potting. This completes a non-contact inspection closed loop for the potting production line, thereby more accurately detecting various quality defects in products passing through the line and improving product quality.

[0005] To achieve the above objectives, the present invention provides an automated visual closed-loop inspection method for a filling production line, comprising the following steps: Data acquisition for the S1 calibration database: A hyperspectral camera is used to continuously photograph the filling fluid sample at fixed time intervals to record the spectral curves. A 3D vision camera is used to scan the filling sample before and after filling to collect three-dimensional data. S2 The establishment of respective mathematical models: Spectral features are extracted using MATLAB tools, and characteristic wavelengths related to curing strength are screened through partial least squares discriminant analysis. Three-dimensional features are extracted using 3D point clouds, and the volume of the point cloud is calculated using voxelization or convex hull algorithm. The average roughness is calculated using the height distribution of points on the point cloud surface. S3 Coupling Model Construction: A linear weighted fusion model is used to describe the variable relationships between the above features.

[0006] Since the potting fluid will produce corresponding states during the solidification process, this application divides the solidification process of the potting fluid into three stages: liquid, gel and solid. A hyperspectral camera records the spectral curves of each stage, and the ENVI or Python library is used for data extraction and analysis to capture the changing trend of spectral characteristics over time.

[0007] To ensure stable experimental conditions and avoid interference from environmental fluctuations (such as temperature changes or humidity fluctuations) on spectral characteristics, thereby ensuring the accuracy and repeatability of the data, key environmental variables are controlled according to the specific type of potting solution when the hyperspectral camera acquires data. This is achieved by using an environmental control box or a constant temperature and humidity chamber to ensure stable experimental conditions and avoid environmental fluctuations.

[0008] To accurately extract spectral data, the spectral feature extraction method in step S2 is as follows: Establish a calibration model: Let the initial reflectance at the characteristic wavelength λ be , and the reflectance at time t be . Then the degree of curing of the spectrum is: Save and apply: Save the data model to the software calibration library and set it as the default; the software will automatically calculate the spectral curing degree during subsequent sample measurements.

[0009] To accurately extract 3D feature data, the 3D feature extraction method in step S2 is as follows: Establish a mathematical model: Calculate the point cloud volume using voxelization or convex hull algorithms, assuming an initial volume of... The volume at time t is The volume shrinkage rate is: in, (The volume can be considered a linear change); Calculate the average roughness using the height distribution of points on the point cloud surface: in, Let be the height coordinates of the i-th point on the point cloud surface at time t. Let (N) be the average height, and (N) be the number of surface points. Saving and Application: The relevant equations are saved into the software calibration library and set as the default. During subsequent sample measurements, the software automatically substitutes the data collected by the 3D vision camera for calculation to complete the extraction of the target's three-dimensional features.

[0010] To ensure the accuracy of data coupling, the linear weighted fusion model in step S3 is as follows: Assuming spectral curing degree Volume shrinkage rate Surface roughness With total curing degree The relationship is linear, and the weights are fitted using the least squares method. in: For feature weights; b is the bias term.

[0011] The present invention also provides an automated filling production line visual closed-loop inspection system, the system being used in the above-mentioned automated filling production line visual closed-loop inspection method, including a 3D vision camera module, a hyperspectral camera module and a central data processing host; The 3D vision camera module collects 3D data of various products on the production line; The hyperspectral camera module collects the spectral characteristics of the surface or refractive layer of the potting fluid used in various products at different stages before and after normal curing. The central data processing host uses data processing software to perform real-time analysis and coupling of data collected by the 3D vision camera module and the hyperspectral camera module, and provides real-time feedback through the display.

[0012] The present invention also provides a visual closed-loop inspection device for an automated filling production line. The device is used in the aforementioned visual closed-loop inspection system for an automated filling production line, and includes an automated filling production line, a 3D vision camera and a hyperspectral camera correspondingly installed on the automated filling production line, and a host computer connected to the 3D vision camera and the hyperspectral camera.

[0013] In summary, the present invention has the following beneficial effects: (1) By fusing multi-dimensional data from hyperspectral cameras and 3D cameras, the synchronous detection of spectral features and three-dimensional structural parameters of potting layers is realized, upgrading the traditional single-dimensional judgment of manual visual inspection to multi-modal composite analysis, greatly improving the detection accuracy and reliability, especially enabling accurate judgment on whether the potting adhesive of potting products has cured. (2) The detection data can be interacted with the MES system in real time through the host computer to establish a closed-loop feedback mechanism of potting quality and process parameters. When a curing abnormality is detected, the production line abnormality alarm function is automatically triggered, which makes it convenient for technicians to solve the abnormality of the potting production line in a timely manner. (3) With the updating and replacement of data processing software, the detection results and detection efficiency of the present invention will be further improved, and it has the potential for growth; (4) The participation of the QR code traceability system provides a basis for establishing a database of potting quality data for various potting products, and further provides a traceable quantitative basis for process improvement. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a schematic structural diagram of the vision closed-loop detection device for the automated potting production line of the present invention; Figure 2 is a schematic structural diagram of the vision closed-loop detection system for the automated potting production line of the present invention; Figure 3 is a flowchart of the vision closed-loop detection method for the automated potting production line of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0015] The following describes the preferred embodiments of the present invention in detail with reference to the accompanying drawings, so that the advantages and features of the present invention can be more easily understood by those skilled in the art, and thus a clearer and more definite definition of the protection scope of the present invention can be made.

[0016] As Figure 1 shown, an automated potting production line vision closed-loop detection device includes an automated potting production line 1, a 3D vision camera 3 and a hyperspectral camera 2 correspondingly arranged on the automated potting production line, and a host computer 4 connected to the 3D vision camera 3 and the hyperspectral camera 2.

[0017] After the product enters the automated potting production line 1, it first reaches the three-dimensional detection area. First, the 3D vision camera 3 performs QR code recognition to confirm the product model, and the corresponding calibration data (total curing degree ) is retrieved from the calibration information database by the host computer 4. Immediately, the 3D vision camera 3 measures and collects the three-dimensional data (volume, surface roughness of the potting surface) of the product, and transmits the data to the host computer 4 to calculate the target three-dimensional data (volume shrinkage rate and surface roughness ). After the data collection is completed, the product is transferred to the spectral detection station, and the hyperspectral camera 2 performs spectral characteristic data collection. The spectral characteristic data is also transmitted to the host computer 4 to calculate the spectral curing degree ). The host computer 4 substitutes all the data into the coupling model. If the three-dimensional data and the spectral data conform to the coupling model, it is determined to be qualified; otherwise, it is unqualified.

[0018] As Figure 2 shown, an automated potting production line vision closed-loop detection system is applied to the above detection device, and includes a 3D vision camera module, a hyperspectral camera module and a central data processing host; 3D vision camera module: QR code recognition, collecting three-dimensional data (potting depth, potting surface height, potting flatness, bubbles, cavities, etc.) before and after product potting; Hyperspectral camera module: Collects spectral data of the surface (non-transparent potting compound) or refractive layer (transparent potting compound) of various commonly used potting compounds before and after curing, as well as spectral data of the surface (non-transparent potting compound) or refractive layer (transparent potting compound) of each potting product under test at the test position; Central data processing host (host computer): Through installed data processing software such as spectral analysis, 3D data analysis, and comprehensive data analysis, it performs real-time analysis and coupling of data collected by the 3D vision camera module and hyperspectral camera module, and provides real-time feedback through the display.

[0019] like Figure 3 The automated filling production line visual closed-loop inspection method shown above, based on the above-mentioned inspection device and inspection system, includes the following steps: S1 Data collection for the calibration information database; S2 Establishment of respective mathematical models; S3 Construction of the coupled model; S4 Determination of whether it is a qualified product.

[0020] The following is a detailed explanation.

[0021] Data acquisition from the S1 calibration information database: (1) Hyperspectral camera data acquisition Stage Division: The solidification process of the potting fluid is scientifically divided into three key stages: liquid (not yet solidified), gel (in the process of solidification), and solid (completely solidified). The liquid stage is characterized by a fluid state and high fluidity; in the gel stage, the potting fluid begins to undergo cross-linking or solidification reactions, and its viscosity increases significantly; in the solid stage, it is completely solidified, forming a stable solid structure that cannot flow.

[0022] Data Acquisition: A hyperspectral camera was used to continuously photograph the filling solution sample at fixed time intervals (e.g., once per minute) to ensure coverage of the entire solidification process. The spectral curves of each stage were recorded, and data extraction and analysis were performed using specialized software (such as ENVI or Python libraries) to capture the changing trends of spectral characteristics over time.

[0023] Environmental control: During data collection, key environmental variables such as temperature and humidity were strictly controlled according to the product instructions, depending on the specific type of potting solution (e.g., epoxy resin or silicone-based potting solution). This included maintaining the temperature at 25℃±1℃ and the relative humidity at 50%RH±5%. Stable experimental conditions were ensured by using an environmental control chamber or constant temperature and humidity room, preventing interference from environmental fluctuations (such as temperature changes or humidity fluctuations) on spectral characteristics, thereby guaranteeing the accuracy and repeatability of the data.

[0024] (2) 3D vision camera data acquisition Sample Preparation: Prepare product samples with the potting solution completely solidified and in a qualified state, following the same standards and requirements as the hyperspectral camera. Ensure the sample is free of bubbles and defects in its solid state, and has a smooth surface, to ensure the accuracy of subsequent data acquisition. Specific steps include checking the degree of solidification, confirming that the sample is free of contamination, and placing it on a stable platform.

[0025] Data Acquisition: Based on actual application requirements, use a 3D vision camera to scan the prepared qualified potting product samples, acquiring relevant data (point cloud) such as volume and surface roughness. Ensure the camera is correctly calibrated, the ambient lighting is suitable, and the sample is kept still during the operation to obtain accurate measurement results. Data should be recorded and backed up in real time for subsequent analysis.

[0026] The establishment of the respective mathematical models for S2: (1) Hyperspectral camera Spectral Feature Extraction: The curing process of potting solutions (such as epoxy resin and silicone rubber) is essentially a functional group cross-linking reaction (such as the consumption of hydroxyl-OH and epoxy-COC groups), and its spectral features (such as reflectance / absorbance) change monotonically with the reaction progress. Using MATLAB, spectral features are extracted, and partial least squares discriminant analysis (PLS-DA) is used to screen for characteristic wavelengths that are strongly correlated with the degree of curing.

[0027] Establish a calibration model: Assume the initial reflectance at the characteristic wavelength λ is... The reflectance at time t is Then the degree of curing of the spectrum is: Save and apply: Save the data model to the software calibration library and set it as the default; the software will automatically calculate the spectral curing degree during subsequent sample measurements.

[0028] (2) 3D vision camera 3D feature extraction: During the curing process of potting fluid, volume shrinkage (a typical feature of thermosetting materials) and surface morphology changes (such as shrinkage lines and increased roughness) will occur. 3D point cloud can quantify these physical features.

[0029] Establish a mathematical model: Volume change rate (the most direct physical indicator) Calculate the point cloud volume using voxelization or the convex hull algorithm, assuming the initial volume is... The volume at time t is The volume shrinkage rate is: in, (The volume can be considered a linear change) Surface roughness (reflects curing uniformity): Calculate the average roughness using the height distribution of points on the point cloud surface: in, Let be the height coordinates of the i-th point on the point cloud surface at time t. Let (N) be the average height, and (N) be the number of surface points.

[0030] Saving and Application: The relevant equations are saved into the software calibration library and set as the default. During subsequent sample measurements, the software automatically substitutes the data collected by the 3D vision camera for calculation to complete the target 3D data acquisition.

[0031] S3 Coupling Model Construction: Degree of Curing (DOC) is a comprehensive indicator of the degree of chemical cross-linking (reflected by spectral features) and changes in physical structure (reflected by 3D point cloud). It is necessary to establish a quantitative relationship between DOC and spectral and 3D features.

[0032] Here, a linear weighted fusion model is used to describe the variable relationships among the above features: Assuming spectral curing degree Volume shrinkage rate Surface roughness With total curing degree The relationship is linear, and the weights are fitted using the least squares method: in: Feature weights (satisfying) (obtained by fitting experimental data); b is the bias term (correction for systematic error).

[0033] For example: For epoxy resin potting solutions, the true degree of cure is measured by DSC (differential scanning calorimetry), and the result is fitted as follows: (The weighting reflects that spectral characteristics contribute the most to the degree of curing, followed by volume shrinkage, with surface roughness serving as a secondary correction.) S4 determines whether the product is qualified: This testing device and system are used to extract the spectral and three-dimensional features of the potting samples to be tested, and the degree of spectral curing is calculated using corresponding formulas. Volume shrinkage rate and surface roughness Substitute the values ​​into the coupled model to obtain the results; then compare these results with the total degree of solidification of the product's potting fluid in the solid stage in the calibration library. The comparison is performed. If the deviation is within ±5%, it is considered qualified; otherwise, it is unqualified.

[0034] The following example illustrates this.

[0035] The initial curing time of a certain epoxy potting solution at room temperature (25℃) is 4-6 hours (gel state), and the complete curing time (solid state) is 24 hours. The total degree of curing after complete curing is... The value is 1 (100%), and its calibrated coupling model is: During testing, the spectral curing degree at a 25-hour time point was measured using this testing device. The volume shrinkage rate is 0.98. The surface roughness is 0.08 (8%). The thickness is 1.2 μm. Substituting this into the coupling model, we can obtain the current state of this epoxy potting solution. It is 0.981, which is consistent with the calibration. The deviation value is 1.9% < 5%, which is considered acceptable and confirms that it has been completely cured.

[0036] The above are all preferred embodiments of this application, and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape, principle and application direction of this application should be covered within the scope of protection of this application.

Claims

1. A visual closed-loop inspection method for an automated potting production line, characterized in that, Includes the following steps: Data acquisition for the S1 calibration database: A hyperspectral camera is used to continuously photograph the filling fluid sample at fixed time intervals to record the spectral curves. A 3D vision camera is used to scan the filling sample before and after filling to collect three-dimensional data. S2 The establishment of respective mathematical models: Spectral features are extracted using MATLAB tools, and characteristic wavelengths related to curing strength are screened through partial least squares discriminant analysis. Three-dimensional features are extracted using 3D point clouds, and the volume of the point cloud is calculated using voxelization or convex hull algorithm. The average roughness is calculated using the height distribution of points on the point cloud surface. S3 Coupling Model Construction: A linear weighted fusion model is used to describe the variable relationships between the above features.

2. The visual closed-loop detection method for automated filling production lines according to claim 1, characterized in that, In step S1, the solidification process of the potting fluid is divided into three stages: liquid, gel, and solid. A hyperspectral camera records the spectral curves of each stage, and the data is extracted and analyzed using ENVI or Python libraries to capture the changing trend of spectral features over time.

3. The visual closed-loop detection method for automated filling production lines according to claim 2, characterized in that, When acquiring data with a hyperspectral camera, key environmental variables are controlled according to the specific type of potting solution. By using an environmental control box or a constant temperature and humidity chamber, experimental conditions are kept stable and environmental fluctuations are avoided.

4. The visual closed-loop detection method for automated potting production lines according to claim 1, characterized in that, In step S2, the spectral feature extraction method is as follows: Establish a calibration model: Let the initial reflectance at the characteristic wavelength λ be , and the reflectance at time t be . Then the degree of curing of the spectrum is: Save and apply: Save the data model to the software calibration library and set it as the default; the software will automatically calculate the spectral curing degree during subsequent sample measurements.

5. The visual closed-loop detection method for automated potting production lines according to claim 4, characterized in that, In step S2, the three-dimensional feature extraction method is as follows: Establish a mathematical model: Calculate the point cloud volume using voxelization or convex hull algorithms, assuming an initial volume of... The volume at time t is The volume shrinkage rate is: in, (The volume can be considered a linear change); Calculate the average roughness using the height distribution of points on the point cloud surface: in, Let be the height coordinates of the i-th point on the point cloud surface at time t. Let (N) be the average height, and (N) be the number of surface points. Saving and Application: The relevant equations are saved into the software calibration library and set as the default. During subsequent sample measurements, the software automatically substitutes the data collected by the 3D vision camera for calculation to complete the extraction of the target's three-dimensional features.

6. The visual closed-loop detection method for automated filling production lines according to claim 1, characterized in that, In step S3, the linear weighted fusion model is as follows: Assuming spectral curing degree Volume shrinkage rate Surface roughness With total curing degree The relationship is linear, and the weights are fitted using the least squares method. in: For feature weights; b is the bias term.

7. A visual closed-loop inspection system for an automated potting production line, characterized in that, The system is used for the automated filling production line visual closed-loop inspection method as described in any one of claims 1-6, including a 3D vision camera module, a hyperspectral camera module, and a central data processing host; The 3D vision camera module collects 3D data of various products on the production line; The hyperspectral camera module collects the spectral characteristics of the surface or refractive layer of the potting fluid used in various products at different stages before and after normal curing. The central data processing host uses data processing software to perform real-time analysis and coupling of data collected by the 3D vision camera module and the hyperspectral camera module, and provides real-time feedback through the display.

8. A visual closed-loop inspection device for an automated potting production line, characterized in that, The device is used in the automated filling production line visual closed-loop inspection system as described in claim 7, including an automated filling production line, a 3D vision camera and a hyperspectral camera correspondingly installed on the automated filling production line, and a host computer connected to the 3D vision camera and the hyperspectral camera.

Images