Protein chip automatic quality inspection method based on machine vision and related equipment

Abstract

The invention relates to the technical field of machine vision detection, in particular to a protein chip automatic quality inspection method based on machine vision and related equipment.The method comprises the steps that parameter information and image data are obtained, communication connection is established, mechanical motion errors are compensated through datum point positioning, images are preprocessed and spliced, and defect types are recognized. And outputting an analysis conclusion and establishing closed-loop feedback and data tracing. According to the method, the protein chip quality inspection automation is realized, the detection precision and efficiency are improved, the problems of large error, low efficiency and data non-traceability in traditional manual detection are solved, and reliable quality control technical support is provided for industrial production.

Description

Technical Field

[0001] The present application relates to the field of machine vision detection technology, and in particular to a method for automatic quality inspection of protein chips based on machine vision and related equipment. Background Art

[0002] In the biomedical field, protein chips serve as core vehicles for disease diagnosis and drug screening. The quality of their surface dot arrays (such as the accuracy of protein dot placement and morphological integrity) directly impacts the reliability of subsequent test results. Traditionally, at the quality inspection station on protein chip spotting production lines, manual visual inspection is used: Using a desk lamp to illuminate the chip, inspectors visually inspect the array for defects such as missing, offset, or dimensional anomalies.

[0003] This method uses artificial vision to optically observe the dot array on the surface of a protein chip, relying on the human eye to discern defect characteristics (such as color, shape, and positional deviation). This method requires no additional equipment investment, has extremely low initial costs, and is suitable for small-scale, low-precision, temporary inspections. However, the limited resolution of the human eye, coupled with the subjective judgment criteria of different quality inspectors, leads to high rates of missed detections and false positives, making it unable to meet the quality control requirements of high-end protein chips.

[0004] Manual inspection of a single chip takes as long as 3-5 minutes, while modern spotting production lines can produce 10-20 chips per hour. This creates a production bottleneck, and prolonged visual inspection can lead to visual fatigue, further exacerbating efficiency declines. Test results rely solely on paper records, unable to automatically correlate with chip batches, production equipment parameters, and other information. This makes it difficult to trace the root cause of defects and violates the data traceability requirements of pharmaceutical manufacturing quality management regulations.

[0005] As the protein chip industry develops towards high-density and automation, traditional manual testing can no longer meet the needs of industrial quality inspection in cleanroom environments. A protein chip quality inspection technology that is highly accurate, automated, and data traceable is urgently needed to address the core issues of existing methods, such as large random errors, low efficiency, and lack of data management. Summary of the Invention

[0006] The purpose of this application is to provide a protein chip automatic quality inspection method and related equipment based on machine vision, aiming to solve the problems existing in traditional manual visual inspection methods, such as the detection accuracy is greatly affected by individual differences, the efficiency is low, and the data cannot be traced.

[0007] The first object of this application is to provide a method for automatic quality inspection of protein chips based on machine vision, comprising: Acquiring parameter information and image data related to protein chip quality inspection, wherein the parameter information includes communication setting parameters, chip-related information, scanning parameters, and dot matrix parameters, and the image data is collected by an external hardware system and transmitted to the software system; Establishing a communication connection between the external hardware system and the software system based on the acquired communication setting parameters; Obtaining the position information of multiple reference points on the protein chip, determining the actual coordinate offset of the protein chip in the X-axis and Y-axis directions by locating the reference points, and compensating for the mechanical motion error of the external hardware system in combination with the preset chip spacing parameters; Divide the image data of the intact protein chip into multiple regional sub-images, perform zooming, shrinking, and reloading operations on each of the regional sub-images, and simultaneously apply normalization and equalization image enhancement processing methods to improve the image contrast for more accurate subsequent image analysis; Based on an image feature matching algorithm, the pre-processed sub-images of the region are automatically spliced into a complete and clear protein chip image according to coordinate positions; Automatically analyzing the spliced complete protein chip image according to the lattice parameters, and identifying the defect type in the protein chip lattice using an integrated target detection algorithm; Output the analysis conclusion of the protein chip quality inspection, which includes the chip qualification judgment, the serial number of the unqualified small chip, the location of the unqualified point and the specific defect type.

[0008] By adopting the above technical solutions, it is possible to fully automate the protein chip quality inspection process based on machine vision technology, avoid random errors caused by individual differences in manual visual inspection, and significantly improve the consistency and accuracy of defect identification; through reference point positioning and mechanical motion error compensation, the coordinate offset of the hardware system can be effectively calibrated to ensure the position accuracy of the regional sub-images collected in blocks, and combined with image enhancement processing and feature matching and stitching algorithms, high-definition complete chip images can be generated, providing a high-quality data foundation for subsequent defect analysis; the target detection algorithm is used to automatically identify the type of dot matrix defects and output detailed analysis conclusions including qualification judgment, defect location and type, which not only improves the single-chip inspection efficiency to the minute level, adapting to the needs of high-speed production lines, but also realizes the digital management and traceability of inspection data, solving the problems of low efficiency and non-traceability of data in traditional methods, providing high-precision and automated technical support for quality control in the industrial production of protein chips, and meeting the quality inspection requirements of high-density protein chips in clean workshop environments.

[0009] In one possible embodiment of the present application, the steps of obtaining parameter information and image data related to protein chip quality inspection include: Acquiring original multispectral image data using an integrated multispectral imaging module, wherein the original multispectral image data includes visible light image data, infrared image data, and ultraviolet image data; Performing preliminary noise reduction and filtering processing on the original multispectral image data; Performing wavelet transform on the pre-processed original multispectral image data to decompose it into characteristic sub-images of different scales and directions; Performing principal component analysis on the characteristic sub-images of each spectral channel based on wavelet transform, calculating the covariance matrix of each characteristic sub-image and obtaining the principal components through eigenvalue decomposition; The extracted characteristic components of each spectral channel are weightedly fused, and weight coefficients are assigned according to the sensitivity and importance of each spectral channel to different features to generate a fused image containing information on the surface material composition, thickness and defect depth of the protein chip.

[0010] By adopting the above technical solution, an integrated multispectral imaging module is used to acquire raw image data containing visible light, infrared, and ultraviolet light. This allows the material properties of the protein chip surface to be captured from multiple dimensions (such as visible light reflecting morphology, infrared characterizing thickness, and ultraviolet identifying chemical groups). Combined with the decomposition capabilities of noise reduction filtering and wavelet transform for features at different scales, detailed information such as edges and textures of subtle defects can be effectively extracted. By screening key feature components through principal component analysis and performing weighted fusion, weights can be dynamically assigned based on the sensitivity of each spectral channel to different defect types, generating a high-dimensional fused image containing information on material composition, thickness, and defect depth. This not only solves the problem of insufficient information from single spectral detection, but also significantly improves the ability to identify three-dimensional defects such as depressions and protrusions, as well as hidden defects such as abnormal material composition. This provides multimodal data support including spatial structure and chemical properties for the subsequent automated analysis of defect types, improving the detection accuracy of protein chip surface defects from two-dimensional morphological recognition to a comprehensive determination of three-dimensional features and material composition, effectively meeting the complex quality inspection requirements of high-density, multifunctional protein chips.

[0011] In one possible implementation of the present application, the steps of performing principal component analysis on the characteristic sub-image of each spectral channel based on wavelet transform, calculating the covariance matrix of each characteristic sub-image and obtaining the principal component by eigenvalue decomposition include: Calculating the covariance matrix of each of the feature sub-images; Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and corresponding eigenvectors; Sort the eigenvectors in descending order of the eigenvalues, select the first several eigenvectors whose cumulative contribution rate reaches the set threshold, and the components corresponding to the first several eigenvectors are the extracted principal components; The projection coefficient of the principal component on each of the characteristic sub-images is used as the characteristic component of each of the characteristic sub-images.

[0012] By adopting the above technical solution, the high-dimensional data dimensionality reduction processing is performed on the characteristic sub-image after wavelet transformation with the help of principal component analysis. By calculating the covariance matrix and eigenvalue decomposition, the principal component with the highest contribution to defect recognition can be screened out from the complex multi-spectral features, and redundant information and noise interference (such as background fluorescence fluctuations, sensor inherent errors) can be effectively eliminated; by sorting by eigenvalue and selecting the principal components with a cumulative contribution rate reaching a set threshold, while retaining more than 95% of the key defect features, the data dimension can be reduced by 60%-80%, significantly improving the computational efficiency of subsequent feature fusion and defect analysis; the principal components can be combined in the characteristic The projection coefficient on the sub-image serves as a characteristic component, which can quantify the defect sensitivity under different spectral channels and scales (such as the high-weight response of the ultraviolet channel to chemical group anomalies), and provide a scientific basis for weight distribution for the weighted fusion of multi-spectral features. The final fusion image contains both the macroscopic morphological features of the protein chip surface and highlights microscopic defect information such as the depth of the depression and the difference in material composition, thereby increasing the signal-to-noise ratio of the defect feature by more than 3 times, effectively solving the problem of interference of irrelevant variables in multi-spectral data on defect identification, and providing a low-redundancy, high-discrimination feature data foundation for high-precision automated quality inspection.

[0013] In one possible implementation of the present application, the method further includes: Acquire 3D topographic data of the protein chip surface, calibrate local details through data registration algorithms, and form a high-precision 3D point cloud model covering the entire chip; Performing spatial registration on the three-dimensional point cloud model and the fused image corresponding to the original multispectral image data to establish a pixel-level spatial coordinate mapping relationship; Extracting surface curvature from the three-dimensional point cloud model, extracting spectral features of the defect area from the fused image, establishing a linear correlation model between the two through principal component analysis, and determining a mapping formula between surface curvature change and defect depth; Based on the mapping formula, three-dimensional depth inversion is performed on the suspicious area in the fused image to automatically identify stereo defects.

[0014] By adopting the above technical solution, and leveraging spatial registration and feature correlation analysis of 3D topography data and multispectral fusion images, the limitations of traditional 2D visual inspection can be overcome, enabling the precise identification of 3D defects on protein chip surfaces. The high-precision 3D point cloud model, obtained by fusion of structured light scanning and atomic force microscopy, accurately characterizes changes in chip surface curvature. Combined with the spectral characteristics of the defect area in the multispectral fusion image (such as material composition and thickness differences), a linear correlation model established using principal component analysis can quantitatively map surface curvature changes to defect depth. This enables the system to not only identify 3D geometric defects such as depressions and protrusions, but also detect hidden 3D defects caused by substrate damage and protein aggregation through the combined analysis of spectral characteristics and curvature changes. Pixel-level spatial registration ensures the precise correspondence between 3D structural information and spectral features, avoiding detection blind spots in single-modality data, improving the accuracy of 3D defect detection, and enabling automated quantitative assessment of defect depth. This provides multi-dimensional data support encompassing geometric morphology, material composition, and spatial structure for quality grading and process optimization of high-density protein chips, effectively addressing the industry challenge of difficult-to-detect complex 3D defects.

[0015] In one possible implementation of the present application, the method further includes: During multispectral imaging, continuous frames of fluorescence images of the same area are collected synchronously, and the fluorescence intensity decay rate between adjacent frames is calculated; When the fluorescence intensity decay rate exceeds a preset threshold, it is determined that fluorescence quenching occurs; Establish a spatial distribution map of fluorescence quenching, mark the location and degree of quenching areas, and generate a quenching risk heat map; Based on the quenching risk heat map, the excitation light parameters are dynamically adjusted for high-risk areas. Pulsed excitation light is used instead of continuous light, and the duty cycle and frequency are optimized. At the same time, the excitation light intensity is increased and the exposure time is shortened for areas with severe quenching. A feedback control algorithm is used to keep the fluorescence signal intensity in the target area within the linear response range of the detection system. For the multispectral image data of the quenched region, a joint spatiotemporal denoising algorithm is used. In the temporal dimension, the features of adjacent unquenched frames are used for interpolation and restoration. In the spatial dimension, the surface curvature information in the 3D topography data is combined to suppress artifacts. A generative adversarial network is then used to reconstruct the molecular distribution image of the quenched region. When it is detected that the average fluorescence decay rate of the entire batch of chips exceeds the set ratio, the early warning mechanism is automatically triggered and prompts to replace the fluorescent labeling reagent or adjust the biological sample pretreatment process. A quenching history database is established to analyze the quenching characteristics of different batches of chips and fluorescent dyes, providing parameter optimization suggestions for subsequent quality inspection tasks.

[0016] The above technical solution effectively addresses the problem of detection failure caused by fluorescence quenching of biomolecules. Real-time monitoring of fluorescence intensity decay rates in multispectral imaging and generation of quenching risk heat maps accurately locate high-risk areas. Dynamic adjustment of excitation light parameters (such as pulsed excitation and optimized duty cycle) stabilizes fluorescence signal intensity within the detection system's linear response range, improving the signal-to-noise ratio of the fluorescence signal in the target area and avoiding photobleaching caused by strong light exposure. A combined spatiotemporal denoising algorithm and a generative adversarial network are combined to reconstruct molecular distribution images in quenched regions. Three-dimensional topographic data is used to suppress artifacts and improve feature recovery in quenched regions, thereby enhancing defect recognition accuracy in fluorescence quenching scenarios. A monitoring and early warning mechanism for fluorescence decay rates across entire batches of chips automatically correlates with production parameters and provides recommendations for reagent replacement and process adjustments. The established quenching history database supports adaptive optimization of subsequent quality inspection parameters, reducing the incidence of fluorescence quenching in similar scenarios. This represents a technological upgrade from "passive detection" to "active compensation," ensuring signal stability and data reliability for fluorescently labeled protein chips in high-throughput detection and providing a systematic solution for quality control of quench-prone samples in biomedical testing.

[0017] In one possible embodiment of the present application, the step of compensating for the mechanical motion error of the external hardware system includes: Integrate a multi-dimensional sensor array on the motion platform of the external hardware system to collect vibration, displacement and posture data of the motion platform in real time during its motion; Build a dynamic error model based on the collected vibration, displacement and posture data, adjust motion parameters in real time, and compensate for positioning deviations caused by mechanical vibration and thermal drift; Combining real-time position feedback with image feature matching, the scanning path planning is dynamically optimized to ensure that the overlapping accuracy of adjacent sub-images meets the preset stitching requirements; To address image blur caused by motion, we use physical model-based image restoration technology combined with edge enhancement algorithm to improve the recognizability of defect features and ensure detection accuracy under complex motion conditions.

[0018] By adopting the above technical solution, the vibration, displacement and posture changes of the motion platform can be perceived in real time, and the positioning deviation caused by factors such as vibration and thermal drift during mechanical movement can be accurately compensated, solving the lag problem of traditional fixed parameter compensation; combining real-time position feedback and dynamic optimization of the scanning path can significantly improve the stitching accuracy of sub-images in adjacent areas, avoid image geometric distortion caused by motion deviation, and provide complete chip images with accurate position for subsequent defect analysis; for image blur caused by motion, image restoration and edge enhancement processing can effectively improve the clarity and recognizability of defect features, so that the system can still stably capture subtle defects in high-speed motion or complex vibration environments, ensuring the detection reliability of the hardware system under dynamic working conditions, and realizing high-precision automated quality inspection of protein chips.

[0019] In one possible embodiment of the present application, the steps after outputting the analysis conclusion of the protein chip quality inspection include: Establish closed-loop feedback with sample production equipment via industrial Ethernet; When the same type of defects are detected on several protein chips in succession, the defect position coordinates and corresponding scanning parameters are automatically extracted; Based on the spatial cluster analysis of the defect position coordinate distribution, determine whether the defect is caused by the positioning deviation of the spotting head or the abnormal injection parameters; If the sample head positioning deviation, adjust the sample head X / Y axis motion parameters; If the injection parameters of the sample head are abnormal, the injection pressure and pulse width of the sample head will be dynamically corrected; A quality inspection data traceability system is established to associate the defect type, three-dimensional morphology data, and multi-spectral characteristic components of each chip with the production batch and equipment parameters to form a quality control database.

[0020] By adopting the above technical solution, an automated closed-loop feedback mechanism can be established between the quality inspection system and the spotting production equipment, and the inspection results and production parameters can be correlated in real time. When similar defects occur continuously, the root cause of the defect (such as deviation in the positioning of the spotting head or abnormal injection parameters) can be accurately located through spatial clustering analysis, and the dynamic adjustment of the corresponding equipment parameters can be automatically triggered, realizing the automation of the entire process of "detection-analysis-calibration", reducing the occurrence of repetitive defects from the source; at the same time, the quality inspection data traceability system deeply associates defect information with production batches and equipment parameters, providing complete data chain support for subsequent process optimization, solving the problems of disconnection between production and inspection and difficulty in tracing the root causes of defects in traditional quality inspection, improving the quality control capability and intelligence level of the protein chip production process, and ensuring the stability of the production process and the consistency of product quality.

[0021] The second purpose of this application is to provide a protein chip automatic quality inspection system based on machine vision, which includes: Parameter information and image data acquisition module: acquires parameter information and image data related to protein chip quality inspection, wherein the parameter information includes communication setting parameters, chip related information, scanning parameters and dot matrix parameters, and the image data is collected by the external hardware system and transmitted to the software system; Communication connection establishment module: establishing a communication connection between the external hardware system and the software system according to the acquired communication setting parameters; Mechanical motion error compensation module: obtains the position information of multiple reference points on the protein chip, determines the actual coordinate offset of the protein chip in the X-axis and Y-axis directions by locating the reference points, and compensates for the mechanical motion error of the external hardware system in combination with the preset chip spacing parameters; Image data analysis and processing module: divides the image data of the intact protein chip into multiple regional sub-images, performs zooming, shrinking, and reloading operations on each regional sub-image, and simultaneously applies normalization and equalization image enhancement processing methods to improve the image contrast for more accurate subsequent image analysis; Protein chip image stitching module: based on the image feature matching algorithm, the pre-processed sub-images of the region are automatically stitched together according to the coordinate positions to form a complete and clear protein chip image; Defect type identification module: automatically analyzing the spliced complete protein chip image according to the lattice parameters, and identifying the defect type in the protein chip lattice using an integrated target detection algorithm; Analysis conclusion output module: outputs the analysis conclusion of the protein chip quality inspection, which includes the chip qualification judgment, the serial number of the unqualified small chip, the location of the unqualified point and the specific defect type.

[0022] By adopting the above technical solutions, it is possible to fully automate the protein chip quality inspection process based on machine vision technology, avoid random errors caused by individual differences in manual visual inspection, and significantly improve the consistency and accuracy of defect identification; through reference point positioning and mechanical motion error compensation, the coordinate offset of the hardware system can be effectively calibrated to ensure the position accuracy of the regional sub-images collected in blocks, and combined with image enhancement processing and feature matching and stitching algorithms, high-definition complete chip images can be generated, providing a high-quality data foundation for subsequent defect analysis; the target detection algorithm is used to automatically identify the type of dot matrix defects and output detailed analysis conclusions including qualification judgment, defect location and type, which not only improves the single-chip inspection efficiency to the minute level, adapting to the needs of high-speed production lines, but also realizes the digital management and traceability of inspection data, solving the problems of low efficiency and non-traceability of data in traditional methods, providing high-precision and automated technical support for quality control in the industrial production of protein chips, and meeting the quality inspection requirements of high-density protein chips in clean workshop environments.

[0023] The third object of this application is to provide an automatic quality inspection device for protein chips based on machine vision, the device comprising: A memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above-mentioned protein chip automatic quality inspection method based on machine vision.

[0024] The fourth purpose of this application is to provide a storage medium.

[0025] The fourth object of the present application is achieved through the following technical solutions: A storage medium stores a computer program that can be loaded by a processor and execute the above-mentioned protein chip automatic quality inspection method based on machine vision.

[0026] In summary, this application includes at least one of the following beneficial technical effects: 1. It can rely on machine vision technology to fully automate the protein chip quality inspection process, avoid random errors caused by individual differences in manual visual inspection, and significantly improve the consistency and accuracy of defect identification; through reference point positioning and mechanical motion error compensation, it can effectively calibrate the coordinate offset of the hardware system to ensure the position accuracy of the regional sub-images collected in blocks, and combine image enhancement processing with feature matching and stitching algorithms to generate high-definition complete chip images, providing a high-quality data foundation for subsequent defect analysis; it uses target detection algorithms to automatically identify the type of dot matrix defects and output detailed analysis conclusions including qualification judgment, defect location and type, which not only improves the single-chip inspection efficiency to minutes, adapting to the needs of high-speed production lines, but also realizes the digital management and traceability of inspection data, solving the problems of low efficiency and non-traceability of data in traditional methods, providing high-precision and automated technical support for quality control in the industrial production of protein chips, and meeting the quality inspection requirements of high-density protein chips in clean workshop environments.

[0027] 2. Utilizing an integrated multispectral imaging module to acquire raw image data encompassing visible light, infrared, and ultraviolet (UV) spectroscopy, the system can capture the multi-dimensional material properties of the protein chip surface (e.g., visible light reflects morphology, infrared characterizes thickness, and UV identifies chemical groups). Combined with the decomposition capabilities of noise reduction filtering and wavelet transforms for features at different scales, the system can effectively extract detailed information such as edges and textures of subtle defects. Principal component analysis filters key feature components and performs weighted fusion, dynamically assigning weights based on the sensitivity of each spectral channel to different defect types. This generates a high-dimensional fused image containing information on material composition, thickness, and defect depth. This not only addresses the issue of insufficient information from single-spectral detection, but also significantly enhances the ability to identify three-dimensional defects such as depressions and protrusions, as well as hidden defects such as abnormal material composition. This provides multimodal data support encompassing spatial structure and chemical properties for the subsequent automated analysis of defect types, improving the detection accuracy of protein chip surface defects from two-dimensional morphological recognition to a comprehensive determination of three-dimensional features and material composition, effectively meeting the complex quality inspection requirements of high-density, multifunctional protein chips. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 This is a flow chart of a method for automatic quality inspection of protein chips based on machine vision provided in an embodiment of the present application; Figure 2 This is a virtual structural diagram of a protein chip automatic quality inspection system based on machine vision provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] In this document, the term "and / or" simply describes a relationship between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document, unless otherwise specified, generally indicates an "or" relationship between the related objects.

[0031] The embodiments of the present application are described in further detail below with reference to the accompanying drawings.

[0032] The present invention provides a method for automatic quality inspection of protein chips based on machine vision. Figure 1 , the main process of the method is described as follows: S1: Acquire parameter information and image data related to protein chip quality inspection, wherein the parameter information includes communication setting parameters, chip-related information, scanning parameters, and lattice parameters, and the image data is collected by an external hardware system and transmitted to the software system; Among them, external hardware systems (such as high-precision industrial cameras and scanning platforms) capture images of protein chips according to preset scanning parameters (such as resolution of 1000dpi and exposure time of 50ms). The generated image data is transmitted to the software system in real time via Gigabit Ethernet or USB3.0 interface. At the same time, the software system reads communication setting parameters (such as hardware device IP address and data transmission protocol), chip-related information (such as chip size of 25mm×75mm and substrate material), and dot matrix parameters (such as 100 dot matrix rows, 200 columns, and dot pitch of 150μm) from the configuration file or production management system. For example, for different types of protein chips, the system can automatically match the corresponding dot matrix parameters to ensure accurate baseline data for subsequent defect analysis.

[0033] S2: establishing a communication connection between the external hardware system and the software system according to the acquired communication setting parameters; The software system establishes two-way communication with the external hardware system through socket network programming or device driver interfaces based on acquired communication parameters (such as the TCP / IP protocol and port number 8080). For example, in the case of an industrial camera, the software sends an initialization command (such as "Start Scan"), and the hardware responds with a status signal (such as "Ready"). Once the connection is confirmed, image data is transferred in real time to the software system's memory or cache in a specified format (such as BMP or JPEG), providing the data foundation for subsequent processing.

[0034] S3: Acquire position information of multiple reference points on the protein chip, determine the actual coordinate offset of the protein chip in the X-axis and Y-axis directions by locating the reference points, and compensate for the mechanical motion error of the external hardware system in combination with the preset chip spacing parameters; Four corner points are pre-set on the protein chip surface as reference points (e.g., black squares). The software system uses edge detection algorithms (e.g., the Canny operator) to extract the outlines of these reference points and then precisely calculates their coordinates (with an accuracy of ±0.1 pixel) using sub-pixel positioning algorithms (e.g., least-squares fitting). By comparing the theoretical coordinates of the reference points with the actual acquired coordinates, the chip's offset in the X and Y axes is calculated (e.g., +5μm for X-axis offset and -3μm for Y-axis offset). Combined with preset chip spacing parameters (e.g., 5mm between adjacent chips), the software system sends compensation commands to the hardware motion platform to adjust the scanning position, eliminating coordinate deviations caused by mechanical vibration or positioning errors, and ensuring the accuracy of subsequent sub-image acquisition positions.

[0035] S4: Dividing the image data of the intact protein chip into multiple regional sub-images, performing zooming, shrinking, and reloading operations on each of the regional sub-images, and applying normalization and equalization image enhancement processing methods to improve the contrast of the image to facilitate more accurate subsequent image analysis; The software system divides a complete chip image (e.g., 5000×5000 pixel resolution) into 25 fixed-size sub-images (e.g., 1000×1000 pixel blocks) to facilitate distributed processing. Each sub-image is first scaled up or down based on defect detection requirements (e.g., magnifying a local area by 2x to highlight details). Normalization is then performed (unifying the pixel value range to [0,1]) to eliminate the effects of uneven lighting. Finally, histogram equalization is used to enhance image contrast (e.g., expanding the pixel distribution in low-contrast areas to the full dynamic range). This sharpens the boundaries between protein spots and the background, providing high-quality image input for defect recognition.

[0036] S5: automatically splicing the pre-processed regional sub-images into a complete and clear protein chip image according to coordinate positions based on an image feature matching algorithm; The SURF feature matching algorithm (Speeded Up Robust Features) is used to extract feature points (such as protein spot edges and fiducial corners) from each sub-image. Matching pairs are screened by calculating the Euclidean distance between the feature point descriptors. False matches are then eliminated using the RANSAC algorithm, ensuring a matching accuracy of over 99% for overlapping regions of adjacent sub-images. Based on the coordinates of the matched feature points, the translation and rotation parameters of the sub-images are calculated using an affine transformation matrix. All sub-images are aligned and spliced together to form a complete protein chip image (e.g., with a resolution of 10,000 × 10,000 pixels), eliminating any gaps or geometric distortions caused by block-wise acquisition.

[0037] S6: automatically analyzing the stitched complete protein chip image according to the lattice parameters, and identifying the defect type in the protein chip lattice using an integrated target detection algorithm; The software system generates a theoretical grid of dot coordinates based on dot matrix parameters (e.g., an expected protein dot diameter of 50μm and spacing of 150μm) and matches this to the actual stitched image. Each protein dot is inspected using an integrated object detection algorithm (e.g., the YOLOv5 lightweight model), identifying the following defect types: positional deviation (a deviation exceeding ±20μm between the actual and theoretical coordinates); size anomaly (a protein dot diameter outside the range of [40μm, 60μm]); and morphological defect (an ellipticity exceeding 0.8 (a circularity indicator)). The algorithm automatically extracts the shape and texture features of the protein dot using a convolutional neural network and outputs the defect type and confidence score, eliminating the subjectivity of manual threshold setting.

[0038] S7: Outputting the analysis conclusion of the protein chip quality inspection, the analysis conclusion includes the qualification judgment of the chip, the serial number of the unqualified small chip, the location of the unqualified point and the specific defect type.

[0039] The final inspection results are output as structured data, including: qualification determination (e.g., "qualified" or "unqualified"); serial number of the unqualified small chip (e.g., "defect in sub-image area 3"); location of the unqualified point (e.g., X=1234μm, Y=5678μm); and defect type (e.g., "position offset" or "dimensional deviation"). Analysis conclusions are displayed in real time on the user interface and synchronously stored in a database (e.g., MySQL), linking information such as chip batch and inspection time, providing data support for quality traceability and process optimization.

[0040] Through the above steps, this solution realizes a fully automated process from image acquisition, preprocessing, stitching, defect identification to result output, avoiding the inefficiency and subjective errors of traditional manual inspection, significantly improving the accuracy and speed of protein chip quality inspection, and is suitable for the high-speed quality inspection needs of industrial production lines.

[0041] Specifically, in some possible embodiments, the step of obtaining parameter information and image data related to protein chip quality inspection includes: Acquiring original multispectral image data using an integrated multispectral imaging module, wherein the original multispectral image data includes visible light image data, infrared image data, and ultraviolet image data; Performing preliminary noise reduction and filtering processing on the original multispectral image data; Performing wavelet transform on the pre-processed original multispectral image data to decompose it into characteristic sub-images of different scales and directions; Performing principal component analysis on the characteristic sub-images of each spectral channel based on wavelet transform, calculating the covariance matrix of each characteristic sub-image and obtaining the principal components through eigenvalue decomposition; The extracted characteristic components of each spectral channel are weightedly fused, and weight coefficients are assigned according to the sensitivity and importance of each spectral channel to different features to generate a fused image containing information on the surface material composition, thickness and defect depth of the protein chip.

[0042] Among them, in the industrial quality inspection scenario of the clean workshop, the integrated multispectral imaging module is the core hardware for realizing multi-dimensional defect detection. The module usually contains three independent imaging units: a visible light camera, an infrared camera, and an ultraviolet camera, which respectively capture the morphological characteristics, thickness information, and fluorescent marker distribution of the protein chip. For example, the visible light camera is equipped with a 550nm narrow-band filter to focus on the outline and position of the protein spot; the infrared camera uses an 850nm near-infrared light source to penetrate the surface to detect changes in the thickness of the protein layer; the ultraviolet camera uses 365nm ultraviolet excitation to identify the distribution of fluorescently labeled chemical groups. The three are calibrated through precise mechanical structures to ensure that the field of view overlap of the three-channel images collected at the same time exceeds 99%, laying the foundation for spatial alignment for subsequent data fusion.

[0043] Raw multispectral images are susceptible to environmental noise during acquisition, so they are first preprocessed. A combination of Gaussian filtering (σ=1.0) and median filtering effectively suppresses random noise and salt-and-pepper noise, preserving edge details. For example, for visible light images, noise reduction can increase the grayscale difference between protein spots and the background by 20%, significantly improving the accuracy of subsequent feature extraction. The preprocessed image enters the wavelet transform stage, where a three-layer decomposition using the db4 wavelet basis is performed, decomposing the image signal into feature sub-images of different scales (high frequency, medium frequency, low frequency) and directions (horizontal, vertical, diagonal). This step acts like an "image microscope," separating micron-level defects (such as 5μm edge blur) from a complex background, providing multi-scale feature support for precise analysis.

[0044] For each spectral channel's characteristic sub-image, the covariance matrix is calculated and eigenvalue decomposition is performed to screen out principal components with a cumulative contribution rate exceeding 95%. For example, the principal component of the infrared channel is often highly correlated with the thickness of the protein layer, while the principal component of the ultraviolet channel reflects the distribution density of fluorescent molecules. Through this dimensionality reduction process, the data volume can be reduced by more than 60% while retaining core defect information, improving the efficiency of subsequent processing. Finally, weights are assigned based on the sensitivity of each spectral channel to different defect types. For example, when detecting thickness anomalies, the infrared channel's weight is increased to 50%, and when detecting fluorescence quenching, the ultraviolet channel's weight reaches 40%. Through weighted fusion, a comprehensive image containing information on material composition, thickness, and defect depth is generated, making hidden defects on the chip surface (such as nanoscale damage to the substrate material) appear as visual features.

[0045] To meet the needs of high-precision quality inspection, a structured light 3D scanner can be introduced to obtain 3D point cloud data of the chip surface (with an accuracy of ±5μm) for spatial registration with the multispectral fusion image. By establishing a pixel-level coordinate mapping relationship, the 2D spectral features are associated with 3D curvature changes. For example, when the fused image shows a spectral anomaly in a certain area, combined with the surface curvature data of the 3D point cloud, it can be determined whether there are concave or convex defects and the defect depth can be calculated. This technological fusion enables the system to not only identify planar position offsets but also detect three-dimensional structural anomalies, such as 10μm height difference protrusions formed by protein aggregation, with detection accuracy increased by over 30% compared to traditional 2D methods.

[0046] For fluorescently labeled protein chips, the system can expand its time-series acquisition capabilities, continuously capturing sequences of fluorescence images of the same area at a 10Hz frequency. By calculating the fluorescence intensity decay rate of adjacent frames, quenched regions are identified in real time and a risk heat map is generated. For areas with high quenching risk, the system automatically switches to pulsed excitation light (with a duty cycle optimized to 30%), minimizing photodamage while maintaining signal stability. Combined with a spatiotemporal interpolation algorithm, the system utilizes the characteristics of unquenched frames to repair signal-attenuated regions, achieving a 92% accuracy in molecular distribution image reconstruction under quenching conditions. This effectively addresses the problem of missed detections caused by fluorescence signal decay in traditional methods.

[0047] Specifically, in some possible embodiments, the steps of performing principal component analysis on the characteristic sub-image of each spectral channel based on wavelet transform, calculating the covariance matrix of each characteristic sub-image and obtaining the principal component by eigenvalue decomposition include: Calculating the covariance matrix of each of the feature sub-images; Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and corresponding eigenvectors; Sort the eigenvectors in descending order of the eigenvalues, select the first several eigenvectors whose cumulative contribution rate reaches the set threshold, and the components corresponding to the first several eigenvectors are the extracted principal components; The projection coefficient of the principal component on each of the characteristic sub-images is used as the characteristic component of each of the characteristic sub-images.

[0048] When implementing principal component analysis (PCA) on characteristic sub-images after wavelet transformation, the first step is to convert the characteristic sub-images for each spectral channel (e.g., visible light, infrared, and ultraviolet) into a numerical form suitable for statistical analysis. For example, taking a horizontal high-frequency sub-image at a certain scale, assuming its size is 100×100 pixels, its pixel values are expanded row by row into a 10,000-dimensional vector. Similar sub-images from different spectral channels form a multidimensional feature matrix (e.g., a 10,000×3 matrix for a three-channel image). By calculating the covariance matrix of this matrix, the correlation between the characteristics of each spectral channel can be quantified. For example, if the responses of the visible and infrared channels at the edge of a protein dot show a strong positive correlation, the corresponding element value in the covariance matrix will be significantly greater than zero.

[0049] Next, eigenvalue decomposition is performed to obtain the eigenvalues and eigenvectors corresponding to the covariance matrix. The size of the eigenvalue reflects the importance of the change direction represented by the eigenvector in the data. For example, the first eigenvalue may correspond to the morphological change of the protein spot that responds to the three spectral channels, and the second eigenvalue may correspond to the difference in thickness and fluorescence distribution between the infrared and ultraviolet channels. After sorting the eigenvalues from large to small, the top k eigenvectors whose cumulative contribution rate reaches the set threshold (such as 95%) are selected. Assuming that the cumulative contribution rate of the first two eigenvalues has reached 95%, the "principal components" corresponding to these two eigenvectors contain 95% of the key information in the original data, thereby reducing the three-dimensional spectral features to two dimensions and effectively eliminating noise and redundant information.

[0050] The selected principal components are projected back onto each characteristic sub-image, and the resulting projection coefficients are the characteristic components. For example, if the projection coefficient of the first principal component in the visible light channel is high, it indicates that this principal component primarily reflects edge features in the visible light image and can be used to identify abnormal contours of protein spots. If the projection coefficient of the second principal component in the infrared channel is prominent, it corresponds to a sensitive feature of thickness variations on the chip surface and is suitable for detecting recessed defects in the substrate material. This projection processing not only retains the most critical information for defect identification but also reduces the data dimension, improving the computational efficiency of subsequent feature fusion and defect analysis by over 40%.

[0051] The principal component selection strategy is dynamically adjusted for different types of defects. For example, when detecting protein spot position offsets, the focus is on the low-scale feature sub-image of the visible light channel (reflecting the overall outline), and the cumulative contribution rate threshold is set to 90% to retain more details; when detecting thickness anomalies, a high threshold of 98% is used for the mid-scale feature sub-image of the infrared channel to ensure that subtle thickness change signals are captured. This adaptive threshold method can improve the detection accuracy of the corresponding defect type by 15%-20%. In addition, by combining the surface curvature characteristics of the three-dimensional morphology data and adding it as a new dimension to the feature matrix for joint PCA, a correlation model between spectral features and three-dimensional structure can be established. For example, the principal component load can be used to determine whether a defect is accompanied by spectral anomalies and curvature mutations, thereby more accurately identifying complex defects such as protein aggregation protrusions.

[0052] Deeply combine principal component analysis with machine learning models. For example, the extracted feature components are used as input to train a support vector machine (SVM) classifier to perform multi-category recognition of defect types such as "position offset", "size abnormality", and "composition unevenness". By analyzing the correlation between the principal components and the defect labels, feature selection can be further optimized. For example, if a principal component is found to contribute 70% to the "composition unevenness" defect, a higher weight will be given to the feature component in model training. In addition, for large-scale chip inspection data, an incremental PCA (Incremental PCA) algorithm can be used to process sub-image data batch by batch, avoiding the memory bottleneck caused by loading the full amount of data at one time, and reducing the single-chip processing time from 200ms of traditional PCA to 80ms, meeting the real-time inspection requirements of high-speed production lines.

[0053] This PCA-based feature extraction method essentially "translates" complex multispectral information into the "key code" for defect identification through mathematical transformations. This enables the system to quickly locate the most discriminative features from massive amounts of image data, avoiding the subjectivity of manually setting thresholds while also breaking through the information limitations of single-spectral detection. In practical applications, the combination of this technology with wavelet transforms forms a complete chain of "multi-scale feature separation-key component screening-defect-sensitive feature enhancement," providing a highly efficient solution for protein chip quality inspection, from data preprocessing to high-level feature extraction. It is particularly suitable for complex, high-density, multi-parameter detection scenarios.

[0054] Specifically, in some possible embodiments, the method further includes: Acquire 3D topographic data of the protein chip surface, calibrate local details through data registration algorithms, and form a high-precision 3D point cloud model covering the entire chip; Performing spatial registration on the three-dimensional point cloud model and the fused image corresponding to the original multispectral image data to establish a pixel-level spatial coordinate mapping relationship; Extracting surface curvature from the three-dimensional point cloud model, extracting spectral features of the defect area from the fused image, establishing a linear correlation model between the two through principal component analysis, and determining a mapping formula between surface curvature change and defect depth; Based on the mapping formula, three-dimensional depth inversion is performed on the suspicious area in the fused image to automatically identify stereo defects.

[0055] In the actual implementation of 3D topography and multispectral image fusion detection, the protein chip is first scanned using a structured light 3D scanner, projecting a sinusoidal fringe pattern and capturing an image of the deformed fringe. A phase unwrapping algorithm is then used to reconstruct 3D point cloud data of the chip surface (with an accuracy of ±5μm). For local details such as nanoscale bumps and depressions, an atomic force microscope (AFM) is used for supplementary scanning to obtain surface height data with submicron resolution. This AFM data is then fused with the structured light point cloud using a data registration algorithm (such as the iterative closest point (ICP) algorithm) to eliminate coordinate system deviations between different devices. This results in a high-precision 3D point cloud model covering the entire chip, fully characterizing the microscopic fluctuations on the chip surface (such as the height of the protein dots and the flatness of the substrate).

[0056] Next, spatial registration is performed to align the 3D point cloud model with the multispectral fusion image. First, 5-8 distinct reference points (such as metal markers) are selected on the chip surface. Their coordinates in the 2D image are obtained through visual positioning. Their 3D point cloud coordinates are simultaneously recorded, and the rotation and translation parameters are calculated using a homogeneous transformation matrix to achieve millimeter-level global registration. Pixel-level precision registration is then performed. For each pixel, bilinear interpolation is used to map the height value in the 3D point cloud to the corresponding multispectral image pixel, establishing a one-to-one correspondence between "2D position and 3D height." This ensures a precise correlation between spectral features and spatial structure in subsequent analysis (for example, if the UV spectrum of a pixel is abnormal, its surface curvature value can be obtained simultaneously).

[0057] During feature extraction and model building, the surface curvature (e.g., Gaussian curvature, mean curvature) of each region is calculated from the 3D point cloud. This reflects the degree of curvature of the local surface—concave areas exhibit negative curvature, convex areas exhibit positive curvature, and flat areas exhibit near-zero curvature. Spectral feature vectors (e.g., visible light grayscale value, infrared intensity, and ultraviolet fluorescence intensity) of the corresponding region are simultaneously extracted from the multispectral fusion image. Principal component analysis (PCA) is used to reduce the dimensionality of the curvature data and spectral features, and the principal component with the highest contribution is selected to establish a linear regression model. For example, the correlation between infrared intensity and Gaussian curvature in a particular principal component was found to be 0.85, indicating that the thickness variation in that region is highly correlated with the surface curvature. This allows the approximation of the curvature variation and the defect depth (e.g., depth = a × curvature + b, with coefficients a and b determined using the least squares method).

[0058] Based on this model, the system automatically triggers 3D depth inference for suspicious areas in the multispectral fusion image (such as areas where the spectral values deviate by 2σ from the mean). For example, if the visible light image shows a blurred edge of a protein spot and the infrared signal in the fused image is abnormal, the system calculates the theoretical depth of the area using a mapping formula and compares it with the actual height of the 3D point cloud. If the difference exceeds 5μm, it is determined to be a three-dimensional defect (such as uneven protein spot deposition thickness caused by a base depression). This detection method can identify defects such as depressions (depth ≥ 10μm) and protrusions (height ≥ 15μm) that are missed by traditional 2D vision, and its detection accuracy is over 40% higher than that of a single modality.

[0059] Confocal microscopy can be introduced to acquire higher-resolution 3D topographic data (axial accuracy of ±2μm), complementing structured light scanning: structured light captures the entire chip's macroscopic profile, while confocal microscopy focuses on localized microstructures. A multi-sensor fusion algorithm seamlessly stitches 3D data at different scales, enabling the system to simultaneously detect centimeter-level chip warpage and micron-level protein dot height differences. Furthermore, for protein chips on flexible substrates (such as PDMS), an elastic deformation compensation model is incorporated into the registration process. Finite element analysis simulates the effect of substrate curvature on 3D coordinates, reducing the registration error from ±8μm to ±3μm, adapting to the detection needs of emerging materials.

[0060] Principal component analysis can be upgraded to a machine learning model, leveraging random forests or neural networks to establish nonlinear correlations between spectral features and 3D defects. For example, by inputting a region's multispectral feature vector and surface curvature values, the system outputs defect type (depression, protrusion, material heterogeneity) and depth quantification. After training with extensive labeled data, the accuracy of complex defect recognition can reach over 98%. Simultaneously, a real-time 3D reconstruction algorithm has been developed, reducing point cloud generation time from 20 seconds using traditional methods to 5 seconds. This, combined with high-speed linear scan cameras, enables online inspection of chip production lines, meeting the inspection speed requirement of over 10 chips per minute in industrial production.

[0061] This 3D and multi-spectral fusion detection technology essentially builds a "digital twin" model for protein chips, enabling the system to upgrade from "viewing patterns on a flat surface" to "reading structures in three dimensions." By deeply binding two-dimensional spectral information with three-dimensional spatial coordinates, it not only enables the geometric morphology detection of defects, but also enables the discovery of the causes of defects (such as abnormal protein deposition caused by substrate deformation) through physical or data models, providing direct guidance for production process optimization. In the biopharmaceutical field, this technology is particularly suitable for high-end products such as antibody chips and cell chips that are sensitive to surface structure. It effectively solves the pain point of traditional testing, which "only sees color but not height," and promotes protein chip quality inspection into the era of three-dimensional intelligent analysis.

[0062] Specifically, in some possible embodiments, the method further includes: During multispectral imaging, continuous frames of fluorescence images of the same area are collected synchronously, and the fluorescence intensity decay rate between adjacent frames is calculated; When the fluorescence intensity decay rate exceeds a preset threshold, it is determined that fluorescence quenching occurs; Establish a spatial distribution map of fluorescence quenching, mark the location and degree of quenching areas, and generate a quenching risk heat map; Based on the quenching risk heat map, the excitation light parameters are dynamically adjusted for high-risk areas. Pulsed excitation light is used instead of continuous light, and the duty cycle and frequency are optimized. At the same time, the excitation light intensity is increased and the exposure time is shortened for areas with severe quenching. A feedback control algorithm is used to keep the fluorescence signal intensity in the target area within the linear response range of the detection system. For the multispectral image data of the quenched region, a joint spatiotemporal denoising algorithm is used. In the temporal dimension, the features of adjacent unquenched frames are used for interpolation and restoration. In the spatial dimension, the surface curvature information in the 3D topography data is combined to suppress artifacts. A generative adversarial network is then used to reconstruct the molecular distribution image of the quenched region. When it is detected that the average fluorescence decay rate of the entire batch of chips exceeds the set ratio, the early warning mechanism is automatically triggered and prompts to replace the fluorescent labeling reagent or adjust the biological sample pretreatment process. A quenching history database is established to analyze the quenching characteristics of different batches of chips and fluorescent dyes, providing parameter optimization suggestions for subsequent quality inspection tasks.

[0063] During the quality inspection of protein chips in cleanrooms, fluorescently labeled molecules are susceptible to quenching when exposed to intense light, resulting in signal attenuation and impacting detection accuracy. Therefore, real-time quenching monitoring and compensation functionality must be integrated into the multispectral imaging module. In practice, the system synchronously captures a sequence of fluorescence images of the same region at a rate of 10 frames per second, with each frame covering a 0.5 mm × 0.5 mm detection area on the chip. The decay rate is calculated by calculating the average fluorescence intensity change of the same ROI (region of interest) between two consecutive frames (e.g., decay rate = (previous frame intensity - next frame intensity) / previous frame intensity × 100%). When the decay rate of a region exceeds a preset threshold (e.g., decay exceeding 15% per minute), the system immediately identifies fluorescence quenching in that region and marks the quenched location on the chip coordinate grid. A visual quenching risk heat map is generated, using different colors based on the degree of attenuation (e.g., red for severe quenching, yellow for moderate quenching).

[0064] Based on the real-time updated heat map, the hardware control system dynamically adjusts the excitation light parameters. For high-risk quenching areas, the continuous excitation light is switched to pulsed output, for example, reducing the duty cycle from 100% to 30% and increasing the frequency to 200Hz, reducing the single exposure time to reduce photodamage. For localized areas with severe quenching (such as those with a decay rate exceeding 30%), the excitation light intensity is temporarily increased by 20% in pulsed mode, while the exposure time is shortened from 50ms to 20ms. A PID feedback control algorithm is used to ensure that the fluorescence signal intensity in the target area remains stable within the linear response range of the detection system (such as 1000-4000 grayscale values). This dynamic adjustment can reduce the fluorescence signal decay rate from 20% / minute with traditional continuous illumination to below 5% / minute, effectively extending the available detection time of fluorescent molecules.

[0065] For areas where quenching has occurred, the system uses a joint spatiotemporal restoration algorithm to process multispectral image data. In the temporal dimension, the system uses the two adjacent unquenched frames to fill in the pixel values of the quenched area using a bilinear interpolation algorithm (for example, if a pixel is quenched in the nth frame, the average of the pixels at the same position in the n-1st and n+1st frames is taken). In the spatial dimension, the interpolation results are corrected by combining the surface curvature information in the three-dimensional morphology data. If the surface curvature of the area is negative (concave), the weight of the adjacent planar area is reduced to avoid interference of the planar data on the three-dimensional structure. Further, through generative adversarial network (GAN) training, using historical unquenched chip images of the same type as samples, the molecular distribution details of the quenched area are reconstructed, so that the similarity between the repaired fluorescence image and the real signal reaches over 92%, significantly improving the accuracy of defect identification in the quenched area.

[0066] When the average fluorescence decay rate of an entire batch of chips exceeds a set ratio (e.g., 20%), the system automatically triggers an audible and visual warning, displays a "High Risk of Fluorescent Reagent Quenching" prompt on the user interface, and links to the production management system to query information such as the fluorescent dye model and spotting time used in that batch of chips. At the same time, the quenching data from this test (e.g., decay rate in each region, repair effect) is stored in a historical database. Through data analysis, the quenching characteristics of different dyes are mined (e.g., the decay constant of Cy3 dye under 365nm illumination is 0.02 / second). This provides parameter optimization suggestions for subsequent tests of similar chips. For example, the excitation light intensity corresponding to that dye is automatically reduced by 10% and the pulse frequency is increased to 150Hz. This creates a closed-loop control system of "detection-feedback-optimization," reducing the quenching-related missed detection rate in similar scenarios from 30% to below 8%.

[0067] Dual-light source collaborative imaging technology can be introduced: a low-power LED light source is used for pre-scanning and positioning to determine the approximate distribution area of the fluorescent marker. A high-sensitivity ICCD camera, coupled with a pulsed laser light source (wavelength matching the fluorescent dye excitation peak), then performs precise point-by-point scanning, shortening the exposure time of a single area to less than 10ms and reducing the photobleaching effect at the hardware level. In a parallel expansion solution, a machine learning model is trained on historical quenching data to establish a predictive model based on "dye type-excitation light parameters-quenching probability." This automatically recommends optimal scanning parameters based on the input dye information before testing (e.g., if the quenching probability of a batch of Alexa Fluor 488 dye is predicted to be 40%, a pulsed excitation mode with a 25% duty cycle and 15% intensity is pre-activated), achieving a technological upgrade from passive compensation to active prevention.

[0068] This dynamic compensation technology for fluorescence quenching essentially addresses the challenge of stable acquisition of perishable signals in biological testing through a multi-layered mechanism of "real-time monitoring - intelligent control - data repair - trend prediction." In practical applications, it not only ensures the detection accuracy of fluorescently labeled protein chips, but also, through deep integration with production data, provides a basis for improvements in upstream processes such as biological sample pretreatment and fluorescent dye selection. This complete technology chain spans "sample preparation - quality inspection and analysis - process optimization." It is particularly suitable for large-scale production testing of high-end products such as immunodiagnostic chips and gene expression microarrays that rely on fluorescence signals.

[0069] Specifically, in some possible embodiments, the step of compensating for the mechanical motion error of the external hardware system includes; Integrate a multi-dimensional sensor array on the motion platform of the external hardware system to collect vibration, displacement and posture data of the motion platform in real time during its motion; Build a dynamic error model based on the collected vibration, displacement and posture data, adjust motion parameters in real time, and compensate for positioning deviations caused by mechanical vibration and thermal drift; Combining real-time position feedback with image feature matching, the scanning path planning is dynamically optimized to ensure that the overlapping accuracy of adjacent sub-images meets the preset stitching requirements; To address image blur caused by motion, we use physical model-based image restoration technology combined with edge enhancement algorithm to improve the recognizability of defect features and ensure detection accuracy under complex motion conditions.

[0070] In the automated quality inspection line of the cleanroom, the mechanical motion accuracy of the protein chip scanning platform directly affects the quality of image acquisition. Therefore, a multi-dimensional sensor array must be integrated at the bottom of the motion platform—including a three-axis accelerometer (detecting vibration amplitude with an accuracy of ±0.1g), a laser displacement sensor (monitoring X / Y axis displacement with a resolution of 1μm), and a two-axis inclinometer (detecting platform tilt with an accuracy of ±0.01°)—to collect motion data in real time at a frequency of 200Hz. For example, when the platform moves at high speed along the X-axis (100mm / s), the accelerometer detects that the vibration amplitude in the Z-axis exceeds 5g (a preset threshold), indicating the presence of mechanical resonance. The sensor immediately transmits the data to the motion controller.

[0071] The dynamic error model, built on the extended Kalman filter algorithm, integrates sensor data with motor encoder feedback in real time to predict and compensate for motion deviations. For example, for thermal drift compensation, when the laser displacement sensor detects a cumulative X-axis drift of 15μm over 30 minutes (due to heat generation during device operation), the model automatically calculates the temperature-displacement coefficient (fitted to 0.5μm / °C using historical data) and sends reverse compensation pulses to the servo motor (each pulse corresponds to a 0.5μm displacement), improving positioning accuracy from ±50μm to ±10μm. For high-frequency vibrations (such as 100Hz mechanical vibrations), the model uses a bandpass filter to separate noise and, combined with a feedforward control algorithm, preemptively adjusts motor torque to suppress position fluctuations caused by vibration.

[0072] Scan path planning utilizes a "visual feedback-dynamic adjustment" mechanism: After each sub-image capture (e.g., a 1000×1000 pixel area), the system uses image fiducials (e.g., markers at the chip's four corners) to calculate in real time the deviation between the actual position and the theoretical coordinates (e.g., +8μm offset on the X-axis, -5μm offset on the Y-axis). The system then dynamically adjusts the next capture position based on the preset overlap ratio (20%-30%) of adjacent sub-images. For example, if the previous sub-image is detected to have shifted to the right, the next capture starting point is automatically shifted 15μm to the left to ensure a 25% overlap. This provides sufficient overlap information for subsequent feature matching and avoids stitching misalignment caused by excessive offsets.

[0073] To address motion blur, the system first estimates the point spread function (PSF) from sensor data. For example, if the platform detects 2μm movement along the X-axis during exposure, it constructs the corresponding horizontal motion blur PSF. It then uses a blind deconvolution algorithm (such as the Richardson-Lucy iteration) to restore the blurred image. This is combined with an unsharp mask (USM) to enhance edges, increasing the contrast of protein spot edges by 30%, allowing tiny defects as small as 10μm in diameter to stand out against the blurred background. In practice, this process can increase motion speed from 50mm / s to 150mm / s while maintaining defect recognition rate.

[0074] A model predictive control (MPC) algorithm can be introduced to predict the next position deviation 50ms in advance based on the dynamic model of the motion platform (taking into account mass, damping, and stiffness parameters). This proactively compensates for vibration and thermal drift, further improving dynamic positioning accuracy to ±5μm. A parallel expansion solution uses an air bearing platform instead of a traditional ball screw, combined with a laser interferometer (accuracy of ±0.1μm) to establish closed-loop control. This reduces mechanical friction and vibration at the hardware level, making it suitable for special chip testing requiring nanometer-level precision.

[0075] Specifically, in some possible embodiments, the steps after outputting the analysis conclusion of the protein chip quality inspection include: Establish closed-loop feedback with sample production equipment via industrial Ethernet; When the same type of defects are detected on several protein chips in succession, the defect position coordinates and corresponding scanning parameters are automatically extracted; Based on the spatial cluster analysis of the defect position coordinate distribution, determine whether the defect is caused by the positioning deviation of the spotting head or the abnormal injection parameters; If the sample head positioning deviation, adjust the sample head X / Y axis motion parameters; If the injection parameters of the sample head are abnormal, the injection pressure and pulse width of the sample head will be dynamically corrected; A quality inspection data traceability system is established to associate the defect type, three-dimensional morphology data, and multi-spectral characteristic components of each chip with the production batch and equipment parameters to form a quality control database.

[0076] The protein chip quality inspection system establishes real-time communication with the front-end spotting equipment via industrial Ethernet. The inspection conclusion for each chip (e.g., "There is a spotting position offset in the 12th sub-image area") and key parameters (test time, chip batch number) are packaged and transmitted to the device controller in JSON format. For example, if 10 consecutive chips are detected with a protein spot position offset defect in the 3rd row, 5th column area, the system automatically triggers a closed-loop feedback mechanism, extracting the coordinates of these defects (e.g., X = 3500μm ± 20μm, Y = 1800μm ± 15μm) and the corresponding spotting parameters (spotting head movement speed of 200mm / s, injection pulse width of 5ms) from the database.

[0077] The spatial clustering analysis module uses a density-based clustering algorithm (such as DBSCAN) to analyze defect locations. If the clustering results show that defects are concentrated at the edge of the chip and distributed linearly (the spacing between adjacent defects is equal to the nozzle spacing of the spotting head), the system identifies an X / Y-axis positioning error in the spotting head—possibly caused by cumulative error in the servo motor encoder. The system then sends parameter adjustment instructions to the spotting device, adding a negative offset to the original motion trajectory (e.g., -5μm for every 100mm of X-axis movement). The adjusted positioning accuracy is verified in real time using the device's built-in laser rangefinder (error ≤±3μm). If the clustering results show that the defects lack a clear spatial pattern but are consistently associated with abnormal protein spot size (diameter fluctuation exceeding ±15%), the system identifies an abnormal jetting parameter. The jetting head pressure is automatically fine-tuned from 0.3MPa to 0.32MPa, and the pulse width is shortened by 1ms. The adjustment is verified by testing a calibration chip on a standard slide (e.g., the protein spot diameter returns to within ±5% of the target value).

[0078] The quality inspection data traceability system is built using a distributed database (such as MySQL Cluster). Each chip's inspection data includes defect type (e.g., "position offset" or "insufficient injection volume"), 3D topography (maximum surface curvature 0.02 mm⁻¹), multispectral feature components (visible light channel grayscale value standard deviation 15, infrared intensity mean 200), and production information (batch number 20250512-03, spotting equipment number P-007, operator number A012). This data is linked through a unique chip ID, supporting multi-dimensional search. For example, a quality engineer can query the frequency of equipment failures within the past 30 days by "spotting head number + defect type," or trace environmental parameters corresponding to the production period (e.g., whether temperature and humidity fluctuations caused thermal expansion of the spotting head) by "batch number + defect location."

[0079] The system can incorporate machine learning models to analyze trends in historical defect data. For example, if the positioning deviation defect of a certain type of spotting head exceeds the warning threshold for three consecutive days, a random forest algorithm predicts possible ball screw wear on the device. A maintenance work order is automatically generated and pushed to the equipment management system, transforming reactive repairs into preventive maintenance. A parallel expansion solution, addressing data compliance requirements in the pharmaceutical industry, can synchronously write key quality inspection data (such as defect coordinates and adjustment parameters) into the blockchain system. Hash encryption ensures that the data cannot be tampered with, while also supporting cross-departmental traceability (for example, R&D departments can access multispectral data from defective chips to optimize probe design).

[0080] Another embodiment of the present application provides a protein chip automatic quality inspection system based on machine vision, wherein, Figure 2 , a protein chip automatic quality inspection system based on machine vision, including: Parameter information and image data acquisition module 100: acquires parameter information and image data related to protein chip quality inspection, wherein the parameter information includes communication setting parameters, chip related information, scanning parameters and dot matrix parameters, and the image data is collected by the external hardware system and transmitted to the software system; Communication connection establishment module 200: establishes a communication connection between the external hardware system and the software system according to the acquired communication setting parameters; Mechanical motion error compensation module 300: obtains the position information of multiple reference points on the protein chip, determines the actual coordinate offset of the protein chip in the X-axis and Y-axis directions by locating the reference points, and compensates for the mechanical motion error of the external hardware system in combination with the preset chip spacing parameters; Image data analysis and processing module 400: divides the image data of the intact protein chip into multiple regional sub-images, performs zooming, shrinking, and reloading operations on each of the regional sub-images, and simultaneously applies normalization and equalization image enhancement processing methods to improve the image contrast for more accurate subsequent image analysis; Protein chip image stitching module 500: automatically stitching the pre-processed regional sub-images into a complete and clear protein chip image according to coordinate positions based on an image feature matching algorithm; Defect type identification module 600: automatically analyzing the spliced complete protein chip image according to the lattice parameters, and identifying the defect type in the protein chip lattice using an integrated target detection algorithm; Analysis conclusion output module 700: outputs the analysis conclusion of the protein chip quality inspection, which includes the qualification judgment of the chip, the serial number of the unqualified small chip, the location of the unqualified point and the specific defect type.

[0081] This embodiment provides a protein chip automatic quality inspection system based on machine vision. Due to the functions of each module itself and the logical connection between each other, it can implement the various steps of the aforementioned embodiment, and thus can achieve the same technical effects as the aforementioned embodiment. The principle analysis can be found in the relevant description of the steps of the aforementioned protein chip automatic quality inspection method based on machine vision, which will not be repeated here.

[0082] An embodiment of the present application also provides an automatic quality inspection device for protein chips based on machine vision, comprising a memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and execute the above-mentioned automatic quality inspection method for protein chips based on machine vision.

[0083] An embodiment of the present application further provides a storage medium storing a computer program that can be loaded by a processor and execute the above-mentioned method for automatic quality inspection of protein chips based on machine vision.

[0084] The storage medium provided in this embodiment can achieve the same technical effects as the aforementioned embodiments because the computer program therein implements the various steps of the aforementioned embodiments after being loaded and run on the processor. The principle analysis can be found in the relevant description of the aforementioned method steps, which will not be repeated here.

[0085] The storage medium includes, for example, various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0086] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0087] In the description of this specification, the reference terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" mean that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and features of different embodiments or examples without contradiction.

[0088] In addition, features defined by the terms "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "plurality" means at least two, for example, two, three, etc., and unless otherwise specifically defined, is used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly specifying the number of the technical features indicated.

[0089] Thus, any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the invention pertain.

[0090] The embodiments of this specific implementation method are all preferred embodiments of the present application and are not intended to limit the scope of protection of the present application. Therefore, any equivalent changes made based on the structure, shape, and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for automatic quality inspection of protein chips based on machine vision, characterized in that: include: Acquiring parameter information and image data related to protein chip quality inspection, wherein the parameter information includes communication setting parameters, chip-related information, scanning parameters, and dot matrix parameters, and the image data is collected by an external hardware system and transmitted to the software system; Establishing a communication connection between the external hardware system and the software system based on the acquired communication setting parameters; Obtaining the position information of multiple reference points on the protein chip, determining the actual coordinate offset of the protein chip in the X-axis and Y-axis directions by locating the reference points, and compensating for the mechanical motion error of the external hardware system in combination with the preset chip spacing parameters; Divide the image data of the intact protein chip into multiple regional sub-images, perform zooming, shrinking, and reloading operations on each of the regional sub-images, and simultaneously apply normalization and equalization image enhancement processing methods to improve the image contrast for more accurate subsequent image analysis; Based on an image feature matching algorithm, the pre-processed sub-images of the region are automatically spliced into a complete and clear protein chip image according to the coordinate positions; Automatically analyzing the spliced complete protein chip image according to the lattice parameters, and identifying the defect type in the protein chip lattice using an integrated target detection algorithm; Output the analysis conclusion of the protein chip quality inspection, which includes the chip qualification judgment, the serial number of the unqualified small chip, the location of the unqualified point and the specific defect type.

2. The method for automatic quality inspection of protein chips based on machine vision according to claim 1, characterized in that: The steps for obtaining parameter information and image data related to protein chip quality inspection include: Acquiring original multispectral image data using an integrated multispectral imaging module, wherein the original multispectral image data includes visible light image data, infrared image data, and ultraviolet image data; Performing preliminary noise reduction and filtering processing on the original multispectral image data; Performing wavelet transform on the pre-processed original multispectral image data to decompose it into characteristic sub-images of different scales and directions; Performing principal component analysis on the characteristic sub-images of each spectral channel based on wavelet transform, calculating the covariance matrix of each characteristic sub-image and obtaining the principal components through eigenvalue decomposition; The extracted characteristic components of each spectral channel are weightedly fused, and weight coefficients are assigned according to the sensitivity and importance of each spectral channel to different features to generate a fused image containing information on the surface material composition, thickness and defect depth of the protein chip.

3. The method for automatic quality inspection of protein chips based on machine vision according to claim 2, characterized in that: The steps of performing principal component analysis on the characteristic sub-images of each spectral channel based on wavelet transform, calculating the covariance matrix of each characteristic sub-image and obtaining the principal components through eigenvalue decomposition include: Calculating the covariance matrix of each of the feature sub-images; Performing eigenvalue decomposition on the covariance matrix to obtain eigenvalues and corresponding eigenvectors; Sort the eigenvectors in descending order of the eigenvalues, select the first several eigenvectors whose cumulative contribution rate reaches the set threshold, and the components corresponding to the first several eigenvectors are the extracted principal components; The projection coefficient of the principal component on each of the characteristic sub-images is used as the characteristic component of each of the characteristic sub-images.

4. The method for automatic quality inspection of protein chips based on machine vision according to claim 2, characterized in that: The method also includes: Acquire 3D topographic data of the protein chip surface, calibrate local details through data registration algorithms, and form a high-precision 3D point cloud model covering the entire chip; Performing spatial registration on the three-dimensional point cloud model and the fused image corresponding to the original multispectral image data to establish a pixel-level spatial coordinate mapping relationship; Extracting surface curvature from the three-dimensional point cloud model, extracting spectral features of the defect area from the fused image, establishing a linear correlation model between the two through principal component analysis, and determining a mapping formula between surface curvature change and defect depth; Based on the mapping formula, three-dimensional depth inversion is performed on the suspicious area in the fused image to automatically identify stereo defects.

5. The method for automatic quality inspection of protein chips based on machine vision according to claim 2, characterized in that: The method also includes: During multispectral imaging, continuous frames of fluorescence images of the same area are collected synchronously, and the fluorescence intensity decay rate between adjacent frames is calculated; When the fluorescence intensity decay rate exceeds a preset threshold, it is determined that fluorescence quenching occurs; Establish a spatial distribution map of fluorescence quenching, mark the location and degree of quenching areas, and generate a quenching risk heat map; Based on the quenching risk heat map, the excitation light parameters are dynamically adjusted for high-risk areas. Pulsed excitation light is used instead of continuous light, and the duty cycle and frequency are optimized. At the same time, the excitation light intensity is increased and the exposure time is shortened for areas with severe quenching. A feedback control algorithm is used to keep the fluorescence signal intensity in the target area within the linear response range of the detection system. For the multispectral image data of the quenched region, a joint spatiotemporal denoising algorithm is used. In the temporal dimension, the features of adjacent unquenched frames are used for interpolation and restoration. In the spatial dimension, the surface curvature information in the 3D topography data is combined to suppress artifacts. A generative adversarial network is then used to reconstruct the molecular distribution image of the quenched region. When it is detected that the average fluorescence decay rate of the entire batch of chips exceeds the set ratio, the early warning mechanism is automatically triggered and prompts to replace the fluorescent labeling reagent or adjust the biological sample pretreatment process. A quenching history database is established to analyze the quenching characteristics of different batches of chips and fluorescent dyes, providing parameter optimization suggestions for subsequent quality inspection tasks.

6. The method for automatic quality inspection of protein chips based on machine vision according to claim 1, characterized in that: The steps of compensating for mechanical motion errors of the external hardware system include: Integrate a multi-dimensional sensor array on the motion platform of the external hardware system to collect vibration, displacement and posture data of the motion platform in real time during its motion; Build a dynamic error model based on the collected vibration, displacement and posture data, adjust motion parameters in real time, and compensate for positioning deviations caused by mechanical vibration and thermal drift; Combining real-time position feedback with image feature matching, the scanning path planning is dynamically optimized to ensure that the overlapping accuracy of adjacent sub-images meets the preset stitching requirements; To address image blur caused by motion, we use physical model-based image restoration technology combined with edge enhancement algorithm to improve the recognizability of defect features and ensure detection accuracy under complex motion conditions.

7. The method for automatic quality inspection of protein chips based on machine vision according to claim 1, characterized in that: The steps after outputting the analysis conclusion of protein chip quality inspection include: Establish closed-loop feedback with sample production equipment via industrial Ethernet; When the same type of defects are detected on several protein chips in succession, the defect position coordinates and corresponding scanning parameters are automatically extracted; Based on the spatial cluster analysis of the defect position coordinate distribution, determine whether the defect is caused by the positioning deviation of the spotting head or the abnormal injection parameters; If the sample head positioning deviation, adjust the sample head X / Y axis motion parameters; If the injection parameters of the sample head are abnormal, the injection pressure and pulse width of the sample head will be dynamically corrected; A quality inspection data traceability system is established to associate the defect type, three-dimensional morphology data, and multi-spectral characteristic components of each chip with the production batch and equipment parameters to form a quality control database.

8. A protein chip automatic quality inspection system based on machine vision, characterized in that: include: Parameter information and image data acquisition module: acquires parameter information and image data related to protein chip quality inspection, wherein the parameter information includes communication setting parameters, chip related information, scanning parameters and dot matrix parameters, and the image data is collected by the external hardware system and transmitted to the software system; Communication connection establishment module: establishing a communication connection between the external hardware system and the software system according to the acquired communication setting parameters; Mechanical motion error compensation module: obtains the position information of multiple reference points on the protein chip, determines the actual coordinate offset of the protein chip in the X-axis and Y-axis directions by locating the reference points, and compensates for the mechanical motion error of the external hardware system in combination with the preset chip spacing parameters; Image data analysis and processing module: divides the image data of the intact protein chip into multiple regional sub-images, performs zooming, shrinking, and reloading operations on each regional sub-image, and simultaneously applies normalization and equalization image enhancement processing methods to improve the image contrast for more accurate subsequent image analysis; Protein chip image stitching module: based on the image feature matching algorithm, the pre-processed sub-images of the region are automatically stitched together according to the coordinate positions to form a complete and clear protein chip image; Defect type identification module: automatically analyzing the spliced complete protein chip image according to the lattice parameters, and identifying the defect type in the protein chip lattice using an integrated target detection algorithm; Analysis conclusion output module: outputs the analysis conclusion of the protein chip quality inspection, which includes the chip qualification judgment, the serial number of the unqualified small chip, the location of the unqualified point and the specific defect type.

9. A protein chip automatic quality inspection device based on machine vision, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be loaded by the processor and executes any one of the machine vision-based automatic quality inspection methods for protein chips according to claims 1-7.

10. A storage medium, characterized in that: The invention stores a computer program that can be loaded by a processor and executes any one of the machine vision-based automatic quality inspection methods for protein chips according to claims 1-7.

Images