Multi-spectrum and machine vision fused intelligent product surface quality detection system

Abstract

The invention relates to the technical field of product quality detection, and discloses a multispectral and machine vision fused product surface quality intelligent detection system, which comprises a data acquisition module, a surface feature extraction module, a quality risk analysis module and a defect evaluation module, the data acquisition module acquires a product surface image in real time through a multispectral sensor and a machine vision camera; the surface feature extraction module constructs a defect feature information set and environment illumination feature information based on the collected images; the quality risk analysis module obtains a quality defect risk value in combination with the environment illumination feature information, the product standard state information, the current detection path and the detection device state information; the defect evaluation module generates a plurality of defect detection schemes when the risk value is abnormal, calculates interference influence values of the schemes on the detection state, and screens out the optimal scheme, and the system realizes multi-technology deep fusion, comprehensively considers the environment and device states, and improves the detection adaptability and reliability.

Description

Technical Field

[0001] This invention relates to the field of product quality inspection technology, specifically to an intelligent product surface quality inspection system that integrates multispectral and machine vision. Background Technology

[0002] In modern manufacturing, product surface quality directly affects product performance, lifespan, and market competitiveness. Therefore, surface quality inspection has always been a crucial part of the production process. Traditional product surface quality inspection relies heavily on manual visual inspection. This method not only consumes a lot of manpower but also makes the inspection results susceptible to factors such as the inspector's experience, fatigue, and subjective judgment. This results in problems such as low inspection efficiency, poor consistency, and high rates of missed and false detections, making it difficult to meet the requirements of modern large-scale production for inspection accuracy and efficiency.

[0003] With the development of automation and intelligent technologies, machine vision-based inspection technology is increasingly being applied to product surface quality inspection. Machine vision inspection technology acquires images of the product surface through cameras and analyzes these images using image processing algorithms to identify and judge surface defects. However, single machine vision inspection technologies often have limitations when faced with complex product surface features or changing environmental conditions. For example, under unstable lighting conditions, images acquired by machine vision cameras may suffer from insufficient contrast and blurred details, leading to inaccurate defect feature extraction and affecting the reliability of the inspection results.

[0004] To overcome the limitations of single-mode detection technologies, multispectral detection technology has gradually gained attention. Multispectral detection technology can image the surface of a product using spectral information at different wavelengths, thereby acquiring more surface feature information and helping to improve the ability to identify subtle or hidden defects. However, multispectral detection technology also has its own shortcomings, such as large data processing volume and sensitivity to changes in the state of the detection device. In practical applications, the state of the detection device changes with time, environment, and other factors. If the impact of these changes on the detection results is not considered in a timely manner, it may lead to a decrease in detection accuracy.

[0005] Currently, although some research has attempted to combine machine vision and multispectral technology for product surface quality inspection, most existing fusion inspection systems simply overlay data from both technologies, failing to fully utilize their advantages for deep integration. During defect detection, the impact of factors such as changes in ambient lighting and fluctuations in the state of the inspection equipment on the results is often overlooked. Furthermore, there is a lack of assessment and optimization of the interference effects of different defect detection schemes, resulting in insufficient adaptability and robustness of the inspection system, making it difficult to stably and efficiently complete product surface quality inspection tasks in complex and ever-changing production environments. Therefore, developing an intelligent product surface quality inspection system that can achieve deep fusion of multispectral and machine vision and comprehensively consider the influence of environmental and equipment conditions has become a problem that needs to be solved in the manufacturing industry. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent product surface quality inspection system that integrates multispectral and machine vision technologies to solve the problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides an intelligent product surface quality inspection system that integrates multispectral and machine vision technologies, the system comprising:

[0008] The data acquisition module acquires images of the product surface in real time using a multispectral sensor and a machine vision camera.

[0009] A surface feature extraction module, which constructs a set of defect feature information and ambient light feature information of the product surface based on the acquired image;

[0010] The quality risk analysis module, based on the ambient light characteristic information, combined with the product's standard status information, the current detection path, and the current time of the detection device's status information, obtains the product's quality defect risk value based on the defect characteristic information set.

[0011] The defect assessment module generates various defect detection schemes based on defect feature information when the quality defect risk value is abnormal; it calculates the cooperative interference value corresponding to the current time when the detection device switches to the standard state by combining the ambient light feature information, obtains the interference impact value of each defect detection scheme on the detection state, selects the best defect detection scheme for the product, and generates the best defect detection scheme for the current time.

[0012] Preferably, the system further includes:

[0013] The detection scheme execution module executes surface quality detection actions based on the optimal defect detection scheme at the current time and collects actual quality data.

[0014] The calibration analysis module determines the detection error based on the deviation between the actual quality data and the expected quality, and corrects the optimal defect detection scheme based on the detection error to generate the corrected optimal defect detection scheme.

[0015] The detection scheme update feedback module updates the current detection path of the product in the database in real time based on the corrected optimal defect detection scheme; and generates detection log information based on the update results and feeds it back to the administrator in real time.

[0016] Preferably, in the data acquisition module, the multispectral sensor includes an infrared spectral sensor, a visible spectral sensor, or an ultraviolet spectral sensor;

[0017] The surface feature extraction module extracts a set of defect feature information and ambient light feature information of the product surface based on the images acquired by the multispectral sensor and machine vision camera.

[0018] Preferably, in the surface feature extraction module, each element in the defect feature information set corresponds to a defect feature information, and the defect feature information includes the defect location, defect size, and defect type extracted based on the acquired image;

[0019] The quality risk analysis module calculates the quality defect risk value of the product based on the defect feature information set.

[0020] Preferably, in the surface feature extraction module, the ambient lighting feature information includes the lighting intensity and lighting uniformity of different image regions in the acquired image;

[0021] The defect assessment module, in conjunction with the ambient light characteristic information, calculates the cooperative interference value corresponding to the current time detection device switching to the standard state.

[0022] Preferably, in the quality risk analysis module, the standard status information of the product includes the orthographic projection of the product's surface contour under standard conditions onto the detection plane and a first spacing; the status information of the current time detection device includes the orthographic projection of the focal point of the current time detection device onto the detection plane and a second spacing; the first spacing represents the maximum value of the distance between the detection point and the boundary within the corresponding orthographic projection in the standard status information; the second spacing represents the maximum value of the distance between the detection point and the boundary within the corresponding orthographic projection in the current time detection device's status information;

[0023] The quality risk analysis module calculates the quality defect risk value based on the first spacing, the second spacing, and the overlap width of the defect contour.

[0024] Preferably, when the quality risk analysis module determines whether the quality defect risk value is abnormal, it will determine the quality defect risk value that is greater than a preset threshold as abnormal;

[0025] When the quality defect risk value is abnormal, the defect assessment module generates various defect detection schemes based on defect feature information.

[0026] Preferably, when the defect assessment module generates each defect detection scheme, it obtains the detection boundary nodes corresponding to each defect in the defect feature information, marks the defect outline and detection boundary nodes in the current detection path, divides the safe detection area by the line connecting the current position of the detection device and each detection boundary node, and obtains the defect detection scheme for each safe detection area based on the area division algorithm.

[0027] When the detection scheme execution module performs the surface quality detection action, it collects actual quality data based on the defect detection scheme of the safe detection area.

[0028] Preferably, when the defect assessment module calculates the collaborative interference value, it calculates the first interference value based on the number of operation instructions for state switching in the database, calculates the second interference value in combination with the second interval, and obtains the collaborative interference value by combining the first interference value and the second interference value.

[0029] When the correction analysis module corrects the optimal defect detection scheme based on the detection error, it uses the cooperative interference value to generate the corrected optimal defect detection scheme.

[0030] Preferably, when the correction analysis module determines the detection error based on the detection error, it inputs the deviation value between the actual quality data and the expected quality into the adaptive control model, generates basic control parameter correction values, and corrects the optimal defect detection scheme based on the basic control parameter correction values.

[0031] The detection scheme update feedback module updates the current detection path of the product according to the corrected optimal defect detection scheme.

[0032] When the detection scheme update feedback module generates detection log information, the detection log information includes the log generation time, the corresponding corrected optimal defect detection scheme, the defect feature information set obtained before obtaining the corresponding corrected optimal defect detection scheme, ambient light feature information, current detection path, and the status information of the detection device at the current time.

[0033] The correction analysis module optimizes the adaptive control model based on the defect feature information set and ambient light feature information in the detection log information.

[0034] Compared with the prior art, the beneficial effects of the present invention are:

[0035] By simultaneously acquiring product surface images using a multispectral sensor and a machine vision camera through a data acquisition module, the advantages of both technologies can be integrated to capture multidimensional feature information of the product surface. The multispectral sensor provides image data across different spectral bands, helping to uncover hidden defects on the product surface, while the machine vision camera can quickly acquire high-resolution surface images. The combination of the two results in richer and more comprehensive image information, providing more reliable raw data for subsequent defect detection.

[0036] The surface feature extraction module constructs a defect feature information set and ambient lighting feature information based on the acquired images, enabling simultaneous analysis of product surface defects and ambient lighting conditions. This processing method does not simply extract single features from the image, but rather correlates defect features with ambient lighting features, allowing the system to clearly understand the impact of environmental factors on image quality, laying the foundation for subsequent quality risk analysis and defect assessment.

[0037] When analyzing the quality defect risk value, the quality risk analysis module comprehensively considers ambient lighting characteristics, the product's standard state information, the current testing path, and the current state of the testing device. This means that when assessing product surface quality, it not only focuses on the product's own defect characteristics but also fully incorporates the influence of external factors such as the environment and device status. This makes the obtained quality defect risk value more consistent with actual testing scenarios and more accurately reflects the possibility of quality problems on the product surface.

[0038] When the quality defect risk value is abnormal, the defect assessment module generates multiple defect detection schemes and evaluates the interference impact of different schemes on the detection status by calculating the collaborative interference value, thereby selecting the optimal defect detection scheme. This approach avoids the fluctuations in the detection status that may be caused by blindly selecting a detection scheme, and can reduce the impact on the stability of the detection system while ensuring the defect detection effect. By quantitatively analyzing the interference impact of different schemes, the most suitable detection scheme can be selected according to the actual situation, making the system more adaptable and flexible in the face of different types of defects and complex environmental conditions.

[0039] The entire system achieves a deep integration of multispectral and machine vision technologies, going beyond a simple superposition of the two technologies. It organically combines data acquisition, feature extraction, risk analysis, and solution evaluation across all stages. By comprehensively considering factors such as ambient lighting and the status of the detection equipment, the system's adaptability to complex environments is enhanced. During defect detection, optimized selection of detection schemes reduces unnecessary interference and ensures the stability of the detection process. This system can more comprehensively and accurately identify product surface defects, maintaining stable detection performance in complex and changing production environments, and meeting the demands of modern production for product surface quality inspection. Attached Figure Description

[0040] Figure 1 This is a timing diagram of the intelligent product surface quality inspection system that integrates multispectral and machine vision as described in this invention.

[0041] Figure 2 A flowchart for the execution, correction, and update feedback of the testing plan;

[0042] Figure 3 A flowchart for calculating the risk value of quality defects;

[0043] Figure 4 A flowchart for the generation and execution of a defect detection scheme;

[0044] Figure 5 This is a flowchart for the work of correcting, updating and optimizing the detection scheme. Detailed Implementation

[0045] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0046] Please see Figure 1 This invention provides an intelligent product surface quality inspection system that integrates multispectral and machine vision technologies, the system comprising:

[0047] By fusing multispectral sensors and machine vision cameras, intelligent inspection of product surface quality is achieved. During system operation, the data acquisition module acquires real-time images of the product surface using the multispectral sensors and machine vision cameras. The surface feature extraction module constructs a set of defect feature information and ambient lighting feature information based on the acquired images. The quality risk analysis module, based on the ambient lighting feature information, combined with the product's standard state information, the current inspection path, and the current state information of the inspection device, obtains the product's quality defect risk value based on the defect feature information set. When the quality defect risk value is abnormal, the defect assessment module generates various defect detection schemes based on the defect feature information; combined with the ambient lighting feature information, it calculates the cooperative interference value corresponding to the current time when the inspection device's state switches to the standard state, obtains the interference impact value of each defect detection scheme on the inspection state, selects the optimal defect detection scheme for the product, and generates the optimal defect detection scheme for the current time.

[0048] Example 1: See Figure 2The system is deployed at a surface defect inspection station for precision electronic components. Once the data acquisition module is activated, its built-in infrared spectral sensor and a high-resolution machine vision camera begin working synchronously. The infrared sensor can penetrate the tiny coating on the component's surface to detect structural anomalies beneath, while the visible light camera captures scratches, dents, or contaminants on the surface. They acquire images of the products on the conveyor belt at a rate of several frames per second, ensuring full surface coverage. The acquired multispectral and visible light images are transmitted in real time to the surface feature extraction module.

[0049] After receiving the image data, the surface feature extraction module first performs preprocessing. Due to vibrations in the production line environment, the image may be slightly blurred; therefore, a digital image stabilization algorithm is used for compensation. Subsequently, the registered multispectral image and the visible light image are fused to enhance the contrast of image features. The module runs a deep learning-based feature recognition algorithm, trained on a large number of defect samples, capable of accurately segmenting potential defect regions from the fused image. For each identified region, the algorithm calculates its geometric center coordinates as the defect location, calculates its pixel area, and converts it to actual physical dimensions as the defect size based on calibration parameters. Simultaneously, it classifies the defect type based on its texture and spectral response characteristics, such as "cracks," "impurities," or "uneven coating." All this information is structured and stored, collectively forming the product's defect feature information set. Meanwhile, the module analyzes the global grayscale distribution of the visible light image, dividing the image into multiple grid regions. It calculates the average pixel brightness value of each region to obtain the illumination intensity and calculates the standard deviation of the brightness values ​​across all regions to quantify the illumination uniformity. These data are recorded as ambient lighting feature information.

[0050] The quality risk analysis module then receives the aforementioned feature information. This module contains standard state information for this type of electronic component, including the orthographic projection image of the component's surface contour onto the detection plane under ideal laboratory lighting and calibrated focal length, and the maximum distance (first gap) from any point within the orthographic projection image to its contour boundary. Simultaneously, the module acquires the actual state of the current production line inspection device via sensors: the actual projection position of its optical focal point onto the detection plane, and the maximum distance (second gap) from the focal point within this projection to the boundary. The module compares the current defect feature information set with the standard state, and, combining the difference between the first and second gaps and the defect distribution, calculates a comprehensive quality defect risk value. If this risk value does not exceed a preset threshold, the system determines the product risk is controllable, and the process continues; if the risk value is abnormal, the defect assessment module is immediately triggered.

[0051] Once activated, the defect assessment module's primary task is to generate multiple alternative defect detection schemes based on the abnormal defect feature information set. It extracts the boundary node coordinates of each defect from the information set and precisely marks the contours of all defects and these nodes on the current default detection path map. Subsequently, the module connects the current physical location of the detection device to each defect boundary node, thereby dividing the detection plane into multiple independent safe detection zones. Each zone contains one or more defects. For each zone, the module generates a customized detection scheme, which may include adjusting the camera's macro focal length to obtain higher resolution images, changing the band of the multispectral sensor to highlight specific defect features, or adjusting the angle and intensity of the auxiliary light source to overcome the effects of uneven ambient lighting.

[0052] The defect assessment module needs to evaluate the potential negative impacts of adjusting the detection device's state, i.e., calculate the cooperative interference value. It queries the system database to obtain the number of control commands required to switch the current detection device state (such as focal length and light source intensity) back to the standard state. A higher number of commands indicates more complex operation, thus generating the first interference value. Simultaneously, the deviation between the current second spacing and the standard first spacing is used to calculate the second interference value; a larger deviation indicates a greater deviation from the current state. Combining these two factors, the module calculates a cooperative interference value. This value quantifies the degree of interference each alternative detection scheme causes to the overall system stability and detection efficiency during execution. Finally, the module selects the scheme with the lowest cooperative interference value while maintaining detection accuracy, determining it as the optimal defect detection scheme for the current time.

[0053] The detection scheme execution module receives and executes this optimal scheme. It controls the robotic arm and optical sensors to rescan and image the designated safe detection area according to the planned path and parameters, collecting more accurate actual quality data during this process, such as the precise depth of defects and clear spectral signatures. This high-precision actual quality data is sent to the calibration analysis module, which compares the actual data with the system's preset expected quality data based on product standards, calculates the deviation value, and determines the detection error. This error reveals potential minor deviations in the optimal defect detection scheme. Subsequently, the calibration analysis module uses the previously calculated cooperative interference value as a correction coefficient to fine-tune the parameters of the optimal defect detection scheme, such as slightly adjusting the scanning speed or exposure time, to compensate for system errors, thereby generating a corrected optimal defect detection scheme.

[0054] The testing plan update feedback module then takes action, writing the corrected plan into the central database and updating the current testing path for that product model, enabling subsequent testing of similar products to directly utilize this optimized path. Simultaneously, the module automatically generates a detailed testing log, including the timestamp of the event, the testing plan before and after correction, the original defect feature information set and ambient lighting characteristics that triggered the adjustment, and the original status data of the testing equipment. This log is pushed in real-time to the quality administrator's monitoring terminal via the factory network, providing the administrator with a complete audit trail. The entire process is now complete, and the system continues to run, constantly optimizing itself using feedback data.

[0055] Example 2: After the data acquisition module is started, the configured ultraviolet spectral sensor and high frame rate machine vision camera synchronously acquire images. The ultraviolet sensor is highly sensitive to fine scratches and oil stains on the surface of the board, while the visible light camera captures surface irregularities and color anomalies. The acquired image data stream is transmitted in real time to the surface feature extraction module for processing.

[0056] After receiving the multispectral image stream, the surface feature extraction module first performs image registration and fusion. Due to the extremely high speed of the production line, the module adopts a fast registration algorithm based on feature points to ensure that images of the same area acquired by different sensors are spatially aligned. Subsequently, an image fusion algorithm is applied to merge ultraviolet and visible light image information to enhance the contrast between defects and the background. Next, the module runs a multi-scale feature extraction algorithm. This algorithm first segments all potential defect areas from the fused image through edge detection and region growing techniques. For each segmented region, the module calculates the coordinates of the center point of its minimum bounding rectangle as the defect location information. At the same time, it calculates the total number of pixels in the region and converts it into the actual physical area according to a pre-calibrated millimeter-to-pixel ratio coefficient, which is taken as the defect size. For defect type identification, the module adopts a lightweight convolutional neural network model. This model classifies defects into categories such as "scratches," "indentations," "oil stains," or "oxidation spots" by analyzing the texture features, morphological features, and response differences of the region in the ultraviolet and visible light bands. All this information—the location, size, and type of each defect—is encapsulated into a structured data set, namely, a defect feature information set.

[0057] While generating the defect feature information set, this module simultaneously extracts ambient lighting feature information. This process is primarily based on visible light image analysis. The module divides each frame of the image into several regular grid regions and calculates the average grayscale value of all pixels within each grid, representing the lighting intensity of that grid region. Subsequently, it calculates the standard deviation of all these grid lighting intensity values, which is used to quantify the lighting uniformity of the entire image. The lighting intensity distribution and uniformity data are jointly recorded as ambient lighting feature information. This information reflects the ambient light conditions at the time of detection, and may include uneven lighting caused by the angle of overhead lighting or reflections from surrounding equipment.

[0058] After receiving the two sets of feature information, the quality risk analysis module begins operation. This module accesses the database to obtain the standard quality parameters for this batch of aluminum alloy sheets. The module internally presets risk weight coefficients for different defect types; for example, "cracks" have the highest weight coefficient, followed by "indentations," and "oil stains" have relatively lower weights. The module iterates through each element in the defect feature information set, obtains the corresponding weight based on its labeled defect type, and then combines this with the defect size to perform a weighted calculation. Finally, it sums the weighted risks of all defects to obtain a comprehensive quality defect risk value. This value is a quantitative assessment of the overall quality risk of the current sheet surface.

[0059] The process enters the defect assessment phase. One of the core tasks of this module is to calculate the co-interference value, where ambient lighting characteristics play a crucial role. The module simulates the process of adjusting the current detection device (including the camera and light source) parameter settings back to a standard state. It combines the currently extracted ambient lighting characteristics—the actual light intensity and uniformity of each area—to predict how this adjustment will change the current imaging environment. For example, if the current ambient light has significant non-uniformity (high standard deviation), while the standard state requires uniform illumination, the switching process may cause local overexposure or underexposure of the image. This potential image quality degradation is quantified as a co-interference value. This interference value reflects the new uncertainties that may be introduced in pursuit of a standard detection state.

[0060] The defect assessment module comprehensively considers quality defect risk values ​​and co-interference values ​​to generate and select the optimal defect detection solution. This solution strikes a balance between detailed defect re-inspection and maintaining overall system stability. For example, it might decide to use a higher resolution scan for high-risk defect areas while accepting slight gradients in ambient lighting, rather than performing a radical lighting adjustment that could cause greater interference. The entire process demonstrates the system's deep integration of multi-source information and intelligent decision-making capabilities.

[0061] Example 3: See Figure 3At the start of the inspection, the data acquisition module uses a configured multispectral sensor and a high-resolution industrial camera to acquire images of the glass cover plate on the conveyor belt. The surface feature extraction module processes the acquired images, identifies defects such as microscopic scratches and uneven melting areas on the surface, and constructs a defect feature information set containing the location, size, and type of each defect. Simultaneously, this module analyzes the ambient lighting conditions during imaging and generates ambient lighting feature information.

[0062] Once the quality risk analysis module is activated, it first retrieves the standard status information for this model of glass cover from the system database. This information is pre-measured and stored under ideal laboratory conditions, and one of its core components is the orthographic projection of the product's surface contour onto the inspection plane. This orthographic projection is a precise two-dimensional geometric figure that defines the standard boundaries of the product within the inspection field of view. More importantly, the first spacing associated with this orthographic projection ( ) are called together. First spacing ( The first spacing is a geometric parameter defined as follows: within the orthographic projection of the standard state information, the shortest distance from any virtual detection point to the boundary of the orthographic projection has a maximum value, which is the first spacing. It can be intuitively understood as the diameter of the largest inscribed circle that can be accommodated within the orthographic projection, and its size reflects the coverage area of ​​the standard detection region.

[0063] Almost simultaneously, the module acquires the current status information of the detection device in real time through displacement and attitude sensors integrated on the detection device. This includes the actual orthographic projection position of its optical focal point on the detection plane. Due to vibrations in the production line, minor deformations of the equipment, or thermal drift, this projection position typically deviates slightly from the standard state. A second spacing associated with this projection ( The second spacing is calculated in real time. The definition of the first spacing is similar, but its calculation is based on the orthographic projection in the current time detection device status information. It represents the maximum value of the shortest distance from the detection point to the boundary within the current actual projected pattern, reflecting the actual size of the effective detection area at the current moment.

[0064] This module incorporates geometric information extracted from the defect feature information set—the overlap width of the defect contour ( This parameter measures the degree of overlap between the identified defect area and the currently effective detection area (defined by the current orthographic projection). A larger overlap width means that the defect is mainly located inside the detection area and can be effectively monitored; while a smaller overlap width may indicate that the defect is located at the edge or even outside the detection area, posing a risk of missed detection.

[0065] Based on the above three core parameters: the first spacing ( ), second spacing ( ) and the width of the overlap of the defect profile ( The quality risk analysis module performs a comprehensive calculation process to quantify risk. This process integrates these geometric and positional factors through a mathematical formula:

[0066]

[0067] In this relation: This represents the final calculated risk value for quality defects. The calculation logic for this relationship is as follows: ratio This reflects the proportion of the degree of overlap between the defect and the currently detected area relative to the size of the currently detected area. (Logarithmic terms) This is used to assess the degree of deviation of the current detection device state from the standard state, and it introduces a logarithmic function to smoothly handle different degrees of deviation. The entire relationship shows that the risk value... It is simultaneously affected by two factors: "whether the defect is well covered" and "whether the detection device is in standard condition." The more inadequate the defect coverage or the more serious the deviation of the device condition, the worse the calculated results. The larger the value, the better.

[0068] The system has a preset risk threshold. The quality risk analysis module will calculate the real-time risk value. Compare with this threshold. If The system determines that the current overall risk is within an acceptable range, and the testing process continues as planned. If the quality defect risk value is deemed abnormal, it indicates that the current detection status may not reliably capture identified defects, posing a potential quality control risk. This abnormal signal immediately triggers the downstream defect assessment module. The defect assessment module then generates a series of alternative, more targeted defect detection schemes based on the extracted defect feature information set. These schemes aim to recapture and confirm defects by adjusting the detection path or parameters, thereby addressing this risk anomaly. The entire analysis process embodies the logic of making objective and quantitative decisions about quality control risks based on geometric and physical relationships.

[0069] Example 4: See Figure 4In the automotive manufacturing industry, the exterior metal panels of a high-end sedan require surface quality inspection after painting. These panels are large and have complex curvature, placing high demands on the path planning and anti-interference capabilities of the inspection system. The implementation method described in this embodiment is applied in this scenario. After the inspection system is initialized, the data acquisition module scans the panel using a multispectral sensor and a 3D machine vision camera. The surface feature extraction module analyzes the acquired images, identifies several potential defects, including "paint impurities," "orange peel texture," and "minor runs," and constructs a defect feature information set containing the location, size, and type of these defects. Based on the current inspection status and ambient lighting information, the quality risk analysis module calculates a safety risk value exceeding a threshold, determines that there is a risk of missed detection in the current inspection state, and then activates the defect assessment module.

[0070] The primary task of the defect assessment module is to generate multiple detailed alternative inspection schemes. It extracts the geometric information of each defect from the defect feature information set. For point defects such as "paint impurities," the center point is taken as the inspection boundary node; for area defects such as "orange peel texture," the four vertices of the bounding rectangle are taken as nodes; and for linear defects such as "minor runs," the two endpoints are taken as nodes. All these nodes are precisely marked on a digital 3D mesh map formed by the current inspection path, and the outline of the defect is delineated accordingly.

[0071] The module connects the real-time coordinates of the detection device (a robotic arm equipped with sensors) to each detection boundary node marked on the map. These lines intertwine in three-dimensional space, dividing the panel surface to be inspected into several non-overlapping safe inspection zones. Each zone contains one or a group of spatially adjacent defects. The module runs a zone division algorithm to generate a tailored defect detection plan for each safe inspection zone. The plan is not a simple parameter adjustment, but a complete set of detection instructions, which may include: for "paint impurities" areas, instructing the robotic arm to perform low-altitude, close-range scanning using a zigzag path, while simultaneously increasing the sensitivity of the ultraviolet spectral sensor; for large areas of "orange peel texture," instructing the robotic arm to sweep at a uniform speed along the long side, using a side light source at a specific angle to highlight the texture; for "minor runs" areas, instructing high-precision line scanning and simultaneously triggering multiple images for three-dimensional reconstruction.

[0072] These initially generated solutions all aim to capture defects more accurately, but they all require the detection device to switch from its current state to a new operating state. The defect assessment module needs to evaluate the system overhead and potential risks of this switching behavior itself, i.e., calculate the co-interference value. This calculation process is divided into two parts. First, the module accesses the system database to query the number of low-level control commands required to execute each alternative solution, such as the number of commands to adjust the robotic arm joint angle, change the focal length, or switch the light source mode. The more commands there are, the more complex the operation, the longer it takes, and the greater the impact on the overall detection cycle time, thus generating a first interference value. Second, the module calculates the second distance (a geometric parameter reflecting the degree of deviation of the current detection focus from the standard state) in the current detection device state information. The larger the value of the second distance, the more the current state deviates from the ideal standard, and the greater the "inertia" or deviation that needs to be overcome to adjust it back to the standard state, thus generating a second interference value. Finally, the module combines the first and second interference values ​​through an internal algorithm to obtain a co-interference value that characterizes the state switching cost and instability, as shown in Table 1.

[0073] Table 1: Defect detection scheme and evaluation of synergistic interference value.

[0074]

[0075] With the evaluation data shown in the table, the decision-making logic of the defect evaluation module becomes clear. It needs to select a solution that achieves the best balance between detection accuracy and system stability. A solution with excessively high co-interference values, even if theoretically the best in terms of detection performance, may be counterproductive due to the introduction of too many unstable factors. The module selects a solution that effectively covers the target defect while keeping the co-interference value within an acceptable range, identifying it as the optimal defect detection solution for the current time. The detection solution execution module receives and executes this solution, controlling the robotic arm and sensors to perform a fine scan of the designated safe detection area, collecting more detailed actual quality data, such as the precise composition spectrum of impurities, the roughness quantification value of orange peel texture, and the cross-sectional shape data of drips.

[0076] This high-precision data is fed into the calibration analysis module, which compares the actual data with the expected standard and calculates the minute detection error. It's important to note that the calibration process is not simply a matter of adjusting a few parameters. The module uses previously calculated co-interference values ​​as one of the key inputs. These co-interference values ​​quantify the "cost" of system adjustments; therefore, in the calibration algorithm, they serve as a constraint to ensure that the generated calibration scheme does not blindly require the system to undergo significant secondary adjustments in pursuit of ultimate accuracy, thus avoiding a cycle of over-adjustment. The calibration analysis module ultimately outputs a more robust and optimal defect detection scheme after calibration, taking into account historical operating costs. This scheme is then used to update the database, guiding the detection process of subsequent similar panels, completing the closed loop from perception to decision-making to optimization.

[0077] Example 5: See Figure 5 In the manufacturing process of precision optical components, the requirements for surface cleanliness and microscopic scratch detection are extremely stringent. The detection process is initiated by the data acquisition module, where a multispectral sensor and a high-resolution camera work together to capture microscopic images of the component surface. The surface feature extraction module processes the images, identifying nanoscale dust particles, watermarks, and fine scratches, constructing a defect feature information set containing their location, size, and type, and simultaneously recording the ambient lighting characteristics during the capture. After the quality risk analysis module determines an abnormal risk, it triggers subsequent modules to generate and execute a targeted optimal defect detection plan. The detection plan execution module controls the detection device to re-inspect the component, collecting more accurate actual quality data about the defects, such as the precise height information of dust particles and the spectral reflectance characteristics of watermarks.

[0078] These high-fidelity actual quality data are transmitted to the calibration analysis module, whose core task is to handle the deviation between actual measurements and theoretical expected values. The actual quality data it receives includes quantitative measurements of defects, while the expected quality data is derived from the design specifications of this type of optical element and the standard range defined in the historical database of qualified products. The module uses a comparison algorithm to calculate the deviation values ​​between these quantitative characteristics item by item, such as the difference between the measured scratch depth and the maximum allowable depth according to the standard, or the difference between the actual area covered by dust particles and the area under ideal clean conditions. These deviation values ​​are comprehensively evaluated to determine the systematic and random errors present in the current inspection process, i.e., the inspection error.

[0079] After determining the detection error, the correction analysis module activates its core adaptive control function. This module incorporates an adaptive control model, an algorithmic system capable of dynamically adjusting its internal parameters based on input data. The calculated detection error value is fed into this model as the primary input signal. The model processes these error signals, analyzes their patterns and trends, and then outputs a set of specific correction instructions, namely, basic control parameter correction values. These correction values ​​directly target adjustable parameters in the optimal defect detection scheme. For example, it might suggest fine-tuning the scanning speed by 5% to accommodate more precise capture requirements, adjusting the color temperature of the illumination source to a specific value to better highlight specific types of defects, or increasing the camera's exposure time by a small increment to improve the signal-to-noise ratio. Based on these correction values, the module then updates the parameters of the currently used optimal defect detection scheme, generating a finely tuned and corrected optimal defect detection scheme. This new scheme aims to compensate for the identified systematic errors and improve the accuracy of subsequent detections.

[0080] The inspection scheme update feedback module then begins operation. It receives the corrected optimal defect inspection scheme, and its first action is to access the central database, locate the current inspection path record corresponding to the optical component of that model being inspected, and update it with the new, optimized scheme path. This means that when the next component of the same model passes through the inspection station, the system will directly use this newly corrected and more reliable path for inspection.

[0081] Simultaneously, the module automatically generates a structured inspection log. This log is not simply a record of events, but a technical snapshot containing complete context. It details the timestamp of log generation, the specific content and parameter differences of the optimal defect detection scheme before and after correction, and the original data that triggered this entire correction cycle—namely, the defect feature information set and ambient lighting feature information initially generated by the surface feature extraction module. Furthermore, the original state parameters of the detection device and the version number of the current detection path during re-inspection are also fully recorded. This comprehensive log information is pushed to the quality administrator's monitoring interface in real time, providing a complete basis for audit trails and process diagnosis.

[0082] The functionality of the calibration analysis module doesn't end there; it possesses a crucial long-term self-optimization capability. The module periodically retrieves historical detection logs accumulated by the system, extracting historical defect feature information sets and corresponding ambient lighting characteristics from these massive amounts of log records. This vast amount of historical data, originating from actual production environments, constitutes a valuable dataset for training machine learning models. The calibration analysis module utilizes this data to periodically retrain and optimize the parameters of its internal adaptive control model. By analyzing the characteristics of different defect types under varying lighting conditions and the successful calibration strategies employed historically, the model can continuously learn and adjust its internal decision-making logic. This enables it to generate basic control parameter correction values ​​more accurately and quickly based on detection errors in the future, forming a continuously improving intelligent closed loop.

[0083] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0084] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A product surface quality intelligent inspection system integrating multispectral and machine vision, characterized in that, The system includes: The data acquisition module acquires images of the product surface in real time using a multispectral sensor and a machine vision camera. A surface feature extraction module, which constructs a set of defect feature information and ambient light feature information of the product surface based on the acquired image; The quality risk analysis module, based on the ambient light characteristic information, combined with the product's standard status information, the current detection path, and the current time of the detection device's status information, obtains the product's quality defect risk value based on the defect characteristic information set. The defect assessment module generates various defect detection schemes based on defect feature information when the quality defect risk value is abnormal; it calculates the cooperative interference value corresponding to the current time when the detection device switches to the standard state by combining the ambient light feature information, obtains the interference impact value of each defect detection scheme on the detection state, selects the best defect detection scheme for the product, and generates the best defect detection scheme for the current time.

2. The intelligent product surface quality inspection system integrating multispectral and machine vision according to claim 1, characterized in that, Also includes: The detection scheme execution module executes surface quality detection actions based on the optimal defect detection scheme at the current time and collects actual quality data. The calibration analysis module determines the detection error based on the deviation between the actual quality data and the expected quality, and corrects the optimal defect detection scheme based on the detection error to generate the corrected optimal defect detection scheme. The detection scheme update feedback module updates the current detection path of the product in the database in real time according to the corrected optimal defect detection scheme; and generates detection log information based on the update results and feeds it back to the administrator in real time.

3. The intelligent product surface quality inspection system integrating multispectral and machine vision according to claim 1, characterized in that, In the data acquisition module, the multispectral sensor includes an infrared spectral sensor, a visible spectral sensor, or an ultraviolet spectral sensor; The surface feature extraction module extracts a set of defect feature information and ambient light feature information of the product surface based on the images acquired by the multispectral sensor and machine vision camera.

4. The intelligent product surface quality inspection system integrating multispectral and machine vision according to claim 3, characterized in that, In the surface feature extraction module, each element in the defect feature information set corresponds to a defect feature information, which includes the defect location, defect size, and defect type extracted based on the acquired image. The quality risk analysis module calculates the quality defect risk value of the product based on the defect feature information set.

5. The intelligent product surface quality inspection system fusion of multispectral and machine vision according to claim 4, characterized in that, In the surface feature extraction module, the ambient lighting feature information includes the lighting intensity and uniformity of different image regions in the acquired image; The defect assessment module, in conjunction with the ambient light characteristic information, calculates the cooperative interference value corresponding to the current time detection device switching to the standard state.

6. The intelligent product surface quality inspection system fusion of multispectral and machine vision according to claim 5, characterized in that, In the quality risk analysis module, the standard status information of the product includes the orthographic projection of the product's surface contour under standard conditions onto the detection plane and a first spacing; the status information of the current time detection device includes the orthographic projection of the focal point of the current time detection device onto the detection plane and a second spacing; the first spacing represents the maximum distance between the detection point and the boundary within the corresponding orthographic projection in the standard status information; the second spacing represents the maximum distance between the detection point and the boundary within the corresponding orthographic projection in the current time detection device's status information; The quality risk analysis module calculates the quality defect risk value based on the first spacing, the second spacing, and the overlap width of the defect contour.

7. The intelligent product surface quality inspection system fusion of multispectral and machine vision according to claim 6, characterized in that, When the quality risk analysis module determines whether the quality defect risk value is abnormal, it will determine the quality defect risk value that is greater than the preset threshold as abnormal. When the quality defect risk value is abnormal, the defect assessment module generates various defect detection schemes based on defect feature information.

8. The intelligent product surface quality inspection system fusion of multispectral and machine vision according to claim 7, characterized in that, When the defect assessment module generates each defect detection scheme, it obtains the detection boundary nodes corresponding to each defect in the defect feature information, marks the defect outline and detection boundary nodes in the current detection path, divides the safe detection area by the line connecting the current position of the detection device and each detection boundary node, and obtains the defect detection scheme for each safe detection area based on the area division algorithm. When the detection scheme execution module performs the surface quality detection action, it collects actual quality data based on the defect detection scheme of the safe detection area.

9. The intelligent product surface quality inspection system fusion of multispectral and machine vision according to claim 8, characterized in that, When the defect assessment module calculates the collaborative interference value, it calculates the first interference value based on the number of state switching operation instructions in the database, calculates the second interference value in combination with the second interval, and obtains the collaborative interference value by combining the first interference value and the second interference value. When the correction analysis module corrects the optimal defect detection scheme based on the detection error, it uses the cooperative interference value to generate the corrected optimal defect detection scheme.

10. The intelligent product surface quality inspection system fusion of multispectral and machine vision according to claim 9, characterized in that, When the correction analysis module determines the detection error based on the detection error, it inputs the deviation value between the actual quality data and the expected quality into the adaptive control model, generates the basic control parameter correction value, and corrects the optimal defect detection scheme based on the basic control parameter correction value. The detection scheme update feedback module updates the current detection path of the product according to the corrected optimal defect detection scheme. When the detection scheme update feedback module generates detection log information, the detection log information includes the log generation time, the corresponding corrected optimal defect detection scheme, the defect feature information set obtained before obtaining the corresponding corrected optimal defect detection scheme, ambient light feature information, current detection path, and the status information of the detection device at the current time. The correction analysis module optimizes the adaptive control model based on the defect feature information set and ambient light feature information in the detection log information.

Images