Intelligent quality detection system based on machine vision

Abstract

The application discloses an intelligent quality detection system based on machine vision and relates to the technical field of data processing, solves the technical problems that it is difficult to automatically adjust light source parameters according to environmental parameter influence coefficients and by using a control algorithm, it is difficult to obtain optimized image information by using a Q-learning algorithm in combination with image quality scoring after image data is subjected to multi-scale fusion alignment, it is difficult to perform defect recognition detection and three-dimensional deformation detection, and it is difficult to analyze quality abnormality coefficients and perform control feedback and visualization. The application realizes efficient and accurate quality detection by adaptive light source control and image optimization in combination with defect detection and three-dimensional deformation evaluation, and by real-time feedback and visualization, it is convenient to quickly understand the defect and deformation state of an object.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and specifically relates to an intelligent quality detection system based on machine vision. BACKGROUND

[0002] Machine vision mainly uses a computer to simulate the visual function of a human being, extracts information from images of objective things, processes and understands the information, and finally is used for actual detection, measurement and control. With the development of machine vision technology, image acquisition devices are constantly improving, and the existing quality detection technology has many deficiencies in light source control, image processing, defect detection, deformation analysis and feedback mechanism, and it is difficult to meet the needs of efficient, accurate and automated quality detection in industrial production. Through the application of deep learning algorithm, intelligent quality detection can accurately identify and classify complex targets in real time and efficiently.

[0003] The existing intelligent quality detection system based on machine vision has the following problems: it is difficult to automatically adjust the light source parameters according to the environmental parameter influence coefficient and by using a control algorithm; it is difficult to obtain optimized image information by using a Q-learning algorithm combined with image quality scoring after multi-scale fusion alignment of image data; it is difficult to perform defect recognition detection and three-dimensional deformation detection; and it is difficult to analyze the quality abnormality coefficient and perform control feedback and visualization. SUMMARY

[0004] The present application aims to at least solve one of the technical problems existing in the prior art; for this purpose, the first aspect of the present application provides an intelligent quality detection system based on machine vision, comprising the following modules:

[0005] Data image acquisition module: real-time acquisition of environmental parameters of a detected object by combining various types of sensors, and automatic adjustment of light source parameters by calculating environmental parameter influence coefficients and using a control algorithm; real-time acquisition of image data of the detected object by combining an optical camera, an infrared camera and a laser scanner according to the automatic adjustment results;

[0006] Data image preprocessing module: data cleaning and normalization preprocessing of environmental parameters; filtering and denoising and image enhancement preprocessing of image data; multi-scale fusion alignment of image data by using a coordinate transformation matrix;

[0007] Quality detection analysis module: automatic adjustment of light source parameters to obtain optimized image information by training light source control behavior using a Q-learning algorithm and combining image quality scoring calculated and analyzed according to the fused image information; defect recognition detection using a YOLOv8 model according to the optimized image information; three-dimensional deformation detection according to the optimized image information; calculation of a quality abnormality coefficient according to the defect and deformation detection results;

[0008] Control feedback module: real-time control feedback according to the quality anomaly coefficient and visual detection of defects and deformation state of the object.

[0009] Further scheme, combined with a variety of types of sensor real-time acquisition of environmental parameters of the detection object, and through the calculation of environmental parameter influence coefficient combined with the use of control algorithm to automatically adjust the light source parameters, including the following steps:

[0010] Using light sensor, temperature sensor, humidity sensor, laser dust sensor and acceleration sensor, real-time acquisition of environmental parameters of the detection object includes: environmental light intensity, environmental temperature, environmental humidity, environmental dust concentration and object motion speed;

[0011] According to the environmental parameters of the detection object, the environmental parameter influence coefficient is calculated and combined with the use of control algorithm to automatically adjust the light source parameters, including: brightness, angle and stroboscopic;

[0012] The formula for calculating the environmental parameter influence coefficient is:

[0013] Where, G(t) is the environmental light intensity at time t; T(t) is the environmental temperature at time t; S(t) is the environmental humidity at time t; F(t) is the environmental dust concentration at time t; V(t) is the object motion speed at time t; t is the time variable, representing the time length from the initial time 0 to the current collection of environmental parameter data;

[0014] The target brightness is: L 目标 = L base *(1+H), where L base represents the reference light source brightness; the target brightness is subtracted from the current brightness to obtain the light source brightness deviation; the light source brightness deviation is input into the PID control algorithm, and the current new light source brightness is automatically adjusted to the current brightness plus the sum of the output values of the PID control algorithm; similarly, the above automatic adjustment of light source brightness step, the current new light source angle and new stroboscopic can be obtained.

[0015] Further scheme, through coordinate transformation matrix, multi-scale fusion alignment of image data, including the following steps:

[0016] According to the results of automatic adjustment of light source parameters, real-time acquisition of image data includes: using optical camera to obtain the appearance image of the detection object, using infrared camera to capture the thermal radiation information of the detection object, and using laser scanner to emit light beam to measure the point cloud data of the three-dimensional structure of the detection object;

[0017] The chessboard calibration method or Zhang Zhengyou calibration method is used to calibrate the optical camera, the infrared camera and the laser scanner respectively, and the internal parameters and external parameters are calculated; the internal parameters include the focal length and the principal point position, and the external parameters include the rotation matrix and the translation vector;

[0018] The transformation matrix of the world coordinate system to the target coordinate system of the three sensors is calculated for the optical camera, the infrared camera and the laser scanner respectively, and the transformation matrix formula is: Wherein, R is the rotation matrix, and X is the translation vector.

[0019] According to the transformation matrix, the optical camera image is transformed from the optical camera coordinate system to the world coordinate system, and then to the target coordinate system; the infrared camera image is transformed from the infrared camera coordinate system to the world coordinate system, and then to the target coordinate system; the laser scanner point cloud is transformed from the laser scanner coordinate system to the world coordinate system, and then to the target coordinate system;

[0020] The image data is spatially aligned and temporally aligned in the target coordinate system, and the spatial alignment includes spatial position and attitude; the image data is projected to the same target coordinate system through the coordinate transformation matrix;

[0021] The optical image, the infrared image and the point cloud data after coordinate transformation are fused in multiple scales to obtain fused and aligned image information.

[0022] In a further scheme, according to the fused image information, the light source control behavior is trained by using the Q-learning algorithm, and the image quality score is calculated and analyzed to automatically adjust the light source parameters to obtain optimized image information, including the following steps:

[0023] According to the material data of the detected object, including metal, plastic and glass, the collected environmental parameters and object material data are used as the characteristics of the state space of the Q-learning algorithm to construct a state space of the current environment and object state; the dimension and range of the state space are determined according to the accuracy of the sensor;

[0024] The adjustable range and step size of the light source parameters are defined to construct an action space; each action in the action space corresponds to an adjustment mode of the light source brightness, angle and stroboscopic;

[0025] According to the multi-scale fused image information, the image is decomposed into multiple scale subbands, and the structural similarity SSIM, the mean square error MSE and the peak signal-to-noise ratio PSNR are calculated at each scale respectively; the image quality score formula is calculated as:

[0026] Wherein, H is the environmental parameter influence coefficient; S is the number of decomposed scales; α s, β s and γ s are weight coefficients of each scale respectively, the sum of α s , β s and γ s is 1, and they are 0.55, 0.25 and 0.2 respectively;

[0027] According to the image quality score, a reward function is designed to ensure that the reward function reflects the influence of the light source control behavior on the image quality; an initialized Q table is created to store the Q values under different state and action combinations, and the initial value of the Q table is set to zero or a random value;

[0028] According to the current environment state, select the action with the maximum current Q value, otherwise randomly select an action; after executing the selected action, observe the new environment state and reward value, and update the Q value in the Q table according to the update formula of the Q-learning algorithm; use the Q-learning algorithm to train the light source control behavior, and obtain the optimized image quality score and optimized image information by continuously adjusting the light source parameters.

[0029] Further scheme, according to the optimized image information, using YOLOv8 model for defect recognition detection, including the following steps:

[0030] Collect the optimized historical image information training set, which contains multiple defect types and image quality scores; label the image information set, including: defect category and position information; the defect type includes: crack, scratch, bubble, depression and hole;

[0031] Select YOLOv8 model as the defect detection model, which is used to identify and locate the defect type and area and output the bounding box; input the labeled historical image information training set into the YOLOv8 model for training; input the real-time collected and preprocessed image information into the trained YOLOv8 model for defect detection and output including: defect category, position bounding box;

[0032] Introduce online learning technology, continuously collect new defect data including new defect types and samples, label the newly collected defect data, supplement the image information training set; deploy the incremental learning framework to continuously update the defect detection model using new defect data.

[0033] Further scheme, according to the optimized image information, three-dimensional deformation detection is carried out, including the following steps:

[0034] In the initial state or non-deformation state, the optical image, infrared thermal image and point cloud data are acquired and stored, and set as reference image data; a three-dimensional reference model including geometric shape, surface texture and thermal distribution is generated through multi-scale fusion; the optimized image information of the current state including optical image, infrared thermal image and point cloud data is collected at fixed time intervals; wherein the reference image data and the image information of the current state are time stamped and aligned;

[0035] Geometric deformation detection is performed based on the point cloud data, the current point cloud and the reference point cloud are registered using the ICP algorithm, the surface strain distribution is calculated through finite element analysis or local mesh deformation model; thermal radiation anomaly detection is performed based on the infrared image, the current infrared image and the reference infrared image are aligned, and the temperature field abnormal area is detected; the thermal expansion coefficient of temperature change and geometric deformation is analyzed based on the thermal elastic theory; surface texture change detection is performed based on the optical image, the feature matching algorithm is used to compare the surface texture of the current and reference optical images; the micro displacement field is calculated by the optical flow method or digital image correlation method; the deformation degree value is obtained by weighted sum of the deformation detection results of the above three modalities;

[0036] By verifying whether the geometric deformation area coincides with the thermal anomaly and texture change area in space, interference factors are excluded; when the deformation degree value exceeds the threshold value, it is determined that the object has deformed.

[0037] Further scheme, according to the defect and deformation detection result, the quality anomaly coefficient is calculated, including the following steps:

[0038] The defect type and position recognized in real time by the YOLOv8 model output the boundary box of the defect area; the image area positioned to the defect position is input into the instance segmentation algorithm in the semantic segmentation algorithm; the instance segmentation algorithm analyzes and classifies each pixel in the area to determine whether it belongs to the defect part, and generates a pixel-level defect mask; in the defect mask, the pixel value of the defect area is set to 255, and the pixel value of the non-defect area is set to 0; the defect area is separated from the background of the boundary box;

[0039] After obtaining the defect mask, the number of pixels in the defect area is obtained by counting the pixel points with a pixel value of 255 in the mask; the pixel quantity is converted into actual area units and multiplied by the pixel quantity of the defect area to obtain the defect area according to the pixel size information of the image; after obtaining the defect mask, the contour of the defect area is extracted by using the edge detection algorithm, and the defect perimeter is obtained by calculating the sum of the distances between adjacent pixels on the contour edge;

[0040] The formula for calculating the quality anomaly coefficient is:

[0041] A quality anomaly coefficient Y is obtained; wherein, A q is the area of the i-th defect; C i is the perimeter of the i-th defect; q is the number of defects; B is the deformation degree value; Q represents the image quality score; H is the environmental parameter influence coefficient.

[0042] Further, the control feedback module comprises: setting a threshold value of the quality anomaly coefficient, returning to step S1 to automatically adjust the light source parameters and reacquire the image when the quality anomaly coefficient exceeds the set threshold value; projecting the defect category label and defect position of the detected object in the three-dimensional model for visual display; combining the segmentation mask and point cloud data to generate a three-dimensional model of the detected object using a 3D reconstruction algorithm, including the geometric shape, surface texture and thermal distribution information of the detected object; and generating a deformation thermal map or a three-dimensional deformation model using a color mapping method according to the deformation degree value.

[0043] Compared with the prior art, the beneficial effects of the present application are:

[0044] The present application detects the environmental parameters of the object and automatically adjusts the light source parameters using a control algorithm to ensure that high-quality image data can still be obtained in complex environments; the adaptive light source control mechanism significantly improves the stability and accuracy of image acquisition.

[0045] The present application uses a Q-learning algorithm to train the light source control behavior, dynamically optimizes the light source parameters in combination with the image quality score, and further improves the usability of the image. At the same time, the YOLOv8 model is used for defect recognition and detection, which can quickly and accurately locate and identify the defects on the surface of the object, greatly improving the detection efficiency and accuracy.

[0046] The present application performs three-dimensional deformation detection on the optimized image information, can accurately evaluate the deformation of the object, and obtains a quality anomaly coefficient in combination with the defect detection result.

[0047] The present application adjusts the detection process in real time according to the quality anomaly coefficient, and visualizes the detection result, which is convenient for the operator to quickly understand the defect and deformation state of the object. This real-time feedback mechanism significantly improves the response speed and operation convenience of the system, and reduces the need for manual intervention. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0049] Figure 1 System module diagram of the present application. DETAILED DESCRIPTION

[0050] The technical solutions of the present application will be described clearly and completely below in conjunction with embodiments. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] Please refer to Figure 1 The first aspect embodiment of the present application provides an intelligent quality detection system based on machine vision, comprising the following modules:

[0052] Data image acquisition module: real-time acquisition of environmental parameters of the detected object by combining various types of sensors, and automatic adjustment of light source parameters by calculating environmental parameter influence coefficients and using control algorithms; according to the automatic adjustment result, real-time acquisition of image data of the detected object by combining optical cameras, infrared cameras and laser scanners;

[0053] Data image preprocessing module: data cleaning and normalization preprocessing of environmental parameters; image data filtering and denoising and image enhancement preprocessing; multi-scale fusion alignment of image data through coordinate transformation matrix;

[0054] Quality detection analysis module: according to the fused image information, training light source control behavior by using Q-learning algorithm, combining with the calculated image quality score, automatically adjusting the light source parameters to obtain the optimized image information; defect recognition detection according to the optimized image information using YOLOv8 model; three-dimensional deformation detection according to the optimized image information; calculating the quality abnormality coefficient according to the defect and deformation detection results;

[0055] Control feedback module: real-time control feedback according to the quality abnormality coefficient and visual detection of the defect and deformation state of the object.

[0056] Specifically, the infrared thermal imager, temperature and humidity sensor, light intensity sensor, etc. are integrated to collect the environmental temperature, humidity, light conditions and other parameters of the detected object in real time. The environmental parameter influence coefficient is calculated to dynamically adjust the light source parameters. Combined with the optical camera, infrared camera and laser scanner, the image data of the detected object is collected in real time. The frame synchronization mechanism is adopted to ensure the consistency of the image data output by multiple sensors in time. The collected environmental parameters are subjected to data cleaning and normalization processing to eliminate noise and outliers. The image is denoised using a filtering algorithm, and image enhancement techniques including contrast adjustment and sharpening are applied to improve image quality. The images collected by different sensors are fused and aligned through a coordinate transformation matrix to ensure the uniformity of the image data. The Q-learning algorithm is used to train the light source control behavior, and the light source parameters are dynamically adjusted according to the image quality score to obtain optimized images. The YOLOv8 model is used to detect defects in the optimized images to ensure the accuracy and real-time performance of the detection. Based on the optimized image information, three-dimensional deformation detection is performed. According to the defect and deformation detection results, the quality anomaly coefficient is calculated to provide a basis for subsequent control feedback. According to the quality anomaly coefficient, the threshold of the quality anomaly coefficient is set, and the control action is triggered when the threshold is exceeded to further visualize the defect and deformation state of the detected object.

[0057] In one embodiment, a variety of sensors are combined to collect environmental parameters of the detected object in real time, and the light source parameters are automatically adjusted using a control algorithm by calculating the environmental parameter influence coefficient, including the following steps:

[0058] Light sensors, temperature sensors, humidity sensors, laser dust sensors and acceleration sensors are used to collect environmental parameters of the detected object in real time, including environmental light intensity, environmental temperature, environmental humidity, environmental dust concentration and object movement speed.

[0059] According to the environmental parameters of the detected object, the light source parameters including brightness, angle and stroboscopic are automatically adjusted by calculating the environmental parameter influence coefficient and using a control algorithm.

[0060] The formula for calculating the environmental parameter influence coefficient is:

[0061] Where G(t) is the environmental light intensity at time t; T(t) is the environmental temperature at time t; S(t) is the environmental humidity at time t; F(t) is the environmental dust concentration at time t; V(t) is the object movement speed at time t; t is the time variable, representing the time length from the initial time 0 to the current collection of environmental parameter data;

[0062] The target brightness is L 目标 = L base(1+H), wherein, L base The target brightness is obtained by subtracting the current brightness from the reference light source brightness; the light source brightness deviation is obtained by subtracting the current brightness from the target brightness; the light source brightness deviation is input into the PID control algorithm, and the current new light source brightness is automatically adjusted to be the current brightness plus the sum of the output values of the PID control algorithm; similarly, the current new light source angle and the new stroboscopic can be obtained through the above automatic adjustment of the light source brightness.

[0063] Specifically, according to the environment parameters collected and preprocessed in real time, the environment parameter influence coefficient is calculated by substituting the environment parameters into the environment parameter influence coefficient formula. The target brightness is calculated according to the environment parameter influence coefficient and the preset brightness reference value. The target brightness is compared with the current light source brightness to obtain the brightness deviation. The deviation is input into the PID control algorithm to calculate the control amount for adjusting the brightness of the light source. Similarly, the target angle and the target stroboscopic are calculated according to the environment parameter influence coefficient and the angle and the stroboscopic reference value of the light source. The angle and the stroboscopic of the light source are adjusted through the corresponding control algorithm to adapt to different environmental conditions. After the adjustment of the light source parameters, the actual brightness, angle and stroboscopic of the light source are monitored in real time, and compared with the target parameters. If there is a deviation, the parameters of the control algorithm are adjusted in time or the environment parameter influence coefficient is recalculated to ensure the accuracy and stability of the light source parameters.

[0064] In one embodiment, the image data is aligned through multi-scale fusion by a coordinate transformation matrix, including the following steps:

[0065] According to the results of the automatically adjusted light source parameters, the image data is collected in real time, including: obtaining the appearance image of the detected object by using an optical camera, capturing the thermal radiation information of the detected object by using an infrared camera, and obtaining the point cloud data by emitting a light beam to measure the three-dimensional structure of the detected object by using a laser scanner;

[0066] The optical camera, the infrared camera and the laser scanner are calibrated by using a chessboard calibration method or a Zhang Zhengyou calibration method to calculate the intrinsic parameters and the extrinsic parameters; wherein, the intrinsic parameters include: focal length and principal point position, and the extrinsic parameters include: rotation matrix and translation vector;

[0067] The transformation matrix from the world coordinate system to the target coordinate system of the three sensors is calculated respectively, and the transformation matrix formula is: Wherein, R is the rotation matrix, and X is the translation vector;

[0068] According to the transformation matrix, the optical camera image is transformed from the optical camera coordinate system to the world coordinate system, and then to the target coordinate system; the infrared camera image is transformed from the infrared camera coordinate system to the world coordinate system, and then to the target coordinate system; the laser scanner point cloud is transformed from the laser scanner coordinate system to the world coordinate system, and then to the target coordinate system;

[0069] spatially aligning and temporally aligning the image data under a target coordinate system, the spatial alignment comprising: spatial position and posture; projecting the image data data to the same target coordinate system through a coordinate transformation matrix;

[0070] performing multi-scale fusion on the coordinate-transformed optical image, infrared image and point cloud data to obtain fused and aligned image information.

[0071] Specifically, according to the automatic adjustment of the light source parameter results, the data of the optical camera, the infrared camera and the laser scanner are collected in real time. Ensure that the collected image data is synchronized in time to reduce the complexity of subsequent time alignment. The collected image data is preprocessed, including denoising, filtering and other operations, to improve the quality and processing efficiency of the data. The optical camera, the infrared camera and the laser scanner are calibrated respectively by using the chessboard calibration method or the Zhang Zhengyou calibration method. The chessboard calibration method uses a chessboard pattern as a calibration board, and calculates the intrinsic and extrinsic parameters of the camera by shooting multiple images of the chessboard at different angles. The Zhang Zhengyou calibration algorithm calculates the intrinsic and extrinsic parameters of the camera by shooting multiple images of a planar calibration board. Place the chessboard calibration board in the field of view of the optical camera and shoot multiple images at different angles. Use the calibration functions in computer vision libraries such as OpenCV to calculate the intrinsic parameters of the optical camera, including focal length and principal point position, and extrinsic parameters, including rotation matrix and translation vector. Similarly, use the chessboard calibration board or the Zhang Zhengyou calibration method to calibrate the infrared camera to obtain its intrinsic and extrinsic parameters. By scanning an object with a known geometric shape, point cloud data is obtained and matched with the known geometric model to calculate the intrinsic and extrinsic parameters of the laser scanner. Select a suitable reference point as the origin of the world coordinate system, which can usually be a feature point of the detected object or a corner point of the calibration board. For the optical camera, the infrared camera and the laser scanner, calculate their transformation matrices from the world coordinate system to the target coordinate system. According to the extrinsic parameters of the optical camera, construct the transformation matrix from the optical camera coordinate system to the world coordinate system. Then, according to the relationship between the target coordinate system and the world coordinate system, calculate the transformation matrix from the optical camera coordinate system to the target coordinate system. Similarly, according to the extrinsic parameters of the infrared camera, calculate the transformation matrix from the infrared camera coordinate system to the target coordinate system. According to the extrinsic parameters of the laser scanner, calculate the transformation matrix from the laser scanner coordinate system to the target coordinate system. Use the transformation matrices calculated above to transform the optical camera image, infrared camera image and laser scanner point cloud from their respective coordinate systems to the target coordinate system. Convert the two-dimensional image pixel coordinates obtained by the optical camera to three-dimensional coordinates in the target coordinate system through the transformation matrix. Similarly, convert the thermal radiation image pixel coordinates obtained by the infrared camera to the target coordinate system. Convert the three-dimensional point cloud data obtained by the laser scanner to the target coordinate system through the transformation matrix. In the target coordinate system, spatially align the transformed image data. Spatial alignment includes alignment of spatial position and attitude to ensure consistency of data obtained by different sensors in space. Adjust the position of the transformed data to make the coordinates consistent in the target coordinate system. Rotate and translate the data of different sensors to keep them consistent in attitude. Due to the difference in collection time of different sensors, time alignment is needed. Interpolation and other methods can be used to adjust the data collected at different times to the same time point.The transformed and aligned optical image, infrared image and point cloud data are subjected to multi-scale decomposition. Methods such as Gaussian pyramid or Laplacian pyramid can be used to decompose the image data into different resolution levels. According to different application scenarios and requirements, appropriate fusion strategies are selected, including weighted average, maximum value selection, wavelet transform fusion, etc. Starting from the lowest resolution layer, gradually upward fusion is performed. At each layer, according to the selected fusion strategy, the optical image, infrared image and point cloud data are fused to obtain the fused image information. The fused multi-scale image information is reconstructed to obtain the final fused and aligned image containing optical, infrared and three-dimensional structure information, which can more comprehensively reflect the characteristics of the detected object.

[0072] In one embodiment, according to the fused image information, the light source control behavior is trained by using the Q-learning algorithm, and the image quality score calculated by analysis and calculation is combined to automatically adjust the light source parameters to obtain the optimized image information, including the following steps:

[0073] According to the material data of the detected object including metal, plastic and glass, the collected environmental parameters and object material data are used as the characteristics of the state space of the Q-learning algorithm to construct a state space of the current environment and object state. The dimension and range of the state space are determined according to the accuracy of the sensor;

[0074] The adjustable range and step size of the light source parameters are defined to construct the action space. Each action in the action space corresponds to an adjustment method for the light source brightness, angle and stroboscopic;

[0075] According to the multi-scale fused image information, the image is decomposed into multiple scale subbands, and the structural similarity SSIM, mean square error MSE and peak signal-to-noise ratio PSNR are calculated at each scale. The image quality score formula is calculated as:

[0076] Where H is the environmental parameter influence coefficient; S is the number of scales decomposed; a s , b s and g s are the weight coefficients of each scale, a s , b s and g s sum up to 1, and are 0.55, 0.25 and 0.2, respectively;

[0077] According to the image quality score, a reward function is designed to ensure that the reward function reflects the influence of the light source control behavior on the image quality. An initialized Q table is created to store the Q values under different state and action combinations, and the initial value of the Q table is set to zero or a random value;

[0078] According to the current environment state, the maximum Q value of the current action is selected, otherwise a random action is selected; after executing the selected action, the Q value in the Q table is updated according to the update formula of the Q-learning algorithm through observation of the new environment state and the reward value; the light source control behavior is trained using the Q-learning algorithm, and the optimized image quality score and the optimized image information are obtained by continuously adjusting the light source parameters.

[0079] Specifically, the environmental parameters of the detected object are collected in real time through light sensors, temperature sensors, humidity sensors, laser dust sensors, and acceleration sensors, including: environmental light intensity, environmental temperature, environmental humidity, environmental dust concentration, and object movement speed. According to the material of the detected object, including: metal, plastic, glass, etc., the corresponding material data can be obtained, which can be obtained in real time through a spectrum sensor. The collected environmental parameters are normalized and converted to the same numerical range for subsequent state space construction. According to the difference of the material of the object, the corresponding features are extracted. For example, for metal material, its reflectivity, thermal conductivity, etc. can be extracted; for plastic material, its transparency, hardness, etc. can be extracted; for glass material, its refractive index, light transmittance, etc. can be extracted. According to the accuracy of the sensor and the diversity of the material features of the object, the dimension of the state space is determined. The environmental light intensity can be divided into 10 levels, the environmental temperature can be divided into 5 levels, the environmental humidity can be divided into 5 levels, the environmental dust concentration can be divided into 5 levels, the object movement speed can be divided into 5 levels, and the object material features can be divided into 3 levels including: metal, plastic, and glass. According to the measurement range of the sensor and the value range of the material features of the object, the range of the state space is determined. The range of the environmental light intensity can be 0 to 1000lx, the range of the environmental temperature can be -20℃ to 50℃, the range of the environmental humidity can be 0% to 100%, the range of the environmental dust concentration can be 0 to 1000μg / m 3The range of the object motion speed can be 0 to 10 m / s. The adjustable range and step size of the light source parameters are defined to construct the action space; each action in the action space corresponds to an adjustment mode of the light source brightness, angle and strobe; the adjustable range of the light source brightness is defined as 0 to 100% and the step size is 5%; the adjustable range of the light source angle is defined as 0° to 360° and the step size is 10°; the adjustable range of the light source strobe is defined as 0 to 100 Hz and the step size is 5 Hz. According to the adjustable range and step size of the light source parameters, the dimension of the action space is determined. The multi-scale fused image information is decomposed into multiple scale subbands, which can be decomposed into different resolution levels by methods such as Gaussian pyramid or Laplacian pyramid. At each scale, the structural similarity SSIM of the optical image, the infrared image and the point cloud data is calculated, which reflects the structural similarity between the images, and the larger the value, the higher the structural similarity. At each scale, the mean square error MSE of the optical image, the infrared image and the point cloud data is calculated, which reflects the error degree between the images, and the smaller the value, the smaller the error. At each scale, the peak signal-to-noise ratio PSNR of the optical image, the infrared image and the point cloud data is calculated, which reflects the signal-to-noise ratio of the image, and the larger the value, the higher the signal-to-noise ratio. According to historical experience or experimental data, the weight coefficients of each scale are determined, and the number of scales S can be 3, and the weight coefficients of each scale can be 0.55, 0.25 and 0.2 respectively. According to the image quality score formula, the image quality score is calculated. The reward function should be able to accurately reflect the influence of the light source control behavior on the image quality score. When the light source control behavior leads to an increase in the image quality score, a positive reward is given; when the light source control behavior leads to a decrease in the image quality score, a negative reward is given. The reward function can be designed to be proportional to the difference between the image quality score. The Q table is a two-dimensional table, with rows representing the state space and columns representing the action space. The initial value of the Q table can be set to zero or a random value. Use array or hash table data structure to store Q table for fast lookup and update. According to the current environment state, select the action with the largest current Q value, otherwise randomly select an action. After executing the selected action, observe the new environment state and reward value. Update the Q value in the Q table according to the update formula of the Q-learning algorithm. In each training iteration, select an action according to the current environment state, observe the new state and reward value after executing the action, and update the Q table. Repeat this process until the Q table converges or reaches the preset number of training times. According to the Q value stored in the Q table, select the optimal action, i.e. the optimal light source parameter adjustment mode, and update the light source parameters; by continuously adjusting the light source parameters, the optimized image quality score and the optimized image information are obtained.

[0080] In one embodiment, defect recognition and detection are performed on the optimized image information using a YOLOv8 model, including the following steps:

[0081] Collect an optimized historical image information training set containing various defect types and image quality scores; label the image information set, including defect categories and location information; the defect types include cracks, scratches, bubbles, depressions, and holes;

[0082] Select the YOLOv8 model as the defect detection model for identifying and locating defect types and regions and outputting bounding boxes; input the labeled historical image information training set into the YOLOv8 model for training; input the real-time collected and preprocessed image information into the trained YOLOv8 model for defect detection and output including defect categories and location bounding boxes;

[0083] Introduce online learning technology, continuously collect new defect data including newly emerging defect types and samples, label the newly collected defect data, and supplement the image information training set; deploy an incremental learning framework to continuously update the defect detection model using new defect data.

[0084] Specifically, collect an optimized historical image information training set, which should contain various defect types, including cracks, scratches, bubbles, depressions, and holes, etc., and record the image quality score of each image. Label the image information set, including defect categories and location information. Location information is usually represented in the form of a bounding box, containing the coordinates of the defect in the image. To increase the diversity of the data set and improve the generalization ability of the model, data augmentation techniques such as image rotation, flipping, scaling, cropping, and adding noise are used. Select the YOLOv8 model as the defect detection model, which performs well in target detection tasks and can quickly and accurately identify and locate defects. Input the labeled historical image information training set into the YOLOv8 model for training. During training, adjust the model's parameters, such as learning rate, batch size, etc., to optimize the model's performance. During training, regularly evaluate the model's performance, such as calculating accuracy, recall rate, and F1 score, to ensure that the model can accurately identify and locate defects. Introduce online learning technology, continuously collect new defect data including newly emerging defect types and samples, label the newly collected defect data, and supplement the image information training set. Use new defect data to continuously update the defect detection model. During incremental learning, appropriate strategies such as regular updates, batch updates, etc. are used to ensure that the model can adapt to new defect types and samples while avoiding catastrophic forgetting. Real-time acquisition and preprocessing of new image information, input of preprocessed new image information into the trained YOLOv8 model, defect detection, and output of detected defect categories and location bounding boxes.

[0085] In one embodiment, three-dimensional deformation detection is performed based on the optimized image information, including the following steps:

[0086] In the initial state or non-deformation state, the optical image, infrared thermal image and point cloud data are acquired and stored, and set as reference image data; a three-dimensional reference model including geometric shape, surface texture and thermal distribution is generated through multi-scale fusion; the optimized image information of the current state including optical image, infrared thermal image and point cloud data is collected at fixed time intervals; wherein the reference image data and the image information of the current state are time-stamped aligned;

[0087] Geometric deformation detection is performed based on point cloud data, the current point cloud and the reference point cloud are registered using the ICP algorithm, the surface strain distribution is calculated through finite element analysis or local mesh deformation model; thermal radiation anomaly detection is performed based on infrared image, the current infrared image and the reference infrared image are aligned, and the temperature field abnormal area is detected; the thermal expansion coefficient of temperature change and geometric deformation is analyzed based on the thermal elastic theory; surface texture change detection is performed based on optical image, the surface texture of the current and reference optical images is compared using a feature matching algorithm; the micro-displacement field is calculated through the optical flow method or digital image correlation method; the deformation degree value is obtained by weighted sum of the deformation detection results of the above three modalities;

[0088] By verifying whether the geometric deformation area coincides with the thermal anomaly and texture change area in space, interference factors are excluded; by setting a deformation threshold value, when the deformation degree value exceeds the threshold value, it is determined that the object has deformed.

[0089] Specifically, in the initial state or non-deformation state, the optical image, infrared thermal image and point cloud data are acquired and stored, and set as the reference image data. The optimized image information of the current state is collected in real time, including: optical image, infrared thermal image and point cloud data. The reference image data and the image information of the current state are timestamped and aligned to ensure the consistency of the data in time. Through multi-scale fusion technology, the optical image, infrared thermal image and point cloud data are fused to generate a three-dimensional reference model containing geometric shape, surface texture and thermal distribution. The current point cloud is registered with the reference point cloud using the ICP algorithm to ensure the consistency of the point cloud data in space. Through finite element analysis or local mesh deformation model, the surface strain distribution is calculated. Finite element analysis can be used to simulate the stress and strain distribution of materials under force or deformation. Align the current infrared image with the reference infrared image to detect temperature field abnormal areas. Combined with the thermal elastic theory, analyze the thermal expansion coefficient of temperature change and geometric deformation, and evaluate the influence of thermal expansion on deformation. Use feature matching algorithm to compare the surface texture of the current and reference optical images to detect texture changes. Calculate the micro displacement field by optical flow method or digital image correlation method to analyze the subtle changes of surface texture. The results of geometric deformation detection, thermal radiation anomaly detection and surface texture change detection are weighted and summed to obtain the deformation degree value; the weights of the results of geometric deformation detection, thermal radiation anomaly detection and surface texture change detection are 0.55, 0.2 and 0.25 respectively, which can be adjusted by experiment and historical data. Verify whether the geometric deformation area coincides with the thermal anomaly and texture change area in space to exclude interference factors. By setting the deformation threshold value to 0.2, when the deformation degree value exceeds the threshold value, it is determined that the object has deformed. The setting of the deformation degree value threshold needs to be adjusted and determined according to the specific application scene, the characteristics of the monitored object and the statistical analysis of historical data.

[0090] In one embodiment, according to the defect and deformation detection results, the quality anomaly coefficient is calculated, including the following steps:

[0091] The defect type and position recognized in real time by the YOLOv8 model output the bounding box of the defect area; the image area positioned to the defect position is input into the instance segmentation algorithm in the semantic segmentation algorithm using the instance segmentation algorithm; the instance segmentation algorithm analyzes and classifies each pixel in the area to determine whether it belongs to the defect part, generating a pixel-level defect mask; in the defect mask, the pixel value of the defect area is set to 255, and the pixel value of the non-defect area is set to 0; the defect area is separated from the background of the bounding box;

[0092] After obtaining the defect mask, the pixel number of the defect area is obtained by counting the pixel points with a pixel value of 255 in the mask; the defect area is obtained by converting the pixel number into an actual area unit according to the pixel size information of the image and then multiplying the pixel number of the defect area; after obtaining the defect mask, the contour of the defect area is extracted by using an edge detection algorithm, and the defect perimeter is obtained by calculating the sum of the distances between adjacent pixels on the contour edge;

[0093] The formula for calculating the quality anomaly coefficient is:

[0094] The quality anomaly coefficient Y is obtained; wherein A q is the i th defect area; C i is the i th defect perimeter; q is the number of defects; B is the deformation degree value; Q represents the image quality score; and H is the environmental parameter influence coefficient.

[0095] Specifically, the defect type and position recognized in real time by the YOLOv8 model are used to output the bounding box of the defect area. The optimized image information is collected, including the optical image, the infrared thermal image, and the point cloud data. The image area positioned to the defect position is input into the instance segmentation algorithm Mask R-CNN. The instance segmentation algorithm analyzes and classifies each pixel in the area to generate a pixel-level defect mask. In the defect mask, the pixel value of the defect area is set to 255, and the pixel value of the non-defect area is set to 0, so as to separate the defect area from the background of the bounding box. The pixel points with a pixel value of 255 in the mask are counted to obtain the pixel number of the defect area, which is converted into an actual area unit according to the pixel size information of the image, and then multiplied by the pixel number of the defect area to obtain the defect area. The contour of the defect area is extracted by using an edge detection algorithm, and the defect perimeter is obtained by calculating the sum of the distances between adjacent pixels on the contour edge. The deformation degree value is calculated by a three-dimensional deformation detection method. The image quality score is calculated using the multi-scale fused image information. The environmental parameter influence coefficient is calculated according to the collected environmental parameters. The quality anomaly coefficient of the detected object is obtained by substituting the quality anomaly coefficient formula.

[0096] In one embodiment, the control feedback module includes setting a threshold value of the quality anomaly coefficient, returning to step S1 to automatically adjust the light source parameters and re-collect the image when the quality anomaly coefficient exceeds the set threshold value; projecting the defect category label and the defect position of the detected object on the three-dimensional model to visually display; using a 3D reconstruction algorithm to generate a three-dimensional model of the detected object, including the geometric shape, surface texture, and thermal distribution information of the detected object, by combining the segmentation mask and the point cloud data; and generating a deformation thermal map or a three-dimensional deformation model using a color mapping method according to the deformation degree value.

[0097] Specifically, a large amount of historical quality anomaly coefficient data is collected, the mean and standard deviation are calculated, and according to the 3-sigma principle, the threshold is set as the mean ± 3 times the standard deviation, which can cover most of the quality anomaly coefficient fluctuations under normal circumstances, effectively identifying abnormal situations. Through consulting relevant industry standards or consulting field experts, a reasonable quality anomaly coefficient threshold range is obtained, and combined with the actual situation, it is adjusted appropriately. By continuously collecting new quality anomaly coefficient data, regularly recalculating statistical indicators, and dynamically adjusting the threshold to adapt to changes in data distribution. Using 3D reconstruction algorithm, combined with segmentation mask and point cloud data, generate 3D model of the detected object, including: the geometric shape, surface texture and thermal distribution information of the detected object. Traditional computer vision-based 3D reconstruction methods such as stereo vision, structured light, etc. can be used, or deep learning-based 3D reconstruction algorithms such as 3D U-Net can be used; Project the defect class label and defect position of the detected object into the 3D model, which can be visualized by marking the defect position on the 3D model, labeling the defect class label, etc. Realize the visual display of defect information. According to the size of the deformation degree value, select the appropriate color mapping method, such as using gradient color to represent different deformation degrees, and color from light to dark or from cold to warm to represent the gradual increase of deformation degree. On the two-dimensional image, according to the deformation degree value, give each pixel point the corresponding color, generate the deformation heat map, and intuitively display the deformation distribution of the object surface. On the three-dimensional model, according to the deformation degree value, give each point or each face of the model surface the corresponding color, generate the three-dimensional deformation model, and more comprehensively and intuitively display the three-dimensional deformation of the object.

[0098] The above embodiments are only used to illustrate the technical method of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present application can be modified or replaced equivalently without departing from the spirit and scope of the technical method of the present application.

Claims

1. A machine vision based intelligent quality detection system, characterized in that, The application comprises the following modules: Data image acquisition module: real-time acquisition of environmental parameters of the detected object by combining various types of sensors, automatic adjustment of light source parameters by calculating environmental parameter influence coefficients and using control algorithms; real-time acquisition of image data of the detected object by combining optical cameras, infrared cameras and laser scanners according to the automatic adjustment results; Data image preprocessing module: data cleaning and normalization preprocessing of environmental parameters; image filtering, denoising and image enhancement preprocessing of image data; Through a coordinate transformation matrix, the image data is fused and aligned at multiple scales; Quality detection and analysis module: training of light source control behavior by using Q-learning algorithm according to the fused image information, automatic adjustment of light source parameters to obtain optimized image information according to the image quality score calculated by analysis; defect recognition and detection by using YOLOv8 model according to the optimized image information; three-dimensional deformation detection according to the optimized image information; According to the defect and deformation detection results, the quality abnormality coefficient is calculated; Control feedback module: real-time control feedback according to the quality abnormality coefficient and visual detection of the defect and deformation state of the object; Real-time acquisition of environmental parameters of the detected object by combining various types of sensors, automatic adjustment of light source parameters by calculating environmental parameter influence coefficients and using control algorithms, comprising the following steps: Real-time acquisition of environmental parameters of the detected object by using light sensors, temperature sensors, humidity sensors, laser dust sensors and acceleration sensors, including: environmental light intensity, environmental temperature, environmental humidity, environmental dust concentration and object movement speed; According to the environmental parameters of the detected object, the light source parameters are automatically adjusted by calculating the environmental parameter influence coefficient and using the control algorithm, including: brightness, angle and stroboscopic; The formula for calculating the environmental parameter influence coefficient is: wherein G is the ambient light intensity at the time instant t ; T is the ambient temperature at the time instant t ; S is the ambient humidity at the time instant t ; F is the ambient dust concentration at the time instant t ; V is the object motion speed at the time instant t ; t is the time variable, representing the time length from the initial time instant 0 to the current collection of the environmental parameter data; The target brightness is: wherein, The reference light source brightness is represented; the light source brightness deviation is obtained by subtracting the current brightness from the target brightness; the light source brightness deviation is input into the PID control algorithm, and the current new light source brightness is automatically adjusted to be the current brightness plus the sum of the output values of the PID control algorithm; similarly, the above automatic adjustment of the light source brightness step can obtain the current new light source angle and the new stroboscopic. According to the fused image information, the light source control behavior is trained by using Q-learning algorithm, the image quality score is calculated by analysis, and the light source parameters are automatically adjusted to obtain optimized image information, comprising the following steps: According to the material data of the detected object including: metal, plastic, glass; the collected environmental parameters and object material data are used as the characteristics of the state space of Q-learning algorithm to construct a state space of the current environment and object state; the dimension and range of the state space are determined according to the accuracy of the sensor; Define the adjustable range and step size of the light source parameters to construct the action space; each action in the action space corresponds to an adjustment method for light source brightness, angle and stroboscopic; According to the multi-scale fused image information, the image is decomposed into multiple scale subbands, and the structural similarity SSIM, mean square error MSE and peak signal-to-noise ratio PSNR are calculated at each scale; the image quality score formula is: Wherein, H is the environmental parameter influence coefficient; S is the scale number of decomposition; , and are the weight coefficients of each scale respectively, , and The sum of and is 1, which are 0.55, 0.25 and 0.2 respectively; According to the image quality score, design the reward function to ensure that the reward function reflects the influence of light source control behavior on image quality; create an initialized Q table to store the Q values under different state and action combinations, and the initial value of the Q table is set to zero or a random value; According to the current environment state, the maximum Q value of the current action is selected, otherwise a random action is selected; after executing the selected action, the Q value in the Q table is updated according to the update formula of the Q-learning algorithm through observation of the new environment state and the reward value; the light source control behavior is trained using the Q-learning algorithm, and the optimized image quality score and the optimized image information are obtained by continuously adjusting the light source parameters.

2. The machine vision based intelligent quality detection system as claimed in claim 1, wherein, The image data is aligned through multi-scale fusion by a coordinate transformation matrix, including the following steps: According to the automatic adjustment of the light source parameter result, real-time image data acquisition includes: using an optical camera to obtain the appearance image of the detected object, using an infrared camera to capture the thermal radiation information of the detected object, and using a laser scanner to emit a light beam to measure the point cloud data of the three-dimensional structure of the detected object; A checkerboard calibration method or Zhang Zhengyou calibration method is used to calibrate the optical camera, infrared camera and laser scanner respectively, and the internal and external parameters are calculated; the internal parameters include focal length and principal point position, and the external parameters include rotation matrix and translation vector; For optical camera, infrared camera and laser scanner, the transformation matrix of world coordinate system to target coordinate system of three sensors is calculated respectively, and the transformation matrix formula is: wherein R is a rotation matrix, and X is a translation vector; According to the transformation matrix, the optical camera image is transformed from the optical camera coordinate system to the world coordinate system, and then to the target coordinate system; the infrared camera image is transformed from the infrared camera coordinate system to the world coordinate system, and then to the target coordinate system; the laser scanner point cloud is transformed from the laser scanner coordinate system to the world coordinate system, and then to the target coordinate system; The image data is spatially aligned and temporally aligned in the target coordinate system, and the spatial alignment includes spatial position and attitude; the image data is projected to the same target coordinate system through the coordinate transformation matrix; The optical image, infrared image and point cloud data after coordinate transformation are multi-scale fused to obtain the fused and aligned image information. 3.The machine vision-based intelligent quality detection system of claim 1, wherein, According to the optimized image information, a YOLOv8 model is used for defect recognition detection, including the following steps: Collect the optimized historical image information training set, which contains multiple defect types and image quality scores; label the image information set, including defect category and position information; the defect types include cracks, scratches, bubbles, depressions and holes; Select the YOLOv8 model as the defect detection model, which is used to identify and locate the defect type and area and output the bounding box; input the labeled historical image information training set into the YOLOv8 model for training; input the real-time collected and preprocessed image information into the trained YOLOv8 model for defect detection and output including defect type, position and bounding box; Introduce online learning technology, continuously collect new defect data including new defect types and samples, label the newly collected defect data, supplement the image information training set; deploy an incremental learning framework to continuously update the defect detection model using new defect data.

4. The machine vision based intelligent quality detection system as claimed in claim 1, wherein, According to the optimized image information, three-dimensional deformation detection is performed, including the following steps: In the initial state or non-deformation state, the optical image, infrared thermal image and point cloud data are acquired and stored, and set as the reference image data; a three-dimensional reference model including geometric shape, surface texture and thermal distribution is generated through multi-scale fusion; the optimized image information of the current state including optical image, infrared thermal image and point cloud data is collected at fixed time intervals; wherein the reference image data and the image information of the current state are time stamped and aligned; Geometric deformation detection based on point cloud data, the current point cloud and the reference point cloud are registered using the ICP algorithm, the surface strain distribution is calculated through finite element analysis or local mesh deformation model; thermal radiation anomaly detection based on infrared image, the current infrared image and the reference infrared image are aligned, and the temperature field abnormal area is detected; combined with the thermal elastic theory, the thermal expansion coefficient of temperature change and geometric deformation is analyzed; surface texture change detection based on optical image, the surface texture of the current and reference optical image is compared using a feature matching algorithm; the micro displacement field is calculated by optical flow method or digital image correlation method; the deformation degree value is obtained by weighted sum of the deformation detection results of the above geometric deformation detection based on point cloud data, thermal radiation anomaly detection based on infrared image and surface texture change detection based on optical image; By verifying whether the geometric deformation area coincides with the thermal anomaly and texture change area in space, interference factors are excluded; by setting a deformation threshold, when the deformation degree value exceeds the threshold, it is determined that the object has deformed.

5. The machine vision based intelligent quality detection system as claimed in claim 1, wherein, According to the defect and deformation detection results, the quality anomaly coefficient is calculated, including the following steps: The defect type and position recognized in real time by the YOLOv8 model output the boundary box of the defect area; the image area positioned to the defect position is input into the instance segmentation algorithm in the semantic segmentation algorithm using the instance segmentation algorithm; the instance segmentation algorithm analyzes and classifies each pixel in the area to determine whether it belongs to the defect part, and generates a pixel-level defect mask; in the defect mask, the pixel value of the defect area is set to 255, and the pixel value of the non-defect area is set to 0; the defect area is separated from the background of the boundary box; After obtaining the defect mask, the number of pixels in the defect area is obtained by counting the pixel points with a pixel value of 255 in the mask; the pixel number is converted into an actual area unit and multiplied by the pixel number of the defect area to obtain the defect area according to the pixel size information of the image; after obtaining the defect mask, the contour of the defect area is extracted using an edge detection algorithm, and the sum of the distances between adjacent pixels on the contour edge is calculated to obtain the defect perimeter; The calculation of the quality anomaly coefficient formula is as follows: A quality abnormality coefficient Y is obtained; wherein, is the i-th defect area; is the i-th defect perimeter; q is the number of defects; B is a deformation degree value; Q represents an image quality score; and H is an environmental parameter influence coefficient.

6. The machine vision-based intelligent quality detection system of claim 1, wherein The control feedback module comprises: setting a threshold value of the quality anomaly coefficient, returning to step S1 when the quality anomaly coefficient exceeds the set threshold value, automatically adjusting the light source parameters and reacquiring the image; projecting the defect category label and the defect position of the detected object in the three-dimensional model to visually display; combining the segmentation mask and the point cloud data, using a 3D reconstruction algorithm to generate a three-dimensional model of the detected object, including the geometric shape, surface texture and thermal distribution information of the detected object; and generating a deformation thermal map or a three-dimensional deformation model using a color mapping method according to the deformation degree value.

Images