Notebook computer screen quality detection system and method based on machine vision

Abstract

The invention discloses a laptop screen quality detection system and method based on machine vision, and relates to the technical field of machine vision detection.The laptop screen quality detection system comprises an image acquisition module, a data processing module, a multi-modal fusion module, an intelligent algorithm optimization module, a result output module, a storage module and a parameter adjustment module; the method has the advantages that feature vectors of image data, ultraviolet data and depth data can be comprehensively processed by introducing an infrared camera, an ultraviolet camera and a depth camera through multi-modal fusion detection and introducing a fusion index function F (T, I, U and D) into a multi-modal fusion module, accurate fusion of the multi-modal data is realized through calculation of weight coefficients, and the accuracy of fusion of the multi-modal data is improved. The infrared camera can detect the heat distribution condition of the screen, so that potential quality problems, such as the influence of local heating caused by faults of electronic components on the display quality of the screen, are warned in advance, and comprehensive detection of the notebook computer screen from multiple angles and multiple physical characteristics is realized.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision detection, and particularly to a notebook screen quality detection system and method based on machine vision. Background Technique

[0002] With the wide application of notebook computers in modern society, the quality of their screens has become a key factor affecting user experience. A high-quality notebook screen can not only provide clear and vivid image display but also reduce the harm to users' eyes. Therefore, it is crucial to ensure the quality of notebook screens. In the past few decades, the manufacturing technology of notebook screens has been continuously developing, and performance indicators such as screen resolution, color reproduction, and brightness have been significantly improved. However, at the same time, screen quality detection technology also faces new challenges;

[0003] There are certain technical defects in the existing notebook screen quality detection. First of all, the existing technology mainly relies on a single detection modality, such as traditional optical cameras, which are difficult to comprehensively detect various potential problems of the screen, cannot effectively detect defects related to temperature, depth, etc., and are prone to missed detections, thus affecting the accuracy and reliability of detection. Secondly, the existing detection algorithms may be difficult to automatically adjust detection parameters and algorithm models according to different screen models, production processes, and environmental conditions, cannot adapt to these changes in a timely manner, resulting in unsatisfactory detection accuracy and stability, increasing maintenance costs and operation difficulties. Finally, in the existing technology, the exposure time, gain parameters of the camera, and the intensity and angle of the light source may not be well adapted to different lighting conditions, cannot be intelligently adjusted according to the reflection characteristics of the screen and the brightness distribution of the image, thus affecting the stability and accuracy of image quality and reducing the sensitivity of defect detection. For this reason, we propose a notebook screen quality detection system and method based on machine vision. Summary of the Invention

[0004] The purpose of the present invention is to provide a notebook screen quality detection system and method based on machine vision.

[0005] To solve the problems raised in the above background technique, the present invention provides the following technical solution: A notebook screen quality detection system based on machine vision, the detection system includes an image acquisition module, a data processing module, a multi-modal fusion module, an intelligent algorithm optimization module, a result output module, a storage module, and a parameter adjustment module;

[0006] The image acquisition module obtains the notebook screen image through a high-resolution industrial camera and a ring LED light source device, and transmits the image data to the data processing module in TIFF format. The multi-modal fusion module combines the multi-sensor data of an infrared thermal imaging camera, an ultraviolet detection camera, and a structured light depth camera, and performs fusion analysis with the data of the image acquisition module. The intelligent algorithm optimization module continuously adjusts the algorithm according to the data processing results, and adjusts the algorithm parameters according to the feedback of each detection result. After the data processing module processes the image and multi-modal data using an image segmentation algorithm based on deep learning, the results are transmitted to the result output module and the storage module. The parameter adjustment module adjusts the parameters of each module according to the instructions of the intelligent algorithm optimization module. When the system works, the image acquisition module first obtains the image data, and then the multi-modal fusion module introduces the multi-sensor data of the infrared, ultraviolet, and depth cameras. The data processing module processes these data, and the intelligent algorithm optimization module optimizes according to the processing results. The final result is output by the result output module and stored in the storage module.

[0007] As a further solution of the present invention: The image acquisition module includes an XoF16K industrial camera and an LDR2-150W-C ring LED light source device. The resolution of the industrial camera is 48 million pixels, and the frame rate is 50 frames per second. The ring LED light source device provides uniform and stable illumination, and the illumination intensity is between 2500 lx - 3200 lx. The image acquisition module adjusts the camera parameters and the light source intensity according to different screen materials and brightness data. During the acquisition process, the image sensor of the camera uses CMOS technology. After converting the optical signal into an electrical signal, it is converted into a digital signal by a high-speed analog-to-digital converter and transmitted to the data processing module in TIFF format.

[0008] As a further solution of the present invention: The multi-modal fusion module is provided with an infrared thermal imaging camera, an ultraviolet detection camera, and a structured light depth camera. The infrared thermal imaging camera is used to obtain the thermal distribution image of the screen, and the data format is 16-bit grayscale image format, which is transmitted to the data processing module through high-speed USB3.0. The ultraviolet detection camera detects the response data of the screen to ultraviolet rays, and is also transmitted in JPEG2000 format. The structured light depth camera obtains the three-dimensional information of the screen, and the data is transmitted in point cloud data format;

[0009] Let the temperature distribution function of the screen at a certain moment be T(x, y, z) (x, y, z represent spatial coordinates), and introduce the fusion index function F(T, I, U, D), where I represents the image data feature vector, U represents the ultraviolet data feature vector, and D represents the depth data feature vector. The specific function definition is as follows:

[0010]

[0011] Where: α, β, γ, and δ are weight coefficients, and their values are determined by deep learning and optimization of a large amount of historical data, |I i |, |U j |, and |D k | are respectively the elements in the feature vectors of image data, ultraviolet data, and depth data, and ‖I‖, ‖U‖, and ‖D‖ respectively represent the norms of the feature vectors of image data, ultraviolet data, and depth data.

[0012] As a further solution of the present invention: The data processing module uses an image segmentation algorithm based on deep learning to process the image data transmitted by the image acquisition module. First, the image is grayscale processed, and the specific grayscale processing formula is as follows:

[0013] Y = 0.299R + 0.587G + 0.114B

[0014] Where: Y represents the grayscale value, and R, G, and B are respectively the red, green, and blue channel values of the color image. Then, filtering processing is performed to remove noise. The median filtering function is used, and the parameter is set to a window size of 3x3. Then, edge detection is performed on the image, and the Canny edge detection algorithm is used to extract the screen features. When processing the data of the multi-modal fusion module, the Bayesian fusion algorithm is used. Let the image data be D1, the infrared data be D2, the ultraviolet data be D3, and the depth data be D4. The fused data D satisfies the following formula:

[0015] P(D|D1,D2,D3,D4) = P(D1|D) × P(D2|D) × P(D3|D) × P(D4|D)

[0016] × P(D) / P(D1) × P(D2) × P(D3) × P(D4)

[0017] After calculation, the data processing module integrates different types of data and transmits the data to the storage module and the result output module.

[0018] As a further solution of the present invention: The intelligent algorithm optimization module can receive the data of the storage module, adopt the reinforcement learning algorithm, and adjust the algorithm parameters according to the feedback of each detection result. The comprehensive evaluation index function E is introduced, and the specific comprehensive evaluation index function is as follows:

[0019] E = g(Acc,Rec,F1,Cov,Spe)

[0020] Where: Acc represents the accuracy rate, Rec represents the recall rate, F1 represents the harmonic balance coefficient, Cov represents the coverage rate, and Spe represents the specificity. The specific calculation formula of Acc is as follows:

[0021] Acc = TP / (TP + FP + FN)

[0022] Among them: TP represents the number of true positives (the number of actual positive samples predicted as positive samples), FP represents the number of false positives (the number of actual negative samples predicted as positive samples), FN represents the number of false negatives (the number of actual positive samples predicted as negative samples), and the specific calculation formulas for Rec, F1, Spe, and Cov are as follows:

[0023] Rec = TP / (TP + FN)

[0024]

[0025] Cov = (TP + FN) / Total

[0026] Spe = TN / (TN + FP)

[0027] Among them: Total represents the total number of samples, and TN represents the number of true negatives (the number of actual negative samples predicted as negative samples);

[0028] And a reward function is set as R = h(E), where the function h is a function that determines the reward value according to the comprehensive evaluation index function E, comprehensively considering the importance and mutual relationship of each evaluation index to reflect the performance of the system. During the optimization process, the intelligent algorithm optimization module continuously tries different parameter combinations and determines the optimal parameters by evaluating the value of the reward function.

[0029] As a further solution of the present invention: the result output module outputs the detection result in a visual form, including the screen quality level, specific defect types, and location information. The output methods include display on a display screen, printing a report, and transmitting it to other devices through a network. The result output module receives data from the data processing module and the intelligent algorithm optimization module. After format conversion and collation, it displays the screen quality level in a color-coded manner, with red indicating serious defects, yellow indicating minor defects, and green indicating qualified. For specific defect types, a combination of text description and icon identification is used to clearly show the nature of the defects. At the same time, a three-dimensional model of the screen is displayed on the display screen, marking the location of the defects, facilitating users to intuitively understand the screen quality situation. Subsequently, a print report of the detection result is generated, providing detailed detection data and analysis results.

[0030] As a further solution of the present invention: The storage module is used to store the image data, multi-modal data, detection results, and intermediate data during the algorithm optimization process. The solid-state drive array storage technology is adopted to ensure the security and traceability of the data. The storage module can store the data of the image acquisition module, data processing module, intelligent algorithm optimization module, and result output module. During the storage process, the data is classified and compressed. The image data is classified and stored according to the screen model and production date information to improve the storage efficiency and retrieval speed. The lossless compression algorithm is used to compress the data to reduce the storage space occupancy.

[0031] As a further solution of the present invention: The parameter adjustment module adjusts the parameters of the image acquisition module, data processing module, and multi-modal fusion module according to the instructions of the intelligent algorithm optimization module. The parameter adjustment module automatically adjusts the parameters of each module through a parameter adjustment algorithm based on fuzzy logic according to the current detection requirements and system performance, including adjusting the exposure time and gain parameters of the camera to adapt to different lighting conditions. When the ambient light intensity changes, the parameter adjustment module adjusts the exposure time and gain of the camera according to the pre-set fuzzy rules. When the light intensity is strong, the exposure time is reduced and the gain is decreased. When the light intensity is weak, the exposure time is increased and the gain is increased. The intensity and angle of the light source are adjusted to improve the image quality. According to the reflection characteristics of the screen and the brightness distribution of the image, the angle and intensity of the light source are adjusted to make the defects in the image more obvious. For the multi-modal fusion module, the parameter settings of each sensor in the multi-modal fusion module are adjusted to optimize the data fusion effect.

[0032] In addition, the present invention also provides a method for detecting the quality of a notebook screen based on machine vision. The detection method includes the following steps:

[0033] Step 1: Use a high-resolution industrial camera and a ring-shaped LED light source device to obtain a notebook screen image. Adjust the camera parameters and light source intensity according to different screen materials and brightness. The image sensor uses CMOS technology. After converting the optical signal into an electrical signal, it is converted into a digital signal by a high-speed analog-to-digital converter and transmitted to the data processing module in TIFF format.

[0034] Step 2: The multimodal fusion module is equipped with an infrared thermal imaging camera, an ultraviolet detection camera, and a structured light depth camera. The infrared thermal imaging camera acquires the thermal distribution image of the screen (in the format of a 16-bit grayscale image), the ultraviolet detection camera detects the response data of the screen to ultraviolet rays (in the JPEG2000 format), and the structured light depth camera acquires the three-dimensional information of the screen (in the format of point cloud data). A fusion metric function F(T, I, U, D) is introduced to fuse the multimodal data, including the processing of the feature vectors of the image data, ultraviolet data, and depth data, and the calculation through weight coefficients. The data processing module processes the image data using a deep learning-based image segmentation algorithm, including grayscale processing, filtering processing, and edge detection. When processing the data of the multimodal fusion module, a Bayesian fusion algorithm is used to integrate different types of data;

[0035] Step 3: The intelligent algorithm optimization module uses a reinforcement learning algorithm to adjust the algorithm parameters according to the detection result feedback, and introduces a comprehensive evaluation metric function E (E = g(Acc, Rec, F1, Cov, Spe)), and sets a reward function R = h(E). The importance and mutual relationship of each evaluation metric are comprehensively considered to reflect the performance of the system. Different parameter combinations are continuously tried, and the optimal parameters are determined by evaluating the value of the reward function. The parameter adjustment module adjusts the parameters of the image acquisition module, data processing module, and multimodal fusion module according to the instructions of the intelligent algorithm optimization module, including the camera exposure time, gain, light source intensity, and angle;

[0036] Step 4: The result output module outputs the detection results in a visual form, including the screen quality level, specific defect types, and location information. The output methods include display on the display screen, printing a report, and transmitting through the network to other devices. The storage module stores various data during the detection process and intermediate data during the algorithm optimization process, and uses a solid-state drive array storage technology to classify and compress the data.

[0037] With the above technical solutions, compared with the prior art, the beneficial effects of the present invention are as follows:

[0038] 1. The present invention performs multi-modal fusion detection by introducing an infrared camera, an ultraviolet camera, and a depth camera. The fusion metric function F(T, I, U, D) introduced in the multi-modal fusion module can comprehensively process the feature vectors of image data, ultraviolet data, and depth data. Through the calculation of weight coefficients, precise fusion of multi-modal data is achieved. The infrared camera can detect the thermal distribution of the screen and can early warn of potential quality problems caused by overheating or local temperature anomalies, such as problems where local heating caused by electronic component failures affects the screen display quality. The ultraviolet camera can detect the response of the screen material to ultraviolet rays and evaluate the quality of the anti-ultraviolet coating on the screen. The depth camera can obtain the three-dimensional information of the screen, accurately measure the flatness and thickness uniformity of the screen, and effectively detect problems such as local protrusions or depressions that are crucial for the performance and user experience of the touch screen. This multi-modal fusion detection method greatly improves the accuracy and reliability of detection, reduces the missed detection rate, and can comprehensively detect the notebook screen from multiple angles and various physical characteristics;

[0039] 2. The present invention greatly improves the adaptability and flexibility of the system by automatically adjusting the detection parameters and algorithm models according to different screen models, production processes, and environmental conditions. In the initial stage of operation, a basic detection model is established by learning a certain number of screen samples with known quality. Subsequently, new data is continuously collected during the detection process, and the differences from the existing model are analyzed in real time. When it is found that there are large deviations between the new data and the existing model, the system automatically triggers the algorithm optimization module for adjustment and update. By introducing the comprehensive evaluation metric function E and the reward function R, the system can comprehensively consider the importance and mutual relationships of multiple evaluation metrics, accurately reflect the performance of the system, and by evaluating the value of the reward function, the system can continuously try different parameter combinations to determine the optimal parameters and achieve an improvement in the detection ability for specific types of defects. This adaptive intelligent algorithm optimization greatly reduces the maintenance cost and operation difficulty, ensures that the system always maintains high detection accuracy and stability, and becomes more intelligent and efficient over time;

[0040] 3. The present invention can adapt to different lighting conditions by adjusting the exposure time and gain parameters of the camera. When the light intensity is strong, the exposure time is reduced and the gain is lowered, and when the light intensity is weak, the exposure time is increased and the gain is raised, thereby ensuring the stability and accuracy of the image quality. In addition, by adjusting the intensity and angle of the light source, according to the reflection characteristics of the screen and the brightness distribution of the image, the defects in the image can be made more obvious, improving the sensitivity of defect detection. For the multi-modal fusion module, by adjusting the parameter settings of each sensor, the data fusion effect can be optimized, further enhancing the system's ability to comprehensively judge the screen quality. This intelligent parameter adjustment function enhances the stability and reliability of the system, improves the detection efficiency and accuracy, and enables the system to better adapt to various complex detection environments and requirements. Brief Description of the Drawings

[0041] Figure 1 This is a schematic diagram of the system process in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the method steps in an embodiment of the present invention. Detailed Embodiments

[0043] The following further describes the detailed embodiments of the present invention in conjunction with the accompanying drawings. It should be noted here that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation to the present invention.

[0044] In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0045] Embodiment 1:

[0046] Please refer to the attached Figure 1 - attached Figure 2 , a notebook screen quality detection system based on machine vision of the present invention, the detection system includes an image acquisition module, a data processing module, a multi-modal fusion module, an intelligent algorithm optimization module, a result output module, a storage module, and a parameter adjustment module;

[0047] The image acquisition module obtains the notebook screen image through a high-resolution industrial camera and a ring-shaped LED light source device, and transmits the image data to the data processing module in TIFF format. The multi-modal fusion module combines the multi-sensor data of an infrared thermal imaging camera, an ultraviolet detection camera, and a structured light depth camera, and performs fusion analysis with the data of the image acquisition module. The intelligent algorithm optimization module continuously adjusts the algorithm according to the data processing result, and adjusts the algorithm parameters according to the feedback of each detection result. After the data processing module processes the image and multi-modal data using an image segmentation algorithm based on deep learning, the result is transmitted to the result output module and the storage module. The parameter adjustment module adjusts the parameters of each module according to the instructions of the intelligent algorithm optimization module. When the system works, the image acquisition module first obtains the image data, then the multi-modal fusion module introduces the multi-sensor data of the infrared, ultraviolet, and depth cameras, the data processing module processes these data, the intelligent algorithm optimization module optimizes according to the processing result, and the final result is output by the result output module and stored in the storage module;

[0048] Specific workflow: The intelligent algorithm optimization module receives data from the storage module and optimizes the algorithm based on this. Then, the reinforcement learning algorithm is adopted to continuously adjust the algorithm parameters according to the feedback of each detection result to improve the performance of the algorithm. At the same time, a comprehensive evaluation index function E = g(Acc, Rec, F1, Cov, Spe) is introduced. This function can comprehensively evaluate the performance of the system and consider the performance of the system from multiple dimensions. Then, through the reward function R = h(E), considering the importance and mutual relationship of each evaluation index, the reward value is accurately determined, thus providing a clear direction for parameter adjustment. During the optimization process, different parameter combinations are continuously tried, and the value of the reward function is evaluated to find the optimal parameter combination. In addition, historical data is used to update and improve the model, enabling the system to better adapt to different screen types and production environments. Finally, according to the optimization results, the parameter adjustment module is instructed to adjust the parameters of each module, thereby improving the detection performance of the entire system and ensuring that the system can accurately and reliably detect the quality of notebook screens;

[0049] Furthermore, through multi-modal fusion detection, an infrared camera, an ultraviolet camera, and a depth camera are introduced. The fusion index function F(T, I, U, D) introduced in the multi-modal fusion module can comprehensively process the feature vectors of image data, ultraviolet data, and depth data. Through the calculation of weight coefficients, accurate fusion of multi-modal data is achieved. The infrared camera can detect the thermal distribution of the screen and can give early warnings of potential quality problems caused by overheating or local temperature anomalies, such as local heating caused by electronic component failures affecting the screen display quality and other problems. The ultraviolet camera can detect the response of the screen material to ultraviolet rays and evaluate the quality of the anti-ultraviolet coating on the screen. The depth camera can obtain the three-dimensional information of the screen, accurately measure the flatness and thickness uniformity of the screen, and effectively detect problems such as local protrusions or depressions that are crucial for the performance and user experience of touch screens. This multi-modal fusion detection method greatly improves the accuracy and reliability of detection, reduces the missed detection rate, and can comprehensively detect notebook screens from multiple angles and various physical characteristics.

[0050] Embodiment 2:

[0051] On the basis of Embodiment 1, as shown in the accompanying drawings of the specification Figures 1 - 2As shown in the figure, the image acquisition module includes an XoF16K industrial camera and an LDR2-150W-C ring-shaped LED light source device. The resolution of the industrial camera is 48 million pixels, and the frame rate is 50 frames per second. The ring-shaped LED light source device provides uniform and stable illumination, and the illumination intensity is between 2500 lx and 3200 lx. The image acquisition module adjusts the camera parameters and light source intensity according to different screen materials and brightness data. During the acquisition process, the image sensor of the camera uses CMOS technology. After converting the optical signal into an electrical signal, it is converted into a digital signal by a high-speed analog-to-digital converter and transmitted to the data processing module in the form of TIFF format data. The multi-modal fusion module is equipped with an infrared thermal imaging camera, an ultraviolet detection camera, and a structured light depth camera. The infrared thermal imaging camera is used to obtain the thermal distribution image of the screen, and the data format is a 16-bit grayscale image format, which is transmitted to the data processing module through high-speed USB3.0. The ultraviolet detection camera detects the response data of the screen to ultraviolet rays and is also transmitted in the JPEG2000 format. The structured light depth camera obtains the three-dimensional information of the screen, and the data is transmitted in the form of point cloud data format;

[0052] Let the temperature distribution function of the screen at a certain moment be T(x, y, z) (x, y, z represent spatial coordinates), and introduce the fusion index function F(T, I, U, D), where I represents the image data feature vector, U represents the ultraviolet data feature vector, and D represents the depth data feature vector. The specific function definition is as follows:

[0053]

[0054] Among them: α, β, γ, δ are weight coefficients, and their values are determined by deep learning and optimization of a large amount of historical data. |I i |, |U j |, and |D k | are the respective elements in the image data, ultraviolet data, and depth data feature vectors, and ‖I‖, ‖U‖, and ‖D‖ respectively represent the norms of the image data, ultraviolet data, and depth data feature vectors. The data processing module uses an image segmentation algorithm based on deep learning to process the image data transmitted by the image acquisition module. First, the image is grayscale processed, and the specific grayscale processing formula is as follows:

[0055] Y = 0.299R + 0.587G + 0.114B

[0056] Where: Y represents the grayscale value, and R, G, and B are the red, green, and blue channel values of the color image respectively. Then, filtering processing is performed to remove noise. The median filtering function is used with the parameter set as the window size of 3x3. Then, edge detection is performed on the image using the Canny edge detection algorithm to extract screen features. When processing the data of the multimodal fusion module, the Bayesian fusion algorithm is adopted. Let the image data be D1, the infrared data be D2, the ultraviolet data be D3, and the depth data be D4. The fused data D satisfies the following formula:

[0057] P(D|D1,D2,D3,D4) = P(D1|D) × P(D2|D) × P(D3|D) × P(D4|D)

[0058] × P(D) / P(D1) × P(D2) × P(D3) × P(D4)

[0059] After calculation, the data processing module integrates different types of data and transmits the data to the storage module and the result output module. The intelligent algorithm optimization module can receive the data from the storage module and adopts the reinforcement learning algorithm to adjust the algorithm parameters according to the feedback of each detection result. The comprehensive evaluation index function E is introduced. The specific comprehensive evaluation index function is as follows:

[0060] E = g(Acc,Rec,F1,Cov,Spe)

[0061] Where: Acc represents the accuracy rate, Rec represents the recall rate, F1 represents the harmonic balance coefficient, Cov represents the coverage rate, and Spe represents the specificity. The specific calculation formula of Acc is as follows:

[0062] Acc = TP / (TP + FP + FN)

[0063] Where: TP represents the number of true positive examples (the number of samples that are actually positive and are predicted to be positive), FP represents the number of false positive examples (the number of samples that are actually negative but are predicted to be positive), FN represents the number of false negative examples (the number of samples that are actually positive but are predicted to be negative). The specific calculation formulas of Rec, F1, Spe, and Cov are as follows:

[0064] Rec = TP / (TP + FN)

[0065]

[0066] Cov = TP + FN / Total

[0067] Spe = TN / (TN + FP)

[0068] Where: Total represents the total number of samples, and TN represents the number of true negative examples (the number of samples that are actually negative and are predicted to be negative);

[0069] And a reward function is set as R = h(E), where the function h is a function that determines the reward value according to the comprehensive evaluation index function E. By comprehensively considering the importance and mutual relationship of each evaluation index, it reflects the performance quality of the system. During the optimization process, the intelligent algorithm optimization module continuously tries different parameter combinations and determines the optimal parameters by evaluating the value of the reward function;

[0070] Specific working process: The image acquisition module includes a high-resolution industrial camera and a ring-shaped LED light source device. There are specific requirements for the camera resolution and frame rate. The ring-shaped LED light source provides a specific illumination intensity. The image acquisition module adjusts the parameters according to the instructions of the parameter adjustment module. The camera uses CMOS technology, converts the optical signal into a digital signal, and transmits the data in TIFF format. In practical applications, for example, on a notebook computer production line, the image acquisition module can be installed at a specific detection station. When the notebook screen passes through this station, the industrial camera quickly captures the screen image, and at the same time, the ring-shaped LED light source provides uniform and stable illumination to ensure the image quality. The parameter adjustment module can automatically adjust the frame rate and exposure time of the camera according to the speed of the production line and the type of the screen to obtain the best image effect;

[0071] The data processing module uses an image segmentation algorithm based on deep learning to process images and multi-modal data, including steps such as grayscale conversion, filtering, and edge detection. When processing the data of the multi-modal fusion module, the Bayesian fusion algorithm is used to provide an accurate data basis for the subsequent intelligent algorithm optimization module. The multi-modal fusion module includes an infrared thermal imaging camera, an ultraviolet detection camera, and a structured light depth camera, which obtain data such as the thermal distribution, ultraviolet response, and three-dimensional information of the screen, and transmit the data to the data processing module in a specific format. A fusion index function is introduced for data fusion to comprehensively judge the screen quality. The intelligent algorithm optimization module uses a reinforcement learning algorithm to adjust the algorithm parameters according to the detection result feedback. The reward function is related to indicators such as accuracy and recall. By continuously trying different parameter combinations and using historical data for model update and improvement, the detection performance of the system is improved. In practical applications, the intelligent algorithm optimization module can be combined with an artificial intelligence chip to improve the running speed and efficiency of the algorithm. For example, on a large-scale production line, a dedicated artificial intelligence chip can be used to accelerate the calculation of the reinforcement learning algorithm, enabling the system to adapt to different screen types and production environments faster. At the same time, the intelligent algorithm optimization module can also be combined with a big data analysis platform. By analyzing a large amount of historical data, potential rules of screen quality problems are discovered, and possible quality problems are warned in advance;

[0072] Furthermore, by automatically adjusting the detection parameters and algorithm models according to different screen models, production processes, and environmental conditions, the adaptability and flexibility of the system are greatly improved. In the initial stage of operation, a basic detection model is established by learning a certain number of screen samples with known quality. Subsequently, new data is continuously collected during the detection process, and the differences from the existing model are analyzed in real time. When a large deviation is found between the new data and the existing model, the system automatically triggers the algorithm optimization module for adjustment and update. By introducing the comprehensive evaluation index function E and the reward function R, the system can comprehensively consider the importance and mutual relationship of multiple evaluation indicators, accurately reflect the performance of the system, and determine the optimal parameters by evaluating the value of the reward function, thereby improving the detection ability for specific types of defects. This adaptive intelligent algorithm optimization greatly reduces the maintenance cost and operation difficulty, ensures that the system always maintains high detection accuracy and stability, and becomes more intelligent and efficient over time.

[0073] Embodiment 3:

[0074] On the basis of Embodiment 2, as shown in the accompanying drawings of the specification Figures 1 - 2As shown, the result output module outputs the detection results in a visual form, including the screen quality level, specific defect types, and location information. The output methods include display on the display screen, printing reports, and transmitting them to other devices via the network. The result output module receives data from the data processing module and the intelligent algorithm optimization module. After format conversion and collation, it displays the screen quality level in a color-coded manner. Red indicates serious defects, yellow indicates minor defects, and green indicates qualified. For specific defect types, a combination of text descriptions and icon identifications is used to clearly show the nature of the defects. At the same time, a 3D model of the screen is displayed on the display screen, marking the locations of the defects to facilitate users to intuitively understand the screen quality situation. Subsequently, a print report of the detection results is generated, providing detailed detection data and analysis results. The storage module is used to store the image data, multi-modal data, detection results, and intermediate data during the algorithm optimization process. The solid-state drive array storage technology is adopted to ensure the security and traceability of the data. The storage module can store the data of the image acquisition module, data processing module, intelligent algorithm optimization module, and result output module. During the storage process, the data is classified and compressed. The image data is classified and stored according to the screen model and production date information to improve the storage efficiency and retrieval speed. A lossless compression algorithm is used to compress the data to reduce the storage space occupancy. The parameter adjustment module adjusts the parameters of the image acquisition module, data processing module, and multi-modal fusion module according to the instructions of the intelligent algorithm optimization module. The parameter adjustment module automatically adjusts the parameters of each module through a parameter adjustment algorithm based on fuzzy logic according to the current detection requirements and system performance, including adjusting the exposure time and gain parameters of the camera to adapt to different lighting conditions. When the ambient light intensity changes, the parameter adjustment module adjusts the exposure time and gain of the camera according to the pre-set fuzzy rules. When the light intensity is strong, the exposure time is reduced and the gain is lowered. When the light intensity is weak, the exposure time is increased and the gain is raised. The intensity and angle of the light source are adjusted to improve the image quality. According to the reflection characteristics of the screen and the brightness distribution of the image, the angle and intensity of the light source are adjusted to make the defects in the image more obvious. For the multi-modal fusion module, the parameter settings of each sensor in the multi-modal fusion module are adjusted to optimize the data fusion effect;

[0075] Specific work process: The result output module outputs the detection results in a visual form, including information such as the screen quality level, specific defect types and locations, etc. The output methods are diverse. It receives data from the data processing module and the intelligent algorithm optimization module, organizes and presents them. The storage module is used to store various data and intermediate data during the detection process. It adopts solid-state drive array storage technology to ensure data security and traceability, conducts data interaction with other modules, classifies and compresses the data. The parameter adjustment module adjusts the parameters of each module according to the instructions of the intelligent algorithm optimization module. Through a parameter adjustment algorithm based on fuzzy logic, it automatically adjusts the parameters according to the detection requirements and system performance, including the parameters of cameras, light sources, and various sensors in the multi-modal fusion module. In practical applications, the parameter adjustment module can be integrated with an automated control system to achieve automatic parameter adjustment. For example, when the ambient light intensity changes, the parameter adjustment module can automatically adjust the exposure time and gain of the camera, as well as the intensity and angle of the light source, without manual intervention. At the same time, the parameter adjustment module can also automatically adjust the parameters of various sensors in the multi-modal fusion module according to the model and production batch of the screen to ensure the accuracy and consistency of detection;

[0076] Furthermore, by adjusting the exposure time and gain parameters of the camera, it can adapt to different lighting conditions. When the light intensity is strong, reduce the exposure time and lower the gain; when the light intensity is weak, increase the exposure time and raise the gain, thus ensuring the stability and accuracy of the image quality. In addition, by adjusting the intensity and angle of the light source, according to the reflection characteristics of the screen and the brightness distribution of the image, the defects in the image can be made more obvious, improving the sensitivity of defect detection. For the multi-modal fusion module, adjusting the parameter settings of each sensor can optimize the data fusion effect and further enhance the system's ability to comprehensively judge the screen quality. This intelligent parameter adjustment function enhances the stability and reliability of the system, improves the detection efficiency and accuracy, and enables the system to better adapt to various complex detection environments and requirements.

[0077] Meanwhile, this application uses specific terms to describe the implementation manners of this application. Such as "one implementation manner", "one implementation manner", and / or "some implementation manners" mean a certain feature, structure, or characteristic related to at least one implementation manner of this application. Therefore, it should be emphasized and noted that the "an implementation manner" or "one implementation manner" or "an alternative implementation manner" mentioned twice or more at different positions in this specification does not necessarily refer to the same implementation manner. In addition, certain features, structures, or characteristics in one or more implementation manners of this application can be appropriately combined.

[0078] Some aspects of the present application can be executed entirely by hardware, entirely by software (including firmware, resident software, microcode, etc.), or by a combination of hardware and software. The above-mentioned hardware or software can all be referred to as "data block", "module", "engine", "unit", "component" or "system". The processor can be one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, or combinations thereof. In addition, aspects of the present application may be embodied as a computer product located in one or more computer-readable media, which includes computer-readable program code. For example, the computer-readable media may include, but is not limited to, magnetic storage devices (such as hard disks, floppy disks, magnetic tapes...), optical disks (such as compact disks CD, digital versatile disks DVD...), smart cards, and flash memory devices (such as cards, sticks, key drives...).

[0079] The computer-readable medium may contain a propagated data signal having computer program code embodied therein, for example, on a baseband or as part of a carrier wave. The propagated signal may take many forms, including electromagnetic, optical, or the like, or suitable combinations thereof. The computer-readable medium can be any computer-readable medium other than a computer-readable storage medium that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. The program code located on the computer-readable medium can be propagated through any appropriate medium, including radio, cable, fiber optic cable, radio frequency signal, or similar media, or any combination of the above media.

[0080] Similarly, it should be noted that, in order to simplify the description disclosed in this application and thus help the understanding of one or more embodiments of the invention, in the foregoing description of the embodiments of this application, multiple features are sometimes grouped into one embodiment, drawing or description thereof. However, this disclosure method does not mean that the features required by the subject matter of this application are more than those mentioned in the claims. In fact, the features of the embodiments are fewer than all the features of the individual embodiments disclosed above. In some embodiments, numbers describing components and the quantity of attributes are used. It should be understood that such numbers used for the description of the embodiments are modified by the modifiers "about", "approximate" or "substantially" in some examples. Unless otherwise stated, "about", "approximate" or "substantially" indicate that the numbers allow a variation of ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, and this approximate value can be changed according to the characteristics required by individual embodiments. In some embodiments, the numerical parameters should consider the specified significant digits and adopt the method of retaining the general number of digits. Although the numerical ranges and parameters used to confirm the breadth of the scope in some embodiments of this application are approximate values, in specific embodiments, such numerical settings are as precise as possible within the feasible range.

[0081] Although the present invention is disclosed above in preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make possible changes and modifications without departing from the spirit and scope of the present invention. Therefore, all modifications, equivalent changes and decorations made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention fall within the protection scope defined by the claims of the present invention.

Claims

1. A notebook screen quality inspection system based on machine vision, characterized by: The detection system includes an image acquisition module, a data processing module, a multimodal fusion module, an intelligent algorithm optimization module, a result output module, a storage module and a parameter adjustment module; The image acquisition module acquires the laptop screen image through a high-resolution industrial camera and a ring-shaped LED light source device, and transmits the image data to the data processing module in TIFF format. The multimodal fusion module combines the multiple sensor data of the infrared thermal imaging camera, the ultraviolet detection camera and the structured light depth camera, and performs fusion analysis with the data of the image acquisition module. The intelligent algorithm optimization module continuously adjusts the algorithm according to the data processing results, and adjusts the algorithm parameters according to the feedback of each detection result. After the data processing module processes the image and multimodal data using the image segmentation algorithm based on deep learning, the results are transmitted to the result output module and the storage module. The parameter adjustment module adjusts the parameters of each module according to the instructions of the intelligent algorithm optimization module. When the system is working, the image acquisition module first acquires the image data, and then the multimodal fusion module introduces the multi-sensor data of the infrared, ultraviolet and depth cameras. The data processing module processes these data, and the intelligent algorithm optimization module optimizes according to the processing results. The final result is output by the result output module and stored in the storage module.

2. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The image acquisition module includes an XoF16K industrial camera and an LDR2-150W-C annular LED light source device. The industrial camera has a resolution of 48 million pixels and a frame rate of 50 frames per second. The annular LED light source device provides uniform and stable lighting, and the lighting intensity is between 2500lx and 3200lx. The image acquisition module adjusts camera parameters and light source intensity according to different screen materials and brightness data. During the acquisition process, the camera's image sensor uses CMOS technology to convert optical signals into electrical signals, which are then converted into digital signals through a high-speed analog-to-digital converter and transmitted to the data processing module in the form of TIFF data.

3. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The multimodal fusion module is provided with an infrared thermal imaging camera, an ultraviolet detection camera and a structured light depth camera. The infrared thermal imaging camera is used to obtain the thermal distribution image of the screen. The data format is a 16-bit grayscale image format and is transmitted to the data processing module via a high-speed USB3.

0. The ultraviolet detection camera detects the response data of the screen to ultraviolet rays and is also transmitted in the JPEG2000 format. The structured light depth camera obtains the three-dimensional information of the screen and the data is transmitted in the point cloud data format. Assume that the temperature distribution function of the screen at a certain moment is T(x, y, z), (x, y, z represent spatial coordinates), introduce the fusion index function F(T, I, U, D), where I represents the image data feature vector, U represents the ultraviolet data feature vector, and D represents the depth data feature vector. The specific function definition is as follows: Among them: α, β, γ, δ are weight coefficients, and their values ​​are determined by deep learning and optimization of a large amount of historical data. i |、|U j | and |D k |represent the elements in the feature vectors of image data, ultraviolet data and depth data respectively, ‖I‖, ‖U‖ and ‖D‖ represent the norms of the feature vectors of image data, ultraviolet data and depth data respectively.

4. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The data processing module uses an image segmentation algorithm based on deep learning to process the image data transmitted by the image acquisition module. First, the image is grayed. The specific gray processing formula is as follows: Y=0.299R+0.587G+0.114B Among them: Y represents the gray value, R, G and B are the red, green and blue channel values ​​of the color image respectively, and then filtering is performed to remove noise. The median filter function is used, and the parameter is set to the window size of 3x3. Then the image is edge detected, and the Canny edge detection algorithm is used to extract the screen features. When processing the data of the multimodal fusion module, the Bayesian fusion algorithm is used. Let the image data be D1, the infrared data be D2, the ultraviolet data be D3, and the depth data be D4. The fused data D satisfies the following formula: P(D|D1,D2,D3,D4)=P(D1|D)×P(D2|D)×P(D3|D)×P(D4|D) ×P(D) / P(D1)×P(D2)×P(D3)×P(D4) The data processing module integrates different types of data after calculation and transmits the data to the storage module and the result output module.

5. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The intelligent algorithm optimization module can receive data from the storage module, adopt a reinforcement learning algorithm, adjust the algorithm parameters according to the feedback of each detection result, and introduce a comprehensive evaluation index function E. The specific comprehensive evaluation index function is as follows: E=g(Acc,Rec,F1,Cov,Spe) Among them: Acc represents accuracy, Rec represents recall, F1 represents harmonic balance coefficient, Cov represents coverage, Spe represents specificity, and the specific calculation formula of Acc is as follows: Acc=TP / (TP+FP+FN) Among them: TP represents the number of true positive samples (the number of samples that are actually positive and predicted as positive samples), FP represents the number of false positive samples (the number of samples that are actually negative but predicted as positive samples), FN represents the number of false negative samples (the number of samples that are actually positive but predicted as negative samples), and the specific calculation formulas of Rec, F1, Spe and Cov are as follows: Rec=TP / (TP+FN) Cov=TP+FN / Total Spe=TN / (TN+FP) Among them: Total represents the total number of samples, TN represents the number of true negative examples (the number of samples that are actually negative and predicted to be negative); A reward function is set to R=h(E). Function h is a function that determines the reward value based on the comprehensive evaluation index function E. The importance and mutual relationship of each evaluation index are comprehensively considered to reflect the performance of the system. During the optimization process, the intelligent algorithm optimization module continuously tries different parameter combinations and determines the optimal parameters by evaluating the value of the reward function.

6. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The result output module outputs the test results in a visual form, including screen quality grade, specific defect type and location information. The output methods include display screen display, printed report and transmission to other devices via the network. The result output module receives data from the data processing module and the intelligent algorithm optimization module, and after format conversion and sorting, displays the screen quality grade in a color-coded manner. Red indicates serious defects, yellow indicates minor defects, and green indicates qualified. For specific defect types, a combination of text description and icon identification is used to clearly display the nature of the defect. At the same time, a three-dimensional model of the screen is displayed on the display screen, and the location of the defect is marked to facilitate users to intuitively understand the screen quality. The test results are then generated into a printed report, providing detailed test data and analysis results.

7. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The storage module is used to store image data, multimodal data, detection results and intermediate data in the algorithm optimization process during the detection process. It adopts solid-state hard disk array storage technology to ensure the security and traceability of the data. The storage module can store data of the image acquisition module, data processing module, intelligent algorithm optimization module and result output module. During the storage process, the data is classified and compressed. The image data is classified and stored according to the screen model and production date information to improve storage efficiency and retrieval speed. A lossless compression algorithm is used to compress the data to reduce storage space occupancy.

8. The notebook screen quality inspection system based on machine vision according to claim 1, characterized in that: The parameter adjustment module adjusts the parameters of the image acquisition module, the data processing module and the multimodal fusion module according to the instructions of the intelligent algorithm optimization module. The parameter adjustment module automatically adjusts the parameters of each module according to the current detection requirements and system performance through the parameter adjustment algorithm based on fuzzy logic, including adjusting the exposure time and gain parameters of the camera to adapt to different lighting conditions. When the ambient light intensity changes, the parameter adjustment module adjusts the exposure time and gain of the camera according to the pre-set fuzzy rules. When the light intensity is strong, the exposure time is reduced and the gain is lowered. When the light intensity is weak, the exposure time is increased and the gain is increased. The intensity and angle of the light source are adjusted to improve the image quality. According to the reflection characteristics of the screen and the brightness distribution of the image, the angle and intensity of the light source are adjusted to make the defects in the image more obvious. For the multimodal fusion module, the parameter settings of each sensor in the multimodal fusion module are adjusted to optimize the data fusion effect.

9. A method for detecting the quality of a notebook screen based on machine vision, applicable to the notebook screen quality detection system based on machine vision according to any one of claims 1 to 8, characterized in that: The detection method comprises the following steps: Step 1: Use a high-resolution industrial camera and a ring-shaped LED light source device to obtain the laptop screen image. Adjust the camera parameters and light source intensity according to different screen materials and brightness. The image sensor uses CMOS technology to convert the optical signal into an electrical signal, which is then converted into a digital signal by a high-speed analog-to-digital converter and transmitted to the data processing module in TIFF format. Step 2: The multimodal fusion module is equipped with an infrared thermal imaging camera, an ultraviolet detection camera and a structured light depth camera. The infrared thermal imaging camera obtains the screen thermal distribution image (16-bit grayscale image format), the ultraviolet detection camera detects the screen's response data to ultraviolet rays (JPEG2000 format), and the structured light depth camera obtains the screen's three-dimensional information (point cloud data format). The fusion index function F (T, I, U, D) is introduced to fuse the multimodal data, including processing the feature vectors of image data, ultraviolet data and depth data, and calculating through weight coefficients. The data processing module uses an image segmentation algorithm based on deep learning to process the image data, including grayscale processing, filtering processing and edge detection. The Bayesian fusion algorithm is used to process the data of the multimodal fusion module to integrate different types of data. Step 3: The intelligent algorithm optimization module adopts the reinforcement learning algorithm, adjusts the algorithm parameters according to the feedback of the detection results, introduces the comprehensive evaluation index function E (E = g (Acc, Rec, F1, Cov, Spe)), and sets the reward function R = h (E), comprehensively considers the importance and mutual relationship of each evaluation index to reflect the performance of the system, continuously tries different parameter combinations, and determines the optimal parameters by evaluating the value of the reward function. The parameter adjustment module adjusts the parameters of the image acquisition module, the data processing module and the multimodal fusion module according to the instructions of the intelligent algorithm optimization module, including the camera exposure time, gain, light source intensity and angle; Step 4: The result output module outputs the test results in a visual form, including screen quality grade, specific defect type and location information. The output methods include display screen display, printed report and transmission to other devices via the network. The storage module stores various data in the test process and intermediate data in the algorithm optimization process, and uses solid-state hard disk array storage technology to classify and compress the data.

Images