Machine vision-based liquid crystal panel card automatic detection system and method
By combining machine vision and deep learning models, fully automated inspection of LCD boards has been achieved, solving the problems of low efficiency and insufficient stability in traditional inspection methods and improving inspection accuracy and efficiency.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING SIFANG JIBAO ENG TECH
- Filing Date
- 2026-02-06
- Publication Date
- 2026-05-29
Smart Images
Figure CN122115359A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of liquid crystal screen inspection technology, and more specifically, relates to an automatic inspection system and method for liquid crystal boards based on machine vision. Background Technology
[0002] In human-computer interaction, LCD screens are crucial devices for information input and output, and their quality directly affects the user's reading and interpretation of the input and output information. During the production process, many uncertainties exist; manufacturing processes and materials themselves can cause some pixels to display abnormally, directly leading to product defects. Therefore, rigorous inspection is necessary before the product leaves the factory.
[0003] Traditional inspections are conducted manually by visual inspection. However, manual inspection is easily affected by factors such as environment and attention, and has a certain degree of randomness. This randomness can lead to significant human error. At the same time, employees may experience visual fatigue during long hours of continuous work. All these factors contribute to low inspection efficiency and have an adverse impact on the company's profitability.
[0004] With the development of automation technology in China, image processing and machine vision technologies are becoming increasingly mature in application. Machine vision technology for surface defect detection has been applied in multiple industrial fields. Technological advancements in LCD screen defect detection show that existing solutions have achieved certain breakthroughs: for detecting small spots, some solutions use singular value decomposition (SVD) image reconstruction to identify minute defects; others achieve micron-level detection accuracy through local image template matching, but these algorithms are complex and time-consuming, and template matching requires a large number of templates, which is not conducive to engineering automation; still others use the Laplacian Gaussian (LOG) operator to detect defects in the frequency domain, achieving good results, but the DOG operator has been proven to have a more continuous and smooth edge detection advantage than the LOG operator. Existing technologies mostly focus on single defect detection, and are easily affected by screen brightness fluctuations and flickering interference in screen dead pixel and button testing, resulting in insufficient detection stability.
[0005] Most of the methods mentioned above are based on traditional machine learning, requiring the manual setting of numerous features during testing. They perform well when the screen brightness is stable and unaffected by external light. However, during screen dead pixel and button testing, the screen is illuminated, and its brightness is affected by various factors. For example, the screen brightness value set by the software directly affects the screen's brightness, and the flickering of the LCD screen at a fixed frequency after power-on also impacts the detection results. Summary of the Invention
[0006] To address the shortcomings of existing technologies, this invention provides an automatic inspection system and method for LCD boards based on machine vision.
[0007] The present invention adopts the following technical solution.
[0008] A first aspect of the present invention provides an automatic inspection system for liquid crystal display boards based on machine vision, comprising: The system includes positioning fixtures, an image acquisition module, a key simulation module, an electrical control module, and an industrial computer, among which: The positioning fixture includes a testing table 1, the upper end of which is provided with a groove that matches the shape of the LCD screen to be tested. This groove is used to place the LCD board card upside down and make the screen face downwards. The image acquisition module is located above the positioning fixture and is used to acquire screen images of the LCD board. The button simulation module is set on the positioning fixture and is used to simulate the physical button actions on the LCD board. The electrical control module is communicatively connected to the button simulation module to drive it to perform button actions; The industrial computer is communicatively connected to the electrical control module and the image acquisition module, respectively, for executing: The electrical control module drives the button triggering device to execute button actions. Before and after the button press, the control image acquisition module acquires the screen image of the LCD board respectively; By analyzing the captured screen images and considering the changes in the screen display before and after button presses, the effectiveness of the button functions and the defect status of the screen can be determined.
[0009] Optionally, the button simulation module includes a finger cylinder 5 and a slide. The finger cylinder 5 is positioned above the slide. After the board is placed on the slide, it slides to a designated position so that the finger cylinder 5 is positioned directly above the button.
[0010] Optionally, the electrical control module includes a network relay 6 and a solenoid valve, wherein the solenoid valve is connected to the air circuit of the finger cylinder 5 and the air circuit is controlled to open or close via the network relay 6.
[0011] A second aspect of the present invention provides a machine vision-based automatic inspection method for liquid crystal display (LCD) boards, and a machine vision-based automatic inspection system for LCD boards based on the first aspect of the present invention, comprising: Fix the LCD panel to be tested with the screen facing down onto the positioning fixture; The initial screen image of the LCD board is acquired through the image acquisition module; The control button simulation module clicks the physical buttons on the LCD panel. After clicking the button, the screen image of the LCD board is captured again by the image acquisition module after the trigger. Based on the initial screen image and the screen image after triggering, the changes in the screen image are analyzed to determine the effectiveness of the button function and the defect status of the screen.
[0012] Optionally, determining the validity of a button function includes: The initial screen image and the screen image after triggering are respectively input into the deep learning model to extract the corresponding first feature vector and second feature vector; Calculate the Euclidean distance between the first eigenvector and the second eigenvector; The Euclidean distance is compared with a preset threshold, and the screen image is judged based on the comparison result to determine whether the button function is normal.
[0013] Optionally, the deep learning model is a neural network based on the ResNet architecture, and its deep residual blocks contain deformable convolutional layers.
[0014] Optionally, after acquiring the initial screen image, the process also includes segmenting the screen region: Convert the acquired color image to a grayscale image; Based on the brightness distribution characteristics of grayscale images, a dynamic threshold segmentation method is used to segment the display area of the LCD screen and generate an image of the screen area to be analyzed.
[0015] Optionally, the display area of the LCD screen is segmented using a dynamic threshold segmentation method, including: Obtain the maximum and minimum pixel grayscale values of the screen region image to be analyzed; Calculate the dynamic segmentation threshold based on the maximum and minimum values; Binarization of grayscale images is performed using a dynamic segmentation threshold. Edge extraction is performed on the binarized image, and the edge contour of the LCD screen is selected based on the fixed area attribute of the LCD screen. The fitted edge contour is a rectangle, and this rectangular region is cropped as the screen region image to be analyzed.
[0016] A third aspect of the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program, when loaded onto the processor, implements an automatic detection method for liquid crystal display panels based on machine vision according to a second aspect of the present invention.
[0017] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements an automatic detection method for liquid crystal display panels based on machine vision according to a second aspect of the present invention.
[0018] Compared with the prior art, the beneficial effects of the present invention include at least the following: 1. This invention, by setting up a dedicated positioning fixture and a button simulation module (such as a finger cylinder), realizes the automated testing of the physical button functions of LCD boards based on the automatic acquisition of screen images, solving the problem that traditional manual inspection and general vision platforms cannot effectively verify button functions.
[0019] 2. This invention uses an improved deep learning model (such as ResNet with deformable convolution in deep residual blocks) to extract features and quantize differences in screen images before and after button triggering. This solves the problems of insufficient ability of traditional image algorithms to distinguish highly similar LCD screens and susceptibility to interference from screen brightness fluctuations, thereby improving the accuracy and stability of function judgment.
[0020] 3. This invention integrates hardware modules for positioning, image acquisition, button triggering, and intelligent analysis, and is uniformly scheduled and controlled by an industrial control computer. This enables integrated and fully automatic detection of both the appearance defects (such as dead pixels) and internal functions (such as buttons and software responses) of LCD boards, thereby improving detection efficiency and coverage.
[0021] 4. The system design of this invention supports integration with production line control systems (such as PLC and MES), realizing a closed-loop automated process from board loading, positioning, testing to result feedback, providing a stable and traceable quality inspection solution for large-scale production. Attached Figure Description
[0022] Figure 1 This is a schematic diagram of the hardware components of the detection system provided according to an embodiment of the present invention, including: 1-detection table (1), 2-industrial camera, 3-lens, 4-camera bracket, 5-finger cylinder, 6-network relay; Figure 2 This is a schematic diagram of the detection process provided according to an embodiment of the present invention. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of this invention.
[0024] In Embodiment 1, this invention provides an automatic inspection system for LCD boards based on machine vision, such as... Figure 1 As shown, the system includes: The system includes positioning fixtures, an image acquisition module, a key simulation module, an electrical control module, and an industrial computer, among which: The positioning fixture includes a testing table (1), which has a groove at its upper end that matches the shape of the LCD screen to be tested. The groove is used to place the LCD board card upside down and make the screen face downward. The image acquisition module is located above the positioning fixture and is used to acquire screen images of the LCD board. The button simulation module is set on the positioning fixture and is used to simulate the physical button actions on the LCD board. The electrical control module is communicatively connected to the button simulation module to drive it to perform button actions; The industrial computer is communicatively connected to the electrical control module and the image acquisition module, respectively, for executing: The electrical control module drives the button triggering device to execute button actions. Before and after the button press, the control image acquisition module acquires the screen image of the LCD board respectively; By analyzing the captured screen images and considering the changes in the screen display before and after button presses, the effectiveness of the button functions and the defect status of the screen can be determined.
[0025] Preferably, the button simulation module includes a finger cylinder 5, which is positioned above the slide. After the board is placed into the slide and the slide retracts into its position, the finger cylinder 5 is located directly above the button, used to simulate physical button actions and control the LCD screen to switch images.
[0026] Preferably, the image acquisition module includes an industrial camera 2, a lens 3, and a camera bracket 4. The industrial camera 2 is used to capture the display screen of the LCD board to obtain image data. The industrial camera 2 is fixed above the inspection table 1 by the camera bracket 4. The lens 3 is mounted on the industrial camera 2 to ensure the clarity and stability of the captured image.
[0027] Preferably, the electrical control module includes a network relay 6 and a solenoid valve, the solenoid valve being connected to the air path of the finger cylinder 5; Preferably, the industrial control computer is connected to the electrical control module and the image acquisition module respectively, and is used to generate test commands and send them to the electrical control module to trigger the action of the actuator; after the action is completed, it controls the image acquisition module to capture the image; it receives and analyzes the image data and outputs the test results.
[0028] Preferably, the industrial control computer includes an instruction scheduling unit and a vision analysis unit, wherein: The instruction scheduling unit is used to generate and send test instructions to the electrical control module to trigger the action of the actuator; after receiving the action completion signal, it controls the image acquisition module to capture the image. The visual analysis unit is used to receive and analyze image data captured by the image acquisition module, and output the detection results of screen defects and button functions.
[0029] Preferably, the machine vision-based automatic inspection system for LCD boards further includes circuitry, including a power supply, connectors, and solenoid valves, for connecting and controlling the various hardware components to achieve electrical functions.
[0030] Preferably, the electrical control unit includes: The electrical structure of the test fixture mainly includes the following parts: Spring pins: used to connect various electrical components (such as J1, J2, J4, J5 connectors, etc.) and equipment to achieve circuit conduction and information acquisition.
[0031] Solenoid valves: The negative terminals of each solenoid valve are connected in parallel to the 0V terminal block of the LM450. The on / off state of the solenoid valves is controlled by the network relay (6) to realize the action of the finger cylinder 5.
[0032] Network Relay 6: As one of the core components of circuit control, it uses Ethernet communication to remotely control the circuit on and off by receiving software commands. It can coordinate the collaborative work of devices such as cameras, cylinders, and sensors. For example, it can cut off unnecessary circuits before image acquisition to reduce interference, or control the power supply to shut down after the test is completed, thereby improving the automation and safety of the test system.
[0033] Sensor switches: such as switches, are used to detect the status and position of tooling and provide feedback signals for system control.
[0034] Power supply: Provides the power required by the test system, such as 5V for the MMI plug-in and 24V for network relays and solenoid valves.
[0035] The purpose of this invention is to propose a liquid crystal screen testing device that can be applied to multiple testing items. In the button testing section, a deep learning-based recognition method is applied, which can accurately distinguish between two similar images and accurately detect button hardware problems.
[0036] In Embodiment 2, this invention provides an automatic inspection method for LCD boards based on machine vision, which, based on the automatic inspection system for LCD boards based on machine vision described in Embodiment 1, includes the following steps: Step 1: Fix the LCD panel to be tested with the screen facing down onto the positioning fixture.
[0037] Preferably, step 1 includes: Step 1.1: Place the LCD panel to be tested, screen down, in the groove of the testing stage 1. The groove size matches the LCD screen. Step 1.2: Connect the system power supply, start the industrial camera 2 and the detection software, and complete the system initialization.
[0038] Step 2: Acquire the initial screen image of the LCD board using the image acquisition module.
[0039] Specifically, the image acquisition device used in this invention is a Hikvision industrial camera MV-CS050-10GC, and image acquisition is uniformly performed using the Hikvision industrial camera interface. Since the algorithm module processes images slowly, synchronous image detection would affect the real-time performance of the test; therefore, asynchronous image acquisition is used.
[0040] Open the image acquisition device to begin capturing images. OpenCV only supports BGR channel image display format; the camera acquisition format must be set to BGR8. Use the `future` method to asynchronously create multiple threads for image detection. After writing the TCP communication module in MFC, use `if` statements to use message instructions as conditions for image capture. Then, by calling the Hikvision camera API, you can obtain one or more images from the opened camera.
[0041] Preferably, after acquiring the initial screen image, the method further includes segmenting the screen region: Convert the acquired color image to a grayscale image; Based on the brightness distribution characteristics of grayscale images, a dynamic threshold segmentation method is used to segment the display area of the LCD screen and generate an image of the screen area to be analyzed.
[0042] More preferably, the display area of the LCD screen is segmented using a dynamic threshold segmentation method, including: Obtain the maximum and minimum pixel grayscale values of the screen region image to be analyzed; Calculate the dynamic segmentation threshold based on the maximum and minimum values; Binarization of grayscale images is performed using a dynamic segmentation threshold. Edge extraction is performed on the binarized image, and the edge contour of the LCD screen is selected based on the fixed area attribute of the LCD screen. The fitted edge contour is a rectangle, and this rectangular region is cropped as the screen region image to be analyzed.
[0043] Specifically, this invention only detects the LCD screen area, so the LCD screen area needs to be segmented as an image to be processed, and then the screen is detected. Based on the image contrast characteristics, an image edge processing method is used to obtain the rectangular area of the LCD screen to achieve accurate segmentation of the LCD screen.
[0044] Image grayscale conversion is a method for processing color images. Grayscale images, like color images, still reflect the overall and local distribution and characteristics of chromaticity and brightness levels. As a single-channel image, it is simpler to analyze and process, so color images can be converted to grayscale. Color images come in various formats; for ease of channel conversion in OpenCV, a BGR format image is used here. There are generally three methods for converting BGR to grayscale: the average method, the maximum value method, and the weighted average method. This invention uses the commonly used weighted average method: (1) In the formula, It is the converted grayscale value. , and These are the grayscale values of the Red, Green, and Blue channels corresponding to the BGR color image, respectively.
[0045] First, the BRG image is converted into a grayscale image using formula (1). Then, the grayscale image threshold is processed to separate the obvious foreground and background. Based on the area attribute of the segmented region, the LCD screen area is extracted.
[0046] Thresholding involves setting a threshold value. If the grayscale value of a pixel is higher than this threshold, it is considered a point of interest; otherwise, it is considered a background point. Assuming a grayscale image... The pixel value at that location is After thresholding, its value is The threshold is ,but (2) In the formula, 1 represents a pixel on the LCD screen and 0 represents a background pixel.
[0047] When this system captures images of the LCD screen, the image quality is slightly uneven due to lens reflection and diffuse light reflection, and the frame area of the LCD screen is also captured. This invention employs a fast global thresholding method to quickly segment the LCD screen area. Assuming the LCD screen area... The maximum grayscale value of that pixel is The smallest is ,but: (3) A threshold of 120 was set. After binarization, the image was first smoothed using a mean filter, and then an edge extraction algorithm was used to obtain the edge lines of the liquid crystal portion. Since the area of the liquid crystal screen is fixed, the area was set... The edge lines of the LCD screen are filtered out.
[0048] At this point, the obtained edge line is an irregular curve. To extract the rectangular region of the LCD screen, this edge line is fitted into an approximate rectangle, and the geometric information of the fitted rectangle is obtained. The largest bounding rectangle within the closed region is selected as the region of interest. Thus, the rectangular region of the LCD screen has been obtained. The rectangular region is extracted using a screenshot method, and the image of this rectangular region of the LCD screen is then subjected to detection.
[0049] Step 3: Control the button simulation module to click the physical buttons on the LCD board.
[0050] Step 4: After clicking the button, the image acquisition module will capture the screen image of the LCD board after the trigger.
[0051] Step 5: Based on the initial screen image and the screen image after triggering, analyze the changes in the screen image to determine the effectiveness of the button function and the defect status of the screen.
[0052] Preferably, determining the validity of a button function includes: The initial screen image and the screen image after triggering are respectively input into the deep learning model to extract the corresponding first feature vector and second feature vector; Calculate the Euclidean distance between the first eigenvector and the second eigenvector; The Euclidean distance is compared with a preset threshold, and the comparison result is used to determine whether the screen image has undergone a valid change, thereby determining whether the button function is normal. More preferably, the deep learning model is a neural network based on the ResNet architecture, and its deep residual blocks contain deformable convolutional layers.
[0053] Specifically, after the LCD board is powered on, it can control the finger cylinder (5) to click the button to change the screen image. Therefore, the effectiveness of the button can be determined by detecting whether the screen changes. Since there are many types of LCD boards, the screen changes caused by the button are not the same, so the problem can be regarded as a multi-classification problem. In order to balance reasoning speed and classification effect, this invention uses RESNET18 as the classification model.
[0054] RESNET is a widely used network architecture that solves the parameter explosion problem in neural networks, laying a solid foundation for the deep development of neural networks. Furthermore, this network performs excellently in numerous classification tasks. Therefore, this invention selects RESNET as the backbone network for improvement. Its network architecture, after initial convolutional pooling, enters the residual module and channel sampling residual module, and finally performs average pooling, with the result output by a fully connected layer.
[0055] The main idea is to map facial images onto a high-dimensional feature space and train a neural network to learn a mapping function, ensuring that features from different images are significantly different in the feature space, while features from the same images are very similar. In the feature space, image similarity can be determined by calculating Euclidean distance. Since the Euclidean distance between the same image is small, while the Euclidean distance between different images is large, setting a threshold for the Euclidean distance between image feature vectors can effectively distinguish between LCD screens. Let two n-dimensional vectors... and The Euclidean distance between them is as follows: (4) in, This represents the value of the first vector a in the k-th dimension; This represents the value of the second vector b in the k-th dimension.
[0056] Based on the principles of RESNET, the algorithm flow is designed as follows: 1) Image preprocessing: Convert the image to a 640*640 RGB image and input it into the model.
[0057] 2) Load the template, convert the template image to a 640*640 RGB image, and pass it into the model.
[0058] 3) Similarity comparison: Perform Euclidean distance calculation on the vectors output by the model as shown in equation (4).
[0059] 4) Processing results: If the Euclidean distance is less than 1, the images are considered to be consistent; otherwise, they are considered inconsistent.
[0060] It should be noted that, in order to achieve high-precision differentiation of subtle differences in LCD screen images and adapt to high-resolution input, this invention has made targeted optimizations to the ResNet model used, mainly including: (1) Introduce deformable convolution modules in stages: To enhance the model's ability to extract irregular and deformable features (such as thin scratches and local brightness distortions) from screen images, deformable convolutions are introduced in the deep network stages. Specifically, in the deep residual blocks of the ResNet architecture (e.g., stages 3 and 4 of the original network), some standard 3×3 convolutions are replaced with deformable convolutions. Deformable convolutions learn additional offset parameters, allowing the sampling position of the convolution kernel to adapt to the image content, thus achieving a more flexible receptive field. This design is based on the characteristics of feature hierarchy: shallow features mainly contain basic information such as edges and textures, for which standard convolutions are sufficient; while deep features contain higher-level semantic information, and deformable convolutions can better model their potential irregular morphological changes.
[0061] (2) Computational efficiency balance design: The aforementioned optimizations improve performance while also considering computational efficiency. A "selective enhancement" strategy is formed by introducing deformable convolutions only in deeper layers and retaining efficient standard convolutions in shallower layers. Combined with optimizations to the input downsampling strategy, the incremental computational complexity and memory usage of the improved model are kept within a reasonable range (e.g., approximately 15%-20% more than the standard ResNet-18), ensuring the algorithm's real-time performance and feasibility in industrial inspection environments.
[0062] Based on this optimized ResNet model (which can be called Deformable ResNet), high-dimensional feature vectors of screen images are extracted, and Euclidean distance is used for similarity measurement. This can more accurately quantify the subtle differences between the screen before and after a button is triggered, thereby reliably determining whether the button function is normal.
[0063] In addition, the testing equipment provided by this invention has multiple functions such as dead pixel detection, text detection, LED detection, and button detection, and can perform comprehensive testing on the performance of LCD PCBs.
[0064] If a button is successfully pressed, it causes a change in the LCD screen. An image is taken before and after the change. If the images are different, the button function is considered normal. However, the images on the LCD screen have a high similarity, making it difficult to distinguish between them using traditional methods. While deep learning classification algorithms can accurately differentiate images, they require retraining for each new board model. Therefore, a FaceNet-based template matching algorithm was used. Images are converted into 128-dimensional vectors, and the similarity is determined by calculating the distance between two vectors. This ensures accurate button recognition without requiring repeated model training and updates.
[0065] In Embodiment 3, the present invention provides a machine vision-based method for detecting dead pixels on liquid crystal display panels, which can be used as an independent or parallel detection function of the detection system.
[0066] The specific steps are as follows: Step 1: Switch the control screen to the dead pixel detection interface and trigger the screen to display a specific solid color detection screen. In this embodiment, the detection screen includes alternating displays of a pure white screen and a pure blue screen. Dead pixels are typically displayed as black or other colored pixel blocks in the pure white screen, and as white or other colored pixel blocks in the pure blue screen.
[0067] Step 2: When the screen displays a full white image and a full blue image, control the industrial camera to capture a frame of the screen image to obtain the white field detection image and the blue field detection image.
[0068] Step 3: Perform image processing on the white-point detection image and the blue-point detection image respectively to identify bad pixels. The image processing includes edge feature extraction and bad pixel identification and classification, as follows: The edge feature extraction employs the DoG (Difference of Gaussians) operator. The DoG operator enhances edge and blob features in the image by using Gaussian kernels with two different standard deviations, effectively filtering uniform backgrounds, suppressing noise, and enhancing the contrast of minor defects. Its principle is as follows: Representing the image matrix, the Gaussian function can be expressed as: (5) Use a convolutional kernel with a width of Gaussian kernel function Convolution is performed to obtain Obtain the Gaussian difference image ,Right now: (6) From equation (5), we can see that Images processed by Gaussian difference filtering. Now, define the Gaussian difference filter. Then equation (6) becomes: (7) Substituting the Gaussian function into equation (7) yields the DOG operator.
[0069] In its specific implementation, the algorithm flow of this embodiment includes: 1. For the input image matrix Perform a Fourier transform to obtain the frequency domain image. .
[0070] 2. In the frequency domain, based on the preset spatial Gaussian standard deviation parameter... and (For example, =5, =1), construct the DoG operator .
[0071] 3. Calculation .
[0072] 4. To Perform an inverse Fourier transform to obtain the enhanced binary feature map. .
[0073] The defect identification and classification uses the YOLOv8 object detection model. The binary feature map obtained in step 3... The data is input into a pre-trained YOLOv8 model. This model can identify abnormal regions in the feature map and classify them into different types of bad points (such as bright spots, dark spots, colored spots, etc.), while outputting the location coordinates and confidence scores of the bad points.
[0074] Step 4: Based on the recognition results of the YOLOv8 model, determine the status of dead pixels on the screen. If dead pixels are detected, record the type, location, and number of dead pixels, and mark the screen as defective; if no dead pixels are detected, the screen dead pixel detection is considered passed.
[0075] The defect detection method provided in this embodiment realizes automated and high-precision detection of screen defects, improving detection efficiency and reliability.
[0076] Embodiment 4 of the present invention provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements an automatic detection method for liquid crystal display boards based on machine vision as described in Embodiment 2.
[0077] Embodiment 5 of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements an automatic detection method for liquid crystal boards based on machine vision as described in Embodiment 2.
[0078] In Embodiment 6, this invention provides an application example of the automatic inspection process of an LCD board based on machine vision, which is based on the automatic inspection method for LCD boards based on machine vision described in Embodiment 2, and includes: Preparation stage: Place the LCD screen face down in the groove of the same size as the LCD screen on the top of the test fixture, connect the power supply and camera, and open the test software.
[0079] Button testing and screen switching: The testing software controls the finger cylinder 5 to click the button on the LCD panel, thereby controlling the LCD screen to switch the displayed screen, realizing button testing and software version testing.
[0080] Image acquisition: After the finger cylinder 5 is clicked, the software drives the industrial camera 2 to take a picture once to obtain the image of the current display screen on the LCD board.
[0081] Image processing and analysis: The detection software processes and analyzes the captured images to determine whether the LCD board has defects or malfunctions.
[0082] Test complete: Once all test items are completed, the LCD screen test is complete.
[0083] Furthermore, this embodiment provides the hardware components of the detection system, the hardware of which includes, for example,... Figure 1 As shown, it mainly consists of the following components: Testing station 1: Used to place LCD panels and provide a stable support platform for testing.
[0084] Industrial camera 2: Installed directly above the LCD board, used to capture the LCD board display and acquire image data.
[0085] Lens 3: A fixed-focus lens 3 is used to ensure the clarity and stability of the captured images.
[0086] Camera bracket 4: Secures the industrial camera 2, keeping it in a suitable shooting position.
[0087] Finger cylinder 5: Its movement is triggered by the testing software to produce an effect similar to a finger pressing a button, which is used to control the switching of images on the LCD screen.
[0088] Circuit equipment: including power supplies, connectors, solenoid valves, etc., used to connect and control various hardware components to realize electrical functions.
[0089] The test fixture structure is labeled as follows: ① Inspection table, ② Industrial camera, ③ Lens, ④ Camera bracket, ⑤ Finger cylinder, ⑥ Network relay. The structure of the test fixture is as follows: Figure 1 As shown, its upper end has a groove of the same size as the LCD screen. The LCD screen is placed face down in the groove. This design ensures the accuracy and stability of the LCD board placement, making it convenient for the industrial camera 2 to take pictures and inspect.
[0090] This testing system, by introducing a finger-cylinder 5-button mechanism, can not only detect basic performance characteristics of LCD screens such as dead pixels and brightness, but also detect the software version and hardware I / O performance of the LCD, thus overcoming the shortcomings of traditional LCD screen testing methods. Furthermore, the entire testing process is automatically controlled by the system, improving the degree of automation and accuracy of the testing.
[0091] It should be noted that the machine vision inspection system provided by this invention, through automated image acquisition and algorithm analysis, reduces the single inspection cycle from an average of 30 seconds to less than 20 seconds. When used in conjunction with production line operations, a single device can inspect an average of 800-1000 pieces per day, which is about 50% more efficient than manual inspection and significantly reduces the inspection bottleneck in large-scale production.
[0092] The technical effects of the present invention also include: 1. Improved detection accuracy and stability: Manual inspection is susceptible to visual fatigue and ambient lighting interference, resulting in a high rate of missed detections of minute defects, and its accuracy decreases after continuous operation. The system of this invention uses a fixed-focus lens and optimized algorithms (such as the DOG operator) to accurately identify pixel-level defects, reducing the error rate by approximately 20% compared to manual inspection, and maintaining a stable inspection level during long-term continuous operation, unaffected by changes in working hours or ambient lighting.
[0093] 2. Achieve fully automated, multi-dimensional integrated detection: The system can seamlessly integrate with the production line's PLC and MES systems, enabling fully automated operation from loading, positioning, and detection to sorting. By introducing finger cylinders to simulate button presses, the system can not only detect physical defects on the screen (such as dead pixels and brightness issues), but also verify functional logic such as software version compatibility and hardware I / O response speed, expanding the detection dimensions from a single visual inspection to a comprehensive evaluation of "hardware performance + software logic".
[0094] 3. Brings economic benefits and quality control value: Taking a production line with an annual output of 500,000 boards as an example, adopting this system can significantly reduce the number of inspection personnel, saving 500,000 to 600,000 yuan in annual labor costs. Furthermore, the system can be put into operation within a week of deployment, avoiding the months-long manual training period. The system automatically records complete inspection data and defect information for each board, forming a traceable quality database. Through data analysis, process problems can be located, helping to improve the overall yield by 3%-5% and reducing annual losses from defective products by approximately 800,000 yuan.
[0095] 4. Strong environmental adaptability, convenient maintenance, and high reliability: Employing industrial-grade hardware and a closed design, the system operates stably under various lighting conditions, overcoming the dependence of manual inspection on ambient light. The hardware utilizes a modular design, supporting rapid and independent replacement of key components (such as cameras and cylinders). With a mean time to repair (MTTR) of less than 30 minutes, it ensures long-term continuous operation and reduces maintenance costs by approximately 35% compared to traditional equipment. This system is a front-end and back-end system developed based on MFC (Microsoft Foundation Classes) and the C++ language. MFC provides rich class libraries and frameworks, enabling developers to easily and quickly create Windows applications, while C++, as a powerful programming language, provides the system with high performance and flexible programming capabilities.
[0096] The front-end utilizes MFC's visual design tools to create an intuitive and user-friendly interface. This includes various windows, menus, buttons, text boxes, and other controls to facilitate user interaction with the system. For example, the main window has a logical layout, clearly dividing different functional areas so users can quickly find the desired operation. The MFC software also handles cylinder control and photo triggering. Operators can configure templates to set up test items. After configuration, the front-end controls the cylinder to click the button, then sends a TCP message to the back-end to take a picture and perform functional testing. The front-end and back-end collaborate via TCP message communication to complete this testing process.
[0097] The backend is a detection system developed in C++ using OpenCV and ONNX algorithm libraries for image processing. The image processing primarily utilizes OpenCV version 4.8.0. OpenCV is a cross-platform computer vision library developed by Intel and continuously optimized by the open-source community, providing over 2500 optimized algorithms covering image processing, pattern recognition, and machine learning. Given OpenCV's powerful image processing capabilities, the image processing in this system is mainly handled by the OpenCV API. MFC triggers image capture via TCP commands, sending the captured image from the Hikvision camera to OpenCV for processing. The test results are then converted into TCP packet format and sent back to MFC. The frontend and backend work together to complete this detection software. The software of this detection system includes four modules: image acquisition module, LCD screen extraction module, detection module, and result output and storage module.
[0098] The software control flow of the detection system is as follows: Figure 2 As shown, its working process is as follows: After the testing software starts and completes system initialization, it enters a loop testing process. This process first determines whether the preset number of testing tasks has been completed (e.g., all four tests have been completed): If not completed, wait for and execute the instructions sent by the host computer, drive the camera to take pictures, detect and analyze the images in sequence, update the loop state according to the single detection result, and then proceed to the next round of judgment; If completed, all test results are summarized and a final report is output, and the process ends.
[0099] This loop control logic enables automated and serialized testing of multiple functions of the LCD board.
[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the protection scope of the claims of the present invention.
Claims
1. An automatic inspection system for LCD circuit boards based on machine vision, characterized in that, include: The system includes positioning fixtures, an image acquisition module, a key simulation module, an electrical control module, and an industrial computer, among which: The positioning fixture includes a testing table (1), which has a groove at its upper end that matches the shape of the LCD screen to be tested. The groove is used to place the LCD board card upside down and make the screen face downward. The image acquisition module is located above the positioning fixture and is used to acquire screen images of the LCD board. The button simulation module is set on the positioning fixture and is used to simulate the physical button actions on the LCD board. The electrical control module is communicatively connected to the button simulation module to drive it to perform button actions; The industrial computer is communicatively connected to the electrical control module and the image acquisition module, respectively, for executing: The electrical control module drives the button triggering device to execute button actions. Before and after the button press, the control image acquisition module acquires the screen image of the LCD board respectively; By analyzing the captured screen images and considering the changes in the screen display before and after button presses, the effectiveness of the button functions and the defect status of the screen can be determined.
2. The automatic inspection system for LCD boards based on machine vision according to claim 1, characterized in that: The button simulation module includes a finger cylinder (5) and a slide. The finger cylinder (5) is positioned above the slide. After the board is placed on the slide, it is slid to the designated position so that the finger cylinder (5) is positioned directly above the button.
3. The automatic inspection system for LCD boards based on machine vision according to claim 2, characterized in that: The electrical control module includes a network relay (6) and a solenoid valve. The solenoid valve is connected to the air circuit of the finger cylinder (5) and controls the opening and closing of the air circuit through the network relay (6).
4. A machine vision-based automatic inspection method for LCD circuit boards, based on the machine vision-based automatic inspection system for LCD circuit boards according to any one of claims 1-3, characterized in that, include: Fix the LCD panel to be tested with the screen facing down onto the positioning fixture; The initial screen image of the LCD board is acquired through the image acquisition module; The control button simulation module clicks the physical buttons on the LCD panel. After clicking the button, the screen image of the LCD board is captured again by the image acquisition module after the trigger. Based on the initial screen image and the screen image after triggering, the changes in the screen image are analyzed to determine the effectiveness of the button function and the defect status of the screen.
5. The automatic detection method for LCD boards based on machine vision according to claim 4, characterized in that: Determining the validity of button functions includes: The initial screen image and the screen image after triggering are respectively input into the deep learning model to extract the corresponding first feature vector and second feature vector; Calculate the Euclidean distance between the first eigenvector and the second eigenvector; The Euclidean distance is compared with a preset threshold, and the screen image is judged based on the comparison result to determine whether the button function is normal.
6. The automatic detection method for LCD boards based on machine vision according to claim 5, characterized in that: The deep learning model is a neural network based on the ResNet architecture, and its deep residual blocks contain deformable convolutional layers.
7. The automatic detection method for LCD boards based on machine vision according to claim 4, characterized in that: After acquiring the initial screen image, the process also includes segmenting the screen region: Convert the acquired color image to a grayscale image; Based on the brightness distribution characteristics of grayscale images, a dynamic threshold segmentation method is used to segment the display area of the LCD screen and generate an image of the screen area to be analyzed.
8. The automatic detection method for LCD boards based on machine vision according to claim 7, characterized in that: The display area of the LCD screen is segmented using a dynamic threshold segmentation method, including: Obtain the maximum and minimum pixel grayscale values of the screen region image to be analyzed; Calculate the dynamic segmentation threshold based on the maximum and minimum values; Binarization of grayscale images is performed using a dynamic segmentation threshold. Edge extraction is performed on the binarized image, and the edge contour of the LCD screen is selected based on the fixed area attribute of the LCD screen. The fitted edge contour is a rectangle, and this rectangular region is cropped as the screen region image to be analyzed.
9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is loaded into the processor, it implements an automatic detection method for LCD boards based on machine vision according to any one of claims 4-8.
10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements an automatic detection method for LCD boards based on machine vision according to any one of claims 4-8.