LED display screen lamp bead missing automatic detection system based on machine vision
By combining machine vision with multi-physics sensors and deep learning technology, high-precision, multi-dimensional inspection of LED display chips has been achieved. This solves the problems of low efficiency and difficulty in identifying internal defects in traditional inspection methods, thereby improving the accuracy of inspection and the reliability of the production line.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- ANHUI NINGCAI NEW DISPLAY DEVICE CO LTD
- Filing Date
- 2026-02-10
- Publication Date
- 2026-05-05
AI Technical Summary
Existing automatic detection systems for missing LED display chips are inefficient, labor-intensive, susceptible to subjective human factors, and struggle to detect potential welding or structural defects inside the chips, making it difficult to make effective judgments.
An automatic detection system for missing LED beads based on machine vision is adopted. It uses terahertz scanning imaging, laser-induced ultrasound and hyperspectral microscopic imaging sensors for multi-dimensional detection. Combined with a deep learning multi-source information fusion network and expert rule base, it can achieve non-contact and high-precision identification of the deep physical state and internal defects of the beads.
It enables high-precision detection of the deep physical state and internal defects of LED beads under non-powered conditions, significantly improving the defect detection rate and classification accuracy. It solves the problems of traditional detection methods being susceptible to environmental interference and unable to detect internal defects, and enhances the compatibility and reliability of the production line.
Smart Images
Figure CN121978129A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of LED display technology, and in particular to an automatic detection system for missing LED display beads based on machine vision. Background Technology
[0002] LED displays are widely used in advertising, information dissemination, stage performances, and command and control, among other fields. Their display quality and reliability directly impact the viewing experience and user experience. An LED display is composed of tens of thousands, or even millions, of tiny LED beads (pixels) arranged in a matrix. During production, transportation, or long-term use, individual beads may experience defects such as complete absence, physical damage, internal solder joint defects, or broken gold wires. These defects can lead to persistent dark spots, color shifts, or affect the overall image uniformity, and in severe cases, even cause localized or large-scale failures. Therefore, rapid and accurate defect detection of LED beads before shipment or during maintenance is a crucial step in ensuring product quality.
[0003] Traditional defect detection mainly relies on manual visual inspection, which is inefficient, labor-intensive, easily affected by subjective factors, and difficult to detect potential welding or structural defects inside the LED beads. It can no longer meet the requirements of efficiency and consistency for modern large-scale production.
[0004] To improve the level of automation in inspection, machine vision-based automatic inspection technology is gradually being applied. A common approach is to transport the display screen to the inspection station via a conveyor line, power it on (e.g., displaying a full white field or a specific test pattern), use an industrial camera to capture the displayed image, and then use image processing algorithms (e.g., comparison with standard templates, brightness / color analysis) to identify non-lit or abnormal LEDs. While this method achieves a degree of automation, it has significant limitations: First, it requires powering the display screen, increasing the complexity of electrical interface connections and the risk of poor contact, and it cannot be used when the screen cannot be lit (e.g., due to power module failure). Second, the inspection results are highly susceptible to interference from ambient light, screen surface reflections, and different display content, resulting in a high false positive rate. Third, it can only determine whether an LED is "lit" or "not lit," and it struggles to effectively identify "sub-healthy" states where electrical connections are unstable but not completely failed due to poor internal soldering, chip micro-cracks, or broken gold wires, as well as situations where the LED is physically present but its internal structure is damaged.
[0005] In recent years, some improved technologies have emerged, such as using infrared thermal imaging to detect the heating of LED chips after power-on. While this method can provide some internal information, it still requires power-on excitation, and its thermal response is greatly affected by heat dissipation conditions, making it insensitive to minute thermal differences. Furthermore, some research has attempted to use non-contact flaw detection techniques (such as ultrasound) for electronic component inspection. However, in scenarios involving rapid, high-precision, and online inspection of LED display chips, a mature and effective systematic solution for achieving simultaneous and reliable identification of multiple types of defects still lacks a mature and effective solution. Summary of the Invention
[0006] To address the technical problems of existing automatic detection systems for missing LED display beads being inefficient, labor-intensive, susceptible to subjective human factors, and unable to detect potential welding or structural defects inside the beads, or situations where the beads are physically present but their internal structure is damaged, making it difficult to make effective judgments, this invention proposes an automatic detection system for missing LED display beads based on machine vision.
[0007] The present invention proposes an automatic detection system for missing LED beads in an LED display screen based on machine vision, including a conveyor line, wherein a detection component is provided on the outer surface of the frame of the conveyor line, the detection component including a detection camera, and the detection camera detects missing LED beads in the LED display screen;
[0008] A clamping component is fixedly installed on the outer surface of the frame of the conveyor line. The clamping component includes a driving device and a fixing device. The driving device includes a lifting frame. The lifting of the lifting frame drives the fixing device to lift. The fixing device includes a contact airbag. The contact airbag fixes the LED display screen that needs to be tested.
[0009] It also includes a detection system, which comprises a multi-physics sensing unit, an intelligent analysis and decision unit, an actuator and feedback unit, and a system monitoring and early warning unit.
[0010] Preferably, the detection component further includes a detection frame, which is fixedly installed on the upper surface of the frame of the conveyor line. A moving component is fixedly installed on the inner wall of the detection frame. The lower surface of the slider of the moving component is fixedly installed with the outer surface of the detection camera. An illumination component is fixedly installed on the outer surface of the slider.
[0011] Preferably, the driving device further includes a lifting hydraulic cylinder, which is fixedly installed on the outer surface of the frame of the conveyor line. A push frame is fixedly installed at one end of the piston rod of the lifting hydraulic cylinder. The outer surface of the limiting rod of the push frame is slidably inserted into the inner wall of the lifting frame. A tension spring is fixedly installed on the outer surface of the limiting rod of the push frame. One end of the tension spring is fixedly installed on the outer surface of the lifting frame. A limit plate is fixedly installed on the outer surface of the frame of the conveyor line. The lower surface of the limit plate contacts the upper surface of the lifting frame.
[0012] Preferably, a limiting guide rail is fixedly installed on the upper surface of the lifting frame, a movable frame is slidably inserted into the upper surface of the limiting guide rail, a limiting ball is rotatably connected to the lower surface of the movable frame through a bearing, a guide groove plate is fixedly installed on the outer surface of the lifting frame, and the outer surface of the limiting ball is slidably connected to the inner wall of the guide groove plate.
[0013] Preferably, the fixing device further includes an air pump, which is fixedly installed on the outer surface of the frame of the conveyor line. A filter is fixedly installed on the outer surface of the frame of the conveyor line. The outlet end of the filter is fixedly connected to the inlet end of the air pump. The outlet end of the air pump is fixedly connected to a diversion pipe with a control valve. An outlet air pump is fixedly installed on the outer surface of the frame of the conveyor line. The outer surface of the diversion pipe is fixedly connected to the inlet end of the outlet air pump through a three-way valve.
[0014] Preferably, the outer surfaces of the plurality of contact airbags are fixedly installed with the outer surfaces of the movable frame and the lifting frame, and a telescopic airbag assembly is fixedly installed on the inner wall of the contact airbag. The plurality of telescopic airbag assemblies are fixedly connected to each other through connecting hoses. A delivery hose is fixedly connected to the outer surface of the telescopic airbag assembly. One end of the delivery hose is fixedly connected to one end of the diversion pipe. The two sets of delivery hoses are located inside the lifting frame and the pushing frame, respectively.
[0015] Preferably, the multi-physics sensing unit includes a multi-physics sensor array and a signal fusion and feature extraction module;
[0016] The multiphysics sensor array includes a terahertz scanning imaging sensor, a laser-induced ultrasound sensor, and a hyperspectral microscopic imaging sensor, which collects deep information from the screen from three physical domains: electromagnetic waves, mechanical vibration, and material spectra.
[0017] The transceiver probe array of the terahertz scanning imaging sensor is mounted on the detection frame to perform area array scanning on the LED display screen that is stationary at the detection station, and to obtain chip structure information below the encapsulation layer.
[0018] The pulsed laser emitter and laser interferometer of the laser-induced ultrasonic sensor are integrated on the moving component, which moves with the component and aligns with each lamp bead unit to excite and detect its micro-vibration spectrum.
[0019] The hyperspectral microscopic imaging sensor is integrated into the detection camera position to acquire the microscopic reflectance spectral characteristics of each LED bead region in a wide spectral range.
[0020] The signal fusion and feature extraction module is used to synchronize, filter, and reduce noise in the raw data collected by the three sensors, and extract key features, including terahertz reflection intensity map, ultrasonic feature spectrum, and material reflection spectrum, to provide a stable and multi-dimensional fusion data source for subsequent decision-making.
[0021] Preferably, the intelligent analysis and decision unit includes a core analysis module and a multimodal defect decision algorithm module;
[0022] The core analysis module receives fused feature data from the multi-physics sensing unit and executes analysis logic;
[0023] The multimodal defect judgment algorithm module has a built-in deep learning-based multi-source information fusion network and expert rule base. It is used to calculate the physical existence confidence and internal health score of each LED position based on the fused multi-physics field feature data. By comparing with the preset defect feature mapping model, it dynamically generates defect type judgment results, including complete absence, chip breakage, internal cold solder joint, and gold wire breakage, so as to achieve accurate identification and classification of defects.
[0024] The multimodal defect decision algorithm module achieves accurate defect identification and classification through the following mathematical process:
[0025] Multi-source confidence fusion calculation: First, calculate the independent confidence level based on the signals from each sensor: ,in, For sensor type index, For sensors The raw signal feature values were collected and preprocessed. , , These are, respectively, terahertz reflection intensity, ultrasonic vibration amplitude, and hyperspectral characteristic values. For the corresponding sensor The decision threshold For sensors The sensitivity coefficient, It is a natural constant. Based on sensors Independent confidence level for signal calculation;
[0026] Then, a sensor reliability factor is introduced. ,in, For the sensor's operating temperature, and Its optimal operating temperature and stability parameters. For sensors Real-time reliability weighting factor;
[0027] Finally, the overall presence confidence of the LED chips is calculated: ,in, For sensors The dynamic weighting coefficients in the fusion process satisfy... Its value can be adaptively adjusted based on the sensor signal quality and historical performance.
[0028] Defect pattern classification: Constructing feature vectors ;
[0029] in, The signal difference is denoted as ; where, A comprehensive feature vector used for defect classification. For sensors and Signal difference between them To represent the transpose of a vector, The independent confidence level of the terahertz sensor. The independent confidence level of the laser ultrasonic sensor. The independent confidence level of the hyperspectral sensor. The difference between terahertz and laser ultrasound signals. The difference between laser ultrasound and hyperspectral signals. The difference between hyperspectral and terahertz signals, For sensors Standardized eigenvalues;
[0030] Will Input a multi-classifier based on the Softmax function to calculate the defect category. The probability of:
[0031] ,in, Given a feature vector The tested LED chip belongs to the defect category. The conditional probability, For category indexing, To indicate the first Class defects, Sum index for defect categories, and For category The corresponding weight vector and bias term, The total number of defect categories preset by the system. for The row vector form, for The row vector form, For the first The weight vector of class defects, For the first Class bias term;
[0032] Parameter adaptive adjustment: Based on feedback from historical decision results, the loss function is minimized. The decision threshold is dynamically optimized using the gradient descent method. and weighting coefficients , For the first A historical sample for category The true label, The number of historical samples used for parameter optimization. This is the sample index in the historical sample batch.
[0033] Preferably, the actuator and feedback unit includes a report generation and tag triggering module and a system self-test and status feedback module;
[0034] The report generation and marking trigger module is used to receive defect coordinates and type information from the intelligent analysis and judgment unit, automatically generate a structured inspection report, drive external marking devices to mark the location of screen defects, or send sorting instructions to the production line control system.
[0035] The system self-test and status feedback module is used to automatically calibrate the reference parameters of the multi-physics sensor before the detection begins or during idle periods, and to provide real-time feedback on the working status of each sensor, moving component, and lighting unit to the monitoring center.
[0036] Preferably, the system monitoring and early warning unit includes a comprehensive fault diagnosis module and a graded early warning and indication module;
[0037] The integrated fault diagnosis module is used to monitor system-level or process-level faults such as abnormal sensor signals, interrupted data streams, low confidence levels in analysis algorithms, and actuator response timeouts, and generates diagnostic logs containing fault levels and location information.
[0038] The comprehensive fault diagnosis module calculates a decision uncertainty metric. To monitor the consistency between sensors, when A self-check is triggered when the threshold is exceeded. The average confidence level;
[0039] The graded early warning and indication module is used to trigger graded alarms through prominent color blocks and sounds on the local human-machine interface (HMI) and the connected production line central control console when a system fault is diagnosed, the screen defect rate is detected to exceed the process threshold, or a high-risk defect type is identified, and to suspend the conveyor line for further processing.
[0040] The beneficial effects of this invention are as follows:
[0041] 1. By setting up a driving component, this clamping component achieves flexible adaptive fixation and non-destructive clamping of the LED display screen. Specifically, the lifting frame, driven by the driving device, brings the contact airbags into contact with the screen body. By filling them with filtered clean gas, multiple contact airbags can adaptively conform to the side of the display screen, evenly distributing the pressure. This effectively avoids the micro-cracks caused by screen crushing, scratching, or stress concentration that may be caused by traditional rigid clamps. At the same time, the unique telescopic airbag group can buffer the vibration and impact during the conveying and positioning process. While providing sufficient fixing force, it maximizes the protection of the precision display screen, creating stable and non-damaging working conditions for subsequent high-precision testing. This improves the compatibility and reliability of the production line and ensures product yield.
[0042] 2. By setting up a detection system, this system achieves non-contact, high-precision, and multi-dimensional intelligent detection of the deep physical state and internal defects of LED beads under non-powered conditions. Specifically, the system uses three complementary sensing technologies—terahertz scanning, laser-induced ultrasound, and hyperspectral microscopy—to directly extract information from electromagnetic wave reflection, mechanical vibration response, and material spectral characteristics, determining the physical presence of the LED beads without turning them on. Furthermore, through an embedded fusion confidence calculation model and defect pattern classifier, the system can effectively distinguish various defect types that traditional vision cannot identify, such as "complete absence," "chip breakage," "internal cold solder joint," and "gold wire breakage," and continuously optimizes the system with a parameter adaptive mechanism. This fundamentally overcomes the limitations of traditional power-on testing methods, which are susceptible to environmental interference and unable to detect internal defects. It elevates the detection dimension from surface light signals to the material and structural level, significantly improving the defect detection rate, classification accuracy, and robustness of production line testing. At the same time, it provides key data support for product reliability assessment and predictive maintenance. It solves the technical problems of existing automatic detection systems for missing LED display beads, such as low efficiency, high labor intensity, susceptibility to subjective human factors, difficulty in detecting potential welding or structural defects inside the beads, and difficulty in making effective judgments when the beads are physically present but their internal structure is damaged. Attached Figure Description
[0043] Figure 1 This is a schematic diagram of an automatic detection system for missing LED beads in an LED display screen based on machine vision, as proposed in this invention.
[0044] Figure 2 This is a perspective view of the detection camera structure of an automatic detection system for missing LED beads in an LED display screen based on machine vision, as proposed in this invention.
[0045] Figure 3 This is a perspective view of the filter structure of an automatic detection system for missing LED beads in an LED display screen based on machine vision, as proposed in this invention.
[0046] Figure 4 This is a perspective view of the air pump structure of an automatic detection system for missing LED beads in an LED display screen based on machine vision, as proposed in this invention.
[0047] Figure 5 This is a perspective view of the lifting hydraulic cylinder structure of an automatic detection system for missing LED beads in an LED display screen based on machine vision, as proposed in this invention.
[0048] Figure 6 This is a perspective view of the pusher frame structure of an automatic detection system for missing LED beads in an LED display screen based on machine vision, as proposed in this invention.
[0049] Figure 7 This is a three-dimensional view of the contact airbag structure of an automatic detection system for missing LED display beads based on machine vision proposed in this invention.
[0050] Figure 8 This is a perspective view of the telescopic airbag assembly structure of an automatic detection system for missing LED display beads based on machine vision proposed in this invention.
[0051] Figure 9 This is a block diagram of an automatic detection system for missing LED display beads based on machine vision, as proposed in this invention.
[0052] In the diagram: 1. Conveyor line; 2. Detection frame; 21. Moving component; 22. Detection camera; 23. Lighting component; 3. Lifting hydraulic cylinder; 31. Push frame; 32. Lifting frame; 33. Tension spring; 34. Limiting plate; 4. Limiting guide rail; 41. Moving frame; 42. Limiting ball; 43. Guide groove plate; 5. Air pump; 51. Filter; 52. Diversion pipe; 53. Air pump; 54. Contact airbag; 55. Telescopic airbag assembly; 56. Conveying hose. Detailed Implementation
[0053] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0054] Reference Figures 1-9An automatic detection system for missing LED display beads based on machine vision includes a conveyor line 1, which carries and sequentially transports the LED display to the detection station.
[0055] like Figure 2 As shown, a detection component is provided on the outer surface of the frame of the conveyor line 1. The detection component includes a detection camera 22, which integrates a hyperspectral microscopic imaging sensor. The detection camera 22 is not a single visible light imaging device, but an integrated sensing unit. Its core function is to obtain key information about the physical existence and internal health status of the LED without relying on the active light emission of the LED beads, thereby overcoming the limitation of traditional detection that must be powered on.
[0056] Specifically, the detection component also includes a detection frame 2, which can be a gantry or arch structure, providing a stable, closed or semi-closed space for detection that is protected from external light interference. The detection frame 2 is fixedly installed on the upper surface of the frame of the conveyor line 1. A moving component 21 is fixedly installed on the inner wall of the detection frame 2. The moving component 21 is usually a high-precision two-dimensional linear module (XY axis) driven by a servo motor and a ball screw (or synchronous belt). The slider is the actuator of the moving component 21, driving the integrated detection camera 22 and illumination component 23 to perform programmed, full-coverage scanning motion above the display screen. The lower surface of the slider of the moving component 21 is fixedly installed with the outer surface of the detection camera 22. The illumination component 23 is fixedly installed on the outer surface of the slider. The illumination component 23 provides controllable and uniform illumination with adjustable brightness and color temperature to optimize the imaging conditions of different sensors (especially hyperspectral imaging), rather than to light up the LED screen. This enables the detection camera 22 to accurately locate and repeatedly scan any position on the display screen surface, ensuring the comprehensiveness and consistency of data acquisition. Controllable illumination provides a guarantee for high-quality imaging and further improves the stability of the detection signal.
[0057] like Figures 3-8 As shown, in order to clamp the LED screen without damage, a clamping component is fixedly installed on the outer surface of the frame of the conveyor line 1. The clamping component includes a driving device and a fixing device. The driving device includes a lifting frame 32. The lifting of the lifting frame 32 drives the fixing device to lift. The fixing device includes a contact airbag 54. The contact airbag 54 fixes the LED display screen that needs to be tested.
[0058] Specifically, the drive device also includes a lifting hydraulic cylinder 3, which is fixedly installed on the outer surface of the frame of the conveyor line 1. The lifting hydraulic cylinder 3 serves as the main power source, providing a stable and powerful linear thrust. A push frame 31 is fixedly installed at one end of the piston rod of the lifting hydraulic cylinder 3. The outer surface of the limiting rod of the push frame 31 is slidably inserted into the inner wall of the lifting frame 32. The sliding insertion of the limiting rod and the lifting frame 32 forms a precision guide pair, ensuring no deflection during the lifting process. To allow the movement of the push frame 31 to drive the movement of the lifting frame 32, the limiting rod of the push frame 31... A tension spring 33 is fixedly installed on the outer surface of the position rod. One end of the tension spring 33 is fixedly installed on the outer surface of the lifting frame 32. The push frame 31 drives the lifting frame 32 to rise and fall through the tension spring 33. A limit plate 34 is fixedly installed on the outer surface of the frame of the conveyor line 1. The lower surface of the limit plate 34 contacts the upper surface of the lifting frame 32. The limit plate 34 limits the lifting frame 32. After the lifting frame 32 reaches the set position, its upper surface contacts the lower surface of the limit plate 34, and the rise of the push frame 31 cannot drive the rise of the lifting frame 32.
[0059] Specifically, to facilitate clamping the other two sides of the raised LED screen, a limiting guide rail 4 is fixedly installed on the upper surface of the lifting frame 32. A movable frame 41 is slidably inserted into the upper surface of the limiting guide rail 4. A guide assembly is fixedly installed on the outer surface of the movable frame 41. The guide assembly includes a connecting rod installed on the movable frame 41. The connecting rod is slidably inserted into the push frame 31. The connecting rod and the push frame 31 are connected by a spring. A limiting ball 42 is rotatably connected to the lower surface of the movable frame 41 through a bearing. A guide groove plate 43 is fixedly installed on the outer surface of the lifting frame 32. The outer surface of the limiting ball 42 is slidably connected to the inner wall of the guide groove plate 43. The limiting ball 42 slides into the groove in the guide groove plate 43. When the limiting ball 42 is restricted, it will push the movable frame 41 to move on the limiting guide rail 4, so that the fixing device can move out from under the LED screen, thereby achieving clamping of the sides of the LED screen.
[0060] Specifically, the fixed device also includes an air pump 5, which is fixedly installed on the outer surface of the frame of the conveyor line 1. A filter 51 is fixedly installed on the outer surface of the frame of the conveyor line 1. The outlet end of the filter 51 is fixedly connected to the inlet end of the air pump 5. The filter 51 ensures that the air being filled is clean and dust-free, preventing blockage. The outlet end of the air pump 5 is fixedly connected to a diversion pipe 52 with a control valve. The diversion pipe 52 with the control valve is used to precisely adjust the inflation speed and final pressure, and can independently control multiple sets of telescopic airbags 55. An exhaust air pump 53 is fixedly installed on the outer surface of the frame of the conveyor line 1. The outer surface of the diversion pipe 52 is fixedly connected to the inlet end of the exhaust air pump 53 through a three-way valve. The exhaust air pump 53 and the three-way valve constitute an active rapid deflation circuit, which can quickly extract the air from the airbag after the test, causing it to quickly retract and detach from the screen, greatly improving the cycle time compared to natural deflation.
[0061] Specifically, the outer surfaces of multiple contact airbags 54 are fixedly installed with the outer surfaces of the moving frame 41 and the lifting frame 32. Telescopic airbag assemblies 55 are fixedly installed on the inner walls of the contact airbags 54. These telescopic airbag assemblies 55 are connected by connecting hoses. Each telescopic airbag assembly 55 is an independent small cavity inside the main airbag, and they are interconnected via connecting hoses. This design allows internal air to flow to other areas through the connecting hoses when an airbag is locally compressed and deformed, thereby achieving dynamic pressure balance across the entire airbag contact surface and preventing "hard spots" with excessive local pressure. A delivery hose 56 connects the air source to the entire airbag network. A delivery hose 56 is fixedly connected to the outer surface of each telescopic airbag assembly 55. One end of the delivery hose 56 is fixedly connected to one end of the diversion pipe 52. The two sets of delivery hoses 56 are located inside the lifting frame 32 and the pushing frame 31, respectively.
[0062] like Figure 9 As shown, it also includes a detection system, which comprises a multi-physics sensing unit, an intelligent analysis and decision unit, an actuator and feedback unit, and a system monitoring and early warning unit.
[0063] The detection system is a software and intelligent control core integrated on top of the hardware. The multiphysics sensing unit is responsible for collecting raw data from different physical dimensions; the intelligent analysis and decision unit is the "brain," responsible for in-depth processing and intelligent judgment of the fused data; the actuator and feedback unit transforms the decision results into control commands and operable reports; and the system monitoring and early warning unit ensures the reliability and stability of the entire system. These four units work together to form a complete "perception-decision-execution-monitoring" closed loop.
[0064] Specifically, the multi-physics sensing unit includes a multi-physics sensor array and a signal fusion and feature extraction module.
[0065] Once the LED display screen is stably fixed in the testing station by the clamping components, the multi-physics sensor array begins to work collaboratively.
[0066] The multi-physics sensor array includes a terahertz scanning imaging sensor, a laser-induced ultrasound sensor, and a hyperspectral microscopic imaging sensor, which collects deep information from the screen from three physical domains: electromagnetic waves, mechanical vibration, and material spectra.
[0067] The transceiver array of the terahertz scanning imaging sensor is mounted on the detection frame 2. The transceiver array performs a rapid area scan of the stationary display screen from above the detection frame 2. Terahertz waves penetrate the surface encapsulation material of the LED chips and are reflected by the internal semiconductor chip and metal structure, forming a three-dimensional structural image (signal) reflecting the physical existence of the chip and its internal connection state. );
[0068] The pulsed laser emitter and laser interferometer of the laser-induced ultrasonic sensor are integrated on the moving component 21. The moving component 21 drives its pulsed laser emitter to move precisely above each LED unit, emitting high-frequency laser pulses to induce ultrasonic waves on the LED surface. The integrated laser interferometer synchronously detects the microscopic vibration response on the LED surface, obtaining its vibration spectrum characteristics (signal). );
[0069] A hyperspectral microscopic imaging sensor is integrated into the detection camera 22. Under the positioning of the moving component 21, it performs microscopic imaging on each LED chip area and acquires reflectance spectra over a broad spectral range from visible light to short-wave infrared, obtaining spectral "fingerprint" characteristics (signals) characterizing the LED chip, solder joints, and packaging materials. ).
[0070] The signal fusion and feature extraction module performs synchronous acquisition. , , The three raw data streams were time-domain aligned, filtered and denoised, and key features, such as terahertz reflection intensity maps, ultrasonic characteristic frequency amplitudes, and spectral reflectance of specific bands, were extracted to form a multi-dimensional fusion data package for subsequent analysis.
[0071] Specifically, the intelligent analysis and decision unit includes a core analysis module and a multimodal defect decision algorithm module;
[0072] The core analysis module receives fused feature data from the multi-physics sensing unit and executes analysis logic;
[0073] The multimodal defect judgment algorithm module has a built-in deep learning-based multi-source information fusion network and expert rule base. It is used to calculate the physical existence confidence and internal health score of each LED bead position based on the fused multi-physics field feature data. By comparing with the preset defect feature mapping model, it dynamically generates defect type judgment results, including complete missing, chip crack, internal poor soldering, and gold wire breakage, so as to achieve accurate identification and classification of defects.
[0074] After receiving the fused feature data, the multimodal defect decision algorithm module of this unit performs calculations and decisions according to the following steps:
[0075] Multi-source confidence fusion calculation: First, calculate the independent confidence level based on the signals from each sensor: ,in, For sensor type index, For sensors The raw signal feature values were collected and preprocessed. , , These are, respectively, terahertz reflection intensity, ultrasonic vibration amplitude, and hyperspectral characteristic values. For the corresponding sensor The decision threshold For sensors The sensitivity coefficient, It is a natural constant. Based on sensors Independent confidence level for signal calculation;
[0076] Then, a sensor reliability factor is introduced. ,in, For the sensor's operating temperature, and Its optimal operating temperature and stability parameters. For sensors Real-time reliability weighting factor;
[0077] Finally, the overall presence confidence of the LED chips is calculated: ,in, For sensors The dynamic weighting coefficients in the fusion process satisfy... Its value can be adaptively adjusted based on the sensor signal quality and historical performance.
[0078] Defect pattern classification: Constructing feature vectors ;
[0079] in, The signal difference is denoted as ; where, A comprehensive feature vector used for defect classification. For sensors and Signal difference between them To represent the transpose of a vector, The independent confidence level of the terahertz sensor. The independent confidence level of the laser ultrasonic sensor. The independent confidence level of the hyperspectral sensor. The difference between terahertz and laser ultrasound signals. The difference between laser ultrasound and hyperspectral signals. The difference between hyperspectral and terahertz signals, For sensors Standardized eigenvalues;
[0080] Will Input a multi-classifier based on the Softmax function to calculate the defect category. The probability of:
[0081] ,in, Given a feature vector The tested LED chip belongs to the defect category. The conditional probability, For category indexing, To indicate the first Class defects, Sum index for defect categories, and For category The corresponding weight vector and bias term, The total number of defect categories preset by the system. for The row vector form, for The row vector form, For the first The weight vector of class defects, For the first Class bias term;
[0082] Parameter adaptive adjustment: Based on feedback from historical decision results, the loss function is minimized. The decision threshold is dynamically optimized using the gradient descent method. and weighting coefficients , For the first A historical sample for category The true label, The number of historical samples used for parameter optimization. This is the sample index in the historical sample batch.
[0083] The detection system uses gradient descent to iteratively update parameters.
[0084] , ,in, Threshold Updated learning rate, Weighting Updated learning rate, The updated decision threshold, The current (before) the update threshold. The updated weighting coefficients, The loss function, specifically the cross-entropy loss, is used here. For loss function Regarding thresholds The partial derivatives, For loss function Regarding weight The partial derivatives of .
[0085] Specifically, the actuator and feedback unit includes a report generation and tag triggering module and a system self-test and status feedback module;
[0086] The report generation and marking trigger module is used to receive defect coordinates and type information from the intelligent analysis and decision unit, automatically generate a structured inspection report, drive external marking equipment to mark the location of screen defects, or send sorting instructions to the production line control system.
[0087] The system self-test and status feedback module is used to automatically calibrate the reference parameters of the multi-physics sensor before the detection begins or during idle periods, and to provide real-time feedback on the working status of each sensor, moving component 21, and lighting component 23 to the monitoring center.
[0088] Specifically, the system monitoring and early warning unit includes a comprehensive fault diagnosis module and a hierarchical early warning and indication module;
[0089] The integrated fault diagnosis module is used to monitor system-level or process-level faults such as abnormal sensor signals, interrupted data streams, low confidence levels in analysis algorithms, and actuator response timeouts, and generates diagnostic logs containing fault levels and location information.
[0090] The integrated fault diagnosis module calculates the decision uncertainty measure. To monitor the consistency between sensors, when A self-check is triggered when the threshold is exceeded. For average confidence level, ;
[0091] The graded early warning and indication module is used to trigger graded alarms through prominent color blocks, sounds, and the connected production line central control console when a system fault is diagnosed, the screen defect rate is detected to exceed the process threshold, or a high-risk defect type is identified. It also suspends conveyor line 1 for further processing.
[0092] Working principle: First stage: Conveying, positioning and flexible adaptive clamping
[0093] Conveying and Initial Positioning: The LED display screen to be inspected is carried by conveyor line 1, transported to the preset inspection station, and precisely positioned; the clamping components located on both sides of the frame of conveyor line 1 are activated; the lifting hydraulic cylinder 3 drives the push frame 31 to rise, and the push frame 31 drives the lifting frame 32 to rise smoothly through the tension spring 33. The contact airbags 54 on both sides of the push frame 31 are located on both sides of the LED display screen. After the air pump 5 is activated, it draws in external air through the filter 51, enters the diversion pipe 52, and then enters the telescopic airbag group 55. After the telescopic airbag group 55 expands, it drives the contact airbags 54 to expand and contact the long sides of the LED display screen, thereby adaptively and evenly wrapping and pressing the sides of the display screen to form a stable, stress-free environment. With the force of flexible clamping, the lifting frame 32 continues to rise, pushing the LED display screen upward until the top of the lifting frame 32 contacts the limiting plate 34, completing the hard limit of the rising stroke. The lifting hydraulic cylinder 3 continues to rise, pushing the push frame 31 to rise, so that the limiting guide rail 4 and the moving frame 41 rise. At the same time, the limiting ball 42 of the moving frame 41 can roll in the guide groove plate 43 and move along the trajectory of the guide groove plate 43, pushing the two opposing moving frames 41 to move in opposite directions and outward. After moving out of the periphery of the LED display screen from below, it rises. By inflating the telescopic airbag group 55 on the moving frame 41, the contact airbag 54 can expand, flexibly clamping the wide edge of the LED display screen.
[0094] Phase Two: Multiphysics Collaborative Scanning and Data Acquisition
[0095] Environmental Preparation and Scanning: After the clamping is stabilized, the illumination assembly 23 installed within the detection frame 2 provides optimized illumination. The moving assembly 21 drives the probes of the integrated detection camera 22 (including a hyperspectral microscopic imaging sensor) and the laser-induced ultrasonic sensor to perform a programmed scan above the display screen. Simultaneously, the terahertz scanning imaging sensor probe array fixed to the detection frame 2 performs a top-down area scan.
[0096] Synchronous signal acquisition: At each scanning point, the terahertz sensor acquires the reflected signal from the internal structure. ), laser ultrasonic sensors excite and acquire microscopic vibration spectra ( Hyperspectral sensors collect reflectance spectra from microscopic regions. ).
[0097] Phase 3: Intelligent Signal Processing and Defect Detection
[0098] Signal preprocessing and feature fusion: The signal fusion and feature extraction module performs spatiotemporal synchronization and filtering / denoising on the three raw data streams (e.g., for...). Wavelet denoising, and Bandpass filtering, (Spectral smoothing), and extracted the standardized feature value of each LED position: terahertz average reflection intensity. , Amplitude of the dominant ultrasonic frequency Characteristic wavelength normalized reflectivity .
[0099] Multimodal intelligent decision-making: The multimodal defect decision-making algorithm module performs core calculations.
[0100] Confidence fusion: Calculating the independent confidence scores of each sensor. Combined with its real-time reliability factor Weighted fusion is performed to obtain the overall existence confidence level. .
[0101] Defect classification: Constructing feature vectors The data is then input into a Softmax classifier to calculate the probability of belonging to each type of defect (such as intact, missing, cracked, poorly soldered, or broken gold wire). To output the final defect type with the highest probability.
[0102] Parameter self-optimization: The system utilizes historical verification samples to minimize the loss function. The decision threshold is dynamically optimized using the gradient descent method. With fusion weight .
[0103] Phase 4: Result Execution, System Monitoring, and Process Reset
[0104] Results Output and Execution: The report generation and marking trigger module receives the judgment results (defect coordinates, type, confidence level), automatically generates an inspection report, and drives the marking equipment to mark the defect location or send sorting instructions to the production line.
[0105] Real-time monitoring and early warning: The integrated fault diagnosis module continuously calculates the uncertainty measure of the decision. ,like If the threshold is exceeded or abnormal data is detected, the system self-test is triggered, and an alarm of the corresponding level is issued through the graded early warning and indication module. If necessary, the production line is suspended. The system self-test and status feedback module periodically calibrates the sensors and provides feedback on the status of each unit.
[0106] Release and Reset: After the test is completed, the exhaust air pump 53 starts, quickly drawing air out of the airbag through the diversion pipe 52 and the three-way valve. The contact airbag 54 quickly retracts and detaches from the screen body, the upgraded hydraulic cylinder descends, and the push frame 31 drives the lifting frame 32 to descend and reset smoothly under the action of the tension spring 33. The conveyor line 1 starts, sending the tested screen out of the workstation and into the next screen to be tested, and the cycle begins.
[0107] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. An automatic detection system for missing LED display beads based on machine vision, comprising a conveyor line (1), characterized in that: The outer surface of the frame of the conveyor line (1) is provided with a detection component, which includes a detection camera (22) for detecting the missing LED beads of the LED display screen. A clamping component is fixedly installed on the outer surface of the frame of the conveyor line (1). The clamping component includes a driving device and a fixing device. The driving device includes a lifting frame (32). The lifting of the lifting frame (32) drives the fixing device to lift. The fixing device includes a contact airbag (54). The contact airbag (54) fixes the LED display screen that needs to be tested. It also includes a detection system, which comprises a multi-physics sensing unit, an intelligent analysis and decision unit, an actuator and feedback unit, and a system monitoring and early warning unit.
2. The automatic detection system for missing LED display beads based on machine vision according to claim 1, characterized in that: The detection component also includes a detection frame (2), which is fixedly installed on the upper surface of the frame of the conveyor line (1). A moving component (21) is fixedly installed on the inner wall of the detection frame (2). The lower surface of the slider of the moving component (21) is fixedly installed with the outer surface of the detection camera (22). An illumination component (23) is fixedly installed on the outer surface of the slider.
3. The automatic detection system for missing LED display beads based on machine vision according to claim 2, characterized in that: The driving device also includes a lifting hydraulic cylinder (3), which is fixedly installed on the outer surface of the frame of the conveyor line (1). A pusher frame (31) is fixedly installed on one end of the piston rod of the lifting hydraulic cylinder (3). The outer surface of the limit rod of the pusher frame (31) is slidably inserted into the inner wall of the lifting frame (32). A tension spring (33) is fixedly installed on the outer surface of the limit rod of the pusher frame (31). One end of the tension spring (33) is fixedly installed on the outer surface of the lifting frame (32). A limit plate (34) is fixedly installed on the outer surface of the frame of the conveyor line (1). The lower surface of the limit plate (34) contacts the upper surface of the lifting frame (32).
4. The automatic detection system for missing LED display beads based on machine vision according to claim 3, characterized in that: The upper surface of the lifting frame (32) is fixedly installed with a limiting guide rail (4), and a movable frame (41) is slidably inserted into the upper surface of the limiting guide rail (4). The lower surface of the movable frame (41) is rotatably connected with a limiting ball (42) through a bearing. The outer surface of the lifting frame (32) is fixedly installed with a guide groove plate (43), and the outer surface of the limiting ball (42) is slidably connected to the inner wall of the guide groove plate (43).
5. The automatic detection system for missing LED display beads based on machine vision according to claim 4, characterized in that: The fixed device also includes an air pump (5), which is fixedly installed on the outer surface of the frame of the conveying line (1). A filter (51) is fixedly installed on the outer surface of the frame of the conveying line (1). The outlet end of the filter (51) is fixedly connected to the inlet end of the air pump (5). The outlet end of the air pump (5) is fixedly connected to a diversion pipe (52) with a control valve. An outlet air pump (53) is fixedly installed on the outer surface of the frame of the conveying line (1). The outer surface of the diversion pipe (52) is fixedly connected to the inlet end of the outlet air pump (53) through a three-way valve.
6. The automatic detection system for missing LED display beads based on machine vision according to claim 5, characterized in that: The outer surfaces of the multiple contact airbags (54) are fixedly installed on the outer surfaces of the movable frame (41) and the lifting frame (32). The inner wall of the contact airbag (54) is fixedly installed with a telescopic airbag assembly (55). The multiple telescopic airbag assemblies (55) are fixedly connected to each other through connecting hoses. The outer surface of the telescopic airbag assembly (55) is fixedly connected with a delivery hose (56). One end of the delivery hose (56) is fixedly connected to one end of the diversion pipe (52). The two sets of delivery hoses (56) are located inside the lifting frame (32) and the pushing frame (31), respectively.
7. The automatic detection system for missing LED display beads based on machine vision according to claim 6, characterized in that: The multi-physics sensing unit includes a multi-physics sensor array and a signal fusion and feature extraction module; The multiphysics sensor array includes a terahertz scanning imaging sensor, a laser-induced ultrasound sensor, and a hyperspectral microscopic imaging sensor, which collects deep information from the screen from three physical domains: electromagnetic waves, mechanical vibration, and material spectra. The transceiver probe array of the terahertz scanning imaging sensor is mounted on the detection frame (2) to perform area array scanning on the LED display screen that is stationary at the detection station and obtain chip structure information below the encapsulation layer. The pulsed laser emitter and laser interferometer of the laser-induced ultrasonic sensor are integrated on the moving component (21), which moves with it and aligns with each lamp bead unit to excite and detect its micro-vibration spectrum; The hyperspectral microscopic imaging sensor is integrated into the detection camera (22) to acquire the microscopic reflectance spectral characteristics of each lamp bead region in a wide spectral range; The signal fusion and feature extraction module is used to synchronize, filter, and reduce noise in the raw data collected by the three sensors, and extract key features, including terahertz reflection intensity map, ultrasonic feature spectrum, and material reflection spectrum, to provide a stable and multi-dimensional fusion data source for subsequent decision-making.
8. The automatic detection system for missing LED display beads based on machine vision according to claim 7, characterized in that: The intelligent analysis and decision unit includes a core analysis module and a multimodal defect decision algorithm module; The core analysis module receives fused feature data from the multi-physics sensing unit and executes analysis logic; The multimodal defect judgment algorithm module has a built-in deep learning-based multi-source information fusion network and expert rule base. It is used to calculate the physical existence confidence and internal health score of each LED position based on the fused multi-physics field feature data. By comparing with the preset defect feature mapping model, it dynamically generates defect type judgment results, including complete absence, chip breakage, internal cold solder joint, and gold wire breakage, so as to achieve accurate identification and classification of defects. The multimodal defect decision algorithm module achieves accurate defect identification and classification through the following mathematical process: Multi-source confidence fusion calculation: First, calculate the independent confidence level based on the signals from each sensor: ,in, For sensor type index, For sensors The raw signal feature values were collected and preprocessed. , , These are, respectively, terahertz reflection intensity, ultrasonic vibration amplitude, and hyperspectral characteristic values. For the corresponding sensor The decision threshold For sensors The sensitivity coefficient, It is a natural constant. Based on sensors Independent confidence level for signal calculation; Then, a sensor reliability factor is introduced. ,in, For the sensor's operating temperature, and Its optimal operating temperature and stability parameters. For sensors Real-time reliability weighting factor; Finally, the overall presence confidence of the LED chips is calculated: ,in, For sensors The dynamic weighting coefficients in the fusion process satisfy... Its value can be adaptively adjusted based on the sensor signal quality and historical performance. Defect pattern classification: Constructing feature vectors ; in, The signal difference is denoted as ; where, A comprehensive feature vector used for defect classification. For sensors and Signal difference between them To represent the transpose of a vector, The independent confidence level of the terahertz sensor. The independent confidence level of the laser ultrasonic sensor. The independent confidence level of the hyperspectral sensor. The difference between terahertz and laser ultrasound signals. The difference between laser ultrasound and hyperspectral signals. The difference between hyperspectral and terahertz signals, For sensors Standardized eigenvalues; Will Input a multi-classifier based on the Softmax function to calculate the defect category. The probability of: ,in, Given a feature vector The tested LED chip belongs to the defect category. The conditional probability, For category indexing, To indicate the first Class defects, Sum index for defect categories, and For category The corresponding weight vector and bias term, The total number of defect categories preset by the system. for The row vector form, for The row vector form, For the first The weight vector of class defects, For the first Class bias term; Parameter adaptive adjustment: Based on feedback from historical decision results, the loss function is minimized. The decision threshold is dynamically optimized using the gradient descent method. and weighting coefficients , For the first A historical sample for category The true label, The number of historical samples used for parameter optimization. This is the sample index in the historical sample batch.
9. The automatic detection system for missing LED display beads based on machine vision according to claim 8, characterized in that: The actuator and feedback unit includes a report generation and tag triggering module and a system self-test and status feedback module; The report generation and marking trigger module is used to receive defect coordinates and type information from the intelligent analysis and judgment unit, automatically generate a structured inspection report, drive external marking devices to mark the location of screen defects, or send sorting instructions to the production line control system. The system self-test and status feedback module is used to automatically calibrate the reference parameters of the multi-physics sensor before the detection begins or during idle periods, and to provide real-time feedback on the working status of each sensor, the moving component (21), and the lighting component (23) to the monitoring center.
10. The automatic detection system for missing LED display beads based on machine vision according to claim 9, characterized in that: The system monitoring and early warning unit includes a comprehensive fault diagnosis module and a hierarchical early warning and indication module; The integrated fault diagnosis module is used to monitor system-level or process-level faults such as abnormal sensor signals, interrupted data streams, low confidence levels in analysis algorithms, and actuator response timeouts, and generates diagnostic logs containing fault levels and location information. The comprehensive fault diagnosis module calculates a decision uncertainty metric. To monitor the consistency between sensors, when A self-check is triggered when the threshold is exceeded. The average confidence level; When the graded early warning and indication module diagnoses a system fault, detects that the screen defect rate exceeds the process threshold, or determines a high-risk defect type, it triggers a graded alarm through a prominent color block and sound on the local human-machine interface (HMI) and the connected production line central control console, and suspends the conveyor line (1) for processing.