Positioning diagnosis device of circuit board component

Abstract

The invention discloses a circuit board component positioning diagnosis device, which relates to the technical field of component detection, and comprises a positioning diagnosis device and a positioning assembly, and the positioning diagnosis device is arranged at the top of the positioning assembly. A multi-modal information acquisition module, an information processing and fusion module, a positioning diagnosis module and a man-machine interaction and control module are integrated in the positioning diagnosis device; the multi-modal information acquisition module comprises an optical imaging unit, an X-ray imaging unit and an infrared thermal imaging unit; a non-contact omnibearing detection system is constructed through a multi-mode information acquisition module integrating optics, X-ray and infrared thermal imaging, so that the technical bottlenecks that the efficiency is low, a circuit board is easy to damage, internal and thermal faults are difficult to find and the like in the traditional probe contact detection are effectively overcome; the full-dimension nondestructive diagnosis of the circuit board component from the appearance, the internal structure to the working state is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of component detection, in particular to a positioning diagnosis device for circuit board components. BACKGROUND

[0002] Under the background of the vigorous development of modern electronic industry, the quality and performance of circuit boards, as the core hub of electronic products, directly determine the stability and reliability of terminal equipment. With the rapid rise of 5G communication, artificial intelligence, automotive electronics and other fields, the integration of circuit boards continues to rise, and the density of components is getting higher and higher. Miniaturization and precision have become the development trend. Under this situation, the positioning diagnosis technology of circuit board components has become an indispensable link to ensure product quality.

[0003] When the existing circuit board components are positioned and diagnosed, the physical contact between the probe and the test point of the circuit board is mainly relied on to obtain the electrical parameters, and the point-by-point detection method leads to low detection efficiency. It takes a long time to complete the comprehensive detection of complex circuit boards, which is difficult to meet the production needs of rapid iteration of electronic products. In addition, frequent physical contact can easily damage the circuit board, which may cause problems such as wear and tear of the test point plating layer, open circuit and short circuit, resulting in the scrapping of the circuit board, increasing production costs, and may also affect the detection accuracy due to the position deviation of the probe. Therefore, the present application provides a positioning diagnosis device for circuit board components. SUMMARY

[0004] The purpose of the present application is to provide a positioning diagnosis device for circuit board components to solve the problem that the existing circuit board components are mainly relied on the physical contact between the probe and the test point of the circuit board to obtain the electrical parameters when positioned and diagnosed, which leads to low detection efficiency. It takes a long time to complete the comprehensive detection of complex circuit boards, which is difficult to meet the production needs of rapid iteration of electronic products. In addition, frequent physical contact can easily damage the circuit board, which may cause problems such as wear and tear of the test point plating layer, open circuit and short circuit, resulting in the scrapping of the circuit board, increasing production costs, and may also affect the detection accuracy due to the position deviation of the probe.

[0005] In order to achieve the above purpose, the present application provides the following technical scheme:

[0006] A positioning diagnosis device for circuit board components, comprising a positioning diagnosis device and a positioning assembly, the positioning diagnosis device is installed on the top of the positioning assembly, the positioning diagnosis device is integrated with a multi-modal information acquisition module, an information processing and fusion module, a positioning diagnosis module and a man-machine interaction and control module;

[0007] The multimodal information acquisition module includes an optical imaging unit, an X-ray imaging unit, and an infrared thermal imaging unit, which are used to acquire optical image information, X-ray image information, and infrared thermal image information of circuit board components, respectively. The optical imaging unit, X-ray imaging unit, and infrared thermal imaging unit are all communicatively connected to the information processing and fusion module, and transmit the acquired image information to the information processing and fusion module.

[0008] The information processing and fusion module is used to preprocess, extract features and fuse information on the image information transmitted by the multimodal information acquisition module. It includes an image preprocessing submodule, a feature extraction submodule and an information fusion submodule. The information processing and fusion module is communicatively connected to the positioning diagnosis module and transmits the fused information to the positioning diagnosis module.

[0009] The positioning and diagnosis module performs positioning and fault diagnosis on circuit board components based on the fused information transmitted by the information processing and fusion module. It includes a component positioning submodule and a fault diagnosis submodule. The positioning and diagnosis module is communicatively connected to the human-machine interaction and control module and transmits the positioning and diagnosis results to the human-machine interaction and control module.

[0010] The human-computer interaction and control module is used to receive control commands input by the user and control the working status of the multimodal information acquisition module, information processing and fusion module and positioning diagnosis module according to the control commands. It is also used to display the positioning and diagnosis results output by the positioning diagnosis module. The human-computer interaction and control module, the multimodal information acquisition module, the information processing and fusion module and the positioning diagnosis module are all connected to each other through a data bus.

[0011] Optionally, the optical imaging unit includes a high-resolution industrial camera for acquiring optical images of the surface features of the circuit board and components, wherein the high-resolution industrial camera has a resolution of 5000×5000 pixels or higher.

[0012] Optionally, the X-ray imaging unit includes a microfocus X-ray generator and a corresponding flat panel detector. The focal size of the microfocus X-ray generator is ≤5μm. The flat panel detector is communicatively connected to the information processing and fusion module for transmitting three-dimensional imaging data.

[0013] Optionally, the infrared thermal imaging unit is equipped with an uncooled focal plane detector with a response band of 8-14μm and a thermal sensitivity of ≤50mK, and is connected to the information processing and fusion module via a data cable.

[0014] Optionally, the image preprocessing submodule includes a noise reduction unit, an enhancement unit, and a correction unit. The noise reduction unit uses a nonlocal mean filtering algorithm to perform noise reduction processing on the acquired image. The enhancement unit uses a histogram equalization method to enhance the contrast of the image. The correction unit corrects the distortion of the image using a geometric correction algorithm.

[0015] Optionally, the feature extraction submodule uses different algorithms to extract optical, X-ray, and infrared features respectively; for optical features, the Canny edge detection algorithm is used to extract the edge features of components; for X-ray features, the U-Net segmentation network is used to extract solder joints and internal structural features; for infrared features, the threshold segmentation method is used to extract temperature anomaly region features.

[0016] Optionally, the information fusion submodule employs a multi-stream convolutional neural network fusion method to perform multi-level fusion of preprocessed optical, X-ray, and infrared images and extracted features.

[0017] Optionally, the component localization submodule uses the YOLO algorithm to perform real-time component localization using the fused information through a trained model, and the fault diagnosis submodule establishes a ResNet-based fault diagnosis model, inputting the fused features into the model for classification and judgment.

[0018] Optionally, it also includes a carrier plate, a template, and a transmission assembly. The positioning assembly is disposed on the carrier plate, and the template is disposed on the top of the carrier plate in a front-to-back manner. A transmission assembly for transporting the circuit board is provided on the opposite side of both sides of the template.

[0019] Optionally, the positioning assembly includes a positioning frame and a lifting component mounted on top of the support plate. The top side of the positioning frame is provided with a loading plate, and the bottom front and rear sides of the loading plate are provided with two sets of buffer positioning rods acting on the circuit board via fixed seats. The lifting component is mounted on the support plate, and the top output end of the lifting component is connected to a support plate for supporting the circuit board to be diagnosed.

[0020] The beneficial effects of this invention are:

[0021] 1. This invention constructs a non-contact, all-round detection system by integrating multi-modal information acquisition modules of optics, X-rays and infrared thermal imaging. It effectively overcomes the technical bottlenecks of traditional probe contact detection, such as low efficiency, easy damage to circuit boards, and difficulty in detecting internal and thermal faults. It realizes non-destructive diagnosis of circuit board components from appearance, internal structure to working status.

[0022] 2. This invention employs an information fusion method based on multi-stream convolutional neural networks to deeply fuse preprocessed multi-source image features, generating a more comprehensive and discriminative fusion feature map. Combined with the fast and accurate localization achieved by the YOLO algorithm and the intelligent fault classification based on the ResNet model, it significantly improves the localization accuracy of components in complex backgrounds and the diagnostic accuracy and reliability of various hidden faults. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0024] Figure 1 This is a block diagram of the overall architecture of the integrated module inside the positioning diagnostic device of the present invention;

[0025] Figure 2 This is a structural block diagram of the multimodal information acquisition module in this invention;

[0026] Figure 3 This is a structural block diagram of the information processing and fusion module in this invention;

[0027] Figure 4 This is a structural block diagram of the positioning and diagnosis module in this invention;

[0028] Figure 5 This is a schematic diagram of the structure of a circuit board component positioning and diagnostic device according to the present invention;

[0029] Figure 6 A schematic diagram of the structure of the present invention after removing the positioning diagnostic device.

[0030] The numbers on the map are:

[0031] 1. Positioning diagnostic device;

[0032] 2. Positioning components; 201. Positioning frame; 202. Loading plate; 203. Fixing base; 204. Buffer positioning rod; 205. Lifting component; 206. Support plate;

[0033] 3. Support plate; 4. Profile plate;

[0034] 5. Transmission components. Detailed Implementation

[0035] To make the technical means, creative features, objectives and effects of this invention easier to understand, the invention will be further described below in conjunction with specific embodiments.

[0036] As attachedFigure 1 To be continued Figure 6 As shown, the present invention provides a positioning and diagnostic device for circuit board components, including a positioning diagnostic device 1 and a positioning component 2. The positioning diagnostic device 1 is installed on the top of the positioning component 2. The positioning diagnostic device 1 integrates a multimodal information acquisition module, an information processing and fusion module, a positioning diagnostic module, and a human-computer interaction and control module.

[0037] The multimodal information acquisition module includes an optical imaging unit, an X-ray imaging unit, and an infrared thermal imaging unit, which are used to acquire optical image information, X-ray image information, and infrared thermal image information of circuit board components, respectively. The optical imaging unit, X-ray imaging unit, and infrared thermal imaging unit are all communicatively connected to the information processing and fusion module, and transmit the acquired image information to the information processing and fusion module.

[0038] The information processing and fusion module is used to preprocess, extract features and fuse information on the image information transmitted by the multimodal information acquisition module. It includes an image preprocessing submodule, a feature extraction submodule and an information fusion submodule. The information processing and fusion module is communicatively connected to the positioning and diagnosis module and transmits the fused information to the positioning and diagnosis module.

[0039] The positioning and diagnosis module, based on the fused information transmitted by the information processing and fusion module, performs positioning and fault diagnosis on the circuit board components. It includes a component positioning submodule and a fault diagnosis submodule. The positioning and diagnosis module is communicatively connected to the human-machine interaction and control module and transmits the positioning and diagnosis results to the human-machine interaction and control module.

[0040] The human-computer interaction and control module is used to receive control commands input by the user and control the working status of the multimodal information acquisition module, information processing and fusion module and positioning diagnosis module according to the control commands. It is also used to display the positioning and diagnosis results output by the positioning diagnosis module. The human-computer interaction and control module communicates with the multimodal information acquisition module, information processing and fusion module and positioning diagnosis module through a data bus.

[0041] like Figure 2 As shown, in one embodiment of the present invention, the optical imaging unit includes a high-resolution industrial camera for acquiring optical images of the surface features of circuit boards and components. The high-resolution industrial camera has a resolution of 5000×5000 pixels or higher. The high-resolution industrial camera can be a Basler-acA4112-30uc model and is equipped with an adjustable focus lens and a ring-shaped lighting system. The ring-shaped lighting system includes ring-shaped white light, coaxial light, and infrared light, and the switching between multiple light sources is achieved through a programmable logic controller.

[0042] Specifically, the CMOS image sensor of the industrial camera converts the surface-reflected light signals of the circuit board components into electrical signals, which are then quantized into digital image data by a 12-bit A / D converter. For components with different package types, the imaging effect can be optimized by adjusting the lens focal length and the supplementary lighting mode. For example, when inspecting tiny surface-mount components, switching to blue light supplementary lighting reduces surface reflection interference, and using a 50mm telephoto lens to magnify local details; when inspecting chip pins, using a ring of white light to uniformly illuminate the pin area avoids misjudgments of pin breakage caused by shadows.

[0043] The X-ray imaging unit includes a microfocus X-ray generator and a corresponding flat panel detector. The microfocus X-ray generator has a focal size ≤5μm, and the flat panel detector is communicatively connected to the information processing and fusion module for transmitting three-dimensional imaging data. The microfocus X-ray generator is model Hamamatsu-L8121-01, and the flat panel detector is made of amorphous silicon with a pixel size of 127μm and an effective detection area of ​​200×200mm.

[0044] Specifically, the microfocus X-ray generator produces an adjustable-energy X-ray beam. When it penetrates the circuit board, materials of different densities (such as component solder, PCB substrate, and air gaps) absorb the X-rays to varying degrees. The remaining rays are received by a flat panel detector and converted into electrical signals. These signals are then used in a reconstruction algorithm (using a filtered back-projection algorithm) to generate a three-dimensional tomographic image of the circuit board. This unit can penetrate the PCB substrate, clearly revealing the morphology of internal solder joints (such as cold solder joints, bridging, and voids) and the location of components hidden beneath the substrate, thus overcoming the technical blind spot of traditional optical inspection's inability to penetrate the substrate.

[0045] It should be further described that the X-ray generator is equipped with a dose control module that automatically adjusts the radiation dose according to the thickness of the circuit board (pre-measured by the optical imaging unit). For example, when inspecting a conventional PCB with a thickness of 1.6 mm, the dose is set to 1.2 mGy; when inspecting a multilayer PCB with a thickness of 3.2 mm, the dose is increased to 2.5 mGy, so as to ensure the clarity of the image while avoiding excessive radiation damage to the components.

[0046] The infrared thermal imaging unit is equipped with an uncooled focal plane array detector with a response band of 8-14μm and a thermal sensitivity of ≤50mK. It is connected to the information processing and fusion module via a data cable. The uncooled focal plane array detector is model FLIR-Lepton-3.5.

[0047] Specifically, the detector senses the infrared radiation energy on the surface of components through a microbolometer array, converts temperature differences into electrical signals, and generates thermal image data. During the detection process, the circuit board needs to be powered on first (DC voltage can be provided through the built-in adjustable power supply module) to bring the components into normal working condition. At this time, faulty components (such as short-circuited capacitors or overloaded resistors) will generate local overheating due to abnormal energy loss, which will appear as obvious "hot spots" in the thermal image.

[0048] like Figure 3 As shown, in one embodiment of the present invention, the image preprocessing submodule includes a noise reduction unit, an enhancement unit, and a correction unit. The noise reduction unit uses a nonlocal mean filtering algorithm to denoise the acquired image. This algorithm is based on the characteristic that there are a large number of similar structures in the image. It achieves the purpose of noise reduction by searching for regions similar to the current pixel block in the entire image range and performing a weighted average of these similar regions. The enhancement unit uses a histogram equalization method to enhance the contrast of the image. By redistributing the gray values ​​of pixels in the image, the gray-level histogram of the image is distributed as evenly as possible, thereby expanding the gray-level dynamic range of the image and improving the contrast between the target and the background in the image. The correction unit corrects the distortion of the image using a geometric correction algorithm. During the image acquisition process, due to factors such as lens optical characteristics and shooting angle, the image will produce radial and tangential distortion, which will cause deviations in the shape and position of objects in the image, affecting the accuracy of subsequent feature extraction and localization diagnosis.

[0049] Specifically, for each pixel in the image, the nonlocal mean filtering algorithm searches for other blocks with high similarity to the local block containing that pixel within a search window centered on that pixel. The similarity is calculated based on the difference in grayscale values ​​between corresponding pixels in the two blocks; the smaller the difference, the higher the similarity, and the greater the corresponding weight. In this way, the algorithm can remove noise while preserving as much detail as possible in the image, such as the edges of components and the contours of solder joints.

[0050] First, the frequency of each gray level in the image is statistically analyzed, and the cumulative distribution function is calculated. Then, the gray values ​​of the original image are mapped to new gray values ​​based on the cumulative distribution function, thereby achieving image enhancement. In this invention, adaptive histogram equalization (CLAHE) is used, which divides the image into multiple sub-blocks, performs histogram equalization on each sub-block, and limits the contrast gain of each sub-block to avoid noise amplification due to excessive local enhancement. This effectively improves the clarity of component features, such as the gray level difference between chip pins and the substrate, and the contrast between solder joints and surrounding areas.

[0051] The calibration unit first acquires images from multiple angles using standard references such as a checkerboard calibration board. It then extracts the corner coordinates from the calibration board using a corner detection algorithm. Next, based on classic algorithms such as Zhang's calibration method, it calculates the lens distortion parameters (including radial and tangential distortion coefficients) as well as the image's rotation, translation, and scaling parameters. Finally, based on the calculated parameters, it performs an inverse transformation on the original image, correcting the distorted image to conform to actual physical coordinates. This accurately restores the position and shape of the components, providing a precise data foundation for subsequent positioning and diagnosis.

[0052] In one embodiment of the present invention, the feature extraction submodule uses different algorithms to extract optical, X-ray and infrared features respectively; for optical features, the Canny edge detection algorithm is used to extract the edge features of components; for X-ray features, the U-Net segmentation network is used to extract solder joints and internal structural features; for infrared features, the threshold segmentation method is used to extract temperature anomaly region features.

[0053] Specifically, optical feature extraction involves using the Canny edge detection algorithm. First, the preprocessed optical image is Gaussian blurred to suppress image noise that might interfere with edge detection. Then, operators like Sobel are used to calculate the gradient magnitude and direction of each pixel in the image. Areas with large gradient magnitudes are potential edges with drastic grayscale changes. Along the gradient direction, only pixels with local maxima of gradient magnitude are retained, thus refining wide "edge bands" into precise edges of single pixel width. Two thresholds are set: gradient magnitudes above the high threshold are considered strong edges, those below the low threshold are suppressed, and those in between are considered weak edges. Ultimately, only weak edges connected to strong edges are retained as true edges, effectively connecting broken edges and suppressing spurious responses, thereby accurately extracting the edge features of the components.

[0054] X-ray Feature Extraction: The U-Net segmentation network is used to extract features of weld points and internal structures. U-Net is an image segmentation model based on a fully convolutional neural network with a symmetrical encoder-decoder structure. The encoder progressively downsamples through a series of convolutional and pooling layers to extract high-level semantic features of the image, while simultaneously increasing the receptive field to capture global information within the image. The decoder then progressively restores the features extracted by the encoder to a segmentation result with the same resolution as the original image through upsampling and skip connections. In X-ray image processing, the U-Net network can learn complex feature patterns of weld points and internal structures. After the encoder extracts features from the input X-ray image, the decoder maps these features to the pixel level, outputting the probability that each pixel belongs to a weld point or internal structure, thereby achieving accurate segmentation of weld points and internal structures and extracting their features.

[0055] Infrared Feature Extraction: This method uses threshold segmentation to extract features from temperature anomaly regions. In infrared thermal imaging, faulty components (such as short-circuited capacitors or overloaded resistors) can cause localized overheating due to abnormal energy loss, appearing as obvious "hot spots" in the thermal image. Threshold segmentation categorizes pixels in the thermal image by setting one or more thresholds. First, appropriate thresholds are determined based on the temperature distribution characteristics of normal and faulty components. Then, the temperature value of each pixel in the thermal image is compared with the threshold. If the pixel temperature value is higher than the threshold, it is classified as a temperature anomaly region; otherwise, it is classified as a normal region. This allows for the extraction of features from the infrared thermal image of temperature anomaly regions, providing a basis for subsequent fault diagnosis.

[0056] It should be further noted that, due to the dense components and complex background of the circuit board image, the high threshold of the Canny algorithm can be set to 150 and the low threshold to 50.

[0057] In one embodiment of the present invention, the information fusion submodule employs a multi-stream convolutional neural network fusion method to perform multi-level fusion of preprocessed optical, X-ray and infrared images and extracted features.

[0058] Specifically, a multi-stream convolutional neural network (CNN) fusion method is employed to fuse preprocessed optical, X-ray, and infrared images along with extracted features at multiple levels. The CNN has multiple input streams, each processing images and features from different modalities. Each input stream extracts high-level features for its respective modality through operations such as convolutional and pooling layers. Then, these features from different modalities are fused at specific layers of the network, using methods such as feature concatenation or feature addition. Through multi-level fusion, the network can comprehensively utilize information from different modalities of images, learning more comprehensive and discriminative feature representations, thereby improving the accuracy of subsequent component localization and fault diagnosis.

[0059] Fusion process: The input size of the optical image branch is 512x512x3 (RGB), and the output feature map size after convolution is 128x128x256;

[0060] The input X-ray image is 512x512x1 (grayscale), and the output feature map size after convolution is 128x128x128.

[0061] The infrared thermal image input is 512x512x1 (grayscale), and the output feature map size after convolution is 128x128x128.

[0062] In the fusion layer, the three feature maps are concatenated to obtain a multimodal fusion feature map with a size of 128x128x(256+128+128)=128x128x512.

[0063] like Figure 4 As shown, in one embodiment of the present invention, the component positioning submodule uses the YOLO algorithm to perform real-time component positioning based on the fused information through a trained model, and the fault diagnosis submodule establishes a fault diagnosis model based on ResNet and inputs the fused features into the model for classification and judgment.

[0064] Specifically, the YOLO algorithm is a deep learning-based object detection algorithm that transforms the object detection problem into a regression problem. The algorithm divides the input image into multiple grids, with each grid responsible for predicting a certain number of bounding boxes and their corresponding class probabilities. During training, the model learns the mapping relationship from image features to object bounding boxes and classes. When the fused image information is input, the trained YOLO model can quickly and accurately predict the location and class of components in the image, achieving real-time component localization.

[0065] A fault diagnosis model based on ResNet is established, in which fused features are input into the model for classification. ResNet is a deep convolutional neural network that solves the gradient vanishing problem during deep neural network training by introducing residual blocks, allowing the network to be trained deeper and learn more complex features. In fault diagnosis, the fused features are input into the ResNet model, which further extracts and transforms the features through multiple convolutional and fully connected layers. The output layer uses the Softmax activation function, and finally, the softmax layer outputs the probability distribution of fault categories. The fault type of the circuit board components is determined based on the probability magnitude.

[0066] like Figures 5-6 As shown, in one embodiment of the present invention, it further includes a carrier plate 3, a template 4 and a transmission component 5. The positioning component 2 is disposed on the carrier plate 3, and the template 4 is disposed on the top of the carrier plate 3 in a front-to-back manner. The two templates 4 are each provided with a transmission component 5 for transporting the circuit board on their opposite sides.

[0067] The positioning component 2 includes a positioning frame 201 mounted on the top of the support plate 3 and a lifting component 205. The top side of the positioning frame 201 is provided with a loading plate 202. The bottom front and rear sides of the loading plate 202 are provided with two sets of buffer positioning rods 204 that act on the circuit board through the fixing seat 203. The ends of the buffer positioning rods 204 may be provided with polyurethane buffer pads. The lifting component 205 is mounted on the support plate 3, and the top output end of the lifting component 205 is connected to a support plate 206 for supporting the circuit board to be diagnosed. The lifting component 205 may be a servo electric cylinder.

[0068] Specifically, when the circuit board to be diagnosed is automatically fed onto the positioning component 2 by the transmission component 5, the lifting component 205 is activated, driving the support plate 206 and the circuit board to move upward into the diagnostic station. When the circuit board approaches the loading plate 202, the buffer positioning rod 204 fixed on it contacts the edge of the circuit board to provide flexible buffering and position the circuit board.

[0069] It should be further described that the transmission component 5 includes a transmission bracket disposed on the side of the template 4, a transmission bar wrapped around the transmission bracket, and an external drive unit for driving the transmission bar.

[0070] Working principle: The circuit board to be diagnosed is automatically fed into the device by the transmission component 5 and arrives smoothly at the initial working position. Then, the positioning component 2 starts to work, and its lifting component 205 drives the support plate 206 to lift the circuit board vertically. During this process, the buffer positioning rod 204 installed on the loading plate 202 contacts and guides the circuit board through the flexible buffer pad at its end, so as to achieve precise mechanical positioning.

[0071] After positioning, the human-machine interaction and control module receives the start command and sends control signals to each module. First, it coordinates the multimodal information acquisition module to start working together. The high-resolution industrial camera of the optical imaging unit first acquires high-definition images of the circuit board surface, clearly capturing the appearance features such as component shapes, silkscreen printing, and pins. The micro-focus X-ray source of the X-ray imaging unit starts simultaneously. The emitted X-rays penetrate the circuit board and are received by the flat panel detector, reconstructing a three-dimensional tomographic image, revealing internal defects and hidden structures of solder joints non-destructively. At the same time, with the circuit board powered on, the uncooled focal plane detector of the infrared thermal imaging unit scans synchronously, generating a thermal image reflecting the real-time working status of the components, making any abnormal heating or low temperature areas immediately apparent. The acquired multi-source data is transmitted in real time to the information processing and fusion module for in-depth analysis via the data bus.

[0072] The image preprocessing submodule first "cleans" and "standardizes" the original image, successively using nonlocal mean filtering for noise reduction, histogram equalization to enhance contrast, and geometric correction based on calibration parameters to ensure data quality for subsequent analysis. The feature extraction submodule then operates in parallel, employing proprietary algorithms for different modalities of image characteristics. For optical images, the Canny algorithm is used to extract precise edge features; for X-ray images, the U-Net segmentation network is used to extract weld points and internal structural features with pixel-level precision; and for infrared thermal images, an adaptive threshold segmentation method is used to lock features in temperature anomaly regions. The information fusion submodule finally adopts a multi-stream convolutional neural network architecture to stitch and fuse the high-level feature maps extracted from the optical, X-ray, and infrared streams at specific layers, generating a unified fusion feature map rich in multi-dimensional information.

[0073] Subsequently, the localization and diagnosis module performs the final decision based on the deep fusion feature map. The component localization submodule calls the trained YOLO algorithm model to quickly and accurately identify all components in the image and mark their precise bounding box coordinates and categories. The fault diagnosis submodule inputs the fused features into a deep classification model built on ResNet. This model analyzes complex fault modes through its deep residual network. Finally, the Softmax output layer calculates the probability distribution of various faults and determines the specific health status of each component based on the maximum probability value.

[0074] Finally, the positioning and diagnosis module transmits the positioning and diagnosis results to the human-computer interaction and control module. This module receives the results and displays them on the display and operation unit, allowing users to intuitively understand the status of the circuit board components. This achieves fully automated diagnosis from circuit board input, information collection, processing and fusion to positioning diagnosis and result display.

[0075] The foregoing has only described certain exemplary embodiments of the present invention by way of illustration. Undoubtedly, those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the foregoing drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

Claims

1. A positioning and diagnostic device for circuit board components, characterized in that: It includes a positioning diagnostic device (1) and a positioning component (2). The positioning diagnostic device (1) is installed on the top of the positioning component (2). The positioning diagnostic device (1) integrates a multimodal information acquisition module, an information processing and fusion module, a positioning diagnostic module, and a human-computer interaction and control module. The multimodal information acquisition module includes an optical imaging unit, an X-ray imaging unit, and an infrared thermal imaging unit, which are used to acquire optical image information, X-ray image information, and infrared thermal image information of circuit board components, respectively. The optical imaging unit, X-ray imaging unit, and infrared thermal imaging unit are all communicatively connected to the information processing and fusion module, and transmit the acquired image information to the information processing and fusion module. The information processing and fusion module is used to preprocess, extract features and fuse information on the image information transmitted by the multimodal information acquisition module. It includes an image preprocessing submodule, a feature extraction submodule and an information fusion submodule. The information processing and fusion module is communicatively connected to the positioning diagnosis module and transmits the fused information to the positioning diagnosis module. The positioning and diagnosis module performs positioning and fault diagnosis on circuit board components based on the fused information transmitted by the information processing and fusion module. It includes a component positioning submodule and a fault diagnosis submodule. The positioning and diagnosis module is communicatively connected to the human-machine interaction and control module and transmits the positioning and diagnosis results to the human-machine interaction and control module. The human-computer interaction and control module is used to receive control commands input by the user and control the working status of the multimodal information acquisition module, information processing and fusion module and positioning diagnosis module according to the control commands. It is also used to display the positioning and diagnosis results output by the positioning diagnosis module. The human-computer interaction and control module, the multimodal information acquisition module, the information processing and fusion module and the positioning diagnosis module are all connected to each other through a data bus.

2. The positioning and diagnostic device for circuit board components according to claim 1, characterized in that: The optical imaging unit includes a high-resolution industrial camera for acquiring optical images of the surface features of circuit boards and components, wherein the high-resolution industrial camera has a resolution of 5000×5000 pixels or higher.

3. The positioning and diagnostic device for circuit board components according to claim 2, characterized in that: The X-ray imaging unit includes a microfocus X-ray generator and a flat panel detector that works in conjunction with it. The focal size of the microfocus X-ray generator is ≤5μm. The flat panel detector is communicatively connected to the information processing and fusion module for transmitting three-dimensional imaging data.

4. The positioning and diagnostic device for circuit board components according to claim 3, characterized in that: The infrared thermal imaging unit is equipped with an uncooled focal plane detector with a response band of 8-14μm and a thermal sensitivity of ≤50mK. It is connected to the information processing and fusion module via a data cable.

5. The positioning and diagnostic device for circuit board components according to claim 4, characterized in that: The image preprocessing submodule includes a noise reduction unit, an enhancement unit, and a correction unit. The noise reduction unit uses a nonlocal mean filtering algorithm to reduce noise in the acquired image. The enhancement unit uses a histogram equalization method to enhance the contrast of the image. The correction unit corrects the distortion of the image using a geometric correction algorithm.

6. The positioning and diagnostic device for circuit board components according to claim 5, characterized in that: The feature extraction submodule employs different algorithms to extract optical, X-ray, and infrared features; for optical features, the Canny edge detection algorithm is used to extract the edge features of the components; for... For X-ray features, the U-Net segmentation network was used to extract weld joint and internal structural features; for infrared features, a threshold segmentation method was used to extract features of temperature anomaly regions.

7. The positioning and diagnostic device for circuit board components according to claim 6, characterized in that: The information fusion submodule employs a multi-stream convolutional neural network fusion method to fuse preprocessed optical, X-ray, and infrared images with extracted features at multiple levels.

8. The positioning and diagnostic device for circuit board components according to claim 1, characterized in that: The component localization submodule uses the YOLO algorithm to perform real-time component localization using the fused information through a trained model. The fault diagnosis submodule establishes a ResNet-based fault diagnosis model and inputs the fused features into the model for classification and judgment.

9. The positioning and diagnostic device for circuit board components according to claim 1, characterized in that: It also includes a carrier plate (3), a template (4) and a transmission component (5). The positioning component (2) is disposed on the carrier plate (3). The template (4) is disposed on the top of the carrier plate (3) in a front-to-back manner. The template (4) on both sides of the template (4) is provided with a transmission component (5) for transporting the circuit board.

10. The positioning and diagnostic device for circuit board components according to claim 9, characterized in that: The positioning component (2) includes a positioning frame (201) mounted on the top of the support plate (3) and a lifting component (205). The top side of the positioning frame (201) is provided with a loading plate (202). The bottom front and rear sides of the loading plate (202) are provided with two sets of buffer positioning rods (204) that act on the circuit board through a fixing seat (203). The lifting component (205) is mounted on the support plate (3), and the top output end of the lifting component (205) is connected to a support plate (206) for carrying the circuit board to be diagnosed.

Images