Printed circuit board defect detection system based on virtual reality technology

Abstract

A virtual reality technology-based printed circuit board defect detection system relates to the technical field of printed circuit board detection, and comprises the following steps: shooting a printed circuit board according to initial shooting parameters, obtaining visual image data of the printed circuit board, and obtaining other modal data of the printed circuit board; performing feature analysis on the visual image data, and performing adaptive adjustment on shooting parameters of the printed circuit board according to a feature analysis result; performing adaptive denoising preprocessing on each pixel point in the grayscale image to obtain a denoised grayscale image; performing feature extraction and detection on the grayscale image to obtain abnormal key features in the grayscale image; outputting types and severity coefficients of the abnormal key features; and visually displaying the type, the severity coefficient and the coverage area of the abnormal key features of the printed circuit board. According to the invention, the efficiency and accuracy of printed circuit board defect detection are obviously improved.

Description

Technical Field

[0001] The invention relates to the technical field of printed circuit board detection, in particular to a printed circuit board defect detection system based on virtual reality technology. Background Art

[0002] CN118261881B "A PCB circuit board defect detection method and system" includes: obtaining feature coordinates of feature positions in a standard design image; obtaining a circuit board image to be detected; obtaining at least one local difference image; obtaining a feature position consistent with the position coordinates of the local difference image; obtaining at least one conventional defect type corresponding to the target feature position; obtaining at least one feature defect image corresponding to the target conventional defect type; obtaining all combinations of at least one target feature defect image corresponding to the target feature position; obtaining a fused image; obtaining a fused image whose difference with the local difference image is less than a preset value; and obtaining the defect type of the PCB circuit board image to be detected.

[0003] CN118261905B "AI-enabled flexible circuit board defect detection method and detection equipment" includes: generating image samples of the flexible circuit board under different deformation conditions; constructing a deep learning network model for generating a corrected image based on the input restored image of the flexible circuit board, and the loss function of the deep learning network model includes a deformation degree term, a structural similarity term and a mean square error term; training the deep learning network model through the image samples; inputting the current image of the flexible circuit board into the trained deep learning network model to obtain a corrected target image; and performing defect judgment based on the target image.

[0004] As a key component of electronic equipment, the quality of printed circuit boards (PCBs) directly affects the performance and reliability of the entire electronic product. In the PCB production process, various defects are prone to occur due to the complex process, such as short circuits, broken circuits, open circuits, missing or offset components, etc. Traditional PCB defect detection methods mainly rely on manual visual inspection and automated inspection systems based on machine vision. Manual visual inspection is inefficient and prone to missed inspections and misjudgments due to factors such as fatigue and experience differences of inspectors; although the automated inspection system based on machine vision has improved the inspection efficiency and accuracy to a certain extent, the inspection results are usually presented in the form of two-dimensional images or data reports. Inspectors still face great difficulties in analyzing complex defects and it is difficult to intuitively obtain comprehensive information about the defects.

[0005] Virtual reality (VR) technology has developed rapidly in recent years. It can create highly realistic virtual environments and provide immersive interactive experiences. Applying VR technology to the field of PCB defect detection is expected to solve the above-mentioned problems of traditional detection methods, improve the efficiency and accuracy of PCB defect detection, and provide strong support for PCB production quality control. Summary of the invention

[0006] In order to solve the above technical problems, the object of the present invention is to provide a printed circuit board defect detection system based on virtual reality technology, including a monitoring center, wherein the monitoring center is communicatively connected to a multimodal data acquisition module, an image enhancement module, an adaptive denoising module, an abnormal feature detection module, a multimodal defect assessment module and a visualization module;

[0007] The multimodal data acquisition module is used to shoot the printed circuit board according to the initial shooting parameters, obtain visual image data of the printed circuit board, and obtain other modal data of the printed circuit board;

[0008] The image enhancement module is used to perform feature analysis on the visual image data and adaptively adjust the shooting parameters of the printed circuit board according to the feature analysis results;

[0009] The adaptive denoising module is used to perform adaptive denoising preprocessing on each pixel in the grayscale image to obtain a grayscale image with noise removed;

[0010] The abnormal feature detection module is used to extract and detect features of grayscale images to obtain abnormal key features in grayscale images;

[0011] The multimodal defect assessment module is used to output the type and severity coefficient of abnormal key features;

[0012] The visualization module is used to visualize the type, severity coefficient and coverage area of ​​the abnormal key features of the printed circuit board.

[0013] Furthermore, the image enhancement module performs feature analysis on the visual image data, and the process of adaptively adjusting the shooting parameters of the printed circuit board according to the feature analysis results includes:

[0014] The visual image data of the printed circuit board is gray-scaled to generate a gray-scale image, the pixel intensity value of each pixel in the gray-scale image is statistically analyzed to generate a gray-scale image histogram, feature extraction is performed on the gray-scale image histogram, the pixel intensity mean and the pixel intensity distribution range are obtained, the pixel intensity mean and the pixel intensity distribution range are compared with the corresponding preset threshold intervals, and if the pixel intensity mean or the pixel intensity distribution range is not within the corresponding preset threshold interval, the shooting parameters of the printed circuit board are adaptively adjusted.

[0015] Furthermore, the process of adaptively adjusting the shooting parameters of the printed circuit board includes:

[0016] An adaptive fuzzy control model is constructed, wherein the input of the adaptive fuzzy control model is the pixel intensity mean and the pixel intensity distribution range, and the output is the shooting parameters, wherein the shooting parameters include brightness and contrast, a membership function of the adaptive fuzzy control model, a plurality of brightness threshold intervals and a contrast threshold interval are preset, and a shooting parameter lookup table is constructed, wherein the shooting parameter lookup table includes shooting parameters corresponding to different pixel intensity means and pixel intensity distribution ranges, and the shooting parameter lookup table is used as a fuzzy control rule table of the adaptive fuzzy control model;

[0017] The pixel intensity mean and the pixel intensity distribution range are input into the adaptive fuzzy control model, and the fuzzy comprehensive evaluation is performed through the adaptive fuzzy control model to obtain the brightness and contrast, update the initial shooting parameters of the printed circuit board according to the brightness and contrast, and re-collect the visual image data of the printed circuit board.

[0018] Furthermore, the process of constructing a shooting parameter lookup table includes:

[0019] Preset a training printed circuit board and an original grayscale image, simulate shooting the training printed circuit board under different lighting conditions according to the initial shooting parameters, generate grayscale images corresponding to the initial shooting parameters under different lighting conditions, obtain the pixel intensity mean and pixel intensity distribution range corresponding to the grayscale image, compare the grayscale image corresponding to the initial shooting parameters under different lighting conditions with the original grayscale image, obtain the structural similarity of the grayscale images corresponding to the initial shooting parameters under different lighting conditions, screen out grayscale images whose structural similarity is less than a preset structural similarity threshold, and mark the grayscale image as a key grayscale image;

[0020] Obtaining the illumination conditions corresponding to each key grayscale image, performing simulated shooting of the training printed circuit board under the illumination conditions corresponding to different key grayscale images by using different shooting parameters, generating grayscale images corresponding to different shooting parameters under the illumination conditions corresponding to different key grayscale images, obtaining the structural similarity of the grayscale images corresponding to different shooting parameters under the illumination conditions corresponding to different key grayscale images, screening out grayscale images whose structural similarity is greater than or equal to a preset structural similarity threshold value from the grayscale images corresponding to different shooting parameters under the illumination conditions corresponding to different key grayscale images, and marking the grayscale images as qualified grayscale images;

[0021] The shooting parameters and the key grayscale image corresponding to the qualified grayscale image are obtained, and the pixel intensity mean and the pixel intensity distribution range corresponding to the key grayscale image are matched with the shooting parameters corresponding to the qualified grayscale image and stored in a shooting parameter lookup table.

[0022] Furthermore, the adaptive denoising module performs adaptive denoising preprocessing on each pixel in the grayscale image to obtain a grayscale image with noise removed, including:

[0023] Preset the neighborhood range and type, obtain the pixel intensity variance and grayscale co-occurrence matrix within the neighborhood range of each pixel point in the grayscale image, perform texture feature extraction on the grayscale co-occurrence matrix within the neighborhood range of each pixel point, obtain the entropy value within the neighborhood range of each pixel point, obtain the pixel point with the lowest entropy value, and mark the pixel intensity variance within the neighborhood range of the pixel point as the global noise level;

[0024] The pixel intensity variance within the neighborhood of each pixel in the grayscale image is compared with the global noise level to obtain the noise difference value within the neighborhood of each pixel, and the filtering parameters of each pixel are set according to the noise difference value within the neighborhood of each pixel. According to the filtering parameters of each pixel, Gaussian filtering is performed on each pixel using the Gaussian filtering method to obtain a grayscale image with noise removed.

[0025] Furthermore, the abnormal feature detection module extracts and detects features of the grayscale image, and the process of obtaining abnormal key features in the grayscale image includes:

[0026] Perform feature extraction on the grayscale image to obtain key features of the grayscale image, preset a standard grayscale image of the printed circuit board, obtain standard key features of the standard grayscale image, match and compare the key features of the grayscale image with the standard key features of the standard grayscale image, obtain key feature similarity between the grayscale image and the standard grayscale image, compare the key feature similarity with a preset key feature similarity threshold, if the key feature similarity is less than the key feature similarity threshold, mark the key feature as an abnormal key feature, and obtain the coverage area of ​​the abnormal key feature in the grayscale image.

[0027] Furthermore, the process of the multimodal defect assessment module outputting the type and severity coefficient of the abnormal key feature includes:

[0028] Acquire other modal data within the coverage area of ​​the abnormal key feature, perform feature extraction on the other modal data, acquire other modal features, and associate the other modal features with the abnormal key feature;

[0029] A multimodal defect assessment model is constructed based on deep learning, the associated abnormal key features and other modal features are input into the multimodal defect assessment model, and the type and severity coefficient of the abnormal key features are output according to the multimodal defect assessment model.

[0030] Furthermore, the process of visualizing the type, severity coefficient and coverage area of ​​the abnormal key features of the printed circuit board by the visualization module includes:

[0031] The printed circuit board is 3D modeled using 3D modeling technology to generate a virtual model of the printed circuit board. When the key features of the grayscale image of the printed circuit board are marked as abnormal key features, the type, severity coefficient and coverage area of ​​the abnormal key features are obtained, and the coverage area of ​​the virtual model of the printed circuit board is highlighted and the type is marked according to the type, severity coefficient and coverage area of ​​the abnormal key features.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. Accurate image quality optimization: The image enhancement module conducts in-depth analysis of visual image data and adaptively adjusts shooting parameters, effectively overcoming the image quality fluctuation problem caused by different printed circuit board materials, colors and lighting conditions. Based on the comparison of the pixel intensity mean and distribution range with the preset threshold range, the adaptive fuzzy control model is used to accurately adjust shooting parameters such as brightness and contrast. This intelligent adjustment mechanism ensures that high-quality, high-definition and distinctive visual images can be obtained under various complex working conditions, providing a solid and reliable data foundation for subsequent defect detection, greatly improving the accuracy and reliability of defect detection, and reducing misjudgments and missed judgments caused by poor image quality.

[0034] 2. Efficient adaptive denoising capability: The adaptive denoising module adopts an innovative denoising method based on pixel neighborhood feature analysis. By calculating the variance and entropy of pixel intensity within the pixel neighborhood, the global noise level is accurately determined, and personalized filter parameters are set for each pixel for Gaussian filtering. This adaptive denoising method can effectively remove noise while retaining the image's detail information and edge features to the greatest extent, avoiding the image blur or detail loss caused by over-smoothing in traditional denoising methods, making the defect features in the image clearer and more discernible, which helps to improve the accuracy and efficiency of abnormal feature detection.

[0035] 3. Deep defect assessment based on multimodal data fusion: The multimodal defect assessment module combines the abnormal key features in the visual image with other modal data (such as electrical test data, temperature data, etc.) and builds a multimodal defect assessment model based on deep learning, thus achieving a comprehensive and in-depth assessment of the nature and severity coefficient of the defect. The fusion of multimodal data gives full play to the complementary advantages of different detection methods and can reveal the inherent nature and potential impact of defects from multiple dimensions. The application of deep learning models enables it to automatically learn and mine the complex correlations and characteristic patterns in multimodal data, so as to more accurately judge the defect type (such as short circuit, open circuit, component performance degradation, etc.) and quantitatively assess the severity of the defect. This precise defect assessment result provides a valuable basis for subsequent maintenance decisions, quality control and production process optimization, which helps to reduce production costs, improve product quality and production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 This is a schematic diagram of a printed circuit board defect detection system based on virtual reality technology according to an embodiment of the present application. DETAILED DESCRIPTION

[0037] The following is a clear and complete description of the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0038] like Figure 1 As shown, a printed circuit board defect detection system based on virtual reality technology includes a monitoring center, wherein the monitoring center is communicatively connected to a multimodal data acquisition module, an image enhancement module, an adaptive denoising module, an abnormal feature detection module, a multimodal defect assessment module and a visualization module;

[0039] The multimodal data acquisition module is used to shoot the printed circuit board according to the initial shooting parameters, obtain visual image data of the printed circuit board, and obtain other modal data of the printed circuit board;

[0040] The image enhancement module is used to perform feature analysis on the visual image data and adaptively adjust the shooting parameters of the printed circuit board according to the feature analysis results;

[0041] The adaptive denoising module is used to perform adaptive denoising preprocessing on each pixel in the grayscale image to obtain a grayscale image with noise removed;

[0042] The abnormal feature detection module is used to extract and detect features of grayscale images to obtain abnormal key features in grayscale images;

[0043] The multimodal defect assessment module is used to output the type and severity coefficient of abnormal key features;

[0044] The visualization module is used to visualize the type, severity coefficient and coverage area of ​​the abnormal key features of the printed circuit board.

[0045] It should be further explained that, in a specific implementation process, the image enhancement module performs feature analysis on the visual image data, and the process of adaptively adjusting the shooting parameters of the printed circuit board according to the feature analysis results includes:

[0046] The visual image data of the printed circuit board is grayed to generate a gray image. The pixel intensity value of each pixel in the gray image is statistically analyzed. The pixel intensity value is between 0 (black) and 255 (white). A gray image histogram is generated. The horizontal axis of the gray image histogram is the pixel intensity value (0-255), and the vertical axis is the frequency of occurrence of the pixel intensity value. Feature extraction is performed on the gray image histogram to obtain the pixel intensity mean and the pixel intensity distribution range. The pixel intensity mean and the pixel intensity distribution range are compared with the corresponding preset threshold intervals. If the pixel intensity mean or the pixel intensity distribution range is not within the corresponding preset threshold interval, the shooting parameters of the printed circuit board are adaptively adjusted. The contrast and brightness of the image are adaptively adjusted to adapt to changes in lighting conditions of different printed circuit boards, ensuring that clear and high-quality images are always obtained for subsequent detection.

[0047] It should be further explained that, in a specific implementation process, the process of adaptively adjusting the shooting parameters of the printed circuit board includes:

[0048] An adaptive fuzzy control model is constructed, wherein the input of the adaptive fuzzy control model is the pixel intensity mean and the pixel intensity distribution range, and the output is the shooting parameters, wherein the shooting parameters include brightness and contrast, a membership function of the adaptive fuzzy control model, a plurality of brightness threshold intervals and a contrast threshold interval are preset, and a shooting parameter lookup table is constructed, wherein the shooting parameter lookup table includes shooting parameters corresponding to different pixel intensity means and pixel intensity distribution ranges, and the shooting parameter lookup table is used as a fuzzy control rule table of the adaptive fuzzy control model;

[0049] The pixel intensity mean and the pixel intensity distribution range are input into the adaptive fuzzy control model, and the fuzzy comprehensive evaluation is performed through the adaptive fuzzy control model to obtain the brightness and contrast, update the initial shooting parameters of the printed circuit board according to the brightness and contrast, and re-collect the visual image data of the printed circuit board.

[0050] It should be further explained that, in the specific implementation process, the process of constructing the shooting parameter lookup table includes:

[0051] Preset a training printed circuit board and an original grayscale image of the training printed circuit board, simulate shooting the training printed circuit board under different lighting conditions according to the initial shooting parameters, generate grayscale images corresponding to the initial shooting parameters under different lighting conditions, obtain the pixel intensity mean and pixel intensity distribution range corresponding to the grayscale image, compare the grayscale image corresponding to the initial shooting parameters under different lighting conditions with the original grayscale image, obtain the structural similarity of the grayscale images corresponding to the initial shooting parameters under different lighting conditions, screen out grayscale images whose structural similarity is less than a preset structural similarity threshold, and mark the grayscale image as a key grayscale image;

[0052] Obtaining the illumination conditions corresponding to each key grayscale image, performing simulated shooting of the training printed circuit board under the illumination conditions corresponding to the different key grayscale images by using different shooting parameters, generating grayscale images corresponding to the different shooting parameters under the illumination conditions corresponding to the different key grayscale images, comparing the grayscale images corresponding to the different shooting parameters under the illumination conditions corresponding to the different key grayscale images with the original grayscale images, obtaining the structural similarity of the grayscale images corresponding to the different shooting parameters under the illumination conditions corresponding to the different key grayscale images, screening out grayscale images whose structural similarity is greater than or equal to a preset structural similarity threshold from the grayscale images corresponding to the different shooting parameters under the illumination conditions corresponding to the different key grayscale images, and marking the grayscale images as qualified grayscale images;

[0053] The shooting parameters and the key grayscale image corresponding to the qualified grayscale image are obtained, and the pixel intensity mean and the pixel intensity distribution range corresponding to the key grayscale image are matched with the shooting parameters corresponding to the qualified grayscale image and stored in a shooting parameter lookup table.

[0054] It should be further explained that, in the specific implementation process, the calculation formula of the structural similarity of the grayscale image is:

[0055] For a size of Grayscale image, assuming that the pixel intensity value of each pixel in the original grayscale image is , the pixel intensity value of each pixel in the grayscale image after re-collection is ,x=1,2,...,M,x=1,2,...,N;

[0056] ;

[0057] in, Indicates structural similarity, Indicates the maximum value of the pixel intensity value. In this embodiment, an 8-bit grayscale image is used, so =255.

[0058] It should be further explained that, in a specific implementation process, the adaptive denoising module performs adaptive denoising preprocessing on each pixel in the grayscale image, and the process of obtaining the grayscale image with noise removed includes:

[0059] Preset the neighborhood range and type, obtain the pixel intensity variance and grayscale co-occurrence matrix within the neighborhood range of each pixel point in the grayscale image, perform texture feature extraction on the grayscale co-occurrence matrix within the neighborhood range of each pixel point, obtain the entropy value within the neighborhood range of each pixel point, obtain the pixel point with the lowest entropy value, and mark the pixel intensity variance within the neighborhood range of the pixel point as the global noise level;

[0060] A low entropy value indicates a simpler and more regular texture, and a smoother area. Since the pixel value changes in the smooth area are mainly caused by noise, the noise intensity is directly reflected by the pixel intensity variance in the smooth area, which improves the accuracy of the subsequent noise denoising results;

[0061] The pixel intensity variance within the neighborhood of each pixel in the grayscale image is compared with the global noise level to obtain the noise difference value within the neighborhood of each pixel, and the filtering parameters of each pixel are set according to the noise difference value within the neighborhood of each pixel. According to the filtering parameters of each pixel, Gaussian filtering is performed on each pixel using the Gaussian filtering method to obtain a grayscale image with noise removed.

[0062] Among them, the calculation formula for obtaining the pixel intensity variance within the neighborhood range is:

[0063] ;

[0064] ;

[0065] in, represents the pixel intensity variance, d represents the pixel intensity mean, Represents pixel The pixel intensity value at , W represents the set of all pixel points in the neighborhood, and n1 represents the total number of pixel points in the neighborhood.

[0066] The process of setting the filtering parameters of each pixel point according to the noise difference value within the neighborhood of each pixel point includes:

[0067] A filtering parameter control model is pre-constructed, and the filtering parameter control model determines the filtering parameters of each pixel point according to the noise difference value within the neighborhood of each pixel point. The filtering parameters include the Gaussian kernel size and the standard deviation. For example, when the pixel intensity variance is close to the global noise level, it indicates that this area is mainly composed of noise, and the filtering parameter control model increases the σ value to enhance the smoothing effect; conversely, if the pixel intensity variance is much larger than the noise level, it indicates that there may be important image features such as edges, and the filtering parameter control model reduces the σ value to reduce the impact on these features.

[0068] It should be further explained that, in the specific implementation process, the abnormal feature detection module extracts and detects features of the grayscale image, and the process of obtaining abnormal key features in the grayscale image includes:

[0069] Feature extraction is performed on the grayscale image to obtain key features of the grayscale image, wherein the key features include geometric features (including shape, width, and direction of the circuit, position and size of the pad, position and contour of the component) and texture features (including contrast, correlation, energy, and entropy, etc.), a standard grayscale image of the printed circuit board is preset, feature extraction is performed on the standard grayscale image, standard key features of the standard grayscale image are obtained, and the key features of the grayscale image are matched and compared with the standard key features of the standard grayscale image. The matching and comparison process specifically includes pixel alignment of the grayscale image and the standard grayscale image using image alignment technology, comparing the geometric features and texture features of the grayscale image with the standard geometric features and standard texture features of the standard grayscale image in the same pixel coverage area, obtaining key feature similarity between the grayscale image and the standard grayscale image, comparing the key feature similarity with a preset key feature similarity threshold, and if the key feature similarity is less than the key feature similarity threshold, marking the key feature as an abnormal key feature, and obtaining the coverage area of ​​the abnormal key feature in the grayscale image.

[0070] It should be further explained that, in a specific implementation process, the process of the multimodal defect assessment module outputting the type and severity coefficient of the abnormal key feature includes:

[0071] Acquire other modal data (including electrical test data, temperature data, and ultrasonic data) within the coverage area of ​​the abnormal key feature, perform feature extraction on the other modal data, acquire other modal features, and associate the other modal features with the abnormal key feature;

[0072] A multimodal defect assessment model is constructed based on deep learning, the associated abnormal key features and other modal features are input into the multimodal defect assessment model, and the type and severity coefficient of the abnormal key features are output according to the multimodal defect assessment model.

[0073] Among them, other modal features include the characteristics of electrical parameters, such as the rate of change of resistance, the deviation value of capacitance, the fluctuation range of inductance, etc. These characteristics can reflect the conductivity of the circuit, the performance of the components, and whether there are defects such as short circuits or open circuits; temperature distribution characteristics, such as the highest temperature, the lowest temperature, the average temperature, the temperature gradient, etc. Abnormal temperature distribution can indicate local overheating caused by circuit overload, component damage or poor contact; ultrasonic signal characteristics, such as the amplitude, frequency, and arrival time of the reflected wave, these characteristics can be used to determine whether there are delaminations, voids or other internal structural defects inside the circuit board;

[0074] It should be further explained that, in the specific implementation process, the process of building a multimodal defect assessment model based on deep learning includes:

[0075] Generate a large amount of annotated data in advance, the annotated data includes different types of defect samples, as well as corresponding abnormal key features and other modal data and known defect properties and severity, use the annotated data as a training set and a test set, input the training set into the multimodal defect assessment model for training until the loss function training is stable, and save the model parameters, test the multimodal defect assessment model with the test set until it meets the preset requirements, and output the multimodal defect assessment model, the multimodal defect assessment model outputs the nature and severity of the defect according to the input abnormal key features and other modal data;

[0076] Among them, this embodiment selects a multimodal deep belief network (DBN) as a deep learning architecture. After determining the model architecture, the next step is to define a loss function. Since the goal of the multimodal defect assessment model is to output the nature and severity of the defect, where the defect nature is a classification task (such as short circuit, open circuit, component damage, etc.), the cross entropy loss function can be used to measure the classification prediction error, and the defect severity is a regression task (such as severity is represented by a numerical value between 0 and 1), and the mean square error (MSE) loss function can be used to measure the difference between the predicted severity and the actual severity. Therefore, this embodiment combines the cross entropy loss with the mean square error loss to generate a classification and regression combined loss function (for simultaneously predicting the defect nature and severity). The classification and regression combined loss function is specifically: ;in is the cross entropy loss, Mean square error loss, is a weight parameter used to balance the importance of classification and regression tasks;

[0077] The prepared training set is then input into the selected deep learning model to start training. During the training process, the weights are continuously updated through the back-propagation algorithm, so that the loss function gradually decreases until a stable state is reached. During this period, techniques such as Early Stopping are used to avoid overfitting. In addition to the basic training process, various parameters of the model are tuned through Grid Search, including learning rate, batch size, regularization coefficient, etc.

[0078] After the model training is completed and the parameters are adjusted, the final evaluation is performed through the test set to obtain the evaluation results of the model, which include classification indicators such as accuracy, recall, F1 score, etc. Based on the evaluation results on the test set, it is judged whether the model has reached the expected standards. If the requirements are met, the model parameters are saved and ready for deployment; if not ideal, it is necessary to return to a previous stage to re-examine issues such as data quality, model structure or training strategy.

[0079] It should be further explained that, in a specific implementation process, the process of the visualization module visually displaying the type, severity coefficient and coverage area of ​​the abnormal key features of the printed circuit board includes:

[0080] The printed circuit board is 3D modeled by using 3D modeling technology to generate a virtual model of the printed circuit board. The process of 3D modeling of the printed circuit board by using 3D modeling technology includes using advanced rendering technology, such as ray tracing rendering algorithm, to render the virtual model so that it presents a highly realistic 3D effect, including the material texture, color texture and lighting effect of the circuit board, etc., to provide an immersive visual experience for the inspectors. When the key features of the grayscale image of the printed circuit board are marked as abnormal key features, the type, severity coefficient and coverage area of ​​the abnormal key features are obtained, and the coverage area of ​​the virtual model of the printed circuit board is highlighted and the type is marked according to the type, severity coefficient and coverage area of ​​the abnormal key features. For example, for a short circuit defect, the short-circuited line segment can be highlighted in the virtual model with a specific color or flashing effect; for a component missing defect, the outline or mark of the missing component can be displayed at the corresponding position.

[0081] The above embodiments are only used to illustrate the technical method of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A printed circuit board defect detection system based on virtual reality technology, characterized in that: The monitoring center includes a multimodal data acquisition module, an image enhancement module, an adaptive denoising module, an abnormal feature detection module, a multimodal defect assessment module and a visualization module in communication connection with the monitoring center; The multimodal data acquisition module is used to shoot the printed circuit board according to the initial shooting parameters, obtain visual image data of the printed circuit board, and obtain other modal data of the printed circuit board; The image enhancement module is used to perform feature analysis on the visual image data and adaptively adjust the shooting parameters of the printed circuit board according to the feature analysis results; The adaptive denoising module is used to perform adaptive denoising preprocessing on each pixel in the grayscale image to obtain a grayscale image with noise removed; The abnormal feature detection module is used to extract and detect features of grayscale images to obtain abnormal key features in grayscale images; The multimodal defect assessment module is used to output the type and severity coefficient of abnormal key features; The visualization module is used to visualize the type, severity coefficient and coverage area of ​​the abnormal key features of the printed circuit board.

2. A printed circuit board defect detection system based on virtual reality technology according to claim 1, characterized in that: The image enhancement module performs feature analysis on the visual image data, and the process of adaptively adjusting the shooting parameters of the printed circuit board according to the feature analysis results includes: The visual image data of the printed circuit board is gray-scaled to generate a gray-scale image, the pixel intensity value of each pixel in the gray-scale image is statistically analyzed to generate a gray-scale image histogram, feature extraction is performed on the gray-scale image histogram, the pixel intensity mean and the pixel intensity distribution range are obtained, the pixel intensity mean and the pixel intensity distribution range are compared with the corresponding preset threshold intervals, and if the pixel intensity mean or the pixel intensity distribution range is not within the corresponding preset threshold interval, the shooting parameters of the printed circuit board are adaptively adjusted.

3. A printed circuit board defect detection system based on virtual reality technology according to claim 2, characterized in that: The process of adaptively adjusting the shooting parameters of the printed circuit board includes: An adaptive fuzzy control model is constructed, wherein the input of the adaptive fuzzy control model is the pixel intensity mean and the pixel intensity distribution range, and the output is the shooting parameters, wherein the shooting parameters include brightness and contrast, a membership function of the adaptive fuzzy control model, a plurality of brightness threshold intervals and a contrast threshold interval are preset, and a shooting parameter lookup table is constructed, wherein the shooting parameter lookup table includes shooting parameters corresponding to different pixel intensity means and pixel intensity distribution ranges, and the shooting parameter lookup table is used as a fuzzy control rule table of the adaptive fuzzy control model; The pixel intensity mean and the pixel intensity distribution range are input into the adaptive fuzzy control model, and the fuzzy comprehensive evaluation is performed through the adaptive fuzzy control model to obtain the brightness and contrast, update the initial shooting parameters of the printed circuit board according to the brightness and contrast, and re-collect the visual image data of the printed circuit board.

4. A printed circuit board defect detection system based on virtual reality technology according to claim 3, characterized in that: The process of building a shooting parameter lookup table includes: Preset a training printed circuit board and an original grayscale image, simulate shooting the training printed circuit board under different lighting conditions according to the initial shooting parameters, generate grayscale images corresponding to the initial shooting parameters under different lighting conditions, obtain the pixel intensity mean and pixel intensity distribution range corresponding to the grayscale image, compare the grayscale image corresponding to the initial shooting parameters under different lighting conditions with the original grayscale image, obtain the structural similarity of the grayscale images corresponding to the initial shooting parameters under different lighting conditions, screen out grayscale images whose structural similarity is less than a preset structural similarity threshold, and mark the grayscale image as a key grayscale image; Obtaining the illumination conditions corresponding to each key grayscale image, performing simulated shooting of the training printed circuit board under the illumination conditions corresponding to different key grayscale images by using different shooting parameters, generating grayscale images corresponding to different shooting parameters under the illumination conditions corresponding to different key grayscale images, obtaining the structural similarity of the grayscale images corresponding to different shooting parameters under the illumination conditions corresponding to different key grayscale images, screening out grayscale images whose structural similarity is greater than or equal to a preset structural similarity threshold value from the grayscale images corresponding to different shooting parameters under the illumination conditions corresponding to different key grayscale images, and marking the grayscale images as qualified grayscale images; The shooting parameters and the key grayscale image corresponding to the qualified grayscale image are obtained, and the pixel intensity mean and the pixel intensity distribution range corresponding to the key grayscale image are matched with the shooting parameters corresponding to the qualified grayscale image and stored in a shooting parameter lookup table.

5. A printed circuit board defect detection system based on virtual reality technology according to claim 4, characterized in that: The adaptive denoising module performs adaptive denoising preprocessing on each pixel in the grayscale image to obtain a grayscale image with noise removed, comprising: Preset the neighborhood range and type, obtain the pixel intensity variance and grayscale co-occurrence matrix within the neighborhood range of each pixel point in the grayscale image, perform texture feature extraction on the grayscale co-occurrence matrix within the neighborhood range of each pixel point, obtain the entropy value within the neighborhood range of each pixel point, obtain the pixel point with the lowest entropy value, and mark the pixel intensity variance within the neighborhood range of the pixel point as the global noise level; The pixel intensity variance within the neighborhood of each pixel in the grayscale image is compared with the global noise level to obtain the noise difference value within the neighborhood of each pixel, and the filtering parameters of each pixel are set according to the noise difference value within the neighborhood of each pixel. According to the filtering parameters of each pixel, Gaussian filtering is performed on each pixel using the Gaussian filtering method to obtain a grayscale image with noise removed.

6. A printed circuit board defect detection system based on virtual reality technology according to claim 5, characterized in that: The abnormal feature detection module extracts and detects features of the grayscale image, and the process of obtaining abnormal key features in the grayscale image includes: Perform feature extraction on the grayscale image to obtain key features of the grayscale image, preset a standard grayscale image of the printed circuit board, obtain standard key features of the standard grayscale image, match and compare the key features of the grayscale image with the standard key features of the standard grayscale image, obtain key feature similarity between the grayscale image and the standard grayscale image, compare the key feature similarity with a preset key feature similarity threshold, if the key feature similarity is less than the key feature similarity threshold, mark the key feature as an abnormal key feature, and obtain the coverage area of ​​the abnormal key feature in the grayscale image.

7. A printed circuit board defect detection system based on virtual reality technology according to claim 6, characterized in that: The process of the multimodal defect assessment module outputting the type and severity coefficient of abnormal key features includes: Acquire other modal data within the coverage area of ​​the abnormal key feature, perform feature extraction on the other modal data, acquire other modal features, and associate the other modal features with the abnormal key feature; A multimodal defect assessment model is constructed based on deep learning, the associated abnormal key features and other modal features are input into the multimodal defect assessment model, and the type and severity coefficient of the abnormal key features are output according to the multimodal defect assessment model.

8. The printed circuit board defect detection system based on virtual reality technology according to claim 7 is characterized in that: The process of visualizing the type, severity coefficient and coverage area of ​​the abnormal key features of the printed circuit board by the visualization module includes: The printed circuit board is 3D modeled using 3D modeling technology to generate a virtual model of the printed circuit board. When the key features of the grayscale image of the printed circuit board are marked as abnormal key features, the type, severity coefficient and coverage area of ​​the abnormal key features are obtained, and the coverage area of ​​the virtual model of the printed circuit board is highlighted and the type is marked according to the type, severity coefficient and coverage area of ​​the abnormal key features.

Images