Electromigration region detection method, device, computer equipment and storage medium

Abstract

The present application relates to an electromigration region detection method, device, computer equipment and storage medium. The method comprises: acquiring a circuit board image; performing edge detection on the circuit board image to obtain edge information of the circuit board image; identifying interference objects in the circuit board image based on the edge information; removing the interference objects from the circuit board image to obtain a target circuit board image without interference objects; and extracting the electromigration region in the target circuit board image through a visual saliency detection algorithm. The use of this method can improve the efficiency of electromigration region detection.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an electromigration region detection method, device, computer equipment and storage medium. Background Art

[0002] Electromigration has attracted widespread attention as a typical failure form of electronic products. Electromigration refers to the migration of metal atoms / ions along the direction of electron movement driven by electron wind force under high current density. Valleys are formed at the negative electrode due to the migration of materials, and hillocks are formed at the positive electrode due to the accumulation of materials. The formation of valleys and hillocks causes the circuit to open or short, resulting in device failure.

[0003] Currently, the identification of electromigration regions on devices mainly relies on manual visual inspection by inspectors, which is prone to errors or missed inspections and requires multiple inspections, resulting in low efficiency in electromigration region detection. Summary of the invention

[0004] Based on this, it is necessary to provide an electromigration region detection method, apparatus, computer equipment and storage medium to address the technical problem of low efficiency of electromigration region detection.

[0005] A method for detecting an electromigration region, the method comprising:

[0006] Get the circuit board image;

[0007] Performing edge detection on the circuit board image to obtain edge information of the circuit board image;

[0008] Identifying interference objects in the circuit board image based on the edge information;

[0009] Removing the interference object from the circuit board image to obtain a target circuit board image without the interference object;

[0010] The electromigration region in the target circuit board image is extracted by a visual saliency detection algorithm.

[0011] In one embodiment, the method further comprises:

[0012] Obtaining a preset number of sample circuit board images;

[0013] Annotating the sample circuit board image to obtain a training sample;

[0014] Inputting the training samples into a target detection network for training to obtain an interference object recognition model;

[0015] The identifying the interference object in the circuit board image based on the edge information includes:

[0016] Interference objects in the circuit board image are identified based on the edge information using the interference object identification model.

[0017] In one embodiment, before performing edge detection on the circuit board image, the method further includes:

[0018] A preprocessing operation is performed on the circuit board image, wherein the preprocessing operation includes image graying, morphological processing, and median filtering processing.

[0019] In one embodiment, extracting the electromigration region in the target circuit board image by using a visual saliency detection algorithm includes:

[0020] Converting the circuit board image without interference objects into a preset color space to obtain a circuit board image without interference objects in the preset color space;

[0021] The electromigration region in the circuit board image without interference objects in the preset color space is extracted by a visual saliency detection algorithm.

[0022] In one embodiment, after extracting the electromigration region in the target circuit board image by a visual saliency detection algorithm, the method further includes:

[0023] Calculating the area of ​​the electromigration region in the circuit board image;

[0024] Determining a short circuit risk level of a circuit board corresponding to the circuit board image according to the area;

[0025] Generate risk warning information corresponding to the short circuit risk level and provide a warning.

[0026] An electromigration region detection device, the device comprising:

[0027] An image acquisition module, used for acquiring a circuit board image;

[0028] An edge detection module, used to perform edge detection on the circuit board image to obtain edge information of the circuit board image;

[0029] An image recognition module, used to identify interference objects in the circuit board image based on the edge information;

[0030] An interference object removal module, used to remove the interference object from the circuit board image to obtain a target circuit board image without interference objects;

[0031] The electromigration region extraction module is used to extract the electromigration region in the target circuit board image through a visual saliency detection algorithm.

[0032] In one embodiment, the apparatus further comprises:

[0033] A sample circuit board image acquisition module is used to acquire a preset number of sample circuit board images;

[0034] A labeling module, used to label the sample circuit board image to obtain a training sample;

[0035] A model training module, used for inputting the training samples into a target detection network for training to obtain an interference object recognition model;

[0036] The image recognition module is further used for:

[0037] Interference objects in the circuit board image are identified based on the edge information using the interference object identification model.

[0038] In one embodiment, the apparatus further comprises:

[0039] The preprocessing module is used to perform preprocessing operations on the circuit board image, and the preprocessing operations include image graying, morphological processing, and median filtering processing.

[0040] In one embodiment, the electromigration region extraction module is further used to:

[0041] Converting the circuit board image without interference objects into a preset color space to obtain a circuit board image without interference objects in the preset color space;

[0042] The electromigration region in the circuit board image without interference objects in the preset color space is extracted by a visual saliency detection algorithm.

[0043] In one embodiment, the apparatus further comprises:

[0044] An area calculation module, used to calculate the area of ​​the electromigration region in the circuit board image;

[0045] A short circuit risk level determination module, used to determine the short circuit risk level of the circuit board corresponding to the circuit board image according to the area;

[0046] The prompt module is used to generate risk prompt information corresponding to the short circuit risk level and provide a prompt.

[0047] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above method when executing the computer program.

[0048] A computer-readable storage medium stores a computer program, which implements the steps of the above method when executed by a processor.

[0049] The above-mentioned electromigration area detection method, device, computer equipment and storage medium, after acquiring the circuit board image, perform edge detection on the circuit board image to obtain edge information of the circuit board image; identify interferers in the circuit board image based on the edge information; remove the interferers from the circuit board image to obtain a target circuit board image without interferers; and extract the electromigration area in the target circuit board image through a visual saliency detection algorithm, so that the electromigration area on the circuit board device can be detected without manual work, thereby improving the efficiency and accuracy of electromigration area detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 A diagram showing an application environment of an electromigration region detection method in an embodiment;

[0051] Figure 2 A schematic diagram of a flow chart of an electromigration region detection method in one embodiment;

[0052] Figure 3 An original circuit board image in one embodiment;

[0053] Figure 4 is a circuit board image to be detected in one embodiment;

[0054] Figure 5 is a detection result image in one embodiment;

[0055] Figure 6 is a schematic flow chart of an electromigration region detection method in another embodiment;

[0056] Figure 7 is a structural block diagram of an electromigration region detection device in one embodiment;

[0057] Figure 8 is a structural block diagram of an electromigration region detection device in another embodiment;

[0058] Fig. 9 is an internal structure diagram of a computer device in one embodiment;

[0059] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0061] The electromigration region detection method provided in this application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The electromigration area detection method can be applied to the terminal or the server, and can also be implemented through the interaction between the terminal and the server. Taking the application of this method to the terminal as an example, the terminal 102 obtains the circuit board image; performs edge detection on the circuit board image to obtain the edge information of the circuit board image; identifies the interference in the circuit board image based on the edge information; removes the interference from the circuit board image to obtain the target circuit board image without interference; and extracts the electromigration area in the target circuit board image through the visual saliency detection algorithm. Among them, the terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server 104 can be implemented with an independent server or a server cluster composed of multiple servers.

[0062] In one embodiment, Figure 2 As shown, a method for detecting an electromigration region is provided, and the method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0063] S202, obtaining a circuit board image.

[0064] The circuit board image is an image obtained by capturing images of electronic components such as circuit boards and chips to be inspected using an image acquisition device.

[0065] In one embodiment, an image acquisition device is used to acquire images of electronic components such as circuit boards and chips to be inspected, and an original circuit board image is obtained, and the acquired original circuit board image is sent to a terminal so that the terminal performs electromigration region detection on the original circuit board image. The image acquisition device may be a high-resolution camera.

[0066] In one embodiment, after acquiring the original circuit board image uploaded by the image acquisition device, the terminal can directly use the original circuit board image as the circuit board image to be detected, or extract the area to be detected from the original circuit board image and use the extracted area to be detected as the circuit board image to be detected. The terminal can use the GrabCut algorithm to perform image segmentation on the original circuit board image. During the image segmentation, the area to be detected is used as the foreground, and the area outside the area to be detected in the original circuit board image is used as the background, so that the area to be detected is extracted from the original circuit board image.

[0067] For example, Figure 3 An original circuit board image is shown. When segmenting the image, the area inside the rectangular frame (the area to be detected) is used as the foreground, and the area outside the rectangular frame is used as the background, so that the area to be detected is extracted from the circuit board image. The extraction result is shown in FIG. Figure 4 shown.

[0068] S204, performing edge detection on the circuit board image to obtain edge information of the circuit board image.

[0069] Among them, edge detection is the measurement, detection and positioning of grayscale changes in circuit board images. Edge information is an important feature of an image. An edge refers to a collection of pixels with step changes or roof changes in the grayscale of surrounding pixels. Edge detection algorithms include differential operators, Laplace Gaussian operators, and Canny operators.

[0070] In one embodiment, after acquiring a circuit board image (i.e., a circuit board image to be detected), the terminal uses an edge detection algorithm to perform edge detection on the circuit board image, thereby obtaining edge information of the circuit board image. Specifically, the terminal uses a canny operator to perform edge detection on the circuit board image, wherein the goal of the canny operator is to find an optimal edge detection algorithm, and the meaning of optimal edge detection is: good detection - the algorithm can identify as many actual edges in the image as possible, good positioning - the identified edges should be as close as possible to the actual edges in the actual image, minimum response - the edges in the image can only be identified once, and possible image noise should not be identified as edges.

[0071] S206: Identify interference objects in the circuit board image based on the edge information.

[0072] The interferer is an interferer region that affects the determination of the electromigration region, and specifically may be an interferer object that affects the determination of the electromigration region, such as characters in a circuit board image. Figure 4 The character "103" in is the interference object to be identified.

[0073] In one embodiment, after obtaining edge information of the circuit board image, the terminal uses the edge information as the image feature of the circuit board image, and uses a pre-trained interference recognition model to recognize interference in the circuit board image based on the image feature. The interference recognition model can be pre-trained based on the YOLOv3 network. YOLOv3 is a target detection algorithm based on a regression model.

[0074] S208, removing the interference object from the circuit board image to obtain a target circuit board image without the interference object.

[0075] In one embodiment, after identifying the interference object in the circuit board image, the terminal removes the identified interference object from the circuit board image, wherein the terminal may use the GrabCut algorithm to remove the identified interference object from the circuit board image.

[0076] Specifically, the GrabCut algorithm is used to perform image segmentation on the circuit board image. During image segmentation, the identified interference object is used as the background, and the area outside the interference object in the circuit board image is used as the foreground, so that the interference object is removed from the circuit board image to obtain a target circuit board image without interference objects.

[0077] S210, extracting the electromigration region in the target circuit board image through a visual saliency detection algorithm.

[0078] The visual saliency detection algorithm is used to extract salient regions (i.e., regions of human interest) in an image by simulating human visual characteristics. The electromigration region in this application is the salient region.

[0079] Specifically, the terminal first calculates the color mean of the target circuit board image, calculates the second norm between the color value and the color mean for each pixel in the target circuit board image, generates a saliency map based on the second norm corresponding to each pixel in the target circuit board image, and extracts the electromigration area from the saliency map. Generating a saliency image based on the second norm corresponding to each pixel in the target circuit board image includes: determining the second norm corresponding to each pixel in the target circuit board image as the color value of the saliency image to be generated.

[0080] In the above embodiment, after acquiring the circuit board image, the terminal performs edge detection on the circuit board image to obtain edge information of the circuit board image; identifies interference objects in the circuit board image based on the edge information; removes interference objects from the circuit board image to obtain a target circuit board image without interference objects; and extracts the electromigration area in the target circuit board image through a visual saliency detection algorithm, thereby detecting the electromigration area on the circuit board device without manual work, thereby improving the efficiency and accuracy of electromigration area detection.

[0081] In one embodiment, the electromigration region detection method further includes the following steps: obtaining a preset number of sample circuit board images; annotating the sample circuit board images to obtain training samples; and inputting the training samples into a target detection network for training to obtain an interference object recognition model. The target detection network may be a YOLOv3 network.

[0082] Specifically, the terminal can capture the sample circuit board image through a high-resolution camera, or download the sample circuit board image from the Internet, and after obtaining the sample circuit board image, use a rectangular bounding box to annotate the sample circuit board image, and the annotated area is the interference area in the sample circuit board image. For example, the interference area in the sample circuit board image is marked as category 1, and the area outside the interference is marked as category 2.

[0083] In the above embodiment, the terminal obtains a preset number of sample circuit board images; annotates the sample circuit board images to obtain training samples, and inputs the training samples into the target detection network for training, thereby obtaining an interference recognition model, and then identifies the interference area in the circuit board image through the interference recognition model, thereby realizing the extraction of the electromigration area, so that the electromigration area on the circuit board device can be detected without manual work, thereby improving the efficiency and accuracy of the electromigration area detection.

[0084] In one embodiment, before the terminal performs edge detection on the circuit board image, it may also perform preprocessing operations on the circuit board image, and the preprocessing operations include image graying, morphological processing, and median filtering.

[0085] Grayscale is to convert a color image into a grayscale image, and transform three-dimensional data into one-dimensional data, so as to reduce the amount of later operations and reduce the complexity of operations. Specifically, the following image grayscale formula is used in this application to perform image grayscale operations on the circuit board image. The image grayscale formula is as follows:

[0086] f(i,j)=0.30R(i,j)+0.59G(i,j)+0.11B(i,j)

[0087] Among them, f(i,j) is the pixel value of the pixel with coordinates (i,j) in the circuit board image after the image is grayed, and R(i,j), G(i,j) and B(i,j) are the color values ​​corresponding to the three color channels R, G and B in the RGB color space of the pixel with coordinates (i,j) in the circuit board image before the image is grayed.

[0088] Morphological processing refers to extracting components of interest from image regions to depict the most essential shapes and structures of each region in the image. It is often used to enhance images, extract connected components, fill regions, etc. Basic morphological processing includes erosion, dilation, opening and closing operations.

[0089] Dilation can make the connected area in the image larger. The shape and size of the structural element will affect the result of the dilation. The dilation operation is mainly used to fill the gaps and connect the disconnected connected domains. Erosion can reduce the connected domains in the image and ablate the boundaries of the image structure. Similarly, the shape and size of the structural element will affect the result of the erosion. The erosion operation is mainly used to disconnect and eliminate small parts. The opening operation is to erode the image first and then dilate it; the closing operation is to dilate the image first and then erode it. The morphological processing performed on the circuit board image in this application can be a closing operation. The specific morphological dilation formula, morphological erosion formula and morphological closing formula used are as follows:

[0090]

[0091]

[0092]

[0093] Where F is the circuit board image, b is the structural element used for morphological expansion, Indicates that the pixel with coordinates (i, j) in the circuit board image is morphologically expanded. It means that the pixel with coordinates (i, j) in the circuit board image is morphologically eroded, and s and t are the morphological processing offsets corresponding to i and j respectively.

[0094] Median filtering is a nonlinear smoothing technique used to eliminate noise introduced during image acquisition and transmission. The formula used for median filtering is as follows:

[0095] g(i,j)=med{f(ik,jl),(k,l∈W)}

[0096] Among them, g(i,j) is the pixel value corresponding to the pixel point with coordinates (i,j) in the circuit board image after median filtering, k and l are the filter offsets corresponding to i and j respectively, and W is the filter offset set.

[0097] In the above embodiment, before the terminal performs edge detection on the circuit board image, the circuit board image can be preprocessed to effectively remove noise and retain effective image information, thereby reducing the amount of calculation in the electromigration area detection process and improving the efficiency of electromigration area detection.

[0098] In one embodiment, the terminal extracts the electromigration area in the target circuit board image through a visual saliency detection algorithm, including: converting the circuit board image without interference objects to a preset color space to obtain the circuit board image without interference objects in the preset color space; and extracting the electromigration area in the circuit board image without interference objects in the preset color space through a visual saliency detection algorithm.

[0099] Specifically, the circuit board image obtained by the terminal is an image in the RGB color space, that is, the target circuit board image without interference obtained by the terminal is also an image in the RGB color space. After obtaining the target circuit board image, the terminal performs image space conversion on the target circuit board image, specifically converting the target circuit board image from the RGB color space to the LAB color space, that is, converting the RGB target circuit board image to the LAB target circuit board image. After obtaining the LAB target circuit board image, the terminal can also perform Gaussian kernel blurring on the LAB target image to obtain the blurred target circuit board image, and calculate the color mean of the target circuit board image in the LAB color space, calculate the second norm between the color value and the color mean of each pixel in the blurred target circuit board image, generate a saliency map based on the second norm corresponding to each pixel in the blurred target circuit board image, and extract the electromigration area from the saliency map. Among them, the calculation formula of the second norm is as follows:

[0100] S(i,j)=||I u -I whc (i,j)||2

[0101] In the above formula, S(i,j) is the binary norm corresponding to the pixel point with coordinates (i,j) in the target circuit board image after blur processing, and I u is the color mean of the target circuit board image of LAB, I whc (i, j) is the color value corresponding to the pixel point (i, j) in the target circuit board image after blur processing.

[0102] In the above embodiment, the terminal obtains a circuit board image without interference objects in the preset color space by converting the circuit board image without interference objects into a preset color space, so that in the preset color space, the circuit board image without interference objects can be extracted more quickly through a visual saliency detection algorithm, thereby improving the efficiency of electromigration area detection.

[0103] In one embodiment, after obtaining the saliency map through the visual saliency detection algorithm, the terminal can also perform binarization and median filtering on the saliency map to obtain a processed saliency map, and use the processed saliency map as a mask to align and superimpose the processed saliency map and the circuit board image to obtain a detection result image with saliency marked electromigration areas. The formula used to binarize the saliency map is:

[0104]

[0105] Among them, value(i,j) is the grayscale value (color value) corresponding to the pixel point with coordinates (i,j) in the saliency map after binarization, and gray(i,j) is the grayscale value (color value) corresponding to the pixel point with coordinates (i,j) in the saliency map before binarization.

[0106] In the above embodiment, the terminal obtains a processed saliency map by binarizing and median filtering the saliency map, and uses the processed saliency map as a mask to align and superimpose the processed saliency map and the circuit board image to obtain a detection result image with the electromigration area significantly marked, thereby intuitively displaying the position of the electromigration area in the circuit board image.

[0107] In one embodiment, during the training of the interference object recognition model, after regressing the prediction frame, the terminal records the rotation angle and prediction frame parameters of the prediction frame, wherein the prediction frame parameters include the center point coordinates, width and height of the prediction frame, and before using the interference object recognition model to identify the interference in the circuit board image, the preprocessing operation performed on the circuit board image also includes performing a first affine transformation on the circuit board image based on the prediction frame parameters to obtain the circuit board image after the affine transformation, and then using the interference object recognition model to identify the interference in the circuit board image after the affine transformation, and performing operations such as interference removal and electromigration area extraction, and after obtaining the detection result image of the significantly marked electromigration area, a second affine transformation is performed on the detection result image of the significantly marked electromigration area based on the prediction frame parameters to obtain a detection result image of the significantly marked electromigration area that is consistent with the circuit board image. The second affine transformation is the inverse process of the first affine transformation. Figure 5 , which is a detection result image of a significantly marked electromigration area consistent with a circuit board image obtained in one embodiment.

[0108] In the above embodiment, before the terminal identifies interference objects on the circuit board image, it performs image affine transformation on the circuit board image based on the prediction frame parameters, thereby improving the accuracy of identifying interference objects and further improving the accuracy of electromigration area detection.

[0109] In one embodiment, after extracting the electromigration region in the target circuit board image through a visual saliency detection algorithm, the terminal can also calculate the area of ​​the electromigration region in the circuit board image, determine the short circuit risk level of the circuit board corresponding to the circuit board image according to the area, generate risk warning information corresponding to the short circuit risk level and provide a prompt. Among them, the larger the ratio of the area of ​​the electromigration region in the circuit board image to the area of ​​the circuit board image, the higher the corresponding short circuit risk level.

[0110] In the above embodiment, the terminal calculates the area of ​​the electromigration region in the circuit board image, determines the short-circuit risk level of the circuit board corresponding to the circuit board image according to the area, generates risk warning information corresponding to the short-circuit risk level and issues a prompt, thereby avoiding continued use of the circuit board when the short-circuit risk of the circuit board is high, thereby reducing safety hazards.

[0111] In one embodiment, Figure 6 As shown, a method for detecting an electromigration region is also provided, and the method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0112] S602, obtaining a circuit board image.

[0113] S604, performing a preprocessing operation on the circuit board image to obtain a preprocessed circuit board image; the preprocessing operation includes image graying, morphological processing, and median filtering processing.

[0114] S606, performing a first image affine transformation on the preprocessed image to obtain a circuit board image after the first image affine transformation.

[0115] S608, performing edge detection on the circuit board image after affine transformation of the first image to obtain edge information of the circuit board image.

[0116] S610: Identify interference objects in the circuit board image based on edge information.

[0117] S612, removing the interference object from the circuit board image to obtain a target circuit board image without the interference object.

[0118] S614, converting the circuit board image without interference objects into a preset color space to obtain the circuit board image without interference objects in the preset color space.

[0119] S616, extracting the electromigration region in the circuit board image without interference objects in a preset color space through a visual saliency detection algorithm, and generating a corresponding saliency map.

[0120] S618, registering and superimposing the saliency map and the circuit board image to obtain a detection result image with saliency marked electromigration areas.

[0121] S620, performing a second affine transformation on the detection result image with the significantly marked electromigration region to obtain a detection result image with the significantly marked electromigration region consistent with the circuit board image.

[0122] It should be understood that although Figure 2 and 6The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2 and 6 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0123] In one embodiment, Figure 7 As shown, an electromigration region detection device is provided, comprising: an image acquisition module 702, an edge detection module 704, an image recognition module 706, an interference removal module 708 and an electromigration region extraction module 710, wherein:

[0124] An image acquisition module 702 is used to acquire a circuit board image;

[0125] The edge detection module 704 is used to perform edge detection on the circuit board image to obtain edge information of the circuit board image;

[0126] An image recognition module 706, used to identify interference objects in the circuit board image based on edge information;

[0127] The interference object removal module 708 is used to remove interference objects from the circuit board image to obtain a target circuit board image without interference objects;

[0128] The electromigration region extraction module 710 is used to extract the electromigration region in the target circuit board image by using a visual saliency detection algorithm.

[0129] In the above embodiment, after acquiring the circuit board image, the terminal performs edge detection on the circuit board image to obtain edge information of the circuit board image; identifies interference objects in the circuit board image based on the edge information; removes interference objects from the circuit board image to obtain a target circuit board image without interference objects; and extracts the electromigration area in the target circuit board image through a visual saliency detection algorithm, thereby detecting the electromigration area on the circuit board device without manual work, thereby improving the efficiency and accuracy of electromigration area detection.

[0130] In one embodiment, Figure 8 As shown, the device also includes: a sample circuit board image acquisition module 712, a labeling module 714 and a model training module 716, wherein:

[0131] The sample circuit board image acquisition module 712 is used to acquire a preset number of sample circuit board images;

[0132] A labeling module 714 is used to label the sample circuit board image to obtain a training sample;

[0133] A model training module 716 is used to input training samples into the target detection network for training to obtain an interference object recognition model;

[0134] The image recognition module 706 is further used for:

[0135] Interference objects in circuit board images are identified based on edge information through the interference object recognition model.

[0136] In the above embodiment, the terminal obtains a preset number of sample circuit board images; annotates the sample circuit board images to obtain training samples, and inputs the training samples into the target detection network for training, thereby obtaining an interference recognition model, and then identifies the interference area in the circuit board image through the interference recognition model, thereby realizing the extraction of the electromigration area, so that the electromigration area on the circuit board device can be detected without manual work, thereby improving the efficiency and accuracy of the electromigration area detection.

[0137] In one embodiment, the apparatus further comprises:

[0138] The preprocessing module is used to perform preprocessing operations on the circuit board image. The preprocessing operations include image grayscale, morphological processing, and median filtering.

[0139] In the above embodiment, before the terminal performs edge detection on the circuit board image, the circuit board image can be preprocessed to effectively remove noise and retain effective image information, thereby reducing the amount of calculation in the electromigration area detection process and improving the efficiency of electromigration area detection.

[0140] In one embodiment, the electromigration region extraction module 710 is further configured to:

[0141] Converting the circuit board image without interference objects into a preset color space to obtain the circuit board image without interference objects in the preset color space;

[0142] The electromigration regions in the distractor-free PCB images in a preset color space are extracted using a visual saliency detection algorithm.

[0143] In the above embodiment, the terminal obtains a circuit board image without interference objects in the preset color space by converting the circuit board image without interference objects into a preset color space, so that in the preset color space, the circuit board image without interference objects can be extracted more quickly through a visual saliency detection algorithm, thereby improving the efficiency of electromigration area detection.

[0144] In one embodiment, the apparatus further comprises:

[0145] An area calculation module, used to calculate the area of ​​the electromigration region in the circuit board image;

[0146] A short circuit risk level determination module, used to determine the short circuit risk level of the circuit board corresponding to the circuit board image according to the area;

[0147] The prompt module is used to generate risk prompt information corresponding to the short circuit risk level and provide prompts.

[0148] In the above embodiment, the terminal calculates the area of ​​the electromigration region in the circuit board image, determines the short-circuit risk level of the circuit board corresponding to the circuit board image according to the area, generates risk warning information corresponding to the short-circuit risk level and issues a prompt, thereby avoiding continued use of the circuit board when the short-circuit risk of the circuit board is high, thereby reducing safety hazards.

[0149] For the specific definition of the electromigration region detection device, please refer to the definition of the electromigration region detection method above, which will not be repeated here. Each module in the above-mentioned electromigration region detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0150] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store circuit board image data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an electromigration area detection method is implemented.

[0151] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.10As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an electromigration area detection method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0152] Those skilled in the art will understand that Fig. 9 The structure shown in or 10 is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0153] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the following steps: acquiring a circuit board image; performing edge detection on the circuit board image to obtain edge information of the circuit board image; identifying interferers in the circuit board image based on the edge information; removing the interferers from the circuit board image to obtain a target circuit board image without interferers; and extracting an electromigration region in the target circuit board image through a visual saliency detection algorithm.

[0154] In one embodiment, when the computer program is executed by a processor, the processor executes the following steps: obtaining a preset number of sample circuit board images; annotating the sample circuit board images to obtain training samples; inputting the training samples into a target detection network for training to obtain an interference recognition model; when the computer program is executed by the processor to perform the step of identifying interferences in the circuit board image based on edge information, the processor also executes the following step: identifying interferences in the circuit board image based on edge information using the interference recognition model.

[0155] In one embodiment, when the computer program is executed by a processor, the processor further executes the following steps: performing preprocessing operations on the circuit board image, the preprocessing operations including image graying, morphological processing, and median filtering.

[0156] In one embodiment, when the computer program is executed by the processor to extract the electromigration area in the target circuit board image through a visual saliency detection algorithm, the processor executes the following steps: converting the circuit board image without interference objects to a preset color space to obtain a circuit board image without interference objects in the preset color space; and extracting the electromigration area in the circuit board image without interference objects in the preset color space through a visual saliency detection algorithm.

[0157] In one embodiment, when the computer program is executed by a processor, the processor performs the following steps: calculating the area of ​​the electromigration region in the circuit board image; determining the short circuit risk level of the circuit board corresponding to the circuit board image based on the area; generating risk warning information corresponding to the short circuit risk level and providing a warning.

[0158] In one embodiment, a computer-readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the processor executes the following steps: acquiring a circuit board image; performing edge detection on the circuit board image to obtain edge information of the circuit board image; identifying interferers in the circuit board image based on the edge information; removing the interferers from the circuit board image to obtain a target circuit board image without interferers; and extracting an electromigration region in the target circuit board image through a visual saliency detection algorithm.

[0159] In one embodiment, when the computer program is executed by a processor, the processor executes the following steps: obtaining a preset number of sample circuit board images; annotating the sample circuit board images to obtain training samples; inputting the training samples into a target detection network for training to obtain an interference recognition model; when the computer program is executed by the processor to perform the step of identifying interferences in the circuit board image based on edge information, the processor also executes the following step: identifying interferences in the circuit board image based on edge information using the interference recognition model.

[0160] In one embodiment, when the computer program is executed by a processor, the processor further executes the following steps: performing preprocessing operations on the circuit board image, the preprocessing operations including image graying, morphological processing, and median filtering.

[0161] In one embodiment, when the computer program is executed by the processor to extract the electromigration area in the target circuit board image through a visual saliency detection algorithm, the processor executes the following steps: converting the circuit board image without interference objects to a preset color space to obtain a circuit board image without interference objects in the preset color space; and extracting the electromigration area in the circuit board image without interference objects in the preset color space through a visual saliency detection algorithm.

[0162] In one embodiment, when the computer program is executed by a processor, the processor performs the following steps: calculating the area of ​​the electromigration region in the circuit board image; determining the short circuit risk level of the circuit board corresponding to the circuit board image based on the area; generating risk warning information corresponding to the short circuit risk level and providing a warning.

[0163] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0164] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0165] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A method for detecting an electromigration region, characterized in that: The method comprises: Acquire an original circuit board image, use the GrabCut algorithm to perform image segmentation on the original circuit board image, extract the area to be detected, and use the extracted area to be detected as the circuit board image; Using a canny operator to perform edge detection on the circuit board image to obtain edge information of the circuit board image; An interference object recognition model is used to identify interference objects in the circuit board image based on the edge information; the interference object recognition model is pre-trained based on a YOLOv3 network; The GrabCut algorithm is used to segment the circuit board image. During the image segmentation, the identified interference object is used as the background, and the area outside the interference object in the circuit board image is used as the foreground, so as to remove the interference object from the circuit board image and obtain a circuit board image without interference object. The interference-free circuit board image is converted to a preset color space to obtain an interference-free circuit board image in the preset color space; a saliency map corresponding to the interference-free circuit board image in the preset color space is obtained by a visual saliency detection algorithm; the saliency map is binarized and median filtered to obtain a processed saliency map; the processed saliency map and the interference-free circuit board image are aligned and superimposed using the processed saliency map as a mask to obtain a detection result image with a significantly marked electromigration area.

2. The method according to claim 1, characterized in that The method further comprises: Obtaining a preset number of sample circuit board images; Annotating the sample circuit board image to obtain a training sample; The training samples are input into the target detection network for training to obtain an interference object recognition model.

3. The method according to claim 1, characterized in that Before using the canny operator to perform edge detection on the circuit board image, the method further includes: The circuit board image is preprocessed, and the preprocessing operation includes image graying, morphological processing, and median filtering.

4. The method according to claim 1, characterized in that: After obtaining the detection result image with significantly marked electromigration areas, the method further includes: Calculating the area of ​​the electromigration region in the circuit board image; Determining a short circuit risk level of a circuit board corresponding to the circuit board image according to the area; Generate risk warning information corresponding to the short circuit risk level and provide a warning.

5. An electromigration region detection device, characterized in that: The device comprises: An image acquisition module is used to acquire an original circuit board image, perform image segmentation on the original circuit board image using a GrabCut algorithm, extract an area to be detected, and use the extracted area to be detected as a circuit board image; An edge detection module, used to perform edge detection on the circuit board image using a Canny operator to obtain edge information of the circuit board image; An image recognition module, used to identify the interferers in the circuit board image based on the edge information using an interferer recognition model; the interferer recognition model is pre-trained based on a YOLOv3 network; An interference removal module is used to perform image segmentation on the circuit board image using the GrabCut algorithm. During image segmentation, the identified interference is used as the background, and the area outside the interference in the circuit board image is used as the foreground, so as to remove the interference from the circuit board image and obtain a circuit board image without interference; The electromigration region extraction module is used to convert the interference-free circuit board image into a preset color space to obtain the interference-free circuit board image in the preset color space; obtain a saliency map corresponding to the interference-free circuit board image in the preset color space through a visual saliency detection algorithm; perform binarization and median filtering on the saliency map to obtain a processed saliency map; and use the processed saliency map as a mask to align and superimpose the processed saliency map and the interference-free circuit board image to obtain a detection result image with saliency-marked electromigration regions.

6. The device according to claim 5, characterized in that The device also includes: A sample circuit board image acquisition module is used to acquire a preset number of sample circuit board images; A labeling module, used to label the sample circuit board image to obtain a training sample; The training samples are input into the target detection network for training to obtain an interference object recognition model.

7. The device according to claim 5, characterized in that The device also includes: The preprocessing module is used to perform preprocessing operations on the circuit board image, and the preprocessing operations include image graying, morphological processing, and median filtering processing.

8. The device according to claim 6, Its feature is that the device also includes: An area calculation module, used to calculate the area of ​​the electromigration region in the circuit board image; A short circuit risk level determination module, used to determine the short circuit risk level of the circuit board corresponding to the circuit board image according to the area; The prompt module is used to generate risk prompt information corresponding to the short circuit risk level and provide a prompt.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.

Images