Electronic device polarity detection method, device, equipment and medium

Abstract

The invention relates to the technical field of electronic device polarity detection, in particular to an electronic device polarity detection method and device, equipment and a medium. The method comprises the following steps: constructing a deep learning model; inputting the template image and the to-be-detected device image into a deep learning model to respectively obtain a first image feature and a second image feature; calculating the similarity between the first image feature and the second image feature; calculating a quality matching score between the template image and the to-be-detected device image based on the similarity; and judging whether the polarity of the device is correct based on the quality matching score. According to the invention, automatic inspection of the polarity condition of the SMT / DIP device can be realized, the labor cost can be effectively reduced, and the generalization and the detection precision can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of polarity detection of electronic devices, and in particular to a method, device, equipment and medium for detecting polarity of electronic devices. Background Art

[0002] Polarity detection of electronic devices is an important step in the production process of SMT (Surface Mount Technology) / DIP (dual in-line package). Many electronic devices have positive and negative polarities, such as capacitors and diodes. If the polarity is reversed when the device is installed, it will cause device damage or circuit failure. Polarity detection can detect this problem before the device is installed, avoiding wasting time and cost. Although the traditional visual inspection method that relies on manual inspection is simple and easy, it is prone to human errors and has low efficiency, which does not meet the economic benefits of the market. In recent years, with the rapid development of computer software and hardware, electronic device polarity detection methods based on automatic optical inspection (AOI) technology have also appeared on the market. However, these methods often have several disadvantages, which are mainly related to the following limitations in the SMT / DIP polarity detection scenario: 1. The ambient lighting conditions on the SMT / DIP production line are usually complex, and there may be problems such as light and dark differences, shadows, and nearby device occlusion, which will affect the image acquisition quality. Traditional image processing algorithms are usually sensitive to light, resulting in poor accuracy in polarity identification in SMT / DIP scenarios. 2. Due to the unstable production process, the polarity marks on different production devices may jitter. At the same time, the transportation and mounting process of the device will also cause the polarity marks to shift and wear. These situations will cause deviations in the polarity characteristics of different devices. The traditional AOI detection algorithm is sensitive to the polarity characteristics of the device, resulting in poor robustness in the recognition of device polarity marks in such complex scenarios, thereby increasing the debugging cost. 3. The number of defective devices in SMT / DIP production is generally low, and defective samples are very rare. Traditional machine vision algorithms and neural network deep learning models based on supervised learning generally require large-scale and diverse defective samples to achieve better detection performance, and waiting for the accumulation of defective samples is not feasible. 4. SMT / DIP production involves a large number of electronic devices of different types and sizes, including resistors, capacitors, diodes, ICs, etc. These devices vary greatly in appearance, pin arrangement, etc., and their polarity identification also exists in various forms. Most existing neural network deep learning models generally have poor generalization. Deep learning models trained on a single data set often cannot effectively adapt to the polarity detection tasks of different electronic devices; and targeted training and optimization for different device characteristics requires a lot of manpower and time costs, which is difficult to put into actual production.

[0003] Therefore, the problem of how to improve the accuracy and efficiency of polarity detection of electronic devices in the SMT / DIP production process while reducing costs in the prior art needs to be solved urgently. Summary of the invention

[0004] The main purpose of this application is to provide a method for detecting polarity of electronic devices, aiming to solve the technical problem of how to improve the accuracy and efficiency of polarity detection of electronic devices in the SMT / DIP production process while reducing costs.

[0005] The first aspect of the present application provides a method for detecting polarity of an electronic device, characterized by comprising:

[0006] Build deep learning models;

[0007] Inputting the template image and the image of the device to be detected into the deep learning model to obtain a first image feature and a second image feature respectively;

[0008] Calculating the similarity between the first image feature and the second image feature;

[0009] Calculating a quality matching score between the template image and the image of the device to be inspected based on the similarity;

[0010] Whether the device polarity is correct is determined based on the quality match score.

[0011] Furthermore, the deep learning model includes: a deep learning feature extractor, used to extract features from the template image and the image of the device to be detected respectively, and output the first image features corresponding to the template image and the second image features corresponding to the image of the device to be detected; and a template matching layer, used to calculate the similarity between the first image features and the second image features, and calculate the quality matching score between the template image and the image of the device to be detected based on the similarity.

[0012] Furthermore, before the step of inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively, the step includes:

[0013] On the image of the device to be inspected, a plurality of candidate regions containing polarity identification symbols are identified;

[0014] Each identified candidate area is extracted from the image of the device to be inspected to form an independent sub-image, which is used to input into the deep learning model together with the template image.

[0015] Furthermore, the step of calculating the quality matching score between the template image and the image of the device to be inspected based on the similarity comprises:

[0016] The quality matching score between the template image and the image of the device to be detected is:

[0017] q(s)=max{TML(s't)};

[0018] Where t is the template image; S is the image of the device to be detected; TML(s′t) is the matching output result of the deep learning model; TML(s′t) is obtained by the following formula:

[0019] TML(s′t)=L(s|t)·L(t|s)

[0020] Wherein, L(s|t) is the matching degree between the image s of the device to be detected and the template image t; L(t|s) is the matching degree between the template image t and the image s of the device to be detected;

[0021] The matching degree between the image s of the device to be detected and the template image t is calculated by the following formula:

[0022]

[0023] Among them, t' represents a template, T represents the set of all considered models; f t and f s Represent the first image feature and the second image feature respectively; ρ(f t ,f s ) is the similarity between the first image feature and the second image feature; α is a positive hyperparameter used to adjust the weight distribution; ρ(f t′ ,f s ) is the similarity between the image feature of t' and the second image feature.

[0024] Furthermore, the step of judging whether the polarity of the device is correct based on the quality matching score includes:

[0025] Map the quality matching score to a grid in two-dimensional space; wherein the value of each grid represents the quality matching score of the corresponding position;

[0026] Convert the quality matching scores of the corresponding positions into colors to form a matching degree heat map;

[0027] Whether the device polarity is correct is determined based on the matching degree heat map.

[0028] Furthermore, after the step of inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively, the method further comprises:

[0029] Calculating the mean and standard deviation of the first image feature and the second image feature;

[0030] Based on the mean value and standard deviation, the characteristic value of each dimension is standardized;

[0031] The standardization process is implemented by the following formula:

[0032]

[0033] Among them, z is the standardized image feature value, x is the original feature value of the image, μ is the mean of the image feature, and σ is the standard deviation of the image feature.

[0034] Furthermore, the deep learning model is processed in conjunction with a hardware acceleration unit to improve the reasoning speed of a single image.

[0035] The second aspect of the present application provides an electronic device polarity detection device, comprising:

[0036] Model building module, used to build deep learning models;

[0037] A model input module, used to input the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively;

[0038] A similarity calculation module, used to calculate the similarity between the first image feature and the second image feature;

[0039] A matching calculation module, used for calculating a quality matching score between the template image and the image of the device to be detected based on the similarity;

[0040] A judgment module is used to judge whether the polarity of the device is correct based on the quality matching score.

[0041] The third aspect of the present application provides an electronic device polarity detection device, comprising:

[0042] Industrial cameras, used to capture images of devices to be inspected;

[0043] processor;

[0044] Memory for storing computer programs;

[0045] Wherein, when the processor executes the computer program, any one of the above-mentioned methods for detecting polarity of electronic devices is implemented.

[0046] The fourth aspect of the present application proposes a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, any of the above-mentioned electronic device polarity detection methods is implemented.

[0047] Beneficial effects:

[0048] This application can realize automatic detection of the polarity of the image of the device to be detected by constructing a deep learning model and calculating the similarity of image features, without the need for special model training, and is suitable for polarity detection scenarios where there are no or no positive samples and labeled samples. The polarity matching of the device to be detected can be obtained through the quality matching score finally output by the model, which can not only accurately quantify the degree of image matching, but also better adapt to polarity detection tasks in different scenarios, and can adapt to polarity detection of different types of devices, with high mobility and generalization. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a schematic diagram of the steps of a method for detecting polarity of an electronic device according to an embodiment of the present application;

[0050] Figure 2 It is a schematic block diagram of the structure of an electronic device polarity detection device according to an embodiment of the present application;

[0051] Figure 3 is a schematic block diagram of the structure of a computer device according to an embodiment of the present application;

[0052] Figure 4 It is a template image used in the polarity detection process of electronic devices;

[0053] Figure 5 It is an image of the device to be tested during the polarity detection process of an electronic device;

[0054] Figure 6 This is the detection result diagram under the traditional template matching method;

[0055] Figure 7 This is the detection result diagram under the general supervised learning neural network model;

[0056] Figure 8 This is a diagram of detection results under a method for detecting polarity of an electronic device according to an embodiment of the present application;

[0057] Fig. 9 is a template image in a method for detecting polarity of an electronic device according to another embodiment of the present application;

[0058] Fig.10 is an image of a device to be detected in a method for detecting polarity of an electronic device according to another embodiment of the present application;

[0059] Fig.11 is a matching degree heat map under a polarity detection method of an electronic device according to another embodiment of the present application;

[0060] Fig.12 is a diagram of detection results under a method for detecting polarity of an electronic device according to another embodiment of the present application;

[0061] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0063] It will be appreciated by those skilled in the art that, unless expressly stated, the singular forms "one", "above", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of features, integers, steps, operations, elements, modules, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, modules, components, and / or groups thereof. It should be understood that when an element is said to be "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may also be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any module and all combinations of one or more associated listed items.

[0064] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0065] Reference Figure 1 The present application embodiment provides a method for detecting polarity of an electronic device, comprising the following steps S1-S5:

[0066] S1. Build a deep learning model;

[0067] S2, inputting the template image and the image of the device to be detected into a deep learning model to obtain a first image feature and a second image feature respectively;

[0068] S3, calculating the similarity between the first image feature and the second image feature;

[0069] S4, calculating a quality matching score between the template image and the image of the device to be detected based on the similarity;

[0070] S5. Determine whether the device polarity is correct based on the quality matching score.

[0071] In step S1, first, an unsupervised deep learning model is constructed to extract features from a template image and an image of a device to be detected, wherein the image of the device to be detected is an image of an electronic device that needs to be polarity detected and is taken by an industrial camera. The deep learning model includes selecting a lightweight convolutional neural network (CNN) architecture, such as ResNet (residual network) or a lightweight Transformer network such as SwinTransformer, etc. These architectures can reduce the demand for computing resources while ensuring performance. In the actual production process, the device sizes in the production of electronic devices are often different, such as the different device sizes in the production of SMT / DIP, and these structures can ensure that the convolution layer of the model can ensure that input images of different sizes are processed. First, the model is initialized, and then a template matching layer (TML) is added. This layer is responsible for calculating the similarity between the features of the template image and the image of the device to be detected. TML can use metrics such as cosine similarity or correlation coefficient to quantify the degree of matching between two feature vectors. The use of lightweight models reduces the consumption of computing resources, allowing the model to run quickly on industrial-grade hardware. In addition, a normalization layer can be integrated inside the model for subsequent image normalization processing to reduce the differences caused by changes in image illumination and contrast, and ensure the robustness of the model to illumination conditions. The normalization layer can be a simple pixel normalization, or a more complex local response normalization. The addition of a normalization layer improves the model's resistance to illumination changes and ensures detection accuracy under complex illumination conditions. The lightweight architecture and efficient feature matching mechanism ensure that the model can make decisions quickly in real-time applications and meet the speed requirements of industrial production lines. This step can build a deep learning model that efficiently and accurately detects the polarity of SMT / DIP devices, which can cope with common challenges in SMT / DIP production, such as the diversity of device sizes, changes in illumination conditions, and the complexity of polarity identification, thereby significantly improving detection accuracy and efficiency.

[0072] In step S2, first, the template image and the image of the device to be detected need to be preprocessed, including adjusting the template image and the image of the device to be detected to the same size so that the model can process, including scaling, cropping or filling operations. Then confirm whether the model receives a color image or a grayscale image, and convert the image mode as needed. Then the image pixel values ​​are normalized, for example, the RGB value is scaled from 0-255 to between 0-1 to enhance the numerical stability and generalization ability of the model. Then, the preprocessed template image and the image of the device to be detected are input into the deep learning model. At this point, the feature extractor of the model will start working to extract high-dimensional features from the image. The output is captured at the appropriate layer of the model (usually the last convolution layer or a layer before it), and these outputs represent the feature vectors of the image. Optionally, multiple candidate area processing is performed during the implementation process, and multiple candidate areas containing polarity marks are identified in the image of the device to be detected, including by sliding windows, image segmentation or edge-based detection technology. Independent sub-images are extracted from each candidate area, and these sub-images will be processed separately to improve the accuracy and efficiency of detection. Each sub-image is input into the model together with the template image to obtain the feature vector of each area. This enables the feature information of different positions in the image of the device to be inspected to be obtained, so as to more comprehensively evaluate its matching degree with the template image. This step ensures that the template image and the image of the device to be inspected are properly prepared and input into the deep learning model, providing high-quality image features for the subsequent feature similarity calculation and polarity judgment. The design and optimization of this process aims to improve the accuracy and efficiency of polarity detection of electronic devices and reduce potential production errors and cost waste.

[0073] In step S3, after obtaining the first image feature of the template image and the second image feature of the device image to be detected, it is necessary to calculate the similarity between the two feature vectors to evaluate the degree of matching between the template image and the device image to be detected. Prior to this, the first image feature and the second image feature obtained are first standardized, that is, the dimension value of each feature vector is scaled to the same range, such as unit length or zero mean unit variance. The similarity can be calculated by, for example, cosine similarity or Euclidean distance. By calculating the similarity between the feature vectors, the degree of matching between the template image and the device image to be detected can be quantified, which is the key basis for judging whether the polarity is correct. Using methods such as cosine similarity can reduce the impact of image size and brightness changes on the matching results and improve the robustness of the detection method. The flexible selection of similarity measurement methods can also enable this step to be flexibly adapted to different application scenarios and feature types, thereby improving the versatility and adaptability of the method. Through the above steps, the degree of matching between the template image and the device image to be detected can be accurately evaluated, which is an important basis for subsequent quality matching score calculation and polarity judgment. It can improve the accuracy and reliability of polarity detection of electronic devices and reduce potential errors and costs in production.

[0074] In step S4, after calculating the similarity of the feature vectors of the template image and the image of the device to be detected, we need to further calculate the quality match score based on this similarity information to determine whether the polarity of the device is correct. If there are multiple candidate regions, the region with the highest quality match score is selected from these regions as the final result. This is because the polarity mark may exist in multiple locations, and we are concerned with the region that best matches the template image. First, obtain the image features (denoted as f t and f s ), and calculate the similarity between the two ρ(f t ,f s ), such as the Euclidean distance between the two

[0075] distance) and cosine similarity, and then calculate the degree of match between the two:

[0076]

[0077] Among them, f t and f s Represent the eigenvectors of the template image t and the detection device image s respectively; ρ(f t ,f s ) is the similarity between the two image features; α is a positive hyperparameter used to adjust the weight distribution. T is a set that contains all possible template images t'; ρ(f t′,f s ) is the similarity between the image feature of t' and the second image feature.

[0078] The output of the final template matching layer (TML) is defined as:

[0079] TML(s,t)=L(s|t)·L(t|s);

[0080] Wherein, L(s|t) is the matching degree between the image s of the device to be detected and the template image t; L(t|s) is the matching degree between the template image t and the image s of the device to be detected.

[0081] The quality matching score between the template image and the image of the device to be inspected can be calculated as:

[0082] q(s)=max{TML(s,t)}

[0083] The calculation idea of ​​L(t|s) is the same as that of L(s|t), which will not be repeated here. This embodiment converts the similarity metric into a specific quality matching score, which can more intuitively understand the degree of matching between the device polarity and the template image, thereby improving the accuracy of detection. This step can calculate the quality matching score based on the similarity of the feature vectors. This score will be used for subsequent judgment of the correctness of the device polarity, which can improve the accuracy and reliability of electronic device polarity detection while maintaining the flexibility and scalability of the method.

[0084] In step S5, first, a threshold needs to be set, which is used to distinguish high-quality matches from low-quality matches. This threshold can be determined based on experience, experimental data or cross-validation to ensure that it can minimize misjudgments while ensuring detection accuracy. The quality match score obtained in step S4 is compared with a preset threshold. If the quality match score is higher than the threshold, it is considered that the match between the template image and the image of the device to be detected is of high quality, which means that the polarity detection may be correct. On the contrary, if the quality match score is lower than the threshold, it may indicate that the match quality is not high and the polarity of the device may be mislabeled. Finally, the judgment result is output, which may be a binary flag (for example, 1 indicates correct polarity and 0 indicates incorrect polarity) or a label with confidence, indicating the probability that the polarity of the device is correct. This output can be integrated into more complex systems for automatic assembly lines, quality control or any other application scenarios that require accurate polarity detection. In addition, a feedback mechanism can also be included to adjust the threshold or improve the model. For example, if it is found that the system frequently has false positive or false negative results, the model can be fine-tuned or the threshold can be recalibrated through a machine learning algorithm to improve the overall performance of the system.

[0085] In the judgment process, a heat map can also be generated based on the quality match score. In the deep learning model, the attention mechanism or Gradient-weighted Class Activation Mapping (Grad-CAM) and other technologies can be used to calculate the importance of each pixel in the prediction process of the model. Using the calculated attention weight, it can be superimposed on the original image to generate a heat map. The color intensity on the heat map reflects the degree of attention of the model at that location. The darker the color, the more attention the model pays to the area. By analyzing the heat map, you can visually check whether the model has correctly paid attention to the key features of the electronic device. The heat map can also be used to assist in optimizing the threshold setting. If you find that the model is prone to false alarms in some cases, you can understand the decision-making basis of the model by checking the heat map in these cases, and then adjust the threshold to reduce the false alarm rate.

[0086] Reference Figure 4-Figure 8 In the same detection scenario, traditional computer vision algorithms are sensitive to lighting conditions and image features, and have problems such as weak generalization, poor performance stability, and poor adaptability to complex scenarios, making it difficult to inspect the polarity of SMT production components; the detection results are as follows Figure 6 As shown in the figure, the general supervised learning neural network model requires a large number of diverse training samples to achieve good detection performance. When defect samples are scarce and the types of devices and polarity identification are complex, this solution is difficult to achieve good detection results. The detection results are shown in the figure. Figure 7 As shown in the figure, the solution proposed in this application mainly relies on the feature matching algorithm based on deep learning, and can achieve excellent detection performance without special training of the model. It is lightweight and, with the acceleration hardware unit, the inference speed of a single image can be as low as 30ms, which meets the needs of lightweight industrial quality inspection. At the same time, through the template image ( Figure 4 );and the image of the device to be detected ( Figure 5 ) to judge the polarity, so it can adapt to the polarity detection of different types of devices and has stronger generalization. At the same time, the standardized operation in the reasoning process also enables it to accurately measure and quantify the matching degree between the image of the device to be detected and the template image, so as to better judge the polarity of the device. The detection results are as follows Figure 8 shown.

[0087] Reference Figure 9-12 In another detection scenario, this application uses a template image ( Fig. 9 );and the image of the device to be detected ( Fig.10 ) is matched to make a polarity judgment, and after model inference, the quality matching score between the template image and the image of the device to be detected is output. The generated matching degree heat map is as follows Fig.11As shown in the figure; the quality matching score can be used to locate the position of the polarity mark in the image of the device to be tested and determine whether the polarity of the device is correct. The test results are shown in Fig.12 shown.

[0088] In one embodiment, the deep learning model includes: a deep learning feature extractor, used to extract features from the template image and the image of the device to be detected, respectively, and output the first image features corresponding to the template image and the second image features corresponding to the image of the device to be detected; and a template matching layer, used to calculate the similarity between the first image features and the second image features, and calculate the quality matching score between the template image and the image of the device to be detected based on the similarity.

[0089] In this embodiment, the deep learning model includes a lightweight bias-free deep learning feature extractor (for example, deep learning backbone networks such as ResNet, VGG, Inception (a deep learning structure), ViT (Vision Transformer, a visual model architecture) and a template matching layer (Template matching layer, TML). The feature extractor is used to extract and generate high-dimensional image feature vectors from the image, and the TML module is used to calculate the matching degree between the input template image and the image of the device to be detected and output the final quality matching score heat map. First, the deep learning feature extractor is the core part of the model, which is responsible for extracting and generating high-dimensional image feature vectors from the input image to achieve deep understanding and representation of the image content. Next, the template matching layer is used to receive the high-dimensional feature vectors from the feature extractor and calculate the matching degree between the input template image and the image of the device to be detected. In order to achieve this goal, the template matching layer adopts the structure of a twin neural network (Siamese Network), that is, two identical sub-networks share weights to process the template image and the image of the device to be detected respectively. By calculating the similarity between the feature vectors output by the two sub-networks (such as cosine similarity or Euclidean distance), the matching degree between the input images can be obtained. In this embodiment, the design of the deep learning model and the template matching layer enables the model to learn a more robust and discriminative feature representation, thereby more accurately judging the polarity of the device, ensuring the consistency of the template image and the image of the device to be detected in the feature space, so that the model can more stably distinguish between defects and normal areas, reducing the possibility of false detection.

[0090] In one embodiment, before the step of inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively, the step includes:

[0091] S10, identifying a plurality of candidate regions containing polarity identification symbols on the image of the device to be inspected;

[0092] S11. Extract each identified candidate area from the image of the device to be inspected to form an independent sub-image, which is used to input into the deep learning model together with the template image.

[0093] In this embodiment, first, the image of the device to be detected is searched for areas that may contain polarity identification symbols. For example, edge detection, color segmentation, texture analysis and other techniques are used to preliminarily screen out areas with specific features, which are considered to contain polarity identification symbols. Once the candidate areas are identified, these areas are cropped or segmented from the original image to form independent sub-images. The purpose of this is to reduce the amount of calculation for subsequent processing, because only small areas containing key information are focused on, rather than the entire device image. Finally, each sub-image is input into the deep learning model together with the template image. The template image should be a known, clear sample of polarity identification, and the task of the model is to determine whether the sub-image matches the template, that is, whether it does contain the polarity identification. By focusing only on the candidate areas that may contain polarity identification, unnecessary calculations are reduced, the overall detection process is accelerated, and irrelevant background information is removed, so that the model can focus more on key features and improve the accuracy of recognition. At the same time, since the model is run on standardized sub-images, this helps to reduce false detections caused by factors such as background complexity and illumination changes. By pre-processing and screening image data, the efficiency and accuracy of deep learning models in the tasks of identifying and locating polarity identification symbols can be significantly improved.

[0094] In one embodiment, the step of calculating the quality matching score between the template image and the image of the device to be inspected by similarity comprises:

[0095] S30, the quality matching score between the template image and the image of the device to be detected is:

[0096] q(s)=max{TML(s't)};

[0097] Where t is the template image; s is the image of the device to be detected; TML(s′t) is the matching output result of the deep learning model; TML(s′t) is obtained by the following formula:

[0098] TML(s′t)=L(s|t)·L(t|s)

[0099] Wherein, L(s|t) is the matching degree between the image s of the device to be detected and the template image t; L(t|s) is the matching degree between the template image t and the image s of the device to be detected.

[0100] In this embodiment, the quality matching score q(s) is defined as the maximum matching degree between the template image t and the image s of the device to be detected, that is, the maximum value of TML(s't). Here, q(s) = max{TML(s't)}, where t is the template image, s is the image of the device to be detected, and TML(s't) is the matching output result of the deep learning model. TML(s't) is obtained by calculating the matching degree L(s|t) between the image s of the device to be detected and the template image t and the matching degree L(t|s) between the template image t and the image s of the device to be detected. Specifically, TML(s't) = L(s|t)*L(t|s). This means that TML(s't) is the product of the matching degree in two directions, that is, the two-way matching of the template image t and the image s of the device to be detected is considered. By calculating L(s|t) and L(t|s), the system can simultaneously consider the matching degree of the template image and the image of the device to be detected, which is more stringent than unidirectional matching and improves the accuracy of matching. By maximizing TML(s't) to match the image s of the device to be detected, the accuracy of polarity detection can be improved. This method can better deal with problems such as noise, blur or occlusion in the image, because only when the matching degree in both directions is high will it be considered a high-quality match. The matching degree between the image s of the device to be detected and the template image t is calculated by the following formula:

[0101]

[0102] Among them, t' represents a template, T represents the set of all considered models; f t and f s Represent the first image feature and the second image feature respectively; ρ(f s ,f t ) is the similarity between the first image feature and the second image feature; α is a positive hyperparameter used to adjust the weight distribution; ρ(f t′ ,f s ) is the similarity between the image feature of t' and the second image feature.

[0103] In this embodiment, L(s|t) is the matching degree between the image s of the device to be detected and the template image t, which is calculated by calculating the first image feature f t and the second image feature f sThe similarity of the two feature vectors represents the closeness of the two feature vectors, which can be measured using cosine similarity, Euclidean distance or other suitable metrics. The hyperparameter α is a positive value used to adjust the weight distribution of similarity. The larger the α value, the greater the influence of features with higher similarity on the degree of matching. By dividing by the sum of the similarities of all candidate template images t'∈T, the matching degree L(s|t) is normalized to the range of [0,1] to facilitate subsequent comparison and decision-making. By calculating the similarity between the first image feature ft and the second image feature fs, the system can comprehensively consider the two features to improve the accuracy of matching. The α parameter allows the weight distribution of similarity to be adjusted in order to emphasize important features or balance the contributions of different features. The normalized matching degree L(s|t) is robust to noise, blur or occlusion in the image, because a high degree of matching can only be obtained when a sufficient number of features are highly matched.

[0104] In one embodiment, the step of determining whether the polarity of the device is correct based on the quality matching score comprises:

[0105] S50, mapping the quality matching score to a grid in a two-dimensional space; wherein the value of each grid represents the quality matching score of the corresponding position;

[0106] S51, converting the quality matching scores of the corresponding positions into colors to form a matching degree heat map;

[0107] S52: Determine whether the polarity of the device is correct based on the matching degree heat map.

[0108] In this embodiment, the quality match score is mapped to a two-dimensional grid, where each grid cell represents a specific position or area on the image of the device to be detected. The value of each grid cell corresponds to the quality match score of the position with the template image. In this way, the degree of match on the entire device image is visualized as a two-dimensional array, and each array element represents the matching quality of the corresponding point in the image. Next, the quality match score in the grid is converted into different colors to form a matching degree heat map. Generally, the depth of the color or the degree of temperature (such as blue represents low matching degree and red represents high matching degree) is proportional to the quality match score, so that it can be intuitively seen which areas on the image match well with the template image and which areas match poorly. The heat map provides a visually easy-to-understand way to present complex data distribution. Finally, by analyzing the matching degree heat map, it can be determined whether the polarity of the device is correct. This usually involves the identification and analysis of high matching areas in the heat map. If these areas match the polarity identification of the device and the matching degree is high enough, it can be considered that the polarity of the device is correctly identified. In addition, thermal maps can help identify possible detection errors, such as false positives or negatives, and possible quality issues on the device. On the production line, thermal maps can be used to monitor the accuracy of device polarity detection in real time, helping to promptly detect and correct potential manufacturing defects.

[0109] In one embodiment, after the step of inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively, the following steps are included:

[0110] S60, calculating the mean and standard deviation of the first image feature and the second image feature;

[0111] S61, based on the mean value and the standard deviation, standardize the feature value of each dimension;

[0112] S62, wherein the standardization process is implemented by the following formula:

[0113]

[0114] Among them, z is the standardized image feature value, x is the original feature value of the image, μ is the mean of the image feature, and σ is the standard deviation of the image feature.

[0115] In this embodiment, after the features are extracted, the scale difference between the image features is uniformly performed, which can effectively eliminate the scale difference between the image features and make the different features comparable in value. In the tasks of template matching and polarity detection, the scale difference of the image features may affect the accuracy and stability of the matching. Therefore, by calculating the mean and standard deviation of the first image feature and the second image feature, and standardizing the eigenvalues ​​of each dimension, the different features can be made more consistent in value, thereby improving the accuracy of the matching. Specifically, the standardization is achieved by subtracting the mean from the original eigenvalue and dividing it by its standard deviation. Such an operation can make the standardized eigenvalue distribution conform to the standard normal distribution, that is, the mean is 0 and the standard deviation is 1. After the standardization, the numerical range of the eigenvalue will be limited to a relatively small interval, thereby avoiding the matching error caused by the large scale difference of the eigenvalue. In addition, the standardization can also improve the generalization ability of the deep learning model. During the calculation process, if the scale difference of the eigenvalue of the input data is large, it may cause the model to be difficult to converge or overfitting. Through standardization, the eigenvalues ​​of the input data can have similar scales, making it easier for the model to learn and adapt to the distribution of different data. At the same time, when the model is deployed to actual application scenarios, the model can be made more robust and better able to adapt to changes in different data through standardization. Therefore, in this embodiment, by inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature, their mean and standard deviation are calculated, and the eigenvalues ​​of each dimension are standardized, the accuracy and stability of the template matching can be further improved, thereby improving the accuracy of polarity detection. At the same time, this processing method can also improve the generalization ability of the deep learning model, making the model more suitable for device polarity detection tasks in actual production scenarios.

[0116] In one embodiment, the deep learning model is processed in conjunction with a hardware acceleration unit to improve the reasoning speed of a single image.

[0117] In this embodiment, by introducing a hardware acceleration unit to cooperate with a deep learning model, it is intended to significantly improve the reasoning speed of single image processing. The advantage of this combination is that although the deep learning model performs well in image feature extraction and matching, its computational complexity and huge amount of operations often lead to speed limitations when processing real-time or large-scale data. Hardware acceleration units, such as graphics processing units (GPUs), are designed for large-scale parallel computing and can greatly accelerate the calculation process of deep learning models. Specifically, when a deep learning model is used in combination with a hardware acceleration unit, the computationally intensive tasks in the model can be offloaded to the hardware acceleration unit for execution. Since the hardware acceleration unit has a strong parallel computing capability, it can complete a large number of computing tasks in a short time, thereby significantly improving the reasoning speed. This acceleration effect is particularly critical for application scenarios with high real-time requirements, such as device polarity detection tasks on production lines. In addition, the hardware acceleration unit also has the advantages of low power consumption and high energy efficiency. Compared with traditional central processing units (CPUs), hardware acceleration units consume less power when performing the same computing tasks, which makes the entire system more energy-efficient and efficient. In the case of large-scale deployment and long-term operation, this low power consumption feature can significantly reduce operating costs and maintenance costs.

[0118] In summary, by combining the deep learning model with the hardware acceleration unit, we can significantly improve the inference speed of a single image while maintaining the model performance. This combination not only meets the needs of application scenarios with high real-time requirements, but also reduces the operating and maintenance costs of the system.

[0119] refer to Figure 2 , a polarity detection device for an electronic device, comprising:

[0120] A model building module 100, used to build a deep learning model;

[0121] A model input module 200, used to input the template image and the image of the device to be detected into the deep learning model to obtain a first image feature and a second image feature respectively;

[0122] A similarity calculation module 300, configured to calculate the similarity between the first image feature and the second image feature;

[0123] A matching calculation module 400, configured to calculate a quality matching score between the template image and the image of the device to be detected based on the similarity;

[0124] The judgment module 500 is used to judge whether the polarity of the device is correct based on the quality matching score.

[0125] Furthermore, the above-mentioned determination module 500 includes:

[0126] A mapping unit, used for mapping the quality matching score to a grid in a two-dimensional space; wherein the value of each grid represents the quality matching score of the corresponding position;

[0127] A color conversion unit, used to convert the quality matching scores of corresponding positions into colors to form a matching degree heat map;

[0128] The first judgment subunit is used to judge whether the polarity of the device is correct based on the matching degree heat map.

[0129] Furthermore, the above-mentioned model building module 100 includes:

[0130] A deep learning unit, including a deep learning feature extractor, for extracting and generating a high-dimensional image feature vector from an image;

[0131] The template matching unit includes a template matching layer, which is used to calculate the matching degree between the input template image and the image of the device to be detected and output a matching degree heat map.

[0132] Furthermore, the above-mentioned model input module 200 includes:

[0133] A candidate region division unit, used for identifying a plurality of candidate regions containing polarity identification symbols on the image of the device to be detected;

[0134] The sub-image forming unit is used to extract each identified candidate area from the image of the device to be inspected to form an independent sub-image.

[0135] Furthermore, the matching calculation module 400 includes:

[0136] The first calculation unit is used for the quality matching score between the template image and the image of the device to be detected:

[0137] q(s)=max{TML(s't)};

[0138] Where t is the template image; S is the image of the device to be detected; TML(s′t) is the matching output result of the deep learning model; TML(s′t) is obtained by the following formula:

[0139] TML(s′t)=L(s|t)·L(t|s)

[0140] Wherein, L(s|t) is the matching degree between the image s of the device to be detected and the template image t; L(t|s) is the matching degree between the template image t and the image s of the device to be detected;

[0141] The second calculation unit is used to calculate the matching degree between the image s of the device to be detected and the template image t by the following formula:

[0142]

[0143] Among them, t' represents a template, T represents the set of all considered models; f t and f s Represent the first image feature and the second image feature respectively; ρ(f s ,f t ) is the similarity between the first image feature and the second image feature; α is a positive hyperparameter used to adjust the weight distribution; ρ(f t′ ,f s ) is the similarity between the image feature of t' and the second image feature.

[0144] In one embodiment, the above device further includes a standardization module:

[0145] A mean calculation unit, used to calculate the mean and standard deviation of the first image feature and the second image feature;

[0146] A standardization processing unit, used for performing standardization processing on the feature value of each dimension based on the mean value and the standard deviation;

[0147] A formula application unit, wherein the standardization process is implemented by the following formula:

[0148]

[0149] Among them, z is the standardized image feature value, x is the original feature value of the image, μ is the mean of the image feature, and σ is the standard deviation of the image feature.

[0150] In one embodiment, the above-mentioned device also includes a cooperation module for coordinating the deep learning model with the hardware acceleration unit to improve the reasoning speed of a single image.

[0151] Reference Figure 3 In an embodiment of the present application, a computer device is also provided. The computer device may be a server, and its internal structure may be as follows: Figure 3As shown. The computer device includes a processor, a memory, a network interface and a database connected by a bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores operations, computer programs and databases. The memory provides an environment for the operation of the operations and computer programs in the non-volatile storage medium. The database of the computer device is used to store data such as a method for detecting the polarity of electronic devices. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for detecting the polarity of an electronic device is implemented, including the steps of: constructing a deep learning model; inputting a template image and an image of a device to be detected into the deep learning model to obtain a first image feature and a second image feature respectively; calculating the similarity between the first image feature and the second image feature; calculating the quality matching score between the template image and the image of the device to be detected based on the similarity; judging whether the polarity of the device is correct based on the quality matching score.

[0152] An embodiment of the present application also provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, a method for detecting polarity of an electronic device is implemented, comprising the steps of: constructing a deep learning model; inputting a template image and an image of a device to be detected into the deep learning model to obtain a first image feature and a second image feature, respectively; calculating the similarity between the first image feature and the second image feature; calculating a quality matching score between the template image and the image of the device to be detected based on the similarity; and judging whether the polarity of the device is correct based on the quality matching score.

[0153] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided in the present application and used in the embodiments can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0154] The above description is only a preferred embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for detecting polarity of an electronic device, characterized in that: include: Build deep learning models; Inputting the template image and the image of the device to be detected into the deep learning model to obtain a first image feature and a second image feature respectively; Calculating the similarity between the first image feature and the second image feature; Calculating a quality matching score between the template image and the image of the device to be inspected based on the similarity; Whether the device polarity is correct is determined based on the quality match score.

2. The method for detecting polarity of an electronic device according to claim 1, characterized in that: The deep learning model includes: a deep learning feature extractor, used to extract features from the template image and the image of the device to be detected respectively, and output the first image features corresponding to the template image and the second image features corresponding to the image of the device to be detected; and a template matching layer, used to calculate the similarity between the first image features and the second image features, and calculate the quality matching score between the template image and the image of the device to be detected based on the similarity.

3. The method for detecting polarity of an electronic device according to claim 1, characterized in that: Before the step of inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively, the method includes: On the image of the device to be inspected, a plurality of candidate regions containing polarity identification symbols are identified; Each identified candidate area is extracted from the image of the device to be inspected to form an independent sub-image, which is used to input into the deep learning model together with the template image.

4. The method for detecting polarity of an electronic device according to claim 1, characterized in that: The step of calculating the quality matching score between the template image and the image of the device to be detected based on the similarity comprises: The quality matching score between the template image and the image of the device to be detected is: q(s)=max{TML(s't)}; Where t is the template image; s is the image of the device to be detected; TML(s′t) is the matching output result of the deep learning model; TML(s′t) is obtained by the following formula: TML(s′t)=L(s|t)·L(t|s) Wherein, L(s|t) is the matching degree between the image s of the device to be detected and the template image t; L(t|s) is the matching degree between the template image t and the image s of the device to be detected; The matching degree between the image s of the device to be detected and the template image t is calculated by the following formula: Where t′ represents a template, T represents the set of all considered models; f t and f s Represent the first image feature and the second image feature respectively; ρ(f t , f s ) is the similarity between the first image feature and the second image feature; α is a positive hyperparameter used to adjust the weight distribution; ρ(f t′ , f s ) is the similarity between the image feature of t′ and the second image feature.

5. The method for detecting polarity of an electronic device according to claim 1, characterized in that: The step of judging whether the polarity of the device is correct based on the quality matching score comprises: Map the quality matching score to a grid in two-dimensional space; wherein the value of each grid represents the quality matching score of the corresponding position; Convert the quality matching scores of the corresponding positions into colors to form a matching degree heat map; Whether the device polarity is correct is determined based on the matching degree heat map.

6. The method for detecting polarity of an electronic device according to claim 1, characterized in that: After the step of inputting the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively, the method includes: Calculating the mean and standard deviation of the first image feature and the second image feature; Based on the mean value and standard deviation, the characteristic value of each dimension is standardized; The standardization process is implemented by the following formula: Among them, z is the standardized image eigenvalue, x is the original eigenvalue of the image, μ is the mean of the image feature, and σ is the standard deviation of the image feature.

7. The method for detecting polarity of an electronic device according to claim 1, characterized in that: The deep learning model is processed in conjunction with a hardware acceleration unit to improve the reasoning speed of a single image.

8. An electronic device polarity detection device, characterized in that: include: Model building module, used to build deep learning models; A model input module, used to input the template image and the image of the device to be detected into the deep learning model to obtain the first image feature and the second image feature respectively; A similarity calculation module, used to calculate the similarity between the first image feature and the second image feature; A matching calculation module, used for calculating a quality matching score between the template image and the image of the device to be detected based on the similarity; A judgment module is used to judge whether the polarity of the device is correct based on the quality matching score.

9. An electronic device polarity detection device, characterized in that: include: Industrial cameras, used to capture images of devices to be inspected; processor; Memory for storing computer programs; Wherein, when the processor executes the computer program, the electronic device polarity detection method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the electronic device polarity detection method according to any one of claims 1 to 7 is implemented.

Images