Image processing method and device, equipment, readable storage medium and program product

Abstract

The invention relates to an image processing method and device, equipment, a readable storage medium and a program product. The method comprises the following steps: acquiring a circuit board image, wherein the circuit board image is obtained by performing image acquisition on a target circuit board; inputting the circuit board image into a target detection model to obtain the position information and the device type of each component in the target circuit board in the circuit board image; wherein the target detection model is obtained by training based on a component feature library and / or a plurality of first sample circuit board images, and the component feature library comprises component features extracted from a plurality of second sample circuit board images. By adopting the method, the verification accuracy and the maintenance efficiency of the returned electric energy meter can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular, to an image processing method, apparatus, device, readable storage medium, and program product. Background Art

[0002] During the verification process of returned electricity meters, the acquisition of the positions of components in the printed circuit board (PCB) is a key link to ensure the verification quality and efficiency of returned electricity meters.

[0003] Currently, the positions of components in the circuit board are usually manually acquired and confirmed. This method not only consumes time and effort but also has a high error rate, which is not conducive to improving the verification accuracy and maintenance efficiency of returned electricity meters. Summary of the Invention

[0004] Based on this, it is necessary to provide an image processing method, apparatus, device, readable storage medium, and program product that can improve the verification accuracy and maintenance efficiency of returned electricity meters for the above technical problems.

[0005] In a first aspect, this application provides an image processing method, and the method includes:

[0006] Obtain a circuit board image, where the circuit board image is obtained by performing image acquisition on a target circuit board;

[0007] Input the circuit board image into a target detection model to obtain the position information and device types of each component in the target circuit board in the circuit board image;

[0008] Wherein, the target detection model is trained based on a component feature library and / or a plurality of first sample circuit board images, and the component feature library includes component features extracted from a plurality of second sample circuit board images.

[0009] In one embodiment, the position information includes center point coordinates and bounding box coordinates, and the method further includes:

[0010] Perform error correction on the center point coordinates and / or the bounding box coordinates of each component according to the position correction strategy corresponding to the position information.

[0011] In one embodiment, the method further includes:

[0012] Extract component features from a plurality of the second sample circuit board images and store the extracted component features in the component feature library;

[0013] Train the initial target detection model according to the component feature library and / or the multiple first sample circuit board images to obtain the target detection model.

[0014] In one embodiment, the training the initial target detection model according to the component feature library and / or the multiple first sample circuit board images to obtain the target detection model includes:

[0015] Train the initial target detection model according to each of the component features to obtain the target detection model; or,

[0016] Train the initial target detection model according to each of the first sample circuit board images to obtain the target detection model; or,

[0017] Pre-train the initial target detection model according to each of the component features to obtain an intermediate target detection model, and fine-tune the intermediate target detection model according to each of the first sample circuit board images to obtain the target detection model; or,

[0018] Generate multiple generalized sample circuit board images according to each of the component features, and train the initial target detection model according to each of the generalized sample circuit board images and each of the first sample circuit board images to obtain the target detection model.

[0019] In one embodiment, the extracting component features from the multiple second sample circuit board images includes at least one of the following:

[0020] Use the Hough transform detection algorithm to extract the center information and radius information of circular components or elliptical components from each of the second sample circuit board images as each of the component features;

[0021] Use the contour detection algorithm to extract the contour information of rectangular components from each of the second sample circuit board images as each of the component features;

[0022] Use the gray-level co-occurrence matrix to extract the texture direction, texture contrast, and texture correlation of textured components from each of the second sample circuit board images as each of the component features;

[0023] Use the color histogram algorithm to extract color features from each of the second sample circuit board images as each of the component features.

[0024] In one embodiment, the method further includes:

[0025] Use a three-coordinate measuring instrument to verify the accuracy of the position information to obtain a verification result.

[0026] In a second aspect, the present application provides an image processing apparatus, which includes:

[0027] An acquisition module, configured to acquire a circuit board image, where the circuit board image is obtained by performing image acquisition on a target circuit board;

[0028] A target detection module, configured to input the circuit board image into a target detection model to obtain the position information and device type of each component in the target circuit board in the circuit board image;

[0029] Wherein, the target detection model is trained based on a component feature library and / or a plurality of first sample circuit board images, and the component feature library includes component features extracted from a plurality of second sample circuit board images.

[0030] In a third aspect, the present application provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the method described in the first aspect above are implemented.

[0031] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0032] In a fifth aspect, the present application provides a computer program product, including a computer program. When the computer program is executed by a processor, the steps of the method described in the first aspect above are implemented.

[0033] In the above image processing method, apparatus, device, readable storage medium, and program product, a circuit board image is acquired. The circuit board image is obtained by performing image acquisition on a target circuit board, and the target circuit board may be a circuit board in a watt-hour meter. Then, the circuit board image is input into a target detection model to obtain the position information and device type of each component in the target circuit board in the circuit board image. In this way, a target detection model is pre-trained based on a component feature library (the component feature library includes component features extracted from a plurality of second sample circuit board images) and / or a plurality of first sample circuit board images, and the target detection model is used to accurately and quickly identify the position information and device type of each component in the target circuit board, improving the verification accuracy and maintenance efficiency of the watt-hour meter. Description of the Drawings

[0034] To more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following will briefly introduce the drawings required for the description of the embodiments of the present application or related technologies. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other related drawings can also be obtained based on these drawings.

[0035] Figure 1 It is a schematic flowchart of an image processing method in an embodiment;

[0036] Figure 2 It is a schematic flowchart of an image processing method in another embodiment;

[0037] Figure 3 It is a schematic flowchart of an image processing method in another embodiment;

[0038] Figure 4 It is a schematic flowchart of an image processing method in another embodiment;

[0039] Figure 5 It is a structural block diagram of an image processing device in an embodiment;

[0040] Figure 6 It is an internal structure diagram of a computer device in an embodiment. Detailed implementation manners

[0041] In order to make the purpose, technical solutions and advantages of the present application clearer, the following further details the present application in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0042] A printed circuit board (PCB), also known as a printed wiring board or printed circuit board, is an essential component in modern electronic devices. It is a support for electronic components and a carrier for the electrical connection of electronic components. The circuit board is formed by patterning and laying conductive copper foils on the surface of an insulating material (such as fiberglass, epoxy resin, etc.). These copper foil circuits achieve the electrical connection between electronic components and support the fixation of electronic components on the circuit board. The circuit board can miniaturize and visualize the circuit, facilitating the assembly and maintenance of electronic devices.

[0043] During the verification process of the returned electricity meters, the position acquisition of the components in the circuit board is a key link to ensure the verification quality and efficiency of the returned electricity meters.

[0044] Currently, the positions of components in a circuit board are usually manually collected and confirmed. This method not only consumes time and effort, but also has a high error rate, which is not conducive to improving the verification accuracy and maintenance efficiency of returned electricity meters.

[0045] In view of this, the present application provides an image processing method, device, equipment, readable storage medium and program product, which can improve the verification accuracy and maintenance efficiency of returned electricity meters.

[0046] In one embodiment, as Figure 1 shown, an image processing method is provided. In the embodiment of the present application, this method is illustrated by taking its application to a computer device as an example. The computer device can be a terminal or a server. This method can also be applied to a system including a terminal and a server and implemented through the interaction between the terminal and the server.

[0047] Among them, the terminal can be, but is not limited to, various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, projection devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server can be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0048] In this embodiment, the method includes the following steps 101 and 102:

[0049] Step 101, obtain a circuit board image.

[0050] The circuit board image is obtained by image acquisition of the target circuit board.

[0051] In the embodiment of the present application, a high-resolution industrial camera can be used to perform image acquisition on the target circuit board, and a clear circuit board image can be obtained through shooting environment control during the image acquisition process.

[0052] Among them, the resolution of the camera can be flexibly adjusted according to the size of the target circuit board and the size of the components in the target circuit board. For the optical parameters of the camera (such as focal length and aperture), they can also be flexibly selected before image acquisition. A suitable focal length can ensure a proper imaging ratio of the target circuit board, and the aperture size will affect the depth of field and clarity of the circuit board image. For a multi-layer target circuit board or components with height differences, it is more necessary to select a suitable aperture to ensure that the entire target circuit board can be clearly imaged.

[0053] In addition, the lighting condition is also one of the key factors for clear imaging of the target circuit board. During the implementation process, uniform and stable lighting should be ensured to avoid interference caused by shadows and reflections to the later identification of components. At the same time, the shooting angle should be perpendicular to the surface of the target circuit board to reduce image distortion.

[0054] Optionally, a high-precision robotic arm or fixture can be used to fix the camera and the target circuit board to ensure the accuracy of the shooting angle, so as to ensure that the details of the components on the target circuit board can be clearly captured.

[0055] After the camera captures the circuit board image, the computer device obtains the circuit board image. Exemplarily, the camera can exist in the form of a camera module in the computer device; Exemplarily, the camera can also be independently set outside the computer device, and so on.

[0056] Step 102: Input the circuit board image into the target detection model to obtain the position information and device type of each component in the target circuit board in the circuit board image.

[0057] Among them, the target detection model is trained based on the component feature library and / or multiple first sample circuit board images. The component feature library includes component features extracted from multiple second sample circuit board images. The training method of the target detection model will be introduced in the following embodiments.

[0058] Through the knowledge learned during the training process, after the target detection model converges, the computer device inputs the obtained circuit board image into the pre-trained target detection model, and then the position information and device type of each component in the circuit board image output by the target detection model can be obtained. The position information can include, for example, the coordinates of the position box.

[0059] In the above embodiment, the computer device obtains the circuit board image by image acquisition of the target circuit board. The target circuit board can be the circuit board in the electric energy meter. Then, the circuit board image is input into the target detection model to obtain the position information and device type of each component in the target circuit board in the circuit board image. In this way, the target detection model is pre-trained based on the component feature library (the component feature library includes component features extracted from multiple second sample circuit board images) and / or multiple first sample circuit board images, and the target detection model is used to accurately and quickly identify the position information and device type of each component in the target circuit board, improving the verification accuracy and maintenance efficiency of the electric energy meter.

[0060] In a possible implementation, after the computer device obtains the circuit board image, it can also perform preprocessing on the obtained circuit board image, such as gray conversion, noise removal, image enhancement, etc., so as to improve the image quality of the circuit board image.

[0061] Exemplarily, when the recognition of components mainly depends on features such as shape and size (rather than color information), the color circuit board image can be converted into a grayscale image. For example, the computer device can use a grayscale formula to convert each pixel in the color circuit board image into a grayscale value. The grayscale formula is, for example, the weighted average method. For each pixel, the grayscale value = 0.299*R + 0.587*G + 0.311*B, where R, G, and B are the channel values of the red, blue, and green channels of the pixel respectively. This can not only reduce the amount of data processing but also speed up the operation speed of subsequent algorithms.

[0062] Exemplarily, the computer device can use methods such as Gaussian filtering and median filtering to remove noise in the circuit board image. Gaussian filtering is based on the Gaussian function to perform weighted averaging on the pixels in the circuit board image, thereby effectively removing Gaussian noise; median filtering replaces the grayscale value of the pixel in the circuit board image with the median of the grayscale values of its neighboring pixels, thereby being able to process salt-and-pepper noise.

[0063] Exemplarily, the computer device can also use means such as histogram equalization and contrast stretching to improve the contrast and clarity of the circuit board image. Histogram equalization expands the grayscale range of the circuit board image by redistributing the grayscale histogram of the circuit board image, highlighting the originally blurred component edges and details in the circuit board image; contrast stretching linearly stretches the grayscale interval according to the maximum and minimum grayscale values of the circuit board image, enhancing the visual effect of the circuit board image.

[0064] The computer device then inputs the preprocessed circuit board image into the target detection model to obtain the position information and device types of each component in the target circuit board in the circuit board image.

[0065] Through the above one or more preprocessing methods, the computer device can effectively remove the noise interference in the circuit board image, highlight the component features in the circuit board image, make the circuit board image cleaner and the features more obvious, thereby improving the accuracy and efficiency of subsequent steps such as edge detection and shape feature extraction, and indirectly improving the position acquisition accuracy of the components.

[0066] Based on any of the above embodiments, taking the Figure 1 illustrated embodiment as an example, in this embodiment, the position information of the components in the circuit board image includes the center point coordinates and the bounding box coordinates. Refer to Figure 2 , and the image processing method of this embodiment further includes the following step 201:

[0067] Step 201: According to the position correction strategy corresponding to the position information, perform error correction on the center point coordinates and / or bounding box coordinates of each component.

[0068] Due to factors such as possible deformation of the target circuit board and lens distortion of the camera, there are errors in the center point coordinates and bounding box coordinates of the components in the circuit board image.

[0069] In the embodiments of the present application, the error sources can be analyzed, and the error sources may include one or more of the following:

[0070] 1) Prediction error of the target detection model: Taking the YOLO model framework as an example for the target detection model, the position information predicted by the target detection model may have problems such as offset of the center point coordinates and inaccurate width-to-height ratio of the bounding box coordinates.

[0071] 2) Data annotation error: The annotations (labels) in the training data of the target detection model may be inaccurate or deviated.

[0072] 3) Image distortion error: The camera lens has distortion (such as lens distortion), resulting in deviation of the center point coordinates and bounding box coordinates.

[0073] 4) Environmental factors: Environmental factors such as illumination, occlusion, and background interference may also affect the target detection results.

[0074] Then, for different error sources, corresponding error correction measures are adopted, that is, the computer device determines the position correction strategy corresponding to the error source, and according to this position correction strategy, performs error correction on the center point coordinates and / or bounding box coordinates of each component.

[0075] Hereinafter, the analysis and identification of error sources and the position correction strategy are introduced exemplarily.

[0076] The computer device can collect multiple target detection results obtained by the target detection model for target detection at historical times. The target detection results are such as historical position information. The computer device also collects the true annotation corresponding to each historical position information, or it can be called the true position information.

[0077] Then, for each historical position information, calculate its error value from the true annotation. For example, calculate the distance between the historical center point coordinates included in the historical position information and the true center point coordinates to obtain the center point offset (Δx, Δy); the computer device calculates the width (w) and height (h) differences between the historical bounding box coordinates included in the historical position information and the true bounding box coordinates to obtain the width-to-height ratio error (Δw, Δh); the computer device calculates the overlap degree between the historical bounding box coordinates and the true bounding box coordinates to obtain the IoU (Intersection over Union).

[0078] The computer device performs statistical analysis on each calculated error value to identify the distribution law of the errors. For example, the computer device can calculate statistical quantities such as the mean, variance, and distribution histogram of each error value.

[0079] Next, the computer device determines whether the error values are concentrated in a certain specific direction (such as always shifting to the right) or scale (such as always predicting too large) based on the statistical quantities, and the computer device determines the error source through the statistical quantities. For example, if the center point shift shows a systematic deviation (such as always shifting to the right), the error source may be caused by the prediction error of the target detection model or the image distortion error; for example, if the aspect ratio error is large, the error source may be that the target detection model does not sufficiently learn the target scale, and so on.

[0080] After the computer device determines the error source, it can determine the corresponding position correction strategy. For example, for the case where the error source is lens distortion, the position correction strategy can be to correct the error using the radial distortion and tangential distortion models; for the case where the error source is the deformation of the circuit board, multiple reference points can be set on the target circuit board, and according to the difference between the actual positions of the reference points and the collected position information, the position information of its components can be compensated and corrected, such as performing offset correction on the center point coordinates and / or adjusting the aspect ratio of the bounding box, and so on.

[0081] Exemplarily, a regression model (such as linear regression, support vector regression) can also be used to learn the error compensation function; if the error is complex and non-linear, a neural network can be used to learn the error compensation function, and the learned model can be used for error correction, and so on.

[0082] In the embodiments of the present application, by performing error correction on the position information output by the target detection model, the position information can be accurately determined, providing accurate data support for subsequent processes such as the repair and detection of the target circuit board.

[0083] Based on any of the above embodiments, taking the embodiment shown in Figure 2 as an example, in this embodiment, referring to Figure 3 , the image processing method of this embodiment further includes the following step 301:

[0084] Step 301, verify the accuracy of the position information using a coordinate measuring machine to obtain a verification result.

[0085] As a high-precision measuring device, the coordinate measuring machine can verify the position information output by the target detection model.

[0086] Exemplarily, the computer device can randomly select a certain number (such as 10% - 20%) of components from each component (hereinafter referred to as target components for distinction), and record the position information of these target components predicted by the target detection model. Then, use a coordinate measuring instrument to measure the positions of the extracted target components with high precision to obtain the position information of the target components. Since the image coordinate system usually only provides two-dimensional coordinates (x, y), and the position information measured by the coordinate measuring instrument is three-dimensional coordinates, the Z-axis coordinate in the position information measured by the coordinate measuring instrument can be ignored to align it with the position information in the image coordinate system.

[0087] The computer device compares the error between the position information of the target components predicted by the target detection model and the position information of the target components measured by the coordinate measuring instrument. If the error exceeds the allowable range, a prompt message can be output. This prompt message is used to prompt to re-examine each link such as image acquisition, preprocessing, recognition, and positioning, check for problems and make improvements. For example, for the image acquisition process, the camera resolution, lighting conditions, lens distortion, etc. can be checked; for the preprocessing process, it can be checked whether steps such as image denoising, enhancement, and binarization are reasonable. For the target detection process, the training data, annotation quality, model performance, etc. of the target detection model can be checked. After adjusting the relevant links, it can be retested until the error does not exceed the allowable range to improve the accuracy of the position information of each component.

[0088] Hereinafter, an exemplary introduction to the training process of the target detection model in the embodiments of the present application will be given.

[0089] See Figure 4 , in this embodiment, the training process of the target detection model includes the following steps 401 and 402:

[0090] Step 401, extract component features from multiple second sample circuit board images, and store the extracted component features in the component feature library.

[0091] The second sample circuit board images can be obtained by image acquisition of circuit boards in multiple different states and environments. The computer device performs feature extraction on each second sample circuit board image to obtain component features.

[0092] Exemplarily, the computer device extracting component features from multiple second sample circuit board images includes at least one of the following:

[0093] 1) Use the Hough transform detection algorithm to extract the center information and radius information of circular components or elliptical components from each second sample circuit board image as each component feature.

[0094] Circular components or oval components, such as common components like capacitors and inductors. The Hough transform detection algorithm can convert the circle problem in the image space into a peak search problem in the parameter space. By finding the accumulator peak in the parameter space, the center information and radius information can be determined.

[0095] 2) Use the contour detection algorithm to extract the contour information corresponding to the rectangular components from each second sample circuit board image as the features of each component.

[0096] For rectangular components such as chips, by using the contour detection algorithm in combination with the calculation of the minimum bounding rectangle, the vertex coordinates of the rectangle can be obtained, and then its shape information (i.e., contour information) can be determined.

[0097] 3) Use the gray-level co-occurrence matrix to extract the texture direction, texture contrast, and texture correlation of the components with texture from each second sample circuit board image as the features of each component.

[0098] Through the gray-level co-occurrence matrix, based on the gray values of the pixels in the circuit board image and the gray values of their adjacent pixels, multiple parameters such as the direction, contrast, and correlation of the texture of the components with texture (such as inductors with special patterns on the surface) can be calculated as component features, which can effectively distinguish them from other components without texture.

[0099] 4) Use the color histogram algorithm to extract the color features from each second sample circuit board image as the features of each component.

[0100] When the color of the component plays a key role in target detection, the color histogram algorithm can be used to extract color features. The color histogram shows the distribution of different colors in the circuit board image in a statistical manner. For example, for a colored light-emitting diode, by comparing its color histogram with the known standard color histogram, its color category can be judged to assist the target detection process.

[0101] Step 402: Train the initial target detection model according to the component feature library and / or multiple first sample circuit board images to obtain the target detection model.

[0102] In a possible implementation manner of step 402, the computer device can train the initial target detection model according to the features of each component to obtain the target detection model.

[0103] Directly training the initial target detection model according to the features of each component can reduce the computational amount in the training process of the target detection model, and extracting accurate and effective component features provides a direct basis for the training of the target detection model, making the calculation of coordinates based on feature points more reliable and improving the accuracy and reliability of the overall coordinate acquisition.

[0104] In another possible implementation of step 402, the computer device may train an initial target detection model based on each first sample circuit board image to obtain a target detection model.

[0105] Exemplarily, similar to the second sample circuit board image, the first sample circuit board image may be obtained by collecting images of circuit boards in multiple different states and environments. Then, the first sample circuit board image is finely annotated, and annotations (labels) are added. The annotation information includes the category of components, bounding box coordinates, etc.

[0106] The computer device inputs each first sample circuit board image with added annotations into the initial target detection model for model training. After the model converges, a target detection model is obtained. Here, the initial target detection model refers to a target detection model with parameter initialization.

[0107] In another possible implementation of step 402, the computer device may pre-train the initial target detection model based on each component feature to obtain an intermediate target detection model, and then fine-tune the intermediate target detection model based on each first sample circuit board image to obtain a target detection model.

[0108] The component feature library can be used to initialize the model weights of the target detection model, that is, for pre-training the weights, to help the target detection model have certain recognition capabilities in the initial stage of training, thereby accelerating convergence.

[0109] In another possible implementation of step 402, the computer device may generate multiple generalized sample circuit board images based on each component feature, and train the initial target detection model based on each generalized sample circuit board image and each first sample circuit board image to obtain a target detection model.

[0110] The component features in the component feature library can be used to generate more diverse training samples, thereby improving the generalization ability of the model target detection model.

[0111] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or steps or stages in other steps.

[0112] Based on the same inventive concept, an embodiment of the present application further provides an image processing apparatus for implementing the above-mentioned image processing method. The solution provided by this apparatus for solving problems is similar to the solution described in the above method. Therefore, the specific limitations in one or more embodiments of the following image processing apparatus can refer to the limitations on the image processing method in the foregoing, and will not be repeated here.

[0113] In an exemplary embodiment, as Figure 5 shown, an image processing apparatus is provided, including:

[0114] An acquisition module 501, configured to acquire a circuit board image, where the circuit board image is obtained by performing image acquisition on a target circuit board;

[0115] A target detection module 502, configured to input the circuit board image into a target detection model, and obtain the position information and device type of each component in the target circuit board in the circuit board image;

[0116] Wherein, the target detection model is trained based on a component feature library and / or a plurality of first sample circuit board images, and the component feature library includes component features extracted from a plurality of second sample circuit board images.

[0117] In one embodiment, the position information includes center point coordinates and bounding box coordinates, and the apparatus further includes:

[0118] An error correction module, configured to perform error correction on the center point coordinates and / or the bounding box coordinates of each component according to a position correction strategy corresponding to the position information.

[0119] In one embodiment, the apparatus further includes:

[0120] An extraction module, configured to extract component features from a plurality of the second sample circuit board images, and store the extracted component features in the component feature library;

[0121] A training module, configured to perform model training on an initial target detection model according to the component feature library and / or a plurality of the first sample circuit board images, to obtain the target detection model.

[0122] In one embodiment, the training module is specifically configured to train the initial target detection model according to each of the component features to obtain the target detection model; or, train the initial target detection model according to each of the first sample circuit board images to obtain the target detection model; or, pre-train the initial target detection model according to each of the component features to obtain an intermediate target detection model, and fine-tune the intermediate target detection model according to each of the first sample circuit board images to obtain the target detection model; or, generate a plurality of generalized sample circuit board images according to each of the component features, and train the initial target detection model according to each of the generalized sample circuit board images and each of the first sample circuit board images to obtain the target detection model.

[0123] In one embodiment, the extraction module is specifically configured to perform at least one of the following:

[0124] Use the Hough transform detection algorithm to extract the center information and radius information of circular components or elliptical components from each of the second sample circuit board images as each of the component features;

[0125] Use the contour detection algorithm to extract the contour information of rectangular components from each of the second sample circuit board images as each of the component features;

[0126] Use the gray-level co-occurrence matrix to extract the texture direction, texture contrast, and texture correlation of textured components from each of the second sample circuit board images as each of the component features;

[0127] Use the color histogram algorithm to extract color features from each of the second sample circuit board images as each of the component features.

[0128] In one embodiment, the device further includes:

[0129] A verification module, configured to verify the accuracy of the position information using a coordinate measuring instrument to obtain a verification result.

[0130] Each module in the above image processing device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor of the computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to the above respective modules.

[0131] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 6As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store image processing data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. The computer program, when executed by the processor, implements an image processing method.

[0132] Those skilled in the art can understand that Figure 6 the structure shown in is only a block diagram of some structures related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine some components, or have different component arrangements.

[0133] In an exemplary embodiment, a computer device is provided, including a memory and a processor. A computer program is stored in the memory. When the processor executes the computer program, the following steps are implemented:

[0134] Obtain a circuit board image, where the circuit board image is obtained by image acquisition of a target circuit board;

[0135] Input the circuit board image into a target detection model to obtain the position information and device type of each component in the circuit board image of the target circuit board;

[0136] Among them, the target detection model is trained based on a component feature library and / or multiple first sample circuit board images, and the component feature library includes component features extracted from multiple second sample circuit board images.

[0137] In an embodiment, the position information includes center point coordinates and bounding box coordinates. When the processor executes the computer program, the following steps are also implemented:

[0138] According to the position correction strategy corresponding to the position information, error correction is performed on the center point coordinates and / or the bounding box coordinates of each component.

[0139] In an embodiment, when the processor executes the computer program, the following steps are also implemented:

[0140] Extract component features from multiple second sample circuit board images, and store the extracted component features in the component feature library;

[0141] Train the initial target detection model according to the component feature library and / or multiple first sample circuit board images to obtain the target detection model.

[0142] In one embodiment, when the processor executes the computer program, the following steps are specifically implemented:

[0143] Train the initial target detection model according to each component feature to obtain the target detection model; or,

[0144] Train the initial target detection model according to each first sample circuit board image to obtain the target detection model; or,

[0145] Pre-train the initial target detection model according to each component feature to obtain an intermediate target detection model, and fine-tune the intermediate target detection model according to each first sample circuit board image to obtain the target detection model; or,

[0146] Generate multiple generalized sample circuit board images according to each component feature, and train the initial target detection model according to each generalized sample circuit board image and each first sample circuit board image to obtain the target detection model.

[0147] In one embodiment, when the processor executes the computer program, at least one of the following steps is specifically implemented:

[0148] Use the Hough transform detection algorithm to extract the center information and radius information of circular components or elliptical components from each second sample circuit board image as each component feature;

[0149] Use the contour detection algorithm to extract the contour information of rectangular components from each second sample circuit board image as each component feature;

[0150] Use the gray-level co-occurrence matrix to extract the texture direction, texture contrast, and texture correlation of textured components from each second sample circuit board image as each component feature;

[0151] Use the color histogram algorithm to extract color features from each second sample circuit board image as each component feature.

[0152] In one embodiment, when the processor executes the computer program, the following steps are further implemented:

[0153] Verify the accuracy of the position information using a coordinate measuring machine to obtain a verification result.

[0154] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented:

[0155] Obtain a circuit board image, where the circuit board image is obtained by collecting an image of a target circuit board;

[0156] Input the circuit board image into a target detection model to obtain the position information and device types of each component in the circuit board image of the target circuit board;

[0157] Among them, the target detection model is trained based on a component feature library and / or multiple first sample circuit board images, and the component feature library includes component features extracted from multiple second sample circuit board images.

[0158] In one embodiment, the position information includes center point coordinates and bounding box coordinates. When the computer program is executed by a processor, the following steps are also implemented:

[0159] Perform error correction on the center point coordinates and / or the bounding box coordinates of each component according to the position correction strategy corresponding to the position information.

[0160] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0161] Extract component features from multiple second sample circuit board images and store the extracted component features in the component feature library;

[0162] Train an initial target detection model according to the component feature library and / or multiple first sample circuit board images to obtain the target detection model.

[0163] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0164] Train the initial target detection model according to the component features of each component to obtain the target detection model; or,

[0165] Train the initial target detection model according to each first sample circuit board image to obtain the target detection model; or,

[0166] Pre-train the initial target detection model according to each of the component features to obtain an intermediate target detection model, and fine-tune the intermediate target detection model according to each of the first sample circuit board images to obtain the target detection model; or,

[0167] Generate a plurality of generalized sample circuit board images according to each of the component features, and train the initial target detection model according to each of the generalized sample circuit board images and each of the first sample circuit board images to obtain the target detection model.

[0168] In one embodiment, when the computer program is executed by a processor, it specifically implements at least one of the following steps:

[0169] Use the Hough transform detection algorithm to extract the center information and radius information of circular or elliptical components from each of the second sample circuit board images as each of the component features;

[0170] Use the contour detection algorithm to extract the contour information of rectangular components from each of the second sample circuit board images as each of the component features;

[0171] Use the gray-level co-occurrence matrix to extract the texture direction, texture contrast, and texture correlation of the components with texture from each of the second sample circuit board images as each of the component features;

[0172] Use the color histogram algorithm to extract color features from each of the second sample circuit board images as each of the component features.

[0173] In one embodiment, when the computer program is executed by a processor, it also implements the following steps:

[0174] Use a three-coordinate measuring instrument to verify the accuracy of the position information to obtain a verification result.

[0175] In one embodiment, when the computer program is executed by a processor, it also implements the following steps:

[0176] Obtain a circuit board image, which is obtained by image acquisition of a target circuit board;

[0177] Input the circuit board image into the target detection model to obtain the position information and device type of each component in the target circuit board in the circuit board image;

[0178] Wherein, the target detection model is trained based on a component feature library and / or a plurality of first sample circuit board images, and the component feature library includes component features extracted from a plurality of second sample circuit board images.

[0179] In one embodiment, the location information includes the center point coordinates and the bounding box coordinates. When the computer program is executed by a processor, the following steps are further implemented:

[0180] According to the position correction strategy corresponding to the location information, error correction is performed on the center point coordinates and / or the bounding box coordinates of each of the components.

[0181] In one embodiment, when the computer program is executed by a processor, the following steps are further implemented:

[0182] Extract component features from multiple second sample circuit board images and store the extracted component features in the component feature library;

[0183] Train an initial target detection model according to the component feature library and / or multiple first sample circuit board images to obtain the target detection model.

[0184] In one embodiment, when the computer program is executed by a processor, the following steps are specifically implemented:

[0185] Train the initial target detection model according to each of the component features to obtain the target detection model; or,

[0186] Train the initial target detection model according to each of the first sample circuit board images to obtain the target detection model; or,

[0187] Pre-train the initial target detection model according to each of the component features to obtain an intermediate target detection model, and fine-tune the intermediate target detection model according to each of the first sample circuit board images to obtain the target detection model; or,

[0188] Generate multiple generalized sample circuit board images according to each of the component features, and train the initial target detection model according to each of the generalized sample circuit board images and each of the first sample circuit board images to obtain the target detection model.

[0189] In one embodiment, when the computer program is executed by a processor, at least one of the following steps is specifically implemented:

[0190] Use the Hough transform detection algorithm to extract the center information and radius information corresponding to circular components or elliptical components from each of the second sample circuit board images as each of the component features;

[0191] Use the contour detection algorithm to extract the contour information corresponding to rectangular components from each of the second sample circuit board images as each of the component features;

[0192] Extract the texture direction, texture contrast, and texture correlation of the textured components from each of the second sample circuit board images using the gray-level co-occurrence matrix as the features of each of the components;

[0193] Extract color features from each of the second sample circuit board images using the color histogram algorithm as the features of each of the components.

[0194] In one embodiment, when the computer program is executed by the processor, the following steps are further implemented:

[0195] Verify the accuracy of the position information using a three-coordinate measuring instrument to obtain a verification result.

[0196] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, database, or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The databases involved in the embodiments provided in the present application can include at least one of relational databases and non-relational databases. Non-relational databases can include distributed databases based on blockchain, etc., without limitation. The processors involved in the embodiments provided in the present application can be general-purpose processors, central processors, graphics processors, digital signal processors, programmable logic devices, data processing logics based on quantum computing, artificial intelligence (AI) processors, etc., without limitation.

[0197] The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in the present application.

[0198] The above-described embodiments merely represent several implementation manners of the present application. The description thereof is relatively specific and detailed, but it should not be construed as a limitation on the patent scope of the present application. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all fall within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. An image processing method, characterized in that, The method includes: Obtaining a circuit board image, where the circuit board image is obtained by image acquisition of a target circuit board; Inputting the circuit board image into a target detection model to obtain the position information and device type of each component in the target circuit board in the circuit board image; Wherein, the target detection model is trained based on a component feature library and / or multiple first sample circuit board images, and the component feature library includes component features extracted from multiple second sample circuit board images.

2. The method according to claim 1, wherein The position information includes center point coordinates and bounding box coordinates, and the method further includes: Performing error correction on the center point coordinates and / or the bounding box coordinates of each component according to the position correction strategy corresponding to the position information.

3. The method according to claim 1, wherein The method further includes: Extracting component features from multiple second sample circuit board images and storing the extracted component features in the component feature library; Training an initial target detection model according to the component feature library and / or multiple first sample circuit board images to obtain the target detection model.

4. The method according to claim 3, wherein The training the initial target detection model according to the component feature library and / or multiple first sample circuit board images to obtain the target detection model includes: Training the initial target detection model according to each component feature to obtain the target detection model; or, Training the initial target detection model according to each first sample circuit board image to obtain the target detection model; or, Pre-training the initial target detection model according to each component feature to obtain an intermediate target detection model, and fine-tuning the intermediate target detection model according to each first sample circuit board image to obtain the target detection model; or, Generating multiple generalized sample circuit board images according to each component feature, and training the initial target detection model according to each generalized sample circuit board image and each first sample circuit board image to obtain the target detection model.

5. The method according to claim 3, characterized in that, The extracting component features from multiple second sample circuit board images includes at least one of the following: Using the Hough transform detection algorithm to extract the center information and radius information of circular components or elliptical components from each second sample circuit board image as each component feature; Using the contour detection algorithm to extract the contour information of rectangular components from each second sample circuit board image as each component feature; Using the gray-level co-occurrence matrix to extract the texture direction, texture contrast, and texture correlation of textured components from each second sample circuit board image as each component feature; Using the color histogram algorithm to extract color features from each second sample circuit board image as each component feature.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Verifying the accuracy of the position information using a three-coordinate measuring instrument to obtain a verification result.

7. An image processing apparatus, characterized in that, The device includes: An acquisition module for acquiring a circuit board image, where the circuit board image is obtained by image acquisition of a target circuit board; The target detection module is configured to input the circuit board image into a target detection model to obtain the position information and device types of each component in the target circuit board in the circuit board image; Wherein, the target detection model is trained based on a component feature library and / or a plurality of first sample circuit board images, and the component feature library includes component features extracted from a plurality of second sample circuit board images.

8. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images