Printed circuit board assembly cleaning method, apparatus, storage medium and program product

Abstract

The present application discloses a cleaning method, device, storage medium and program product for a printed circuit board assembly, and relates to the technical field of cleaning printed circuit board assemblies. The present application generates a corresponding three-dimensional cleaning path by combining a deep learning model and a path planning model, thereby realizing automatic cleaning of the printed circuit board assembly to be cleaned, improving the cleaning efficiency of the product, and avoiding foreign matter residue by verifying the cleaning results, thereby improving the quality and reliability of product cleaning. Based on this, the present application can reduce labor costs, improve production efficiency, and also improve the yield rate of the product.

Description

Technical Field

[0001] The present application relates to the technical field of cleaning printed circuit board assemblies, and in particular to a cleaning method, equipment, storage medium, and program product for a printed circuit board assembly. Background Art

[0002] During the manufacturing and transportation of server PCBAs (Printed Circuit Board Assemblies), dust, solder beads, and other foreign matter adsorbed during production and transportation may be adsorbed on the PCBA products. These abnormal conditions will cause product damage and increased defective rates in subsequent production processes. Related technologies use air guns to blow off foreign matter on the PCBA or vacuum cleaners to remove foreign matter on the PCBA, but this has high labor costs. Manual operation can also cause product collisions, indirectly leading to increased product defective rates and low production efficiency. Related technologies can also use automatic cleaning equipment to clean foreign matter on the PCBA. This automatic cleaning equipment requires dividing the product cleaning area for each model and teaching the points. The equipment adjustment and setup time is long and it has poor adaptability to diverse PCBA designs. Summary of the Invention

[0003] The present application provides a cleaning method, device, storage medium and program product for a printed circuit board assembly, so as to at least solve the problems of high labor cost, high product defect rate and low production efficiency in the related art.

[0004] The present application provides a method for cleaning a printed circuit board assembly, comprising:

[0005] Acquire a first target image of the printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on the deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0006] Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, a cleaning device is controlled to perform a cleaning operation;

[0007] In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether foreign matter remains based on the surface state data;

[0008] In response to the presence of foreign matter residue, position information of the foreign matter residue is marked, and the remaining foreign matter is cleaned according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0009] The present application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for implementing the following steps of a printed circuit board assembly cleaning method when executing the computer program:

[0010] Acquire a first target image of the printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on the deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0011] Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, a cleaning device is controlled to perform a cleaning operation;

[0012] In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether foreign matter remains based on the surface state data;

[0013] In response to the presence of foreign matter residue, position information of the foreign matter residue is marked, and the remaining foreign matter is cleaned according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0014] The present application also provides a computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, wherein when the computer program is executed by a processor, the following steps of a method for cleaning a printed circuit board assembly are implemented:

[0015] Acquire a first target image of the printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on the deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0016] Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, a cleaning device is controlled to perform a cleaning operation;

[0017] In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether foreign matter remains based on the surface state data;

[0018] In response to the presence of foreign matter residue, position information of the foreign matter residue is marked, and the remaining foreign matter is cleaned according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0019] The present application also provides a computer program product, comprising a computer program, which, when executed by a processor, implements the following steps of a method for cleaning a printed circuit board assembly:

[0020] Acquire a first target image of the printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on the deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0021] Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, a cleaning device is controlled to perform a cleaning operation;

[0022] In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether foreign matter remains based on the surface state data;

[0023] In response to the presence of foreign matter residue, position information of the foreign matter residue is marked, and the remaining foreign matter is cleaned according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0024] This application combines a deep learning model and a path planning model to generate a corresponding three-dimensional cleaning path, which can realize automatic cleaning of the printed circuit board assembly to be cleaned, improve the cleaning efficiency of the product, and avoid foreign matter residue by verifying the cleaning results, thereby improving the quality and reliability of product cleaning. Based on this, this application can reduce labor costs, improve production efficiency, and also improve the yield rate of products. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0026] Figure 1 An application environment diagram of a printed circuit board assembly cleaning method is provided for an embodiment of the present application;

[0027] Figure 2 An overall flow chart of a method for cleaning a printed circuit board assembly is provided for an embodiment of the present application;

[0028] Figure 3 A schematic diagram of a calibration board for a printed circuit board assembly cleaning method is provided in an embodiment of the present application;

[0029] Figure 4 FIG. 1 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0030] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0031] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0032] It should be noted that the terms "S1", "S2", etc. are used only for the purpose of describing the steps and do not specifically refer to the order or sequence, nor are they used to limit this application. They are merely for the convenience of describing the method of this application and should not be understood as indicating the order of the steps. In addition, the technical solutions between the various embodiments can be combined with each other, but this must be based on the fact that ordinary technicians in this field can implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by this application.

[0033] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0034] The cleaning method of printed circuit board assembly provided in this application can be applied to Figure 1 In the application environment shown, the terminal 102 communicates with a data processing platform provided on the server 104 via a network. The terminal 102 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0035] like Figure 2 As shown, the embodiment of the present application provides a cleaning method for a printed circuit board assembly, wherein the method is applied to Figure 1 The following steps are used as an example to illustrate the terminal in the figure:

[0036] S1: Acquire a first target image of a printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on a deep learning model, where the recognition result at least includes foreign matter distribution data.

[0037] It should be noted that the printed circuit board assembly refers to the PCBA; the printed circuit board assembly to be cleaned refers to the PCBA that needs to be cleaned; the first target image refers to the image of the area to be cleaned on the PCBA, which is obtained by a camera fixedly installed on the cleaning equipment, and the preferred pixel value of the camera is 12 million pixels; the cleaning equipment is also provided with a robotic arm, which cleans the printed circuit board assembly to be cleaned through the end effector of the robotic arm; in addition to the foreign matter distribution data, the recognition results also include the machine type and component layout, among which the machine type refers to the PCBA model, the component layout refers to the distribution of key components on the PCBA, and the foreign matter distribution data refers to the distribution of foreign matter on the PCBA. Foreign matter includes process residue foreign matter (such as solder balls, flux, etc.), environmental attachment foreign matter (such as dust, fingerprints, etc.), and aging and shedding foreign matter (such as sealant debris, etc.).

[0038] S2: Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, the cleaning device is controlled to perform a cleaning operation.

[0039] It should be noted that the path planning model generally includes an input layer, a processing layer, and an output layer. The input layer is used to receive multi-dimensional feature inputs, such as PCBA three-dimensional topology maps (including component height, restricted area coordinates, etc.), foreign matter distribution heat maps (0-1 standardized density values), and equipment motion constraint parameters (such as maximum acceleration, cleaning head working radius, etc.); the processing layer uses a graph convolutional network (GCN) to process component spatial relationships, generate node embedding vectors, and output a path point sequence through a Seq2Seq structure with a dual attention mechanism (spatial attention + temporal attention). It also receives force sensor feedback in real time and predicts the cleaning parameters of the next path point (such as pressure and speed) through an LSTM network; the output layer is used to generate instructions corresponding to the three-dimensional cleaning path (such as coordinates, time, motion speed, and cleaning force, etc.).

[0040] S3: In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is any foreign matter remaining based on the surface state data.

[0041] It should be noted that by comparing the surface state data of the printed circuit board assembly to be cleaned with the surface state data of the cleaned printed circuit board assembly, it can be determined whether there are any foreign matter residues based on the comparison results.

[0042] S4: In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0043] It should be noted that after cleaning the remaining foreign matter according to the position information, the surface state data of the printed circuit board assembly to be cleaned is still compared with the surface state data of the cleaned printed circuit board assembly. Based on the comparison results, it is determined whether there are foreign matter residues. When no foreign matter remains, cleaning is stopped.

[0044] In the above embodiment, by combining the deep learning model and the path planning model to generate the corresponding three-dimensional cleaning path, automatic cleaning of the printed circuit board assembly to be cleaned can be achieved, thereby improving the cleaning efficiency of the product. By verifying the cleaning results, foreign matter residue is avoided, thereby improving the quality and reliability of product cleaning. Based on this, the present application can reduce labor costs, improve production efficiency, and improve the yield rate of products.

[0045] In some specific embodiments, the method for constructing a deep learning model includes:

[0046] Build a multi-task network framework, where the multi-task network framework can choose pytorch, etc.

[0047] Determine a first network model for processing a first target task, where the first target task is to identify the machine type. The first network model can be a ResNet-50 model for identifying PCBA models. The machine type can be classified based on functional type, electrical characteristics, or physical structure features. For example, functional types such as communication modules and industrial control motherboards can be distinguished by onboard identification (QR code, serial number), key component layout (such as CPU / FPGA package type), or electrical parameters (such as impedance matching requirements).

[0048] Determine a second network model for processing the second objective task, which is to determine component layout. The second network model can be a Mask R-CNN model, which is used to perform pixel-level segmentation of key components. The component layout includes the spatial distribution characteristics of components, such as component type: surface mount device (SMD), through-hole plug (THD), and ball grid array (BGA); height topology: the three-dimensional height distribution of different components, and the Z-axis obstacle avoidance strategy that affects the cleaning path; sensitive areas: protection areas for highly fragile components (such as MLCCs and optical sensors) or heat dissipation restricted areas for high-heat dissipation components;

[0049] Determine a third network model for processing the third objective task, which is to determine foreign matter distribution data. The third network model can be a U-Net model, which is used to quantify the foreign matter distribution heat map. The distribution characteristics of the foreign matter include: spatial distribution: discrete (randomly scattered) or aggregated (such as accumulation around a pad); its physical properties include: particle size (0.01mm-5mm), adhesion (weak adsorption / strong adhesion), optical reflectivity (difference in metal / non-metal reflection), etc.

[0050] The first network model, the second network model and the third network model are integrated into a multi-task network framework to generate a deep learning model.

[0051] In some specific embodiments, determining a recognition result of the first target image based on the deep learning model includes:

[0052] Inputting the first target image into the deep learning model;

[0053] Parallel execution of the first network model, the second network model, and the third network model in the deep learning model to output a recognition result of the first target image;

[0054] Among them, the recognition results include the machine type output by the first network model, the component layout output by the second network model, and the foreign matter distribution data output by the third network model. The above-mentioned deep learning model is a model that has been pre-trained through training sets and test sets. Its corresponding training and testing methods are commonly used methods, and the specific process will not be repeated here.

[0055] In the above embodiment, by executing multiple task threads in parallel to generate the machine type, component layout and foreign matter distribution data required for subsequent path planning, the processing efficiency of the deep learning model can be improved, thereby improving the cleaning efficiency.

[0056] In some specific real-time methods, the path planning model construction method includes:

[0057] Constructing a first path planning model, the first path planning model includes an input layer, a processing layer, and an output layer;

[0058] Obtain a training dataset, where the training dataset refers to historical image information, including the paths planned according to the foreign objects, annotating the paths in the historical images, and dividing the historical image information into a training set, a validation set, and a test set in a ratio of 7:2:1 for subsequent model training;

[0059] Determine a training loss function, and train the first path planning model once based on the training data set and the training loss function to obtain a second path planning model. The training once refers to offline pre-training of the model. The training loss function includes:

[0060] ;

[0061] in, L represents the loss value, 、 and Both represent weight coefficients, S 1 represents the path deviation loss,S 2 represents the loss of cleaning coverage, S 3 represents the equipment wear penalty coefficient;

[0062] A clean reward function is determined, and the second path planning model is trained again based on the training data set and the clean reward function. The second training refers to performing online reinforcement learning on the model to obtain a third path planning model. The clean reward function includes:

[0063] ;

[0064] in, K Represents the cleaning reward value, and Both represent weight coefficients, R Indicates the cleaning coverage, T represents the time penalty coefficient, Y Indicates the detergent dosage coefficient;

[0065] In response to the training result meeting the preset criterion, the third path planning model is defined as the path planning model.

[0066] In some embodiments, defining the preset criteria includes the path planning model satisfying an objective function and constraints, wherein the objective function includes:

[0067] ;

[0068] Where t represents the cleaning time, It represents the exchange rate of time and resources in s / ml, and W represents the consumption of cleaning agent in ml;

[0069] Constraints include:

[0070] Δ d ≥ h ;

[0071] z ≤ H +1;

[0072] Among them, Δ d Indicates the distance between the end effector of the robot arm and the components on the printed circuit board assembly, represents the first safety threshold, Indicates the number of turns of the end effector of the robot arm, Represents the second safety threshold, where the first safety threshold can be set according to actual needs, and the second safety threshold is determined by the number of turns required in the planned path.

[0073] In the above embodiment, the cleaning path is planned by setting a path planning model, and the cleaning efficiency is improved through automation and intelligent means. The planned cleaning path can increase the cleaning coverage while reducing the cost of cleaning agents, and can avoid collisions with components while ensuring the cleaning intensity, thereby improving the cleaning quality.

[0074] In some specific embodiments, before controlling the cleaning device to perform a cleaning operation based on the first target three-dimensional cleaning path, the method further includes:

[0075] Obtain a first coordinate point set of the end effector of the robotic arm of the cleaning device in the cleaning device coordinate system, and a second coordinate point set corresponding to the first coordinate point set in the camera coordinate system, wherein the coordinate points in the first coordinate point set correspond one-to-one to the coordinates in the second coordinate point set;

[0076] According to the first coordinate point set and the second coordinate point set, a transformation matrix between the cleaning device coordinate system and the camera coordinate system is determined, where Figure 3 The calibration plate shown in the figure uses a 9-point calibration method to calculate the transformation matrix. The calculation method is a common method and the specific calculation process is not repeated here. The transformation matrix is ​​a 4×4 matrix, specifically:

[0077] ;

[0078] According to the third coordinate point set corresponding to the first target three-dimensional cleaning path, the fourth coordinate point set of the first target three-dimensional cleaning path in the cleaning equipment coordinate system is determined according to the transformation matrix and the third coordinate point set, wherein the coordinate points in the third coordinate point set correspond one-to-one with the coordinates in the fourth coordinate point set, such as the transformation formula corresponding to the (x, y, z) coordinates is: , that is, the transformed coordinates are ( x 1, y 1, z 1);

[0079] Based on the fourth coordinate point set, a second target three-dimensional cleaning path corresponding to the first target three-dimensional cleaning path is determined.

[0080] In some specific embodiments, after determining the second target three-dimensional cleaning path corresponding to the first target three-dimensional cleaning path, the method further includes:

[0081] determining a target cleaning area according to the first target three-dimensional cleaning path, wherein the target cleaning area only needs to cover the first target three-dimensional cleaning path;

[0082] Determine a reference point based on the target cleaning area, wherein the reference point is any point close to the boundary of the area corresponding to the two end points of the first target three-dimensional cleaning path;

[0083] Calculate the Euclidean distance between the coordinate point corresponding to the first target three-dimensional cleaning path and the reference point, where the coordinates used to calculate the Euclidean distance only include the x-axis and y-axis coordinates of the coordinate point and the reference point. The calculation formula for the Euclidean distance is:

[0084] ;

[0085] in,( x 1, y 1) represents the coordinates of the reference point, ( x 2, y 2) Indicates the coordinates of the coordinate point;

[0086] Based on the priority calculation function, the sorting results corresponding to the multiple Euclidean distances are determined, wherein the smaller the y value, the higher the priority. According to the sorting results, the first target coordinate point is determined, that is, the coordinate point corresponding to the Euclidean distance with the highest priority is defined as the first target coordinate point. The priority calculation function includes:

[0087] ;

[0088] Among them, y represents the priority coefficient, x represents the Euclidean distance, and k represents the correction parameter;

[0089] A second target coordinate point corresponding to the first target coordinate point on the second target three-dimensional cleaning path is determined, and the second target coordinate point is defined as a starting point of the cleaning path.

[0090] In some specific embodiments, before controlling the cleaning device to perform the cleaning operation, the method further includes:

[0091] Build a digital twin model of the PCBA cleaning system and rehearse the cleaning path in the twin environment to execute the cleaning operation;

[0092] Determine the offset of the end effector of the robotic arm, the amount of cleaning agent, and environmental data at each time node based on the motion state of the physical device;

[0093] Normalize the offset, detergent inventory, and environmental data. Based on the normalization results and the correction function, determine the correction amount at different time points. The normalization method is a common method, and the specific processing process is not repeated here. The correction function includes:

[0094] ;

[0095] in, G Indicates the correction amount, 、 and Both represent weight coefficients, B 1 represents the offset coefficient,B 2 represents the detergent stock coefficient, B 3 represents the environmental data coefficient;

[0096] Generate a one-to-one mapping relationship between the time node and the correction amount, and generate a mapping table, wherein the time node is used to represent the cleaning time;

[0097] The correction amount at the same time point in multiple preview results is subjected to deviation calculation, and in response to the deviation value being greater than a third preset value, the cleaning path is adjusted until the deviation value is less than or equal to the third preset value, wherein the deviation calculation refers to the difference calculation of any two correction amounts.

[0098] In the above embodiment, by simulating the cleaning path to perform the corresponding cleaning operation, the cleaning path with a large deviation can be corrected, thereby improving the cleaning reliability.

[0099] In some embodiments, controlling the cleaning device to perform a cleaning operation based on the first target three-dimensional cleaning path includes:

[0100] The cleaning device is controlled to perform a cleaning operation according to the cleaning path starting point and the second target three-dimensional cleaning path corresponding to the first target three-dimensional cleaning path.

[0101] By calculating and determining a transformation matrix to convert a cleaning path determined based on a camera image into a cleaning path of a robot end effector and determining a starting point of the cleaning path to clean a printed circuit board assembly to be cleaned, cleaning reliability can be improved.

[0102] In some specific embodiments, in response to cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is residual foreign matter based on the surface state data includes:

[0103] In response to the cleaning being completed, acquiring a second target image of the printed circuit board assembly after the cleaning is completed;

[0104] Extracting surface state data of the second target image, and comparing the surface state data of the second target image with the surface state data of the first target image, wherein the surface state data may include two-dimensional morphological data, three-dimensional topographic data, etc.;

[0105] In response to the comparison being inconsistent, determining that no foreign matter remains;

[0106] In response to the comparison being consistent, it is determined that foreign matter remains.

[0107] In some specific embodiments, extracting the surface state data of the second target image includes:

[0108] Acquire a cleaning point corresponding to the first target image;

[0109] Performing circle expansion processing based on the cleaning points to obtain a circle expansion processing result. The circle expansion processing refers to drawing a circle with the coordinate as the center and fitting the drawn circle into the second target image;

[0110] determining an extraction range of the surface state data of the second target image based on the circle expansion processing result;

[0111] Surface state data of the second target image is extracted according to the extraction range.

[0112] In some specific embodiments, after determining that foreign matter remains, the method further comprises:

[0113] Determine the relevant attribute information of the residual foreign matter, including coverage, material attributes, and the area to which it belongs. The material attributes include whether it is a conductive material, the coverage includes the single-point residual coverage and the total coverage, and the area to which it belongs includes whether it is a critical area (such as a high-frequency signal area).

[0114] Standardize the relevant attribute information and determine the grade of the residual foreign matter based on the standardization results and the grading function. The standardization method is a commonly used method and the specific processing process will not be repeated here. The grading function includes:

[0115] ;

[0116] in, F represents the grading coefficient, 、 and Both represent weight coefficients, G 1 represents the single point coverage, G 2 Represents the material property coefficient, G 3 Indicates the regional coefficient;

[0117] In response to the classification coefficient being less than a first preset value, defining the grade of the residual foreign matter as a first-grade residual;

[0118] In response to the classification coefficient being greater than or equal to a first preset value and less than a second preset value, defining the level of the residual foreign matter as level 2 residual;

[0119] In response to the classification coefficient being greater than the second preset value, the grade of the residual foreign matter is defined as grade three residual.

[0120] Specifically, the first preset value and the second preset value can be set according to actual needs. Level 1 residue is an acceptable residue, level 2 residue is a residue that requires human intervention, and level 3 residue is a residue that requires early warning.

[0121] In some specific embodiments, in response to the presence of foreign matter residue, marking the location information of the foreign matter residue, and cleaning the remaining foreign matter according to the location information until no foreign matter residue is left on the printed circuit board assembly to be cleaned includes:

[0122] In response to the residual foreign matter being classified as level one or level two, the residual foreign matter is cleaned according to the position information until no foreign matter remains on the printed circuit board assembly to be cleaned, wherein when the residual foreign matter is classified as level one or level two, cleaning parameters may be adjusted to perform cleaning, such as increasing a dosage of a detergent or increasing a cleaning intensity.

[0123] In response to the level of the residual foreign matter being level three, a prompt message is sent to the terminal, where the prompt message is used to remind the user to deal with the residual foreign matter in a timely manner.

[0124] In the above embodiment, by verifying the cleaned products and identifying residual foreign matter, the cleaning reliability is improved, and by grading the residual foreign matter, the cleaning process is accurately controlled to avoid excessive cleaning, save resources, and improve cleaning reliability.

[0125] The above-mentioned cleaning method for a printed circuit board assembly includes: obtaining a first target image of the printed circuit board assembly to be cleaned, determining a recognition result of the first target image based on a deep learning model, wherein the recognition result includes at least foreign matter distribution data; based on the recognition result, generating a first target three-dimensional cleaning path using a pre-built path planning model, and controlling a cleaning device to perform a cleaning operation based on the first target three-dimensional cleaning path; in response to the completion of cleaning, obtaining surface state data of the printed circuit board assembly to be cleaned, and judging whether there are foreign matter residues based on the surface state data; in response to the presence of foreign matter residues, marking the position information of the foreign matter residues, and cleaning the remaining foreign matter based on the position information until there are no foreign matter residues on the printed circuit board assembly to be cleaned. This application combines the deep learning model and the path planning model to generate a corresponding three-dimensional cleaning path, thereby realizing automatic cleaning of the printed circuit board assembly to be cleaned, improving the cleaning efficiency of the product, and avoiding foreign matter residues by verifying the cleaning results, thereby improving the quality and reliability of product cleaning. Based on this, this application can reduce labor costs, improve production efficiency, and improve the yield rate of products.

[0126] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0127] It should be understood that although Figure 2The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. In addition, Figure 2 At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0128] In one embodiment, an electronic device is provided. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 4 As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected via a system bus. The processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the electronic device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for cleaning a printed circuit board assembly is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the electronic device, or an external keyboard, touchpad or mouse, etc.

[0129] Those skilled in the art will understand that Figure 4 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the electronic device to which the solution of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0130] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to perform the steps of an embodiment of a method for cleaning a printed circuit board assembly, including:

[0131] S1: Acquire a first target image of a printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on a deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0132] S2: Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, the cleaning device is controlled to perform a cleaning operation;

[0133] S3: in response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is any foreign matter remaining based on the surface state data;

[0134] S4: In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0135] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored therein, wherein the computer program is configured to execute the steps of an embodiment of a cleaning method for a printed circuit board assembly when executed, including:

[0136] S1: Acquire a first target image of a printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on a deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0137] S2: Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, the cleaning device is controlled to perform a cleaning operation;

[0138] S3: in response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is any foreign matter remaining based on the surface state data;

[0139] S4: In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0140] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0141] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the embodiment of the method for cleaning a printed circuit board assembly are implemented, including:

[0142] S1: Acquire a first target image of a printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on a deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0143] S2: Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, the cleaning device is controlled to perform a cleaning operation;

[0144] S3: in response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is any foreign matter remaining based on the surface state data;

[0145] S4: In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0146] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the steps of the embodiment of the method for cleaning a printed circuit board assembly are implemented, including:

[0147] S1: Acquire a first target image of a printed circuit board assembly to be cleaned, and determine a recognition result of the first target image based on a deep learning model, wherein the recognition result includes at least foreign matter distribution data;

[0148] S2: Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, the cleaning device is controlled to perform a cleaning operation;

[0149] S3: in response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is any foreign matter remaining based on the surface state data;

[0150] S4: In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned.

[0151] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0152] The above is a detailed introduction to the cleaning method, device, electronic device, and storage medium for a printed circuit board assembly provided by the present application. Specific examples are used herein to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only intended to help understand the method and core concept of the present application. It should be noted that, for those skilled in the art, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the present application.

Claims

1. A method for cleaning a printed circuit board assembly, characterized in that: The method includes: acquiring a first target image of a printed circuit board assembly to be cleaned, and determining a recognition result of the first target image based on a deep learning model, wherein the recognition result includes at least foreign matter distribution data; Based on the recognition result, a first target three-dimensional cleaning path is generated using a pre-built path planning model, and based on the first target three-dimensional cleaning path, a cleaning device is controlled to perform a cleaning operation; In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and determining whether there is any foreign matter remaining based on the surface state data; In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned; The method for constructing a path planning model includes: training a first path planning model using a training data set and a training loss function to determine a second path planning model, and training the second path planning model using the training data set and a clean reward function to determine the path planning model; the training loss function includes: L=α1S1+α2S2+α3S3; Where L represents the loss value, α1, α2, and α3 represent weight coefficients, S1 represents the path deviation loss, S2 represents the cleaning coverage loss, and S3 represents the equipment wear penalty coefficient; The cleaning reward function includes: K=R×(1-δ1×T)+δ2Y; Among them, K represents the cleaning reward value, δ1 and δ2 represent weight coefficients, R represents the cleaning coverage rate, T represents the time penalty coefficient, and Y represents the detergent usage coefficient.

2. The method for cleaning a printed circuit board assembly according to claim 1, wherein: The recognition result also includes the machine type and component layout. The method for constructing the deep learning model includes: Build a multi-task network framework; determining a first network model for processing a first target task, wherein the first target task is to identify the type of the machine model; determining a second network model for processing a second target task, wherein the second target task is determining the component layout; Determining a third network model for processing a third target task, wherein the third target task is determining foreign body distribution data; The first network model, the second network model, and the third network model are integrated into the multi-task network framework to generate the deep learning model.

3. The method for cleaning a printed circuit board assembly according to claim 2, wherein: Determining a recognition result of the first target image based on the deep learning model includes: inputting the first target image into the deep learning model; executing the first network model, the second network model, and the third network model in the deep learning model in parallel, and outputting a recognition result of the first target image; The recognition result includes the machine type output by the first network model, the component layout output by the second network model, and the foreign matter distribution data output by the third network model.

4. The method for cleaning a printed circuit board assembly according to claim 1, wherein: The method for constructing the path planning model includes: Constructing a first path planning model, wherein the first path planning model includes an input layer, a processing layer, and an output layer; Get the training dataset; Determine a training loss function, and train the first path planning model once based on the training data set and the training loss function to obtain a second path planning model; determining a clean reward function, and performing secondary training on the second path planning model based on the training data set and the clean reward function to obtain a third path planning model; In response to the training result meeting a preset standard, the third path planning model is defined as the path planning model.

5. The method for cleaning a printed circuit board assembly according to claim 4, wherein: The method further comprises: Defining the preset criteria includes the path planning model satisfying an objective function and constraints, wherein the objective function includes: min(∑(t+μW)); Where t represents the cleaning time, μ represents the exchange ratio of time to resources, in s / ml, and W represents the consumption of cleaning agent, in ml; The constraints include: Δd≥h; z≤H+1; Wherein, Δd represents the distance between the end effector of the robot arm and the components on the printed circuit board assembly, h represents the first safety threshold, z represents the number of turns of the end effector of the robot arm, and H represents the second safety threshold.

6. The method for cleaning a printed circuit board assembly according to claim 1, wherein: Before controlling the cleaning device to perform a cleaning operation based on the first target three-dimensional cleaning path, the method further includes: Acquire a first coordinate point set of the robot arm end effector of the cleaning device in the cleaning device coordinate system, and a second coordinate point set corresponding to the first coordinate point set in the camera coordinate system; Determining a transformation matrix between the cleaning device coordinate system and the camera coordinate system based on the first coordinate point set and the second coordinate point set; According to the third coordinate point set corresponding to the first target three-dimensional cleaning path, according to the transformation matrix and the third coordinate point set, determining a fourth coordinate point set of the first target three-dimensional cleaning path in the cleaning equipment coordinate system; Based on the fourth coordinate point set, a second target three-dimensional cleaning path corresponding to the first target three-dimensional cleaning path is determined.

7. The method for cleaning a printed circuit board assembly according to claim 6, wherein: After determining a second target three-dimensional cleaning path corresponding to the first target three-dimensional cleaning path, the method further includes: determining a target cleaning area according to the first target three-dimensional cleaning path; Determining a reference point based on the target cleaning area; Calculating the Euclidean distance between the coordinate point corresponding to the first target three-dimensional cleaning path and the reference point; Determine sorting results corresponding to a plurality of Euclidean distances based on a priority calculation function, and determine a first target coordinate point based on the sorting results, wherein the priority calculation function includes: Among them, y represents the priority coefficient, x represents the Euclidean distance, and k represents the correction parameter; A second target coordinate point corresponding to the first target coordinate point on the second target three-dimensional cleaning path is determined, and the second target coordinate point is defined as a starting point of the cleaning path.

8. The method for cleaning a printed circuit board assembly according to claim 7, wherein: Controlling the cleaning device to perform a cleaning operation based on the first target three-dimensional cleaning path includes controlling the cleaning device to perform a cleaning operation according to the cleaning path starting point and a second target three-dimensional cleaning path corresponding to the first target three-dimensional cleaning path.

9. The method for cleaning a printed circuit board assembly according to claim 1, wherein: In response to the cleaning being completed, obtaining surface state data of the printed circuit board assembly to be cleaned, and judging whether there is foreign matter remaining according to the surface state data includes: In response to the cleaning being completed, acquiring a second target image of the cleaned printed circuit board assembly; extracting surface condition data of the second target image, and comparing the surface condition data of the second target image with the surface condition data of the first target image; In response to the comparison being inconsistent, determining that no foreign matter remains; In response to the comparison being consistent, it is determined that foreign matter remains.

10. The method for cleaning a printed circuit board assembly according to claim 9, wherein: Extracting the surface state data of the second target image includes: Acquire a cleaning point corresponding to the first target image; Performing a circle expansion process based on the cleaning points to obtain a circle expansion process result; determining an extraction range of the surface state data of the second target image based on the circle expansion processing result; Surface state data of the second target image is extracted according to the extraction range.

11. The method for cleaning a printed circuit board assembly according to claim 1, wherein: After determining that foreign matter remains, the method further includes: Determining relevant attribute information of the residual foreign matter, wherein the relevant attribute information includes coverage, material attributes, and area to which it belongs; The relevant attribute information is standardized, and the grade of the residual foreign matter is determined based on the standardized processing result and a grading function, wherein the grading function includes: F=ω1G1+ω2G2+ω3G3; Among them, F represents the classification coefficient, ω1, ω2 and ω3 represent weight coefficients, G1 represents the single point coverage, G2 represents the material attribute coefficient, and G3 represents the region coefficient. In response to the classification coefficient being less than a first preset value, defining the grade of the residual foreign matter as a first-level residual; In response to the classification coefficient being greater than or equal to a first preset value and less than a second preset value, defining the grade of the residual foreign matter as a second-level residual; In response to the classification coefficient being greater than a second preset value, the grade of the residual foreign matter is defined as grade three residual.

12. The method for cleaning a printed circuit board assembly according to claim 11, wherein: In response to the presence of foreign matter residue, marking the position information of the foreign matter residue, and cleaning the remaining foreign matter according to the position information until no foreign matter residue exists on the printed circuit board assembly to be cleaned, comprising: In response to the residual foreign matter being at a level of primary residue or secondary residue, cleaning the residual foreign matter according to the position information until no foreign matter remains on the printed circuit board assembly to be cleaned; In response to the level of the residual foreign matter being level three, a prompt message is sent to the terminal.

13. An electronic device, characterized in that: include: memory for storing computer programs; A processor is configured to implement the steps of the printed circuit board assembly cleaning method according to any one of claims 1 to 12 when executing the computer program.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method for cleaning a printed circuit board assembly according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method for cleaning a printed circuit board assembly according to any one of claims 1 to 12 are implemented.

Images