Site detection and dispensing control method and system based on image information

Abstract

The present invention provides a site detection and dispensing control method and system based on image information, which relates to the field of control technology, including acquiring image information of a workpiece to be dispensed with glue, establishing a dispensing process model based on the initial dispensing parameters according to the characteristics of the material to be dispensed with glue and the area and height information of the area to be dispensed with glue, and obtaining target dispensing parameters by solving the dispensing process model; controlling a dispensing device to dispense glue according to the target dispensing parameters to obtain a dispensing result, acquiring an image of the dispensing area of ​​the workpiece after dispensing and performing quality inspection to obtain a quality inspection result; comparing the quality inspection result with a preset quality standard, and completing the dispensing when the quality requirements are met; when the quality requirements are not met, correcting the parameters in the dispensing process model according to the quality inspection result, re-obtaining the target dispensing parameters and performing dispensing until the quality requirements are met.

Description

Technical Field

[0001] The present invention relates to control technology, and in particular to a method and system for site detection and dispensing control based on image information. Background Art

[0002] Glue dispensing technology is widely used in many fields such as electronic manufacturing, semiconductor packaging, and medical devices. Traditional glue dispensing methods mainly rely on preset coordinate information for positioning and control, but this method often faces many challenges in actual production. First, a slight deviation in the placement of the workpiece may lead to a significant error in the glue dispensing position. Secondly, the manufacturing tolerance of the workpiece itself will also affect the glue dispensing accuracy. In addition, environmental factors such as temperature changes and vibrations will also have an adverse effect on the glue dispensing process.

[0003] In order to improve the accuracy and reliability of dispensing, the industry began to explore dispensing control methods based on vision systems. Early vision-assisted dispensing systems mainly used monocular vision technology to correct the dispensing position by identifying pre-set reference marks. However, this method still has some limitations, such as the inability to effectively handle complex three-dimensional structures and difficulty in adapting to diverse workpiece shapes.

[0004] In recent years, with the rapid development of image processing and machine vision technology, some more advanced visual dispensing methods have been proposed. For example, some researchers have proposed a dispensing position detection method based on stereo vision, which obtains the three-dimensional information of the workpiece through binocular cameras or structured light scanning, thereby achieving more accurate position positioning. Other researchers have explored dispensing target recognition technology based on deep learning, which automatically identifies the dispensing position on complex workpieces by training neural network models.

[0005] Although these methods have improved the accuracy and flexibility of dispensing to a certain extent, there are still some problems that need to be solved:

[0006] Insufficient real-time performance: Many image processing-based methods have high computational complexity and are difficult to meet the needs of high-speed dispensing. Insufficient robustness: The performance of existing methods in dealing with complex environmental factors such as lighting changes and surface reflections still needs to be improved. Lack of adaptive ability: Most methods lack the ability to monitor and dynamically adjust the dispensing process in real time. Low versatility: Many methods are designed for specific types of workpieces and are difficult to quickly adapt to different dispensing application scenarios. Low integration: Existing visual dispensing systems are often external systems independent of dispensing equipment and are not sufficiently integrated with dispensing control.

[0007] Therefore, developing a method for detecting and controlling dispensing sites that can make full use of image information and achieve high precision, high reliability, and high adaptability is of great significance for improving the dispensing process level, product quality, and production efficiency. In view of the above problems, the present invention proposes an innovative method for site detection and dispensing control based on image information, aiming to overcome the shortcomings of the existing technology and provide a more advanced and reliable technical solution for precision dispensing applications. Summary of the invention

[0008] The embodiments of the present invention provide a method and system for site detection and dispensing control based on image information, which can solve the problems in the prior art.

[0009] According to a first aspect of the embodiments of the present invention,

[0010] Provides image-based site detection and dispensing control methods, including:

[0011] Acquire image information of the workpiece to be glued, the image information including contour features and surface features of the workpiece to be glued; determine the position and posture of the workpiece to be glued according to the contour features, and obtain a first point detection result; detect defects on the surface of the workpiece to be glued according to the surface features, and obtain a second point detection result; fuse the first point detection result and the second point detection result to obtain the three-dimensional spatial position coordinates of the glued area of ​​the workpiece to be glued, and determine initial glue dispensing parameters, the initial glue dispensing parameters including glue dispensing trajectory, glue dispensing speed and glue dispensing amount;

[0012] According to the characteristics of the material to be dispensed and the area and height information of the area to be dispensed, a dispensing process model is established on the basis of the initial dispensing parameters, and the target dispensing parameters are obtained by solving the dispensing process model, wherein the target dispensing parameters are dispensing parameters that can make the glue evenly distributed after dispensing and minimize the amount of glue used;

[0013] Control the dispensing equipment to perform dispensing according to the target dispensing parameters to obtain the dispensing results, obtain the dispensing area image of the workpiece after dispensing and perform quality inspection to obtain the quality inspection results; compare the quality inspection results with the preset quality standards, and complete the dispensing when the quality requirements are met; when the quality requirements are not met, correct the parameters in the dispensing process model according to the quality inspection results, re-acquire the target dispensing parameters and perform dispensing until the quality requirements are met.

[0014] In an optional embodiment,

[0015] Acquire image information of the workpiece to be glued, the image information including contour features and surface features of the workpiece to be glued; determine the position and posture of the workpiece to be glued according to the contour features, and acquire the first point detection result including:

[0016] Acquire an image of a workpiece to be glued; preprocess the image of the workpiece to be glued, including applying an adaptive median filter to remove noise and using a contrast-limited adaptive histogram equalization algorithm to enhance the image contrast; construct a multi-scale image pyramid of the preprocessed image, the image pyramid comprising multiple levels of images; perform improved Canny edge detection on each level of the image pyramid, including:

[0017] Calculate the gradient amplitude histogram of each level image; use the Otsu method to determine the initial high and low thresholds; dynamically adjust the high and low thresholds according to the image edge density; perform Canny edge detection; start from the top layer, propagate the detection results downward layer by layer and merge them with the lower layer results, where the strongest edge points in the lower layer area corresponding to the upper layer edge points are retained;

[0018] The fused edge image is morphologically processed, including opening operation to remove noise, closing operation to fill gaps and thinning algorithm to ensure the edge width is single pixel; the Douglas-Peucker algorithm is used to simplify the contour of the processed edge, and the distance threshold is iteratively adjusted to make the number of simplified contour points within the preset range; the least squares ellipse fitting is performed on the simplified contour point set, and the generalized eigenvalue problem is solved to obtain the ellipse parameters; the position coordinates and direction angle of the workpiece are calculated according to the ellipse parameters to obtain the first point detection result.

[0019] In an optional embodiment,

[0020] According to the surface features, the defects on the surface of the workpiece to be glued are detected to obtain the second position detection result; the first position detection result and the second position detection result are integrated to obtain the three-dimensional spatial position coordinates of the glue-dispensing area of ​​the workpiece to be glued, and the initial glue dispensing parameters are determined, including:

[0021] Inputting the preprocessed image into a pre-trained U-Net convolutional neural network to obtain a defect area probability map; binarizing the defect area probability map to obtain a defect mask image;

[0022] Performing connected region analysis on the defect mask image, calculating the centroid coordinates of each defect region, and obtaining a second site detection result;

[0023] Performing feature normalization processing on the first site detection result and the second site detection result respectively; performing nonlinear mapping on the normalized first site detection result and the second site detection result respectively using a multi-layer perceptron to obtain a contour feature vector and a surface feature vector;

[0024] A weight vector is designed to perform weighted fusion of the contour feature vector and the surface feature vector; the weight vector is optimized by using a particle swarm optimization algorithm, wherein the average error of the site detection after fusion is used as the fitness function; the contour feature vector and the surface feature vector are finally fused using the optimized weight vector; the fused feature vector is mapped to a two-dimensional coordinate space through a fully connected layer to obtain the final workpiece site coordinates.

[0025] In an optional embodiment,

[0026] According to the characteristics of the material to be dispensed and the area and height information of the area to be dispensed, a dispensing process model is established on the basis of the initial dispensing parameters, and the target dispensing parameters are obtained by solving the dispensing process model, including:

[0027] Obtain the physical property parameters of the material to be dispensed, including viscosity, surface tension and temperature sensitivity; measure the geometric information of the area to be dispensed, including area and height data; establish a generalized Herschel-Bulkley rheological model, which includes shear stress, yield stress, consistency coefficient, shear rate, flow index and limiting viscosity parameters; use a rheometer to test the material to be dispensed to obtain the specific parameter values ​​of the generalized Herschel-Bulkley rheological model;

[0028] Construct a surface evolution model based on the Level Set method, which describes the spreading behavior of the colloid on different surface shapes during the dispensing process; establish a thermal expansion model of the relationship between temperature and volume change, including initial volume, volume thermal expansion coefficient, initial temperature and current temperature parameters;

[0029] Based on the generalized Herschel-Bulkley rheological model, surface evolution model and thermal expansion model, a comprehensive dispensing process optimization objective function is constructed, which includes three sub-objectives: colloid coverage deviation, colloid thickness uniformity and energy consumption; the weight coefficient of the optimization objective function is set; and the dispensing parameters are initialized, including nozzle pressure, nozzle movement speed and nozzle height;

[0030] The improved particle swarm optimization algorithm is used to solve the optimization objective function. The algorithm introduces adaptive inertia weight and chaotic perturbation mechanism. The parameters of the improved particle swarm optimization algorithm are set, including population size, maximum number of iterations, acceleration constant and chaotic perturbation intensity. The improved particle swarm optimization algorithm is executed to iteratively update the speed and position of the particles. In each iteration, the optimization objective function value under the dispensing parameters corresponding to the current particle is calculated. The individual optimal solution and the global optimal solution are updated.

[0031] Determine whether the convergence condition is met or the maximum number of iterations is reached. If so, output the dispensing parameters corresponding to the global optimal solution as the final target dispensing parameters.

[0032] In an optional embodiment,

[0033] The improved particle swarm optimization algorithm is used to solve the optimization objective function including:

[0034] Construct the dispensing process optimization objective function, which includes three sub-goals: colloid coverage deviation, colloid thickness uniformity and energy consumption, and set the weight coefficient of each sub-goal; determine the optimization variables as the dispensing parameter vector, including nozzle pressure, nozzle moving speed and nozzle height;

[0035] Set the parameters of the improved particle swarm optimization algorithm, including population size, maximum number of iterations, acceleration constant, chaotic perturbation intensity, and upper and lower limits of inertia weight;

[0036] An initial population is randomly generated, each particle represents a set of candidate dispensing parameter solutions, and a position vector, a velocity vector and an individual optimal position are assigned to each particle;

[0037] Calculate the fitness value of each particle and determine the global optimal position;

[0038] Enter the iterative optimization process, for each iteration:

[0039] Calculate the adaptive inertia weight according to the current number of iterations; update the speed and position of each particle, where the position update introduces chaotic perturbations; perform boundary processing on the updated particle position to ensure that it is within the search space; recalculate the fitness value of each particle; update the individual optimal position of each particle; and update the global optimal position;

[0040] Check whether the convergence condition is met or the maximum number of iterations is reached. If so, end the iteration; otherwise, return and output the global optimal position as the final optimized dispensing parameter.

[0041] In an optional embodiment,

[0042] Controlling the dispensing equipment to perform dispensing according to the target dispensing parameters, and obtaining dispensing results includes:

[0043] Receiving optimized target dispensing parameters, including nozzle pressure, nozzle moving speed and nozzle height; converting the target dispensing parameters into specific control instructions through a parameter parsing module; and generating a smooth dispensing path that considers acceleration constraints and curvature continuity using a trajectory planning module according to geometric information of the area to be dispensed and the control instructions;

[0044] A dynamic model of the dispensing process is constructed, which takes into account the effects of nozzle pressure, moving speed and height on colloid flow and spreading behavior; the objective function of the model predictive controller is set, which includes four weighted items: colloid flow deviation, nozzle moving speed deviation, nozzle height deviation and control quantity change rate;

[0045] The pressure sensor, position encoder and vision sensor are integrated in the dispensing system to monitor the pressure change in the nozzle, provide the nozzle position and speed information, and observe the colloid spreading state. The multi-sensor fusion algorithm based on Kalman filtering is used to comprehensively process the data of multiple sensors to obtain accurate and stable system state estimation. The recursive least squares method is used to identify the dispensing process model parameters in real time, and the internal model of the model predictive controller is dynamically adjusted according to the identification results.

[0046] During the dispensing process, the model predictive controller is run at a high frequency to continuously predict the system state in the future and solve the optimal control sequence. In each control cycle, only the first control action of the optimal control sequence is executed, and the optimization and solution process is repeated in the next cycle.

[0047] The machine vision system is used to detect the dispensing results in real time, including the colloid coverage area, thickness distribution and edge contour; the detection results are fed back to the parameter optimization module for further adjustment and optimization of the dispensing parameters; according to the feedback detection results, the objective function weight of the model predictive controller is dynamically adjusted to balance different performance indicators.

[0048] According to a second aspect of the embodiments of the present invention,

[0049] Provides image-based site detection and dispensing control system, including:

[0050] The first unit is used to obtain image information of the workpiece to be glued, wherein the image information includes contour features and surface features of the workpiece to be glued; according to the contour features, the position and posture of the workpiece to be glued are determined to obtain a first point detection result; according to the surface features, defects on the surface of the workpiece to be glued are detected to obtain a second point detection result; the first point detection result and the second point detection result are integrated to obtain the three-dimensional spatial position coordinates of the glued area of ​​the workpiece to be glued, and determine the initial glue dispensing parameters, wherein the initial glue dispensing parameters include a glue dispensing trajectory, a glue dispensing speed, and a glue dispensing amount;

[0051] The second unit is used to establish a dispensing process model based on the initial dispensing parameters according to the characteristics of the material to be dispensed and the area and height information of the dispensing area, and obtain target dispensing parameters by solving the dispensing process model. The target dispensing parameters are dispensing parameters that can make the glue evenly distributed after dispensing and minimize the amount of glue used;

[0052] The third unit is used to control the dispensing equipment to perform dispensing according to the target dispensing parameters, obtain the dispensing results, obtain the dispensing area image of the workpiece after dispensing and perform quality inspection to obtain the quality inspection results; compare the quality inspection results with the preset quality standards, and complete the dispensing when the quality requirements are met; when the quality requirements are not met, correct the parameters in the dispensing process model according to the quality inspection results, re-acquire the target dispensing parameters and perform dispensing until the quality requirements are met.

[0053] According to a third aspect of the embodiments of the present invention,

[0054] An electronic device is provided, comprising:

[0055] processor;

[0056] a memory for storing processor-executable instructions;

[0057] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0058] According to a fourth aspect of the embodiments of the present invention,

[0059] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0060] The advantage of the fusion method of this application is that it can consider the overall shape and local details of the workpiece at the same time, has strong adaptability, and can handle various complex workpiece shapes and surface features. Through particle swarm optimization, the fusion weight is dynamically adjusted. The method has good adaptive ability and can automatically adjust the strategy according to the characteristics of different workpieces to improve the accuracy and stability of the dispensing position. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the process of the site detection and dispensing control method based on image information according to an embodiment of the present invention;

[0062] Figure 2 It is a structural schematic diagram of a site detection and dispensing control system based on image information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0064] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0065] Figure 1 FIG. 1 is a flow chart of a method for detecting and controlling a site based on image information according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0066] S101. Obtain image information of the workpiece to be glued, the image information including contour features and surface features of the workpiece to be glued; determine the position and posture of the workpiece to be glued according to the contour features, and obtain a first point detection result; detect defects on the surface of the workpiece to be glued according to the surface features, and obtain a second point detection result; fuse the first point detection result and the second point detection result to obtain the three-dimensional spatial position coordinates of the glued area of ​​the workpiece to be glued, and determine initial glue dispensing parameters, the initial glue dispensing parameters including glue dispensing trajectory, glue dispensing speed and glue dispensing amount;

[0067] S102. According to the characteristics of the material to be glued and the area and height information of the area to be glued, a glue dispensing process model is established on the basis of the initial glue dispensing parameters, and the target glue dispensing parameters are obtained by solving the glue dispensing process model, wherein the target glue dispensing parameters are glue dispensing parameters that can make the glue evenly distributed after glue dispensing and minimize the glue consumption;

[0068] S103. Control the dispensing equipment to perform dispensing according to the target dispensing parameters to obtain the dispensing results, obtain the dispensing area image of the workpiece after dispensing and perform quality inspection to obtain the quality inspection results; compare the quality inspection results with the preset quality standards, and complete the dispensing when the quality requirements are met; when the quality requirements are not met, correct the parameters in the dispensing process model according to the quality inspection results, re-acquire the target dispensing parameters and perform dispensing until the quality requirements are met.

[0069] In an optional embodiment,

[0070] Acquire image information of the workpiece to be glued, the image information including contour features and surface features of the workpiece to be glued; determine the position and posture of the workpiece to be glued according to the contour features, and acquire the first point detection result including:

[0071] Acquire an image of a workpiece to be glued; preprocess the image of the workpiece to be glued, including applying an adaptive median filter to remove noise and using a contrast-limited adaptive histogram equalization algorithm to enhance the image contrast; construct a multi-scale image pyramid of the preprocessed image, the image pyramid comprising multiple levels of images; perform improved Canny edge detection on each level of the image pyramid, including:

[0072] Calculate the gradient amplitude histogram of each level image; use the Otsu method to determine the initial high and low thresholds; dynamically adjust the high and low thresholds according to the image edge density; perform Canny edge detection; start from the top layer, propagate the detection results downward layer by layer and merge them with the lower layer results, where the strongest edge points in the lower layer area corresponding to the upper layer edge points are retained;

[0073] The fused edge image is morphologically processed, including opening operation to remove noise, closing operation to fill gaps and thinning algorithm to ensure the edge width is single pixel; the Douglas-Peucker algorithm is used to simplify the contour of the processed edge, and the distance threshold is iteratively adjusted to make the number of simplified contour points within the preset range; the least squares ellipse fitting is performed on the simplified contour point set, and the generalized eigenvalue problem is solved to obtain the ellipse parameters; the position coordinates and direction angle of the workpiece are calculated according to the ellipse parameters to obtain the first point detection result.

[0074] For example, the image information-based site detection method first obtains an image of the workpiece to be dispensed. This step usually uses a high-resolution industrial camera with an appropriate lighting system to capture a clear image of the workpiece. To ensure image quality, the camera resolution is at least 2048x1536 pixels and the frame rate is not less than 60fps to meet the needs of real-time processing.

[0075] After acquiring the image, image preprocessing is performed to improve the effect of subsequent processing. First, adaptive median filtering is applied to remove noise. This filtering method can automatically adjust the filter window size according to the local noise level. Usually the window size changes dynamically between 3x3 and 7x7. For areas that are severely contaminated by noise, the window may be expanded to 11x11. Adaptive median filtering can not only effectively remove salt and pepper noise, but also well preserve image details.

[0076] Next, the contrast of the image is enhanced using the contrast-limited adaptive histogram equalization algorithm (CLAHE). The CLAHE algorithm divides the image into multiple small blocks, with a typical block size of 8x8 or 16x16, and performs histogram equalization on each small block. To avoid excessive enhancement of noise, a contrast limit threshold is set, usually 1% to 5% of the grayscale range of the original image. Finally, the results of adjacent blocks are merged using the bilinear interpolation method to eliminate artificial boundaries.

[0077] After preprocessing, a multi-scale image pyramid is constructed. The pyramid usually contains 3 to 5 levels, and the image size of each level is half of the previous level. For example, for an original image of 2048x1536, the pyramid may contain five levels: 2048x1536, 1024x768, 512x384, 256x192, and 128x96. The image of each level is obtained by Gaussian filtering and downsampling, and the standard deviation of the Gaussian kernel is usually set between 0.5 and 1.0.

[0078] Perform improved Canny edge detection on each level of the pyramid. First, calculate the gradient magnitude histogram, using 256 bins to represent the gradient magnitude distribution. Then use the Otsu method to determine the initial high and low thresholds, which automatically finds the best threshold to divide the histogram into two categories. The initial low threshold is usually set to 40% to 60% of the Otsu threshold, and the high threshold is 1.5 to 2 times the Otsu threshold.

[0079] Dynamically adjust the high and low thresholds based on the image edge density. Edge density is defined as the ratio of the number of edge pixels to the total number of pixels. If the edge density is too high (for example, more than 10%), increase the threshold appropriately; if it is too low (for example, less than 1%), decrease the threshold. The adjustment step size is usually 5% to 10% of the initial threshold.

[0080] After performing Canny edge detection, starting from the top layer of the pyramid, the detection results are propagated downward layer by layer and fused with the results of the lower layers. Specifically, for each edge point in the upper layer, the strongest edge point is searched in the 3x3 or 5x5 neighborhood of the corresponding position in the lower layer and retained as the fusion result. This top-down strategy can effectively suppress noise and retain significant edge features.

[0081] Morphological processing is performed on the fused edge image. First, a 3x3 rectangular structure element is used to perform an opening operation to remove small noise points. Then a 5x5 rectangular structure element is used to perform a closing operation to fill small gaps on the edge. Finally, a thinning algorithm is applied to ensure that the edge width is a single pixel, which is crucial for subsequent contour extraction.

[0082] The processed edges are simplified using the Douglas-Peucker algorithm. The initial distance threshold is set to 0.1% of the diagonal length of the original image. If the number of points in the simplified contour exceeds the preset upper limit (e.g. 1000 points), the threshold is increased; if the number of points is less than the preset lower limit (e.g. 100 points), the threshold is decreased. The threshold is adjusted by 20% of the current value each time. Through iterative adjustment, a point set that can accurately represent the contour of the workpiece without being too complex is finally obtained.

[0083] The least squares ellipse fitting is performed on the simplified contour point set. This process involves solving a generalized eigenvalue problem to obtain the parameters describing the ellipse, including the coordinates of the ellipse center, the lengths of the major and minor axes, and the rotation angle. The fitting process is usually solved using the singular value decomposition (SVD) method to improve numerical stability.

[0084] Finally, the position coordinates and direction angle of the workpiece are calculated based on the ellipse parameters. The center of the ellipse is the center position of the workpiece, usually expressed in pixel coordinates, such as (1024, 768). The direction angle is determined by the angle between the major axis of the ellipse and the x-axis of the image, and the value range is 0 to 360 degrees. This information constitutes the first point detection result, which provides an accurate position and posture reference for subsequent dispensing operations.

[0085] Through the above detailed image processing and analysis steps, the position and posture of the workpiece to be dispensed can be accurately detected, laying the foundation for precise dispensing. This method has strong robustness and can adapt to workpieces of different shapes and materials, as well as various complex industrial environments.

[0086] In an optional embodiment,

[0087] According to the surface features, the defects on the surface of the workpiece to be glued are detected to obtain the second position detection result; the first position detection result and the second position detection result are integrated to obtain the three-dimensional spatial position coordinates of the glue-dispensing area of ​​the workpiece to be glued, and the initial glue dispensing parameters are determined, including:

[0088] Inputting the preprocessed image into a pre-trained U-Net convolutional neural network to obtain a defect area probability map; binarizing the defect area probability map to obtain a defect mask image;

[0089] Performing connected region analysis on the defect mask image, calculating the centroid coordinates of each defect region, and obtaining a second site detection result;

[0090] Performing feature normalization processing on the first site detection result and the second site detection result respectively; performing nonlinear mapping on the normalized first site detection result and the second site detection result respectively using a multi-layer perceptron to obtain a contour feature vector and a surface feature vector;

[0091] A weight vector is designed to perform weighted fusion of the contour feature vector and the surface feature vector; the weight vector is optimized by using a particle swarm optimization algorithm, wherein the average error of the site detection after fusion is used as the fitness function; the contour feature vector and the surface feature vector are finally fused using the optimized weight vector; the fused feature vector is mapped to a two-dimensional coordinate space through a fully connected layer to obtain the final workpiece site coordinates.

[0092] For example, defect detection based on surface features is a key step in achieving accurate dispensing. This process first uses a pre-trained U-Net convolutional neural network to analyze the preprocessed image. The U-Net network structure consists of two parts: an encoder and a decoder. The encoder consists of 4 downsampling blocks, each of which contains two 3x3 convolutional layers and a 2x2 maximum pooling layer. The decoder also contains 4 upsampling blocks, each of which contains a 2x2 deconvolutional layer and two 3x3 convolutional layers. The last layer of the network is a 1x1 convolutional layer that outputs a feature map of the same size as the input image.

[0093] The training set of the U-Net network contains a large number of workpiece images with defect annotations, usually requiring thousands of samples. The cross entropy loss function is used in the training process, the Adam optimizer is adopted, the learning rate is initially set to 0.001, and the learning rate decay strategy is used. The training is usually carried out for 50 to 100 epochs to ensure that the network is fully converged.

[0094] The preprocessed image is input into the trained U-Net network to obtain the defect area probability map. The value of each pixel in the probability map represents the probability that the location is a defect, ranging from 0 to 1. In order to obtain a clear defect area, the probability map needs to be binarized. The binarization threshold is usually set to 0.5, that is, pixels with a probability greater than 0.5 are marked as defects, and pixels with a probability less than 0.5 are marked as normal areas. The binary image obtained in this way is the defect mask image.

[0095] Connected region analysis is performed on the defect mask image to identify and locate each independent defect area. Connected region analysis uses the 8-neighborhood connectivity criterion, that is, considering the horizontal, vertical and diagonal adjacent positions of the pixels. In order to reduce the influence of noise, an area threshold can be set, such as 10 pixels, and connected regions smaller than this threshold can be ignored.

[0096] The centroid coordinates are calculated for each valid defect area. The centroid coordinates are calculated by taking into account the positions of all pixels in the defect area and averaging the x- and y-coordinates. For example, for a defect area containing 100 pixels, the centroid coordinates may be (512.5, 384.7). These centroid coordinates constitute the second site detection results, reflecting the distribution of surface defects on the workpiece.

[0097] In order to fuse the first point detection result (based on contour features) and the second point detection result (based on surface features), the two sets of results need to be normalized. The purpose of normalization is to map features of different scales to the same range. Usually, the minimum-maximum normalization method is used to map the feature values ​​to between 0 and 1.

[0098] The normalized features are nonlinearly mapped using a multilayer perceptron to extract higher-level feature representations. The structure of a multilayer perceptron usually contains 2 to 3 hidden layers, and the number of neurons in each layer can be reduced layer by layer, for example, from 128 to 64 and then to 32. ReLU is selected as the activation function to increase the nonlinear expression ability of the network. The output of this step is the contour feature vector and the surface feature vector, which capture the shape information and surface details of the workpiece respectively.

[0099] The weight vector is designed to fuse the contour feature vector and the surface feature vector. The initial weights can be set equal, for example, 0.5 each. In order to find the optimal fusion weight, the particle swarm optimization algorithm is used for optimization. Each particle in the particle swarm represents a set of possible weight configurations. The size of the particle swarm is usually set to 20 to 50 particles, and the number of iterations is 100 to 200.

[0100] The fitness function of particle swarm optimization is defined as the average error of site detection after fusion. Specifically, the features are fused using the current weights, mapped to a two-dimensional coordinate space through a fully connected layer, and then the Euclidean distance between the predicted site and the true site is calculated. During the optimization process, the velocity update parameters of the particles are usually set to: inertia weight w = 0.7, individual learning factor c1 = 1.5, and social learning factor c2 = 1.5.

[0101] After optimization, the optimal weight vector is obtained. This weight vector is used to finally fuse the contour feature vector and the surface feature vector. The fused feature vector is mapped to a two-dimensional coordinate space through a fully connected layer. The input dimension of the fully connected layer is equal to the dimension of the fused feature vector, and the output dimension is 2, corresponding to the x and y coordinates.

[0102] The final output of the workpiece location coordinates takes into account the contour features and surface features of the workpiece, providing more accurate and reliable dispensing location information. For example, for a 2048x1536 pixel image, the final dispensing location coordinates may be (1024.3, 768.7).

[0103] The advantage of this fusion method is that it can consider the overall shape and local details of the workpiece at the same time, and it has strong adaptability and can handle various complex workpiece shapes and surface features. The fusion weight is dynamically adjusted through particle swarm optimization. The method has good adaptive ability and can automatically adjust the strategy according to the characteristics of different workpieces to improve the accuracy and stability of the dispensing position.

[0104] In an optional embodiment,

[0105] According to the characteristics of the material to be dispensed and the area and height information of the area to be dispensed, a dispensing process model is established on the basis of the initial dispensing parameters, and the target dispensing parameters are obtained by solving the dispensing process model, including:

[0106] Obtain the physical property parameters of the material to be dispensed, including viscosity, surface tension and temperature sensitivity; measure the geometric information of the area to be dispensed, including area and height data; establish a generalized Herschel-Bulkley rheological model, which includes shear stress, yield stress, consistency coefficient, shear rate, flow index and limiting viscosity parameters; use a rheometer to test the material to be dispensed to obtain the specific parameter values ​​of the generalized Herschel-Bulkley rheological model;

[0107] Construct a surface evolution model based on the Level Set method, which describes the spreading behavior of the colloid on different surface shapes during the dispensing process; establish a thermal expansion model of the relationship between temperature and volume change, including initial volume, volume thermal expansion coefficient, initial temperature and current temperature parameters;

[0108] Based on the generalized Herschel-Bulkley rheological model, surface evolution model and thermal expansion model, a comprehensive dispensing process optimization objective function is constructed, which includes three sub-objectives: colloid coverage deviation, colloid thickness uniformity and energy consumption; the weight coefficient of the optimization objective function is set; and the dispensing parameters are initialized, including nozzle pressure, nozzle movement speed and nozzle height;

[0109] The improved particle swarm optimization algorithm is used to solve the optimization objective function. The algorithm introduces adaptive inertia weight and chaotic perturbation mechanism. The parameters of the improved particle swarm optimization algorithm are set, including population size, maximum number of iterations, acceleration constant and chaotic perturbation intensity. The improved particle swarm optimization algorithm is executed to iteratively update the speed and position of the particles. In each iteration, the optimization objective function value under the dispensing parameters corresponding to the current particle is calculated. The individual optimal solution and the global optimal solution are updated.

[0110] Determine whether the convergence conditions are met or the maximum number of iterations is reached. If so, output the dispensing parameters corresponding to the global optimal solution as the final target dispensing parameters.

[0111] For example, the establishment and solution of the dispensing process model is the core link to achieve precise dispensing. First, the physical property parameters of the material to be dispensed need to be obtained. These parameters include viscosity, surface tension and temperature sensitivity. Taking epoxy resin as an example, its typical viscosity range is 10,000 to 20,000 centipoise, the surface tension is about 40 to 50 mN / m, and the temperature sensitivity is that the viscosity decreases with increasing temperature, and the viscosity may decrease by 20% to 30% for every 10°C increase.

[0112] The geometric information measurement of the area to be dispensed is equally important. Through a high-precision laser scanner or a structured light 3D scanning system, the precise area and height data of the area to be dispensed can be obtained. For example, the dispensing area of ​​an electronic component is 100 square millimeters, and the height range is 0.5 to 1.5 millimeters.

[0113] Establishing a generalized Herschel-Bulkley rheological model is the key to describing the rheological properties of dispensing materials. The model includes parameters such as shear stress, yield stress, consistency coefficient, shear rate, flow index and limiting viscosity. The specific values ​​of these parameters can be obtained by testing with a high-precision rheometer. During the test, the shear rate usually varies in the range of 0.1 to 10001 / s, and the temperature is controlled at 25°C. For typical epoxy resins, the yield stress may be in the range of 10 to 50 Pa, the flow index is between 0.7 and 0.9, and the limiting viscosity is about 1 / 10 of the initial viscosity.

[0114] A surface evolution model is constructed based on the Level Set method to describe the spreading behavior of colloids on different surface shapes during the dispensing process. The Level Set method represents the interface through implicit functions and can naturally handle topological changes. In implementation, the finite difference method is usually used to discretize the Level Set equation, with the time step set to 0.01 seconds and the spatial grid size to 0.1 mm.

[0115] The thermal expansion model of the relationship between temperature and volume change includes parameters such as initial volume, volume thermal expansion coefficient, initial temperature and current temperature. For epoxy resin, the volume thermal expansion coefficient is usually in the range of 50 to 100 ppm / °C. The initial temperature is set to room temperature 25°C, and the current temperature may vary according to the actual dispensing environment, for example, between 20°C and 30°C.

[0116] The construction of the comprehensive dispensing process optimization objective function takes into account three sub-objectives: colloid coverage deviation, colloid thickness uniformity, and energy consumption. The colloid coverage deviation objective aims to minimize the difference between the actual coverage area and the target area, and the allowable deviation is usually no more than 5%. The thickness uniformity objective requires that the thickness variation of the colloid in the dispensing area be controlled within ±10%. The energy consumption objective considers the impact of pressure, speed, and height parameters on energy consumption during the dispensing process, and usually hopes to minimize energy consumption while meeting quality requirements.

[0117] The weight coefficients of these three sub-goals need to be adjusted according to the specific application scenario. A common configuration is a coverage deviation weight of 0.4, a thickness uniformity weight of 0.4, and an energy consumption weight of 0.2. When initializing the dispensing parameters, the nozzle pressure can be set to 0.3 MPa, the nozzle movement speed to 10 mm / s, and the nozzle height to 1 mm.

[0118] The improved particle swarm optimization algorithm is used to solve the optimization objective function. The algorithm introduces adaptive inertia weight and chaotic perturbation mechanism to enhance the global search ability and the ability to jump out of the local optimum. The algorithm parameters are set as follows: the population size is 50, the maximum number of iterations is 1000, the acceleration constants c1 and c2 are both 2.0, and the chaotic perturbation intensity is 0.001.

[0119] In each iteration, the particle speed and position are updated. The speed update takes into account the influence of the current speed, individual optimal position and global optimal position. The position update is based on the current position and the updated speed. For the dispensing parameters, set a reasonable value range: nozzle pressure 0.1 to 0.5 MPa, nozzle movement speed 5 to 20 mm / s, nozzle height 0.5 to 2 mm.

[0120] After each iteration, the optimization objective function value under the dispensing parameters corresponding to the current particle is calculated. This involves using the generalized Herschel-Bulkley model to calculate fluid behavior, using the Level Set method to simulate colloid spreading, and considering thermal expansion effects. The individual optimal solution and the global optimal solution are updated based on the calculation results.

[0121] The sensitivity condition is usually set to the global optimal solution improvement of less than 0.1% for 50 consecutive iterations, or to the maximum number of iterations of 1000. When any of the conditions is met, the algorithm stops and outputs the dispensing parameters corresponding to the global optimal solution as the final target dispensing parameters.

[0122] The target dispensing parameters obtained by this method, such as nozzle pressure of 0.35 MPa, nozzle moving speed of 12 mm / s, and nozzle height of 0.8 mm, can achieve high-precision and high-uniformity dispensing effects based on material properties, workpiece geometry, and environmental factors. This method based on multi-physics field coupling model and intelligent optimization algorithm has strong versatility and adaptability, and can cope with various complex dispensing scenarios.

[0123] In an optional embodiment,

[0124] The improved particle swarm optimization algorithm is used to solve the optimization objective function including:

[0125] Construct the dispensing process optimization objective function, which includes three sub-goals: colloid coverage deviation, colloid thickness uniformity and energy consumption, and set the weight coefficient of each sub-goal; determine the optimization variables as the dispensing parameter vector, including nozzle pressure, nozzle moving speed and nozzle height;

[0126] Set the parameters of the improved particle swarm optimization algorithm, including population size, maximum number of iterations, acceleration constant, chaotic perturbation intensity, and upper and lower limits of inertia weight;

[0127] An initial population is randomly generated, each particle represents a set of candidate dispensing parameter solutions, and a position vector, a velocity vector and an individual optimal position are assigned to each particle;

[0128] Calculate the fitness value of each particle and determine the global optimal position;

[0129] Enter the iterative optimization process, for each iteration:

[0130] Calculate the adaptive inertia weight according to the current number of iterations; update the speed and position of each particle, where the position update introduces chaotic perturbations; perform boundary processing on the updated particle position to ensure that it is within the search space; recalculate the fitness value of each particle; update the individual optimal position of each particle; and update the global optimal position;

[0131] Check whether the convergence condition is met or the maximum number of iterations is reached. If so, end the iteration; otherwise, return and output the global optimal position as the final optimized dispensing parameter.

[0132] For example, the improved particle swarm optimization algorithm plays a key role in solving the dispensing process optimization objective function. First, the dispensing process optimization objective function is constructed, which comprehensively considers the three sub-objectives of colloid coverage deviation, colloid thickness uniformity and energy consumption. The colloid coverage deviation reflects the difference between the actual dispensing area and the target area, and it is usually hoped that this difference will be controlled within 5%. The colloid thickness uniformity measures the consistency of the colloid thickness in the dispensing area. Ideally, the thickness variation should be controlled within the range of ±10%. Energy consumption takes into account the influence of pressure, speed and height parameters on the total energy consumption during the dispensing process.

[0133] The weight coefficients of these three sub-goals need to be adjusted according to the specific application scenario. A common configuration is: the weight of colloid coverage deviation is 0.4, the weight of colloid thickness uniformity is 0.4, and the weight of energy consumption is 0.2. This configuration attaches equal importance to coverage and uniformity, while also taking energy consumption into consideration.

[0134] The optimization variables are defined as dispensing parameter vectors, including nozzle pressure, nozzle movement speed, and nozzle height. The value ranges of these parameters are usually: nozzle pressure 0.1 to 0.5 MPa, nozzle movement speed 5 to 20 mm / s, and nozzle height 0.5 to 2 mm. These ranges are set based on actual engineering experience and equipment limitations.

[0135] Improving the parameter settings of the particle swarm optimization algorithm has an important impact on the performance of the algorithm. The population size is usually set between 50 and 100, and 80 is taken in this example. The maximum number of iterations is set to 1000, and this value needs to be weighed according to the complexity of the problem and the computing resources. The acceleration constants c1 and c2 are both set to 2.0, which is an empirical value commonly used in particle swarm optimization. The intensity of chaotic perturbation is set to 0.001 to increase the randomness of the algorithm and its ability to jump out of the local optimum. The upper limit of the inertia weight is set to 0.9 and the lower limit is set to 0.4, which allows the algorithm to have a strong global search capability in the early stage, and pay more attention to local fine search in the later stage.

[0136] The initial population is generated using a uniform random distribution. For each particle, its position vector is uniformly randomly generated in each dimension, for example, the nozzle pressure may be initialized to 0.23 MPa, the nozzle movement speed is 15 mm / s, and the nozzle height is 1.2 mm. The velocity vector is initialized to a random value between -10% and 10% of the position vector range. The individual optimal position of each particle is initialized to its initial position.

[0137] Calculating the fitness value of each particle involves using the dispensing process model constructed earlier. For example, for a dispensing parameter combination represented by a certain particle, the possible coverage deviation is 3%, the thickness uniformity deviation is 8%, and the energy consumption is 100 J. Substituting these values ​​into the optimization objective function, the fitness value of the particle is obtained. The position of the particle with the best fitness in the initial population is set as the global optimal position.

[0138] During the iterative optimization process, the adaptive inertia weight is first calculated based on the current iteration number. For example, at the 500th iteration, the inertia weight may be 0.65. This adaptive mechanism enables the algorithm to have a strong global search capability in the early stage, and to focus more on local refined search in the later stage.

[0139] When updating particle velocity, the influence of current velocity, individual optimal position and global optimal position is considered. For example, the velocity of a particle in the nozzle pressure dimension may be updated from 0.01 MPa / iteration to 0.015 MPa / iteration. The position update is based on the current position and the updated velocity, while introducing chaotic perturbations. The chaotic perturbations here are generated using Logistic mapping, which increases the randomness of the search.

[0140] The updated particle position needs to be processed to ensure that it is still within the valid search space. For example, if the updated nozzle pressure exceeds 0.5 MPa, it is truncated to 0.5 MPa. This processing ensures the validity of all candidate solutions.

[0141] Recalculate the fitness value of the updated particle and compare it with the historical optimal position of the particle. If the new position is better, update the individual optimal position. At the same time, the optimal position of all particles is used to update the global optimal position.

[0142] The convergence condition check is usually based on two criteria: the improvement of the global optimal solution is less than 0.1% for 50 consecutive iterations, or the maximum number of iterations is 1000. When either condition is met, the algorithm stops iterating.

[0143] The global optimal position outputted at the end is the optimized dispensing parameters. For example, after optimization, the following parameters may be obtained: nozzle pressure 0.35 MPa, nozzle moving speed 12 mm / s, and nozzle height 0.8 mm. This set of parameters can achieve the best dispensing effect under the given workpiece geometry and material properties.

[0144] This improved particle swarm optimization algorithm enhances the ability to solve complex nonlinear problems through adaptive inertia weights and chaotic perturbation mechanisms. It can effectively balance global exploration and local development, while ensuring the convergence speed and increasing the probability of finding the global optimal solution. This is particularly effective for complex optimization problems such as dispensing processes involving the coupling of multiple physical fields.

[0145] In an optional embodiment,

[0146] Controlling the dispensing equipment to perform dispensing according to the target dispensing parameters, and obtaining dispensing results includes:

[0147] Receiving optimized target dispensing parameters, including nozzle pressure, nozzle moving speed and nozzle height; converting the target dispensing parameters into specific control instructions through a parameter parsing module; and generating a smooth dispensing path that considers acceleration constraints and curvature continuity using a trajectory planning module according to geometric information of the area to be dispensed and the control instructions;

[0148] A dynamic model of the dispensing process is constructed, which takes into account the effects of nozzle pressure, moving speed and height on colloid flow and spreading behavior; the objective function of the model predictive controller is set, which includes four weighted items: colloid flow deviation, nozzle moving speed deviation, nozzle height deviation and control quantity change rate;

[0149] The pressure sensor, position encoder and vision sensor are integrated in the dispensing system to monitor the pressure change in the nozzle, provide the nozzle position and speed information, and observe the colloid spreading state. The multi-sensor fusion algorithm based on Kalman filtering is used to comprehensively process the data of multiple sensors to obtain accurate and stable system state estimation. The recursive least squares method is used to identify the dispensing process model parameters in real time, and the internal model of the model predictive controller is dynamically adjusted according to the identification results.

[0150] During the dispensing process, the model predictive controller is run at a high frequency to continuously predict the system state in the future and solve the optimal control sequence. In each control cycle, only the first control action of the optimal control sequence is executed, and the optimization and solution process is repeated in the next cycle.

[0151] The machine vision system is used to detect the dispensing results in real time, including the colloid coverage area, thickness distribution and edge contour; the detection results are fed back to the parameter optimization module for further adjustment and optimization of the dispensing parameters; according to the feedback detection results, the objective function weight of the model predictive controller is dynamically adjusted to balance different performance indicators.

[0152] For example, controlling the dispensing equipment to perform dispensing according to the target dispensing parameters is a key link in achieving high-precision dispensing. First, receive the optimized target dispensing parameters, including nozzle pressure, nozzle movement speed, and nozzle height. For example, the optimized parameters may be: nozzle pressure 0.35 MPa, nozzle movement speed 12 mm / s, nozzle height 0.8 mm. These parameters are converted into specific control instructions through the parameter parsing module. The parameter parsing module maps the abstract target parameters to an instruction set that the device can understand, such as converting a pressure of 0.35 MPa into the corresponding pressure control valve opening value.

[0153] According to the geometric information of the area to be dispensed and the control instructions, the trajectory planning module generates a smooth dispensing path. The fifth-order polynomial interpolation method is used here to ensure that the path remains continuous in position, velocity and acceleration. For example, for a rectangular dispensing area, a series of S-shaped curve paths may be generated, and the starting and ending points of each curve meet the constraint of zero acceleration. The guarantee of curvature continuity can be achieved by using Bezier curves at key points. This smooth path can reduce mutations in the dispensing process and improve the dispensing quality.

[0154] The construction of the dynamic model of the dispensing process takes into account the complex effects of nozzle pressure, movement speed and height on the colloid flow and spreading behavior. The model can be expressed as a nonlinear state space equation, where the state variables include colloid flow, nozzle position and speed, and the control input is the rate of change of pressure, speed and height. The model includes relevant principles of fluid mechanics and material science, such as the generalized Newtonian fluid model considering the shear thinning effect.

[0155] The objective function setting of the model predictive controller contains four weighted terms: colloid flow deviation, nozzle movement speed deviation, nozzle height deviation and control amount change rate. The initial values ​​of these weights can be set to: flow deviation 0.4, speed deviation 0.3, height deviation 0.2, control amount change rate 0.1. This configuration emphasizes the importance of colloid flow control while also considering the accuracy of motion control and the stability of the system.

[0156] A variety of sensors are integrated into the dispensing system. The pressure sensor is installed inside the nozzle, with a sampling frequency of up to 1000Hz and an accuracy of ±0.1%. The position encoder is integrated on each axis motor with a resolution of 0.1 μm. The visual sensor uses a high-speed camera with a frame rate of up to 500 fps and a resolution of 1280x1024 pixels. These sensors provide rich real-time data, laying the foundation for precise control.

[0157] The multi-sensor fusion algorithm is based on Kalman filter theory. The algorithm first establishes the system state equation and observation equation, where the system noise and observation noise are assumed to be Gaussian white noise. The state prediction and update steps are performed alternately to continuously optimize the state estimation. For example, fusing the data of the pressure sensor and the position encoder can obtain a more accurate colloid flow estimation, and the error can be controlled within ±2%.

[0158] Recursive least squares is used to identify dispensing process model parameters in real time. This method updates the model parameters by minimizing the sum of squares of the prediction errors. In order to improve the tracking ability of the algorithm, a forgetting factor is introduced with a typical value of 0.98. The results of parameter identification are used to dynamically adjust the internal model of the model predictive controller so that the controller can adapt to the time-varying characteristics of the system.

[0159] The model predictive controller runs at a high frequency, with a typical control cycle of 10 ms. In each cycle, the controller predicts the system behavior for the next 100 time steps based on the current state and solves the optimal control sequence. The solution process uses a quadratic programming algorithm, taking into account the constraints of the control quantity and state quantity. For example, the rate of change of the nozzle pressure is limited to ±0.1 MPa / s to avoid drastic pressure fluctuations.

[0160] The machine vision system detects the dispensing results in real time. The system includes a high-resolution camera and a structured light projection device, which can reconstruct the 3D contour of the colloid. The image processing algorithm first performs background segmentation, and then uses morphological operations and edge detection to extract the colloid contour. The 3D reconstruction uses the phase shift method and can achieve a depth resolution of 10 μm. The system is able to measure the colloid coverage area (accuracy ±0.1 mm²), thickness distribution (accuracy ±5 μm) and edge contour (accuracy ±10 μm).

[0161] The detection results are fed back to the parameter optimization module in real time via Ethernet. The optimization module uses a recursive Bayesian estimation method to continuously update the mapping relationship between dispensing parameters and quality indicators. For example, if it is detected that the colloid coverage area is continuously small, the system may appropriately increase the nozzle pressure or reduce the movement speed. This closed-loop optimization mechanism ensures that the dispensing process can adapt to slight changes in material properties or environmental conditions.

[0162] Based on the feedback of the detection results, the system will also dynamically adjust the objective function weight of the model predictive controller. For example, if the colloid thickness distribution is detected to be uneven, the system may increase the weight of the height deviation term from 0.2 to 0.3 to pay more attention to the accuracy of height control. This adaptive mechanism enables the control strategy to be flexibly adjusted according to the actual dispensing effect, further improving the consistency and reliability of dispensing quality.

[0163] This system, which integrates advanced control algorithms, multi-sensor fusion and real-time optimization, can achieve high-precision and high-consistency dispensing effects. In practical applications, the system can control the dispensing coverage error within ±2% and the thickness uniformity deviation within ±5%, while ensuring high production efficiency and energy utilization. This method is not only suitable for traditional dispensing applications, but also can cope with the dispensing needs of complex-shaped workpieces or special materials, showing strong versatility and adaptability.

[0164] Figure 2 FIG. 1 is a schematic diagram of a structure of a site detection and dispensing control system based on image information according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0165] The first unit is used to obtain image information of the workpiece to be glued, wherein the image information includes contour features and surface features of the workpiece to be glued; determine the position and posture of the workpiece to be glued according to the contour features, and obtain a first point detection result; detect defects on the surface of the workpiece to be glued according to the surface features, and obtain a second point detection result; fuse the first point detection result and the second point detection result to obtain the three-dimensional spatial position coordinates of the glued area of ​​the workpiece to be glued, and determine initial glue dispensing parameters, wherein the initial glue dispensing parameters include a glue dispensing trajectory, a glue dispensing speed, and a glue dispensing amount;

[0166] The second unit is used to establish a dispensing process model based on the initial dispensing parameters according to the characteristics of the material to be dispensed and the area and height information of the dispensing area, and obtain target dispensing parameters by solving the dispensing process model. The target dispensing parameters are dispensing parameters that can make the glue evenly distributed after dispensing and minimize the amount of glue used;

[0167] The third unit is used to control the dispensing equipment to perform dispensing according to the target dispensing parameters, obtain the dispensing results, obtain the dispensing area image of the workpiece after dispensing and perform quality inspection to obtain the quality inspection results; compare the quality inspection results with the preset quality standards, and complete the dispensing when the quality requirements are met; when the quality requirements are not met, correct the parameters in the dispensing process model according to the quality inspection results, re-acquire the target dispensing parameters and perform dispensing until the quality requirements are met.

[0168] According to a third aspect of the embodiments of the present invention,

[0169] An electronic device is provided, comprising:

[0170] processor;

[0171] a memory for storing processor-executable instructions;

[0172] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0173] According to a fourth aspect of the embodiments of the present invention,

[0174] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0175] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting and controlling the dispensing of glue based on image information, characterized in that: include: Acquire image information of the workpiece to be glued, wherein the image information includes contour features and surface features of the workpiece to be glued; According to the contour features, the position and posture of the workpiece to be glued are determined to obtain a first point detection result; according to the surface features, defects on the surface of the workpiece to be glued are detected to obtain a second point detection result; The first position detection result and the second position detection result are integrated to obtain the three-dimensional spatial position coordinates of the area to be glued on the workpiece to be glued, and determine the initial glue dispensing parameters, wherein the initial glue dispensing parameters include glue dispensing trajectory, glue dispensing speed and glue dispensing amount; According to the characteristics of the material to be dispensed and the area and height information of the area to be dispensed, a dispensing process model is established on the basis of the initial dispensing parameters, and the target dispensing parameters are obtained by solving the dispensing process model, wherein the target dispensing parameters are dispensing parameters that can make the glue evenly distributed after dispensing and minimize the amount of glue used; Controlling the dispensing equipment to perform dispensing according to the target dispensing parameters to obtain a dispensing result, obtaining an image of the dispensing area of ​​the workpiece after dispensing and performing quality inspection to obtain a quality inspection result; comparing the quality inspection result with a preset quality standard, and completing the dispensing when the quality requirement is met; when the quality requirement is not met, correcting the parameters in the dispensing process model according to the quality inspection result, re-obtaining the target dispensing parameters and performing dispensing until the quality requirement is met; According to the characteristics of the material to be dispensed and the area and height information of the area to be dispensed, a dispensing process model is established on the basis of the initial dispensing parameters, and the target dispensing parameters are obtained by solving the dispensing process model, including: Obtain the physical property parameters of the material to be dispensed, including viscosity, surface tension and temperature sensitivity; measure the geometric information of the area to be dispensed, including area and height data; establish a generalized Herschel-Bulkley rheological model, which includes shear stress, yield stress, consistency coefficient, shear rate, flow index and limiting viscosity parameters; use a rheometer to test the material to be dispensed to obtain the specific parameter values ​​of the generalized Herschel-Bulkley rheological model; Construct a surface evolution model based on the Level Set method, which describes the spreading behavior of the colloid on different surface shapes during the dispensing process; establish a thermal expansion model of the relationship between temperature and volume change, including initial volume, volume thermal expansion coefficient, initial temperature and current temperature parameters; Based on the generalized Herschel-Bulkley rheological model, surface evolution model and thermal expansion model, a comprehensive dispensing process optimization objective function is constructed, which includes three sub-objectives: colloid coverage deviation, colloid thickness uniformity and energy consumption; the weight coefficient of the optimization objective function is set; and the dispensing parameters are initialized, including nozzle pressure, nozzle movement speed and nozzle height; The improved particle swarm optimization algorithm is used to solve the optimization objective function. The algorithm introduces adaptive inertia weight and chaotic perturbation mechanism. The parameters of the improved particle swarm optimization algorithm are set, including population size, maximum number of iterations, acceleration constant and chaotic perturbation intensity. The improved particle swarm optimization algorithm is executed to iteratively update the speed and position of the particles. In each iteration, the optimization objective function value under the dispensing parameters corresponding to the current particle is calculated. The individual optimal solution and the global optimal solution are updated. Determine whether the convergence conditions are met or the maximum number of iterations is reached. If so, output the dispensing parameters corresponding to the global optimal solution as the final target dispensing parameters.

2. The method according to claim 1, characterized in that Acquire image information of the workpiece to be glued, wherein the image information includes contour features and surface features of the workpiece to be glued; Determining the position and posture of the workpiece to be glued according to the contour features, and obtaining the first point detection result includes: Acquire an image of a workpiece to be glued; preprocess the image of the workpiece to be glued, including applying an adaptive median filter to remove noise and using a contrast-limited adaptive histogram equalization algorithm to enhance the image contrast; construct a multi-scale image pyramid of the preprocessed image, the image pyramid comprising multiple levels of images; perform improved Canny edge detection on each level of the image pyramid, including: Calculate the gradient amplitude histogram of each level image; use the Otsu method to determine the initial high and low thresholds; dynamically adjust the high and low thresholds according to the image edge density; perform Canny edge detection; start from the top layer, propagate the detection results downward layer by layer and merge them with the lower layer results, where the strongest edge points in the lower layer area corresponding to the upper layer edge points are retained; The fused edge image is morphologically processed, including opening operation to remove noise, closing operation to fill gaps and thinning algorithm to ensure the edge width is single pixel; the Douglas-Peucker algorithm is used to simplify the contour of the processed edge, and the distance threshold is iteratively adjusted to make the number of simplified contour points within the preset range; the least squares ellipse fitting is performed on the simplified contour point set, and the generalized eigenvalue problem is solved to obtain the ellipse parameters; the position coordinates and direction angle of the workpiece are calculated according to the ellipse parameters to obtain the first point detection result.

3. The method according to claim 2, characterized in that According to the surface features, defects on the surface of the workpiece to be glued are detected to obtain a second site detection result; The first position detection result and the second position detection result are integrated to obtain the three-dimensional spatial position coordinates of the area to be glued on the workpiece to be glued, and the initial glue dispensing parameters are determined, including: Inputting the preprocessed image into a pre-trained U-Net convolutional neural network to obtain a defect area probability map; binarizing the defect area probability map to obtain a defect mask image; Performing connected region analysis on the defect mask image, calculating the centroid coordinates of each defect region, and obtaining a second site detection result; Performing feature normalization processing on the first site detection result and the second site detection result respectively; performing nonlinear mapping on the normalized first site detection result and the second site detection result respectively using a multi-layer perceptron to obtain a contour feature vector and a surface feature vector; A weight vector is designed to perform weighted fusion of the contour feature vector and the surface feature vector; the weight vector is optimized by using a particle swarm optimization algorithm, wherein the average error of the site detection after fusion is used as the fitness function; the contour feature vector and the surface feature vector are finally fused using the optimized weight vector; the fused feature vector is mapped to a two-dimensional coordinate space through a fully connected layer to obtain the final workpiece site coordinates.

4. The method according to claim 1, characterized in that: The improved particle swarm optimization algorithm is used to solve the optimization objective function including: Construct the dispensing process optimization objective function, which includes three sub-goals: colloid coverage deviation, colloid thickness uniformity and energy consumption, and set the weight coefficient of each sub-goal; determine the optimization variables as the dispensing parameter vector, including nozzle pressure, nozzle moving speed and nozzle height; Set the parameters of the improved particle swarm optimization algorithm, including population size, maximum number of iterations, acceleration constant, chaotic perturbation intensity, and upper and lower limits of inertia weight; An initial population is randomly generated, each particle represents a set of candidate dispensing parameter solutions, and a position vector, a velocity vector and an individual optimal position are assigned to each particle; Calculate the fitness value of each particle and determine the global optimal position; Enter the iterative optimization process, for each iteration: Calculate the adaptive inertia weight according to the current number of iterations; update the speed and position of each particle, where the position update introduces chaotic perturbations; perform boundary processing on the updated particle position to ensure that it is within the search space; recalculate the fitness value of each particle; update the individual optimal position of each particle; and update the global optimal position; Check whether the convergence condition is met or the maximum number of iterations is reached. If so, end the iteration; otherwise, return and output the global optimal position as the final optimized dispensing parameter.

5. The method according to claim 1, characterized in that Controlling the dispensing equipment to perform dispensing according to the target dispensing parameters, and obtaining dispensing results includes: Receiving optimized target dispensing parameters, including nozzle pressure, nozzle moving speed and nozzle height; converting the target dispensing parameters into specific control instructions through a parameter parsing module; and generating a smooth dispensing path that considers acceleration constraints and curvature continuity using a trajectory planning module according to geometric information of the area to be dispensed and the control instructions; A dynamic model of the dispensing process is constructed, which takes into account the effects of nozzle pressure, moving speed and height on colloid flow and spreading behavior; the objective function of the model predictive controller is set, which includes four weighted items: colloid flow deviation, nozzle moving speed deviation, nozzle height deviation and control quantity change rate; The pressure sensor, position encoder and vision sensor are integrated in the dispensing system to monitor the pressure change in the nozzle, provide the nozzle position and speed information, and observe the colloid spreading state. The multi-sensor fusion algorithm based on Kalman filtering is used to comprehensively process the data of multiple sensors to obtain accurate and stable system state estimation. The recursive least squares method is used to identify the dispensing process model parameters in real time, and the internal model of the model predictive controller is dynamically adjusted according to the identification results. During the dispensing process, the model predictive controller is run at a high frequency to continuously predict the system state in the future and solve the optimal control sequence. In each control cycle, only the first control action of the optimal control sequence is executed, and the optimization and solution process is repeated in the next cycle. The machine vision system is used to detect the dispensing results in real time, including the colloid coverage area, thickness distribution and edge contour; the detection results are fed back to the parameter optimization module for further adjustment and optimization of the dispensing parameters; according to the feedback detection results, the objective function weight of the model predictive controller is dynamically adjusted to balance different performance indicators.

6. A site detection and dispensing control system based on image information, used to implement the method described in any one of claims 1 to 5, characterized in that: include: The first unit is used to obtain image information of the workpiece to be glued, wherein the image information includes contour features and surface features of the workpiece to be glued; According to the contour features, the position and posture of the workpiece to be glued are determined to obtain a first point detection result; according to the surface features, defects on the surface of the workpiece to be glued are detected to obtain a second point detection result; The first position detection result and the second position detection result are integrated to obtain the three-dimensional spatial position coordinates of the area to be glued on the workpiece to be glued, and determine the initial glue dispensing parameters, wherein the initial glue dispensing parameters include glue dispensing trajectory, glue dispensing speed and glue dispensing amount; The second unit is used to establish a dispensing process model based on the initial dispensing parameters according to the characteristics of the material to be dispensed and the area and height information of the dispensing area, and obtain target dispensing parameters by solving the dispensing process model. The target dispensing parameters are dispensing parameters that can make the glue evenly distributed after dispensing and minimize the amount of glue used; The third unit is used to control the dispensing equipment to perform dispensing according to the target dispensing parameters, obtain the dispensing results, obtain the dispensing area image of the workpiece after dispensing and perform quality inspection to obtain the quality inspection results; compare the quality inspection results with the preset quality standards, and complete the dispensing when the quality requirements are met; when the quality requirements are not met, correct the parameters in the dispensing process model according to the quality inspection results, re-acquire the target dispensing parameters and perform dispensing until the quality requirements are met.

7. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 5.

8. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images