Dispensing machine operation state real-time monitoring method and system

Abstract

The application provides a dispensing machine running state real-time monitoring method and system, relates to the technical field of data processing, and the method comprises the following steps: after all ants complete a search, the global pheromone distribution is updated according to the respective performances, multiple iterations are carried out until a preset iteration number is reached, and finally configured camera parameters are obtained, so that the camera captures a dispensing image formed by the adhesive dots on the substrate according to the finally configured camera parameters; key features are extracted from the dispensing image, and the key features are matched with the features in the predetermined topological data to obtain a matching result; according to the matching result, the similarity scores between the matching features are calculated, the running state of the dispensing machine is classified according to the similarity scores, and a classification result is obtained; and the classification result is saved as a data set together with the corresponding dispensing image, key features and similarity scores. The application can improve the prediction accuracy and reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for real-time monitoring of the operating status of a dispensing machine. Background Art

[0002] In terms of predicting the operating status of dispensing machines, the threshold judgment method in traditional technology is simple and easy to use, but it does have some limitations and challenges that may affect its accuracy and generalization ability. Specifically:

[0003] First, threshold judgment methods typically focus on evaluating a single feature and may fail to fully consider the interactions and correlations between multiple features. In real-world applications, the operating status of a dispensing machine is often influenced by multiple features, and the correlations between these features are crucial for accurately assessing the operating status. Therefore, threshold judgments that rely solely on a single feature may not fully reflect the dispensing machine's true status.

[0004] Secondly, the dispensing machine may be affected by various factors during operation, causing its state to change dynamically. The threshold judgment method is based on a fixed threshold and may not be able to flexibly respond to such changes. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a method and system for real-time monitoring of the operating status of a dispensing machine, which can improve the prediction accuracy and reliability.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] In a first aspect, a method for real-time monitoring of the operating status of a dispensing machine is provided, the method comprising:

[0008] Map the camera parameter space to the corresponding search space, and each ant will search in the solution space according to its own position and movement rules;

[0009] Each ant selects its next location based on the pheromone concentration and heuristic information at its current location. At each new location, the camera's image capture quality is evaluated using an evaluation function.

[0010] After all ants complete a search, they update the global pheromone distribution based on their respective performance and perform multiple iterations until a preset number of iterations is reached to obtain the final configured camera parameters, so that the camera can capture the glue dot image formed on the substrate according to the final configured camera parameters;

[0011] Extract key features from the dispensing image and match the key features with features in the predetermined topological data to obtain matching results;

[0012] Based on the matching results, the similarity scores between the matching features are calculated, and the operating states of the dispensing machines are classified according to the similarity scores to obtain classification results;

[0013] Save the classification results together with the corresponding dispensing images, key features and similarity scores as a dataset;

[0014] Using the data set to train the prediction model to obtain a trained prediction model;

[0015] Based on the trained prediction model and the real-time captured dispensing images, the real-time operating status of the dispensing machine is predicted.

[0016] Furthermore, each ant represents a set of configured camera parameters.

[0017] Furthermore, each ant selects its next location based on the pheromone concentration at its current location and heuristic information, including:

[0018] For each ant, traverse all adjacent positions of the current position and calculate the probability of each adjacent position. The probability calculation formula is:

[0019] ;

[0020] in, Indicates that the ant is moving from its current position Move to adjacent position The probability of selection; Indicates from position To location pheromone concentration; Indicates from position To location pheromone concentration; and Represents heuristic information; 、 and represents weight; and represents environmental factors; Indicates current location The set of all adjacent positions of , where , It's location How often it was selected in the past iteration, is the maximum selection frequency among all adjacent positions; , It's location How often it was selected in past iterations; Indicates the current location of the ant; Indicates that the ant is moving from its current position the next position selected; a particular position of all neighboring positions to the current position; , power consumption from position to position ; maximum value of the environmental condition metric among all neighboring paths; , power consumption from position to position ;

[0021] According to the probability, a neighboring position is selected as the next moving target of the ant.

[0022] Further, the calculation formula of the evaluation function is:

[0023] ;

[0024] wherein, image capture quality score at position ; image resolution at position ; contrast at position ; color saturation at position ; indicates the coefficient.

[0025] Further, key features are extracted from the dispensing image, including:

[0026] The captured dispensing image is preprocessed to obtain a preprocessed image;

[0027] The gradient intensity and direction of each pixel point in the preprocessed image are calculated using the Sobel operator, where the gradient intensity represents the possibility of an edge at the pixel point, and the gradient direction represents the direction of the edge;

[0028] Along the gradient direction, the gradient intensity of the current pixel point is compared with the gradient intensity of its neighboring pixel points;

[0029] The pixel points with gradient intensity higher than the first threshold value are marked as determined edges, and the pixel points with gradient intensity lower than the second threshold value are discarded; for the pixel points with gradient intensity between the two first threshold values and the second threshold value, if they are connected to the determined edges, they are regarded as edges, otherwise they are discarded;

[0030] According to the edges, contours are searched on the binarized edge image, and contours representing dispensing points are screened out;

[0031] According to the contour of the glue point, key points are detected on the pre-processed image;

[0032] The area around the key point is divided into several sub-regions, and for each pixel in each sub-region, the gradient direction and gradient magnitude are calculated;

[0033] According to the gradient direction and gradient magnitude, a gradient histogram is constructed for each sub-region, and the gradient histograms of all sub-regions are spliced to form a long vector, which is used as a feature descriptor of the key point to represent the key features of the key point.

[0034] Further, the key features are matched with the features in the predetermined topological data to obtain a matching result, including:

[0035] Each key feature in the current image is traversed, and the Euclidean distance between the current key feature and the predetermined feature is calculated;

[0036] The Euclidean distance is stored in a distance matrix, and the rows of the matrix represent the key features of the current image, and the columns represent the features in the predetermined feature set;

[0037] For each row in the distance matrix, the column index corresponding to the Euclidean distance is found, and the column index indicates the feature in the predetermined feature set that matches the current key feature;

[0038] According to the column index, a matching pair is established between the key feature of the current image and the feature in the predetermined topological data, wherein each matching pair contains a key feature in the current image and a feature in the predetermined feature set that matches it.

[0039] Further, according to the matching result, the similarity score between the matching features is calculated, including:

[0040] The feature vector of the current image is extracted from the matching pair and the feature vector of the predetermined topological data ;

[0041] A weight is assigned to each dimension of the feature, and let W be the weight vector, then the weighted dot product is calculated as:

[0042] wherein, is the weight of the th feature, and are the th element of the feature vector of the current image and the feature vector of the predetermined topological data, respectively;

[0043] The weighted length is calculated, and the weighted length is calculated as:

[0044] ;

[0045] ;

[0046] The similarity score is obtained by dividing the weighted dot product by the product of the two weighted moduli:

[0047] ;

[0048] wherein, denotes the weighted dot product; denotes the interaction term, ; denotes the similarity score; denotes the total number of dimensions of the features; denotes the bias term; denotes the interaction weight between the th feature and the th feature; denotes the adjustment coefficient; denotes the regularization parameter; denotes the base of the natural logarithm; is the th element of the feature vector of the current image; and denote the index.

[0049] In a second aspect, a dispensing machine running state real-time monitoring system is applied to the method and comprises:

[0050] A search module is configured to map the parameter space of the camera into a corresponding search space, and each ant searches in the solution space according to its own position and movement rule. Each ant selects the next position according to the pheromone concentration and heuristic information of the current position, and evaluates the image capture quality of the camera at each new position through an evaluation function.

[0051] A processing module is configured to update the global pheromone distribution according to the respective performances of all the ants after they complete a search, and perform multiple iterations until a preset number of iterations is reached, so as to obtain the finally configured camera parameters. The camera captures the dispensing image of the adhesive dots formed on the substrate according to the finally configured camera parameters.

[0052] A matching module is configured to extract key features from the dispensing image and match the key features with the features in the predetermined topological data to obtain a matching result. According to the matching result, the similarity scores between the matching features are calculated, and the running state of the dispensing machine is classified according to the similarity scores to obtain a classification result.

[0053] a prediction module configured to save the classification result together with the corresponding dispensing image, key features and similarity scores as a dataset, train the prediction model using the dataset to obtain a trained prediction model, and predict the real-time running state of the dispensing machine according to the trained prediction model and the real-time captured dispensing image.

[0054] In a third aspect, a computing device includes:

[0055] one or more processors;

[0056] a storage device storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method.

[0057] In a fourth aspect, a computer-readable storage medium stores a program, when the program is executed by a processor, the method is implemented.

[0058] The above-mentioned scheme of the present application at least includes the following beneficial effects:

[0059] By optimizing the camera parameter configuration using the ant colony algorithm, the method can capture high-quality dispensing images, thereby effectively improving the accuracy and reliability of the monitoring data. At the same time, combined with the key feature extraction and matching technology, the running state of the dispensing machine can be more accurately identified, reducing the misjudgment and omission.

[0060] The method can capture dispensing images in real time and quickly predict the running state of the dispensing machine according to the trained prediction model. This enables the operator to discover potential running abnormalities or faults in a timely manner and take appropriate measures to intervene, thereby avoiding the occurrence of production accidents and improving production efficiency.

[0061] By iteratively optimizing the camera parameters and updating the global pheromone distribution, the method can adaptively adjust the monitoring strategy to adapt to the dispensing machine running under different working conditions and equipment states. This adaptive ability makes the method more versatile and flexible, and can be widely applied to various dispensing machine running state monitoring scenarios.

[0062] By saving the classification result together with the corresponding dispensing image, key features and similarity scores as a dataset, and training the prediction model using the dataset, the method can continuously enrich and optimize the learning resources of the model. This helps to improve the prediction performance and generalization ability of the model, so that it can make more accurate predictions when facing new dispensing images. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1is a flowchart of a real-time operation state monitoring method of a dispensing machine provided by an embodiment of the present application.

[0064] Figure 2 is a schematic diagram of a real-time operation state monitoring system of a dispensing machine provided by an embodiment of the present application. DETAILED DESCRIPTION

[0065] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it is understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.

[0066] As shown in Figure 1 An embodiment of the present application proposes a real-time operation state monitoring method of a dispensing machine, which comprises the following steps:

[0067] Step 11, mapping the parameter space of the camera into a corresponding search space, each ant will search in the solution space according to its own position and movement rules, and each ant represents a set of configured camera parameters;

[0068] Step 12, each ant selects the next position according to the pheromone concentration and heuristic information of the current position, and at each new position, the image capture quality of the camera is evaluated by an evaluation function;

[0069] Step 13, after all ants complete a search, update the global pheromone distribution according to their respective performances, perform multiple iterations until a predetermined number of iterations is reached, to obtain the final configured camera parameters, so that the camera captures the dispensing image formed by the adhesive dots on the substrate according to the final configured camera parameters;

[0070] Step 14, extracting key features from the dispensing image and matching the key features with features in predetermined topological data to obtain a matching result;

[0071] Step 15, according to the matching result, calculating the similarity score between the matching features, and according to the similarity score, classifying the operation state of the dispensing machine to obtain a classification result;

[0072] Step 16, saving the classification result together with the corresponding dispensing image, key features and similarity score as a data set;

[0073] Step 17, training a prediction model using the data set to obtain a trained prediction model;

[0074] Step 18, according to the trained prediction model and the real-time captured dispensing image, the real-time running state of the dispensing machine is predicted.

[0075] In the embodiments of the present application, by searching for the optimal camera configuration in the parameter space using the ant algorithm, the method can efficiently find the best parameter setting for capturing dispensing images. This not only improves the efficiency of camera configuration, but also ensures the quality of the captured images. The method can accurately identify the key information in the dispensing process by extracting key features from the dispensing image and matching them with the features in the predetermined topological data. This feature-based matching method improves the accuracy of state monitoring, making the evaluation of the dispensing machine running state more accurate. By calculating the similarity score between the matching features and classifying the running state of the dispensing machine according to the score, the method realizes intelligent discrimination of the dispensing machine state. This intelligent state classification helps to discover abnormal situations in time and take appropriate measures for intervention, thereby ensuring the stability and reliability of the dispensing process. The method not only classifies and saves historical data, but also uses these data sets to train prediction models to realize real-time prediction of the dispensing machine running state. This prediction capability enables operators to take preventive measures before problems occur, further improving the maintenance efficiency and production benefit of the dispensing machine. By saving the classification results, dispensing images, key features and similarity scores as data sets and using them to train prediction models, the method realizes a data-driven decision-making process.

[0076] In a preferred embodiment of the present application, step 11 above maps the parameter space of the camera into a corresponding search space, and each ant will search in the solution space according to its own position and movement rules. Each ant represents a set of camera parameter configurations, which can include:

[0077] Step 111, determine the parameter space of the camera, which includes adjustable parameters such as focal length, exposure time, white balance, contrast, brightness, etc. of the camera. Next, set a reasonable value range for each parameter, map the value range of these parameters to a multi-dimensional search space, each dimension corresponds to a camera parameter, set the number of ants, and randomly assign an initial position in the search space to each ant, which represents an initial camera parameter configuration.

[0078] In a preferred embodiment of the present application, step 12 above, each ant selects the next position according to the pheromone concentration and heuristic information of the current position, which can include:

[0079] Step 121, for each ant, traverse all neighboring positions of the current position, and for each neighboring position, calculate its probability, where the calculation formula of the probability is:

[0080] ;

[0081] wherein, denotes the selection probability of the ant to move from the current position to a neighboring position ; denotes the pheromone concentration from position to position ; denotes the pheromone concentration from position to position ; and denote heuristic information; , and denote weights; and denote environmental factors; denotes the set of all neighboring positions of the current position , wherein, , is the frequency of position being selected in past iterations, is the maximum selection frequency among all neighboring positions; , is the frequency of position being selected in past iterations; denotes the position where the ant is currently located; denotes the next position selected by the ant from the current position ; denotes a particular position of all neighboring positions of the current position ; , denotes the power consumption from position to position ; denotes the maximum value of the environmental condition metric among all neighboring paths; , denotes the power consumption from position to position ;

[0082] Step 122, selecting a neighboring position as the next moving target of the ant according to the probability, specifically comprising: selecting a neighboring position as the next moving target of the ant according to the selection probability of each neighboring position calculated in step 121 roulette wheel selection is used to select the next moving target; in roulette wheel selection, the probability of each adjacent position being selected is proportional to its selection probability, which means that a higher selection probability means a greater chance of being selected. Once the next moving target (i.e. the adjacent position ) is selected, the ant is moved from the current position i to the target position, and the position information of the ant is updated for use in subsequent search iterations.

[0083] In an embodiment of the present application, by comprehensively considering pheromone concentration, heuristic information and environmental factors, the method can more accurately guide the ants to move towards the potential high-quality solution region, thereby reducing unnecessary search and improving search efficiency. The introduction of the selection frequency in the past iterations as an adjustment factor can effectively prevent the ants from gathering around the local optimal solution too early, thereby increasing the possibility of finding the global optimal solution. The introduction of the environmental factor enables the algorithm to adjust according to specific conditions (such as power consumption) in the actual application scenario, enhancing the adaptability and flexibility of the algorithm. The method calculates the selection probability by comprehensively considering multiple information, taking into account both known information (such as pheromone concentration and heuristic information) and exploration of unknown areas (by reducing the probability of frequently selected positions), achieving a good balance between exploration and utilization. Through the detailed probability calculation process, each ant can more intelligently select the next moving position, which helps to improve the quality of the final solution found, making the camera parameter configuration more accurate and optimized.

[0084] In a preferred embodiment of the present application, the calculation formula of the evaluation function is:

[0085] ;

[0086] wherein, represents the image capture quality score at position ; represents the image resolution at position ; represents the contrast at position ; represents the color saturation at position ; represents the coefficient.

[0087] In an embodiment of the present application, the function comprehensively considers three key indicators: image resolution , contrast and color saturation , which can more comprehensively reflect the quality of the image; by introducing the coefficient ,This function can adjust the weights of various indicators according to actual needs, making the evaluation more flexible.

[0088] Specific calculation process:

[0089] First, get the Image resolution on , contrast and color saturation The specific value of . According to the given coefficient , calculate the weights of image resolution, contrast and color saturation respectively. The weight calculation formula is as follows:

[0090] Weight of resolution: ;

[0091] Contrast weight: ;

[0092] Weight of color saturation: ;

[0093] These weights reflect the importance of each indicator in the overall evaluation. The weighted average of the original values ​​of each indicator is used to obtain the position. Image capture quality rating on This score takes into account resolution, contrast and color saturation, and is based on the coefficient The contribution of each indicator was adjusted to obtain Finally, the image capture quality at different locations can be compared and analyzed, and this score can be used to guide the setting and optimization of image acquisition equipment.

[0094] In a preferred embodiment of the present invention, the above step 13 is specifically implemented as follows:

[0095] Set an iteration counter, iteration_count, and initialize it to 0, and set a preset number of iterations, max_iterations; enter an iteration loop and repeat the following steps until iteration_count reaches max_iterations; for each ant, perform a search process (as described in step 12), allowing the ant to select the next location based on the pheromone concentration and heuristic information at the current location until a complete search path is completed; when all ants have completed a search, calculate the score of each location on each ant's search path according to the evaluation function (such as the image capture quality scoring formula mentioned above), and record the total score or average score of each ant's search path as the ant's search performance;

[0096] According to the performance of each ant, update the global pheromone distribution, ants with good performance will leave more pheromones on their search paths to attract subsequent ants. Pheromone update can adopt different strategies, such as increasing the pheromone concentration on the paths with excellent performance while reducing the pheromone concentration on the paths with poor performance, or considering the evaporation of pheromones, etc.

[0097] Increase the iteration counter iteration_count by 1, if iteration_count reaches max_iterations, terminate the iteration loop, otherwise, return to continue the next iteration; after the iteration ends, determine the final camera parameter configuration according to the global pheromone distribution or other criteria (such as selecting the configuration with the highest score), output or save the final configured camera parameters.

[0098] Through the above steps, the swarm intelligence of ants can be used for multiple iterations of search, constantly optimizing the global pheromone distribution, and ultimately obtaining a set of excellent camera parameter configurations. This method combines the mechanisms of heuristic search and pheromone update, which helps to find high-quality solutions in complex parameter spaces.

[0099] In a preferred embodiment of the present application, step 14 of extracting key features from the dispensing image can include:

[0100] Step 141, pre-process the captured dispensing image to obtain a pre-processed image; use the Sobel operator to calculate the gradient intensity and direction of each pixel point in the pre-processed image, the gradient intensity represents the possibility of edge at the pixel point, and the gradient direction represents the direction of the edge, specifically including:

[0101] Perform grayscale processing on the captured dispensing image to convert it into a grayscale image to reduce the computational load of subsequent processing; perform denoising processing on the grayscale image, such as using median filtering or Gaussian filtering, etc. to eliminate noise in the image and improve the accuracy of edge detection; use the Sobel operator to perform convolution operation on the pre-processed grayscale image, respectively calculate the gradient intensity of each pixel point in x direction and y direction G x and G y ); according to G x and G y , calculate the total gradient intensity G ) and gradient direction (θ) of each pixel point. The total gradient intensity G can be calculated by the Pythagorean theorem, and the gradient direction θ can be calculated by the arctangent function.

[0102] Step 142, compare the gradient strength of the current pixel point with its adjacent pixel points along the gradient direction; mark the pixel points with gradient strength higher than the first threshold as determined edges, discard the pixel points with gradient strength lower than the second threshold; for the pixel points with gradient strength between the two thresholds, if connected to the determined edges, consider them as edges, otherwise discard, specifically including:

[0103] Set two thresholds: the first threshold (high threshold) and the second threshold (low threshold), which are used to distinguish strong edges, weak edges and non-edge pixels. Traverse each pixel point in the image, and make edge judgment according to its gradient strength G:

[0104] If G is higher than the first threshold, mark the pixel point as a determined edge;

[0105] If G is lower than the second threshold, discard the pixel point, that is, consider it not as an edge;

[0106] If G is between the two thresholds, temporarily mark it as a potential edge, which needs further processing.

[0107] For the pixel points marked as potential edges, check whether they are connected to the pixel points marked as determined edges:

[0108] If connected, consider it as an edge, because it may be part of a real edge;

[0109] If not connected, discard it, because it may be a false edge caused by noise.

[0110] Step 143, according to the edge, search for contours on the binarized edge image and filter out the contours representing the dot points; according to the contours of the dot points, detect key points on the preprocessed image, specifically including:

[0111] Call the findContours function in the OpenCV library, which can detect continuous edge contours in a binary image; the findContours function will return a contour list, each contour is composed of a series of points, representing a continuous edge in the image. Contour filtering:

[0112] a. Traverse the contour list obtained in the last step, for each contour, calculate its area and perimeter, the area can be obtained by counting the number of pixel points inside the contour (or directly calculated using the contourArea function), and the perimeter can be obtained by accumulating the distance between adjacent points on the contour.

[0113] b. According to the actual size and shape characteristics of the dispensing point, set appropriate area threshold and perimeter threshold. These thresholds are used to filter out the contours representing the dispensing point. For example, an area threshold can be set to exclude too small or too large contours, and a perimeter threshold can be set to exclude contours with irregular shapes.

[0114] c. Keep the contours that meet the area and perimeter threshold conditions as candidate contours representing the dispensing point.

[0115] For each filtered dispensing point candidate contour, calculate its centroid as a key point. The centroid can be obtained by averaging the coordinates of all contour points, i.e. calculating the average of the x and y coordinates respectively.

[0116] Step 144, divide the area around the key point into several sub-regions, and for each pixel in each sub-region, calculate its gradient direction and gradient amplitude; according to the gradient direction and gradient amplitude, construct a gradient histogram for each sub-region, and splice the gradient histograms of all sub-regions to form a long vector, which is used as the feature descriptor of the key point to represent the key features of the key point, the specific implementation process is as follows:

[0117] For each detected key point, determine a neighborhood region centered on it, and this region is of fixed size (e.g. 32x32 pixels). If an adaptive size neighborhood region is used, the size of the region can be determined according to the scale information obtained in the key point detection stage (such as the scale factor in the SIFT algorithm). Sub-region division:

[0118] a. Further divide the neighborhood region into several sub-regions, using grid division such as 4x4 or 8x8 grid. In this way, each sub-region will contain a part of the pixels in the neighborhood.

[0119] Gradient histogram construction:

[0120] a. For each pixel in the sub-region, calculate its gradient direction and gradient amplitude, which can be done by using the Sobel operator or other gradient calculation methods used in the previous step.

[0121] b. Create a gradient histogram that uses multiple bins (such as 8 or 16) to quantify the range of gradient directions. Each bin corresponds to a specific range of gradient directions, where a bin is a small interval or unit in the gradient histogram used to quantify the range of gradient directions.

[0122] c. Traverse each pixel in the sub-region, and according to its gradient direction, accumulate the gradient amplitude into the corresponding bin. In this way, the value in each bin will represent the total gradient strength in that gradient direction.

[0123] d. Gradient magnitude is weighted, for example, pixels far from the key point are down-weighted using a Gaussian function to emphasize features near the key point. Feature descriptor is generated:

[0124] a. The gradient histograms of all sub-regions are concatenated to form a long vector. This vector contains detailed gradient information of the region around the key point.

[0125] b. Optionally, the feature descriptor is normalized to eliminate the influence of illumination changes on the feature description. Normalization can be achieved by dividing the bin value of each gradient histogram by the sum of all bin values.

[0126] c. The final long vector is the feature descriptor of the key point, which can be used for subsequent feature matching, target tracking or recognition tasks.

[0127] In the embodiment of the present application, the Sobel operator is used to calculate the gradient intensity and direction of the pixel points, which can accurately detect the edges in the image. This edge detection method is crucial for identifying the outline of the dispensing point, as it can distinguish between true edges and background noise, thereby improving the accuracy of edge detection. The use of a double threshold (first threshold and second threshold) processing mechanism allows the algorithm to flexibly handle pixel points with gradient intensity between the two thresholds while marking the edges. This processing method enhances the robustness of the algorithm, allowing it to stably extract key features in images of different qualities. According to the edge detection results, search for the outline on the binary edge image and select the outline representing the dispensing point. This process quickly locates the region of interest, i.e., the dispensing point, thereby improving the efficiency of subsequent key point detection and feature extraction. According to the outline of the dispensing point, detect the key points on the preprocessed image. This contour-based key point detection method ensures accurate positioning of the key points. By dividing the area around the key point into several sub-regions and constructing a gradient histogram for each sub-region, a long vector is finally formed as the feature descriptor of the key point. This description method not only considers the local gradient information of the key point, but also enhances the richness and discriminability of the features by concatenating the histograms of multiple sub-regions, making the extracted key features more representative and robust.

[0128] In a preferred embodiment of the present application, the step 14 of matching the key features with the features in the predetermined topological data to obtain a matching result further comprises:

[0129] Step 145: Traverse each key feature in the current image, calculate the Euclidean distance between the current key feature and the predetermined feature, and the specific implementation process is as follows:

[0130] A set of key features extracted from the current image is prepared, as well as a predetermined feature set. Each feature is typically represented by a feature vector, which contains information describing the feature (such as features extracted by SIFT, SURF, etc. algorithms). A loop structure is used to iterate through each key feature in the current image. For each key feature, the following operations are performed:

[0131] a. Extract the feature vector of the key feature currently being processed from the set of key features of the current image.

[0132] b. For each feature in the predetermined feature set, calculate the Euclidean distance between the feature vector of the current key feature and the feature vector of the predetermined feature.

[0133] Step 146, store the Euclidean distances in a distance matrix, where the rows represent the key features of the current image and the columns represent the features in the predetermined feature set, as follows:

[0134] According to the number of key features of the current image and the number of features in the predetermined feature set, create a two-dimensional array (i.e., a matrix) to store the Euclidean distance values. The number of rows of the matrix is equal to the number of key features of the current image, and the number of columns is equal to the number of features in the predetermined feature set. Fill the distance matrix with the Euclidean distance values calculated in step 145. Each distance value is placed in the corresponding position of the matrix, where the row index corresponds to the key feature in the current image and the column index corresponds to the feature in the predetermined feature set.

[0135] Step 147, for each row in the distance matrix, find the column index corresponding to the Euclidean distance, which indicates the feature in the predetermined feature set that matches the current key feature, as follows:

[0136] For each row in the distance matrix (representing a key feature in the current image), perform the following operations:

[0137] a. Find the smallest Euclidean distance value in the row by iterating through all elements of the row and comparing their sizes.

[0138] b. Record the column index corresponding to the smallest distance value found. This column index indicates the feature in the predetermined feature set that best matches the current key feature.

[0139] c. Store the found column index in the array or list prepared at initialization.

[0140] Step 148, based on the column index, establish a matching pair between the key feature of the current image and the feature in the predetermined topological data, where each matching pair contains a key feature in the current image and a feature in the predetermined feature set that matches it, as follows:

[0141] An empty list is created to store the matching pairs. For each column index recorded in step 147, the following operations are performed:

[0142] a. Obtain the key feature from the set of key features of the current image using the row index corresponding to the column index.

[0143] b. Obtain the matching feature from the predetermined feature set using the column index.

[0144] c. Combine the key feature of the current image and the matching feature from the predetermined feature set into a matching pair. This matching pair contains two features that match each other.

[0145] d. Add the created matching pair to the list of matching pairs prepared at the initialization. After completing the creation of all matching pairs, return the list containing all matching pairs as the final matching result.

[0146] In an embodiment of the present application, the similarity between each key feature in the current image and the predetermined features can be accurately measured by calculating the Euclidean distance between them. The distance matrix is used to store the Euclidean distance, which can facilitate the retrieval and comparison of the similarity between different features. The rows of the matrix represent the key features of the current image, and the columns represent the features in the predetermined feature set. This structure makes data access and operation efficient. In the matching process, only the minimum distance value and its corresponding column index in the distance matrix need to be found to quickly find the feature that best matches the current key feature. This method can be easily extended to handle large-scale feature sets. The size of the distance matrix is proportional to the number of key features in the current image and the number of features in the predetermined feature set, so it can easily handle any number of features. In addition, by using efficient matrix operation libraries or parallel computing techniques, processing speed can be further improved. Since the Euclidean distance is used as the similarity measure, this method has a certain robustness to scale and rotation changes between features. The Euclidean distance can capture the spatial position relationship between features, thus tolerating these changes to some extent.

[0147] In a preferred embodiment of the present application, step 15 of calculating the similarity score between matching features based on the matching result can include:

[0148] Extracting the feature vector of the current image and the feature vector of the predetermined topological data from the matching pair, respectively ;

[0149] Assigning a weight to each dimension of the feature, and let W be the weight vector, then the weighted dot product calculation is:

[0150] wherein,​ is the weight of the th feature, and is the th element of the feature vector of the current image and the feature vector of the predetermined topological data, respectively;

[0151] the weighted norm is calculated as

[0152]

[0153]

[0154] the similarity score is obtained by dividing the weighted dot product by the product of the two weighted norms:

[0155]

[0156] where denotes the weighted dot product; denotes the interaction term, denotes the similarity score; denotes the total number of dimensions of the features; denotes the bias term; denotes the interaction weight between the th feature and the th feature; denotes the adjustment coefficient; denotes the regularization parameter; denotes the base of the natural logarithm; is the th element of the feature vector of the current image; and denote the indices.

[0157] ​​​​In the embodiments of the present application, by means of the weight vector W, the method can assign different importance to different feature dimensions, thereby realizing more refined similarity evaluation. This weighting processing allows the algorithm to emphasize or weaken the influence of certain features in specific application scenarios, improving the flexibility and accuracy of similarity calculation. The calculation method of the weighted modulus length increases the sensitivity to the change of feature values by introducing the exponential function and the weight factor, so that the change of similarity can be accurately reflected when there is a slight difference in feature values. This design enhances the robustness of the algorithm, so that it can maintain stable performance when facing noise or data disturbance. The calculation of the similarity score CS not only considers the weighted dot product, but also normalizes it by dividing the product of the two weighted modulus lengths, further eliminating the influence of feature vector scale difference on similarity calculation. At the same time, by subtracting the square sum of the weight vector, the algorithm punishes the weight allocation, avoiding excessive dependence on a certain feature, so as to obtain a more comprehensive and balanced similarity measure.

[0158] In a preferred embodiment of the present application, according to the similarity score, the running state of the dispensing machine is classified to obtain a classification result, which can include:

[0159] Collecting historical data of the dispensing machine under different running states, including feature data and corresponding similarity scores of normal operation, abnormal operation and other states; by analyzing these historical data, determine the similarity score threshold between different running states.

[0160] For new dispensing machine running data, first extract its feature vector, use the similarity score calculation method described earlier (step 15) to calculate the similarity score between the new data and various running state template data.

[0161] Compare the similarity score of the new data with the determined threshold, if the similarity score is higher than the threshold of normal running state, it is judged that the dispensing machine is in normal running state. If the similarity score is lower than the threshold of a certain abnormal running state, it is judged that the dispensing machine is in that abnormal running state, and the classification result is recorded in the log or database.

[0162] In a preferred embodiment of the present application, the above step 16, saving the classification result together with the corresponding dispensing image, key feature and similarity score as a data set, can include:

[0163] Organize the classification results, key features, and similarity scores of each sample into a structured data format, such as a CSV file or database records. For the dispensing images, save them as image files (e.g., JPEG, PNG, etc.) and ensure that the file names or paths correspond to the records in the structured data. Create a new dataset directory to store all related data, and save the organized structured data as part of the dataset, for example, named data.csv. Save the dispensing image files to another subdirectory of the dataset, for example, named images.

[0164] In a preferred embodiment of the present application, step 17, training a prediction model using the dataset to obtain a trained prediction model, can include:

[0165] According to the complexity of the problem and the size of the dataset, design a CNN network structure, including determining the number and parameter settings of convolutional layers, pooling layers, and fully connected layers; select activation functions for each layer of the CNN, such as ReLU, Sigmoid, etc., to increase the non-linear representation ability of the network; according to the task type (such as classification or regression), select a suitable loss function, such as cross-entropy loss function. At the same time, select an optimizer to update the network weights, such as Adam or SGD.

[0166] Use a data loading library (such as PyTorch's DataLoader or TensorFlow's tf.data) to load the dataset; normalize the image data, scaling the pixel values to 0-1; standardize the data to have a zero mean and unit variance; to increase the generalization ability of the model, you can perform rotation, translation, scaling, etc. Enhancement operations are performed on the image; adjust all images to a uniform size to meet the input requirements of the CNN.

[0167] Use a random method to divide the dataset into training set, validation set and test set, the proportion can be 70%:15%:15% or 60%:20%:20%; according to the designed CNN structure, initialize the model parameters; set the learning rate, batch size, iteration number, etc. Training parameters; in each iteration, a batch of data is randomly selected from the training set, the loss is calculated by forward propagation, and then the model parameters are updated using backpropagation and optimizer. Record the loss value and accuracy of each iteration; during training, use the validation set to periodically evaluate the performance of the model, and record the best model weight. If the model performance on the validation set is found to be declining or showing signs of overfitting, appropriate measures can be taken, such as adjusting the learning rate, increasing the regularization term, etc.; after training is complete, use the test set to evaluate the model, calculate the test accuracy, recall rate, F1 score, etc. Indexes to comprehensively evaluate the performance of the model, to obtain a trained prediction model.

[0168] In a preferred embodiment of the present application, step 18, predicting the real-time running state of the dispensing machine according to the trained prediction model and the real-time captured dispensing image, can include:

[0169] setting the dispensing machine to capture image data of the dispensing process in real time; performing necessary preprocessing on the captured real-time dispensing image, such as denoising, contrast enhancement, etc.; applying the same feature extraction algorithm as in the training phase to extract the key feature vector from the preprocessed image; using a suitable machine learning library (such as TensorFlow, PyTorch, etc.), loading the previously trained and saved prediction model file; inputting the extracted and preprocessed key feature vector into the loaded prediction model; through the inference process of the model, calculating the prediction result of the current running state of the dispensing machine, which is a classification task, and the model will output the probability distribution of each possible state; according to the output of the model, analyzing and determining the real-time running state of the dispensing machine, which can choose the state with the highest probability as the prediction result, or set a threshold according to specific requirements to determine the state.

[0170] As shown in Figure 2 , the embodiment of the present application also provides a dispensing machine running state real-time monitoring system 20, which includes:

[0171] The search module 21 is used to map the parameter space of the camera into a corresponding search space, and each ant will search in the solution space according to its own position and movement rules; each ant selects the next position according to the pheromone concentration and heuristic information of the current position, and at each new position, the image capture quality of the camera is evaluated by an evaluation function;

[0172] The processing module 22 is used to update the global pheromone distribution according to the respective performances after all ants complete a search, and multiple iterations are performed until a preset number of iterations is reached to obtain the finally configured camera parameters, so that the camera captures the dispensing image of the adhesive dots formed on the substrate according to the finally configured camera parameters;

[0173] The matching module 23 is used to extract key features from the dispensing image and match the key features with the features in the predetermined topological data to obtain a matching result; according to the matching result, a similarity score between the matching features is calculated, and the running state of the dispensing machine is classified according to the similarity score to obtain a classification result;

[0174] The prediction module 24 is used to save the classification result together with the corresponding dispensing image, key features and similarity score as a data set; the data set is used to train the prediction model to obtain a trained prediction model; and the real-time running state of the dispensing machine is predicted according to the trained prediction model and the real-time captured dispensing image.

[0175] It should be noted that the system is a system corresponding to the above method, all the implementation manners in the above method embodiment are applicable to this embodiment, and the same technical effects can also be achieved.

[0176] Embodiments of the present application also provide a computing device, comprising: a processor, a memory storing a computer program, the computer program being executed by the processor to perform the method as described above. All the implementation manners in the above method embodiments are applicable to this embodiment, and the same technical effects can also be achieved.

[0177] Embodiments of the present application also provide a computer readable storage medium storing instructions, when the instructions are executed on a computer, the computer executes the method as described above. All the implementation manners in the above method embodiments are applicable to this embodiment, and the same technical effects can also be achieved.

[0178] The above is the preferred embodiment of the present application, it should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can also be made, these improvements and refinements should also be considered as the protection scope of the present application.

Claims

1. A method for real-time monitoring of the operating status of a dispensing machine, characterized in that: The method comprises: The camera parameter space is mapped to the corresponding search space. Each ant will search in the solution space according to its own position and movement rules. Each ant represents a set of configured camera parameters. Each ant selects its next location based on the pheromone concentration and heuristic information at its current location. At each new location, the camera's image capture quality is evaluated using an evaluation function. After all ants complete a search, they update the global pheromone distribution based on their respective performance and perform multiple iterations until a preset number of iterations is reached to obtain the final configured camera parameters, so that the camera can capture the glue dot image formed on the substrate according to the final configured camera parameters; Extract key features from the dispensing image and match the key features with features in the predetermined topological data to obtain matching results; Based on the matching results, the similarity scores between the matching features are calculated, and the operating states of the dispensing machines are classified according to the similarity scores to obtain classification results; Save the classification results together with the corresponding dispensing images, key features and similarity scores as a dataset; Using the data set to train the prediction model to obtain a trained prediction model; Based on the trained prediction model and the real-time captured dispensing images, the real-time operating status of the dispensing machine is predicted; Each ant chooses its next location based on the pheromone concentration at its current location and heuristic information, including: For each ant, traverse all adjacent positions of the current position and calculate the probability of each adjacent position. The probability calculation formula is: ； in, Indicates that the ant is moving from its current position Move to adjacent position The probability of selection; Indicates from position To location pheromone concentration; Indicates from position To location pheromone concentration; and Represents heuristic information; 、 and represents weight; and represents environmental factors; Indicates current location The set of all adjacent positions of , where , It's location How often it was selected in the past iteration, is the maximum selection frequency among all adjacent positions; , It's location How often it was selected in past iterations; Indicates the current location of the ant; Indicates that the ant is moving from its current position The next position selected; Indicates the current location A specific position of all adjacent positions; , Indicates from position To location Power consumption; represents the maximum value of the environmental condition metric among all adjacent paths; , Indicates from position To location Power consumption; According to the probability, an adjacent position is selected as the next moving target of the ant; The calculation formula of the evaluation function is: ； in, Indicates the location Image capture quality rating on ; Indicates the location Image resolution on ; Indicates the location Contrast on Indicates the location Color saturation on represents the coefficient; Extract key features from dispensing images, including: Pre-processing the captured dispensing image to obtain a pre-processed image; The Sobel operator is used to calculate the gradient intensity and direction of each pixel in the preprocessed image. The gradient intensity indicates the possibility of an edge at the pixel, and the gradient direction indicates the direction of the edge. Compare the gradient strength of the current pixel with the gradient strength of its adjacent pixels along the gradient direction; Pixels with gradient strengths higher than the first threshold are marked as confirmed edges, and pixels with gradient strengths lower than the second threshold are discarded. For pixels with gradient strengths between the first and second thresholds, if they are connected to a confirmed edge, they are considered edges, otherwise they are discarded. According to the edge, the contour is searched on the binary edge image, and the contour representing the dispensing point is screened out; Detect key points on the pre-processed image based on the outline of the dispensing point; The area around the key point is divided into several sub-regions, and for each pixel in the sub-region, its gradient direction and gradient magnitude are calculated; According to the gradient direction and gradient magnitude, a gradient histogram is constructed for each sub-region, and the gradient histograms of all sub-regions are spliced ​​together to form a long vector, which is used as a feature descriptor of the key point to represent the key features of the key point.

2. A method for real-time monitoring of the operating status of a dispensing machine according to claim 1, characterized in that: Match key features with features in predetermined topological data to obtain matching results, including: Traverse each key feature in the current image and calculate the Euclidean distance between the current key feature and the predetermined feature; The Euclidean distance is stored in a distance matrix, where the rows of the matrix represent the key features of the current image and the columns represent the features in the predetermined feature set; For each row in the distance matrix, find the column index corresponding to the Euclidean distance, where the column index indicates the feature in the predetermined feature set that matches the current key feature; According to the column index, matching pairs are established between key features of the current image and features in the predetermined topological data, wherein each matching pair includes a key feature in the current image and a feature matching the key feature in the predetermined feature set.

3. A method for real-time monitoring of the operating status of a dispensing machine according to claim 2, characterized in that: Based on the matching results, the similarity scores between the matching features are calculated, including: Extract the feature vector of the current image from the matching pair and the feature vector of the predetermined topological data ; Assign a weight to the feature of each dimension, let W be the weight vector, then the weighted dot product is calculated as: ,in, It is The weight of the feature, and They are the feature vectors of the current image and the feature vector of the predetermined topological data No. elements; Calculate the weighted modulus length. The weighted modulus length is calculated as: ； ； Divide the weighted dot product by the product of the two weighted moduli to get the similarity score: ； in, represents weighted dot product; represents the interaction term, ; represents the similarity score; The total number of dimensions representing features; represents the deviation term; Indicates the Features and The interaction weights between features; represents the adjustment coefficient; represents the regularization parameter; represents the base of natural logarithms; is the feature vector of the current image No. elements; and Represents an index.

4. A real-time monitoring system for the operation status of a dispensing machine, characterized in that: Applied to the method according to any one of claims 1 to 3, comprising: The search module maps the camera's parameter space to a corresponding search space. Each ant searches the solution space based on its own position and movement rules. Each ant selects its next position based on the pheromone concentration and heuristic information at its current position. At each new position, the camera's image capture quality is evaluated using an evaluation function. A processing module is configured to update the global pheromone distribution according to the performance of each ant after all ants complete a search, and perform multiple iterations until a preset number of iterations is reached to obtain the final configured camera parameters, so that the camera can capture the glue dot image formed on the substrate according to the final configured camera parameters; The matching module is used to extract key features from the dispensing image and match the key features with features in the predetermined topological data to obtain a matching result; based on the matching result, a similarity score between the matching features is calculated, and based on the similarity score, the operating status of the dispensing machine is classified to obtain a classification result; The prediction module is used to save the classification results together with the corresponding dispensing images, key features and similarity scores as a dataset; use the dataset to train the prediction model to obtain a trained prediction model; and predict the real-time operating status of the dispensing machine based on the trained prediction model and the real-time captured dispensing images.

5. A computing device, characterized in that include: one or more processors; A storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a program, which implements the method according to any one of claims 1 to 3 when executed by a processor.

Images