Real-time monitoring method and system for operation state of dispensing machine

Through the ant colony algorithm optimization of camera parameters and feature matching technology, high accuracy monitoring and prediction of the operating status of the dispenser is achieved, and the problem of insufficient accuracy and generalization capabilities in traditional methods is solved, and the monitoring flexibility and production are improved. efficiency.

CN119091359AActive Publication Date: 2024-12-06SHENZHEN JUHENG INTELLIGENT CO LTD
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202411456828.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-18
Publication Date
2024-12-06
Estimated Expiration
2044-10-18

AI Technical Summary

Technical Problem

Traditional threshold judgment methods have problems with insufficient accuracy and generalization capabilities in the prediction of the operating status of the dispenser, especially the failure to fully consider the interactions and dynamic changes between multiple features.

Method used

The ant colony algorithm is used to optimize the camera parameter configuration, and the ants search in the solution space, and finally configure the camera parameters to capture high-quality dispensing images, thereby extracting key features and matching the predetermined topological data, calculating the similarity score, and performing the dispensing machine operation Real-time monitoring and prediction of status.

Benefits of technology

Improve the accuracy and reliability of the operating status monitoring of the dispenser, can adapt to different working conditions and equipment status, reduce misjudgment and misjudgment, enhance the prediction performance and generalization capabilities of the model, promptly detect operation abnormalities, and avoid production. ACCIDENT.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091359A_ABST
    Figure CN119091359A_ABST
Patent Text Reader

Abstract

The invention provides a real-time monitoring method and system for the running state of a dispensing machine, and relates to the technical field of data processing.The method comprises the steps that after all ants complete one-time searching, global pheromone distribution is updated according to respective corresponding expressions, multiple iterations are conducted till the preset number of iterations is reached, and finally configured camera parameters are obtained; enabling the camera to capture an adhesive dispensing image formed by the adhesive point on the substrate according to the finally configured camera parameters; extracting key features from the dispensing image, and matching the key features with features in predetermined topological data to obtain a matching result; according to the matching result, calculating a similarity score between the matching features, and according to the similarity score, classifying the operation state of the dispenser to obtain a classification result; and storing the classification result, the corresponding dispensing image, the key feature and the similarity score as a data set. The method can improve the prediction precision and reliability.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and system for real-time monitoring of the operating status of a dispensing machine. Background Art

[0002] In terms of predicting the operating status of the dispensing machine, the threshold judgment method in traditional technology is simple and easy to use, but it does have some limitations and challenges that may affect its accuracy and generalization ability. Specifically: First, the threshold judgment method usually focuses on the evaluation of a single feature and may fail to fully consider the interaction and correlation between multiple features. In practical applications, the operating status of a dispensing machine is often affected by multiple features, and the correlation between these features is crucial for accurately evaluating the operating status. Therefore, the threshold judgment that relies only on a single feature may not fully reflect the true status of the dispensing machine.

[0003] Secondly, the dispensing machine may be affected by various factors during operation, causing its state to change dynamically. The threshold judgment method is based on a fixed threshold and may not be able to flexibly respond to such changes. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for real-time monitoring of the operating status of a dispensing machine, which can improve the prediction accuracy and reliability.

[0005] In order to solve the above technical problems, the technical solution of the present invention is as follows: In a first aspect, a method for real-time monitoring of the operating status of a dispensing machine is provided, the method comprising: Map the camera parameter space to the corresponding search space, and each ant will search in the solution space according to its own position and movement rules; Each ant selects the next position based on the pheromone concentration and heuristic information at the current position. At each new position, the image capture quality of the camera is evaluated by the evaluation function; After all ants complete a search, they update the global pheromone distribution according to their respective performances, and perform multiple iterations until a preset number of iterations is reached to obtain the final configured camera parameters, so that the camera can capture the glue dot image formed by the glue dot on the substrate according to the final configured camera parameters; Extract key features from the dispensing image, and match the key features with features in the predetermined topological data to obtain a matching result; According to the matching results, the similarity scores between the matching features are calculated, and the operating states of the dispensing machines are classified according to the similarity scores to obtain the classification results; The classification results, together with the corresponding dispensing images, key features, and similarity scores are saved as a dataset; Using the data set to train the prediction model to obtain a trained prediction model; Based on the trained prediction model and the real-time captured dispensing images, the real-time operating status of the dispensing machine is predicted.

[0006] Furthermore, each ant represents a set of configured camera parameters.

[0007] Furthermore, each ant selects the next location based on the pheromone concentration and heuristic information at the current location, including: For each ant, traverse all adjacent positions of the current position, and for each adjacent position, calculate its probability, where the probability calculation formula is: ; in, Indicates that the ant is moving from its current position Move to adjacent location The probability of selection; Indicates from position To location pheromone concentration; Indicates from position To location pheromone concentration; and Represents heuristic information; , and represents weight; and Represents environmental factors; Indicates current location The set of all adjacent positions of , where , It's location how often it was selected in the past iterations, is the maximum selection frequency among all adjacent positions; , It's location How often it was selected in past iterations; Indicates the current location of the ant; Indicates that the ant is moving from its current position The next position selected; Indicates the current location A specific position of all adjacent positions; , Indicates from position To location Power consumption; represents the maximum value of the environmental condition metric among all adjacent paths; , Indicates from position To location Power consumption; Based on the probability, an adjacent position is selected as the next moving target of the ant.

[0008] Furthermore, the calculation formula of the evaluation function is: ; in, Indicates at location Image capture quality rating on ; Indicates at location Image resolution on Indicates at location Contrast on Indicates at location Color saturation on Represents the coefficient.

[0009] Furthermore, key features are extracted from the dispensing images, including: Preprocessing the captured dispensing image to obtain a preprocessed image; The Sobel operator is used to calculate the gradient intensity and direction of each pixel in the preprocessed image. The gradient intensity indicates the possibility of the edge at the pixel, and the gradient direction indicates the direction of the edge. Along the gradient direction, compare the gradient strength of the current pixel with the gradient strength of its adjacent pixels; Pixels with gradient strength higher than the first threshold are marked as confirmed edges, and pixels with gradient strength lower than the second threshold are discarded; for pixels with gradient strength between the first threshold and the second threshold, if they are connected to the confirmed edge, they are regarded as edges, otherwise they are discarded; According to the edge, the contour is searched on the binary edge image, and the contour representing the dispensing point is screened out; According to the outline of the dispensing point, the key points are detected on the pre-processed image; The area around the key point is divided into several sub-areas, and for each pixel in the sub-area, its gradient direction and gradient amplitude are calculated; According to the gradient direction and gradient magnitude, a gradient histogram is constructed for each sub-region, and the gradient histograms of all sub-regions are concatenated to form a long vector, which is used as a feature descriptor of the key point to represent the key features of the key point.

[0010] Further, the key features are matched with features in the predetermined topological data to obtain a matching result, including: Traverse each key feature in the current image and calculate the Euclidean distance between the current key feature and the predetermined feature; The Euclidean distance is stored in a distance matrix, where the rows of the matrix represent the key features of the current image and the columns represent the features in the predetermined feature set; For each row in the distance matrix, find the column index corresponding to the Euclidean distance, where the column index indicates the feature in the predetermined feature set that matches the current key feature; According to the column index, a matching pair is established between the key features of the current image and the features in the predetermined topological data, wherein each matching pair includes a key feature in the current image and a feature matching the key feature in the predetermined feature set.

[0011] Furthermore, based on the matching results, the similarity scores between the matching features are calculated, including: Extract the feature vector of the current image from the matching pair and the feature vector of the predetermined topological data ; Assign a weight to the feature of each dimension, let W be the weight vector, then the weighted dot product is calculated as: ,in, It is The weight of the feature, and They are the feature vectors of the current image. and the feature vector of the predetermined topological data No. elements; Calculate the weighted modulus length. The weighted modulus length is calculated as: ; ; Divide the weighted dot product by the product of the two weighted moduli to get the similarity score: ; in, represents weighted dot product; represents the interaction term, ; represents the similarity score; The total number of dimensions representing features; represents the deviation term; Indicates Features and The interaction weights between features; represents the adjustment coefficient; represents the regularization parameter; represents the base of natural logarithms; is the feature vector of the current image No. elements; and Represents an index.

[0012] In a second aspect, a real-time monitoring system for the operation status of a dispensing machine is used in the method described, comprising: The search module is used to map the parameter space of the camera to the corresponding search space. Each ant will search in the solution space according to its own position and movement rules. Each ant selects the next position according to the pheromone concentration and heuristic information of the current position. At each new position, the image capture quality of the camera is evaluated by the evaluation function. A processing module is used to update the global pheromone distribution according to the corresponding performance of each ant after all ants complete a search, and perform multiple iterations until a preset number of iterations is reached to obtain the camera parameters of the final configuration, so that the camera can capture the glue dot image formed by the glue dot on the substrate according to the camera parameters of the final configuration; A matching module is used to extract key features from the dispensing image and match the key features with features in the predetermined topological data to obtain a matching result; based on the matching result, a similarity score between the matching features is calculated, and based on the similarity score, the operating status of the dispensing machine is classified to obtain a classification result; The prediction module is used to save the classification results together with the corresponding dispensing images, key features and similarity scores as a data set; use the data set to train the prediction model to obtain a trained prediction model; and predict the real-time operating status of the dispensing machine based on the trained prediction model and the real-time captured dispensing images.

[0013] According to a third aspect, a computing device includes: one or more processors; The storage device is used to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the method described.

[0014] In a fourth aspect, a computer-readable storage medium stores a program, and when the program is executed by a processor, the method described is implemented.

[0015] The above solution of the present invention includes at least the following beneficial effects: By using the ant colony algorithm to optimize the camera parameter configuration, this method can capture high-quality dispensing images, thereby effectively improving the accuracy and reliability of monitoring data. At the same time, combined with key feature extraction and matching technology, it can more accurately identify the operating status of the dispensing machine and reduce misjudgment and missed judgment.

[0016] This method can capture dispensing images in real time and quickly predict the operating status of the dispensing machine based on the trained prediction model. This enables operators to promptly detect potential operating anomalies or failures and take appropriate measures to intervene, thereby avoiding production accidents and improving production efficiency.

[0017] By optimizing the camera parameters and updating the global pheromone distribution through multiple iterations, this method can adaptively adjust the monitoring strategy to adapt to the operation of the dispensing machine under different working conditions and equipment states. This adaptive capability makes the method more versatile and flexible, and can be widely used in various dispensing machine operation status monitoring scenarios.

[0018] By saving the classification results together with the corresponding dispensing images, key features and similarity scores as a dataset and using the dataset to train the prediction model, this method can continuously enrich and optimize the learning resources of the model. This helps to improve the prediction performance and generalization ability of the model, enabling it to make more accurate predictions when faced with new dispensing images. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a flow chart of a method for real-time monitoring of the operating status of a dispensing machine provided in an embodiment of the present invention.

[0020] Figure 2 It is a schematic diagram of a real-time monitoring system for the operating status of a dispensing machine provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0021] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in a form and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0022] like Figure 1 As shown, an embodiment of the present invention provides a method for real-time monitoring of the operating status of a dispensing machine, the method comprising the following steps: Step 11, map the camera parameter space to the corresponding search space, each ant will search in the solution space according to its own position and movement rules, and each ant represents a set of configured camera parameters; Step 12, each ant selects the next position according to the pheromone concentration and heuristic information of the current position, and at each new position, the image capture quality of the camera is evaluated by the evaluation function; Step 13, after all ants complete a search, the global pheromone distribution is updated according to their respective corresponding performances, and multiple iterations are performed until a preset number of iterations is reached to obtain the final configured camera parameters, so that the camera can capture the glue dot image formed by the glue dot on the substrate according to the final configured camera parameters; Step 14, extracting key features from the dispensing image, and matching the key features with features in the predetermined topological data to obtain a matching result; Step 15, according to the matching results, calculating the similarity scores between the matching features, and classifying the operating states of the dispensing machines according to the similarity scores to obtain classification results; Step 16, saving the classification results together with the corresponding dispensing images, key features and similarity scores as a data set; Step 17, using the data set to train the prediction model to obtain a trained prediction model; Step 18, predicting the real-time operating status of the dispensing machine based on the trained prediction model and the real-time captured dispensing images.

[0023] In an embodiment of the present invention, by using an ant algorithm to search for the optimal camera configuration in the parameter space, the method can efficiently find the optimal parameter settings for capturing dispensing images. This not only improves the efficiency of camera configuration, but also ensures the quality of the captured images. The method can accurately identify key information in the dispensing process by extracting key features from the dispensing images and matching them with features in predetermined topological data. This feature-based matching method improves the accuracy of state monitoring and makes the evaluation of the operating state of the dispensing machine more accurate. By calculating the similarity scores between matching features and classifying the operating state of the dispensing machine according to the scores, the method realizes intelligent discrimination of the state of the dispensing machine. This intelligent state classification helps to detect abnormal situations in a timely manner and take corresponding measures to intervene, thereby ensuring the stability and reliability of the dispensing process. The method can not only classify and save historical data, but also use these data sets to train prediction models to predict the real-time operating state of the dispensing machine. This prediction capability enables operators to take preventive measures before problems occur, further improving the maintenance efficiency and production benefits of the dispensing machine. This method realizes a data-driven decision-making process by saving the classification results, dispensing images, key features, and similarity scores as a dataset and using them to train a prediction model.

[0024] In a preferred embodiment of the present invention, in step 11, the parameter space of the camera is mapped to a corresponding search space, and each ant searches in the solution space according to its own position and movement rules. Each ant represents a set of configured camera parameters, which may include: Step 111, determine the parameter space of the camera, which includes the focal length, exposure time, white balance, contrast, brightness and other adjustable parameters of the camera. Next, set a reasonable value range for each parameter, map the value ranges of these parameters to a multi-dimensional search space, each dimension corresponds to a camera parameter, set the number of ants, and randomly assign an initial position in the search space to each ant. This position represents a set of initial camera parameter configurations.

[0025] In a preferred embodiment of the present invention, the above step 12, where each ant selects the next location according to the pheromone concentration and heuristic information at the current location, may include: Step 121, for each ant, traverse all adjacent positions of the current position, and for each adjacent position, calculate its probability, wherein the probability calculation formula is: ; in, Indicates that the ant is moving from its current position Move to adjacent location The probability of selection; Indicates from position To location pheromone concentration; Indicates from position To location pheromone concentration; and Represents heuristic information; , and represents weight; and Represents environmental factors; Indicates current location The set of all adjacent positions of , where , It's location how often it was selected in the past iterations, is the maximum selection frequency among all adjacent positions; , It's location How often it was selected in past iterations; Indicates the current location of the ant; Indicates that the ant is moving from its current position The next position selected; Indicates the current location A specific position of all adjacent positions; , Indicates from position To location Power consumption; represents the maximum value of the environmental condition metric among all adjacent paths; , Indicates from position To location Power consumption; Step 122, based on the probability, select an adjacent position as the next moving target of the ant, specifically including: based on the selection probability of each adjacent position calculated in step 121 , use roulette wheel selection to select the next moving target; in roulette wheel selection, the probability of each adjacent position being selected is proportional to its selection probability, which means that a higher selection probability means a greater chance of being selected. Once the next moving target (i.e., adjacent position) is selected ), the ant is moved from the current position i to the target position and the position information of the ant is updated for use in subsequent search iterations.

[0026] In the embodiment of the present invention, by comprehensively considering the pheromone concentration, heuristic information and environmental factors, the method can more accurately guide the ants to move to the potential high-quality solution area, thereby reducing unnecessary searches and improving search efficiency. Introducing the selection frequency in the past iteration as a regulating factor can effectively prevent ants from gathering near the local optimal solution too early, thereby increasing the possibility of finding the global optimal solution. The introduction of environmental factors enables the algorithm to be adjusted according to specific conditions (such as power consumption) in actual application scenarios, enhancing the adaptability and flexibility of the algorithm. The method calculates the selection probability by integrating multiple information, taking into account both known information (such as pheromone concentration and heuristic information) and the exploration of unknown areas (by reducing the probability of frequently selecting positions), achieving a good balance between exploration and utilization. Through the meticulous probability calculation process, each ant can more wisely choose the next moving position, which helps to improve the quality of the final solution found, making the camera parameter configuration more accurate and optimized.

[0027] In a preferred embodiment of the present invention, the calculation formula of the evaluation function is: ; in, Indicates at location Image capture quality rating on ; Indicates at location Image resolution on Indicates at location Contrast on Indicates at location Color saturation on Represents the coefficient.

[0028] In the embodiment of the present invention, the function comprehensively considers the image resolution , Contrast and color saturation Three key indicators can more comprehensively reflect the quality of the image; by introducing coefficients , this function can adjust the weights of various indicators according to actual needs, making the evaluation more flexible.

[0029] Specific calculation process: First, get the Image resolution on , Contrast and color saturation According to the given coefficient , calculate the weights of image resolution, contrast and color saturation respectively. The weight calculation formula is as follows: Weight of resolution: ; Contrast weight: ; Weight of color saturation: ; These weights reflect the importance of each indicator in the overall evaluation. The original values ​​of each indicator are weighted averaged using the above weights to obtain the position Image capture quality rating on , which takes into account resolution, contrast, and color saturation, and is based on a coefficient The contribution of each indicator was adjusted to obtain Finally, the image capture quality at different locations can be compared and analyzed, and this score can be used to guide the setting and optimization of image acquisition equipment.

[0030] In a preferred embodiment of the present invention, the above step 13 is specifically implemented as follows: Set an iteration counter iteration_count and initialize it to 0, set the preset number of iterations max_iterations; enter the iteration loop, repeat the following steps until iteration_count reaches max_iterations; for each ant, perform the search process (as described in step 12), let the ant select the next position according to the pheromone concentration, heuristic information, etc. at the current position, until a complete search path is completed; when all ants have completed a search, calculate the score of each position on each ant's search path according to the evaluation function (such as the image capture quality scoring formula mentioned above), and record the total score or average score of each ant's search path as the performance of the ant's search; The global pheromone distribution is updated based on the performance of each ant. Ants with good performance will leave more pheromones on their search paths to attract subsequent ants. Different strategies can be used to update pheromones, such as increasing the pheromone concentration on paths with excellent performance while reducing the pheromone concentration on paths with poor performance, or considering the volatilization of pheromones.

[0031] Increase the iteration counter iteration_count by 1. If iteration_count reaches max_iterations, terminate the iteration loop. Otherwise, return to the next iteration. After the iteration, determine the final camera parameter configuration based on the global pheromone distribution or other criteria (such as selecting the configuration with the highest score), and output or save the camera parameters of the final configuration.

[0032] Through the above steps, we can use the swarm intelligence of ants to perform multiple iterative searches, continuously optimize the global pheromone distribution, and finally obtain a set of camera parameter configurations with excellent performance. This method combines the heuristic search and pheromone update mechanisms, which helps to find high-quality solutions in complex parameter spaces.

[0033] In a preferred embodiment of the present invention, the above step 14, extracting key features from the dispensing image, may include: Step 141, preprocessing the captured dispensing image to obtain a preprocessed image; using the Sobel operator to calculate the gradient strength and direction of each pixel in the preprocessed image, the gradient strength represents the possibility of the edge at the pixel, and the gradient direction represents the direction of the edge, specifically including: The captured dispensing image is grayed out and converted into a grayscale image to reduce the amount of calculation for subsequent processing; the grayscale image is denoised, such as using median filtering or Gaussian filtering to eliminate noise in the image and improve the accuracy of edge detection; the pre-processed grayscale image is convolved using the Sobel operator to calculate the pixel density of each pixel in the grayscale image. x Direction and y The gradient strength in the direction ( G x and G y );according to G x and G y , calculate the total gradient strength of each pixel ( G ) and the gradient direction (θ). The total gradient strength G It can be calculated by the Pythagorean theorem, and the gradient direction θ can be calculated by the inverse tangent function.

[0034] Step 142, along the gradient direction, compare the gradient strength of the current pixel with the gradient strength of its adjacent pixel; mark the pixel with a gradient strength higher than a first threshold as a determined edge, and discard the pixel with a gradient strength lower than a second threshold; for the pixel whose gradient strength is between the two first thresholds and the second threshold, if it is connected to the determined edge, it is regarded as an edge, otherwise it is discarded, specifically including: Set two thresholds: the first threshold (high threshold) and the second threshold (low threshold). These thresholds are used to distinguish strong edge, weak edge and non-edge pixels. Traverse each pixel in the image and make edge judgment based on its gradient strength G: If G is higher than the first threshold, the pixel is marked as a confirmed edge; If G is lower than the second threshold, the pixel is discarded, that is, it is considered not to be an edge; If G lies between the two thresholds, it is tentatively marked as a potential edge requiring further processing.

[0035] For pixels marked as potential edges, check whether they are connected to pixels marked as definite edges: If it is connected, it is considered an edge, as it may be part of a real edge; If it is not connected, it is discarded as it is probably a spurious edge caused by noise.

[0036] Step 143, based on the edge, searching for contours on the binary edge image, and screening out contours representing the dispensing points; based on the contours of the dispensing points, detecting key points on the preprocessed image, specifically includes: Call the findContours function in the OpenCV library, which can detect continuous edge contours in binary images; the findContours function will return a list of contours, each of which consists of a series of points and represents a continuous edge in the image. Contour screening: a. Traverse the contour list obtained in the previous step, and for each contour, calculate its area and perimeter. The area can be obtained by counting the number of pixels inside the contour (or directly calculating it using the contourArea function), and the perimeter can be obtained by accumulating the distances between adjacent points on the contour.

[0037] b. Set appropriate area thresholds and perimeter thresholds based on the actual size and shape characteristics of the dispensing dots. These thresholds are used to filter out contours that represent dispensing dots. For example, you can set an area threshold to exclude contours that are too small or too large, and set a perimeter threshold to exclude contours with irregular shapes.

[0038] c. The contours that meet the area and perimeter threshold conditions are retained as candidate contours representing the dispensing points.

[0039] For each selected dispensing point candidate contour, calculate its centroid as the key point. The centroid can be obtained by averaging the coordinates of all contour points, that is, calculating the average of the x coordinate and y coordinate respectively.

[0040] Step 144, divide the area around the key point into several sub-areas, and for the pixels in each sub-area, calculate the gradient direction and gradient magnitude; according to the gradient direction and gradient magnitude, construct a gradient histogram for each sub-area, and concatenate the gradient histograms of all sub-areas to form a long vector, which is used as a feature descriptor of the key point to represent the key features of the key point. The specific implementation process is as follows: For each detected key point, a neighborhood area is determined with it as the center. This area is of fixed size (for example, 32×32 pixels). If an adaptive neighborhood area is used, the size of the area can be determined based on the scale information obtained in the key point detection stage (such as the scale factor in the SIFT algorithm). Sub-area division: a. Further divide the neighborhood area into several sub-areas, using a grid-like division, such as a 4×4 or 8×8 grid. In this way, each sub-area will contain a portion of the pixels in the neighborhood.

[0041] Gradient histogram construction: a. For each pixel in the sub-region, calculate its gradient direction and gradient magnitude, which can be done by the Sobel operator or other gradient calculation methods used in the previous step.

[0042] b. Create a gradient histogram that uses multiple bins (such as 8 or 16) to quantify the range of gradient directions. Each bin corresponds to a specific range of gradient directions, where a bin refers to a small interval or unit in the gradient histogram used to quantify the range of gradient directions.

[0043] c. Traverse each pixel in the sub-region and add the gradient magnitude to the corresponding bin according to its gradient direction. In this way, the value in each bin will represent the sum of the gradient magnitudes in that gradient direction.

[0044] d. Perform weighted processing on the gradient magnitude, for example, using a Gaussian function to reduce the weight of pixels far from the key point to emphasize the features near the key point. Feature descriptor generation: a. Concatenate the gradient histograms of all sub-regions to form a long vector. This vector contains detailed gradient information of the area around the key point.

[0045] b. Optionally, normalize the feature descriptors to eliminate the effects of illumination changes on the feature description. Normalization can be achieved by dividing the bin value of each gradient histogram by the sum of all bin values.

[0046] c. The final long vector is the feature descriptor of the key point, which can be used for subsequent feature matching, target tracking or recognition tasks.

[0047] In an embodiment of the present invention, by using the Sobel operator to calculate the gradient intensity and direction of the pixel point, the edge in the image can be accurately detected. This edge detection method is crucial for identifying the contour of the dispensing point because it can distinguish the true edge from the background noise, thereby improving the accuracy of edge detection. The dual threshold (first threshold and second threshold) processing mechanism is adopted, so that the algorithm can flexibly process the pixel points whose gradient intensity is between the two thresholds while marking the determined edge. This processing method enhances the robustness of the algorithm and can stably extract key features from images of different qualities. According to the edge detection result, the contour is searched on the binary edge image, and the contour representing the dispensing point is screened out. This process can quickly locate the area of ​​interest, that is, the dispensing point, thereby improving the efficiency of subsequent key point detection and feature extraction. According to the contour of the dispensing point, the key point is detected on the preprocessed image, and this contour-based key point detection method can ensure the accurate positioning of the key point. By dividing the area around the key point into several sub-areas and constructing a gradient histogram for each sub-area, a long vector is finally formed as the feature descriptor of the key point. This description method not only takes into account the local gradient information of key points, but also enhances the richness and discrimination of features by splicing the histograms of multiple sub-regions, making the extracted key features more representative and robust.

[0048] In a preferred embodiment of the present invention, the above step 14, matching the key features with the features in the predetermined topological data to obtain a matching result, further includes: Step 145, traverse each key feature in the current image, and calculate the Euclidean distance between the current key feature and the predetermined feature. The specific implementation process is as follows: Prepare the key feature set extracted from the current image and the predetermined feature set. Each feature is usually represented by a feature vector, which contains information describing the feature (such as features extracted by algorithms such as SIFT and SURF). Use a loop structure to traverse each key feature in the current image. For each key feature, perform the following operations: a. Extract the feature vector of the key feature currently being processed from the key feature set of the current image.

[0049] b. For each feature in the predetermined feature set, calculate the Euclidean distance between the feature vector of the current key feature and the feature vector of the predetermined feature.

[0050] Step 146, storing the Euclidean distance in a distance matrix, wherein the rows of the matrix represent the key features of the current image, and the columns represent the features in the predetermined feature set. The specific implementation process is as follows: Based on the number of key features of the current image and the number of features of the predetermined feature set, a two-dimensional array (i.e., a matrix) is created to store the Euclidean distance values. The number of rows of the matrix is ​​equal to the number of key features of the current image, and the number of columns is equal to the number of features of the predetermined feature set. The distance matrix is ​​filled with the Euclidean distance values ​​calculated in step 145. Each distance value is placed in a corresponding position of the matrix, where the row index corresponds to the key feature in the current image and the column index corresponds to the feature in the predetermined feature set.

[0051] Step 147, for each row in the distance matrix, find the column index corresponding to the Euclidean distance, the column index indicates the feature in the predetermined feature set that matches the current key feature, and the specific implementation process is as follows: For each row in the distance matrix (representing a key feature in the current image), do the following: a. Find the minimum Euclidean distance value in the row by traversing all elements of the row and comparing their sizes.

[0052] b. Record the column index corresponding to the minimum distance value found. This column index indicates the feature in the predetermined feature set that best matches the current key feature.

[0053] c. Store the found column indices in the array or list prepared during initialization.

[0054] Step 148, according to the column index, establish matching pairs between the key features of the current image and the features in the predetermined topological data, wherein each matching pair includes a key feature in the current image and a feature matching the key feature in the predetermined feature set, and the specific implementation process is as follows: Create an empty list to store the matching pair information. For each column index recorded in step 147, perform the following operations: a. Get the key features from the key feature set of the current image using the row index (corresponding to the column index).

[0055] b. Use the column index to obtain features that match the current key feature from the predetermined feature set.

[0056] c. Combine the key features of the current image and the matching features in the predetermined feature set into a matching pair. This matching pair contains two mutually matching features.

[0057] d. Add the created matching pairs to the matching pair list prepared during initialization. After all matching pairs are created, return a list containing all matching pairs as the final matching result.

[0058] In an embodiment of the present invention, by calculating the Euclidean distance between each key feature in the current image and the predetermined feature, the similarity between them can be accurately measured. Using a distance matrix to store the Euclidean distance can easily retrieve and compare the similarity between different features. The rows of the matrix represent the key features of the current image, and the columns represent the features in the predetermined feature set. This structure makes data access and operation efficient. In the matching process, only the minimum distance value in the distance matrix and its corresponding column index need to be found to quickly find the feature that best matches the current key feature. This method can be easily extended to process large-scale feature sets. The size of the distance matrix is ​​proportional to the number of key features and predetermined feature sets in the current image, so any number of features can be easily processed. In addition, the processing speed can be further improved by using an efficient matrix operation library or parallel computing technology. Since the Euclidean distance is used as the similarity metric, this method has a certain robustness to scale changes and rotation changes between features. The Euclidean distance can capture the spatial position relationship between features, thereby tolerating these changes to a certain extent.

[0059] In a preferred embodiment of the present invention, the above step 15, calculating the similarity score between the matching features according to the matching results, may include: Extract the feature vector of the current image from the matching pair and the feature vector of the predetermined topological data ; Assign a weight to the feature of each dimension, let W be the weight vector, then the weighted dot product is calculated as: ,in, It is The weight of the feature, and They are the feature vectors of the current image. and the feature vector of the predetermined topological data No. elements; Calculate the weighted modulus length. The weighted modulus length is calculated as: ; ; Divide the weighted dot product by the product of the two weighted moduli to get the similarity score: ; in, represents weighted dot product; represents the interaction term, ; represents the similarity score; The total number of dimensions representing features; represents the deviation term; Indicates Features and The interaction weights between features; represents the adjustment coefficient; represents the regularization parameter; represents the base of natural logarithms; is the feature vector of the current image No. elements; and Represents an index.

[0060] In an embodiment of the present invention, through the weight vector W, the method can assign different importance to different feature dimensions, thereby achieving a more refined similarity evaluation. This weighted processing allows the algorithm to emphasize or weaken the influence of certain features in specific application scenarios, thereby improving the flexibility and accuracy of similarity calculation. The calculation method of the weighted modulus increases the sensitivity to changes in eigenvalues ​​by introducing exponential functions and weight factors, so that changes in similarity can still be accurately reflected when there are slight differences in eigenvalues. This design enhances the robustness of the algorithm, allowing it to maintain stable performance in the face of noise or data disturbances. The calculation of the similarity score CS not only takes into account the weighted dot product, but also performs normalization by dividing by the product of two weighted moduli, further eliminating the influence of feature vector scale differences on similarity calculation. At the same time, by subtracting the sum of squares of the weight vector, the algorithm penalizes the weight distribution, avoiding over-reliance on a specific feature, thereby obtaining a more comprehensive and balanced similarity measure.

[0061] In a preferred embodiment of the present invention, the operating states of the dispensing machines are classified according to the similarity scores to obtain classification results, which may include: Collect historical data of the dispensing machine under different operating conditions, including feature data and corresponding similarity scores of normal operation, abnormal operation and other conditions; determine the similarity score thresholds between different operating conditions by analyzing these historical data.

[0062] For the new dispensing machine operation data, its feature vector is first extracted, and the similarity score calculation method described previously (step 15) is used to calculate the similarity score between the new data and various operation status template data.

[0063] The similarity score of the new data is compared with the determined threshold. If the similarity score is higher than the threshold of the normal operating state, the dispensing machine is judged to be in a normal operating state. If the similarity score is lower than the threshold of a certain abnormal operating state, the dispensing machine is judged to be in the abnormal operating state, and the classification result is recorded in a log or database.

[0064] In a preferred embodiment of the present invention, the above step 16, saving the classification results together with the corresponding dispensing images, key features and similarity scores as a data set, may include: Organize the classification results, key features, and similarity scores of each sample into a structured data format, such as a CSV file or database record. For dispensing images, you can save them as image files (such as JPEG, PNG, etc.), and make sure the file name or path corresponds to the record in the structured data. Create a new dataset directory to store all relevant data, and save the organized structured data as part of the dataset, such as data.csv. Save the dispensing image file in another subdirectory of the dataset, such as images.

[0065] In a preferred embodiment of the present invention, step 17, using the data set to train the prediction model to obtain a trained prediction model, may include: According to the complexity of the problem and the size of the data set, design a CNN network structure, including determining the number and parameter settings of convolutional layers, pooling layers, and fully connected layers; select activation functions for each layer of CNN, such as ReLU, Sigmoid, etc., to increase the nonlinear expression ability of the network; according to the task type (such as classification or regression), select a suitable loss function, such as the cross entropy loss function. At the same time, select an optimizer to update the network weights, such as Adam or SGD.

[0066] Use a data loading library (such as PyTorch's DataLoader or TensorFlow's tf.data) to load the dataset; normalize the image data and scale the pixel values ​​to between 0 and 1; standardize the data so that the distribution has zero mean and unit variance; in order to increase the generalization ability of the model, the images can be enhanced by rotation, translation, scaling, etc.; resize all images to a uniform size to meet the input requirements of the CNN.

[0067] Use a random method to divide the data set into a training set, a validation set, and a test set, with a ratio of 70%: 15%: 15% or 60%: 20%: 20%; initialize the model parameters according to the designed CNN structure; set training parameters such as learning rate, batch size, and number of iterations; in each iteration, randomly extract a batch of data from the training set, calculate the loss through forward propagation, and then use backpropagation and optimizer to update the model parameters. Record the loss value and accuracy of each iteration; during the training process, use the validation set to regularly evaluate the performance of the model and record the best model weights. If the performance of the model on the validation set is found to decline or there are signs of overfitting, corresponding measures can be taken, such as adjusting the learning rate, adding regularization terms, etc.; after the training is completed, use the test set to evaluate the model, and calculate indicators such as test accuracy, recall rate, F1 score, etc. to comprehensively evaluate the performance of the model and obtain the trained prediction model.

[0068] In a preferred embodiment of the present invention, step 18, predicting the real-time operating status of the dispensing machine based on the trained prediction model and the real-time captured dispensing image, may include: Set up the dispensing machine to capture real-time image data of the dispensing process; perform necessary preprocessing on the captured real-time dispensing images, such as denoising and contrast enhancement; apply the same feature extraction algorithm as in the training phase to extract key feature vectors from the preprocessed images; use an appropriate machine learning library (such as TensorFlow, PyTorch, etc.) to load the previously trained and saved prediction model file; input the extracted and preprocessed key feature vectors into the loaded prediction model; calculate the current operating status prediction result of the dispensing machine through the model's reasoning process. This is a classification task, and the model will output the probability distribution of each possible state; according to the output of the model, parse and determine the real-time operating status of the dispensing machine. You can choose the state with the highest probability as the prediction result, or set a threshold to judge the state according to specific needs.

[0069] like Figure 2 As shown, an embodiment of the present invention further provides a dispensing machine operating status real-time monitoring system 20, comprising: The search module 21 is used to map the parameter space of the camera to the corresponding search space. Each ant will search in the solution space according to its own position and movement rules. Each ant selects the next position according to the pheromone concentration and heuristic information of the current position. At each new position, the image capture quality of the camera is evaluated by the evaluation function. The processing module 22 is used to update the global pheromone distribution according to the corresponding performance of each ant after all ants complete a search, and perform multiple iterations until a preset number of iterations is reached to obtain the camera parameters of the final configuration, so that the camera can capture the glue dot image formed by the glue dot on the substrate according to the camera parameters of the final configuration; The matching module 23 is used to extract key features from the dispensing image and match the key features with features in the predetermined topological data to obtain a matching result; based on the matching result, calculate the similarity score between the matching features, and classify the operating status of the dispensing machine based on the similarity score to obtain a classification result; The prediction module 24 is used to save the classification results together with the corresponding dispensing images, key features and similarity scores as a data set; use the data set to train the prediction model to obtain a trained prediction model; and predict the real-time operating status of the dispensing machine based on the trained prediction model and the real-time captured dispensing images.

[0070] It should be noted that the system is a system corresponding to the above method, and all implementation methods in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0071] The embodiment of the present invention further provides a computing device, comprising: a processor, a memory storing a computer program, wherein when the computer program is executed by the processor, the method described above is executed. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0072] The embodiment of the present invention also provides a computer-readable storage medium storing instructions, which, when executed on a computer, enable the computer to execute the method described above. All implementations in the above method embodiment are applicable to this embodiment and can achieve the same technical effect.

[0073] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A real-time monitoring method for the operating status of a dispensing machine. It is characterized in that The method comprises: Map the camera parameter space to the corresponding search space, and each ant will search in the solution space according to its own position and movement rules; Each ant selects the next position based on the pheromone concentration and heuristic information at the current position. At each new position, the image capture quality of the camera is evaluated by the evaluation function; After all ants complete a search, they update the global pheromone distribution according to their respective performances, and perform multiple iterations until a preset number of iterations is reached to obtain the final configured camera parameters, so that the camera can capture the glue dot image formed by the glue dot on the substrate according to the final configured camera parameters; Extract key features from the dispensing image, and match the key features with features in the predetermined topological data to obtain a matching result; According to the matching results, the similarity scores between the matching features are calculated, and the operating states of the dispensing machines are classified according to the similarity scores to obtain the classification results; The classification results, together with the corresponding dispensing images, key features, and similarity scores are saved as a dataset; Using the data set to train the prediction model to obtain a trained prediction model; Based on the trained prediction model and the real-time captured dispensing images, the real-time operating status of the dispensing machine is predicted.

2. A method for real-time monitoring of the operating status of a dispensing machine according to claim 1, It is characterized in that Each ant represents a set of configured camera parameters.

3. A method for real-time monitoring of the operating status of a dispensing machine according to claim 2, It is characterized in that Each ant chooses its next location based on the pheromone concentration at its current location and heuristic information, including: For each ant, traverse all adjacent positions of the current position, and for each adjacent position, calculate its probability, where the probability calculation formula is: ; in, Indicates that the ant is moving from its current position Move to adjacent location The probability of selection; Indicates from position To location pheromone concentration; Indicates from position To location pheromone concentration; and Represents heuristic information; , and represents weight; and Represents environmental factors; Indicates current location The set of all adjacent positions of , where , It's location how often it was selected in the past iterations, is the maximum selection frequency among all adjacent positions; , It's location How often it was selected in past iterations; Indicates the current location of the ant; Indicates that the ant is moving from its current position The next position selected; Indicates the current location A specific position of all adjacent positions; , Indicates from position To location Power consumption; represents the maximum value of the environmental condition metric among all adjacent paths; , Indicates from position To location Power consumption; Based on the probability, an adjacent position is selected as the next moving target of the ant.

4. A method for real-time monitoring of the operating status of a dispensing machine according to claim 3, It is characterized in that The calculation formula of the evaluation function is: ; in, Indicates at location Image capture quality rating on ; Indicates at location Image resolution on Indicates at location Contrast on Indicates at location Color saturation on Represents the coefficient.

5. A method for real-time monitoring of the operating status of a dispensing machine according to claim 4, It is characterized in that Extract key features from dispensing images, including: Preprocessing the captured dispensing image to obtain a preprocessed image; The Sobel operator is used to calculate the gradient intensity and direction of each pixel in the preprocessed image. The gradient intensity indicates the possibility of the edge at the pixel, and the gradient direction indicates the direction of the edge. Along the gradient direction, compare the gradient strength of the current pixel with the gradient strength of its adjacent pixels; Pixels with gradient strength higher than the first threshold are marked as confirmed edges, and pixels with gradient strength lower than the second threshold are discarded; for pixels with gradient strength between the first and second thresholds, if they are connected to the confirmed edge, they are regarded as edges, otherwise they are discarded; According to the edge, the contour is searched on the binary edge image, and the contour representing the dispensing point is screened out; According to the outline of the dispensing point, the key points are detected on the pre-processed image; The area around the key point is divided into several sub-areas, and for each pixel in the sub-area, its gradient direction and gradient amplitude are calculated; According to the gradient direction and gradient magnitude, a gradient histogram is constructed for each sub-region, and the gradient histograms of all sub-regions are concatenated to form a long vector, which is used as a feature descriptor of the key point to represent the key features of the key point.

6. A method for real-time monitoring of the operating status of a dispensing machine according to claim 5, It is characterized in that Match the key features with the features in the predetermined topological data to obtain matching results, including: Traverse each key feature in the current image and calculate the Euclidean distance between the current key feature and the predetermined feature; The Euclidean distance is stored in a distance matrix, where the rows of the matrix represent the key features of the current image and the columns represent the features in the predetermined feature set; For each row in the distance matrix, find the column index corresponding to the Euclidean distance, where the column index indicates the feature in the predetermined feature set that matches the current key feature; According to the column index, a matching pair is established between the key features of the current image and the features in the predetermined topological data, wherein each matching pair includes a key feature in the current image and a feature matching the key feature in the predetermined feature set.

7. A method for real-time monitoring of the operating status of a dispensing machine according to claim 6, It is characterized in that Based on the matching results, the similarity scores between the matching features are calculated, including: Extract the feature vector of the current image from the matching pair and the feature vector of the predetermined topological data ; Assign a weight to the feature of each dimension, let W be the weight vector, then the weighted dot product is calculated as: ,in, It is The weight of the feature, and They are the feature vectors of the current image. and the feature vector of the predetermined topological data No. elements; Calculate the weighted modulus length. The weighted modulus length is calculated as: ; ; Divide the weighted dot product by the product of the two weighted moduli to get the similarity score: ; in, represents weighted dot product; represents the interaction term, ; represents the similarity score; The total number of dimensions representing features; represents the deviation term; Indicates Features and The interaction weights between features; represents the adjustment coefficient; represents the regularization parameter; represents the base of natural logarithms; is the feature vector of the current image No. elements; and Represents an index.

8. A real-time monitoring system for the operation status of a dispensing machine. It is characterized in that Applied to the method according to any one of claims 1 to 7, comprising: The search module is used to map the parameter space of the camera to the corresponding search space. Each ant will search in the solution space according to its own position and movement rules. Each ant selects the next position according to the pheromone concentration and heuristic information of the current position. At each new position, the image capture quality of the camera is evaluated by the evaluation function. A processing module is used to update the global pheromone distribution according to the corresponding performance of each ant after all ants complete a search, and perform multiple iterations until a preset number of iterations is reached to obtain the camera parameters of the final configuration, so that the camera can capture the glue dot image formed by the glue dot on the substrate according to the camera parameters of the final configuration; A matching module is used to extract key features from the dispensing image and match the key features with features in the predetermined topological data to obtain a matching result; based on the matching result, a similarity score between the matching features is calculated, and based on the similarity score, the operating status of the dispensing machine is classified to obtain a classification result; The prediction module is used to save the classification results together with the corresponding dispensing images, key features and similarity scores as a data set; use the data set to train the prediction model to obtain a trained prediction model; and predict the real-time operating status of the dispensing machine based on the trained prediction model and the real-time captured dispensing images.

9. A computing device, It is characterized in that include: one or more processors; A storage device for storing one or more programs, when the one or more programs are executed by the one or more processors, the one or more processors implement the method as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Citation Information

Patent Citations

  • Ant colony algorithm based shooting parameter acquiring method

    CN101686410A

  • Shale reservoir fracture identification method and system cooperating with variant ant body

    CN112444859A

  • Dispensing machine automatic control method and system capable of recognizing dispensing quality

    CN115591742A

  • Running state monitoring method of glue supply system and related device

    CN118758375A