SMT production line monitoring method, system and device and storage medium

Abstract

The invention relates to the technical field of image data processing, and discloses an SMT production line monitoring method, system and device and a storage medium, and the method comprises the steps: obtaining an equipment monitoring video and production line equipment data; performing operation according to the equipment monitoring video to obtain an equipment operation efficiency diagram, a target equipment image and an adjacent equipment image; performing operation according to the equipment operation efficiency diagram to obtain an efficiency exception index; operating the target equipment image and the adjacent equipment image to obtain a first enhanced image and a second enhanced image; integrating the first enhanced image to obtain a running exception graph, and obtaining a running exception index based on the running exception graph; calculating the abnormal point probability of the first enhanced image and the second enhanced image, and calculating the abnormal point probability and the weight of the production equipment to obtain a material change coefficient; calculating the efficiency anomaly index, the operation anomaly index and the material change coefficient to obtain an overall evaluation degree; and judging the operation state according to the overall evaluation degree. The method can comprehensively evaluate the state of the production line.

Description

Technical Field

[0001] The present invention relates to the technical field of image data processing, and in particular, to a monitoring method, system, device and storage medium for an SMT production line. Background Art

[0002] At present, Surface Mounted Technology (SMT) is one of the basic processes in the electronic manufacturing process. In the industrial field, in order to monitor the operation status of the SMT production line and ensure the normal operation of the production line, the operation status of the equipment on the SMT production line is usually monitored to timely detect abnormal operation conditions and take corresponding measures.

[0003] In the prior art, the SMT production line collects the operation data of the equipment through devices such as sensors, combines the historical data and machine learning algorithms, and establishes a prediction model to predict the operation status of the equipment in advance. These methods have certain limitations. For example, the accuracy of the prediction model is limited by the quality and update frequency of the historical data, resulting in a lag in the monitoring of faults or abnormal conditions. In addition, the monitoring indicators are relatively single, and only the operation data is used for monitoring, lacking attention to the equipment status, and it is difficult to comprehensively evaluate the operation status of the equipment.

[0004] In summary, the monitoring method in the prior art only monitors through the operation data, and it is difficult to comprehensively evaluate the operation status of the equipment. Summary of the Invention

[0005] The present invention provides a monitoring method, system, device and storage medium for an SMT production line to solve the problem of difficult comprehensive evaluation of the operation status of the equipment.

[0006] In a first aspect, to solve the above technical problem, the present invention provides a monitoring method for an SMT production line, including: Obtaining an equipment monitoring video and production line equipment data, where the production line equipment data includes a production equipment weight and the total number of equipment; Performing image preprocessing operations on the equipment monitoring video to obtain an equipment operation efficiency map, a target equipment image, and an adjacent equipment image; Performing density clustering operations on the equipment operation efficiency map to obtain abnormal clusters, and obtaining an efficiency anomaly index based on the abnormal density of the abnormal clusters; Performing image enhancement operations on the target equipment image and the adjacent equipment image to obtain a first enhanced image and a second enhanced image; Performing pixel integration on the first enhanced image to obtain an operation anomaly map, and taking the gray mean value of all pixel points in the operation anomaly map as the operation anomaly index; Calculate the outlier probabilities of the first enhanced image and the second enhanced image, and calculate the material change coefficient according to the outlier probabilities and the production equipment weights; Based on a preset overall equipment evaluation model, calculate the efficiency anomaly index, the operation anomaly index, and the material change coefficient to obtain an overall evaluation degree; Evaluate the production line status according to the overall evaluation degree. When the overall evaluation degree is greater than a preset evaluation threshold, it is determined that there is an anomaly in the production line.

[0007] In an alternative implementation, obtain production line equipment data, including: Calculate the production equipment weights by the following method: Wherein, is the production equipment weight of equipment i, is the running time of equipment i within a production cycle, and n is the total number of equipment on the production line.

[0008] In an alternative implementation, perform image preprocessing operations on the equipment monitoring video to obtain an equipment operation efficiency map, a target equipment image, and an adjacent equipment image, including: Perform grayscale processing on the equipment monitoring video to obtain a video grayscale map; Take frames from the video grayscale map at preset time intervals to obtain a target equipment image and an adjacent equipment image; Call the real-time detection data of the corresponding equipment in the video grayscale map; Calculate the real-time detection data as an efficiency impact index to obtain an efficiency evaluation coefficient; Take the mean value of the efficiency evaluation coefficients as the equipment operation efficiency, and construct an equipment operation efficiency map based on the equipment operation efficiency; Calculate the efficiency evaluation coefficient by the following formula: Wherein, E is the efficiency evaluation coefficient, is the th real-time detection data of equipment, is the maximum value among all the real-time detection data of equipment, is the weight of the th real-time detection data of equipment, and n is the total number of equipment on the production line.

[0009] In an alternative implementation, perform density clustering operations on the equipment operation efficiency map to obtain abnormal clusters, and obtain an efficiency anomaly index based on the abnormal density of the abnormal clusters, including: Standardize the operation efficiency graph of the device to obtain a standardized efficiency graph; Perform density clustering operation on the standardized efficiency graph according to the DBSCAN algorithm with a preset density to obtain abnormal clusters; Determine the abnormal density according to the abnormal clusters, and calculate the efficiency anomaly index according to the abnormal density; Among them, the efficiency anomaly index is calculated by the following formula: Among them, is the efficiency anomaly index, is the spatial density of the j-th data point in the i-th abnormal cluster, is the total number of data points in the i-th abnormal cluster, S is the sum of the number of data points in all abnormal clusters, is the total number of abnormal clusters.

[0010] In an alternative embodiment, perform image enhancement operations on the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image, including: Normalize the target device image to obtain a normalized image; Enhance the normalized image using a preset image enhancement method to obtain the first enhanced image and the second enhanced image; Among them, the image enhancement method includes histogram equalization enhancement, filtering enhancement, gray-scale transformation enhancement, and image sharpening.

[0011] In an alternative embodiment, perform pixel integration on the first enhanced image to obtain an operation anomaly map, and use the gray-scale mean value of all pixel points in the operation anomaly map as the operation anomaly index, including: Call the first enhanced image to calculate the gray-scale gradient to obtain a gray-scale gradient matrix; Screen pixel points of the first enhanced image to obtain enhanced pixel points; Integrate the gray-scale gradient matrix and the enhanced pixel points to obtain an operation anomaly map; Calculate the gray-scale mean value of all pixel points in the operation anomaly map to obtain the operation anomaly index; Among them, the operation anomaly index is calculated by the following formula: Among them, represents the operation anomaly index, represents the total number of all pixel points in the operation anomaly map, represents the gray-scale value of the k-th pixel point in the operation anomaly map.

[0012] In an alternative embodiment, calculating the outlier probability of the first enhanced image and the second enhanced image, and calculating the material change coefficient according to the outlier probability and the production equipment weight, includes: Constructing a grayscale histogram of the first enhanced image and the second enhanced image; Performing pixel point screening on the calculated grayscale histogram, taking pixel points with brightness within a preset brightness range as normal pixel points, and taking pixel points with brightness greater than or less than the preset brightness range as outlier pixel points; Calculating the outlier probability of the outlier pixel points among all pixel points; Calculating the material change coefficient according to the outlier probability and the production equipment weight; Wherein, the material change coefficient is calculated by the following formula: Wherein, represents the material change coefficient, is the production equipment weight of the i-th device, is the outlier probability within the neighborhood area in the i-th device, is the total number of devices on the production line.

[0013] In an alternative embodiment, based on a preset overall equipment evaluation model, calculating the efficiency anomaly index, the operation anomaly index, and the material change coefficient to obtain an overall evaluation degree, includes: Calculating the overall evaluation degree through the following overall equipment evaluation model: Wherein, Q is the overall evaluation degree, is the efficiency anomaly index, is the operation anomaly index, is the material change coefficient, is the weight of the efficiency anomaly index, is the weight of the operation anomaly index, is the weight of the material change coefficient.

[0014] In a second aspect, the present invention provides a system for an SMT production line, including: A data acquisition module, configured to acquire equipment monitoring videos and production line equipment data, wherein the production line equipment data includes production equipment weights and the total number of devices; An image processing module, configured to perform image preprocessing operations on the equipment monitoring videos to obtain an equipment operation efficiency map, a target equipment image, and an adjacent equipment image; A density clustering module, configured to perform density clustering operations based on the device operation efficiency diagram to obtain abnormal clusters, and obtain an efficiency anomaly index based on the abnormal density of the abnormal clusters; An image enhancement module, configured to perform image enhancement operations based on the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image; A pixel integration module, configured to perform pixel integration on the first enhanced image to obtain an operation anomaly diagram, and use the gray mean value of all pixel points in the operation anomaly diagram as an operation anomaly index; An abnormal point calculation module, configured to calculate the abnormal point probability of the first enhanced image and the second enhanced image, and calculate a material change coefficient based on the abnormal point probability and the production equipment weight; An evaluation degree calculation module, configured to perform calculations on the efficiency anomaly index, the operation anomaly index, and the material change coefficient based on a preset overall device evaluation model to obtain an overall evaluation degree; A status evaluation module, configured to perform production line status evaluation based on the overall evaluation degree, and determine that there is an abnormality in the production line when the overall evaluation degree is greater than a preset evaluation threshold.

[0015] In a third aspect, the present invention further provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the monitoring method of the SMT production line described in any one of the above is implemented.

[0016] In a fourth aspect, the present invention further provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program. When the computer program runs, the device where the computer-readable storage medium is located is controlled to execute the monitoring method of the SMT production line described in any one of the above.

[0017] Compared with the prior art, the present invention has the following beneficial effects: The present invention discloses a monitoring method, system, device and storage medium for an SMT production line, including: obtaining device monitoring videos and production line device data, where the production line device data includes production equipment weights and the total number of devices; performing image preprocessing operations on the device monitoring videos to obtain device operation efficiency diagrams, target device images and adjacent device images; performing density clustering operations on the device operation efficiency diagrams to obtain abnormal clusters, and obtaining efficiency anomaly indicators based on the abnormal densities of the abnormal clusters; performing image enhancement operations on the target device images and the adjacent device images to obtain a first enhanced image and a second enhanced image; performing pixel integration on the first enhanced image to obtain an operation anomaly diagram, and taking the gray mean value of all pixel points in the operation anomaly diagram as the operation anomaly index; calculating the abnormal point probabilities of the first enhanced image and the second enhanced image, and calculating a material change coefficient based on the abnormal point probabilities and the production equipment weights; calculating the efficiency anomaly indicators, the operation anomaly index and the material change coefficient based on a preset overall device evaluation model to obtain an overall evaluation degree; performing production line status evaluation based on the overall evaluation degree, and when the overall evaluation degree is greater than a preset evaluation threshold, determining that there is an anomaly in the production line.

[0018] The prior art collects operation data through devices such as sensors and combines machine learning algorithms of historical data to predict the operation status of devices. In addition, the monitored indicators are relatively single, lacking attention to the device status, and it is difficult to comprehensively evaluate the operation status of devices. The present invention processes and analyzes device monitoring videos to obtain efficiency anomaly indicators, operation anomaly indexes and material change coefficients regarding device operation. By calculating the efficiency anomaly indicators, the operation anomaly indexes and the material change coefficients, an overall evaluation degree regarding the device status is obtained. By comparing the overall evaluation degree with a preset evaluation threshold to determine the production line status, a comprehensive evaluation of the overall operation status of the production line can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic flowchart of the monitoring method for an SMT production line provided in the first embodiment of the present invention; Figure 2 is a schematic structural diagram of the monitoring system for an SMT production line provided in the second embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0020] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0021] Reference Figure 1 , the first embodiment of the present invention provides a monitoring method for an SMT production line, including the following steps: S11, obtain device monitoring videos and production line device data, where the production line device data includes production equipment weights and the total number of devices; S12, perform image preprocessing operations on the device monitoring videos to obtain device operation efficiency diagrams, target device images, and adjacent device images; S13, perform density clustering operations on the device operation efficiency diagrams to obtain abnormal clusters, and obtain efficiency anomaly indicators based on the abnormal densities of the abnormal clusters; S14, perform image enhancement operations on the target device images and the adjacent device images to obtain a first enhanced image and a second enhanced image; S15, perform pixel integration on the first enhanced image to obtain an operation anomaly diagram, and use the gray mean value of all pixel points in the operation anomaly diagram as the operation anomaly index; S16, calculate the abnormal point probabilities of the first enhanced image and the second enhanced image, and calculate the material change coefficient based on the abnormal point probabilities and the production equipment weights; S17, based on a preset overall device evaluation model, calculate the efficiency anomaly indicator, the operation anomaly index, and the material change coefficient to obtain an overall evaluation degree; S18, perform production line status evaluation based on the overall evaluation degree. When the overall evaluation degree is greater than a preset evaluation threshold, it is determined that there is an abnormality in the production line.

[0022] In step S11, device monitoring videos and production line device data are obtained, where the production line device data includes production equipment weights and the total number of devices.

[0023] It should be noted that in the monitoring interface, find the monitoring camera list or video channel list corresponding to the production line area, configure parameters such as the resolution, frame rate, and bit rate of the video. After completing the parameter configuration, through the operation of the client software, send an instruction to obtain the video stream to the selected video channel, and the monitoring system will transmit the real-time video data. The client software will decode and play the video in real time to achieve visual monitoring of the device. At the same time, the video stream is locally stored (such as saved to a specified disk folder, and storage strategies such as storage format and duration are set) for subsequent image processing.

[0024] It should be noted that the production equipment weight is calculated based on the proportion of the working hours of the production equipment on the production line.

[0025] The production equipment weight is calculated by the following method: in, is the production equipment weight of equipment i, is the operating time of equipment i in one production cycle, and n is the total number of equipment on the production line.

[0026] It should be noted that the total number of equipment covered by the current production line can be obtained by finding the module for recording the equipment list in the equipment asset management system.

[0027] In step S12, an image preprocessing operation is performed based on the device monitoring video to obtain a device operation efficiency graph, a target device image, and an adjacent device image.

[0028] Grayscale the device monitoring video to obtain a video grayscale image; Take frames of the video grayscale image at a preset time interval to obtain the target device image and the adjacent device image; Calling real-time detection data of the corresponding device in the video grayscale image; The real-time detection data is used as an efficiency impact index to calculate and obtain an efficiency evaluation coefficient; Taking the mean value of the efficiency evaluation coefficient as the equipment operation efficiency, and constructing an equipment operation efficiency graph based on the equipment operation efficiency; The efficiency evaluation coefficient is calculated by the following formula: Where E is the efficiency evaluation coefficient, It is Real-time detection data of each device, It is the maximum value among all the real-time detection data of the equipment. It is The weight of the real-time detection data of each device, n is the total number of devices on the production line.

[0029] It should be noted that the locally stored equipment monitoring video is grayed out and converted from BGR color format to grayscale format. The preset time interval is determined according to factors such as the speed of equipment operation and the frame rate of the monitoring video. The frame image can be cropped according to the approximate position of the target equipment and the adjacent equipment in the image, so as to obtain the target equipment image and the adjacent equipment image. The real-time detection data is stored in the enterprise's production management system, the database of the equipment monitoring system, or transmitted in real time through a specific sensor network. These data contain information on the equipment's operating parameters (such as temperature, speed, voltage, etc.), working status (for example, operation, shutdown, failure, etc.), etc. Analysis of the parameters in the real-time detection data has a direct impact on the equipment's operating efficiency, and the efficiency evaluation coefficient is calculated through the formula.

[0030] Exemplarily, the monitoring video frame rate is 25 frames per second. If one frame of image is taken every 5 seconds, then the corresponding number of frames in the interval is 5 * 25 = 125 frames.

[0031] In step S13, perform density clustering operation on the device operation efficiency map to obtain abnormal clusters, and obtain an efficiency anomaly index based on the abnormal density of the abnormal clusters.

[0032] Perform normalization processing on the device operation efficiency map to obtain a normalized efficiency map; Perform density clustering operation on the normalized efficiency map according to the DBSCAN algorithm with a preset density to obtain abnormal clusters; Determine the abnormal density according to the abnormal clusters, and calculate the efficiency anomaly index according to the abnormal density; Among them, the efficiency anomaly index is calculated by the following formula: Among them, is the efficiency anomaly index, is the spatial density of the j-th data point in the i-th abnormal cluster, is the total number of data points in the i-th abnormal cluster, S is the total number of data points in all abnormal clusters, is the total number of abnormal clusters.

[0033] It should be noted that the data in the device operation efficiency map are in different magnitude ranges. The purpose of normalization processing (normalization) is to map the data to a specific interval (for example, the interval [0, 1] or an interval that meets the requirements of a specific statistical distribution), so that subsequent clustering and other algorithms can better process the data and avoid analysis deviations caused by too large differences in data magnitudes.

[0034] It should be noted that DBSCAN is a density-based spatial clustering algorithm. It discovers clusters based on the density of data points, divides data points with sufficient density into a cluster, and regards data points in low-density regions as noise points through a preset density value, thereby screening out abnormal clusters (such as clusters with fewer points in the cluster and relatively low density, etc., which do not conform to the density characteristics of the normal operating state).

[0035] It should be noted that it is necessary to calculate the average density or preset density value of the abnormal clusters, and the density lower than the average density value is the abnormal density; the abnormal efficiency index is calculated by the formula based on the above principle.

[0036] In step S14, perform image enhancement operations on the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image.

[0037] Normalize the target device image to obtain a normalized image; Enhance the normalized image using a preset image enhancement method to obtain the first enhanced image and the second enhanced image; It should be noted that since the overall contrast of the production equipment is low and the gray-scale distribution of the image is uneven, normalization processing first processes the image to a suitable analysis range, and image enhancement can effectively improve the overall visual clarity of the image.

[0038] Exemplarily, the image enhancement method is histogram equalization enhancement. By adjusting the histogram of the image, the gray-scale distribution of the image is made more uniform, thereby enhancing the contrast of the image and making the details in the image more clearly visible. Based on the cumulative distribution function (CDF) of the image gray-scale histogram for transformation, for each gray level, according to its proportion and cumulative situation in the original image histogram, it is remapped to a new gray level, so that the gray-level distribution of the transformed image is closer to a uniform distribution.

[0039] Exemplarily, the pixel gray values in the darker areas of the original image are concentrated in the low gray levels. After histogram equalization, these pixels will be assigned to a wider range of gray levels, making the details in the dark areas prominent.

[0040] In step S15, integrate the pixels of the first enhanced image to obtain an operation anomaly map, and use the gray-scale mean value of all pixel points in the operation anomaly map as the operation anomaly index.

[0041] Call the first enhanced image to calculate the gray-scale gradient to obtain a gray-scale gradient matrix; Screen the pixel points of the first enhanced image to obtain enhanced pixel points; Integrate the gray-scale gradient matrix and the enhanced pixel points to obtain an operation anomaly map; Calculate the gray-scale mean value of all pixel points in the operation anomaly map to obtain an operation anomaly index; Among them, the operation anomaly index is calculated by the following formula: Among them, represents the operation anomaly index, represents the total number of all pixel points in the operation anomaly map, represents the th pixel point in the operation anomaly map.

[0042] It should be noted that the gray-scale gradient reflects the change rate of pixel gray values in different directions in the image. By calculating the gray-scale gradient, details such as object edges, textures, and surface temperatures in the image can be obtained, which helps to judge whether the device has abnormal operation conditions subsequently.

[0043] Specifically, a grayscale threshold range is set and only pixels with grayscale values ​​within the range are selected to pick out pixels in the image that are important for determining abnormal device operation, such as pixels located at key parts, edges, or areas with obvious grayscale changes.

[0044] Specifically, the grayscale change information of the image reflected by the grayscale gradient matrix and the key pixel information obtained by screening the pixels are combined to generate an operation abnormality map that can intuitively reflect the abnormal operation of the equipment. The area where the grayscale change is more drastic and the part where the key pixel overlaps are the focus areas. The sum of the grayscale values ​​obtained by statistics is divided by the total number of pixels to obtain the grayscale mean of all pixels in the operation abnormality map, which is the required operation abnormality index.

[0045] For example, when cracks, wear and tear appear on the surface of the equipment, the image often shows a sudden change in grayscale value, and grayscale gradient calculation can highlight these change areas. Also, when there is a large temperature difference in the equipment, the high temperature area will be displayed as a white area with higher brightness after grayscale processing.

[0046] In step S16, the outlier probabilities of the first enhanced image and the second enhanced image are calculated, and the material variation coefficient is calculated based on the outlier probabilities and the production equipment weights.

[0047] Constructing grayscale histograms of the first enhanced image and the second enhanced image; Filtering pixels of the calculated grayscale histogram, taking pixels whose brightness is within a preset brightness range as normal pixels, and taking pixels whose brightness is greater than or less than the preset brightness range as abnormal pixels; Calculate the abnormal point probability of the abnormal pixel among all the pixels; Calculate the material variation coefficient according to the abnormal point probability and the production equipment weight; Among them, the material variation coefficient is calculated by the following formula: in, Indicates the material variation coefficient, is the production equipment weight of the ith equipment, is the probability of anomalies in the neighborhood of the i-th device, is the total number of devices on the production line.

[0048] Specifically, the grayscale histogram is a chart that statistically counts the occurrence frequencies of different grayscale levels in the first enhanced image and the second enhanced image. It can intuitively reflect the grayscale distribution of the image, and characteristics such as the contrast and brightness of the device surface can be understood. For example, if the grayscale values in the histogram are concentrated in darker or brighter areas, it means that the image has under - exposure or over - exposure conditions, and the brightness is also associated with the state changes of the device surface.

[0049] It should be noted that the determination of the preset brightness interval needs to be set in combination with the characteristics of the image when the device is operating normally and the experience of the actual application scenario. For example, a certain same device on multiple production lines can be observed simultaneously, and the interval range where the grayscale values in the grayscale histogram are concentrated is analyzed, and this interval range is determined as the preset brightness interval. Pixel points with brightness greater than or less than the preset brightness interval indicate that deformation has occurred outside the device or the temperature inside the device has changed, providing a basis for judging the operating state of the device.

[0050] In step S17, based on the preset overall device evaluation model, the efficiency anomaly index, the operation anomaly index, and the material change coefficient are calculated to obtain the overall evaluation degree.

[0051] The overall evaluation degree is calculated through the following overall device evaluation model: Among them, Q is the overall evaluation degree, is the efficiency anomaly index, is the operation anomaly index, is the material change coefficient, is the weight of the efficiency anomaly index, is the weight of the operation anomaly index, is the weight of the material change coefficient.

[0052] It should be noted that the overall device evaluation model is pre - constructed according to the type of the device, production process requirements, and relevant professional knowledge, etc. Its form is a simple weighted - sum model or a model containing more complex functional relationships. The weights in this model are determined according to the importance of the factors to the overall operation of the device, which will not be described in this embodiment, and the sum of the weights is 1.

[0053] Exemplarily, by analyzing the respective changes of the efficiency anomaly index, the operation anomaly index, and the material change coefficient when the device has had failures, a decline in operation efficiency, etc. in the past, as well as their correlation degrees with the actual severity of the device failure, production losses, etc., the weights of the overall device evaluation model are determined.

[0054] In step S18, the state of the production line is evaluated according to the overall evaluation degree. When the overall evaluation degree is greater than the preset evaluation threshold, it is determined that the production line has an anomaly.

[0055] Exemplarily, establishing the preset evaluation threshold requires considering various factors. For example, referring to historical data, collecting the overall evaluation data corresponding to the normal operation of the past production line and various abnormal situations, and statistically calculating the average value, maximum value, etc. of the overall evaluation degree during the normal operation of the production line for a period of time; or combining the equipment characteristics and production requirements. For example, considering factors such as the key degree, complexity of various equipment in the production line, the impact on the production process, and the quality standards and output requirements of the produced products.

[0056] Specifically, compare the obtained overall evaluation degree value with the preset evaluation threshold to determine the state of the production line. Judge whether the overall evaluation degree is less than the preset evaluation threshold. If so, continue production. If not, it is determined that there is an abnormality in the production line, and the system sends an alarm instruction to the control end, and the control end stops the production line to eliminate the fault.

[0057] To facilitate the understanding of the present invention, some preferred embodiments of the present invention will be further described below.

[0058] The working process of the present invention will be described below by taking a relatively common scenario as an example. Please also refer to Figure 2 , which is Figure 1 a schematic diagram of the working scenario of the method.

[0059] The working process is as follows: Step 1: The production line is working normally. The equipment monitoring system stores the monitoring video in the computer, and the production line sensors transmit the production line equipment data to the computer system. The computer calls the equipment monitoring video and the production line equipment data.

[0060] Step 2: The computer uses a preset image processing method to perform grayscale processing on the called equipment monitoring video, and takes frames from the grayscale processed video grayscale image to obtain the target equipment image and the adjacent equipment image. The computer retrieves the monitoring data of the corresponding equipment in the video grayscale image, performs calculations to obtain the efficiency evaluation coefficient, takes the average value of the efficiency evaluation coefficient as the equipment operation efficiency, and constructs an equipment operation efficiency map based on the equipment operation efficiency.

[0061] Step 3: The computer performs standardization processing on the equipment operation efficiency map to obtain a standardized efficiency map, and performs density clustering operation on the standardized efficiency map according to the DBSCAN algorithm with a preset density to obtain abnormal clusters, obtains abnormal density through the abnormal clusters, and calculates the efficiency anomaly index.

[0062] Step 4: The computer calls the target equipment image and the adjacent equipment image, performs normalization processing, and enhances the image using histogram equalization to obtain the first enhanced image and the second enhanced image.

[0063] Step Five: The system calls the first enhanced image for gray gradient calculation to obtain a gray gradient matrix, screens the pixel points of the first enhanced image according to a preset brightness interval to obtain enhanced pixel points, integrates the gray gradient matrix and the enhanced pixel points, focuses on the overlapping part, obtains an operation anomaly map, and calculates the gray mean value of all pixel points in the operation anomaly map to obtain an operation anomaly index.

[0064] Step Six: Construct gray histograms of the first enhanced image and the second enhanced image, screen abnormal pixel points within the gray histograms of the first enhanced image and the second enhanced image according to a preset brightness interval, calculate the abnormal point probability of the abnormal pixel points among all pixel points in the gray histograms, and calculate together with the production equipment weight to obtain a material change coefficient.

[0065] Step Seven: Calculate the overall evaluation degree, and use the overall evaluation degree to evaluate the status of the production line.

[0066] Step Eight: If the overall evaluation degree is greater than a preset evaluation threshold, the system issues an alarm, the alarm is transmitted to the control end, the control end stops the production line for troubleshooting. After the problem is properly solved, the control end recalibrates the overall evaluation degree to ensure the accuracy of the evaluation, and resumes the production of the production line.

[0067] In summary, the present invention discloses a monitoring method, system, device and storage medium for an SMT production line, including: obtaining an equipment monitoring video and production line equipment data, where the production line equipment data includes production equipment weight and the total number of equipment; performing image preprocessing operations on the equipment monitoring video to obtain an equipment operation efficiency map, a target equipment image and an adjacent equipment image; performing density clustering operations on the equipment operation efficiency map to obtain abnormal clusters, and obtaining an efficiency anomaly index based on the abnormal density of the abnormal clusters; performing image enhancement operations on the target equipment image and the adjacent equipment image to obtain a first enhanced image and a second enhanced image; performing pixel integration on the first enhanced image to obtain an operation anomaly map, and taking the gray mean value of all pixel points in the operation anomaly map as an operation anomaly index; calculating the abnormal point probability of the first enhanced image and the second enhanced image, and calculating according to the abnormal point probability and the production equipment weight to obtain a material change coefficient; based on a preset equipment overall evaluation model, calculating the efficiency anomaly index, the operation anomaly index and the material change coefficient to obtain an overall evaluation degree; performing production line status evaluation according to the overall evaluation degree, and when the overall evaluation degree is greater than a preset evaluation threshold, determining that the production line has an anomaly.

[0068] In the prior art, operating data is collected through devices such as sensors, and machine learning algorithms combined with historical data are used to predict the operating status of the device. In addition, the monitored indicators are relatively single, lacking attention to the device status, and it is difficult to comprehensively evaluate the operating conditions of the device. In the present invention, by processing and analyzing the device monitoring video, an efficiency anomaly index, an operating anomaly index, and a material change coefficient regarding the device operation are obtained. By calculating the efficiency anomaly index, the operating anomaly index, and the material change coefficient, an overall evaluation degree regarding the device status is obtained. By comparing the overall evaluation degree with a preset evaluation threshold, the production line status can be determined, and a comprehensive evaluation of the overall operating status of the production line can be achieved.

[0069] Referring to Figure 2 , the second embodiment of the present invention provides a monitoring system for an SMT production line, including: A data acquisition module, configured to acquire device monitoring videos and production line device data, where the production line device data includes production equipment weights and the total number of devices; An image processing module, configured to perform image preprocessing operations according to the device monitoring videos to obtain a device operation efficiency map, a target device image, and an adjacent device image; A density clustering module, configured to perform density clustering operations according to the device operation efficiency map to obtain abnormal clusters, and obtain an efficiency anomaly index based on the abnormal density of the abnormal clusters; An image enhancement module, configured to perform image enhancement operations according to the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image; A pixel integration module, configured to perform pixel integration on the first enhanced image to obtain an operating anomaly map, and use the gray scale mean value of all pixel points in the operating anomaly map as the operating anomaly index; An anomaly point calculation module, configured to calculate the anomaly point probabilities of the first enhanced image and the second enhanced image, and calculate a material change coefficient according to the anomaly point probabilities and the production equipment weights; An evaluation degree calculation module, configured to calculate an overall evaluation degree based on a preset overall device evaluation model by calculating the efficiency anomaly index, the operating anomaly index, and the material change coefficient; A status evaluation module, configured to perform production line status evaluation according to the overall evaluation degree, and determine that there is an anomaly in the production line when the overall evaluation degree is greater than a preset evaluation threshold.

[0070] It should be noted that the monitoring system for an SMT production line provided in the embodiment of the present invention is used to execute all the process steps of the monitoring method for an SMT production line in the above embodiment, and the working principles and beneficial effects of the two correspond one by one, so they will not be elaborated here.

[0071] An embodiment of the present invention also provides an electronic device. The electronic device includes: a processor, a memory, and a computer program stored in the memory and executable on the processor, such as a monitoring method for an SMT production line. When the processor executes the computer program, the steps in the above embodiments of the monitoring method for each SMT production line are implemented, such as Figure 1 step S11 shown. Alternatively, when the processor executes the computer program, the functions of each module / unit in the above device embodiments are implemented, such as a data acquisition module.

[0072] Exemplarily, the computer program may be divided into one or more modules / units, and the one or more modules / units are stored in the memory and executed by the processor to complete the present invention. The one or more modules / units may be a series of computer program instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0073] The electronic device may be a computing device such as a desktop computer, a notebook, a palm computer, and a smart tablet. The electronic device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the above components are only examples of the electronic device and do not constitute a limitation on the electronic device. It may include more or fewer components than the above, or combine certain components, or different components. For example, the electronic device may further include input / output devices, network access devices, a bus, etc.

[0074] The so-called processor may be a central processing unit (CPU), or may also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the electronic device and connects various parts of the entire electronic device through various interfaces and lines.

[0075] The memory can be used to store the computer program and / or module. By running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory, the processor realizes various functions of the electronic device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.); the data storage area can store the data created according to the use of the mobile phone (such as audio data, phone book, etc.). In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as hard disks, memory, plug-in hard disks, smart media cards (SMC), secure digital (SD) cards, flash cards, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0076] Among them, if the module / unit integrated in the electronic device is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by instructing relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0077] It should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the attached drawings of the device embodiments provided by the present invention, the connection relationships between modules indicate that they have communication connections, which can be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement without creative efforts.

[0078] The specific embodiments described above further elaborate on the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. In particular, for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A monitoring method for an SMT production line, characterized in that: Executed by a computer, including: Obtaining equipment monitoring videos and production line equipment data, wherein the production line equipment data includes production equipment weights and the total number of equipment; Perform image preprocessing operations according to the device monitoring video to obtain a device operation efficiency graph, a target device image, and an adjacent device image; Performing density clustering operation according to the equipment operation efficiency graph to obtain abnormal clusters, and obtaining efficiency abnormality indicators based on abnormal density of the abnormal clusters; Performing an image enhancement operation on the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image; Performing pixel integration on the first enhanced image to obtain an operation abnormality map, and taking the grayscale mean of all pixels in the operation abnormality map as an operation abnormality index; Calculating the outlier probabilities of the first enhanced image and the second enhanced image, and obtaining the material variation coefficient according to the outlier probabilities and the production equipment weights; Based on a preset equipment overall evaluation model, the efficiency abnormality index, the operation abnormality index and the material variation coefficient are calculated to obtain an overall evaluation degree; The production line status is evaluated according to the overall evaluation degree, and when the overall evaluation degree is greater than a preset evaluation threshold, it is determined that an abnormality exists in the production line.

2. The monitoring method of the SMT production line according to claim 1, characterized in that: Obtain production line equipment data, including: The weight of production equipment is calculated by the following method: in, is the production equipment weight of equipment i, is the operating time of equipment i in one production cycle, and n is the total number of equipment on the production line.

3. The monitoring method of the SMT production line according to claim 1, characterized in that: Performing image preprocessing operations according to the device monitoring video to obtain a device operation efficiency graph, a target device image, and an adjacent device image, including: Grayscale the device monitoring video to obtain a video grayscale image; Take frames of the video grayscale image at a preset time interval to obtain the target device image and the adjacent device image; Calling real-time detection data of the corresponding device in the video grayscale image; The real-time detection data is used as an efficiency impact index to calculate and obtain an efficiency evaluation coefficient; Taking the mean value of the efficiency evaluation coefficient as the equipment operation efficiency, and constructing an equipment operation efficiency graph based on the equipment operation efficiency; The efficiency evaluation coefficient is calculated by the following formula: Where E is the efficiency evaluation coefficient, It is Real-time detection data of each device, It is the maximum value among all the real-time detection data of the equipment. It is The weight of the real-time detection data of each device, n is the total number of devices on the production line.

4. The monitoring method of the SMT production line according to claim 1, characterized in that: Performing density clustering operation according to the equipment operation efficiency graph to obtain abnormal clusters, and obtaining efficiency abnormality indicators based on abnormal density of the abnormal clusters, including: Standardizing the equipment operation efficiency graph to obtain a standardized efficiency graph; Performing a density clustering operation on the standardized efficiency graph according to a DBSCAN algorithm with a preset density to obtain abnormal clusters; Determine an abnormal density according to the abnormal clustering, and calculate an efficiency abnormality index according to the abnormal density; Among them, the efficiency abnormality index is calculated by the following formula: in, is the efficiency abnormality indicator, is the spatial density of the jth data point in the ith abnormal cluster, is the total number of data points in the ith abnormal cluster, S is the sum of the number of data points in all abnormal clusters, is the total number of abnormal clusters.

5. The monitoring method of the SMT production line according to claim 1, characterized in that: Performing an image enhancement operation on the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image includes: Normalizing the target device image to obtain a normalized image; Enhance the normalized image using a preset image enhancement method to obtain the first enhanced image and the second enhanced image; Among them, image enhancement methods include histogram equalization enhancement, filtering enhancement, grayscale transformation enhancement and image sharpening.

6. The monitoring method of the SMT production line according to claim 1, characterized in that: Performing pixel integration on the first enhanced image to obtain an operation abnormality map, and taking the grayscale mean of all pixels in the operation abnormality map as an operation abnormality index, including: Calling the first enhanced image to perform grayscale gradient calculation to obtain a grayscale gradient matrix; Screening pixels of the first enhanced image to obtain enhanced pixels; Integrate the grayscale gradient matrix with the enhanced pixel points to obtain an operation abnormality map; Calculate the grayscale mean of all pixels in the operation abnormality map to obtain an operation abnormality index; The operation abnormality index is calculated by the following formula: in, Indicates the abnormal operation index. represents the total number of all pixels in the operation anomaly graph, Indicates the operation abnormality diagram The gray value of a pixel.

7. The monitoring method of the SMT production line according to claim 1, characterized in that: Calculating the outlier probabilities of the first enhanced image and the second enhanced image, and calculating the material variation coefficient according to the outlier probabilities and the production equipment weights, including: Constructing grayscale histograms of the first enhanced image and the second enhanced image; Filtering pixels of the calculated grayscale histogram, taking pixels whose brightness is within a preset brightness range as normal pixels, and taking pixels whose brightness is greater than or less than the preset brightness range as abnormal pixels; Calculate the abnormal point probability of the abnormal pixel among all the pixels; Calculate the material variation coefficient according to the abnormal point probability and the production equipment weight; Among them, the material variation coefficient is calculated by the following formula: in, Indicates the material variation coefficient, is the production equipment weight of the ith equipment, is the probability of anomalies in the neighborhood of the i-th device, is the total number of devices on the production line.

8. The monitoring method of the SMT production line according to claim 1, characterized in that: Based on the preset equipment overall evaluation model, the efficiency abnormality index, the operation abnormality index and the material variation coefficient are calculated to obtain an overall evaluation degree, including: The overall evaluation degree is calculated by the following equipment overall evaluation model: Among them, Q is the overall evaluation degree, is the efficiency abnormality indicator, is the operation abnormality index, is the material variation coefficient, is the weight of the efficiency anomaly index, is the weight of the abnormal operation index, is the weight of the material variation coefficient.

9. A monitoring system for an SMT production line, characterized in that: include: A data acquisition module, used to acquire equipment monitoring videos and production line equipment data, wherein the production line equipment data includes production equipment weights and total number of equipment; An image processing module is used to perform image preprocessing operations according to the device monitoring video to obtain a device operation efficiency diagram, a target device image, and an adjacent device image; A density clustering module, configured to perform a density clustering operation according to the equipment operation efficiency graph to obtain an abnormal cluster, and obtain an efficiency abnormality index based on the abnormal density of the abnormal cluster; An image enhancement module, configured to perform an image enhancement operation on the target device image and the adjacent device image to obtain a first enhanced image and a second enhanced image; a pixel integration module, configured to perform pixel integration on the first enhanced image to obtain an operation abnormality map, and use the grayscale mean of all pixels in the operation abnormality map as an operation abnormality index; an abnormal point calculation module, used to calculate the abnormal point probability of the first enhanced image and the second enhanced image, and obtain the material variation coefficient according to the abnormal point probability and the production equipment weight; An evaluation degree calculation module is used to calculate the efficiency abnormality index, the operation abnormality index and the material variation coefficient based on a preset equipment overall evaluation model to obtain an overall evaluation degree; The status evaluation module is used to evaluate the status of the production line according to the overall evaluation degree, and when the overall evaluation degree is greater than a preset evaluation threshold, it is determined that the production line is abnormal.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored computer program, wherein when the computer program is executed, the device where the computer-readable storage medium is located is controlled to execute the monitoring method of the SMT production line according to any one of claims 1 to 8.

Images