A digital twin construction method and system for panel production line

Abstract

The embodiment of the present invention provides a method and system for constructing a digital twin of a panel production line, which belongs to the field of data processing technology. The method includes: obtaining a first image of a first target plate and a second image of a second target plate, and obtaining first sensor data of multiple initial sensors for data detection; performing target extraction on the first image to obtain first target information and performing target extraction on the second image to obtain second target information, and determining first abnormal information based on the first target information and the second target information; performing abnormal data identification on the first sensor data to obtain second sensor data, and screening from the initial sensors based on the second sensor data to obtain second abnormal information; determining abnormal equipment based on the first abnormal information and the second abnormal information; generating abnormal pictures based on the first abnormal information and the second abnormal information; constructing a digital twin of the target production line based on the abnormal pictures to obtain a real state model of the target production line.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method and system for constructing a digital twin of a panel production line. Background Art

[0002] The data twin model can monitor the status of the physical system in real time and provide the latest operating information, thereby helping operators to understand the operation of the production line or equipment in a timely manner. However, existing technologies usually display monitoring data by simulating the production line and use the interface to display and analyze the data. When data anomalies occur, the system will issue an alarm, but only with these abnormal reminders, only professionals can deeply analyze the data and identify the specific abnormal status of the equipment. For non-professionals, they can only know that there is abnormal data, but they cannot understand the specific abnormal status of the equipment, which reduces the response ability and processing efficiency of non-professionals to abnormal situations. This limitation may lead to delays in equipment maintenance and troubleshooting, thereby affecting overall production efficiency. Summary of the invention

[0003] The main purpose of the embodiments of the present invention is to provide a method and system for constructing a digital twin of a panel production line, aiming to solve the problem in the related technology that when using a digital twin model for equipment monitoring, non-professionals can only know that there is abnormal data but cannot specifically understand the abnormal state of the equipment, which reduces the non-professionals' response ability and processing efficiency to abnormal situations.

[0004] In a first aspect, an embodiment of the present invention provides a method for constructing a digital twin of a panel production line, comprising:

[0005] Obtaining a first image corresponding to a first target plate obtained under a target production line and a second image corresponding to a second target plate obtained under the target production line, and obtaining first sensor data corresponding to data detection by a plurality of initial sensors in the target production line;

[0006] Performing target extraction on the first image to obtain first target information and performing target extraction on the second image to obtain second target information, and determining first abnormal information corresponding to a target component according to the first target information and the second target information;

[0007] Performing abnormal data identification on the first sensor data to obtain second sensor data, and screening from the initial sensor data according to the second sensor data to obtain second abnormal information;

[0008] Determine the corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information;

[0009] Generate an abnormal picture corresponding to the abnormal device according to the first abnormal information and the second abnormal information;

[0010] A digital twin of the target production line is constructed according to the abnormal image to obtain a real state model corresponding to the target production line.

[0011] In a second aspect, an embodiment of the present invention provides a digital twin construction system for a panel production line, including:

[0012] A data acquisition module, used to obtain a first image corresponding to a first target plate obtained under a target production line and a second image corresponding to a second target plate obtained under the target production line, and obtain first sensor data corresponding to data detection by a plurality of initial sensors in the target production line;

[0013] an abnormality identification module, configured to extract a target from the first image to obtain first target information and to extract a target from the second image to obtain second target information, and to determine first abnormality information corresponding to a target component according to the first target information and the second target information;

[0014] an abnormality screening module, used for performing abnormal data identification on the first sensor data to obtain second sensor data, and screening from the initial sensor data according to the second sensor data to obtain second abnormality information;

[0015] An equipment determination module, configured to determine a corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information;

[0016] An image generation module, configured to generate an abnormal image corresponding to the abnormal device according to the first abnormal information and the second abnormal information;

[0017] A model building module is used to build a digital twin of the target production line according to the abnormal image to obtain a real state model corresponding to the target production line.

[0018] The embodiment of the present invention provides a method and system for constructing a digital twin of a panel production line. The method includes: obtaining a first image corresponding to a first target plate and a second image corresponding to a second target plate in a target production line, and collecting first sensor data of multiple initial sensors in the target production line, and then performing target extraction on the first image to obtain first target information and performing target extraction on the second image to obtain second target information. Thus, based on the first target information and the second target information, the first abnormal information of the target component is determined. Then, the first sensor data is identified for abnormal data to obtain second sensor data. Then, according to the second sensor data, the initial sensor data is screened to obtain second abnormal information. Then, according to the first abnormal information and the second abnormal information, the abnormal equipment in the target production line is determined, and an abnormal picture reflecting the abnormal state is generated, and the digital twin model of the production line is updated using these abnormal pictures. Thus, a digital twin model is constructed according to the updated abnormal picture to obtain a real state model, ensuring that the real state model accurately reflects the actual operating state and abnormal conditions of the target production line, and the constructed real state model is helpful for non-professionals to monitor and manage the production line in real time. By integrating images and sensor data, the method can accurately identify equipment abnormalities and improve the efficiency of fault detection and maintenance. Accurate digital twin models improve production management, help optimize production processes, reduce costs and enhance safety. Updated digital twin models allow non-professionals to more clearly understand the abnormal state of the equipment, thereby improving their ability to respond to abnormal situations and their efficiency in handling them. This method also effectively solves the problem that non-professionals cannot understand the specific abnormal state of the equipment when using digital twin models for equipment monitoring, and improves the monitoring and management capabilities of the overall production line. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0020] Figure 1 A schematic diagram of a process flow of a method for constructing a digital twin of a panel production line provided by an embodiment of the present invention;

[0021] Figure 2 A schematic diagram of the module structure of a digital twin construction system for a panel production line provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0024] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] The embodiment of the present invention provides a method and system for constructing a digital twin of a panel production line. The method for constructing a digital twin of a panel production line can be applied to a terminal device, which can be an electronic device such as a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, and a wearable device. The terminal device can be a server or a server cluster.

[0026] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0027] Please refer to Figure 1 , Figure 1 A schematic flow chart of a method for constructing a digital twin of a panel production line provided in an embodiment of the present invention.

[0028] like Figure 1 As shown, the panel production line digital twin construction method includes steps S101 to S106.

[0029] Step S101, obtaining a first image corresponding to a first target plate obtained on a target production line and a second image corresponding to a second target plate obtained on the target production line, and obtaining first sensor data corresponding to data detection by multiple initial sensors in the target production line.

[0030] Exemplarily, a camera or image acquisition device needs to be installed on the target production line to ensure that the camera is positioned correctly so as to capture the target plate obtained on the target production line, thereby obtaining a first image corresponding to the first target plate produced at the current moment and a second image corresponding to the second target plate produced at the previous moment or the next moment.

[0031] Exemplarily, a plurality of initial sensors (such as temperature sensors, pressure sensors, displacement sensors, etc.) are installed in the target production line, and then different equipment on the target production line is monitored by the plurality of initial sensors, thereby obtaining corresponding first sensor data under the plurality of initial sensors.

[0032] For example, the operating parameters and status of various types of plate processing equipment (cutting saws, edge banding machines, drilling machines, robots, automatic roller lines, transfer lines, belt lines) on the target production line and the data of the currently executed order production tasks (plate size, order, processing technology) are obtained through multiple initial sensors, that is, the corresponding first sensor data is obtained, and then the first sensor data corresponding to the data detection of multiple initial sensors in the target production line is obtained through the corresponding interface.

[0033] It should be noted that after obtaining the first sensor data, the first image and the second image, all the collected data need to be stored in the database, so that big data processing tools can be used to clean, analyze and model, to provide support for the subsequent creation of the real state model. Furthermore, the digital twin construction system of the panel production line also includes a data storage module for obtaining the first sensor data, the first image and the second image. All the collected data need to be stored in the database, so that big data processing tools can be used to clean, analyze and model.

[0034] In addition, after obtaining the first sensor data, the first image and the second image, the above data need to be cleaned and analyzed to provide support for subsequent abnormal information identification.

[0035] Step S102: extracting a target from the first image to obtain first target information and extracting a target from the second image to obtain second target information, and determining first abnormal information corresponding to a target component according to the first target information and the second target information.

[0036] Exemplarily, a contour detection algorithm (such as findContours in OpenCV) is used to identify and extract the contour of the target in the first image, and then the image segmentation technology (such as threshold segmentation and region growing) is used to separate the target from the background to obtain the first target information. Similarly, a contour detection algorithm (such as findContours in OpenCV) is used to identify and extract the contour of the target in the second image, and then the image segmentation technology (such as threshold segmentation and region growing) is used to separate the target from the background to obtain the second target information.

[0037] Exemplarily, when there is an abnormality in the target component, the same target will exist in the same position in the first image and the second image, and then the first contour information and the first position information of the first target information in the first image are determined, and the second contour information and the second position information of the second target information in the second image are determined, and then the first distance information between the first position information and the second position information is calculated, and the second distance information between the first contour information and the second contour information is compared, and then the first distance information and the second distance information are fused to obtain the target distance information, so that when the target distance information is less than the preset distance, it is determined that there is an abnormality in the target component, and the abnormal position corresponding to the target component is determined according to the first target information and the second target information, and then the abnormal position is determined as the first abnormal information corresponding to the target component.

[0038] In some embodiments, the target extraction of the first image to obtain the first target information and the target extraction of the second image to obtain the second target information include: graying the first image to obtain the first grayscale image, and performing noise reduction on the first grayscale image to obtain the first noise reduction image; performing consistency calculation on the first grayscale image and the first noise reduction image to determine the first related information corresponding to different processing when adjacent coordinates in the first image; obtaining the first adjacent pixel information corresponding to any first pixel information in the first grayscale image, and determining the second related information corresponding to the same processing when different coordinates in the first image according to the first pixel information and the first adjacent pixel information; determining the first association information between adjacent pixels in the first image according to the first related information and the second related information; constructing a first fuzzy parameter according to the first association information, and performing pixel clustering on the first grayscale image according to the first fuzzy parameter to obtain a first clustering result; The method comprises the steps of: performing target extraction on the first image based on a clustering result to obtain the first target information; performing grayscale processing on the second image to obtain a second grayscale image, and performing denoising processing on the second grayscale image to obtain a second denoised image; performing consistency calculation on the second grayscale image and the second denoised image to determine the third related information corresponding to different processing at adjacent coordinates in the second image; obtaining the second adjacent pixel information corresponding to any second pixel information in the second grayscale image, and determining the fourth related information corresponding to the same processing at different coordinates in the second image based on the second pixel information and the second adjacent pixel information; determining the second association information between adjacent pixels in the second image based on the third related information and the fourth related information; constructing a second fuzzy parameter based on the second association information, and performing pixel clustering on the second grayscale image based on the second fuzzy parameter to obtain a second clustering result; performing target extraction on the second image based on the second clustering result to obtain the second target information.

[0039] Exemplarily, the first image is grayed using a graying formula to obtain a first grayscale image, and a noise reduction algorithm (such as Gaussian filtering, median filtering, etc.) is used to reduce noise and interference in the image to obtain a clearer first denoised image, and then the first pixel difference between any pixel in the first grayscale image and the adjacent pixel of the pixel in the first denoised image is compared to calculate the first pixel difference, and then the maximum value of the grayscale interval is used for normalization. If the first pixel difference between any pixel in the first grayscale image and the adjacent pixel of the pixel in the first denoised image is larger, the corresponding consistency result is smaller, that is, the first relevant information is smaller; if the first pixel difference between any pixel in the first grayscale image and the adjacent pixel of the pixel in the first denoised image is smaller, the corresponding consistency result is larger, that is, the first relevant information is larger.

[0040] Exemplarily, the first adjacent pixel information corresponding to any first pixel information in the first grayscale image is obtained, and the pixel value corresponding to the first pixel information is calculated to perform difference calculation with the pixel value corresponding to each first adjacent pixel information to obtain the second pixel difference, and then the second pixel difference is normalized using the maximum value of the grayscale interval. If the second pixel difference between the first adjacent pixel information corresponding to any first pixel information in the first grayscale image is larger, the corresponding adjacent correlation is smaller, that is, the second related information is smaller; if the second pixel difference between the first adjacent pixel information corresponding to any first pixel information in the first grayscale image is smaller, the corresponding adjacent correlation is larger, that is, the second related information is larger.

[0041] Exemplarily, the first relevant information and the second relevant information are weighted averaged to obtain first association information between adjacent pixels in the first image, and then a first fuzzy parameter required for clustering is defined based on the first association information, and then the first grayscale image is pixel clustered using the first fuzzy parameter to obtain a first clustering result, thereby identifying and extracting the target area in the image based on the first clustering result. For example, the first target information can be extracted from the first clustering result using techniques such as contour detection and region labeling.

[0042] Exemplarily, the second image is grayed using a graying formula to obtain a second grayscale image, and a noise reduction algorithm (such as Gaussian filtering, median filtering, etc.) is used to reduce noise and interference in the image to obtain a clearer second denoised image, and then the third pixel difference between any pixel in the second grayscale image and the adjacent pixel of the pixel in the second denoised image is compared to calculate the third pixel difference, and then the maximum value of the grayscale interval is used for normalization. If the third pixel difference between any pixel in the second grayscale image and the adjacent pixel of the pixel in the second denoised image is larger, the corresponding consistency result is smaller, that is, the third related information is smaller; if the third pixel difference between any pixel in the second grayscale image and the adjacent pixel of the pixel in the second denoised image is smaller, the corresponding consistency result is larger, that is, the third related information is larger.

[0043] Exemplarily, the second adjacent pixel information corresponding to any second pixel information in the second grayscale image is obtained, and the pixel value corresponding to the second pixel information is calculated to perform difference calculation with the pixel value corresponding to each second adjacent pixel information to obtain a fourth pixel difference, and then the fourth pixel difference is normalized using the maximum value of the grayscale interval. If the fourth pixel difference between the second adjacent pixel information corresponding to any second pixel information in the second grayscale image is larger, the corresponding adjacent correlation is smaller, that is, the fourth related information is smaller; if the fourth pixel difference between the second adjacent pixel information corresponding to any second pixel information in the second grayscale image is smaller, the corresponding adjacent correlation is larger, that is, the fourth related information is larger.

[0044] Exemplarily, the third related information and the fourth related information are weighted averaged to obtain the second association information between adjacent pixels in the second image, and then the second fuzzy parameter required for clustering is defined based on the second association information, and then the second grayscale image is pixel clustered using the second fuzzy parameter to obtain a second clustering result, thereby identifying and extracting the target area in the image based on the second clustering result. For example, the second target information can be extracted from the second clustering result using contour detection, region labeling, and other technologies.

[0045] Specifically, by comparing the consistency of the original grayscale image and the denoised image, the denoising effect can be evaluated and the denoising algorithm can be adjusted to better retain the target information. Analyzing the correlation between pixels and their neighboring pixels helps to identify structural features and changes in the image. This is very important for edge detection and feature extraction in target detection and image segmentation. Thus, constructing fuzzy parameters based on the first association information for pixel clustering helps to process fuzzy boundaries and overlapping areas in the image, thereby improving the accuracy and robustness of target extraction.

[0046] In some embodiments, constructing a first fuzzy parameter according to the first association information, and performing pixel clustering on the first grayscale image according to the first fuzzy parameter to obtain a first clustering result includes: determining a first cluster center, calculating first distance information between the first adjacent pixel information and the first cluster center; calculating a first pixel difference between the first adjacent pixel information and the first pixel information; fusing the first distance information, the first pixel difference and the first association information to determine the first fuzzy parameter corresponding to the first cluster center; determining a first correlation degree between the first pixel information and the first cluster center according to the first fuzzy parameter; updating the first cluster center according to the first correlation degree until the updated first cluster center meets a first preset condition; performing pixel clustering on the first grayscale image according to the updated first cluster center to obtain the first clustering result; wherein the first correlation degree is obtained according to the following formula:

[0047]

[0048] in, represents the first degree of association between the first pixel information m1 and the first cluster center n1, represents the number of clusters of the first cluster center, represents the pixel value corresponding to the first pixel information m1, represents the pixel value corresponding to the first cluster center n1, represents the pixel value corresponding to the first cluster center t, represents the first fuzzy parameter between the first cluster center n1 and the first pixel information m1, Represents the first blur parameter between the first cluster center t and the first pixel information m1.

[0049] Exemplarily, the first cluster center is selected. Random selection, K-means++ algorithm or other heuristic methods can be used to initialize the position of the first cluster center. Generally, the first cluster center should cover different areas in the image to better capture the characteristics of the image, so that for each pixel, information about its neighboring pixels is collected. This information usually includes the gray value or other characteristics of the pixel.

[0050] Exemplarily, the distance between each adjacent pixel information and the current cluster center is calculated. This can be Euclidean distance, Manhattan distance or other suitable distance measurement. The distance information is used to measure the proximity of each pixel to the cluster center. For each adjacent pixel, the first pixel difference between its grayscale value and its central pixel (or reference pixel) is calculated. This first pixel difference reflects the grayscale difference between pixels.

[0051] Exemplarily, the first distance information, the first pixel difference and the first association information are fused using weighted average, linear combination or other fusion techniques to obtain a first fuzzy parameter for data clustering, and then the first fuzzy parameter is used to calculate a first association degree of each pixel to the first cluster center, where the first association degree reflects the degree to which it belongs to a first cluster center, wherein the first association degree can be obtained according to the following formula:

[0052]

[0053] in, represents the first correlation degree between the first pixel information m1 and the first cluster center n1, represents the number of clusters of the first cluster center, represents the pixel value corresponding to the first pixel information m1, represents the pixel value corresponding to the first cluster center n1, represents the pixel value corresponding to the first cluster center t, represents the first fuzzy parameter between the first cluster center n1 and the first pixel information m1, Represents the first fuzzy parameter between the first cluster center t and the first pixel information m1.

[0054] The formula enables the clustering algorithm to be adaptively adjusted according to the characteristics of the image by introducing the first fuzzy parameter. For example, the first correlation degree from the pixel to the cluster center can be dynamically adjusted according to the change of the image content, which improves the flexibility of the algorithm. In addition, using the first fuzzy parameter to calculate the first correlation degree from the pixel to the first cluster center can improve the flexibility and accuracy of clustering.

[0055] Exemplarily, the position of the first cluster center is updated according to the calculated first correlation degree of each pixel. The updating method may include recalculating the weighted average of each first cluster center, where the weight is determined by the first correlation degree, so as to check whether the updated first cluster center meets the preset conditions, such as whether a significant change occurs, or whether a specified number of iterations is reached. If the conditions are met, the updating process is stopped, otherwise the iteration continues.

[0056] Exemplarily, the updated first clustering center is used to perform pixel clustering on the entire first image, and each pixel is then assigned to a cluster corresponding to the first clustering center that is closest to the pixel, thereby generating a first clustering result based on the pixel allocation.

[0057] Specifically, by calculating the distance between adjacent pixel information and the cluster center, as well as the pixel difference, the location of the cluster center can be determined more accurately. This method takes into account the influence of neighboring pixels and helps to find a cluster center that is more consistent with the image data. Fusion of distance information, pixel difference, and association information to determine the fuzzy parameter can more finely adjust the first degree of association and enhance the flexibility of clustering. This optimization can handle fuzzy boundaries and overlapping areas in the image and improve the accuracy of clustering. Especially in the case of uneven pixel distribution or noise, it can better reflect the actual category of the pixel. Thereby, the accuracy of the clustering results can be effectively improved.

[0058] In some embodiments, constructing a second fuzzy parameter according to the second association information, and performing pixel clustering on the second grayscale image according to the second fuzzy parameter to obtain a second clustering result includes: determining a second cluster center, calculating second distance information between the second adjacent pixel information and the second cluster center; calculating a second pixel difference between the second adjacent pixel information and the second pixel information; fusing the second distance information, the second pixel difference and the second association information to determine the second fuzzy parameter corresponding to the second cluster center; determining a second degree of association between the second pixel information and the second cluster center according to the second fuzzy parameter; updating the second cluster center according to the second degree of association until the updated second cluster center meets a second preset condition; performing pixel clustering on the second grayscale image according to the updated second cluster center to obtain the second clustering result; wherein the second degree of association is obtained according to the following formula:

[0059]

[0060] in, represents the second correlation degree between the second pixel information m2 and the second cluster center n2, P represents the number of clusters of the second cluster center, represents the pixel value corresponding to the second pixel information m2, represents the pixel value corresponding to the second cluster center n2, represents the pixel value corresponding to the second cluster center L, represents the second fuzzy parameter between the second cluster center n2 and the second pixel information m2, Represents the second blur parameter between the second cluster center L and the second pixel information m2.

[0061] Exemplarily, a second cluster center is selected. Random selection, K-means++ algorithm or other heuristic methods can be used to initialize the position of the second cluster center. Generally, the second cluster center should cover different areas in the image to better capture the characteristics of the image, so that for each pixel, information about its neighboring pixels is collected. This information usually includes the gray value or other characteristics of the pixel.

[0062] Exemplarily, the distance between each adjacent pixel information and the second cluster center is calculated. This can be Euclidean distance, Manhattan distance or other suitable distance measurement. The distance information is used to measure the proximity of each pixel to the cluster center. For each adjacent pixel, the second pixel difference between its grayscale value and its central pixel (or reference pixel) is calculated. This second pixel difference reflects the grayscale difference between pixels.

[0063] Exemplarily, the second distance information, the second pixel difference and the second association information are fused using weighted average, linear combination or other fusion techniques to obtain a second fuzzy parameter for data clustering, and then the second fuzzy parameter is used to calculate the second association degree of each pixel to the second cluster center, where the second association degree reflects the degree to which it belongs to a second cluster center, wherein the second association degree can be obtained according to the following formula:

[0064]

[0065] in, represents the second correlation degree between the second pixel information m2 and the second cluster center n2, P represents the cluster number of the second cluster center, represents the pixel value corresponding to the second pixel information m2, represents the pixel value corresponding to the second cluster center n2, represents the pixel value corresponding to the second cluster center L, represents the second fuzzy parameter between the second cluster center n2 and the second pixel information m2, Represents the second fuzzy parameter between the second cluster center L and the second pixel information m2.

[0066] The formula enables the clustering algorithm to be adaptively adjusted according to the characteristics of the image by introducing the second fuzzy parameter. For example, the second correlation degree from the pixel to the second cluster center can be dynamically adjusted according to the change of the image content, which improves the flexibility of the algorithm. In addition, the use of the second fuzzy parameter to calculate the second correlation degree from the pixel to the second cluster center can improve the flexibility and accuracy of clustering.

[0067] Exemplarily, the position of the second cluster center is updated according to the calculated second association degree of each pixel. The updating method may include recalculating the weighted average of each second cluster center, where the weight is determined by the second association degree, so as to check whether the updated second cluster center meets the preset conditions, such as whether a significant change occurs, or whether a specified number of iterations is reached. If the conditions are met, the updating process is stopped, otherwise the iteration continues.

[0068] Exemplarily, the updated second clustering center is used to perform pixel clustering on the entire second image, and each pixel is then assigned to the cluster corresponding to the second clustering center that is closest to it, thereby generating a second clustering result based on the pixel allocation.

[0069] Specifically, by calculating the distance between adjacent pixel information and the cluster center, as well as the pixel difference, the location of the cluster center can be determined more accurately. This method takes into account the influence of neighboring pixels and helps to find a cluster center that is more consistent with the image data. By fusing distance information, pixel difference and association information to determine the fuzzy parameters, the second degree of association can be adjusted more finely, enhancing the flexibility of clustering. This optimization can handle fuzzy boundaries and overlapping areas in the image and improve the accuracy of clustering. Especially in the case of uneven pixel distribution or noise, it can better reflect the actual category of the pixel. Thereby, the accuracy of the clustering results can be effectively improved.

[0070] In some embodiments, determining the first abnormal information corresponding to the target component based on the first target information and the second target information includes: determining the first position information corresponding to the first target information in the first image, and determining the second position information corresponding to the second target information in the second image; aligning the first image and the second image to obtain an alignment rule; performing data processing on the first position information and the second position information according to the alignment rule to obtain third position information and fourth position information; determining the proportion of identical information based on the third position information and the fourth position information, and determining the first abnormal information corresponding to the target component based on the proportion of identical information.

[0071] Exemplarily, the first position information of the first target information in the first image is determined using the coordinate system calibration. Similarly, the second position information of the second target information in the first image is determined using the coordinate system calibration.

[0072] Exemplarily, a feature point matching method (such as SIFT, SURF, ORB, etc.) is used to find corresponding feature points in the first image and the second image, and then the transformation matrix for image alignment is calculated based on the feature point matching results. Common transformations include affine transformation, perspective transformation, etc. Among them, the transformation matrix can be calculated using the least squares method, RANSAC algorithm, etc.

[0073] Exemplarily, the first image and the second image are aligned according to the calculated transformation matrix. The aligned images make the same target in the same spatial position in different images, and then according to the alignment rule (transformation matrix), the first position information is converted into a fixed coordinate system to obtain the third position information. Similarly, the second position information is converted into a fixed coordinate system to obtain the fourth position information.

[0074] Exemplarily, the third position information and the fourth position information are compared to determine the overlap between them. The area of ​​the overlapping area, the matching rate, etc. can be calculated, and then the proportion of identical information can be calculated based on the matching and comparison results. That is, the proportion of identical information in the first target information and the second target information. If the proportion of identical information is greater than a certain threshold, it may indicate that there is an abnormality in the target component, which in turn leads to the existence of the same abnormal target in different target plates. The first abnormal information may be the absence, misalignment, deformation, etc. of the target component, which leads to the existence of the same abnormal target in different target plates.

[0075] Specifically, by determining the first position information of the first target information in the first image and the second position information of the second target information in the second image, the target object can be effectively identified and located, providing a reliable basis for subsequent analysis. By aligning the first image and the second image, the visual difference caused by the difference in perspective, position or scaling can be eliminated. The aligned images make the same target in the corresponding position, avoiding misjudgment caused by position change. This can enhance the credibility of the subsequent analysis results, and then apply the alignment rules to process the position information, so that the first position information and the second position information can be clearly compared to achieve effective comparison. The third position information and the fourth position information obtained will be more consistent, reducing potential alignment errors. Finally, according to the proportion of the same information, the state of the target object can be quantitatively analyzed. This quantitative information helps to evaluate the similarity and overlap between targets, facilitating further decision-making.

[0076] Step S103: performing abnormal data identification on the first sensor data to obtain second sensor data, and screening from the initial sensors according to the second sensor data to obtain second abnormal information.

[0077] Exemplarily, a machine learning method such as isolation forest, clustering-based method, density-based method, etc. is used to identify abnormal data on the first sensor data to obtain the second sensor data. Thus, the target sensor corresponding to the second sensor data is determined from the initial sensor, and then the monitoring position corresponding to the target sensor and the monitoring device corresponding to the monitoring position are obtained, so that the monitoring position and the monitoring device are determined as the second abnormal information.

[0078] In some embodiments, the performing abnormal data identification on the first sensor data to obtain the second sensor data includes: determining a normal sequence and an abnormal sequence corresponding to the initial sensor, and determining a window length; segmenting the normal sequence, the abnormal sequence, and the first sensor data according to the window length to obtain a first number of segments, a second number of segments, and a third number of segments, respectively; determining the minimum value among the first number of segments, the second number of segments, and the third number of segments as a target number of segments; determining a target sequence length according to the target number of segments and the window length, and obtaining first sequence information corresponding to the target sequence length from the normal sequence, obtaining second sequence information corresponding to the target sequence length from the abnormal sequence, and obtaining third sequence information corresponding to the target sequence length from the first sensor data; calculating first distance information between the first sequence information and the second sequence information, and determining an abnormal threshold according to the first distance information; calculating second distance information between the third sequence information and the first sequence information and the second sequence information, and performing abnormal value judgment on the first sensor data according to the second distance information and the abnormal threshold to determine the second sensor data.

[0079] Exemplarily, the sensor historical data is analyzed to obtain normal sequences and abnormal sequences corresponding to the initial sensor, and the window length is set based on the frequency of the sensor data and the time scale of the desired analysis. The window length may depend on the sampling frequency of the data and the duration of the pattern to be analyzed.

[0080] Exemplarily, the normal sequence, abnormal sequence and first sensor data are segmented using a set window length to obtain a first number of segments corresponding to the normal sequence, a second number of segments corresponding to the abnormal sequence and a third number of segments corresponding to the first sensor data, and then the minimum value among the first number of segments, the second number of segments and the third number of segments is determined and set as the target number of segments.

[0081] Exemplarily, the target sequence length is obtained by multiplying the target number of segments and the window length, and then data blocks that meet the target sequence length are extracted from the normal sequence, the abnormal sequence and the first sensor data according to the target sequence length, thereby obtaining the first sequence information corresponding to the normal sequence, the second sequence information corresponding to the abnormal sequence, and the third sequence information corresponding to the first sensor data.

[0082] Exemplarily, a distance metric such as Euclidean distance, cosine similarity, etc. is calculated between the data blocks of the normal sequence and the abnormal sequence, that is, the first distance information between the first sequence information and the second sequence information is calculated, and then the first distance information can help determine the degree of difference between the abnormal sequence and the normal sequence. Thus, the minimum value of the first distance information is determined as the abnormal threshold.

[0083] Exemplarily, the distance information between the third sequence information and the first sequence information is calculated, and the distance information between the third sequence information and the second sequence information is calculated, and the maximum value of the distance information is obtained as the second distance information. When the second distance information is greater than the abnormal threshold, it is determined that the sequence information when the second distance information is obtained is the first sequence information, and the third sequence information is determined to be a normal sequence; when it is determined that the sequence information when the second distance information is obtained is the second sequence information, the third sequence information is determined to be an abnormal sequence, and then when the third sequence information is an abnormal sequence, the abnormal sequence is determined as the second sensor data.

[0084] Specifically, in order to avoid misjudgment due to inconsistent data length, the length of all data sequences should be ensured to be consistent. By setting a reasonable window length, the normal sequence, abnormal sequence and second sensor data are uniformly segmented to ensure that they are compared and analyzed under the same data length. Then, according to the characteristics of the actual data and the analysis requirements, the target sequence length is calculated, and sequence information with the same length is extracted from the normal sequence, abnormal sequence and second sensor data. This ensures that all data blocks are compared under the same analysis conditions, reducing the impact of length differences on abnormal judgment results. Thus, by comparing the first sensor data with the normal sequence and the abnormal sequence, the deviation from the normal sequence and the abnormal sequence can be accurately identified, thereby better identifying abnormal situations.

[0085] Step S104: determining corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information.

[0086] Exemplarily, the first abnormal information and the second abnormal information are integrated with relevant data of the equipment in the target production line (such as equipment number, location, function, etc.), so as to determine the abnormal equipment corresponding to the first abnormal information and the second abnormal information from the target production line.

[0087] In some embodiments, determining the corresponding abnormal equipment in the target production line based on the first abnormal information and the second abnormal information includes: obtaining a first abnormal position corresponding to the target component based on the first abnormal information; obtaining a second abnormal position corresponding to abnormal sensor monitoring from the second abnormal information; and determining the corresponding abnormal equipment in the target production line based on the first abnormal position and the second abnormal position.

[0088] Exemplarily, the first abnormal position of the target component is extracted from the first abnormal information. Generally, the first abnormal position can be a description of the position of the abnormality in the target component or the relative position of the target component in the target production line that generates the abnormality. For example, if the first abnormal information shows that the position of a component on the target production line is abnormal, the position can be a specific station or position number.

[0089] Exemplarily, the second abnormal position during monitoring by the abnormal sensor is extracted from the second abnormal information. The second abnormal position is usually related to the area monitored by the abnormal sensor, and may include the specific position or monitoring area of ​​the abnormal sensor.

[0090] Exemplarily, a layout diagram or equipment distribution diagram of the target production line is obtained. This includes the physical location of the equipment, the location of the components, and the distribution of the sensors. The first abnormal location is matched with the production line layout to determine the specific location of the component in the target production line, thereby finding its exact location in the target production line based on the component number or location description. The second abnormal location is then matched with the sensor location in the production line layout to confirm the area or location monitored by the abnormal sensor.

[0091] Exemplarily, the abnormal device is determined based on the correspondence between the first abnormal position and the second abnormal position. For example, assuming that the first abnormal position is a position on the production line, and the second abnormal position is an area monitored by a sensor adjacent thereto. Combining these two positions, the specific position of the abnormal device can be inferred.

[0092] Specifically, accurate positioning of abnormal equipment reduces unnecessary inspections and attempts, thereby reducing manpower and time costs while reducing unnecessary spare parts replacement costs and other expenses, reducing maintenance costs. Through automated abnormality identification and positioning, the uncertainty of human factors is reduced, and the working reliability and stability of the entire system are improved, thereby helping industrial enterprises improve production efficiency, reduce costs, and ensure product quality and equipment safety.

[0093] Step S105: Generate an abnormal picture corresponding to the abnormal device according to the first abnormal information and the second abnormal information.

[0094] Exemplarily, the first abnormality information is used to query the abnormality type according to the mapping table, so as to determine the first abnormality type corresponding to the target component, and then generate the first abnormality picture corresponding to the target component according to the first abnormality type.

[0095] Exemplarily, the second abnormal information is classified according to the machine learning model to obtain the monitoring component corresponding to the abnormal sensor, and then the second abnormal type corresponding to the monitoring component is determined, and then the second abnormal image corresponding to the monitoring component is generated according to the second abnormal type.

[0096] In some embodiments, generating an abnormal picture corresponding to the abnormal device based on the first abnormal information and the second abnormal information includes: determining a target abnormal position corresponding to the first image or the second image based on the proportion of the same information, and determining a first abnormal type corresponding to the abnormal device based on the target abnormal position; performing abnormal classification on the second sensor data to obtain a second abnormal type corresponding to the abnormal device; determining a normal picture corresponding to the abnormal device, adjusting the normal picture based on the first abnormal type and the second abnormal type, and obtaining the abnormal picture corresponding to the abnormal device.

[0097] Exemplarily, the target abnormal position is determined based on the proportion of the same information. Assuming that the area or position with the same information proportion is more likely to be the key position of the abnormality, it is marked as the target abnormal position, and then the target abnormal position is mapped to the device, and the abnormal situation at the position is analyzed to determine the first abnormal type. For example, if the target abnormal position is on a certain component, the abnormal type (such as mechanical failure, electrical problem, etc.) can be determined based on the function and status of the component.

[0098] Exemplarily, a classification algorithm or rule is used to classify the anomalies in the sensor data into different types. A machine learning model (such as a decision tree, a random forest) or a rule-based classification method can be used to identify the anomaly type (such as overheating, vibration anomaly, etc.). Thus, based on the classification result, the second anomaly type of the abnormal device is determined.

[0099] Exemplarily, a normal picture corresponding to the device is determined as a reference image. The normal picture should clearly show the internal state or appearance state of the device in a normal state, and then the normal picture is adjusted according to the first abnormal type and the second abnormal type, for example, the abnormal area is marked on the normal picture. Color marking, box selection tools, etc. can be used to add these features to the normal picture according to the characteristics of the abnormal type (such as cracks, discoloration, warning signs), simulate abnormal conditions, and then adjust the details in the normal picture, for example, increase noise, change color, add icons, etc., so as to more realistically reflect the abnormal state, so that the processed picture is an abnormal picture, reflecting the internal state or appearance state of the device under specific abnormal conditions.

[0100] Specifically, by analyzing the abnormality type and adjusting the normal picture, the generated abnormal picture can intuitively display the abnormal state of the equipment, thereby ensuring that non-professionals have a clear understanding of the fault situation. This provides support for the subsequent solution to the problem that when using data twin models for equipment monitoring in related technologies, non-professionals can only know that there is abnormal data but cannot specifically understand the abnormal state of the equipment, which reduces the non-professionals' response ability and processing efficiency to abnormal situations.

[0101] Step S106: construct a digital twin of the target production line according to the abnormal image to obtain a real state model corresponding to the target production line.

[0102] Exemplarily, the abnormal pictures are carefully analyzed to identify the equipment and abnormal parts in the pictures, and then a 3D model of the equipment is created using 3D modeling software based on the information in the abnormal pictures. These 3D models will represent the appearance and structure of the equipment in an abnormal state. A basic model of the target production line is established in the digital twin platform, including 3D models of all equipment, components, and production environments. The 3D model created from the abnormal picture is then integrated into the digital twin model of the target production line. This step involves replacing the equipment model in the abnormal state with the production line model to reflect the actual abnormal situation.

[0103] Exemplarily, the equipment parts and positions corresponding to the abnormal pictures are found in the digital twin model. The corresponding equipment parts in the digital twin model of the target production line are replaced with the 3D model created from the abnormal pictures. This replacement step will enable the digital twin model to accurately display the abnormal state. By replacing the equipment model, the digital twin model of the target production line is updated to a real state model including the abnormal state.

[0104] For example, based on the process, equipment and panel size data, a model library of panels and equipment corresponding to the target production line can be constructed. Then, the relationship between the models in the model library is effectively marked using a graphic database, thereby forming a 3D model library and model integration module that is easy to quickly retrieve and reuse. This process maps the processed data with the digital twin model to achieve simulation analysis of the entire production process of the panel. At the same time, the equipment part and position corresponding to the abnormal image are determined, and on this basis, the 3D model created from the abnormal image is used to replace the corresponding equipment parts in the digital twin model of the target production line. Through this replacement step, the digital twin model can accurately display the abnormal status of the equipment. This update will make the digital twin model of the target production line more realistically reflect the actual situation.

[0105] In some embodiments, the digital twin of the target production line is constructed according to the abnormal image to obtain a real state model corresponding to the target production line, including: obtaining an initial digital model corresponding to all target devices in the target production line, and obtaining a normal digital model corresponding to the abnormal device from the initial digital model; correcting the normal digital model of the abnormal device according to the abnormal image to obtain a target digital model corresponding to the abnormal device; initially building the target production line according to the initial digital model to obtain an initial state model corresponding to the target production line; displaying the target digital model at a position corresponding to the abnormal device in the target production line to obtain the real state model corresponding to the target production line.

[0106] For example, detailed data of all target equipment in the target production line is obtained, including the size, shape, components and functions of the equipment, and then the initial digital model of all equipment is created or imported using computer-aided design (CAD) software or 3D modeling tools. Ensure that the model reflects the appearance and structure of the equipment in normal working condition.

[0107] Exemplarily, the normal digital model corresponding to the abnormal device is extracted from the created initial digital model, so as to carefully analyze the abnormal image and identify the specific features of the abnormal device in the abnormal state, such as cracks, deformation, component damage, etc., so as to modify the normal digital model according to the features in the abnormal image. This includes adding or adjusting abnormal features in the model to accurately reflect the abnormal state of the device. For example, the digital model is modified using 3D modeling software, for example, by adding defects, changing color or shape to simulate abnormal conditions.

[0108] Exemplarily, in the digital twin model of the target production line initially constructed, the location of the abnormal equipment is found, and then the corrected target digital model (reflecting the abnormal state) is replaced at the location of the abnormal equipment in the target production line, and it is ensured that the target digital model is correctly displayed in the production line model and can accurately represent the abnormal state, and then the digital twin model of the production line including the target digital model of the abnormal equipment is integrated into a complete real state model.

[0109] Specifically, by extracting the normal digital model of the abnormal equipment from the initial digital model and then correcting it based on the abnormal picture, the equipment failure or abnormality can be accurately identified and located. This helps to quickly and accurately find the problem points in the production line. With an accurate digital model, maintenance personnel or non-professionals can understand the specific situation of the abnormal equipment more quickly, so as to formulate a more effective maintenance plan. This reduces the troubleshooting time, improves the efficiency of fault handling, and can monitor the status of the production line in real time, timely discover and deal with potential problems, and improve the stability and efficiency of the production line. It can also significantly improve the operating efficiency, fault handling speed and maintenance quality of the production line, while reducing operating costs.

[0110] The embodiment of the present invention provides a method and system for constructing a digital twin of a panel production line. The method includes: obtaining a first image corresponding to a first target plate and a second image corresponding to a second target plate in a target production line, and collecting first sensor data of multiple initial sensors in the target production line, and then performing target extraction on the first image to obtain first target information and performing target extraction on the second image to obtain second target information. Thus, based on the first target information and the second target information, the first abnormal information of the target component is determined. Then, the first sensor data is identified for abnormal data to obtain second sensor data. Then, according to the second sensor data, the initial sensor data is screened to obtain second abnormal information. Then, according to the first abnormal information and the second abnormal information, the abnormal equipment in the target production line is determined, and an abnormal picture reflecting the abnormal state is generated, and the digital twin model of the production line is updated using these abnormal pictures. Thus, a digital twin model is constructed according to the updated abnormal picture to obtain a real state model, ensuring that the real state model accurately reflects the actual operating state and abnormal conditions of the target production line, and the constructed real state model is helpful for non-professionals to monitor and manage the production line in real time. By integrating images and sensor data, the method can accurately identify equipment abnormalities and improve the efficiency of fault detection and maintenance. Accurate digital twin models improve production management, help optimize production processes, reduce costs and enhance safety. Updated digital twin models allow non-professionals to more clearly understand the abnormal state of the equipment, thereby improving their ability to respond to abnormal situations and their efficiency in handling them. This method also effectively solves the problem that non-professionals cannot understand the specific abnormal state of the equipment when using digital twin models for equipment monitoring, and improves the monitoring and management capabilities of the overall production line.

[0111] See also Figure 2 , Figure 2A digital twin construction system 200 for a panel production line provided in an embodiment of the present application includes a data acquisition module 201, an abnormality recognition module 202, an abnormality screening module 203, an equipment determination module 204, an image generation module 205, and a model construction module 206, wherein the data acquisition module 201 is used to obtain a first image corresponding to a first target plate obtained under a target production line and a second image corresponding to a second target plate obtained under the target production line, and obtain first sensor data corresponding to data detection by multiple initial sensors in the target production line; the abnormality recognition module 202 is used to perform target extraction on the first image to obtain first target information and to perform target extraction on the second image to obtain second target information information, and determines the first abnormal information corresponding to the target component according to the first target information and the second target information; an abnormal screening module 203, used to identify the abnormal data of the first sensor data to obtain the second sensor data, and screen from the initial sensor according to the second sensor data to obtain the second abnormal information; an equipment determination module 204, used to determine the corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information; an image generation module 205, used to generate an abnormal picture corresponding to the abnormal equipment according to the first abnormal information and the second abnormal information; a model construction module 206, used to construct a digital twin of the target production line according to the abnormal picture, and obtain a real state model corresponding to the target production line.

[0112] In some implementations, the abnormality identification module 202 performs, during the process of extracting the target from the first image to obtain the first target information and extracting the target from the second image to obtain the second target information:

[0113] Performing grayscale processing on the first image to obtain a first grayscale image, and performing noise reduction processing on the first grayscale image to obtain a first noise-reduced image;

[0114] Performing consistency calculation on the first grayscale image and the first denoised image to determine first related information corresponding to different processing when adjacent coordinates in the first image are processed;

[0115] Obtaining first adjacent pixel information corresponding to any first pixel information in the first grayscale image, and determining second related information corresponding to the same processing at different coordinates in the first image according to the first pixel information and the first adjacent pixel information;

[0116] Determine first association information between adjacent pixels in the first image according to the first relevant information and the second relevant information;

[0117] constructing a first fuzzy parameter according to the first association information, and performing pixel clustering on the first grayscale image according to the first fuzzy parameter to obtain a first clustering result;

[0118] Extracting a target from the first image according to the first clustering result to obtain the first target information;

[0119] Performing grayscale processing on the second image to obtain a second grayscale image, and performing noise reduction processing on the second grayscale image to obtain a second noise-reduced image;

[0120] performing consistency calculation on the second grayscale image and the second denoised image to determine third related information corresponding to different processing when adjacent coordinates in the second image are processed;

[0121] Obtaining second adjacent pixel information corresponding to any second pixel information in the second grayscale image, and determining fourth related information corresponding to the same processing at different coordinates in the second image according to the second pixel information and the second adjacent pixel information;

[0122] Determine second association information between adjacent pixels in the second image according to the third relevant information and the fourth relevant information;

[0123] constructing a second fuzzy parameter according to the second association information, and performing pixel clustering on the second grayscale image according to the second fuzzy parameter to obtain a second clustering result;

[0124] Perform target extraction on the second image according to the second clustering result to obtain the second target information.

[0125] In some implementations, the abnormality identification module 202, in the process of constructing the first fuzzy parameter according to the first association information, and clustering pixels of the first grayscale image according to the first fuzzy parameter to obtain the first clustering result, executes:

[0126] Determine a first cluster center, and calculate first distance information between the first adjacent pixel information and the first cluster center;

[0127] Calculating a first pixel difference between the first adjacent pixel information and the first pixel information;

[0128] Determine the first fuzzy parameter corresponding to the first cluster center by fusing the first distance information, the first pixel difference and the first association information;

[0129] Determining a first degree of association between the first pixel information and the first cluster center according to the first fuzzy parameter;

[0130] Updating the first cluster center according to the first association degree until the updated first cluster center meets a first preset condition;

[0131] Performing pixel clustering on the first grayscale image according to the updated first cluster center to obtain the first clustering result;

[0132] The first correlation degree is obtained according to the following formula:

[0133]

[0134] in, represents the first degree of association between the first pixel information m1 and the first cluster center n1, represents the number of clusters of the first cluster center, represents the pixel value corresponding to the first pixel information m1, represents the pixel value corresponding to the first cluster center n1, represents the pixel value corresponding to the first cluster center t, represents the first fuzzy parameter between the first cluster center n1 and the first pixel information m1, Represents the first blur parameter between the first cluster center t and the first pixel information m1.

[0135] In some implementations, the abnormality identification module 202, in the process of constructing the second fuzzy parameter according to the second association information, and performing pixel clustering on the second grayscale image according to the second fuzzy parameter to obtain the second clustering result, executes:

[0136] Determine a second cluster center, and calculate second distance information between the second adjacent pixel information and the second cluster center;

[0137] Calculating a second pixel difference between the second adjacent pixel information and the second pixel information;

[0138] Determine the second fuzzy parameter corresponding to the second cluster center by fusing the second distance information, the second pixel difference and the second association information;

[0139] determining a second degree of association between the second pixel information and the second cluster center according to the second fuzzy parameter;

[0140] Updating the second cluster center according to the second association degree until the updated second cluster center meets a second preset condition;

[0141] Performing pixel clustering on the second grayscale image according to the updated second cluster center to obtain the second clustering result;

[0142] The second correlation degree is obtained according to the following formula:

[0143]

[0144] in, represents the second correlation degree between the second pixel information m2 and the second cluster center n2, P represents the number of clusters of the second cluster center, represents the pixel value corresponding to the second pixel information m2, represents the pixel value corresponding to the second cluster center n2, represents the pixel value corresponding to the second cluster center L, represents the second fuzzy parameter between the second cluster center n2 and the second pixel information m2, Represents the second blur parameter between the second cluster center L and the second pixel information m2.

[0145] In some implementations, the abnormality identification module 202, in the process of determining the first abnormality information corresponding to the target component according to the first target information and the second target information, performs:

[0146] Determine first position information corresponding to the first target information in the first image, and determine second position information corresponding to the second target information in the second image;

[0147] Performing image alignment on the first image and the second image to obtain an alignment rule;

[0148] Performing data processing on the first position information and the second position information according to the alignment rule to obtain third position information and fourth position information;

[0149] The proportion of identical information is determined according to the third position information and the fourth position information, and the first abnormal information corresponding to the target component is determined according to the proportion of identical information.

[0150] In some implementations, the abnormality screening module 203 performs, during the process of identifying abnormal data on the first sensor data to obtain the second sensor data:

[0151] Determine a normal sequence and an abnormal sequence corresponding to the initial sensor, and determine a window length;

[0152] Segmenting the normal sequence, the abnormal sequence, and the first sensor data according to the window length to obtain a first number of segments, a second number of segments, and a third number of segments;

[0153] Determine a minimum value among the first segment number, the second segment number, and the third segment number and set it as a target segment number;

[0154] Determine a target sequence length according to the target segment number and the window length, and obtain first sequence information corresponding to the target sequence length from the normal sequence, obtain second sequence information corresponding to the target sequence length from the abnormal sequence, and obtain third sequence information corresponding to the target sequence length from the first sensor data;

[0155] Calculating first distance information between the first sequence information and the second sequence information, and determining an abnormality threshold according to the first distance information;

[0156] The second distance information between the third sequence information and the first sequence information and the second sequence information is calculated, and an abnormal value judgment is performed on the first sensor data according to the second distance information and the abnormal threshold to determine the second sensor data.

[0157] In some implementations, the equipment determination module 204, in the process of determining the corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information, performs:

[0158] Obtaining a first abnormal position corresponding to the target component according to the first abnormal information;

[0159] Obtaining a second abnormal position corresponding to monitoring by the abnormal sensor from the second abnormal information;

[0160] The abnormal device corresponding to the target production line is determined according to the first abnormal position and the second abnormal position.

[0161] In some implementations, the image generation module 205, during the process of generating the abnormal image corresponding to the abnormal device according to the first abnormal information and the second abnormal information, performs:

[0162] Determine a target abnormal position corresponding to the first image or the second image according to the proportion of the same information, and determine a first abnormal type corresponding to the abnormal device according to the target abnormal position;

[0163] Performing abnormal classification on the second sensor data to obtain a second abnormal type corresponding to the abnormal device;

[0164] Determine a normal picture corresponding to the abnormal device, adjust the normal picture according to the first abnormal type and the second abnormal type, and obtain the abnormal picture corresponding to the abnormal device.

[0165] In some implementations, the model building module 206 performs, during the process of building a digital twin of the target production line according to the abnormal image to obtain a real state model corresponding to the target production line:

[0166] Obtaining initial digital models corresponding to all target devices in the target production line, and obtaining normal digital models corresponding to the abnormal devices from the initial digital models;

[0167] Correcting the normal digital model of the abnormal device according to the abnormal image to obtain a target digital model corresponding to the abnormal device;

[0168] Initially construct the target production line according to the initial digital model to obtain an initial state model corresponding to the target production line;

[0169] The target digital model is displayed at a position corresponding to the abnormal equipment in the target production line to obtain the real state model corresponding to the target production line.

[0170] In some embodiments, the panel production line digital twin construction system 200 can be applied to terminal equipment.

[0171] It should be noted that technical personnel in the relevant field can clearly understand that, for the convenience and conciseness of description, the specific working process of the panel production line digital twin construction system 200 described above can refer to the corresponding process in the aforementioned panel production line digital twin construction method embodiment, and will not be repeated here.

[0172] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of any method for constructing a digital twin of a panel production line provided in the description of the embodiment of the present invention.

[0173] The storage medium may be an internal storage unit of the terminal device in the aforementioned embodiment, such as a hard disk or memory of the terminal device. The storage medium may also be an external storage device of the terminal device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal device.

[0174] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those skilled in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0175] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0176] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for constructing a digital twin of a panel production line, characterized in that: The method comprises: Obtaining a first image corresponding to a first target plate obtained under a target production line and a second image corresponding to a second target plate obtained under the target production line, and obtaining first sensor data corresponding to data detection by a plurality of initial sensors in the target production line; Performing target extraction on the first image to obtain first target information and performing target extraction on the second image to obtain second target information, and determining first abnormal information corresponding to a target component according to the first target information and the second target information; Performing abnormal data identification on the first sensor data to obtain second sensor data, and screening from the initial sensor data according to the second sensor data to obtain second abnormal information; Determine the corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information; Generate an abnormal picture corresponding to the abnormal device according to the first abnormal information and the second abnormal information; Constructing a digital twin of the target production line according to the abnormal image to obtain a real state model corresponding to the target production line; Wherein, determining the first abnormal information corresponding to the target component according to the first target information and the second target information includes: Determine first position information corresponding to the first target information in the first image, and determine second position information corresponding to the second target information in the second image; Performing image alignment on the first image and the second image to obtain an alignment rule; Performing data processing on the first position information and the second position information according to the alignment rule to obtain third position information and fourth position information; Determine a proportion of identical information according to the third position information and the fourth position information, and determine the first abnormal information corresponding to the target component according to the proportion of identical information; The step of constructing a digital twin of the target production line according to the abnormal image to obtain a real state model corresponding to the target production line includes: Obtaining initial digital models corresponding to all target devices in the target production line, and obtaining normal digital models corresponding to the abnormal devices from the initial digital models; Correcting the normal digital model of the abnormal device according to the abnormal image to obtain a target digital model corresponding to the abnormal device; Initially construct the target production line according to the initial digital model to obtain an initial state model corresponding to the target production line; The target digital model is displayed at a position corresponding to the abnormal equipment in the target production line to obtain the real state model corresponding to the target production line.

2. The method according to claim 1, characterized in that The step of extracting a target from the first image to obtain first target information and extracting a target from the second image to obtain second target information comprises: Performing grayscale processing on the first image to obtain a first grayscale image, and performing noise reduction processing on the first grayscale image to obtain a first noise-reduced image; Performing consistency calculation based on the first grayscale image and the first denoised image to determine first relevant information corresponding to adjacent coordinates in the first image under different processing; Obtaining first adjacent pixel information corresponding to any first pixel information in the first grayscale image, and determining second related information corresponding to different coordinates in the first image under the same processing according to the first pixel information and the first adjacent pixel information; Determine first association information between adjacent pixels in the first image according to the first relevant information and the second relevant information; constructing a first fuzzy parameter according to the first association information, and performing pixel clustering on the first grayscale image according to the first fuzzy parameter to obtain a first clustering result; Extracting a target from the first image according to the first clustering result to obtain the first target information; Performing grayscale processing on the second image to obtain a second grayscale image, and performing noise reduction processing on the second grayscale image to obtain a second noise-reduced image; Perform consistency calculation based on the second grayscale image and the second denoised image to determine third related information corresponding to adjacent coordinates in the second image under different processing; Obtaining second adjacent pixel information corresponding to any second pixel information in the second grayscale image, and determining fourth related information corresponding to different coordinates in the second image under the same processing according to the second pixel information and the second adjacent pixel information; Determine second association information between adjacent pixels in the second image according to the third relevant information and the fourth relevant information; constructing a second fuzzy parameter according to the second association information, and performing pixel clustering on the second grayscale image according to the second fuzzy parameter to obtain a second clustering result; Perform target extraction on the second image according to the second clustering result to obtain the second target information.

3. The method according to claim 2, characterized in that The constructing a first fuzzy parameter according to the first association information, and performing pixel clustering on the first grayscale image according to the first fuzzy parameter to obtain a first clustering result includes: Determine a first cluster center, and calculate first distance information between the first adjacent pixel information and the first cluster center; Calculating a first pixel difference between the first adjacent pixel information and the first pixel information; Determine the first fuzzy parameter corresponding to the first cluster center by fusing the first distance information, the first pixel difference and the first association information; Determining a first degree of association between the first pixel information and the first cluster center according to the first fuzzy parameter; Updating the first cluster center according to the first association degree until the updated first cluster center meets a first preset condition; Performing pixel clustering on the first grayscale image according to the updated first cluster center to obtain the first clustering result; The first correlation degree is obtained according to the following formula: ； in, represents the first degree of association between the first pixel information m1 and the first cluster center n1, represents the number of clusters of the first cluster center, represents the pixel value corresponding to the first pixel information m1, represents the pixel value corresponding to the first cluster center n1, represents the pixel value corresponding to the first cluster center t, represents the first fuzzy parameter between the first cluster center n1 and the first pixel information m1, represents the first fuzzy parameter between the first cluster center t and the first pixel information m1, represents the difference information between the first pixel information m1 and the first cluster center n1, Represents the difference information between the first pixel information m1 and the first cluster center t.

4. The method according to claim 2, characterized in that: The constructing a second fuzzy parameter according to the second association information, and performing pixel clustering on the second grayscale image according to the second fuzzy parameter to obtain a second clustering result includes: Determine a second cluster center, and calculate second distance information between the second adjacent pixel information and the second cluster center; Calculating a second pixel difference between the second adjacent pixel information and the second pixel information; Determine the second fuzzy parameter corresponding to the second cluster center by fusing the second distance information, the second pixel difference and the second association information; determining a second degree of association between the second pixel information and the second cluster center according to the second fuzzy parameter; Updating the second cluster center according to the second association degree until the updated second cluster center meets a second preset condition; Performing pixel clustering on the second grayscale image according to the updated second cluster center to obtain the second clustering result; The second correlation degree is obtained according to the following formula: ； in, represents the second correlation degree between the second pixel information m2 and the second cluster center n2, P represents the number of clusters of the second cluster center, represents the pixel value corresponding to the second pixel information m2, represents the pixel value corresponding to the second cluster center n2, represents the pixel value corresponding to the second cluster center L, represents the second fuzzy parameter between the second cluster center n2 and the second pixel information m2, represents the second fuzzy parameter between the second cluster center L and the second pixel information m2, represents the difference information between the second pixel information m2 and the second cluster center n2, Represents the difference information between the second pixel information m2 and the second cluster center L.

5. The method according to claim 1, characterized in that The performing abnormal data identification on the first sensor data to obtain second sensor data includes: Determine a normal sequence and an abnormal sequence corresponding to the initial sensor, and determine a window length; Segmenting the normal sequence, the abnormal sequence, and the first sensor data according to the window length to obtain a first number of segments, a second number of segments, and a third number of segments; Determine a minimum value among the first segment number, the second segment number, and the third segment number and set it as a target segment number; Determine a target sequence length according to the target segment number and the window length, and obtain first sequence information corresponding to the target sequence length from the normal sequence, obtain second sequence information corresponding to the target sequence length from the abnormal sequence, and obtain third sequence information corresponding to the target sequence length from the first sensor data; Calculating first distance information between the first sequence information and the second sequence information, and determining an abnormality threshold according to the first distance information; The second distance information between the third sequence information and the first sequence information and the second sequence information is calculated, and an abnormal value judgment is performed on the first sensor data according to the second distance information and the abnormal threshold to determine the second sensor data.

6. The method according to claim 1, characterized in that The determining the corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information includes: Obtaining a first abnormal position corresponding to the target component according to the first abnormal information; Obtaining a second abnormal position corresponding to monitoring by the abnormal sensor from the second abnormal information; The abnormal device corresponding to the target production line is determined according to the first abnormal position and the second abnormal position.

7. The method according to claim 1, characterized in that The generating an abnormal picture corresponding to the abnormal device according to the first abnormal information and the second abnormal information includes: Determine a target abnormal position corresponding to the first image or the second image according to the proportion of the same information, and determine a first abnormal type corresponding to the abnormal device according to the target abnormal position; Performing abnormal classification on the second sensor data to obtain a second abnormal type corresponding to the abnormal device; Determine a normal picture corresponding to the abnormal device, adjust the normal picture according to the first abnormal type and the second abnormal type, and obtain the abnormal picture corresponding to the abnormal device.

8. A digital twin construction system for a panel production line, characterized in that: include: A data acquisition module, used to obtain a first image corresponding to a first target plate obtained under a target production line and a second image corresponding to a second target plate obtained under the target production line, and obtain first sensor data corresponding to data detection by a plurality of initial sensors in the target production line; An abnormality identification module is used to extract the target from the first image to obtain the first target information and to extract the target from the second image to obtain the second target information, and to determine the first abnormality information corresponding to the target component according to the first target information and the second target information; wherein, the determining the first abnormality information corresponding to the target component according to the first target information and the second target information includes: determining the first position information corresponding to the first target information in the first image, and determining the second position information corresponding to the second target information in the second image; aligning the first image and the second image to obtain an alignment rule; performing data processing on the first position information and the second position information according to the alignment rule to obtain third position information and fourth position information; determining the proportion of identical information according to the third position information and the fourth position information, and determining the first abnormality information corresponding to the target component according to the proportion of identical information; an abnormality screening module, used for performing abnormal data identification on the first sensor data to obtain second sensor data, and screening from the initial sensor data according to the second sensor data to obtain second abnormality information; An equipment determination module, configured to determine a corresponding abnormal equipment in the target production line according to the first abnormal information and the second abnormal information; An image generation module, configured to generate an abnormal image corresponding to the abnormal device according to the first abnormal information and the second abnormal information; A model construction module is used to construct a digital twin of the target production line according to the abnormal image to obtain a real state model corresponding to the target production line, wherein the digital twin construction of the target production line according to the abnormal image to obtain the real state model corresponding to the target production line includes: obtaining an initial digital model corresponding to all target devices in the target production line, and obtaining a normal digital model corresponding to the abnormal device from the initial digital model; correcting the normal digital model of the abnormal device according to the abnormal image to obtain a target digital model corresponding to the abnormal device; initially building the target production line according to the initial digital model to obtain an initial state model corresponding to the target production line; displaying the target digital model at a position corresponding to the abnormal device in the target production line to obtain the real state model corresponding to the target production line.

Images