Industrial data analysis method and device, electronic equipment and medium

Abstract

The invention discloses an industrial data analysis method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining a plurality of pieces of physical quantity data outputted by an industrial equipment unit of a to-be-analyzed target, carrying out the image splicing of the plurality of pieces of physical quantity data outputted by each industrial equipment unit, and obtaining a plurality of pieces of physical quantity data; the industrial data matrixes obtained through image splicing form an industrial data matrix sequence, and the target to be analyzed is one industrial device or an industrial device cluster composed of at least two industrial devices or at least two industrial device clusters; and analyzing the industrial data matrix sequence by using an industrial data analysis model to obtain an industrial data analysis result of the to-be-analyzed target, the industrial data analysis model being obtained by pre-training a pre-constructed visual model. According to the industrial data analysis method and device, the electronic equipment and the computer readable storage medium, industrial data analysis can be efficiently carried out, the universality is high, and the universality is high.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of industrial big data, and in particular, to an industrial data analysis method, device, electronic device, and computer-readable storage medium. Background Art

[0002] Industrial data is various data generated by industrial equipment during operation, covering equipment operation, production processes, quality control, energy consumption, etc. There is great value in this data. By analyzing industrial data, it can provide strong support in aspects such as optimizing production processes, improving product quality, reducing costs, and innovating business models.

[0003] However, industrial data has its unique complexity. First, the scale of industrial data is usually extremely large. With the popularization of the industrial Internet of Things, a large number of devices are connected to the network, generating massive amounts of data every moment. Second, the sources of industrial data are extensive and diverse. This diversity makes it extremely difficult to integrate and uniformly process and analyze the data. Based on the characteristics of industrial data above, there is currently no method with strong generality and high universality that can efficiently perform industrial data analysis. Summary of the Invention

[0004] The embodiments of the present application provide an industrial data analysis method, device, electronic device, and computer-readable storage medium, which can achieve the purpose of efficiently performing industrial data analysis with strong universality and high generality.

[0005] On the one hand, the embodiments of the present application provide an industrial data analysis method, including:

[0006] Obtain multiple physical quantity data output by industrial equipment units of the target to be analyzed, perform image stitching on the multiple physical quantity data output by each industrial equipment unit, and form an industrial data matrix sequence from the industrial data matrices obtained by image stitching, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipments or at least two industrial equipment clusters;

[0007] Use an industrial data analysis model to analyze the industrial data matrix sequence to obtain an industrial data analysis result for the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed vision model.

[0008] Optionally, when the target to be analyzed is an industrial equipment, the industrial equipment unit is a sensor, and the performing image stitching on the multiple physical quantity data output by each industrial equipment unit includes:

[0009] Obtain the first physical quantity data among the multiple physical quantity data output by each sensor, create a first pixel block corresponding to the first physical quantity data, and use the first pixel block as the center to splice each pixel block corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block.

[0010] Optionally, the splicing of each pixel block corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block includes:

[0011] Splice the pixel block corresponding to the second physical quantity data among the multiple physical quantity data directly above the first pixel block, splice the pixel block corresponding to the third physical quantity data among the multiple physical quantity data on the right side of the first pixel block, splice the pixel block corresponding to the fourth physical quantity data among the multiple physical quantity data directly below the first pixel block, splice the pixel block corresponding to the fifth physical quantity data among the multiple physical quantity data on the left side of the first pixel block, splice the pixel block corresponding to the sixth physical quantity data among the multiple physical quantity data in the upper right of the first pixel block, splice the pixel block corresponding to the seventh physical quantity data among the multiple physical quantity data in the lower right of the first pixel block, splice the pixel block corresponding to the eighth physical quantity data among the multiple physical quantity data in the lower left of the first pixel block, and splice the pixel block corresponding to the ninth physical quantity data among the multiple physical quantity data in the upper left of the first pixel block.

[0012] Optionally, the splicing of each pixel block corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block includes:

[0013] Splice the pixel block corresponding to the second physical quantity data among the multiple physical quantity data directly above the first pixel block, splice the pixel block corresponding to the third physical quantity data among the multiple physical quantity data on the right side of the first pixel block, splice the pixel block corresponding to the fourth physical quantity data among the multiple physical quantity data directly below the first pixel block, splice the pixel block corresponding to the fifth physical quantity data among the multiple physical quantity data on the left side of the first pixel block, splice the pixel block corresponding to the sixth physical quantity data among the multiple physical quantity data in the upper right of the first pixel block, splice the pixel block corresponding to the seventh physical quantity data among the multiple physical quantity data in the lower right of the first pixel block, splice the pixel block corresponding to the eighth physical quantity data among the multiple physical quantity data in the lower left of the first pixel block, and splice the pixel block corresponding to the ninth physical quantity data among the multiple physical quantity data in the upper left of the first pixel block.

[0014] Optionally, the industrial equipment is any one of PCB board production line equipment, new material smelting furnace, and new energy battery production line equipment.

[0015] Optionally, the vision model is TimeSformer or a hybrid architecture of 3D-CNN and Transformer.

[0016] Optionally, after performing image stitching on the multiple physical quantity data output by each industrial equipment unit, the method further includes:

[0017] Visually displaying the stitched physical quantity data.

[0018] On the one hand, an embodiment of the present application further provides an industrial data analysis device, including:

[0019] An acquisition module, configured to acquire multiple physical quantity data output by industrial equipment units of a target to be analyzed, perform image stitching on the multiple physical quantity data output by each industrial equipment unit, and form an industrial data matrix sequence from the industrial data matrices obtained by image stitching, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipment or at least two industrial equipment clusters;

[0020] An analysis module, configured to analyze the industrial data matrix sequence by using an industrial data analysis model to obtain an industrial data analysis result of the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed vision model.

[0021] On the one hand, an embodiment of the present application further provides an electronic device, including: a memory and a processor; the memory stores executable program code; the processor coupled to the memory calls the executable program code stored in the memory to execute the industrial data analysis method provided in the above embodiment.

[0022] On the one hand, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the industrial data analysis method provided in the above embodiment is implemented.

[0023] As can be seen from the above embodiments of the present application, by obtaining multiple physical quantity data output by the industrial equipment units of the target to be analyzed, image stitching is performed on the multiple physical quantity data output by each industrial equipment unit, and the industrial data matrices obtained by image stitching are formed into an industrial data matrix sequence, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipment or at least two industrial equipment clusters. An industrial data analysis model is used to analyze the industrial data matrix sequence to obtain an industrial data analysis result for the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed vision model. The embodiments of the present application perform image stitching on the physical quantity data corresponding to the industrial equipment units of the target to be analyzed, reflecting the characteristics of most industrial data in the time dimension and space dimension, realizing the unification of spatio-temporal modeling, being applicable to most industrial data, and then using the vision model architecture for analysis, directly reusing the existing architecture, reducing the cost of custom-developing new models in the industrial field, achieving the purpose of improving the accuracy and efficiency of industrial data analysis, and having strong universality and high generality. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0025] Figure 1 It is a flowchart of the implementation of the industrial data analysis method provided by an embodiment of the present application;

[0026] Figure 2 It is a schematic structural diagram of the industrial data analysis device provided by an embodiment of the present application;

[0027] Figure 3 It is a schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0028] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts fall within the scope of protection of the present invention.

[0029] See Figure 1 , Figure 1The flowchart shows the implementation of an industrial data analysis method provided by an embodiment of this application. The method described in this embodiment can be applied to an electronic device, which can be a terminal device or a cloud server. As Figure 1 shown, the method specifically includes:

[0030] Step S11: Obtain multiple physical quantity data output by industrial equipment units of the target to be analyzed, perform image stitching on the multiple physical quantity data output by each industrial equipment unit, and form an industrial data matrix sequence with the industrially data matrices obtained by image stitching. Herein, the target to be analyzed is an industrial equipment, or an industrial equipment cluster composed of at least two industrial equipments, or at least two industrial equipment clusters.

[0031] In this embodiment, the target to be analyzed is an industrial equipment; or, the target to be analyzed is an industrial equipment cluster composed of at least two industrial equipments, and the industrial equipment cluster composed of the at least two industrial equipments is also called a production line; or the target to be analyzed is at least two industrial equipment clusters (i.e., multiple production lines. For example, the target to be analyzed is multiple production lines of a certain factory).

[0032] Specifically, when the target to be analyzed is an industrial equipment, the industrial equipment unit is a sensor on the industrial equipment. When the target to be analyzed is an industrial equipment cluster, the industrial equipment unit is a single industrial equipment in the industrial equipment cluster. When the target to be analyzed is at least two industrial equipment clusters, the industrial equipment unit is a single industrial equipment cluster in the at least two industrial equipment clusters.

[0033] Further, in an optional embodiment of the present invention, the industrial equipment is any one of PCB board production line equipment, new material smelting furnace, and new energy battery production line equipment.

[0034] Specifically, when the target to be analyzed is an industrial equipment, obtain multiple physical quantity data output by each sensor installed on the industrial equipment; perform image stitching on the multiple physical quantity data output by each sensor, and a grid image is obtained after stitching. Specifically, the image data after stitching can be represented in the form of an industrial data matrix. Specifically, the industrial data matrix can have various forms. For example, an industrial data matrix includes the stitching of multiple physical quantity data output by a sensor at a certain moment, or an industrial data matrix can include the stitching of multiple physical quantity data corresponding to multiple sensors at a certain moment.

[0035] Specifically, when the target to be analyzed is an industrial equipment cluster, that is, when the target to be analyzed is a certain production line, specifically obtain the multiple physical quantity data output by each sensor installed on each industrial equipment in the industrial equipment cluster; represent the multiple physical quantity data output by an industrial equipment (i.e., the multiple physical quantities output by the sensors in the industrial equipment) with a pixel block of a certain size (such as a 3*3 pixel block); take each industrial equipment as a pixel block, and perform image stitching between the pixel blocks. After stitching, a grid image is obtained. Specifically, the stitched image data can be represented in the form of an industrial data matrix. Specifically, the industrial data matrix can include the stitching of the multiple physical quantity data corresponding to multiple industrial equipments in an industrial equipment cluster at a certain moment.

[0036] Specifically, when the target to be analyzed is at least two industrial equipment clusters, that is, when the target to be analyzed is at least two production lines, specifically obtain the multiple physical quantity data output by each sensor installed on each industrial equipment in the industrial equipment cluster; represent the multiple physical quantity data output by an industrial equipment cluster (i.e., the multiple physical quantities output by the sensors in a single industrial equipment within the industrial equipment cluster) with a pixel block of a certain size; take each industrial equipment cluster as a pixel block, and perform image stitching between the pixel blocks. After stitching, a grid image is obtained. Specifically, the stitched image data can be represented in the form of an industrial data matrix. Specifically, the industrial data matrix can include the stitching of the multiple physical quantity data corresponding to multiple industrial equipments in multiple industrial equipment clusters at a certain moment.

[0037] In this embodiment, performing image stitching means mapping the physical quantity data to pixel blocks and stitching between the pixel blocks.

[0038] Further, in an optional embodiment of the present invention, after obtaining the physical quantity data, perform independent normalization or standardization processing on each physical quantity data, so that the numerical ranges of all physical quantity data are within a reasonable interval (such as 0-1 or standard normal distribution).

[0039] Further, in an optional embodiment of the present invention, after obtaining the physical quantity data, clean the data to remove obvious noise and outliers. And perform interpolation processing on missing values (such as linear interpolation or model-based interpolation) to ensure the continuity of the data.

[0040] Further, in an optional embodiment of the present invention, after obtaining the physical quantity data, perform interpolation or downsampling on the sensor data with different sampling frequencies to ensure that all sensor data are aligned in the time dimension.

[0041] Further, in an optional embodiment of the present invention, when the target to be analyzed is an industrial device, the industrial device unit is a sensor, and the graphical stitching of the multiple physical quantity data output by each industrial device unit includes:

[0042] Obtain the first physical quantity data among the multiple physical quantity data output by each sensor, create a first pixel block corresponding to the first physical quantity data, and use the first pixel block as the center to stitch the respective pixel blocks corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block.

[0043] In this embodiment, when performing graphical stitching, a corresponding pixel block is created for each physical quantity data to form a grid structure of the image. For example, a sensor outputs less than 10 physical quantity data, one physical quantity data occupies one pixel (each pixel is 4 bytes), an enhancement intensity variable Q is introduced and default value is 1, and the pixel block corresponding to each physical quantity data is a Q*Q pixel block. If the enhancement intensity Q = 3, then the pixel block corresponding to each physical quantity data is a 3*3 pixel block, where one physical quantity repeatedly occupies the 3*3 pixel block, and each pixel in the pixel block is equal to this physical quantity.

[0044] Specifically, using the first pixel block as the center to stitch the respective pixel blocks corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block includes: using the first pixel block as the center and stitching the pixel blocks corresponding to the other physical quantity data around the first pixel block in a clockwise or counterclockwise direction in sequence.

[0045] In this embodiment, the other physical quantity data refers to the physical quantity data other than the first physical quantity data. The graphical stitching is the stitching of multiple physical quantity data in one sensor. In other optional embodiments, the multiple physical quantity data corresponding to multiple sensors can also be stitched.

[0046] Further, in an optional embodiment of the present invention, the stitching of the respective pixel blocks corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block includes:

[0047] The pixel block corresponding to the second physical quantity data among the multiple physical quantity data is spliced ​​directly above the first pixel block, the pixel block corresponding to the third physical quantity data among the multiple physical quantity data is spliced ​​to the right of the first pixel block, the pixel block corresponding to the fourth physical quantity data among the multiple physical quantity data is spliced ​​directly below the first pixel block, the pixel block corresponding to the fifth physical quantity data among the multiple physical quantity data is spliced ​​to the left of the first pixel block, the pixel block corresponding to the sixth physical quantity data among the multiple physical quantity data is spliced ​​to the upper right of the first pixel block, the pixel block corresponding to the seventh physical quantity data among the multiple physical quantity data is spliced ​​to the lower right of the first pixel block, the pixel block corresponding to the eighth physical quantity data among the multiple physical quantity data is spliced ​​to the lower left of the first pixel block, and the pixel block corresponding to the ninth physical quantity data among the multiple physical quantity data is spliced ​​to the upper left of the first pixel block.

[0048] For example, different physical quantity data of a sensor are centered according to the pixel block corresponding to the first physical quantity data, the pixel block corresponding to the second physical quantity data is spliced ​​above the pixel block corresponding to the first physical quantity data, the pixel block corresponding to the third physical quantity data is spliced ​​to the right of the pixel block corresponding to the first physical quantity data, the pixel block corresponding to the fourth physical quantity data is below the pixel block corresponding to the first physical quantity data, the fifth physical quantity data is on the left of the pixel block corresponding to the first physical quantity data, the sixth physical quantity data is on the upper right of the pixel block corresponding to the first physical quantity data, the seventh physical quantity data is on the lower right of the pixel block corresponding to the first physical quantity data, the eighth physical quantity data is on the lower left of the pixel block corresponding to the first physical quantity data, and the ninth physical quantity data is on the upper left of the pixel block corresponding to the first physical quantity data, forming a 3*3 square matrix. If the number of physical quantity data after splicing is less than 3*3, that is, the total number of physical quantity data after splicing is less than 9, the remaining part is filled with blank pixels. For example, fill the 3×3 pixel blocks in the aforementioned order (such as temperature→(0,0), pressure→(0,1)), and fill the insufficient positions with zeros. Specifically, a sensor image matrix parameter S is set, where S is an odd number greater than or equal to 3. The default value of S is 3. If the sensor physical quantity is greater than 9, the parameter is set to 5 or greater, such as 5*5=25 physical quantities. If S is greater than 3, the image stitching method is the same as above.

[0049] In this embodiment, when an industrial device is the target to be analyzed, the physical quantity data output by each sensor in the device can be spliced ​​in a graphical manner.

[0050] In this embodiment, when an industrial equipment cluster (i.e., a certain production line) is the target to be analyzed, the physical quantity data output by each industrial equipment is regarded as a whole, and an industrial equipment with multiple sensors is used as a pixel block for image splicing. At this time, a pixel block of a certain size (for example, a 3*3 pixel block) represents the multiple physical quantity data output by an industrial equipment (the splicing of the pixels in the pixel block can refer to the image splicing of the physical quantity data of the sensors in the industrial equipment, or can also be spliced in a clockwise or counterclockwise manner, etc.), and then each pixel block is spliced. That is, the different sensors of the industrial equipment in an industrial equipment cluster are similar to the splicing of the pixel blocks of the different physical quantity data in the sensors. The device image matrix parameter can be set to D, and the splicing algorithm of the pixel block corresponding to the industrial equipment is basically the same as the image splicing of the physical quantity data of the sensors in the industrial equipment. Specifically, for the first industrial equipment in the industrial equipment cluster, create the first device pixel block corresponding to the first industrial equipment, and use the first device pixel block as the center to splice the pixel blocks corresponding to the other industrial equipment in the industrial equipment cluster except the first industrial equipment around the first device pixel block. Specifically, the pixel block corresponding to the second industrial equipment in the other industrial equipment can be spliced directly above the first device pixel block, the pixel block corresponding to the third industrial equipment in the other industrial equipment can be spliced on the right side of the first device pixel block, the pixel block corresponding to the fourth industrial equipment in the other industrial equipment can be spliced directly below the first device pixel block, the pixel block corresponding to the fifth industrial equipment in the other industrial equipment can be spliced on the left side of the first device pixel block, the pixel block corresponding to the sixth industrial equipment in the other industrial equipment can be spliced in the upper right of the first device pixel block, the pixel block corresponding to the seventh industrial equipment in the other industrial equipment can be spliced in the lower right of the first device pixel block, the pixel block corresponding to the eighth industrial equipment in the other industrial equipment can be spliced in the lower left of the first device pixel block, and the pixel block corresponding to the ninth industrial equipment in the other industrial equipment can be spliced in the upper left of the first device pixel block. In specific implementation, if the number of industrial equipment in the industrial equipment cluster is greater than nine, the first device pixel block can also be centered, and the other device pixel blocks can be spliced around the first device pixel block in a clockwise, counterclockwise or other order.

[0051] In this embodiment, when at least two industrial equipment clusters (i.e., multiple production lines in a factory as a whole) are the targets to be analyzed, the industrial equipment in the industrial equipment cluster is regarded as a whole, and an industrial equipment cluster (i.e., a production line) is used as a pixel block for image splicing (the splicing of pixels within the pixel block can refer to the image splicing of the physical quantity data of sensors in the aforementioned industrial equipment, or the image splicing of the pixel block corresponding to the industrial equipment, or can be spliced in a clockwise or counterclockwise manner, etc.). At this time, a pixel block of a certain size (such as a 9*9 pixel block) represents multiple physical quantity data output by a production line, and then each pixel block is spliced. That is, similar to the splicing of pixel blocks of different physical quantity data of sensors in industrial equipment within an industrial equipment cluster among multiple industrial equipment clusters, the device image matrix parameter can be set to M, and the splicing algorithm of the pixel block corresponding to the industrial equipment cluster is basically the same as the image splicing of the physical quantity data of sensors in the industrial equipment. Specifically, for the first industrial equipment cluster among at least two industrial equipment clusters, a first device cluster pixel block corresponding to this industrial equipment cluster is created, and with this first device cluster pixel block as the center, the respective pixel blocks corresponding to the other industrial equipment clusters except the first industrial equipment cluster among at least two industrial equipment clusters are spliced around the first device cluster pixel block. Specifically, the pixel block corresponding to the second industrial equipment cluster in other industrial equipment clusters can be spliced directly above the first device cluster pixel block, the pixel block corresponding to the third industrial equipment cluster in other industrial equipment clusters can be spliced on the right side of the first device cluster pixel block, the pixel block corresponding to the fourth industrial equipment cluster in other industrial equipment clusters can be spliced directly below the first device cluster pixel block, the pixel block corresponding to the fifth industrial equipment cluster in other industrial equipment clusters can be spliced on the left side of the first device cluster pixel block, the pixel block corresponding to the sixth industrial equipment cluster in other industrial equipment clusters can be spliced in the upper right of the first device cluster pixel block, the pixel block corresponding to the seventh industrial equipment cluster in other industrial equipment clusters can be spliced in the lower right of the first device cluster pixel block, the pixel block corresponding to the eighth industrial equipment cluster in other industrial equipment clusters can be spliced in the lower left of the first device cluster pixel block, and the pixel block corresponding to the ninth industrial equipment cluster in other industrial equipment clusters can be spliced in the upper left of the first device cluster pixel block. In specific implementation, if the number of industrial equipment clusters in at least the industrial equipment clusters is greater than nine, the first device cluster pixel block can also be centered, and the other device cluster pixel blocks can be spliced around the first device cluster pixel block in a clockwise or counterclockwise order, etc.

[0052] The splicing method described in this embodiment can expand from the center with blank spaces on both sides, and there will be no interference during splicing, thereby improving data processing efficiency and reducing data errors.

[0053] Since the sensor relationships of industrial devices and industrial device clusters are fixed, and the relationships of different physical quantities of sensors are also fixed, in this embodiment, spatial relationships are directly established by image stitching, which facilitates the discovery of spatial associations and improves the accuracy of analysis.

[0054] Furthermore, in other alternative embodiments, dynamic position encoding can also be performed during stitching, that is, the real coordinates (x, y, z) of the sensors are encoded as position embeddings, and are input into the industrial data analysis model together with the pixel block stitching, so that the model can understand the spatial relationships of the sensors.

[0055] In this embodiment, the industrial data matrix sequence is a set of industrial data matrices at different times.

[0056] In this embodiment, there are spatial relationships (such as up-down position relationships, horizontal position relationships, etc.) between different sensors on an industrial device, and there are temporal relationships between the data output by the same sensor on an industrial device. Similarly, there are spatial relationships between multiple industrial devices in an industrial device cluster, and there are temporal relationships between the data output by multiple industrial devices in an industrial device cluster. Therefore, image stitching of the multiple physical quantity data output by industrial device units can display the correlations and change trends between the physical quantity data of different industrial device units, which is beneficial to better data analysis. Moreover, there is no need to design specialized conversion methods for various different formats, reducing the risk of data loss or distortion and improving the efficiency of data preprocessing.

[0057] Furthermore, in an alternative embodiment of the present invention, after the image stitching of the multiple physical quantity data output by each industrial device unit, the method further includes:

[0058] Visually display the stitched physical quantity data.

[0059] In this embodiment, image stitching of the multiple physical quantity data output by industrial device units on an industrial device can enhance the data visualization effect and is beneficial to intuitive data analysis. For example, after image stitching of sensor data such as voltage and current in an industrial device, the power operation status at different positions can be visually presented, which is convenient for quickly discovering abnormal areas, and can also clearly show the distribution, trend and mutual relationship of the data, which is beneficial to fault warning and optimizing the production process.

[0060] Step S12, analyze the industrial data matrix sequence by using an industrial data analysis model to obtain an industrial data analysis result of the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed visual model.

[0061] Further, in an alternative embodiment of the present invention, the visual model is TimeSformer or a hybrid architecture of 3D-CNN and Transformer. After obtaining the industrial data matrix sequence, the industrial data matrix sequence is directly input into the pre-trained TimeSformer or the pre-trained hybrid architecture of 3D-CNN and Transformer.

[0062] Further, in an alternative embodiment of the present invention, before analyzing the industrial data matrix sequence using the industrial data analysis model, the method includes:

[0063] Obtain a pre-constructed visual model, and the output layer of the visual model is connected to a unit conversion layer;

[0064] Pre-train the pre-constructed visual model using industrial training data to obtain an industrial data analysis model, wherein during training, physical constraint fine-tuning is performed on the pre-constructed visual model.

[0065] In this embodiment, the unit conversion layer is used to convert the original output of the visual model into a numerical value with physical units. Optionally, the unit conversion layer is a linear transformation layer, which can be scaled or offset according to the unit of the physical quantity.

[0066] In this embodiment, the industrial training data can be data of factories, equipment, and production lines. Specifically, it can be obtained through real-time historical data, experimental data, public data (such as UCI Machine Learning Repository), etc. The industrial data types include various sensor data, such as pressure, vibration, current / voltage, flow rate, rotational speed, humidity, etc. Specifically, such as the wind speed, generator rotational speed, gearbox vibration, temperature, current, voltage, etc. of a wind turbine.

[0067] In other alternative embodiments, the industrial training data can also include environmental data (such as environmental temperature / humidity) and equipment operation data (such as operating status).

[0068] Specifically, after obtaining the industrial training data, data cleaning (including denoising, alignment, normalization) and data augmentation (for example, time augmentation such as time warping, random slicing, or spatial augmentation such as noise injection) are performed on the industrial training data. Map multiple physical quantities of each sensor into 3x3 pixel blocks, splice all sensors according to physical positions into a larger image, splice the images of multiple devices according to the production line layout into a global image, and slice all the images into multiple "frames" according to a fixed time window (such as 1 minute) in time series to form a video sequence. Use a sliding window to generate a continuous frame sequence and retain the temporal continuity. Then use these data for pre-training.

[0069] In this embodiment, during training, physical constraint fine-tuning is performed on the pre-constructed vision model. Specifically, a physical constraint term is added to the loss function. For example, the total loss value in the loss function is the sum of the mean squared error loss and the energy conservation error. This enables the vision model to not only minimize the prediction error during training but also minimize the degree of violation of physical laws. Thus, it can ensure that the output of the vision model conforms to physical laws (such as energy conservation, mass conservation, energy balance, linear kinematics, etc.) and industrial laws, improving the accuracy and efficiency of industrial data analysis and avoiding the output of the vision model violating basic physical principles.

[0070] In this embodiment, the training includes two-stage fine-tuning and reinforcement learning optimization. Specifically, in the first stage of the two-stage fine-tuning, it is unconstrained pre-training, and the MSE loss can be used to fit historical data. The second stage of the two-stage fine-tuning is physical constraint fine-tuning. In this embodiment, the model can be further optimized through reinforcement learning to make its output conform to physical laws and make optimal decisions.

[0071] Furthermore, in an optional embodiment of the present invention, the pre-constructed vision model adopts a modular structure. During training, it is trained with industrial training data of different scenarios, and physical constraints corresponding to different application scenarios are dynamically loaded during training. When applying the trained model, the modules of the corresponding model are directly selected for analysis according to the task requirements (i.e., the specific analysis scenario).

[0072] In this embodiment, by training the pre-constructed vision model, the processing of pixels by the large vision model can be directly reused, and the vision model can be migrated into a model capable of processing industrial data matrix sequences, using a set of models to process all industrial data. Furthermore, the method described in this embodiment can also be used for the processing of general tabular structured data. A table is regarded as a sensor, and multiple associated tables are regarded as devices, and the image splicing method of this solution is used for processing.

[0073] In other optional embodiments, before using the industrial training data to pre-train the pre-constructed vision model, the vision model (such as TimeSformer) can also be compressed into a lightweight version using knowledge distillation. This can improve the running efficiency of the model and thus improve the efficiency of industrial data analysis.

[0074] In an alternative embodiment of the present invention, if the task to be analyzed is the equipment health status of the PCB production line equipment or the module and sensor where a positioning fault occurs. Then, obtaining the sensor data of the PCB production line equipment includes temperature (temperature sensor data of the reflow soldering furnace), pressure (pressure sensor data of the mounter), vibration (vibration sensor data during equipment operation), current / voltage (current and voltage data of motors and electrical equipment). Then, these data are graphically stitched together. Specifically, multiple physical quantities (such as temperature, pressure, vibration) of each sensor are mapped to a 3x3 pixel block. Equipment-level mapping: The sensors of each module are stitched together according to their physical positions into a larger image. For example, the sensors of the mounter are stitched together into one image, and the sensors of the reflow soldering furnace are stitched together into another image (this image can be represented in matrix form), and the image is represented as a matrix. The data within a continuous time window is sliced into multiple frames to form a matrix sequence. During training, an unsupervised learning method (such as Masked Autoencoder, MAE) is used to pre-train the vision model using the data of the PCB production line equipment, randomly masking some sensor data and letting the model predict the masked part. Then, the vision model is fine-tuned using labeled data, and the task can be fault classification (such as normal, minor fault, severe fault) or fault prediction (such as predicting whether a fault will occur within the next 24 hours). A physical constraint term, such as energy conservation or temperature change law, is added to the loss function. During application, the matrix sequence is input into the pre-trained model, and the model outputs the equipment health status. Specifically, it can output the health score of the equipment (such as a value between 0 and 1, where 1 represents healthy and 0 represents faulty). Or, if there is a fault in the model, it can output the fault location.

[0075] In an alternative embodiment of the present invention, if the task to be analyzed is the equipment health status of a new material smelting furnace or the modules and sensors where a positioning fault occurs. Then, obtaining the sensor data of the new material smelting furnace includes temperature (temperature sensor data inside and outside the smelting furnace), pressure (furnace internal gas pressure sensor data), vibration (furnace body vibration sensor data), current / voltage (current and voltage data of heating elements), and gas concentration (gas concentration data of oxygen, nitrogen, etc. inside the furnace). Then, these data are graphically stitched together. Specifically, multiple physical quantities (such as temperature, pressure, vibration, gas concentration) of each sensor are mapped to 3x3 pixel blocks, and device-level mapping: the sensors of each module are stitched together according to their physical positions into a larger image. For example, the temperature sensors inside the furnace are stitched together into one image, and the temperature sensors outside the furnace are stitched together into another image (this image can be represented in matrix form), and the image is represented in matrix form. The data within a continuous time window is sliced into multiple frames to form a matrix sequence. During training, an unsupervised learning method (such as MAE) is used to pre-train the vision model using the smelting furnace data, randomly masking part of the sensor data and letting the model predict the masked part. Then, the vision model is fine-tuned using labeled data, and the task can be smelting furnace status classification (such as normal, overheating, abnormal air pressure) or fault prediction (such as predicting whether a fault will occur within the next 1 hour). A physical constraint term, such as the law of conservation of energy or the law of gas concentration change, is added to the loss function. During application, the matrix sequence is input into the pre-trained model, and the model outputs the health status of the smelting furnace. Specifically, it can output a health score of the smelting furnace (such as a value between 0 and 1, where 1 represents healthy and 0 represents faulty). Or, if the model has a fault, it can output the fault location (such as caused by too high temperature inside the furnace).

[0076] In the embodiments of the present application, by obtaining multiple physical quantity data output by the industrial equipment unit of the target to be analyzed, graphically stitching the multiple physical quantity data output by each industrial equipment unit, and forming an industrial data matrix sequence from the industrially data matrices obtained by the graphical stitching, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipments or at least two industrial equipment clusters. The industrial data matrix sequence is analyzed using an industrial data analysis model to obtain an industrial data analysis result for the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed vision model. The embodiments of the present application graphically stitch the physical quantity data corresponding to the industrial equipment unit of the target to be analyzed, reflecting the characteristics of most industrial data in terms of time dimension and space dimension, realizing the unification of spatio-temporal modeling, being applicable to most industrial data, and then using the vision model architecture for analysis, directly reusing the existing architecture, reducing the cost of custom-developing new models in the industrial field, achieving the purpose of improving the accuracy and efficiency of industrial data analysis, and having strong universality and high generality.

[0077] See Figure 2 , a schematic structural diagram of an industrial data analysis device provided by an embodiment of the present application. For ease of description, only parts related to the embodiments of the present application are shown. The device can be arranged in an electronic device. The industrial data analysis device includes:

[0078] An acquisition module 201, configured to acquire a plurality of physical quantity data output by an industrial equipment unit of a target to be analyzed, perform image stitching on the plurality of physical quantity data output by each industrial equipment unit, and form an industrial data matrix sequence from the industrial data matrices obtained by image stitching, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipments or at least two industrial equipment clusters.

[0079] In this embodiment, the target to be analyzed is an industrial equipment; or, the target to be analyzed is an industrial equipment cluster composed of at least two industrial equipments, and the industrial equipment cluster composed of the at least two industrial equipments is also called a production line; or the target to be analyzed is at least two industrial equipment clusters (that is, multiple production lines, for example, the target to be analyzed is multiple production lines of a certain factory).

[0080] Specifically, when the target to be analyzed is an industrial equipment, the industrial equipment unit is a sensor on the industrial equipment. When the target to be analyzed is an industrial equipment cluster, the industrial equipment unit is a single industrial equipment in the industrial equipment cluster. When the target to be analyzed is at least two industrial equipment clusters, the industrial equipment unit is a single industrial equipment cluster in the at least two industrial equipment clusters.

[0081] Further, in an optional embodiment of the present invention, the industrial equipment is any one of PCB board production line equipment, new material smelting furnace, and new energy battery production line equipment.

[0082] Specifically, when the target to be analyzed is an industrial equipment, acquire a plurality of physical quantity data output by each sensor installed on the industrial equipment; perform image stitching on the plurality of physical quantity data output by each sensor, and a grid image is obtained after stitching. Specifically, the stitched image data can be represented in the form of an industrial data matrix. Specifically, the industrial data matrix can have various forms. For example, an industrial data matrix includes the stitching of a plurality of physical quantity data output by a sensor at a certain moment, or an industrial data matrix can include the stitching of a plurality of physical quantity data corresponding to a plurality of sensors at a certain moment.

[0083] Specifically, when the target to be analyzed is an industrial equipment cluster, that is, when the target to be analyzed is a certain production line, specifically obtain the multiple physical quantity data output by each sensor installed on each industrial equipment in the industrial equipment cluster; represent the multiple physical quantity data output by an industrial equipment (i.e., the multiple physical quantities output by the sensors in the industrial equipment) with a pixel block of a certain size (such as a 3*3 pixel block); take each industrial equipment as a pixel block, perform image splicing between the pixel blocks, and after splicing, obtain a grid image. Specifically, the spliced image data can be represented in the form of an industrial data matrix. Specifically, the industrial data matrix can include the splicing of the multiple physical quantity data corresponding to multiple industrial equipment in an industrial equipment cluster at a certain moment.

[0084] Specifically, when the target to be analyzed is at least two industrial equipment clusters, that is, when the target to be analyzed is at least two production lines, specifically obtain the multiple physical quantity data output by each sensor installed on each industrial equipment in the industrial equipment cluster; represent the multiple physical quantity data output by an industrial equipment cluster (i.e., the multiple physical quantities output by the sensors in a single industrial equipment within the industrial equipment cluster) with a pixel block of a certain size; take each industrial equipment cluster as a pixel block, perform image splicing between the pixel blocks, and after splicing, obtain a grid image. Specifically, the spliced image data can be represented in the form of an industrial data matrix. Specifically, the industrial data matrix can include the splicing of the multiple physical quantity data corresponding to multiple industrial equipment in multiple industrial equipment clusters at a certain moment.

[0085] In this embodiment, performing image splicing means mapping the physical quantity data to pixel blocks and splicing between the pixel blocks.

[0086] Further, in an optional embodiment of the present invention, after obtaining the physical quantity data, perform independent normalization or standardization processing on each physical quantity data, so that the numerical ranges of all physical quantity data are within a reasonable interval (such as 0-1 or standard normal distribution).

[0087] Further, in an optional embodiment of the present invention, after obtaining the physical quantity data, clean the data to remove obvious noise and outliers. And perform interpolation processing on missing values (such as linear interpolation or model-based interpolation) to ensure the continuity of the data.

[0088] Further, in an optional embodiment of the present invention, after obtaining the physical quantity data, perform interpolation or downsampling on the sensor data with different sampling frequencies to ensure that all sensor data are aligned in the time dimension.

[0089] Further, in an optional embodiment of the present invention, when the target to be analyzed is an industrial device, the industrial device unit is a sensor, and the graphical stitching of the multiple physical quantity data output by each industrial device unit includes:

[0090] Obtain the first physical quantity data among the multiple physical quantity data output by each sensor, create a first pixel block corresponding to the first physical quantity data, and use the first pixel block as the center to stitch the respective pixel blocks corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block.

[0091] In this embodiment, when performing graphical stitching, a corresponding pixel block is created for each physical quantity data to form a grid structure of the image. For example, if a sensor outputs less than 10 physical quantity data, one physical quantity data occupies one pixel (each pixel is 4 bytes), a reinforcement intensity variable Q is introduced and default value is 1, and the pixel block corresponding to each physical quantity data is a Q*Q pixel block. If the reinforcement intensity Q = 3, then the pixel block corresponding to each physical quantity data is a 3*3 pixel block. Among them, one physical quantity repeatedly occupies a 3*3 pixel block, and each pixel in the pixel block is equal to this physical quantity.

[0092] Specifically, using the first pixel block as the center to stitch the respective pixel blocks corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block includes: using the first pixel block as the center and stitching the pixel blocks corresponding to the other physical quantity data around the first pixel block in a clockwise or counterclockwise direction in sequence.

[0093] In this embodiment, the other physical quantity data refers to the physical quantity data other than the first physical quantity data. The graphical stitching is the stitching of multiple physical quantity data in one sensor. In other optional embodiments, the multiple physical quantity data corresponding to multiple sensors can also be stitched.

[0094] Further, in an optional embodiment of the present invention, the stitching of the respective pixel blocks corresponding to the other physical quantity data among the multiple physical quantity data around the first pixel block includes:

[0095] The pixel block corresponding to the second physical quantity data among the multiple physical quantity data is spliced ​​directly above the first pixel block, the pixel block corresponding to the third physical quantity data among the multiple physical quantity data is spliced ​​to the right of the first pixel block, the pixel block corresponding to the fourth physical quantity data among the multiple physical quantity data is spliced ​​directly below the first pixel block, the pixel block corresponding to the fifth physical quantity data among the multiple physical quantity data is spliced ​​to the left of the first pixel block, the pixel block corresponding to the sixth physical quantity data among the multiple physical quantity data is spliced ​​to the upper right of the first pixel block, the pixel block corresponding to the seventh physical quantity data among the multiple physical quantity data is spliced ​​to the lower right of the first pixel block, the pixel block corresponding to the eighth physical quantity data among the multiple physical quantity data is spliced ​​to the lower left of the first pixel block, and the pixel block corresponding to the ninth physical quantity data among the multiple physical quantity data is spliced ​​to the upper left of the first pixel block.

[0096] For example, different physical quantity data of a sensor are centered according to the pixel block corresponding to the first physical quantity data, the pixel block corresponding to the second physical quantity data is spliced ​​above the pixel block corresponding to the first physical quantity data, the pixel block corresponding to the third physical quantity data is spliced ​​to the right of the pixel block corresponding to the first physical quantity data, the pixel block corresponding to the fourth physical quantity data is below the pixel block corresponding to the first physical quantity data, the fifth physical quantity data is on the left of the pixel block corresponding to the first physical quantity data, the sixth physical quantity data is on the upper right of the pixel block corresponding to the first physical quantity data, the seventh physical quantity data is on the lower right of the pixel block corresponding to the first physical quantity data, the eighth physical quantity data is on the lower left of the pixel block corresponding to the first physical quantity data, and the ninth physical quantity data is on the upper left of the pixel block corresponding to the first physical quantity data, forming a 3*3 square matrix. If the number of physical quantity data after splicing is less than 3*3, that is, the total number of physical quantity data after splicing is less than 9, the remaining part is filled with blank pixels. For example, fill the 3×3 pixel blocks in the aforementioned order (such as temperature→(0,0), pressure→(0,1)), and fill the insufficient positions with zeros. Specifically, a sensor image matrix parameter S is set, where S is an odd number greater than or equal to 3. The default value of S is 3. If the sensor physical quantity is greater than 9, the parameter is set to 5 or greater, such as 5*5=25 physical quantities. If S is greater than 3, the image stitching method is the same as above.

[0097] In this embodiment, when an industrial device is the target to be analyzed, the physical quantity data output by each sensor in the device can be spliced ​​in a graphical manner.

[0098] In this embodiment, when an industrial equipment cluster (i.e., a certain production line) is the target to be analyzed, the physical quantity data output by each industrial equipment is regarded as a whole, and an industrial equipment with multiple sensors is used as a pixel block for image splicing. At this time, a pixel block of a certain size (for example, a 3*3 pixel block) represents the multiple physical quantity data output by an industrial equipment (the splicing of the pixels in the pixel block can refer to the image splicing of the physical quantity data of the sensors in the industrial equipment, or can also be spliced in a clockwise or counterclockwise manner), and then each pixel block is spliced. That is, the different sensors of the industrial equipment in an industrial equipment cluster are similar to the splicing of the pixel blocks of the different physical quantity data in the sensors, and the device image matrix parameter can be set to D. The splicing algorithm of the pixel block corresponding to the industrial equipment is basically the same as the image splicing of the physical quantity data of the sensors in the industrial equipment. Specifically, for the first industrial equipment in the industrial equipment cluster, create the first device pixel block corresponding to the first industrial equipment, and use the first device pixel block as the center to splice the pixel blocks corresponding to the other industrial equipment in the industrial equipment cluster except the first industrial equipment around the first device pixel block. Specifically, the pixel block corresponding to the second industrial equipment in the other industrial equipment can be spliced directly above the first device pixel block, the pixel block corresponding to the third industrial equipment in the other industrial equipment can be spliced on the right side of the first device pixel block, the pixel block corresponding to the fourth industrial equipment in the other industrial equipment can be spliced directly below the first device pixel block, the pixel block corresponding to the fifth industrial equipment in the other industrial equipment can be spliced on the left side of the first device pixel block, the pixel block corresponding to the sixth industrial equipment in the other industrial equipment can be spliced in the upper right of the first device pixel block, the pixel block corresponding to the seventh industrial equipment in the other industrial equipment can be spliced in the lower right of the first device pixel block, the pixel block corresponding to the eighth industrial equipment in the other industrial equipment can be spliced in the lower left of the first device pixel block, and the pixel block corresponding to the ninth industrial equipment in the other industrial equipment can be spliced in the upper left of the first device pixel block. In specific implementation, if the number of industrial equipment in the industrial equipment cluster is greater than nine, the first device pixel block can also be centered, and the other device pixel blocks can be spliced around the first device pixel block in a clockwise or counterclockwise order.

[0099] In this embodiment, when at least two industrial equipment clusters (i.e., multiple production lines in a factory as a whole) are the targets to be analyzed, the industrial equipment in the industrial equipment cluster is regarded as a whole, and an industrial equipment cluster (i.e., a production line) is used as a pixel block for image splicing (the splicing of pixels within the pixel block can refer to the image splicing of the physical quantity data of sensors in the aforementioned industrial equipment, or the image splicing of the pixel block corresponding to the industrial equipment, or it can also be spliced in a clockwise or counterclockwise manner, etc.). At this time, a pixel block of a certain size (such as a 9*9 pixel block) represents multiple physical quantity data output by a production line, and then each pixel block is spliced. That is, similar to the splicing of pixels of different sensors of industrial equipment within an industrial equipment cluster among multiple industrial equipment clusters, the device image matrix parameter can be set to M, and the splicing algorithm of the pixel block corresponding to the industrial equipment cluster is basically the same as the image splicing of the physical quantity data of sensors in the industrial equipment. Specifically, for the first industrial equipment cluster among at least two industrial equipment clusters, create the first device cluster pixel block corresponding to this industrial equipment cluster, take this first device cluster pixel block as the center, and splice each pixel block corresponding to other industrial equipment clusters other than the first industrial equipment cluster among at least two industrial equipment clusters around the first device cluster pixel block. Specifically, the pixel block corresponding to the second industrial equipment cluster in other industrial equipment clusters can be spliced directly above the first device cluster pixel block, the pixel block corresponding to the third industrial equipment cluster in other industrial equipment clusters can be spliced on the right side of the first device cluster pixel block, the pixel block corresponding to the fourth industrial equipment cluster in other industrial equipment clusters can be spliced directly below the first device cluster pixel block, the pixel block corresponding to the fifth industrial equipment cluster in other industrial equipment clusters can be spliced on the left side of the first device cluster pixel block, the pixel block corresponding to the sixth industrial equipment cluster in other industrial equipment clusters can be spliced in the upper right of the first device cluster pixel block, the pixel block corresponding to the seventh industrial equipment cluster in other industrial equipment clusters can be spliced in the lower right of the first device cluster pixel block, the pixel block corresponding to the eighth industrial equipment cluster in other industrial equipment clusters can be spliced in the lower left of the first device cluster pixel block, and the pixel block corresponding to the ninth industrial equipment cluster in other industrial equipment clusters can be spliced in the upper left of the first device cluster pixel block. In specific implementation, if the number of industrial equipment clusters in at least industrial equipment clusters is greater than nine, the first device cluster pixel block can also be centered, and other device cluster pixel blocks can be spliced around the first device cluster pixel block in a clockwise or counterclockwise order, etc.

[0100] The splicing method described in this embodiment can expand from the center with blank spaces on both sides, and there will be no mutual interference during splicing, thereby improving the data processing efficiency and reducing data errors.

[0101] Since the sensor relationships of industrial devices and industrial device clusters are fixed, and the relationships of different physical quantities of sensors are also fixed, in this embodiment, spatial relationships are directly established by image stitching, which is convenient for discovering spatial associations and improving the accuracy of analysis.

[0102] Furthermore, in other alternative embodiments, dynamic position encoding can also be performed during stitching, that is, the real coordinates (x, y, z) of the sensors are encoded into position embeddings, and are input into the industrial data analysis model together with the pixel block stitching.

[0103] In this embodiment, the industrial data matrix sequence is a set of industrial data matrices at different times.

[0104] In this embodiment, there are spatial relationships (such as upper and lower position relationships, horizontal position relationships, etc.) between different sensors on an industrial device, and there are temporal relationships between the data output by the same sensor on an industrial device. Similarly, there are spatial relationships between multiple industrial devices in an industrial device cluster, and there are temporal relationships between the data output by multiple industrial devices in an industrial device cluster. Therefore, image stitching of the multiple physical quantity data output by the industrial device unit can display the correlation and change trend between the physical quantity data of different industrial device units, which is beneficial to better data analysis. Moreover, there is no need to design a dedicated conversion method for various different formats, reducing the risk of data loss or distortion and improving the efficiency of data preprocessing.

[0105] Furthermore, in an alternative embodiment of the present invention, the device further includes a display module, and the display module is configured to: after performing image stitching on the multiple physical quantity data output by each industrial device unit, perform visual display on the stitched physical quantity data.

[0106] In this embodiment, image stitching of the multiple physical quantity data output by the industrial device unit on the industrial device can enhance the data visualization effect and is beneficial to intuitive data analysis. For example, after image stitching of sensor data such as voltage and current in the industrial device, the power operation status at different positions can be visually presented, which is convenient for quickly discovering abnormal areas, and can also clearly show the distribution, trend and mutual relationship of the data, which is beneficial to fault warning and optimizing the production process.

[0107] The analysis module 202 is configured to analyze the industrial data matrix sequence by using an industrial data analysis model to obtain an industrial data analysis result of the target to be analyzed, wherein the industrial data analysis model is obtained by pre-training a pre-constructed vision model.

[0108] Further, in an alternative embodiment of the present invention, the vision model is TimeSformer or a hybrid architecture of 3D-CNN and Transformer. After obtaining the industrial data matrix sequence, the industrial data matrix sequence is directly input into the pre-trained TimeSformer or the pre-trained hybrid architecture of 3D-CNN and Transformer.

[0109] Further, in an alternative embodiment of the present invention, the device further includes a training module, and the training module is used for:

[0110] Before analyzing the industrial data matrix sequence using the industrial data analysis model, obtain a pre-constructed vision model, and the output layer of the vision model is connected to a unit conversion layer;

[0111] Pre-train the pre-constructed vision model using industrial training data to obtain an industrial data analysis model, wherein during training, physical constraint fine-tuning is performed on the pre-constructed vision model.

[0112] In this embodiment, the unit conversion layer is used to convert the original output of the vision model into a numerical value with a physical unit. Optionally, the unit conversion layer is a linear transformation layer, which can be scaled or offset according to the unit of the physical quantity.

[0113] In this embodiment, the industrial training data can be data of factories, equipment, and production lines. Specifically, it can be obtained through real-time historical data, experimental data, public data (such as UCI Machine Learning Repository), etc. The industrial data types include various sensor data, such as pressure, vibration, current / voltage, flow rate, rotational speed, humidity, etc. Specifically, such as the wind speed, generator rotational speed, gearbox vibration, temperature, current, voltage, etc. of a wind turbine.

[0114] In other alternative embodiments, the industrial training data can also include environmental data (such as environmental temperature / humidity) and equipment operation data (such as operating status).

[0115] Specifically, after obtaining the industrial training data, perform data cleaning (including denoising, alignment, normalization) and data augmentation on the industrial training data (for example, perform time augmentation such as time warping, random slicing, or spatial augmentation such as noise injection). Map multiple physical quantities of each sensor into 3x3 pixel blocks, splice all sensors into a larger image according to physical location, splice the images of multiple devices into a global image according to the production line layout, and slice all the images into multiple "frames" according to a fixed time window (such as 1 minute) in time series to form a video sequence, and use a sliding window to generate a continuous frame sequence to retain the continuity in time. Then use these data for pre-training.

[0116] In this embodiment, during training, the pre-constructed vision model is fine-tuned with physical constraints. Specifically, a physical constraint term is added to the loss function. For example, the total loss value in the loss function is the sum of the mean square error loss and the energy conservation error. This enables the vision model to not only minimize the prediction error but also minimize the degree of violation of physical laws during the training process. Thus, it can ensure that the output of the vision model conforms to physical laws (such as energy conservation, mass conservation, energy balance, linear kinematics, etc.) and industrial laws, improving the accuracy and efficiency of industrial data analysis and preventing the output of the vision model from violating basic physical principles.

[0117] In this embodiment, the training includes two-stage fine-tuning and reinforcement learning optimization. Specifically, in the two-stage fine-tuning, the first stage is unconstrained pre-training, and the MSE loss can be used to fit historical data. The second stage in the two-stage fine-tuning is physical constraint fine-tuning. In this embodiment, the model can be further optimized through reinforcement learning to make its output conform to physical laws and make optimal decisions.

[0118] In this embodiment, by training the pre-constructed vision model, the processing of pixels by the large vision model can be directly reused, and the vision model can be migrated into a model capable of processing industrial data matrix sequences, using a set of models to process all industrial data. Further, the method described in this embodiment can also be used for the processing of general tabular structured data. A table is regarded as a sensor, and multiple associated tables are regarded as devices, and the image stitching method of this solution is adopted for processing.

[0119] In other alternative embodiments, before pre-training the pre-constructed vision model using industrial training data, the vision model (such as TimeSformer) can be compressed into a lightweight version using knowledge distillation. This can improve the running efficiency of the model and thus improve the efficiency of industrial data analysis.

[0120] In an alternative embodiment of the present invention, if the task to be analyzed is the equipment health status of the PCB production line equipment or the modules and sensors where the positioning failure occurs. Then, obtaining the sensor data of the PCB production line equipment includes temperature (temperature sensor data of the reflow soldering furnace), pressure (pressure sensor data of the mounter), vibration (vibration sensor data during equipment operation), current / voltage (current and voltage data of motors and electrical equipment). Then, these data are graphically stitched together. Specifically, multiple physical quantities (such as temperature, pressure, vibration) of each sensor are mapped to a 3x3 pixel block, and equipment-level mapping: the sensors of each module are stitched together by physical position into a larger image. For example, the sensors of the mounter are stitched together into one image, and the sensors of the reflow soldering furnace are stitched together into another image (this image can be represented in matrix form), and the image is represented as a matrix. The data within a continuous time window is sliced into multiple frames to form a matrix sequence. During training, an unsupervised learning method (such as Masked Autoencoder, MAE) is used to pre-train the vision model using the data of the PCB production line equipment, randomly masking some of the sensor data and letting the model predict the masked part. Then, the vision model is fine-tuned using labeled data, and the task can be fault classification (such as normal, minor fault, severe fault) or fault prediction (such as predicting whether a fault will occur within the next 24 hours). A physical constraint term, such as energy conservation or temperature change law, is added to the loss function. During application, the matrix sequence is input into the pre-trained model, and the model outputs the equipment health status. Specifically, it can output the health score of the equipment (such as a value between 0 and 1, where 1 represents healthy and 0 represents faulty). Or, if there is a fault in the model, it can output the fault location.

[0121] In an alternative embodiment of the present invention, if the task to be analyzed is the equipment health status of a new material smelting furnace or the module and sensor where a positioning fault occurs, then obtaining the sensor data of the new material smelting furnace includes temperature (temperature sensor data inside and outside the smelting furnace), pressure (pressure sensor data inside the furnace), vibration (vibration sensor data of the furnace body), current / voltage (current and voltage data of the heating element), and gas concentration (gas concentration data of oxygen, nitrogen, etc. inside the furnace). Then, these data are graphically stitched together. Specifically, multiple physical quantities (such as temperature, pressure, vibration, gas concentration) of each sensor are mapped to 3x3 pixel blocks. At the device level, the sensors of each module are stitched together physically to form a larger image. For example, the temperature sensors inside the furnace are stitched together to form one image, and the temperature sensors outside the furnace are stitched together to form another image (which can be represented in matrix form), and the image is represented as a matrix. The data within a continuous time window is sliced into multiple frames to form a matrix sequence. During training, an unsupervised learning method (such as MAE) is used to pre-train the vision model using the smelting furnace data, randomly masking some sensor data and letting the model predict the masked part. Then, the vision model is fine-tuned using labeled data. The task can be smelting furnace state classification (such as normal, overheating, abnormal air pressure) or fault prediction (such as predicting whether a fault will occur within the next 1 hour). A physical constraint term, such as the law of conservation of energy or the change rule of gas concentration, is added to the loss function. During application, the matrix sequence is input into the pre-trained model, and the model outputs the health status of the smelting furnace. Specifically, it can output a health score of the smelting furnace (such as a value between 0 and 1, where 1 represents healthy and 0 represents faulty). Or, if there is a fault, the model can output the fault location (such as caused by too high temperature inside the furnace).

[0122] In the embodiments of the present application, by obtaining multiple physical quantity data output by the industrial equipment unit of the target to be analyzed, graphically stitching the multiple physical quantity data output by each industrial equipment unit, and forming an industrial data matrix sequence from the industrially data matrices obtained by the graphical stitching, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipment or at least two industrial equipment clusters. The industrial data matrix sequence is analyzed using an industrial data analysis model, and an industrial data analysis result of the target to be analyzed is obtained, where the industrial data analysis model is obtained by pre-training a pre-constructed vision model. In the embodiments of the present application, by graphically stitching the physical quantity data corresponding to the industrial equipment unit of the target to be analyzed, the characteristics of most industrial data in the time dimension and space dimension are reflected, the unification of spatio-temporal modeling is realized, it is applicable to most industrial data, and then analyzed using the vision model architecture, directly reusing the existing architecture, reducing the cost of custom-developing new models in the industrial field, achieving the purpose of improving the accuracy and efficiency of industrial data analysis, and having strong universality and high generality.

[0123] See Figure 3 , a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present application.

[0124] Exemplarily, the electronic device may be a cloud server or a terminal computer. In some cases, the electronic device may perform multiple functions.

[0125] As Figure 3 shown, the electronic device 10 may include a control circuit, and the control circuit may include a storage and processing circuit 30. The storage and processing circuit 30 may include a memory, such as a hard disk drive memory, a non-volatile memory (such as a flash memory or other electronically programmable limit erasable memory for forming a solid state drive, etc.), a volatile memory (such as a static or dynamic random access memory, etc.), etc., which are not limited in the embodiments of the present application. The processing circuit in the storage and processing circuit 30 may be used to control the operation of the electronic device 10. The processing circuit may be implemented based on one or more microprocessors, microcontrollers, digital signal processors, baseband processors, power management units, audio codec chips, application specific integrated circuits, display driver integrated circuits, etc.

[0126] The storage and processing circuit 30 may be used to run the software in the electronic device 10. This software may be used to perform some control operations, for example, ambient light measurement based on an ambient light sensor, proximity sensor measurement based on a proximity sensor, information display function implemented based on a status indicator such as an LED, touch event detection based on a touch sensor, functions associated with displaying information on multiple (such as hierarchical) displays, operations associated with performing wireless communication functions, operations associated with collecting and generating audio signals, control operations associated with collecting and processing button press event data, and other functions in the electronic device 10, which are not limited in the embodiments of the present application.

[0127] Furthermore, the memory stores executable program code, and a processor coupled to the memory calls the executable program code stored in the memory and executes the industrial data analysis method described in the embodiment as described above Figure 1 shown.

[0128] Wherein, the executable program code includes each module in the industrial data analysis device described in the embodiment as described above Figure 2 shown, such as: an acquisition module and an analysis module.

[0129] The electronic device 10 may further include an input / output circuit 42. The input / output circuit 42 can be used to enable the electronic device 10 to achieve data input and output, that is, to allow the electronic device 10 to receive data from an external device and also to allow the electronic device 10 to output data from the electronic device 10 to an external device. The input / output circuit 42 may further include a sensor 32. The sensor 32 may include an ambient light sensor, a proximity sensor based on light and capacitance, a touch sensor (e.g., a light-based touch sensor and / or a capacitive touch sensor, where the touch sensor may be a part of a touch display screen or may be used independently as a touch sensor structure), an acceleration sensor, and other sensors, etc.

[0130] The input / output circuit 42 may further include a communication circuit 38. The communication circuit 38 can be used to provide the electronic device 10 with the ability to communicate with an external device. The communication circuit 38 may include analog and digital input / output interface circuits, and a wireless communication circuit based on radio frequency signals and / or optical signals. The wireless communication circuit in the communication circuit 38 may include a radio frequency transceiver circuit, a power amplifier circuit, a low-noise amplifier, switches, filters, and antennas. For example, the wireless communication circuit in the communication circuit 38 may include a circuit for supporting Near Field Communication (NFC) by transmitting and receiving near-field coupled electromagnetic signals. For example, the communication circuit 38 may include a near-field communication antenna and a near-field communication transceiver. The communication circuit 38 may further include a cellular phone transceiver and antenna, a wireless local area network transceiver circuit and antenna, etc.

[0131] The electronic device 10 may further include a battery, a power management circuit, and other input / output units 40. The input / output units 40 may include buttons, joysticks, click wheels, scroll wheels, touch pads, keypads, keyboards, cameras, light-emitting diodes, and other status indicators, etc.

[0132] The user can input commands through the input / output circuit 42 to control the operation of the electronic device 10, and can use the output data of the input / output circuit 42 to receive status information and other outputs from the electronic device 10.

[0133] Furthermore, an embodiment of the present invention further provides a computer-readable storage medium, which may be disposed in the electronic device in the above embodiments. The computer-readable storage medium may be the memory in the storage and processing circuit 30 in the foregoing Figure 3 illustrated embodiments. A computer program is stored on the computer-readable storage medium, and when the program is executed by a processor, it implements the foregoing Figure 1The industrial data analysis method described in the illustrated embodiment. Further, the computer-readable storage medium may also be various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk, or an optical disc.

[0134] It should be noted that, for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0135] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] The above is the description of the industrial data analysis method, device, and computer-readable storage medium provided by the present invention. For those of ordinary skill in the art, according to the idea of the embodiments of the present invention, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An industrial data analysis method, characterized in that, The method includes: Obtaining a plurality of physical quantity data output by industrial equipment units of a target to be analyzed, performing image stitching on the plurality of physical quantity data output by each industrial equipment unit, and forming an industrial data matrix sequence with the industrial data matrices obtained by image stitching, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipment or at least two industrial equipment clusters; Analyzing the industrial data matrix sequence by using an industrial data analysis model to obtain an industrial data analysis result of the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed vision model.

2. The method according to claim 1, wherein When the target to be analyzed is an industrial equipment, the industrial equipment unit is a sensor, and the performing image stitching on the plurality of physical quantity data output by each industrial equipment unit includes: Obtaining first physical quantity data among the plurality of physical quantity data output by each sensor, creating a first pixel block corresponding to the first physical quantity data, and using the first pixel block as the center to stitch pixel blocks corresponding to other physical quantity data among the plurality of physical quantity data around the first pixel block.

3. The method according to claim 2, characterized in that, The stitching the pixel blocks corresponding to other physical quantity data among the plurality of physical quantity data around the first pixel block includes: Stitching the pixel block corresponding to the second physical quantity data among the plurality of physical quantity data directly above the first pixel block, stitching the pixel block corresponding to the third physical quantity data among the plurality of physical quantity data on the right side of the first pixel block, stitching the pixel block corresponding to the fourth physical quantity data among the plurality of physical quantity data directly below the first pixel block, stitching the pixel block corresponding to the fifth physical quantity data among the plurality of physical quantity data on the left side of the first pixel block, stitching the pixel block corresponding to the sixth physical quantity data among the plurality of physical quantity data in the upper right of the first pixel block, stitching the pixel block corresponding to the seventh physical quantity data among the plurality of physical quantity data in the lower right of the first pixel block, stitching the pixel block corresponding to the eighth physical quantity data among the plurality of physical quantity data in the lower left of the first pixel block, and stitching the pixel block corresponding to the ninth physical quantity data among the plurality of physical quantity data in the upper left of the first pixel block.

4. The method according to claim 1, wherein Before the analyzing the industrial data matrix sequence by using the industrial data analysis model, the method includes: Obtaining a pre-constructed vision model, where an output layer of the vision model is connected to a unit conversion layer; Pre-training the pre-constructed vision model by using industrial training data to obtain an industrial data analysis model, where physical constraint fine-tuning is performed on the pre-constructed vision model during training.

5. The method according to any one of claims 1 to 4, characterized in that, The industrial equipment is any one of PCB board production line equipment, new material smelting furnace, and new energy battery production line equipment.

6. The method according to any one of claims 1 to 4, characterized in that The vision model is TimeSformer or a hybrid architecture of 3D-CNN and Transformer.

7. The method according to any one of claims 1 to 4, characterized in that, After the performing image stitching on the plurality of physical quantity data output by each industrial equipment unit, the method further includes: Visually display the spliced physical quantity data.

8. An industrial data analysis device, characterized in that, The device includes: An acquisition module, configured to acquire multiple physical quantity data output by an industrial equipment unit of a target to be analyzed, perform image splicing on the multiple physical quantity data output by each industrial equipment unit, and form an industrial data matrix sequence from the industrially data matrices obtained by image splicing, where the target to be analyzed is an industrial equipment or an industrial equipment cluster composed of at least two industrial equipments or at least two industrial equipment clusters; An analysis module, configured to analyze the industrial data matrix sequence by using an industrial data analysis model to obtain an industrial data analysis result of the target to be analyzed, where the industrial data analysis model is obtained by pre-training a pre-constructed visual model.

9. An electronic device, characterized in that, The electronic device includes: A memory and a processor; The memory stores executable program code; The processor coupled to the memory calls the executable program code stored in the memory and executes the industrial data analysis method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the industrial data analysis method according to any one of claims 1 to 7 is implemented.

Images