Factory equipment anomaly detection method and equipment based on CNN and Transform, and medium

Abstract

The invention discloses a CNN and Transform-based factory equipment anomaly detection method and device, and a medium, and the method comprises the steps: collecting equipment operation information corresponding to factory equipment according to a preset collection interval; wherein the equipment operation information comprises an equipment image, and the acquisition interval is related to the equipment anomaly distribution probability of a workshop area where the factory equipment is located; inputting the equipment image into a preset anomaly detection model, extracting feature information in the equipment image through a CNN (Convolutional Neural Network) in the anomaly detection model, and positioning anomaly features used for representing factory equipment anomaly in the feature information through a Transform self-attention mechanism; decoding the abnormal features through a CNN network to identify an abnormal region in the equipment image; and under the condition that the abnormal area exists in the equipment image, the equipment position of the factory equipment corresponding to the equipment image is acquired, and the equipment position and the abnormal area are reported to an early warning center, so that abnormal early warning of the factory equipment is realized.

Description

Technical Field

[0001] The present application relates to the technical field of equipment detection, and in particular to a method, equipment and medium for detecting abnormalities of factory equipment based on CNN and Transformer. Background Art

[0002] Factory equipment is an important material and technical basis for enterprises to organize production and is one of the important elements of productivity. In order to ensure the normal operation of factory equipment, it is necessary to conduct regular inspections of factory equipment to avoid irreparable losses due to equipment hazards that are not discovered or handled in a timely manner.

[0003] Traditional equipment inspection methods mostly require inspection personnel to patrol and inspect at fixed points regularly. Although this method is relatively sufficient for equipment inspection, due to the long inspection cycle, it is often lax in supervision of emergencies, making it difficult to achieve real-time monitoring and timely maintenance of equipment. In addition to the above methods, sensors can also be deployed to detect equipment abnormalities, but this method has certain limitations and cannot detect surface abnormalities such as oil leakage and damage. Summary of the invention

[0004] In order to solve the above problems, this application proposes a factory equipment anomaly detection method based on CNN and Transformer, including:

[0005] Collecting equipment operation information corresponding to factory equipment at a preset collection interval; wherein the equipment operation information includes equipment images, and the collection interval is related to the equipment abnormality distribution probability of the workshop area where the factory equipment is located;

[0006] Input the device image into a preset anomaly detection model, extract feature information in the device image through a CNN network in the anomaly detection model, and locate abnormal features in the feature information that are used to characterize abnormalities of factory equipment through a Transformer self-attention mechanism;

[0007] Decoding the abnormal features through the CNN network to identify abnormal areas in the device image;

[0008] When there is an abnormal area in the device image, the device location of the factory equipment corresponding to the device image is obtained, and the device location and the abnormal area are reported to the early warning center to implement abnormal early warning of the factory equipment.

[0009] In one implementation of the present application, the abnormal features used to characterize the abnormality of factory equipment in the feature information are located through the Transformer self-attention mechanism, specifically including:

[0010] Reshape the feature information by a preset reshaping function to obtain sequence information corresponding to the feature information, and add a position code corresponding to the position of the element in the sequence information to each element in the sequence information to obtain a corresponding feature sequence;

[0011] The feature sequence is input into a Transformer layer with a Transformer self-attention mechanism, and the abnormal features in the feature map used to characterize the abnormality of the factory equipment are located through the Transformer layer.

[0012] In one implementation of the present application, the abnormal features used to characterize the abnormality of the factory equipment in the feature map are located through the Transformer layer, and the specific calculation method is:

[0013]

[0014] Among them, head i represents the output of the i-th attention head, H is the number of attention heads, z q is the feature representation of the query matrix q, is the normalized three-dimensional coordinate of the reference point, Λ(z q ) ilqk ∈[0,1] represents the attention weight, σ1 rescales To the l-th level feature, Δ pilqk ∈R 3 represents the sampling offset of the kth sampling point at the lth feature level, and φ(…) represents the linear projection layer used to weight and aggregate the feature representations of all attention heads.

[0015] In one implementation of the present application, decoding the abnormal feature through the CNN network to identify the abnormal area in the device image specifically includes:

[0016] Performing transposed convolution on the abnormal feature through the CNN network to upsample the abnormal feature to an original resolution corresponding to the device image, thereby obtaining an upsampled abnormal feature;

[0017] The upsampled abnormal features are refined by residual blocks to obtain refined abnormal features, and semantic segmentation is performed on the refined abnormal features to identify abnormal areas in the device image.

[0018] In one implementation of the present application, before inputting the device image into a preset anomaly detection model, the method further includes:

[0019] Generate a training sample consisting of a plurality of device images; wherein the training sample carries an actual abnormal region and an abnormal region label for marking the actual abnormal region;

[0020] Pre-training the anomaly detection model according to the training samples, so as to output the predicted abnormal regions existing in the training samples during the pre-training process;

[0021] According to the abnormal region label, the actual abnormal region in the training sample is located, and the predicted abnormal region is compared with the actual abnormal region to calculate the corresponding loss value;

[0022] The anomaly detection model is optimized according to the loss value until the prediction area output by the anomaly detection model can meet the preset conditions.

[0023] In one implementation of the present application, reporting the device location and the abnormal area to an early warning center specifically includes:

[0024] Determine the workshop area to which the factory equipment belongs according to the equipment location;

[0025] Summarizing the number of equipment anomalies in the workshop area and the entire workshop respectively, and calculating the distribution probability of equipment anomalies corresponding to the workshop area according to the summarized number of equipment anomalies;

[0026] The equipment abnormality distribution probability is bound to all factory equipment in its corresponding workshop area, and the equipment location, the abnormal area and the equipment abnormality distribution probability are reported to the early warning center.

[0027] In one implementation of the present application, collecting device images corresponding to factory devices at preset collection intervals specifically includes:

[0028] Determine the workshop area where the factory equipment is located, and obtain the historical detection records corresponding to the workshop area, and determine the historical equipment abnormality distribution probability corresponding to the workshop area in the previous collection interval based on the historical detection information;

[0029] Acquire a mapping relationship between the device abnormality distribution probability and the collection interval, and determine the collection interval corresponding to the historical device abnormality distribution probability according to the mapping relationship; wherein the historical device abnormality distribution probability is negatively correlated with the collection interval;

[0030] According to the collection interval, the device image corresponding to the factory equipment is collected.

[0031] In one implementation of the present application, the method further includes:

[0032] Acquire the device sensor data, and match the device sensor data with a corresponding value range to determine whether the device sensor data is within the value range;

[0033] If not, a corresponding warning message is generated, and the warning message and the device location are reported to the warning center.

[0034] The embodiment of the present application provides a factory equipment anomaly detection device based on CNN and Transformer, the device comprising:

[0035] at least one processor;

[0036] and, a memory communicatively coupled to the at least one processor;

[0037] In which, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a factory equipment anomaly detection method based on CNN and Transformer as described in any of the above items.

[0038] The embodiment of the present application provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0039] A factory equipment anomaly detection method based on CNN and Transformer as described in any of the above items.

[0040] The factory equipment anomaly detection method based on CNN and Transformer proposed in this application can bring the following beneficial effects:

[0041] Through the preset collection interval (the interval is dynamically adjusted with the probability of equipment abnormality distribution), the system can collect equipment operation information, especially equipment images, in real time or near real time, greatly shortening the response time of equipment status detection. Compared with the traditional regular inspection method, this method can capture equipment abnormalities more quickly and avoid the problem of lax supervision of emergencies caused by long inspection cycles.

[0042] Using anomaly detection models to extract features and locate anomalies in device images can not only detect surface anomalies that are difficult for sensors to capture, but also accurately locate abnormal areas through image recognition technology. This method is more comprehensive and accurate than simple sensor detection, and improves the reliability of anomaly detection.

[0043] From image acquisition, feature extraction, anomaly location to anomaly warning, the entire process of factory equipment automation and intelligent processing has been achieved, which greatly reduces the burden of manual inspection and improves work efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0045] Figure 1 A flowchart of a method for detecting abnormality of factory equipment based on CNN and Transformer provided in an embodiment of the present application;

[0046] Figure 2 A schematic diagram of the structure of a factory equipment anomaly detection device based on CNN and Transformer provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0048] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0049] like Figure 1 As shown, a factory equipment anomaly detection method based on CNN and Transformer provided in an embodiment of the present application includes:

[0050] S101: Collecting equipment operation information corresponding to factory equipment at a preset collection interval; wherein the equipment operation information includes equipment images, and the collection interval is related to the equipment abnormality distribution probability of the workshop area where the factory equipment is located.

[0051] Multiple workshop areas can be set up inside the workshop according to the distribution of production lines or equipment types. For example, multiple factory equipment set up on the production lines in a certain area can be used to produce specific products, so this area can be regarded as a workshop area for equipment abnormality detection. Alternatively, the factory equipment inside the workshop is clustered according to the equipment type, so the area with this equipment type can be regarded as a workshop area. There are collection devices around the factory equipment in each workshop area. The collection equipment can collect the equipment operation information corresponding to the factory equipment according to the preset collection interval. Among them, the collection equipment includes a camera for collecting equipment images, and a sensor for assisting in collecting sensor data such as temperature, humidity, and smoke concentration values.

[0052] It should be noted that the collection interval is related to the probability of equipment anomaly distribution in the workshop area where the factory equipment is located. The probability of equipment anomaly distribution is used to indicate the ratio of the number of equipment with abnormalities in the workshop to the total number of abnormal equipment. Generally speaking, the more factory equipment with abnormalities in the workshop area, the higher the corresponding probability of equipment anomaly distribution. The probability of abnormality is higher than that in other workshop areas. For such workshop areas, if equipment anomaly detection is still performed at a fixed collection interval, the anomalies of some factory equipment may not be discovered in time. Therefore, the collection interval needs to be adjusted accordingly according to the probability of equipment anomaly distribution.

[0053] Specifically, the workshop area where each factory equipment is located is determined by using information such as labels, RFID tags or workshop layout diagrams on factory equipment. The historical detection information of the workshop area is queried from the factory's equipment monitoring system. The historical detection information contains key information such as equipment status, abnormality type, occurrence time, and historical equipment abnormality distribution probability of the workshop area. Based on this, after obtaining the historical detection information, the historical equipment abnormality distribution probability corresponding to the workshop area in the previous collection interval can be determined. Among them, there is a certain mapping relationship between the equipment abnormality distribution probability and the collection interval. The mapping relationship can be obtained through the analysis of historical detection records, which will be expressed as a negatively correlated functional relationship, that is, the higher the abnormality probability, the shorter the collection interval; the lower the abnormality probability, the longer the collection interval. According to the acquired historical equipment abnormality distribution probability, the corresponding collection interval can be determined by querying the mapping relationship table. The collection interval obtained at this time is used to determine the time interval between the previous detection time point and the next detection time point. Through the above steps, the factory can realize the dynamic adjustment of the equipment monitoring collection interval in different workshop areas, thereby improving the efficiency and accuracy of monitoring.

[0054] S102: Input the equipment image into a preset anomaly detection model, extract feature information in the equipment image through the CNN network in the anomaly detection model, and locate the abnormal features in the feature information used to characterize the abnormality of the factory equipment through the Transformer self-attention mechanism.

[0055] Equipment images can be used to detect whether there are surface anomalies in the current factory equipment. This process is achieved through an anomaly detection model. The anomaly detection model consists of a codec integrated with CNN and Transformer, which can extract features from equipment images and locate abnormal areas where abnormalities exist. Unlike the Transformer model that treats all image positions equally, the embodiment of the present application introduces a deformable self-attention mechanism that can focus only on key positions in the equipment image. Combining this mechanism with the CNN network can effectively reduce computational and spatial complexity, making it possible to process multi-scale and high-resolution feature maps.

[0056] The anomaly detection model needs to be trained in advance before it can be used in the anomaly detection of the equipment. When pre-training the anomaly detection model, a large number of equipment images should be collected from the factory environment. These images should contain both normal and abnormal states. For each equipment image containing an abnormal area, the actual abnormal area is labeled and the corresponding abnormal area label is generated. According to the labeled equipment images, a training sample consisting of several equipment images can be generated. The training sample is used for pre-training the anomaly detection model. During the training process, the anomaly detection model can output the predicted abnormal area in the training sample, compare the predicted abnormal area with the actual abnormal area located according to the abnormal area label, and calculate the loss value between the two. The loss value can be calculated by the loss function, and the loss function can be selected according to the task requirements, such as cross entropy loss or mean square error loss. According to the calculated loss value, the weight and bias of the model are updated by the back propagation algorithm. The anomaly detection model can be optimized through iterative training until the preset conditions are met (such as reaching the maximum number of iterations, the loss value no longer decreases significantly, etc.). At this time, the obtained anomaly detection model has completed pre-training and can detect anomalies in the equipment image in the subsequent process.

[0057] Most traditional CNN network processes use the network structure of feature pyramid (FPN), which reduces the features by 4 times, 8 times, 16 times, and 32 times through multiple downsampling operations, and then performs feature splicing (upsampling operations) and lateral connection operations from high-level to low-level features. Similar to traditional encoders, the embodiment of the present application includes a Conv-IN-ReLU block and three-stage 3D residual modules, wherein the Conv-IN-ReLU block includes a 3D convolution layer, an instance normalization layer, and a linear activation function ReLU, and the number of 3D residual blocks in the three stages is 3, 3, and 2, respectively. The embodiment of the present application only takes the final three smallest feature layers to form multi-scale feature information for subsequent processing.

[0058] Based on the multi-scale feature information extracted by CNN, the abnormal area of ​​the equipment is located through the multi-scale deformable Transformer self-attention mechanism, which can reduce the area of ​​the abnormal area to be marked, so as to classify and segment the potential abnormal conditions on the equipment through this area, effectively improving the abnormality detection efficiency of the equipment.

[0059] Due to the inherent locality of the convolution operator, the CNN network cannot effectively capture long-range dependencies, so the Transformer is introduced to solve this problem. The multi-scale deformable self-attention mechanism (MS-DMSA) can convert the feature information obtained by CNN into a sequence format and process the information in a sequence-to-sequence manner. However, since the Transformer processes feature information in a sequence-to-sequence manner, it is first necessary to reshape the feature information through a preset reshaping function (such as the Flatten() function) to expand the feature information into one-dimensional sequence information. However, flattening the features will cause some position information to be lost. Therefore, the feature information needs to be sequence-encoded based on position during the reshaping process. That is, for each element in the sequence information, a position code corresponding to the position of the element in the sequence information is added to obtain the corresponding feature sequence.

[0060] The feature sequence containing position encoding is input into the Transformer layer with Transformer self-attention mechanism. The Transformer layer processes the input sequence through a multi-head self-attention mechanism and a position feedforward network. The self-attention mechanism allows the model to pay attention to all other elements in the feature sequence when processing each element in the feature sequence, so that the Transformer can capture the long-distance dependencies between elements in the feature sequence, which is very effective for locating abnormal features in the image. In addition, the self-attention mechanism can also focus on a small number of key sampling points, thereby greatly reducing the calculation and airborne complexity of the Transformer. Therefore, through the Transformer layer, the abnormal features in the feature map used to characterize abnormal factory equipment can be located.

[0061] Among them, the multi-scale deformable self-attention layer (MS-DMSA) can be expressed as the following formula:

[0062]

[0063] Among them, head i represents the output of the i-th attention head, H is the number of attention heads, z q is the feature representation of the query matrix q, is the normalized three-dimensional coordinate of the reference point, Λ(z q) ilqk ∈[0,1] represents the attention weight, σ1 rescales To the l-th level feature, Δ pilqk ∈R 3 represents the sampling offset of the kth sampling point at the lth feature level, and φ(…) represents the linear projection layer used to weight and aggregate the feature representations of all attention heads.

[0064] S103: Decode the abnormal features through the CNN network to identify the abnormal area in the device image.

[0065] For the located abnormal area, the abnormal features are transposed and convolved through the CNN network. This operation can increase the size of the abnormal features (i.e., spatial resolution), thereby upsampling the abnormal features to the original resolution corresponding to the device image to obtain the upsampled abnormal features. The residual block is a commonly used convolutional neural network structure that assists in learning the residual representation between input and output through jump connections. This structure helps solve the gradient vanishing / explosion problem in deep networks and accelerates the training process. The upsampled abnormal features are input into a network composed of multiple residual blocks. These residual blocks will further refine and enhance the abnormal features. Through residual learning, it can learn how to better represent and distinguish abnormal and normal features. For the refined abnormal features, semantic segmentation is performed on them, and each pixel in the image composed of abnormal features can be assigned a category label, thereby identifying the abnormal area in the device image.

[0066] S104: When there is an abnormal area in the device image, the device position of the factory device corresponding to the device image is obtained, and the device position and the abnormal area are reported to the early warning center to implement abnormal early warning of the factory equipment.

[0067] After identifying the abnormal area in the equipment image according to the above steps, the abnormal situation needs to be reported to the early warning center in time, so as to notify the inspection personnel to carry out maintenance and processing. When reporting abnormal information, it is necessary not only to report the abnormal area, but also to push the specific equipment location to the inspection personnel to achieve accurate positioning of abnormal factory equipment.

[0068] It should be noted that the above steps have realized the detection of equipment surface abnormalities. However, in addition to abnormal situations such as surface abnormalities, factory equipment may also have some operating abnormalities that cannot be detected by surface recognition during operation, such as excessive temperature, smoke warning, etc. At this time, it is necessary to analyze it in combination with the collected equipment sensor data. Obtain the equipment sensor data and match the equipment sensor data with its corresponding numerical range to determine whether the equipment sensor data is within the numerical range. If it is within the numerical range, it means that the equipment sensor data meets the normal operating standards of the equipment and no warning is required; if it is not within the numerical range, it means that some abnormalities may have occurred in the factory equipment. At this time, it is necessary to generate corresponding warning information, and report the warning information and equipment location to the warning center to promptly remind the inspection personnel to handle and maintain it.

[0069] In one embodiment, in a factory equipment anomaly detection system, once an equipment anomaly is detected, in addition to immediately handling these anomalies, it is also necessary to summarize the anomalies so that the inspection personnel can fully understand the equipment status and take corresponding preventive measures. First, determine the workshop area to which the factory equipment belongs, and then, for each workshop area, count the number of abnormalities of all equipment in the workshop area, which can be achieved by traversing all equipment in the area and checking its abnormal state. At the same time, it is also necessary to count the number of equipment anomalies in the entire workshop (i.e., the sum of all workshop areas) in order to perform a global analysis. The probability of equipment anomaly distribution refers to the possibility of abnormal equipment appearing in a certain workshop area, which can be calculated by dividing the number of equipment anomalies in the workshop area by the number of equipment anomalies in all workshops. The calculated probability of equipment anomaly distribution is bound to all factory equipment in its corresponding workshop area. In this way, when reporting the early warning information, in addition to the equipment location and abnormal area, the probability of equipment anomaly distribution in the workshop area where the abnormal factory equipment is located can also be reported to the early warning center at the same time, so that the inspection personnel can understand the probability of abnormal occurrence of a certain workshop area from a global perspective based on the probability of equipment anomaly distribution, so as to take corresponding management measures to improve the detection intensity of this area and avoid production losses due to untimely detection.

[0070] The above are embodiments of the method proposed in this application. Based on the same idea, some embodiments of this application also provide devices and non-volatile computer storage media corresponding to the above methods.

[0071] Figure 2 A schematic diagram of the structure of a factory equipment anomaly detection device based on CNN and Transformer provided in an embodiment of the present application. Figure 2 As shown, including:

[0072] at least one processor; and,

[0073] at least one processor is communicatively connected to a memory; wherein,

[0074] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor so that the at least one processor can execute a factory equipment anomaly detection method based on CNN and Transformer as described in any of the above items.

[0075] The embodiment of the present application provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured as follows:

[0076] A factory equipment anomaly detection method based on CNN and Transformer as described in any of the above items.

[0077] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0078] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0079] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0080] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1A device that provides the functions specified in a block or multiple blocks.

[0081] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0082] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0083] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0084] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0085] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0086] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0087] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A factory equipment anomaly detection method based on CNN and Transformer, characterized in that: The method comprises: Collecting equipment operation information corresponding to factory equipment at a preset collection interval; wherein the equipment operation information includes equipment images, and the collection interval is related to the equipment abnormality distribution probability of the workshop area where the factory equipment is located; Input the device image into a preset anomaly detection model, extract feature information in the device image through a CNN network in the anomaly detection model, and locate abnormal features in the feature information that are used to characterize abnormalities of factory equipment through a Transformer self-attention mechanism; Decoding the abnormal features through the CNN network to identify abnormal areas in the device image; When there is an abnormal area in the device image, the device location of the factory equipment corresponding to the device image is obtained, and the device location and the abnormal area are reported to the early warning center to implement abnormal early warning of the factory equipment.

2. According to claim 1, a method for detecting abnormality of factory equipment based on CNN and Transformer, characterized in that: Through the Transformer self-attention mechanism, the abnormal features in the feature information used to characterize the abnormality of the factory equipment are located, specifically including: Reshape the feature information by a preset reshaping function to obtain sequence information corresponding to the feature information, and add a position code corresponding to the position of the element in the sequence information to each element in the sequence information to obtain a corresponding feature sequence; The feature sequence is input into a Transformer layer with a Transformer self-attention mechanism, and the abnormal features in the feature map used to characterize the abnormality of the factory equipment are located through the Transformer layer.

3. The method for detecting abnormality of factory equipment based on CNN and Transformer according to claim 2, characterized in that: Through the Transformer layer, the abnormal features in the feature map used to characterize the abnormality of the factory equipment are located. The specific calculation method is: Among them, head i represents the output of the i-th attention head, H is the number of attention heads, z q is the feature representation of the query matrix q, is the normalized three-dimensional coordinate of the reference point, Λ(z q ) ilqk ∈[0,1] represents the attention weight, σ1 rescales To the l-th level feature, Δ pilqk ∈R 3 represents the sampling offset of the kth sampling point at the lth feature level, and φ(…) represents the linear projection layer used to weight and aggregate the feature representations of all attention heads.

4. The method for detecting abnormality of factory equipment based on CNN and Transformer according to claim 1, characterized in that: The abnormal features are decoded by the CNN network to identify abnormal areas in the device image, specifically including: Performing transposed convolution on the abnormal feature through the CNN network to upsample the abnormal feature to an original resolution corresponding to the device image, thereby obtaining an upsampled abnormal feature; The upsampled abnormal features are refined by residual blocks to obtain refined abnormal features, and semantic segmentation is performed on the refined abnormal features to identify abnormal areas in the device image.

5. The method for detecting abnormality of factory equipment based on CNN and Transformer according to claim 1, characterized in that: Before inputting the device image into a preset anomaly detection model, the method further includes: Generate a training sample consisting of a plurality of device images; wherein the training sample carries an actual abnormal region and an abnormal region label for marking the actual abnormal region; Pre-training the anomaly detection model according to the training samples, so as to output the predicted abnormal regions existing in the training samples during the pre-training process; According to the abnormal region label, the actual abnormal region in the training sample is located, and the predicted abnormal region is compared with the actual abnormal region to calculate the corresponding loss value; The anomaly detection model is optimized according to the loss value until the prediction area output by the anomaly detection model can meet the preset conditions.

6. The method for detecting abnormality of factory equipment based on CNN and Transformer according to claim 1, characterized in that: Reporting the device location and the abnormal area to the early warning center, specifically including: Determine the workshop area to which the factory equipment belongs according to the equipment location; Summarizing the number of equipment anomalies in the workshop area and the entire workshop respectively, and calculating the distribution probability of equipment anomalies corresponding to the workshop area according to the summarized number of equipment anomalies; The equipment abnormality distribution probability is bound to all factory equipment in its corresponding workshop area, and the equipment location, the abnormal area and the equipment abnormality distribution probability are reported to the early warning center.

7. The method for detecting abnormality of factory equipment based on CNN and Transformer according to claim 6, characterized in that: According to the preset collection interval, collect the equipment images corresponding to the factory equipment, including: Determine the workshop area where the factory equipment is located, and obtain the historical detection records corresponding to the workshop area, and determine the historical equipment abnormality distribution probability corresponding to the workshop area in the previous collection interval based on the historical detection information; Acquire a mapping relationship between the device abnormality distribution probability and the collection interval, and determine the collection interval corresponding to the historical device abnormality distribution probability according to the mapping relationship; wherein the historical device abnormality distribution probability is negatively correlated with the collection interval; According to the collection interval, the device image corresponding to the factory equipment is collected.

8. The method for detecting abnormality of factory equipment based on CNN and Transformer according to claim 1, characterized in that: The method further comprises: Acquire the device sensor data, and match the device sensor data with a corresponding value range to determine whether the device sensor data is within the value range; If not, a corresponding warning message is generated, and the warning message and the device location are reported to the warning center.

9. A factory equipment anomaly detection device based on CNN and Transformer, characterized in that: The device comprises: at least one processor; and, a memory communicatively coupled to the at least one processor; Wherein, the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute a factory equipment anomaly detection method based on CNN and Transformer as described in any one of claims 1-8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are configured to: A method for detecting anomalies of factory equipment based on CNN and Transformer as described in any one of claims 1 to 8.

Images