Intelligent factory comprehensive inspection system based on simultaneous implementation of multiple measures

Abstract

The invention, which relates to the technical field of industrial automation and intelligent manufacturing, discloses an intelligent factory comprehensive inspection system based on multi-measure simultaneous work, comprising an acquisition module, a transmission module, an identification module, an analysis module, an alarm module and a processing module. The acquisition module fuses multi-source data of video monitoring, Internet of Things equipment, a robot, an unmanned aerial vehicle and manual inspection, the transmission module is responsible for data transmission, the identification module finds abnormity through a risk identification model and index abnormity identification, and the analysis module diagnoses equipment fault and health state by combining alarm and historical data. The alarm module stores abnormal information and gives an alarm in a linkage mode, and the disposal module completes abnormal disposal. According to the invention, multi-source heterogeneous data fusion and unified management are realized, and the inspection efficiency and quality are improved; the false alarm rate is reduced through an optimized intelligent recognition algorithm, and abnormity is accurately detected; and large model reasoning and digital human interaction are combined to assist in analysis of abnormality reasons and generation of a disposal scheme, so that the abnormality disposal capability is enhanced, and a scientific basis is provided for equipment maintenance.

Description

Technical Field

[0001] The present invention relates to the field of industrial automation and intelligent manufacturing technology, and more particularly to a comprehensive inspection system for intelligent factories based on multiple measures. Background Art

[0002] Currently, chemical plant inspections are a task-intensive, arduous, and labor-intensive endeavor. Traditional factory inspection models are no longer able to meet the demands of modern production. Companies in the chemical and energy sectors are leveraging technologies like drones, inspection robots, and AI-powered visual inspection to replace traditional manual inspections and reduce the risk of safety incidents. However, most chemical plant inspections are still primarily manual, supplemented by digital inspections. Digital inspections primarily rely on data recording or limit alarms, failing to fully leverage the effectiveness of inspection data.

[0003] Traditional manual inspections are time-consuming, inefficient, and costly, with high labor costs, difficulty ensuring accuracy and security, and difficulty managing paper-based data records. Digital inspections reduce labor costs and facilitate data collection and management, but each digital approach has its limitations. For example, inspection robots are expensive to procure and rely heavily on technology, making them ineffective in complex and obstructed areas. Drone inspections are subject to significant weather, wind, and electromagnetic interference, hindering takeoff and affecting inspection performance. IoT sensor devices often capture single-point data, making it difficult to achieve coordinated plant-wide awareness. A single inspection method, or multiple methods that fail to coordinate timely, makes it impossible to verify the authenticity and severity of anomalies when they are detected, leading to misjudgments or missed detections, thus compromising the accuracy and timeliness of problem resolution. Furthermore, data from different inspection methods cannot be effectively shared and integrated, making it difficult to generate comprehensive and accurate equipment status information, hindering comprehensive equipment management and decision-making.

[0004] Therefore, how to realize comprehensive inspection and linkage verification of chemical plant operation is an urgent problem that technical personnel in this field need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a comprehensive inspection system for smart factories based on multiple measures to solve the problems existing in the background technology.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A comprehensive inspection system for smart factories based on multiple measures, including an acquisition module, a transmission module, an identification module, an analysis module, an alarm module, and a disposal module;

[0008] The acquisition module collects multi-source inspection data of the factory;

[0009] The transmission module transmits the factory multi-source inspection data to the identification module;

[0010] The identification module includes, on one hand, constructing a risk identification model, using the risk identification model to identify risks in the video data of the collected factory multi-source inspection data, obtaining an abnormal result, and sending a first alarm signal to the alarm module when the abnormal duration is greater than a first preset threshold; on the other hand, the identification module also includes index abnormality identification, determining whether a value is out of limit by monitoring data in real time according to a second preset threshold range, and determining that an abnormality exists when a real-time monitoring value is greater than a highest threshold value and less than a lowest threshold value; a second alarm signal is sent to the alarm module;

[0011] The analysis module performs intelligent diagnosis and health degree analysis on existing faults or potential faults of the equipment according to the alarm data and historical operation data of each device, generates a third alarm signal when the analysis result is that there is a device fault, and sends the device information required for fault handling and the fault location information of the fault device to the alarm module together;

[0012] The alarm module receives the first alarm signal, the second alarm signal and the third alarm signal and respectively performs alarm and stores abnormal information and fault information;

[0013] The disposal module completes abnormal alarm disposal through verification, pushing, supervision and feedback based on the identified abnormal alarm.

[0014] Optionally, the collection module includes video monitoring, Internet of Things equipment, robots, unmanned aerial vehicles and manual inspection to collect inspection data, and multi-source data fusion forms factory multi-source inspection data;

[0015] Video monitoring data collection: supports video monitoring equipment to monitor fixed point personnel, equipment and environment in real time, and collects on-site video image data at a set frame rate without interruption;

[0016] Internet of Things data collection: supports different types of Internet of Things sensors to collect fixed point equipment monitoring data for 24 hours without interruption;

[0017] Robot data collection: supports collection of visible light, thermal imaging video image data and specified monitoring data, and ground and mobile data collection;

[0018] Unmanned aerial vehicle data collection: supports collection of visible light, thermal imaging video image data and specified monitoring data, and aerial and mobile data collection;

[0019] Manual data collection: supports manual collection of inspection point data according to a specified inspection plan and inspection route.

[0020] Optionally, the risk identification of the video data in the collected multi-source inspection data of the factory by using the risk identification model specifically comprises:

[0021] ① Feature data collection: collect normal operation and abnormal operation picture data of various inspection scenes;

[0022] ② Data labeling: label the collected pictures using a data labeling tool to clearly label the abnormal type and the occurrence position;

[0023] ③ Picture preprocessing: according to the labeling information, pictures containing specific abnormalities are selected;

[0024] ④ Model training: according to the ratio of 8:1:1, the labeled picture dataset is divided into a training set, a validation set and a test set; a risk identification model is established, the risk identification model is trained based on the training set, the model parameters are continuously adjusted through the back propagation mechanism; the model performance is evaluated using the validation set, the training strategy is adjusted in time to prevent overfitting; and the final performance of the risk identification model is evaluated using the test set;

[0025] ⑤ Decode and frame extraction are performed on the collected real-time video stream; at the same time, the extracted video frames are preprocessed to meet the input requirements of the model;

[0026] The preprocessed video frames are input into the trained risk identification model, the target in the picture is identified, and the abnormal category and position are determined;

[0027] In combination with the historical frame detection result, if the abnormal identification duration is greater than a first preset threshold, an alarm is determined, and a first alarm signal is generated.

[0028] Optionally, the risk identification model comprises:

[0029] A C2f_dcnv2 module is constructed;

[0030] The C2f_dcnv2 module comprises a first to a second Conv module, a Split module, a first to a third Bottleneck_dcnv2 module;

[0031] The first to the third Bottleneck_dcnv2 modules have the same structure and each comprises a third Conv module, a deformable convolution layer DCNv2, a batch normalization layer and a SiLU activation function layer connected in sequence, wherein the input end of the third Conv module is connected with an external input, and the input end of the third Conv module and the output end of the SiLU activation function layer are added to serve as an output;

[0032] The first Conv module, the Split module, the first Bottleneck_dcnv2 module, the second Bottleneck_dcnv2 module and the third Bottleneck_dcnv2 module are sequentially connected, the input end of the first Conv module is used as the input end of the C2f_dcnv2 module, the output end of the Split module, the output end of the first Bottleneck_dcnv2 module, the output end of the second Bottleneck_dcnv2 module and the output end of the third Bottleneck_dcnv2 module are spliced in channels and then input to the input end of the second Conv module, and the output end of the second Conv module is used as the output end of the C2f_dcnv2 module;

[0033] The first to third Conv modules have the same structure and each comprises a Conv layer, a batch normalization layer and a SiLU activation function layer connected in sequence;

[0034] For the head network of the YOLOv8 model, all C2f modules are replaced with C2f_dcnv2 modules to obtain a YOLOv8 model introducing deformable convolution;

[0035] For the backbone network of the YOLOv8 model introducing deformable convolution, all Conv modules except the first Conv module are replaced with receptive field attention convolution modules RFAConv to obtain an improved YOLOv8 model.

[0036] Optionally, the alarm module further comprises an association query alarm unit and an alarm information management unit.

[0037] The association query alarm unit identifies an abnormal device through any inspection mode, can check the monitoring situation and historical data of the abnormal moment of the abnormal device through association, and confirms the abnormality through multiple inspection modes and sends an alarm.

[0038] The alarm information management unit receives the abnormal information pushed by the identification module, stores and manages the abnormal information, and displays the abnormal information in the inspection abnormality function. Meanwhile, various types of abnormality are pushed to the disposal module for disposal according to the level, and the alarm module continuously pays attention to the current and historical data information and disposal result information of the abnormality.

[0039] Optionally, the analysis module performs intelligent diagnosis and health degree analysis on existing faults or potential faults of each device according to alarm data and historical operation data of each device, and the intelligent diagnosis and health degree analysis specifically comprises:

[0040] Vibration signals are extracted from the alarm data and historical operation data of each device, and feature vectors are extracted through wavelet packet decomposition;

[0041] Constructing discriminant analysis function for health state evaluation: the feature vectors extracted in each state are grouped into a training set to learn FDA, realize conversion from high-dimensional space to low-dimensional space, and construct each discriminant population and discriminant analysis function; FDA represents Fisher discriminant analysis;

[0042] Calculating Mahalanobis distance between the energy feature vector and the normal population, and evaluating the health degree CV of the equipment at time t by normalizing the Mahalanobis distance;

[0043] Fault detection of rotating machinery equipment: comparing the health degree CV of the equipment in the working state at time t with the third preset health degree threshold HT, if CV>HT, the equipment is in good running state, and if CV≤HT, the equipment has a fault;

[0044] Fault diagnosis of rotating machinery equipment: combining the discriminant analysis function, calculating the Mahalanobis distance between the feature vector x extracted in the working state at time t and each fault population, and determining the fault mode through discriminant analysis.

[0045] Optionally, the discriminant analysis function is: let ∑ -1 The first m eigenvalues of B are λ1≥λ2≥…≥λ m , and the corresponding eigenvectors are a1, a2, …, a m , if the cumulative contribution rate is reaches a set threshold, m mutually independent typical variables y1, y2, … y m are selected, and y m =a' m x is used as the discriminant analysis function, wherein, μ i and ∑ i are the mean vector and the covariance matrix of k discriminant populations G i , i=1, 2, …, k; λ p The subscript p is the dimension of the equipment to be judged, p≥m; a' represents the transpose of vector a, so a' m represents the transpose of vector a m .

[0046] Optionally, the Mahalanobis distance is obtained in the following way:

[0047] Suppose that the feature vector extracted from the real-time vibration signal collected in the working state at time t through wavelet decomposition is the equipment to be judged x, and combining the discriminant analysis function y1, y2, …, y m , y m =a′ m x, in the new low-dimensional space after FDA dimension reduction processing, the equipment to be judged x to the discriminant population G iThe Mahalanobis distance d(x,G i ) is given by y=(y1,y2,…,y m )′ to one-dimensional discriminant population The distance is calculated as:

[0048]

[0049] Using normalization, the obtained Mahalanobis distance is converted into a CV value. The CV value at this time can represent the current health status. A CV close to 1 indicates a good current health status, a downward trend in CV indicates a degenerative state, and a CV close to 0 indicates an abnormal state.

[0050] Optionally, it also includes a digital human module. The digital human module is fully integrated into the Deepseek large model. The digital human module uses multi-source inspection means to analyze result comparison, verification, and upstream and downstream process parameter correlation analysis to assist in judging abnormal production status and tracking the root cause of the abnormality. After confirming the abnormality, it forms a disposal plan through matching plans and operation cards, and realizes intelligent interaction with inspection personnel in a digital human manner.

[0051] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a comprehensive inspection system for smart factories based on multiple measures, which has the following beneficial effects:

[0052] (1) It solves the problem of fusion of multi-source heterogeneous inspection data and realizes unified management and comprehensive application of inspection data. The present invention realizes the unified integration of multi-source heterogeneous data such as the Internet of Things, AI video recognition, mobile devices (drones, robots), and manual inspection data, constructs factory inspection big data, and uses inspection big data as a support to realize correlation query of inspection anomalies and equipment failure prediction analysis, effectively improving the efficiency and quality of factory inspections. At the same time, it gives full play to the potential value of inspection big data, predicts potential equipment failures through analysis modules, and provides scientific guidance for optimizing inspection plans and optimizing equipment inspection and maintenance frequencies.

[0053] (2) It overcomes the problem of high false alarm rate of traditional algorithms, develops high-accuracy intelligent recognition algorithms, and improves the accuracy of anomaly detection and diagnosis. Traditional algorithms rely on threshold alarms or single data sources, have high false alarm rates, and are difficult to adapt to complex working conditions (such as gradual wear of equipment and intermittent failures). The present invention trains video analysis algorithms based on massive historical data to achieve algorithm optimization for coal chemical plant scenarios such as abnormal personnel behavior, unsafe equipment conditions (rust, oil leakage, etc.), and abnormal environmental conditions, thereby improving the accuracy of intelligent recognition.

[0054] (3) Deeply integrate the reasoning and summarizing capabilities of large models, integrate and apply industry large models with industry early warning algorithm models, effectively improve the ability to discover and deal with inspection anomalies, and realize friendly interaction with inspection personnel in a digital human way, further improving the efficiency and quality of inspection work. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0056] Figure 1 This is a schematic diagram of the system structure provided by the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] The embodiment of the present invention discloses a comprehensive inspection system for smart factories based on multiple measures, such as Figure 1 As shown, it includes an acquisition module, a transmission module, an identification module, an analysis module, an alarm module and a disposal module;

[0059] Collection module, collects multi-source inspection data from the factory;

[0060] The transmission module transmits the factory multi-source inspection data to the identification module;

[0061] The identification module, on the one hand, includes building a risk identification model, using the risk identification model to identify risks in the video data collected from the multi-source inspection data of the factory, and obtaining abnormal results. When the abnormality duration is greater than a first preset threshold, a first alarm signal is sent to the alarm module; on the other hand, it also includes indicator abnormality identification. Through real-time monitoring data, it is determined whether the limit value is exceeded according to the second preset threshold range. When the real-time monitoring value is greater than the highest threshold and less than the lowest threshold, it is determined to be abnormal; and a second alarm signal is sent to the alarm module;

[0062] The analysis module performs intelligent diagnosis and health degree analysis on the existing faults or potential faults of each device according to the alarm data and historical operation data of each device, generates a third alarm signal when the analysis result is that there is a device fault, and sends the device information required for fault handling and the fault location information of the faulty device to the alarm module;

[0063] The alarm module receives the first alarm signal, the second alarm signal and the third alarm signal and respectively performs alarm and stores abnormal information and fault information;

[0064] The processing module completes the processing of the abnormal alarm through verification, pushing, supervision and feedback based on the identified abnormal alarm.

[0065] In a specific embodiment, the collection module includes video monitoring, Internet of Things devices, robots, unmanned aerial vehicles and manual inspection to collect inspection data, and multi-source data fusion forms factory multi-source inspection data;

[0066] Video monitoring data collection: visible light, thermal imaging and other video monitoring devices are supported to monitor the fixed point personnel, equipment and environment in the factory production area, warehouse area and other areas in real time, and the on-site video image data is collected at a set frame rate without interruption.

[0067] Internet of Things data collection: temperature, liquid level, vibration, pressure, voiceprint and other types of Internet of Things sensors are supported to collect fixed point device monitoring data for 24 hours without interruption.

[0068] Robot data collection: visible light, thermal imaging video image data and specified monitoring data (such as temperature and humidity, gas concentration, etc.) are supported to collect data on the ground and in mobile mode.

[0069] Unmanned aerial vehicle data collection: visible light, thermal imaging video image data and specified monitoring data (such as temperature and humidity, gas concentration, etc.) are supported to collect data in the air and in mobile mode.

[0070] Manual data collection: manual collection of inspection point data according to the specified inspection plan and inspection route is supported, mainly recording data such as whether the equipment is running normally, whether there is abnormal noise, whether there is leakage, etc.

[0071] The transmission module includes wired data transmission, 5G data transmission and NB-IOT transmission in different ways. The video monitoring data of fixed points is transmitted in a wired manner, with fast transmission speed and high quality. The mobile data collection methods such as robots, unmanned aerial vehicles and Internet of Things sensing data are transmitted in real time through 5G, NB-IOT and other methods. The equipment is internally integrated with 5G and NB-IOT communication modules, which transmit the visible light, thermal infrared monitoring video, temperature, vibration and other time series data collected on site in real time through 5G network, NB-IOT and other methods to the recognition module.

[0072] In a specific embodiment, the risk identification model is used to identify risks in the video data collected from the multi-source inspection data of the factory, specifically:

[0073] ① Feature data collection: Collect normal operation and abnormal operation picture data of various inspection scenes, including belt deviation, belt foreign matter, running, leaking, infrared temperature measurement, and personnel falling, etc. The operation pictures are obtained through open source data sets and on-site collection.

[0074] ② Data labeling: Use data labeling tools to label the collected pictures, and clearly label the abnormal type and location.

[0075] ③ Picture preprocessing: According to the labeling information, pictures containing specific abnormalities are selected. In addition, for pictures with dark and blurred light, histogram equalization, contrast stretching and other methods are used to enhance image clarity and contrast, and highlight the target object. Gaussian filtering, median filtering and other algorithms are used to remove noise interference in the picture and improve image quality.

[0076] ④ Model training: According to the ratio of 8:1:1, the labeled picture dataset is divided into training set, validation set and test set. The risk identification model is established, the risk identification model is trained based on the training set, the risk identification model parameters are continuously adjusted through the back propagation mechanism; the performance of the risk identification model is evaluated using the validation set, and the training strategy is adjusted in time to prevent overfitting; the final performance of the risk identification model is evaluated using the test set to ensure that the model has good generalization ability and robustness.

[0077] ⑤ Risk identification

[0078] Real-time video stream processing: Video stream processing technology is used to decode and frame extract real-time video stream collected by the collection module. At the same time, the preprocessed video frames are preprocessed, including adjusting image size, normalization, etc., to meet the input requirements of the model.

[0079] Abnormal identification: The preprocessed video frames are input into the trained risk identification model to quickly identify personnel, equipment, objects and other targets in the picture and determine the abnormal category and location.

[0080] Risk alarm: Combined with the historical frame detection results, if the abnormal identification duration is greater than the first preset threshold (time), it is determined as an alarm to avoid instantaneous misjudgment.

[0081] (2) Index abnormality identification: including temperature, flow, liquid level, flammable gas, smoke concentration, vibration, pressure, current, voltage, etc. Real-time monitoring data, according to the set threshold range to determine whether it is out of limit, when the real-time monitoring value is greater than the highest threshold, less than the lowest threshold, it is determined as abnormal. The alarm threshold of different indexes is determined in combination with national and industry standards.

[0082] In a specific embodiment, the risk identification model includes:

[0083] Build the C2f_dcnv2 module;

[0084] The C2f_dcnv2 module includes the first to second Conv modules, the Split module, and the first to third Bottleneck_dcnv2 modules;

[0085] The first to third Bottleneck_dcnv2 modules have the same structure, and all include a third Conv module, a deformable convolution layer DCNv2, a batch normalization layer, and a SiLU activation function layer connected in sequence. The input end of the third Conv module is connected to the external input, and the input end of the third Conv module and the output end of the SiLU activation function layer are added as the output;

[0086] The first Conv module, the Split module, the first Bottleneck_dcnv2 module, the second Bottleneck_dcnv2 module, and the third Bottleneck_dcnv2 module are connected in sequence. The input end of the first Conv module serves as the input end of the C2f_dcnv2 module. The output end of the Split module, the output end of the first Bottleneck_dcnv2 module, the output end of the second Bottleneck_dcnv2 module, and the output end of the third Bottleneck_dcnv2 module are spliced ​​by channel and input into the input end of the second Conv module. The output end of the second Conv module serves as the output end of the C2f_dcnv2 module.

[0087] The first to third Conv modules have the same structure, which consists of a Conv layer, a batch normalization layer, and a SiLU activation function layer connected in sequence;

[0088] The deformable convolution layer obtains a set of prediction results of convolution kernel offset and weight prediction results by a convolution operation on the input feature map. The size of the offset feature map is consistent with the input feature map. Figure 1 The number of channels is 2N, where 2 means that each offset is two values ​​(x, y), and N is the number of pixels in the convolution kernel. That is, the deformable convolution learns a set of offsets for each pixel of the input feature map. This embodiment uses DCNv2, which introduces a modulation mechanism that uses weights to adjust the input feature amplitudes from different spatial locations. The size of the weighted feature map is proportional to the input feature. Figure 1 The number of channels is N;

[0089] Given a convolution kernel with K sampling positions, let w k and p krespectively, where x(p) and y(p) denote the features at position p in the input feature map x and the output feature map y, respectively. The expression of the deformable convolution DCNv2 is shown in equation (1).

[0090]

[0091] where Δp k and Δm k are the learnable offset and weight coefficient at the k-th position, respectively. The weight coefficient Δm k is a real number, and the range is not restricted. Δp k and Δm k are initialized to 0 and 0.5 (the default offset is 0, which cannot distinguish the sampling point contribution rate), and the corresponding convolution kernel parameter is initialized to 0, and the learning rate of the convolution layer is set to 0.1 of the existing layer. k

[0092] For the head network of the YOLOv8 model, replace all C2f modules with C2f_dcnv2 modules to obtain a YOLOv8 model with deformable convolution;

[0093] For the backbone network of the YOLOv8 model with deformable convolution, replace all Conv modules except the first Conv module with a receptive field attention convolution module RFAConv to obtain an improved YOLOv8 model.

[0094] The receptive field attention convolution module RFAConv, in order to ensure a small amount of computational overhead and parameter quantity, RFAconv uses AvgPool to aggregate the global features of each receptive field feature, then uses 1×1 grouped convolution for information interaction, and finally uses softmax to emphasize the importance of each feature in the receptive field feature, and the calculation formula of RFA is as follows:

[0095] F = Softmax(g 1×1 (AvgPool(X))) x ReLU(Norm(g k×k (X)))

[0096] = A rf x F rf

[0097] where g 1×1 denotes a 1×1 grouped convolution, k denotes the size of the convolution kernel, and k = 3 is selected, Norm denotes normalization, X denotes the input feature map, and F is obtained by multiplying the attention map A rf and the transformed receptive field spatial feature F rf . ​

[0098] In a specific embodiment, the alarm module further includes an associated query alarm unit and an alarm information management unit;

[0099] (1) Associated query alarm unit: When an abnormal device is identified through any inspection method, the monitoring status and historical data of the abnormality at the time of occurrence of other inspection methods can be viewed in association. Multiple inspection methods can be combined to confirm the abnormality and issue an alarm, thereby improving the accuracy of inspection problem detection.

[0100] (2) Alarm information management unit: Receives abnormal information pushed by the identification module, stores and manages the abnormal information, and displays it uniformly in the inspection abnormality function, including abnormality description, abnormality level, inspection object name, abnormality source, abnormality status, discovery time, etc. At the same time, various abnormalities are pushed to the disposal module for disposal according to their level. The alarm module continuously monitors the current and historical data information of the abnormality and the disposal result information.

[0101] In one specific embodiment, all equipment within the factory is identified by a unified KKS equipment code, linking and associating equipment attribute information, hidden danger information, and inspection and maintenance information. The analysis module leverages each device's alarm data, including abnormal vibration, rotation speed, temperature, sound, displacement, and historical operating data, to perform intelligent diagnosis and health analysis of existing or potential equipment failures. If the analysis indicates an equipment failure, the equipment information required for troubleshooting and the fault location of the faulty device are sent to the inspection and alarm module, which generates an alarm and takes appropriate action. Simultaneously, the module provides analytical reports for predictive maintenance, including abnormal area analysis, abnormal equipment analysis, and equipment performance and early warning analysis, providing data guidance for inspection and maintenance work.

[0102] The analysis module performs intelligent diagnosis and health analysis of existing or potential equipment faults based on the alarm data and historical operation data of each device. Specifically:

[0103] Extract vibration signals from each device's alarm data and historical operating data, and perform wavelet packet decomposition to extract feature vectors;

[0104] Construct a discriminant analysis function to assess health status: The feature vectors extracted from each state are combined into a training set for FDA learning, achieving the conversion from high-dimensional space to low-dimensional space, and constructing each discriminant population and discriminant analysis function; FDA stands for Fisher discriminant analysis;

[0105] Calculate the Mahalanobis distance between the energy eigenvector and the normal population, and evaluate the health CV of the device at time t by normalizing the Mahalanobis distance;

[0106] Fault detection of rotating machinery: Compare the health CV of the equipment in the working state at time t with the preset third health threshold HT. If CV>HT, the equipment is in good operating condition; if CV≤HT, the equipment has a fault.

[0107] Fault diagnosis of rotating machinery: Combined with the discriminant analysis function, the Mahalanobis distance between the feature vector x extracted at the working state at time t and the total number of various faults is calculated, and the fault mode is determined through discriminant analysis. The judgment rules are as follows:

[0108] like Then x∈G1

[0109] Where G j for each fault population. Simply put, the fault state corresponding to the fault population with the smallest Mahalanobis distance to the eigenvector x is the fault mode at time t. Through Fisher discriminant analysis, normal state data, various fault state data, and test data are mapped from the original high-dimensional space to a new low-dimensional space. In this new low-dimensional space, the Mahalanobis distances (MD) are calculated between the mapped points of the test data and each judgment population (normal population and various fault populations). By comparing the MDs, the fault state corresponding to the fault population with the smallest Mahalanobis distance is determined as the current fault mode.

[0110] The vibration signal is decomposed into three layers of wavelet packets so that all peaks can be included in different frequency bands, and the eight frequency band energy indices E are statistically calculated. 3j :

[0111]

[0112] Where x jk (j=0,1,…,7;k=1,2,,n) represents the reconstructed signal S 3j (t) is the discrete point amplitude, n is the number of discrete points.

[0113] When a device or a component on the device fails, it will have a significant impact on the signals in each frequency band. Therefore, we can construct a feature vector T with energy as the element:

[0114] T=[E 30 / E,E 31 / E,E 32 / E,E 33 / E,E 34 / E,E 35 / E,E 36 / E,E 37 E]

[0115] Among them, E 30 , E 37 are the energies of the eight frequency bands, and

[0116]

[0117] The eigenvector T is the energy eigenvector extracted based on wavelet packet decomposition.

[0118] In a specific embodiment, the discriminant analysis function is: Let ∑ -1 The first m eigenvalues ​​of B are λ1≥λ2≥…≥λ m , then the corresponding eigenvectors are a1,a2,…,a m If there is a cumulative contribution rate When the set threshold is reached, m uncorrelated typical variables y1, y2, ...y are selected. m ,y m =a' m x is used as the discriminant analysis function, where μ i and ∑ i There are k discriminant populations G i The mean vector and covariance matrix of , i=1,2…,k; λ p The subscript p is the dimension of the device to be judged, p ≥ m; a' represents the transpose of vector a, so a' m. Represents vector a m The transpose of .

[0119] In a specific embodiment, the Mahalanobis distance is obtained as follows:

[0120] Assume that the feature vector extracted by wavelet decomposition of the real-time vibration signal collected in the working state at time t is the device to be judged x, combined with the discriminant analysis function y1, y2, ..., y m ,y m =a′ m x, in the new low-dimensional space after FDA dimensionality reduction processing, the device to be judged x to the judgment population G i The Mahalanobis distance d(x,G i ) is given by y=(y1,y2,…,y m )′ to one-dimensional discriminant population The distance is calculated as:

[0121]

[0122] Using normalization, the obtained Mahalanobis distance is converted into a CV value. The CV value at this time can represent the current health status. A CV close to 1 indicates a good current health status, a downward trend in CV indicates a degenerative state, and a CV close to 0 indicates an abnormal state.

[0123] The normalization function is:

[0124]

[0125] Where c is the scale parameter, which is determined by the mean of the normal state MD and the corresponding CV benchmark value (which can be set to 0.95). Through Fisher discriminant analysis, the normal state data and the test data are transformed from the original high-dimensional space to a new low-dimensional space, that is, spatial mapping is achieved. The Mahalanobis distance (MD) between the test data and the normal population is calculated in the new low-dimensional space.

[0126] In a specific embodiment, the processing module specifically includes:

[0127] (1) Abnormal alarm handling: Abnormal alarms identified through various means are handled through verification, push, supervision and feedback.

[0128] (2) Dynamic adjustment of inspection and maintenance: After receiving the analysis report generated by the analysis module, adjust the inspection plan and inspection tasks according to the guidance of the analysis report, increase the number of inspections of abnormal areas and abnormal equipment, until the abnormal situation is eliminated from the analysis report.

[0129] In a specific embodiment, it also includes a digital human module. The digital human module is fully connected to the Deepseek large model. The digital human module assists in judging the abnormal production status and tracking the root cause of the abnormality through multi-source inspection means, analysis result comparison, verification, upstream and downstream process parameter correlation analysis, etc., and after confirming the abnormality, it quickly forms a disposal plan through plans, operation cards and similar situation matching, and realizes intelligent interaction with inspection personnel in a digital human manner, providing abnormal reminders, cause analysis, disposal suggestions, inspection optimization and other capabilities, thereby improving the accuracy of factory inspection abnormality identification and disposal efficiency.

[0130] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0131] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A comprehensive inspection system for smart factories based on multiple measures, characterized by: It includes acquisition module, transmission module, identification module, analysis module, alarm module and disposal module; The acquisition module collects multi-source inspection data of the factory; The transmission module transmits the factory multi-source inspection data to the identification module; The identification module, on the one hand, includes building a risk identification model, using the risk identification model to identify risks in the video data collected from the multi-source inspection data of the factory, and obtaining abnormal results. When the abnormality duration is greater than a first preset threshold, a first alarm signal is sent to the alarm module; on the other hand, it also includes indicator abnormality identification, through real-time monitoring data, determining whether the limit value is exceeded according to the second preset threshold range. When the real-time monitoring value is greater than the highest threshold and less than the lowest threshold, it is determined to be abnormal; Sending a second alarm signal to the alarm module; The analysis module performs intelligent diagnosis and health analysis of existing or potential faults of each device based on the alarm data and historical operation data of each device. When the analysis result indicates that a device fault exists, a third alarm signal is generated and the device information required for fault processing and the fault location information of the faulty device are sent to the alarm module. The alarm module receives the first alarm signal, the second alarm signal, and the third alarm signal and respectively issues an alarm and stores abnormal information and fault information; The handling module completes the abnormal alarm handling based on the identified abnormal alarm through verification, push, supervision and feedback.

2. The intelligent factory comprehensive inspection system based on multiple measures according to claim 1 is characterized in that: The acquisition module includes video surveillance, IoT devices, robots, drones and manual inspections to collect inspection data, and multi-source data is integrated to form factory multi-source inspection data; Video surveillance data acquisition: Supports video surveillance equipment to monitor personnel, equipment, and environment at fixed points in real time, and continuously collects on-site video image data at a set frame rate; IoT data collection: supports different types of IoT sensors and collects equipment monitoring data at fixed points 24 hours a day; Robotic data collection: supports the collection of visible light, thermal imaging video data and designated monitoring data, and ground and mobile data collection; UAV data collection: supports the collection of visible light, thermal imaging video data and designated monitoring data, and can be collected in an aerial and mobile manner; Manual data collection: supports manual collection of inspection point data according to specified inspection plans and inspection routes.

3. The intelligent factory comprehensive inspection system based on multiple measures according to claim 1 is characterized in that: The risk identification model is used to identify the risks of the video data in the collected factory multi-source inspection data in detail as follows: ① Feature data collection: Collecting normal operation and abnormal operation image data of various inspection scenarios; ②Data annotation: Use data annotation tools to annotate the collected images, clearly marking the abnormality type and location; ③ Image preprocessing: Filter out images containing specific anomalies based on annotation information; ④Model training: Divide the labeled image dataset into training set, validation set, and test set in a ratio of 8:1:1; Establish a risk identification model, train it based on the training set, and continuously adjust the model parameters through the back-propagation mechanism; use the validation set to evaluate the model performance and adjust the training strategy in a timely manner to prevent overfitting; Use the test set to evaluate the final performance of the risk identification model; ⑤Decode and extract frames from the captured real-time video stream; at the same time, preprocess the extracted video frames to make them meet the input requirements of the model; The pre-processed video frames are fed into the trained risk recognition model to identify the objects in the image and determine the anomaly category and location. Combined with the historical frame detection results, if the abnormality recognition duration is greater than the first preset threshold, it is determined to be an alarm and a first alarm signal is generated.

4. The intelligent factory comprehensive inspection system based on multiple measures according to claim 3 is characterized in that: The risk identification model includes: Build the C2f_dcnv2 module; The C2f_dcnv2 module includes the first to second Conv modules, the Split module, and the first to third Bottleneck_dcnv2 modules; The first to third Bottleneck_dcnv2 modules have the same structure, and all include a third Conv module, a deformable convolution layer DCNv2, a batch normalization layer, and a SiLU activation function layer connected in sequence. The input end of the third Conv module is connected to the external input, and the input end of the third Conv module and the output end of the SiLU activation function layer are added as the output; The first Conv module, the Split module, the first Bottleneck_dcnv2 module, the second Bottleneck_dcnv2 module, and the third Bottleneck_dcnv2 module are connected in sequence. The input end of the first Conv module serves as the input end of the C2f_dcnv2 module. The output end of the Split module, the output end of the first Bottleneck_dcnv2 module, the output end of the second Bottleneck_dcnv2 module, and the output end of the third Bottleneck_dcnv2 module are spliced ​​by channel and input into the input end of the second Conv module. The output end of the second Conv module serves as the output end of the C2f_dcnv2 module. The first to third Conv modules have the same structure, which consists of a Conv layer, a batch normalization layer, and a SiLU activation function layer connected in sequence; For the head network of the YOLOv8 model, all C2f modules are replaced with C2f_dcnv2 modules to obtain the YOLOv8 model that introduces deformable convolution; For the backbone network of the YOLOv8 model that introduces deformable convolution, all Conv modules except the first Conv module are replaced with the receptive field attention convolution module RFAConv to obtain an improved YOLOv8 model.

5. The intelligent factory comprehensive inspection system based on multiple measures according to claim 1 is characterized in that: The alarm module also includes an associated query alarm unit and an alarm information management unit; The associated query alarm unit identifies abnormal equipment through any inspection method, and can view the monitoring status and historical data of other inspection methods at the time of the abnormality through association, and multiple inspection methods can jointly confirm the abnormality and issue an alarm; The alarm information management unit receives the abnormal information pushed by the identification module, stores and manages the abnormal information, and displays it uniformly in the inspection abnormality function; at the same time, various types of abnormalities are pushed to the disposal module for disposal according to the level. The alarm module continuously pays attention to the current and historical data information of the abnormalities and the disposal result information.

6. The intelligent factory comprehensive inspection system based on multiple measures according to claim 1 is characterized in that: The analysis module performs intelligent diagnosis and health analysis of existing or potential faults of each device based on the alarm data and historical operation data of the device. Specifically: Extract vibration signals from each device's alarm data and historical operating data, and perform wavelet packet decomposition to extract feature vectors; Construct a discriminant analysis function to assess health status: The feature vectors extracted from each state are combined into a training set for FDA learning, achieving the conversion from high-dimensional space to low-dimensional space, and constructing each discriminant population and discriminant analysis function; FDA stands for Fisher discriminant analysis; Calculate the Mahalanobis distance between the energy eigenvector and the normal population, and evaluate the health CV of the device at time t by normalizing the Mahalanobis distance; Fault detection of rotating machinery: Compare the health CV of the equipment in the working state at time t with the preset third health threshold HT. If CV>HT, the equipment is in good operating condition; if CV≤HT, the equipment has a fault. Fault diagnosis of rotating machinery: Combined with the discriminant analysis function, the Mahalanobis distance between the feature vector x extracted from the working state at time t and the overall distribution of various faults is calculated, and the fault mode is determined through discriminant analysis.

7. The intelligent factory comprehensive inspection system based on multiple measures according to claim 6 is characterized in that: The discriminant analysis function is: Let ∑ -1 The first m eigenvalues ​​of B are λ1≥λ2≥…≥y m , then the corresponding eigenvectors are a1,a2,…,a m , if there is a cumulative contribution rate When the set threshold is reached, m uncorrelated typical variables y1, y2, ...y are selected. m ,y m =a' m x is used as the discriminant analysis function, where μ i and ∑ i There are k discriminant populations G i The mean vector and covariance matrix of , i=1,2…,k; λ p The subscript p is the dimension of the device to be judged, p ≥ m; a' represents the transpose of vector a, so a' m Represents vector a m The transpose of .

8. The intelligent factory comprehensive inspection system based on multiple measures according to claim 7 is characterized in that: The Mahalanobis distance is obtained as follows: Assume that the feature vector extracted by wavelet decomposition of the real-time vibration signal collected in the working state at time t is the device to be judged x, combined with the discriminant analysis function y1, y2, ..., y m ,y m =a′ m x, in the new low-dimensional space after FDA dimensionality reduction processing, the device to be judged x to the judgment population G i The Mahalanobis distance d(x,G i ) is given by y=(y1,y2,…,y m )′ to one-dimensional discriminant population The distance is calculated as: Using normalization, the obtained Mahalanobis distance is converted into a CV value. The CV value at this time can represent the current health status. A CV close to 1 indicates a good current health status, a downward trend in CV indicates a degenerative state, and a CV close to 0 indicates an abnormal state.

9. The intelligent factory comprehensive inspection system based on multiple measures according to claim 1 is characterized in that: It also includes a digital human module, which is fully integrated into the Deepseek large model. The digital human module uses multi-source inspection methods to analyze result comparison, verification, and upstream and downstream process parameter correlation analysis to assist in judging abnormal production status and track the root cause of the abnormality. After confirming the abnormality, it forms a disposal plan through contingency plans and operation card matching, and realizes intelligent interaction with inspection personnel in a digital human manner.

Images