Safety production hidden danger / risk distinguishing and troubleshooting method and training system based on visual identification

Through multi-source data fusion and deep reinforcement learning algorithms, combined with multi-level spatio-temporal analysis and causal analysis, the problems of low efficiency of traditional security inspection and insufficient intelligent analysis capabilities are solved, real-time and efficient hidden danger assessment and risk prediction are achieved, and the company has significantly improved the Production safety and reliability.

CN119089382AActive Publication Date: 2024-12-06CHIPONT (BEIJING) RES INST OF SAFETY PROD

Patent Information

Application Number
CN202411179125.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-06
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

Traditional security inspections rely on manual, slow identification speed and limited coverage, and cannot detect potential hidden dangers in a timely manner. Existing systems are inefficient when processing large-scale and complex production data, difficult to provide real-time security assessment, lack intelligent analysis capabilities, and cannot Effectively use data to conduct in-depth risk prediction and risk assessment.

Method used

Real-time acquisition and fusion of multi-source data (hyperspectral images, infrared thermal images and vibration sensing data), data monitoring and optimization are used using deep reinforcement learning algorithms, and hidden dangers are evaluated in combination with multi-level spatio-temporal analysis and causal analysis technology to generate comprehensive hidden dangers Evaluation report.

Benefits of technology

Realize real-time and efficient identification and evaluation of hidden dangers in the production process, significantly improve production safety and reliability, and improve the comprehensiveness and accuracy of hidden danger identification through multi-source data fusion and deep learning algorithms, providing dynamic feedback and real-time Adjustment ability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119089382A_ABST
    Figure CN119089382A_ABST
Patent Text Reader

Abstract

The invention relates to the field of industrial automation and intelligent control, in particular to a safety production hidden danger / risk distinguishing and troubleshooting method based on visual identification and a training system, and the method comprises the steps: carrying out the collection of a hyperspectral image, an infrared thermal image and vibration sensing data, carrying out the data cleaning, feature extraction and normalization processing, and generating fusion feature data. Carrying out real-time monitoring and comparison on the fused feature data by utilizing edge computing equipment and a deep reinforcement learning algorithm, carrying out anomaly detection and hidden danger assessment through sparse representation and multi-level space-time analysis, and finally generating a comprehensive hidden danger assessment report; the hidden danger in the production process can be efficiently identified and evaluated in real time, and the production safety and reliability are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of industrial automation and intelligent control, and in particular to a method and training system for identifying and troubleshooting production safety hazards / risks based on visual recognition. Background Art

[0002] With the improvement of industrial automation, ensuring the safety and reliability of the production process has become a top priority. Using visual recognition technology to automatically identify and troubleshoot safety hazards and risks can greatly improve production efficiency, reduce human errors, and improve production safety.

[0003] The current safety production has the following problems:

[0004] Traditional safety inspections rely on manual labor, and have problems such as slow identification speed and limited coverage, making it impossible to detect potential hidden dangers in a timely manner; existing systems are inefficient when processing large-scale, complex production data, and have difficulty providing real-time safety assessments; traditional technologies often lack intelligent analysis capabilities and cannot effectively use data for in-depth hidden danger prediction and risk assessment.

[0005] These problems are mainly caused by the following reasons:

[0006] Manual inspection is time-consuming, inaccurate, and easily affected by subjective factors; traditional data processing technology is unable to cope with the large amount of high-dimensional data generated in modern production environments; the lack of advanced algorithms and models for intelligent analysis makes it impossible to fully tap the potential information in the data.

[0007] The existing technology (Chinese invention patent, publication number: CN116523908B, title: Safety production method, system, equipment and medium based on coil coating production line) has the following shortcomings or defects in the face of these problems:

[0008] Existing technologies rely on a single type of data (such as image data), and it is difficult to integrate multi-source data for comprehensive analysis; the data processing and analysis speed is slow and cannot meet the needs of real-time monitoring and assessment; simple algorithms are used for hidden danger assessment, and it is impossible to deeply mine hidden danger information in the data; there is a lack of effective feedback and adjustment mechanisms, and it is impossible to make dynamic adjustments based on real-time data. Summary of the invention

[0009] In view of the many problems existing in the above-mentioned prior arts, the present invention provides a method and training system for identifying and troubleshooting safety production hazards / risks based on visual recognition. The present invention collects and fuses multi-source data (hyperspectral images, infrared thermal images and vibration sensor data) in real time, uses deep reinforcement learning algorithms to monitor and optimize data, combines multi-level spatiotemporal analysis and causal analysis techniques to conduct hazard assessment, and finally generates a comprehensive hazard assessment report. The present invention can identify and assess hazards in the production process in real time and efficiently, and significantly improve production safety and reliability.

[0010] A method for identifying and troubleshooting production safety hazards / risks based on visual recognition, comprising the following steps:

[0011] Real-time collection of hyperspectral image data, infrared thermal image data and vibration sensor data on the production line, and cleaning, feature extraction and normalization processing to generate fused feature data;

[0012] Using edge computing equipment and deep reinforcement learning algorithms to monitor and compare the fused feature data in real time, including: preliminary comparison, cluster analysis and deep reinforcement learning optimization, to generate optimized comparison result data;

[0013] The optimized comparison result data is subjected to anomaly detection and hidden danger assessment through sparse representation, multi-level spatiotemporal analysis and causal analysis, a comprehensive hidden danger assessment report is generated, and the benchmark model and detection parameters are adjusted according to the feedback of the comprehensive hidden danger assessment report.

[0014] Preferably, the cleaning of the hyperspectral image data, infrared thermal image data and vibration sensor data includes: noise removal and outlier processing, wherein the noise removal is achieved by a filtering algorithm, and the outlier processing is performed by detecting and eliminating abnormal data points through statistical analysis.

[0015] Preferably, the feature extraction includes: extracting edge and contour features from the hyperspectral image data after cleaning, extracting temperature features from the infrared thermal image data after cleaning, and extracting vibration mode features from the vibration sensing data after cleaning, and generating hyperspectral feature data, infrared thermal feature data and vibration feature data respectively.

[0016] Preferably, the normalization processing includes: normalizing the hyperspectral feature data, infrared thermal feature data and vibration feature data to ensure that the data are within the same order of magnitude, and the normalization processing is achieved by a linear normalization method.

[0017] Preferably, the real-time monitoring and comparison includes: performing a preliminary comparison on the fused feature data through an edge computing device to generate preliminary comparison result data, wherein the preliminary comparison is achieved by calculating the Euclidean distance between the real-time feature data and the dynamic benchmark model, and the Euclidean distance calculation formula is:

[0018]

[0019] Where D represents the Euclidean distance; n represents the number of feature dimensions; x i represents the i-th eigenvalue in the real-time feature data; y i represents the i-th eigenvalue in the dynamic baseline model.

[0020] Preferably, the cluster analysis includes: using K-means clustering technology to perform cluster analysis on the preliminary comparison result data to generate cluster result data, wherein K-means clustering is achieved by iteratively optimizing cluster centers, and the objective function of the iterative optimization is:

[0021]

[0022] Among them, J represents the objective function value; k represents the number of clusters; n represents the number of data points; represents the i-th data point belonging to the j-th class; μ j Represents the cluster center of the jth class.

[0023] Preferably, the deep reinforcement learning optimization includes: optimizing the clustering result data using a deep reinforcement learning algorithm to generate optimized comparison result data, wherein the deep reinforcement learning algorithm performs strategy optimization through a deep Q network, and the loss function of the deep Q network is:

[0024]

[0025] Where L(θ) represents the loss function; θ represents the current network parameters; s represents the current state; a represents the current action; r represents the reward; s' represents the next state; γ represents the discount factor; Q represents the Q value function; θ - represents the target network parameters; a' represents the next action.

[0026] Preferably, the anomaly detection and hidden danger assessment includes: performing anomaly pattern recognition on the optimized comparison result data by a sparse representation algorithm to generate anomaly pattern data, wherein the sparse representation algorithm realizes anomaly detection by minimizing reconstruction error, and the calculation formula of the reconstruction error is:

[0027]

[0028] Where E represents the reconstruction error; X represents the optimized comparison result data matrix; D represents the dictionary matrix; α represents the sparse coefficient vector; and λ represents the regularization parameter.

[0029] Preferably, the hidden danger assessment includes: using multi-level spatiotemporal analysis technology to perform time and space feature analysis on the abnormal pattern data to generate spatiotemporal feature data, and using causal analysis technology to perform causal relationship analysis on the spatiotemporal feature data to generate causal relationship data, and finally generating a comprehensive hidden danger assessment report, wherein the causal analysis is implemented by a Granger causality model, and the calculation formula of the Granger causality model is:

[0030]

[0031] Among them, Y t represents the explained variable; X t represents the explanatory variable; α i represents the lag coefficient of the explained variable; β j represents the lag coefficient of the explanatory variable; Y t-i represents the ith lag value of the explained variable; X t-j represents the jth lag value of the explanatory variable; ∈ t represents the error term; p represents the lag order of the explained variable; q represents the lag order of the explanatory variable.

[0032] A training system for the visual recognition-based production safety hazard / risk identification and troubleshooting method, comprising:

[0033] A data acquisition module is used to collect hyperspectral image data, infrared thermal image data and vibration sensor data on the production line in real time through a hyperspectral imaging sensor, an infrared thermal imaging sensor and a vibration sensor;

[0034] A data preprocessing and fusion module is used to clean, extract features and normalize the hyperspectral image data, infrared thermal image data and vibration sensor data to generate fused feature data;

[0035] A real-time monitoring and comparison module, which is used to use edge computing equipment and deep reinforcement learning algorithms to perform real-time monitoring and comparison on the fused feature data, including preliminary comparison, cluster analysis and deep reinforcement learning optimization, and generate optimized comparison result data;

[0036] Anomaly detection and hidden danger assessment module, used to perform anomaly detection and hidden danger assessment on the optimized comparison result data through sparse representation, multi-level spatiotemporal analysis and causal analysis, generate a comprehensive hidden danger assessment report, and adjust the benchmark model and detection parameters according to the feedback of the comprehensive hidden danger assessment report;

[0037] A model training module, used to train and optimize the benchmark model using the fused feature data and the optimized comparison result data to generate a dynamic benchmark model;

[0038] A model verification module, used to verify and evaluate the dynamic benchmark model using a verification data set to ensure the accuracy and robustness of the model;

[0039] The deployment and update module is used to deploy the verified dynamic benchmark model to the production environment and dynamically update the model parameters based on real-time data feedback.

[0040] Compared with the prior art, the advantages and beneficial effects of the present invention are:

[0041] The present invention achieves efficient and comprehensive hidden danger identification effect by introducing multi-source data fusion technology;

[0042] The present invention achieves real-time data monitoring and rapid response effects by utilizing edge computing devices and deep reinforcement learning algorithms;

[0043] The present invention realizes intelligent hidden danger identification and risk assessment through deep reinforcement learning and sparse representation algorithm;

[0044] The present invention realizes dynamic feedback and real-time adjustment effects of hidden danger assessment through multi-level spatiotemporal analysis and cause-effect analysis technology;

[0045] The present invention improves the comprehensiveness and accuracy of hidden danger identification through multi-modal data fusion;

[0046] The present invention improves the efficiency and quality of data processing through efficient data cleaning, feature extraction and normalization processing;

[0047] The present invention is suitable for identifying and troubleshooting safety hazards and risks in different industrial scenarios through the scalability and robustness of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a schematic diagram of the process of the present invention;

[0049] Figure 2 It is a schematic diagram of multi-source data fusion in the present invention;

[0050] Figure 3 It is a real-time monitoring and comparison flow chart in the present invention;

[0051] Figure 4 This is a flowchart for optimizing deep reinforcement learning in the present invention;

[0052] Figure 5 This is a flow chart of anomaly detection and hidden danger assessment in the present invention;

[0053] Figure 6 It is a structural block diagram of the system of the present invention. DETAILED DESCRIPTION

[0054] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present disclosure. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is obvious that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0055] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0056] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification, and should not be interpreted in an idealized or overly rigid manner.

[0057] In the case of using expressions such as "at least one of A, B, and C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.). In the case of using expressions such as "at least one of A, B, or C, etc.", it should generally be interpreted in accordance with the meaning of the expression generally understood by those skilled in the art (for example, "a system having at least one of A, B, or C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0058] Some block diagrams and / or flow charts are shown in the accompanying drawings. It should be understood that some boxes or combinations thereof in the block diagrams and / or flow charts can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that these instructions can create a device for implementing the functions / operations described in these block diagrams and / or flow charts when executed by the processor. The technology of the present disclosure can be implemented in the form of hardware and / or software (including firmware, microcode, etc.). In addition, the technology of the present disclosure can take the form of a computer program product on a computer-readable storage medium storing instructions, which can be used by an instruction execution system or used in conjunction with an instruction execution system.

[0059] like Figure 1 As shown, a method for identifying and troubleshooting production safety hazards / risks based on visual recognition includes the following steps:

[0060] like Figure 2 As shown, the hyperspectral image data, infrared thermal image data and vibration sensor data on the production line are collected in real time, and cleaned, feature extracted and normalized to generate fused feature data;

[0061] Preferably, the cleaning of the hyperspectral image data, infrared thermal image data and vibration sensor data includes: noise removal and outlier processing, wherein the noise removal is achieved by a filtering algorithm, and the outlier processing is performed by detecting and eliminating abnormal data points through statistical analysis.

[0062] In the present invention, noise removal is achieved through a filtering algorithm. The filtering algorithm is a mature technical means in the field of signal processing, and generally includes low-pass filtering, high-pass filtering, and band-pass filtering. The principle is to suppress or remove noise signals of a specific frequency by processing the data in the frequency domain or time domain. The low-pass filter is used to remove high-frequency noise, the high-pass filter is used to remove low-frequency noise, and the band-pass filter is used to retain the signal within a specific frequency band. The noise component of the data processed by the filtering algorithm is significantly reduced, so that subsequent feature extraction and analysis are more accurate and reliable. Specific embodiments include the commonly used mean filtering and median filtering in the processing of hyperspectral image data. The mean filter smoothes out random noise by averaging the image pixel values; the median filter effectively removes salt and pepper noise by selecting the median in the window as the pixel value. In the processing of infrared thermal image data, Gaussian filtering can be used to remove high-frequency noise and maintain the smoothness of the temperature distribution. In the processing of vibration sensor data, low-pass filtering can remove high-frequency vibration signals, making the main vibration mode clearer.

[0063] The main purpose of outlier processing is to detect and remove abnormal data points introduced in the data due to sensor failure, data transmission errors, etc. These outliers may significantly affect the data analysis results and must be processed. The principle is to identify values ​​that deviate from the normal range through statistical analysis methods. Commonly used methods include Z-score, box plot method, and density-based local outlier factor (LOF). After the outlier is detected, it is processed by interpolation, deletion or replacement to ensure the consistency and continuity of the data. After outlier processing, the extreme deviation in the data is eliminated, making the data more representative and stable, ensuring the reliability of subsequent analysis. Specific embodiments include that in the processing of hyperspectral image data, the pixel value anomaly can be detected by the Z-score method, and the pixels beyond a certain standard deviation range are marked as outliers, and then replaced by the average value of the surrounding pixels. In the processing of infrared thermal image data, the box plot method can be used to detect abnormal high or low temperature points in the temperature distribution, and processed by the interpolation method. In vibration sensor data processing, the density-based local outlier factor (LOF) method can effectively detect abnormal vibration signals caused by sensor failure and ensure data continuity by deleting these outliers.

[0064] By removing noise and processing outliers from hyperspectral image data, infrared thermal image data, and vibration sensor data, the high quality and consistency of the data are ensured. This process provides a reliable foundation for subsequent data fusion, feature extraction, real-time monitoring and comparison, anomaly detection, and hidden danger assessment. The application of filtering algorithms and statistical analysis methods not only improves the accuracy and stability of the data, but also effectively reduces the probability of false alarms and missed alarms, making the entire safety production hidden danger / risk identification and investigation system more efficient and reliable.

[0065] Preferably, the feature extraction includes: extracting edge and contour features from the hyperspectral image data after cleaning, extracting temperature features from the infrared thermal image data after cleaning, and extracting vibration mode features from the vibration sensing data after cleaning, and generating hyperspectral feature data, infrared thermal feature data and vibration feature data respectively.

[0066] The principle of extracting edge and contour features from cleaned hyperspectral image data is to use image processing algorithms, such as Canny edge detection and Sobel operator, to extract edges by detecting areas where pixel values ​​change significantly in the image. These algorithms identify edges and contours in images by calculating the gradient and second-order derivative of the image. These edge and contour features can reveal the geometric shape and structural changes of the workpiece, and are an important basis for identifying potential hidden dangers. The data after edge and contour feature extraction is called hyperspectral feature data, which has the effect of accurately reflecting the surface characteristics of the workpiece and helping to identify possible physical defects or damage. For example, in hyperspectral image processing, the use of the Canny edge detection algorithm can effectively extract cracks and scratches on the surface of the workpiece. Through these feature data, the length and direction of the cracks can be further analyzed to provide reliable data support for hidden danger assessment.

[0067] The principle of extracting temperature features from infrared thermal image data after cleaning is to use the pixel values ​​in the image to represent temperature information. By extracting these pixel values, the temperature distribution of the workpiece can be obtained. The temperature features in infrared thermal images can reveal thermal anomalies generated by the workpiece during operation. These thermal anomalies may be caused by equipment overheating, friction or other failures. The data after temperature feature extraction is called infrared thermal feature data, which has the effect of accurately capturing the temperature changes of the workpiece and helping to discover potential thermal hazards. For example, in infrared thermal image processing, the high-temperature area can be extracted through the temperature threshold segmentation method, the overheated parts of the workpiece can be identified, and the cause and scope of overheating can be further analyzed through the temperature feature data, providing key data for preventing equipment failures.

[0068] The principle of extracting vibration mode features from vibration sensor data after cleaning is to use signal processing algorithms, such as Fourier transform and wavelet transform, to convert the vibration signal in the time domain into frequency domain features, and identify the main frequency components and energy distribution in the vibration signal. Vibration mode features can reflect the vibration state of the workpiece during operation, and these vibration states can reveal mechanical failures, looseness or other abnormal conditions of the equipment. The data after vibration mode feature extraction is called vibration feature data, which has the effect of accurately reflecting the vibration characteristics of the workpiece and helping to identify potential mechanical hazards. For example, in the processing of vibration sensor data, the main vibration frequency components can be extracted through Fourier transform to identify whether the workpiece has resonance. Through these feature data, the causes and possible solutions of the resonance can be further analyzed, providing an important basis for mechanical maintenance.

[0069] By extracting edge and contour features, temperature features and vibration mode features from hyperspectral image data, infrared thermal image data and vibration sensor data, hyperspectral feature data, infrared thermal feature data and vibration feature data are generated respectively. These feature data provide rich basic information for subsequent real-time monitoring and comparison, anomaly detection and hidden danger assessment, ensuring that the system can accurately identify and assess safety hazards in production.

[0070] Preferably, the normalization processing includes: normalizing the hyperspectral feature data, infrared thermal feature data and vibration feature data to ensure that the data are within the same order of magnitude, and the normalization processing is achieved by a linear normalization method.

[0071] Normalization is achieved through linear normalization, which works by scaling the data to a predefined range, usually between 0 and 1.

[0072] The calculation formula for linear normalization is:

[0073]

[0074] Among them, x' represents the normalized value, x represents the original value, and x min and x max Represent the minimum and maximum values ​​of the original data respectively. Through linear normalization, all feature data are scaled to the same magnitude range, which allows different types of feature data to be compared and analyzed on the same scale, avoiding deviations and errors caused by differences in data ranges.

[0075] In the present invention, the specific application of normalization processing to hyperspectral feature data, infrared thermal feature data and vibration feature data is as follows:

[0076] For hyperspectral feature data, normalization ensures that edge and contour features extracted from different spectral channels can be compared and analyzed at the same level. The feature values ​​of each channel of a hyperspectral image may vary greatly. Through normalization, these differences can be reduced, making subsequent feature fusion and analysis more accurate. For example, in hyperspectral image processing, the spectral reflectance values ​​of some channels may be between 0 and 100, while the values ​​of other channels may be between 0 and 1. Through normalization, the values ​​of all channels are scaled to the range of 0 to 1, so that the feature values ​​of different channels can be fused and analyzed on the same scale, thereby improving the accuracy of the overall analysis.

[0077] For infrared thermal feature data, normalization ensures that data in different temperature ranges can be compared and analyzed on the same scale. The temperature data in infrared thermal images may have large differences in numerical ranges. Through normalization, the temperature data can be scaled to the range of 0 to 1, so that the temperature feature data can be fused and analyzed on the same scale as other types of feature data. For example, in infrared thermal image processing, the temperature in different areas may vary between 20°C and 100°C. Through normalization, the temperature values ​​are scaled to the range of 0 to 1, so that the data in different temperature areas can be compared on the same scale, which is convenient for identifying temperature anomalies and hidden dangers.

[0078] For vibration feature data, normalization ensures that data in different frequency and amplitude ranges can be compared and analyzed on the same scale. There may be large differences in the frequency and amplitude of vibration sensing data. Through normalization, these differences can be reduced so that vibration mode features can be fused and analyzed on the same order of magnitude as other types of feature data. For example, in vibration sensing data processing, the vibration frequency and amplitude of different mechanical parts may vary significantly. Through normalization, the frequency and amplitude values ​​are scaled to the range of 0 to 1, so that the vibration feature data of different mechanical parts can be compared on the same scale, which is convenient for identifying mechanical failures and hidden dangers.

[0079] Through normalization processing, hyperspectral feature data, infrared thermal feature data, and vibration feature data are compared and analyzed within the same magnitude range, ensuring the accuracy and consistency of data fusion and analysis. This process provides a reliable basis for subsequent real-time monitoring and comparison, anomaly detection, and hidden danger assessment, avoiding analysis deviations caused by differences in data magnitude, thereby improving the overall accuracy and reliability of the system. The application of normalization processing not only enhances the comparability between different feature data, but also effectively improves the efficiency of data processing, making the entire system more efficient and accurate in processing and analyzing multimodal data.

[0080] Using edge computing equipment and deep reinforcement learning algorithms to monitor and compare the fused feature data in real time, including: preliminary comparison, cluster analysis and deep reinforcement learning optimization, to generate optimized comparison result data;

[0081] like Figure 3 As shown, preferably, the real-time monitoring and comparison includes: performing a preliminary comparison on the fused feature data through an edge computing device to generate preliminary comparison result data, wherein the preliminary comparison is achieved by calculating the Euclidean distance between the real-time feature data and the dynamic benchmark model, and the Euclidean distance calculation formula is:

[0082]

[0083] Where D represents the Euclidean distance; n represents the number of feature dimensions; x i represents the i-th eigenvalue in the real-time feature data; y i represents the i-th eigenvalue in the dynamic baseline model.

[0084] Euclidean distance is a common method for measuring the straight-line distance between two points. In the present invention, it is used to compare the difference between real-time feature data and the dynamic benchmark model. The dynamic benchmark model represents the feature data under normal production conditions. By comparing the real-time feature data with the benchmark model, abnormal situations can be effectively identified.

[0085] In actual applications, the principle of the real-time monitoring and comparison step is to process the collected fusion feature data in real time through the edge computing device, and calculate the Euclidean distance between these data and the pre-built dynamic benchmark model. A larger Euclidean distance indicates that the real-time data is significantly different from the benchmark model, and there may be anomalies; while a smaller Euclidean distance indicates that the real-time data is similar to the benchmark model and the production status is normal.

[0086] Effectively, by calculating the Euclidean distance in real time, the system can quickly identify data points that deviate significantly from the benchmark model, indicating potential safety hazards. For example, in the processing of hyperspectral image data, the reflectivity of certain spectral channels in the real-time feature data deviates significantly from the benchmark model, which may indicate the presence of new scratches or cracks on the surface of the workpiece; in the processing of infrared thermal image data, the real-time temperature feature data differs greatly from the benchmark model, which may indicate local overheating of the equipment; in the processing of vibration sensor data, the real-time vibration feature data differs greatly from the benchmark model, which may indicate loose or faulty mechanical parts.

[0087] Specific embodiments include:

[0088] Hyperspectral image data processing: In the real-time acquired hyperspectral image data, the possible physical defects on the workpiece surface are identified by calculating the difference between the reflectance of each spectral channel and the benchmark model. For example, if the reflectance value of a channel is significantly higher than the benchmark model, it indicates that there may be scratches on the surface.

[0089] Infrared thermal image data processing: In the real-time collected infrared thermal image data, the thermal anomaly of the equipment is identified by calculating the difference between the temperature characteristic data and the reference model. For example, if the temperature of a certain area is significantly higher than the reference model, it indicates that there may be overheating.

[0090] Vibration sensor data processing: In the real-time collected vibration sensor data, the vibration anomaly of the mechanical parts is identified by calculating the difference between the vibration characteristic data and the reference model. For example, the vibration amplitude of a certain frequency is significantly higher than the reference model, indicating that there may be mechanical looseness.

[0091] Through the method of the present invention, the real-time monitoring and comparison steps play a key role in the production safety hazard / risk identification and investigation system based on visual recognition, and can timely and accurately identify various hidden dangers that may exist in the production process, thereby improving the safety and reliability of the production process.

[0092] Preferably, the cluster analysis includes: using K-means clustering technology to perform cluster analysis on the preliminary comparison result data to generate cluster result data, wherein K-means clustering is achieved by iteratively optimizing cluster centers, and the objective function of the iterative optimization is:

[0093]

[0094] Among them, J represents the objective function value; k represents the number of clusters; n represents the number of data points; represents the i-th data point belonging to the j-th class; μ j Represents the cluster center of the jth class.

[0095] The cluster analysis step includes using K-means clustering technology to perform cluster analysis on the preliminary comparison result data to generate cluster result data, wherein K-means clustering is achieved by iteratively optimizing the cluster center. The K-means clustering algorithm is a commonly used unsupervised learning method, which is mainly used to assign data points to k clusters so that the data points in the same cluster are as similar as possible, while the data points in different clusters are as different as possible.

[0096] The principle of K-means clustering is to optimize the cluster center in an iterative way so that the distance between each data point and its cluster center is minimized. The specific process includes initializing k cluster centers, then assigning each data point to the cluster center closest to it, and then updating the cluster center to the average value of all data points in the cluster. Repeat this process until the cluster center no longer changes significantly.

[0097] In the present invention, the application effect of the cluster analysis step is reflected in the following aspects:

[0098] First, cluster analysis can effectively classify the preliminary comparison result data to help identify different types of potential hidden dangers. For example, in hyperspectral image data processing, cluster analysis can classify workpieces with different surface features into multiple clusters, thereby identifying workpieces with cracks, scratches or other physical defects. In infrared thermal image data processing, cluster analysis can classify areas with different temperature characteristics into multiple clusters, thereby identifying areas that may have overheating or abnormal temperature distribution. In vibration sensing data processing, cluster analysis can classify mechanical parts with different vibration modes into multiple clusters, thereby identifying parts that may have mechanical failures or abnormal vibrations.

[0099] Secondly, the iterative optimization process of K-means clustering ensures the stability and accuracy of the clustering results. By optimizing the cluster centers, the data points in the same cluster are made more similar and the differences between different clusters are made more significant, thereby improving the effect of cluster analysis. For example, in the cluster analysis process, iterative optimization can ensure that the edge and contour features of all hyperspectral image data points belonging to a cluster are similar, the temperature features of all infrared thermal image data points belonging to a cluster are similar, and the vibration mode features of all vibration sensor data points belonging to a cluster are similar, thereby improving the accuracy of hidden danger identification.

[0100] Specific embodiments include:

[0101] Hyperspectral image data processing: The K-means clustering algorithm is used to divide the different edge and contour feature data points on the workpiece surface into multiple clusters, thereby identifying workpieces with different surface conditions. For example, clustering the hyperspectral data points with cracks on the workpiece surface together can quickly identify workpieces with cracks.

[0102] Infrared thermal image data processing: The K-means clustering algorithm is used to divide the data points of different temperature characteristics of the equipment into multiple clusters, thereby identifying areas with different temperature distributions. For example, clustering the data points of the overheating area of ​​the equipment together can quickly identify parts that may have overheating problems.

[0103] Vibration sensor data processing: The K-means clustering algorithm is used to group the data points of different vibration modes of mechanical parts into multiple clusters, thereby identifying parts with different vibration states. For example, clustering the data points of mechanical resonance together can quickly identify parts with resonance problems.

[0104] Through the above method, the cluster analysis step plays an important role in the safety production hazard / risk identification and investigation system based on visual recognition. It can effectively classify and identify different types of hazards and improve the detection accuracy and reliability of the system.

[0105] like Figure 4 As shown, preferably, the deep reinforcement learning optimization includes: using a deep reinforcement learning algorithm to optimize the clustering result data to generate optimized comparison result data, wherein the deep reinforcement learning algorithm performs strategy optimization through a deep Q network, and the loss function of the deep Q network is:

[0106]

[0107] Where L(θ) represents the loss function; θ represents the current network parameters; s represents the current state; a represents the current action; r represents the reward; s' represents the next state; γ represents the discount factor; Q represents the Q value function; θ -represents the target network parameters; a' represents the next action.

[0108] The present invention uses a deep reinforcement learning algorithm to optimize the clustering result data and generate optimized comparison result data. Specifically, the deep reinforcement learning algorithm performs strategy optimization through a deep Q network and uses the loss function of the deep Q network to optimize the prediction performance of the model.

[0109] Deep Q-Network (DQN) is a commonly used deep reinforcement learning algorithm that combines Q-learning and deep neural networks to process high-dimensional state spaces. The core idea is to approximate the Q-value function through a neural network to find the optimal strategy. In the present invention, the loss function of the deep Q-network is used to evaluate the prediction error of the model and update the parameters through the back-propagation algorithm.

[0110] The optimized comparison result data is subjected to anomaly detection and hidden danger assessment through sparse representation, multi-level spatiotemporal analysis and causal analysis, a comprehensive hidden danger assessment report is generated, and the benchmark model and detection parameters are adjusted according to the feedback of the comprehensive hidden danger assessment report.

[0111] In the present invention, the principle of the deep reinforcement learning optimization step is that through the deep Q network, the system can continuously learn and improve its strategy during real-time monitoring and comparison. The specific process includes: the system collects the current state s at each time step, selects an action a, and obtains an immediate reward r and the next state s' according to the selected action. The system calculates the Q value through the deep Q network and selects the optimal action by maximizing future rewards. During the training process, the system continuously optimizes the Q value function so that it can accurately predict the value of different states and actions, thereby improving the decision-making ability of the system.

[0112] In effect, through deep reinforcement learning optimization, the system can more effectively identify and evaluate hidden dangers in complex production environments. Specifically, the system can dynamically adjust its strategy to respond to different types and levels of risks, and improve the accuracy and timeliness of hidden danger identification. For example, in hyperspectral image data processing, through deep reinforcement learning, the system can more accurately identify tiny cracks on the surface of the workpiece, and adjust the detection strategy according to the development trend of the cracks; in infrared thermal image data processing, the system can optimize its detection model according to real-time temperature changes, and more quickly identify overheating problems of equipment; in vibration sensor data processing, the system can optimize its detection strategy according to changes in vibration patterns, and more accurately identify loose or faulty mechanical parts.

[0113] Specific embodiments include:

[0114] Hyperspectral image data processing: By optimizing the crack detection model through the deep Q network, the system can adjust the detection strategy according to the real-time collected hyperspectral data to improve the accuracy and response speed of crack identification. For example, when the system identifies a new crack on the surface of the workpiece, it can dynamically adjust its detection parameters to ensure continuous monitoring and evaluation of the crack.

[0115] Infrared thermal image data processing: By optimizing the temperature detection model through the deep Q network, the system can optimize the detection strategy according to the real-time temperature changes of the equipment, quickly identify overheating problems and take corresponding measures. For example, when the system detects that the temperature of a certain part of the equipment rises rapidly, it can promptly issue an alarm and adjust the detection frequency to ensure that the problem is quickly handled.

[0116] Vibration sensor data processing: By optimizing the vibration detection model through the deep Q network, the system can optimize the detection strategy according to the real-time vibration pattern of the mechanical parts and accurately identify possible faults and hidden dangers. For example, when the system identifies an abnormal change in the vibration frequency of a mechanical part, it can adjust its detection parameters and strategies to ensure that the fault is discovered and handled in a timely manner.

[0117] Through the above method, the deep reinforcement learning optimization step plays an important role in the safety production hazard / risk identification and investigation system based on visual recognition. It can dynamically adjust and optimize the detection strategy, improve the system's response speed and accuracy, and thus more effectively ensure the safety and reliability of the production process.

[0118] like Figure 5 As shown, preferably, the anomaly detection and hidden danger assessment includes: performing anomaly pattern recognition on the optimized comparison result data through a sparse representation algorithm to generate abnormal pattern data, wherein the sparse representation algorithm realizes anomaly detection by minimizing the reconstruction error, and the calculation formula of the reconstruction error is:

[0119]

[0120] Where E represents the reconstruction error; X represents the optimized comparison result data matrix; D represents the dictionary matrix; α represents the sparse coefficient vector; and λ represents the regularization parameter.

[0121] The anomaly detection and hidden danger assessment step is a key step, which is used to identify and assess abnormal situations and potential hidden dangers in the production process. This step includes using a sparse representation algorithm to perform abnormal pattern recognition on the optimized comparison result data to generate abnormal pattern data. The sparse representation algorithm achieves anomaly detection by minimizing reconstruction errors.

[0122] The principle of the sparse representation algorithm is to represent the data as a linear combination of dictionary matrices, where the sparse coefficient vector represents the weight of the linear combination. By minimizing the reconstruction error, the best approximation to the data can be achieved, thereby effectively identifying abnormal patterns in the data. Specifically, the sparse representation algorithm achieves anomaly detection by finding the sparsest coefficient vector so that the original data can be reconstructed as accurately as possible through the linear combination of the dictionary matrix. The regularization parameter λ is used to balance the reconstruction error and sparsity, so that the model can accurately reconstruct the data while maintaining sparsity.

[0123] In the present invention, the application effects of the anomaly detection and hidden danger assessment steps are reflected in the following aspects:

[0124] First, sparse representation algorithms can effectively extract key features from large amounts of data and identify potential abnormal patterns. For example, in hyperspectral image data processing, sparse representation algorithms can identify spectral anomalies caused by surface defects. Through these abnormal pattern data, the type and severity of the defects can be further analyzed. In infrared thermal image data processing, sparse representation algorithms can identify temperature anomalies caused by equipment overheating. Through these abnormal pattern data, the cause and scope of overheating can be further analyzed. In vibration sensor data processing, sparse representation algorithms can identify vibration anomalies caused by mechanical failures. Through these abnormal pattern data, the location and cause of the failure can be further analyzed.

[0125] Secondly, sparse representation algorithms have significant advantages in processing high-dimensional data, and can provide high-precision anomaly detection results while ensuring computational efficiency. By minimizing reconstruction errors, sparse representation algorithms can accurately locate abnormal patterns in high-dimensional data space, improving the accuracy and reliability of anomaly detection.

[0126] Specific embodiments include:

[0127] Hyperspectral image data processing: In hyperspectral image data processing, sparse representation algorithms generate hyperspectral anomaly pattern data by identifying anomalies in spectral features. For example, if the reflectance of certain spectral channels deviates significantly from the normal value, the sparse representation algorithm can discover these anomalies through dictionary matrix reconstruction and mark them as potential surface defects.

[0128] Infrared thermal image data processing: In infrared thermal image data processing, the sparse representation algorithm generates infrared thermal anomaly pattern data by identifying anomalies in temperature distribution. For example, if the temperature in a certain area is significantly higher than the normal operating range, the sparse representation algorithm can discover these anomalies through dictionary matrix reconstruction and mark them as potential equipment overheating problems.

[0129] Vibration sensor data processing: In vibration sensor data processing, sparse representation algorithms generate vibration anomaly pattern data by identifying anomalies in vibration characteristics. For example, if the vibration frequency and amplitude of a mechanical component deviate significantly from the normal operating state, the sparse representation algorithm can discover these anomalies through dictionary matrix reconstruction and mark them as potential mechanical failures.

[0130] Through the above method, the anomaly detection and hidden danger assessment steps play an important role in the production safety hidden danger / risk identification and investigation system based on visual recognition. It can efficiently and accurately identify and evaluate various abnormal situations and potential hidden dangers in the production process, thereby improving the safety and reliability of the system.

[0131] Preferably, the hidden danger assessment includes: using multi-level spatiotemporal analysis technology to perform time and space feature analysis on the abnormal pattern data to generate spatiotemporal feature data, and using causal analysis technology to perform causal relationship analysis on the spatiotemporal feature data to generate causal relationship data, and finally generating a comprehensive hidden danger assessment report, wherein the causal analysis is implemented by a Granger causality model, and the calculation formula of the Granger causality model is:

[0132]

[0133] Among them, Y t represents the explained variable; X t represents the explanatory variable; α i represents the lag coefficient of the explained variable; β j represents the lag coefficient of the explanatory variable; Y t-i represents the ith lag value of the explained variable; X t-j represents the jth lag value of the explanatory variable; ∈ t represents the error term; p represents the lag order of the explained variable; q represents the lag order of the explanatory variable.

[0134] The hidden danger assessment step is an important part of ensuring the safety of the production process. This step includes using multi-level spatiotemporal analysis technology to analyze the time and space characteristics of abnormal pattern data, generating spatiotemporal characteristic data, and using causal analysis technology to analyze the causal relationship of spatiotemporal characteristic data, and finally generating a comprehensive hidden danger assessment report.

[0135] The principle of multi-level spatiotemporal analysis technology is to identify potential hidden danger patterns by comprehensively analyzing abnormal pattern data in time and space dimensions. Temporal feature analysis focuses on the changing trend and periodicity of abnormal patterns in time, while spatial feature analysis focuses on the distribution and aggregation of abnormal patterns in space. By combining time and space feature analysis, the system can fully understand the occurrence, development and propagation of abnormal patterns.

[0136] Causal analysis techniques further perform causal analysis on spatiotemporal feature data through the Granger causality model. The Granger causality model is used to determine whether a time series can be used to predict another time series. The core idea is that if the past value of a variable can significantly improve the prediction accuracy of the future value of another variable, the former is considered to be the Granger cause of the latter. Specifically, by performing a lag analysis on the explained variable and the explanatory variable, the causal relationship between the two is identified, thereby providing a deep understanding of the causes of abnormal patterns.

[0137] In the present invention, the application effect of the hidden danger assessment step is reflected in the following aspects:

[0138] First, through multi-level spatiotemporal analysis technology, the system can identify the characteristics of abnormal patterns in time and space, and improve the accuracy of hidden danger assessment. For example, in the processing of hyperspectral image data, the development trend of surface defects can be identified through time feature analysis; the distribution of surface defects can be identified through spatial feature analysis. In the processing of infrared thermal image data, the periodic changes of equipment overheating can be identified through time feature analysis; the distribution of overheating areas can be identified through spatial feature analysis. In the processing of vibration sensor data, the changes in vibration patterns of mechanical failures can be identified through time feature analysis; and the locations of vibration abnormalities can be identified through spatial feature analysis.

[0139] Secondly, through causal analysis technology, the system can determine the causal relationship between abnormal patterns and potential hidden dangers, and improve the depth and breadth of hidden danger assessment. For example, through the Granger causality model, the system can identify the causal relationship between surface defects and production process parameters in hyperspectral image data; identify the causal relationship between equipment overheating and ambient temperature changes in infrared thermal image data; identify the causal relationship between mechanical failures and operating loads in vibration sensor data.

[0140] Specific embodiments include:

[0141] Hyperspectral image data processing: Through multi-level spatiotemporal analysis, the temporal variation trend and spatial distribution of cracks on the workpiece surface can be identified; through the Granger causality model, the causal relationship between cracks and specific production process parameters can be identified, thereby providing the causes of cracks and possible solutions.

[0142] Infrared thermal image data processing: Through multi-level spatiotemporal analysis, the temporal variation pattern and spatial distribution characteristics of equipment overheating can be identified; through the Granger causality model, the causal relationship between overheating and ambient temperature fluctuations can be identified, thereby providing the causes and preventive measures for equipment overheating.

[0143] Vibration sensor data processing: Through multi-level spatiotemporal analysis, the time variation pattern and spatial location of abnormal vibration of mechanical parts can be identified; through the Granger causality model, the causal relationship between abnormal vibration and operating load changes can be identified, thereby providing cause analysis of mechanical failures and maintenance recommendations.

[0144] Through the above method, the hidden danger assessment step plays an important role in the production safety hidden danger / risk identification and investigation system based on visual recognition. It can comprehensively and accurately assess various hidden dangers in the production process, provide in-depth cause-effect analysis and detailed hidden danger assessment reports, thereby effectively ensuring the safety and reliability of the production process.

[0145] like Figure 6 As shown, a training system for the method for identifying and troubleshooting production safety hazards / risks based on visual recognition comprises:

[0146] The data acquisition module is used to collect hyperspectral image data, infrared thermal image data and vibration sensing data on the production line in real time through hyperspectral imaging sensors, infrared thermal imaging sensors and vibration sensors; the data acquisition module is used to collect hyperspectral image data, infrared thermal image data and vibration sensing data on the production line in real time through hyperspectral imaging sensors, infrared thermal imaging sensors and vibration sensors. The hyperspectral imaging sensor can capture the surface details of the workpieces on the production line, and can identify surface defects such as cracks and scratches by analyzing the reflectivity of different spectral channels. The infrared thermal imaging sensor captures the temperature distribution of the workpiece and equipment, helping to identify overheated areas and potential thermal hazards. The vibration sensor monitors the vibration of mechanical equipment, and by analyzing the vibration pattern, mechanical failures can be detected early. Through the data collection of these three sensors, the system can obtain comprehensive information about the production line, laying the foundation for subsequent data processing and analysis.

[0147] The data preprocessing and fusion module is used to clean, extract features and normalize the hyperspectral image data, infrared thermal image data and vibration sensor data to generate fused feature data; the data preprocessing and fusion module is used to clean, extract features and normalize the hyperspectral image data, infrared thermal image data and vibration sensor data to generate fused feature data. The data cleaning step includes noise removal and outlier processing to ensure data quality. The feature extraction step extracts edge and contour features from the hyperspectral image, temperature features from the infrared thermal image, and vibration mode features from the vibration sensor data. The normalization process standardizes different data types through a linear normalization method so that they are within the same order of magnitude, which is convenient for subsequent analysis. The fused feature data integrates the above feature data into a comprehensive data set through multi-source data fusion technology to improve the comprehensiveness and accuracy of the data.

[0148] The real-time monitoring and comparison module is used to use edge computing devices and deep reinforcement learning algorithms to perform real-time monitoring and comparison of the fused feature data, including preliminary comparison, cluster analysis and deep reinforcement learning optimization, and generate optimized comparison result data; the real-time monitoring and comparison module is used to use edge computing devices and deep reinforcement learning algorithms to perform real-time monitoring and comparison of the fused feature data, including preliminary comparison, cluster analysis and deep reinforcement learning optimization, and generate optimized comparison result data. The preliminary comparison determines whether the current production status is normal or not by calculating the Euclidean distance between the real-time feature data and the dynamic benchmark model. Cluster analysis uses the K-means algorithm to classify the preliminary comparison results and identify different types of hidden dangers. Deep reinforcement learning optimization optimizes the comparison strategy through a deep Q network to improve the accuracy and timeliness of hidden danger identification. This module can monitor data changes in the production process in real time, quickly identify and classify potential hidden dangers, and ensure production safety.

[0149] The anomaly detection and hidden danger assessment module is used to perform anomaly detection and hidden danger assessment on the optimized comparison result data through sparse representation, multi-level spatiotemporal analysis and causal analysis, generate a comprehensive hidden danger assessment report, and adjust the benchmark model and detection parameters according to the feedback of the comprehensive hidden danger assessment report; the anomaly detection and hidden danger assessment module is used to perform anomaly detection and hidden danger assessment on the optimized comparison result data through sparse representation, multi-level spatiotemporal analysis and causal analysis, generate a comprehensive hidden danger assessment report, and adjust the benchmark model and detection parameters according to the feedback of the comprehensive hidden danger assessment report. The sparse representation algorithm identifies abnormal patterns by minimizing reconstruction errors, and the multi-level spatiotemporal analysis combines time and space characteristics to analyze the changing trends and distribution laws of abnormal data. Causal analysis uses the Granger causality model to identify the causal relationship of potential hidden dangers. Through these technologies, the system can comprehensively evaluate the hidden dangers in the production process, generate detailed assessment reports, and adjust the detection model according to real-time feedback to improve the system's adaptability and accuracy.

[0150] The model training module is used to train and optimize the benchmark model using the fused feature data and the optimized comparison result data to generate a dynamic benchmark model. The model training module is used to train and optimize the benchmark model using the fused feature data and the optimized comparison result data to generate a dynamic benchmark model. This module trains the data through a deep learning algorithm, continuously updates and optimizes the benchmark model so that it can accurately reflect the normal state of the production process. Through continuous model training, the system can cope with changes under different production conditions and improve the robustness and reliability of hidden danger identification.

[0151] The model verification module is used to verify and evaluate the dynamic benchmark model using the verification data set to ensure the accuracy and robustness of the model. The model verification module is used to verify and evaluate the dynamic benchmark model using the verification data set to ensure the accuracy and robustness of the model. This module tests the dynamic benchmark model through the verification data set, evaluates the prediction performance and accuracy of the model, and ensures the reliability of the model in practical applications. Through model verification, the system can continuously improve and optimize the benchmark model and improve the accuracy of hidden danger identification.

[0152] The deployment and update module is used to deploy the verified dynamic benchmark model into the production environment and dynamically update the model parameters according to real-time data feedback. The deployment and update module is used to deploy the verified dynamic benchmark model into the production environment and dynamically update the model parameters according to real-time data feedback. This module applies the verified benchmark model to the actual production process and dynamically adjusts it according to real-time monitoring data to ensure that the model is always in the best state. Through real-time updates, the system can quickly respond to changes in production and improve the real-time and accuracy of hidden danger identification.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The present application is described with reference to flowcharts and / or block diagrams of methods, devices (systems) and computer program products according to embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0155] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0156] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0157] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0158] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0159] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0160] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0161] The above are only embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for identifying and troubleshooting production safety hazards / risks based on visual recognition, characterized in that: The following steps are involved: Real-time collection of hyperspectral image data, infrared thermal image data and vibration sensor data on the production line, and cleaning, feature extraction and normalization processing to generate fused feature data; Using edge computing equipment and deep reinforcement learning algorithms to monitor and compare the fused feature data in real time, including: preliminary comparison, cluster analysis and deep reinforcement learning optimization, to generate optimized comparison result data; The optimized comparison result data is subjected to anomaly detection and hidden danger assessment through sparse representation, multi-level spatiotemporal analysis and causal analysis, a comprehensive hidden danger assessment report is generated, and the benchmark model and detection parameters are adjusted according to the feedback of the comprehensive hidden danger assessment report.

2. The method according to claim 1, characterized in that: The cleaning of the hyperspectral image data, infrared thermal image data and vibration sensor data includes: noise removal and outlier processing, wherein the noise removal is achieved through a filtering algorithm, and the outlier processing is achieved by detecting and eliminating abnormal data points through statistical analysis.

3. The method according to claim 2, characterized in that The feature extraction includes: extracting edge and contour features from the cleaned hyperspectral image data, extracting temperature features from the cleaned infrared thermal image data, and extracting vibration mode features from the cleaned vibration sensing data, and generating hyperspectral feature data, infrared thermal feature data, and vibration feature data respectively.

4. The method according to claim 3, characterized in that The normalization processing includes: normalizing the hyperspectral feature data, infrared thermal feature data and vibration feature data to ensure that the data are within the same magnitude range, and the normalization processing is achieved through a linear normalization method.

5. The method according to claim 1, characterized in that: The real-time monitoring and comparison includes: performing a preliminary comparison on the fused feature data through an edge computing device to generate preliminary comparison result data, wherein the preliminary comparison is achieved by calculating the Euclidean distance between the real-time feature data and the dynamic benchmark model, and the Euclidean distance calculation formula is: Where D represents the Euclidean distance; n represents the number of feature dimensions; x i represents the i-th eigenvalue in the real-time feature data; y i represents the i-th eigenvalue in the dynamic baseline model.

6. The method according to claim 5, characterized in that The cluster analysis includes: using K-means clustering technology to perform cluster analysis on the preliminary comparison result data to generate cluster result data, wherein K-means clustering is achieved by iteratively optimizing cluster centers, and the objective function of the iterative optimization is: Among them, J represents the objective function value; k represents the number of clusters; n represents the number of data points; represents the i-th data point belonging to the j-th class; μ j Represents the cluster center of the jth class.

7. The method according to claim 6, characterized in that The deep reinforcement learning optimization includes: optimizing the clustering result data using a deep reinforcement learning algorithm to generate optimized comparison result data, wherein the deep reinforcement learning algorithm performs strategy optimization through a deep Q network, and the loss function of the deep Q network is: Where L(θ) represents the loss function; θ represents the current network parameters; s represents the current state; a represents the current action; r represents the reward; s' represents the next state; γ represents the discount factor; Q represents the Q value function; θ - represents the target network parameters; a' represents the next action.

8. The method according to claim 1, characterized in that The anomaly detection and hidden danger assessment includes: performing anomaly pattern recognition on the optimized comparison result data through a sparse representation algorithm to generate abnormal pattern data, wherein the sparse representation algorithm realizes anomaly detection by minimizing the reconstruction error, and the calculation formula of the reconstruction error is: Where E represents the reconstruction error; X represents the optimized comparison result data matrix; D represents the dictionary matrix; α represents the sparse coefficient vector; and λ represents the regularization parameter.

9. The method according to claim 8, characterized in that The hidden danger assessment includes: using multi-level spatiotemporal analysis technology to perform time and space feature analysis on the abnormal pattern data to generate spatiotemporal feature data, and using causal analysis technology to perform causal relationship analysis on the spatiotemporal feature data to generate causal relationship data, and finally generating a comprehensive hidden danger assessment report, wherein the causal analysis is implemented by a Granger causality model, and the calculation formula of the Granger causality model is: Among them, Y t represents the explained variable; X t represents the explanatory variable; α i represents the lag coefficient of the explained variable; β j represents the lag coefficient of the explanatory variable; Y t-i represents the ith lag value of the explained variable; X t-j represents the jth lag value of the explanatory variable; ∈ t represents the error term; p represents the lag order of the explained variable; q represents the lag order of the explanatory variable.

10. A training system for the method for identifying and troubleshooting production safety hazards / risks based on visual recognition according to any one of claims 1 to 9, characterized in that: include: A data acquisition module is used to collect hyperspectral image data, infrared thermal image data and vibration sensor data on the production line in real time through a hyperspectral imaging sensor, an infrared thermal imaging sensor and a vibration sensor; A data preprocessing and fusion module is used to clean, extract features and normalize the hyperspectral image data, infrared thermal image data and vibration sensor data to generate fused feature data; A real-time monitoring and comparison module, which is used to use edge computing equipment and deep reinforcement learning algorithms to perform real-time monitoring and comparison on the fused feature data, including preliminary comparison, cluster analysis and deep reinforcement learning optimization, and generate optimized comparison result data; Anomaly detection and hidden danger assessment module, used to perform anomaly detection and hidden danger assessment on the optimized comparison result data through sparse representation, multi-level spatiotemporal analysis and causal analysis, generate a comprehensive hidden danger assessment report, and adjust the benchmark model and detection parameters according to the feedback of the comprehensive hidden danger assessment report; A model training module, used to train and optimize the benchmark model using the fused feature data and the optimized comparison result data to generate a dynamic benchmark model; A model verification module, used to verify and evaluate the dynamic benchmark model using a verification data set to ensure the accuracy and robustness of the model; The deployment and update module is used to deploy the verified dynamic benchmark model to the production environment and dynamically update the model parameters based on real-time data feedback.

Citation Information

Patent Citations

  • Multidirectional distance dynamic monitoring and early warning method based on edge calculation

    CN112291303A

  • Plastic master batch quality inspection method and system based on machine vision

    CN117095005A

  • Building engineering quality real-time monitoring system

    CN117611015A

  • Substation transformer fault infrared image and voiceprint recognition method and system

    CN118051873A

  • Multi-mode artificial intelligence production safety early warning method

    CN118229089A

Cited By

  • Internal defect inspection method and internal defect inspection system

    JP2026141974A

  • Internal defect inspection method and internal defect inspection system

    JP7729523B1