Sensor data analysis method, system and device based on industrial internet of things

Abstract

This application discloses a sensor data analysis method, system, and device based on the Industrial Internet of Things (IIoT), relating to the technical field of IIoT. The method includes: acquiring infrared image data and acoustic emission data corresponding to each workpiece in the same batch under a continuously changing temperature field using sensors, and acquiring static data of the workpiece at room temperature; semantically encoding the infrared image data, acoustic emission data, and static data respectively to obtain infrared image features, acoustic emission features, and static features; semantically associating the infrared image features and acoustic emission features to obtain semantically associated features; semantically enhancing the static features based on the semantically associated features to obtain semantically enhanced features; and outputting workpiece defect analysis results based on the semantically enhanced features. This application has the effect of improving the accuracy of workpiece defect detection.

Description

Technical Field

[0001] This application relates to the technical field of industrial Internet of Things (IIoT), and in particular to sensor data analysis methods, systems and devices based on industrial Internet of Things. Background Technology

[0002] In modern industrial manufacturing, workpiece quality is the cornerstone of ensuring product safety and reliability. Non-destructive testing technologies, such as ultrasonic testing, X-ray testing, infrared thermography, and acoustic emission testing, have become key means of quality control. Each of these technologies has its own advantages: ultrasonic and X-ray testing are good at detecting internal defects, infrared thermography can reflect abnormal surface temperature distribution, and acoustic emission testing can capture stress wave activity inside materials in real time.

[0003] However, existing technologies generally suffer from the following limitations: First, various detection methods typically operate independently, with data analysis processes isolated from each other, forming "information silos." This single-modal analysis method struggles to comprehensively characterize the complete attributes of defects. For example, a microcrack generated during thermal stress and hidden after cooling may be missed by static detection methods. Second, traditional detection schemes cannot effectively correlate the workpiece's response data during dynamic processes with its intrinsic quality data under static conditions, making it impossible to trace the formation mechanism of defects. Furthermore, although the Industrial Internet of Things (IIoT) can acquire multi-source data, the lack of models and methods capable of deeply integrating and intelligently analyzing this heterogeneous data limits further improvements in defect identification accuracy.

[0004] Therefore, there is an urgent need in this field for a technical solution that can integrate industrial IoT resources and perform collaborative analysis and semantic association of multimodal sensing data in order to achieve earlier, more comprehensive and more accurate identification of workpiece defects. Summary of the Invention

[0005] To improve the accuracy of workpiece defect detection, this application provides a sensor data analysis method, system, and device based on the Industrial Internet of Things.

[0006] Firstly, this application provides a sensor data analysis method based on the Industrial Internet of Things, employing the following technical solution:

[0007] A sensor data analysis method based on the Industrial Internet of Things (IIoT) is applied to an IIoT system, which includes a management platform, a sensor network platform, and an object platform connected in sequence. The method is executed by the management platform and includes:

[0008] The infrared image data and acoustic emission data of each workpiece in the same batch are acquired by the sensor in a continuously changing temperature field, and the static data of the workpiece at room temperature are also acquired. The static data includes at least one of ultrasonic detection data and X-ray image data.

[0009] Based on the workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are semantically encoded to obtain infrared image features, acoustic emission features and static features;

[0010] Semantic association is performed on the infrared image features and the acoustic emission features to obtain semantic association features; and semantic enhancement is performed on the static features based on the semantic association features to obtain semantic enhancement features.

[0011] Based on the semantic enhancement features, the workpiece defect analysis results are output.

[0012] By adopting the above technical solution, infrared image data and acoustic emission data corresponding to each workpiece in the same batch under a continuously changing temperature field are acquired through sensors, and static data of the workpiece under normal temperature conditions are also acquired. The static data includes at least one of ultrasonic detection data and X-ray image data. Then, based on the workpiece defect analysis network, the infrared image data, acoustic emission data and static data are semantically encoded to obtain infrared image features, acoustic emission features and static features. Then, the infrared image features and acoustic emission features are semantically correlated to obtain semantically correlated features. Based on the semantically correlated features, the static features are semantically enhanced to obtain semantically enhanced features. Finally, based on the semantically enhanced features, the workpiece defect analysis results are output. In this invention, at the data level, the limitation of a single data source is overcome by integrating dynamic process data and static detection data. Secondly, at the information processing level, deep, semantic-level information complementarity and synergistic enhancement between multimodal data are achieved through semantic encoding, association, and reinforcement, rather than simple data stacking. This enables the system to comprehensively consider the static performance and dynamic evolution of defects, thereby achieving a more comprehensive and accurate identification and evaluation of workpiece defects. This not only reduces the risk of missed detections and misjudgments but also enhances the early warning capability for early defects and potential failure risks, thus improving the overall intelligence level and reliability of industrial quality control.

[0013] Optionally, the step of semantically encoding the infrared image data, the acoustic emission data, and the static data to obtain infrared image features, acoustic emission features, and static features respectively includes:

[0014] The infrared image data, the acoustic emission data, and the static data are loaded into the workpiece defect analysis network, wherein the workpiece defect analysis network includes a semantic coding sub-network, and the semantic coding sub-network includes a first semantic coding unit, a second semantic coding unit, and a third semantic coding unit, each with a different network structure.

[0015] The infrared image data is semantically encoded by the first semantic encoding unit to obtain infrared image features;

[0016] The acoustic emission data is semantically encoded using the second semantic encoding unit to obtain acoustic emission features;

[0017] The static data is subjected to third semantic encoding by the third semantic encoding unit to obtain static features.

[0018] By adopting the above technical solution, in order to extract infrared image features, acoustic emission features, and static features, infrared image data, acoustic emission data, and static data are loaded into a workpiece defect analysis network. The workpiece defect analysis network includes a semantic coding sub-network, which includes a first semantic coding unit, a second semantic coding unit, and a third semantic coding unit with different network structures. Then, the infrared image data is semantically encoded by the first semantic coding unit to obtain infrared image features. Then, the acoustic emission data is semantically encoded by the second semantic coding unit to obtain acoustic emission features. Finally, the static data is semantically encoded by the third semantic coding unit to obtain static features.

[0019] Optionally, the step of performing a first semantic encoding on the infrared image data to obtain infrared image features includes:

[0020] The infrared image data is subjected to multi-level convolution processing to obtain infrared multi-scale feature maps. The multi-level convolution processing includes parallel convolution processing of the same infrared image using convolution kernels of different sizes.

[0021] The infrared multi-scale feature map is subjected to channel attention weighting to obtain infrared channel weighted features, wherein the channel attention weighting is used to enhance the weight of feature channels related to thermal damage;

[0022] Spatial self-attention mining is performed on the weighted features of the infrared channels to obtain infrared self-attention features, wherein the spatial self-attention mining is used to establish semantic associations between different temperature regions;

[0023] Update the first reference feature vector, wherein the first dynamic feature library is initialized or updated. The first dynamic feature library is used to store the infrared self-attention features corresponding to the workpieces that have been identified as qualified in other production batches. The capacity of the dynamic feature library is fixed. The first reference feature vector is obtained by averaging the first dynamic feature library.

[0024] Calculate the first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector, and based on the first difference vector, perform attention weighting on the infrared self-attention feature to generate infrared image features.

[0025] By adopting the above technical solution, in order to obtain infrared image features, multi-level convolution processing is performed on the infrared image data to obtain infrared multi-scale feature maps. The multi-level convolution processing includes parallel convolution processing of the same infrared image using convolution kernels of different sizes. Then, channel attention weighting is applied to the infrared multi-scale feature maps to obtain infrared channel weighted features. Channel attention weighting is used to enhance the weights of feature channels related to thermal damage. Then, spatial self-attention mining is performed on the infrared channel weighted features to obtain infrared self-attention features. Spatial self-attention mining is used to establish semantic associations between different temperature regions. Then, the first reference feature vector is updated. The first dynamic feature library is initialized or updated. The first dynamic feature library is used to store the infrared self-attention features corresponding to workpieces that have been identified as qualified in other production batches. The capacity of the dynamic feature library is fixed. The first reference feature vector is obtained by averaging the first dynamic feature library. Then, the first difference vector between the infrared self-attention features of the current workpiece and the first reference feature vector is calculated. Based on the first difference vector, attention weighting is applied to the infrared self-attention features to generate infrared image features.

[0026] Optionally, the step of performing a second semantic encoding on the acoustic emission data to obtain acoustic emission features includes:

[0027] The acoustic emission data is subjected to time-frequency transformation processing to obtain an acoustic emission time-frequency spectrum. The time-frequency transformation processing is used to convert the acoustic emission data from a time domain representation to a time-frequency domain representation in order to capture the temporal and frequency characteristics of the acoustic emission event.

[0028] The acoustic emission time-spectrum is subjected to two-dimensional convolution processing to obtain acoustic emission convolution features, wherein the two-dimensional convolution processing is used to extract waveform patterns and frequency distribution features in the acoustic emission event;

[0029] Temporal attention weighting is applied to the acoustic emission convolutional features to obtain temporally weighted acoustic emission features, wherein the temporal attention weighting is used to enhance the weights of acoustic emission events related to thermal damage;

[0030] Frequency attention weighting is applied to the acoustic emission time-series weighted features to obtain acoustic emission radio frequency domain weighted features, wherein the frequency attention weighting is used to enhance the weight of characteristic frequency components related to material damage;

[0031] Update the second reference feature vector, wherein the second dynamic feature library is initialized or updated. The second dynamic feature library is used to store the acoustic emission radio frequency domain weighted features corresponding to the workpieces that have been identified as qualified in other production batches. The capacity of the second dynamic feature library is fixed. The second reference feature vector is obtained by averaging the second dynamic feature library.

[0032] Calculate the second difference vector between the acoustic emission radio frequency domain weighted features of the current workpiece and the second reference feature vector, and based on the second difference vector, perform attention weighting on the acoustic emission radio frequency domain weighted features to generate acoustic emission features.

[0033] By employing the above technical solution, in order to obtain acoustic emission characteristics, the acoustic emission data undergoes time-frequency transformation processing to obtain an acoustic emission time-frequency spectrum. This time-frequency transformation converts the acoustic emission data from a time-domain representation to a time-frequency domain representation to capture the temporal and frequency characteristics of the acoustic emission events. Then, a two-dimensional convolution process is performed on the acoustic emission time-frequency spectrum to obtain acoustic emission convolutional features. This two-dimensional convolution is used to extract waveform patterns and frequency distribution features from the acoustic emission events. Finally, temporal attention weighting is applied to the acoustic emission convolutional features to obtain temporally weighted acoustic emission features. This temporal attention weighting is used to enhance the weights of acoustic emission events related to thermal damage. Finally, frequency... Attention weighting is applied to obtain acoustic emission and radio frequency domain weighted features. Frequency attention weighting is used to enhance the weight of characteristic frequency components related to material damage. Then, the second reference feature vector is updated. The second dynamic feature library is initialized or updated. The second dynamic feature library is used to store the acoustic emission and radio frequency domain weighted features corresponding to workpieces that have been identified as qualified in other production batches. The capacity of the second dynamic feature library is fixed. The second reference feature vector is obtained by averaging the second dynamic feature library. Then, the second difference vector between the acoustic emission and radio frequency domain weighted features of the current workpiece and the second reference feature vector is calculated. Based on the second difference vector, attention weighting is applied to the acoustic emission and radio frequency domain weighted features to generate acoustic emission features.

[0034] Optionally, when the static data is the ultrasound detection data and the X-ray image data, the step of performing third semantic encoding on the static data through the third semantic encoding unit to obtain static features includes:

[0035] The static data is subjected to multimodal feature alignment processing to obtain static aligned features, wherein the multimodal feature alignment processing is used to map ultrasound detection data and X-ray image data to a unified feature space;

[0036] The static alignment features are convolved to obtain static convolution features, wherein the convolution process is used to enhance structural features related to internal defects;

[0037] The static convolutional features are subjected to cross-modal attention weighting to obtain static weighted features. The cross-modal attention weighting calculates the mutual enhancement weights between different modal features through a cross-attention mechanism to achieve complementary correlation between ultrasound features and X-ray features.

[0038] The static weighted features are instantiated to obtain a set of defect entities, wherein the defect instantiation process is used to identify defect individuals and extract the geometric parameters of the defect individuals;

[0039] Based on the set of defective entities, a defect topology graph is constructed using a graph neural network;

[0040] Graph feature extraction is performed on the defect topology graph to obtain static features.

[0041] By adopting the above technical solution, in order to obtain static features, multimodal feature alignment processing is performed on the static data to obtain static aligned features. This multimodal feature alignment processing maps ultrasound detection data and X-ray image data to a unified feature space. Then, convolution processing is performed on the static aligned features to obtain static convolutional features. This convolution processing enhances structural features related to internal defects. Next, cross-modal attention weighting is applied to the static convolutional features to obtain static weighted features. Cross-modal attention weighting calculates the mutual enhancement weights between different modal features through a cross-attention mechanism to achieve complementary correlation between ultrasound and X-ray features. Then, defect instantiation is performed on the static weighted features to obtain a set of defect entities. Defect instantiation processing identifies individual defects and extracts their geometric parameters. Based on the set of defect entities, a defect topology graph is constructed using a graph neural network. Finally, graph feature extraction is performed on the defect topology graph to obtain the static features.

[0042] Optionally, the step of semantically associating the infrared image features and the acoustic emission features to obtain semantically associated features includes:

[0043] The infrared image features and the acoustic emission features are loaded into the semantic association unit of the workpiece defect analysis network, wherein the semantic association unit has a built-in learnable first semantic space transformation matrix and second semantic space transformation matrix;

[0044] The first semantic space transformation matrix is ​​used to perform semantic space transformation on the infrared image features to obtain the first transformed features;

[0045] The acoustic emission features are semantically transformed using the second semantic space transformation matrix to obtain the second transformed features, wherein the first transformed features and the second transformed features are in the same semantic space;

[0046] Based on the first conversion feature, the second conversion feature is attention-weighted to obtain a first weighted feature, wherein the first weighted feature is used to enhance the corresponding thermal anomaly region in the infrared feature by utilizing the damage time sequence information in the acoustic emission feature.

[0047] Based on the second conversion feature, the first conversion feature is attention-weighted to obtain the second weighted feature, wherein the second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by utilizing the temperature distribution information in the infrared feature;

[0048] The first weighted feature and the second weighted feature are fused to obtain the semantic association feature.

[0049] By adopting the above technical solution, in order to obtain semantic association features, infrared image features and acoustic emission features are loaded into the semantic association unit of the workpiece defect analysis network. The semantic association unit has built-in learnable first semantic space transformation matrix and second semantic space transformation matrix. Then, the infrared image features are semantically transformed using the first semantic space transformation matrix to obtain the first transformed feature. Then, the acoustic emission features are semantically transformed using the second semantic space transformation matrix to obtain the second transformed feature. The first and second transformed features are in the same semantic space. Then, attention weighting is applied to the second transformed feature based on the first transformed feature to obtain the first weighted feature. The first weighted feature is used to enhance the corresponding thermal anomaly region in the infrared feature by utilizing the damage time sequence information in the acoustic emission feature. Then, attention weighting is applied to the first transformed feature based on the second transformed feature to obtain the second weighted feature. The second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by utilizing the temperature distribution information in the infrared feature. Finally, the first weighted feature and the second weighted feature are fused to obtain the semantic association feature.

[0050] Optionally, the step of semantically strengthening the static features based on the semantic association features to obtain semantically strengthened features includes:

[0051] The semantic association features and the static features are loaded into the semantic enhancement unit of the workpiece defect analysis network, wherein the semantic enhancement unit includes at least a first semantic enhancement subunit and a second semantic enhancement subunit;

[0052] The first semantic enhancement subunit performs first semantic enhancement on the static features based on the semantic association features to obtain the first semantic enhancement features;

[0053] The second semantic enhancement subunit performs second semantic enhancement on the static features based on the semantic association features to obtain the second semantic enhancement features;

[0054] The semantic enhancement features are obtained by averaging the first semantic enhancement features and the second semantic enhancement features.

[0055] By adopting the above technical solution, in order to obtain semantic enhancement features, semantic association features and static features are loaded into the semantic enhancement unit of the workpiece defect analysis network. The semantic enhancement unit includes at least a first semantic enhancement subunit and a second semantic enhancement subunit. Then, through the first semantic enhancement subunit, the static features are semantically enhanced based on the semantic association features to obtain the first semantic enhancement feature. Then, through the second semantic enhancement subunit, the static features are semantically enhanced based on the semantic association features to obtain the second semantic enhancement feature. Finally, the average of the first semantic enhancement feature and the second semantic enhancement feature is calculated to obtain the semantic enhancement feature.

[0056] Optionally, the step of performing first semantic enhancement on the static features based on the semantic association features through the first semantic enhancement subunit to obtain the first semantically enhanced features includes:

[0057] The semantic association features are transformed into third transformed features by the third semantic space transformation matrix built into the first semantic enhancement subunit. The third semantic space transformation matrix is ​​used to transform the semantic association features into the semantic space where the static features are located.

[0058] Determine the parameter mapping relationship between the third transformation feature and the static feature, and based on the parameter mapping relationship, determine the influence weight distribution of the semantic association feature on the static feature, wherein the influence weight distribution is used to quantify the degree of influence of the semantic association feature on the static feature;

[0059] Based on the influence weight distribution, the static features are weighted to obtain the first semantic enhancement feature.

[0060] By adopting the above technical solution, in order to obtain the first semantic enhancement feature, the semantic association feature is transformed into the third transformed feature by the third semantic space transformation matrix built into the first semantic enhancement subunit. The third semantic space transformation matrix is ​​used to transform the semantic association feature into the semantic space where the static feature is located. Then, the parameter mapping relationship between the third transformed feature and the static feature is determined. Based on the parameter mapping relationship, the influence weight distribution of the semantic association feature on the static feature is determined. The influence weight distribution is used to quantify the degree of influence of the semantic association feature on the static feature. Then, based on the influence weight distribution, the static feature is weighted to obtain the first semantic enhancement feature.

[0061] Secondly, this application also provides a sensor data analysis system based on the Industrial Internet of Things, which adopts the following technical solution:

[0062] A sensor data analysis system based on the Industrial Internet of Things (IIoT) includes a management platform, a sensor network platform, and an object platform that are sequentially connected in communication. The management platform is configured with:

[0063] The data acquisition module is used to acquire infrared image data and acoustic emission data of each workpiece in the same batch in a continuously changing temperature field through sensors, and to acquire static data of the workpiece under normal temperature conditions, wherein the static data includes at least one of ultrasonic detection data and X-ray image data.

[0064] A single semantic encoding module is used to perform semantic encoding on the infrared image data, the acoustic emission data and the static data respectively based on the workpiece defect analysis network to obtain infrared image features, acoustic emission features and static features;

[0065] A multi-semantic fusion module is used to semantically correlate the infrared image features and the acoustic emission features to obtain semantically correlated features; and to semantically enhance the static features based on the semantically correlated features to obtain semantically enhanced features.

[0066] The analysis module is used to output the workpiece defect analysis results based on the semantic enhancement features.

[0067] Thirdly, this application also provides a computer device, which adopts the following technical solution:

[0068] A computer device includes a memory and a processor, the memory storing a computer program executable on the processor, the processor executing the computer program to implement the method described in the first aspect.

[0069] In summary, this application includes at least the following beneficial technical effects: Infrared image data and acoustic emission data of each workpiece in the same batch under a continuously changing temperature field are acquired using sensors, and static data of the workpiece under normal temperature conditions are also acquired. The static data includes at least one of ultrasonic testing data and X-ray image data. Then, based on a workpiece defect analysis network, the infrared image data, acoustic emission data, and static data are semantically encoded to obtain infrared image features, acoustic emission features, and static features. Then, the infrared image features and acoustic emission features are semantically correlated to obtain semantically correlated features. Based on the semantically correlated features, the static features are semantically enhanced to obtain semantically enhanced features. Finally, based on the semantically enhanced features, the workpiece defect analysis results are output. In this invention, at the data level, the limitation of a single data source is overcome by integrating dynamic process data and static detection data. Secondly, at the information processing level, deep, semantic-level information complementarity and synergistic enhancement between multimodal data are achieved through semantic encoding, association, and reinforcement, rather than simple data stacking. This enables the system to comprehensively consider the static performance and dynamic evolution of defects, thereby achieving a more comprehensive and accurate identification and evaluation of workpiece defects. This not only reduces the risk of missed detections and misjudgments but also enhances the early warning capability for early defects and potential failure risks, thus improving the overall intelligence level and reliability of industrial quality control. Attached Figure Description

[0070] Figure 1 This is a schematic diagram of the overall process of an embodiment of this application.

[0071] Figure 2 This is a structural diagram of one application scenario of the system in this application embodiment.

[0072] Figure 3 This is a structural diagram of another application scenario of the system according to an embodiment of this application.

[0073] Figure 4 This is a structural block diagram of the computer device described in this application. Detailed Implementation

[0074] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0075] This application discloses a sensor data analysis method based on the Industrial Internet of Things (IIoT).

[0076] Reference Figure 1A sensor data analysis method based on the Industrial Internet of Things (IIoT) is applied to an IIoT system, which includes a management platform, a sensor network platform, and an object platform connected in sequence. The method is executed by the management platform and includes:

[0077] Step S11: The infrared image data and acoustic emission data of each workpiece in the same batch in a continuously changing temperature field are acquired by the sensor, and the static data of the workpiece at room temperature are acquired. The static data includes at least one of ultrasonic detection data and X-ray image data.

[0078] It should be noted that in step S11, the system collects two types of key data through various sensors deployed on the object platform: first, process response data of the workpiece under the action of a dynamic and continuously changing temperature field, namely infrared image data and acoustic emission data, which are used to capture the surface thermodynamic behavior and internal micro-activity during the heating process; second, static data that can reflect the inherent structure of the workpiece under stable conditions at room temperature, such as ultrasonic detection data or X-ray image data, thus forming a multimodal and multidimensional dataset covering "dynamic process monitoring" and "static quality assessment".

[0079] Step S12: Based on the workpiece defect analysis network, semantic encoding is performed on the infrared image data, acoustic emission data and static data respectively to obtain infrared image features, acoustic emission features and static features.

[0080] It should be noted that in step S12, the raw data collected is heterogeneous and contains a lot of redundant information. By introducing the semantic coding module in the workpiece defect analysis network, the system can perform high-level abstraction and understanding of infrared images, acoustic emission signals and static data respectively. This process converts the original pixels, waveforms or images into a series of dense feature vectors that can characterize potential defect patterns, material properties and their physical meanings, namely infrared image features, acoustic emission features and static features.

[0081] Step S13: Semantically correlate the infrared image features and acoustic emission features to obtain semantically correlated features; and based on the semantically correlated features, semantically enhance the static features to obtain semantically enhanced features.

[0082] It should be noted that in step S13, the two types of features obtained from dynamic monitoring, namely infrared image features and acoustic emission features, are subjected to cross-modal semantic association. This operation aims to reveal the intrinsic connection between thermal behavior and acoustic activity, thereby obtaining a semantic association feature that can more comprehensively describe the overall state of the workpiece under dynamic stress. Subsequently, this association feature is used as context or guiding information to semantically enhance the static features extracted from static data. The purpose is to inject the clues revealed by the dynamic process into the understanding of the static structure, so that the enhanced features not only contain internal structural information, but also integrate the behavioral traces of defects in the dynamic process.

[0083] Step S14: Based on semantic enhancement features, output the workpiece defect analysis results.

[0084] It should be noted that in step S14, the semantic enhancement features obtained after the aforementioned steps of fusion and enhancement are input into the final decision layer (e.g., a classifier or regressor) of the workpiece defect analysis network. Based on the learned knowledge, the decision layer parses and judges the input features and finally outputs a structured workpiece defect analysis result. This result may specifically include the type of defect (e.g., cracks, porosity), location, severity level, or a comprehensive health status assessment index, thereby directly serving quality judgment and decision-making.

[0085] In the above embodiment, infrared image data and acoustic emission data corresponding to each workpiece in the same batch in a continuously changing temperature field are acquired by sensors, and static data of the workpiece under normal temperature conditions are also acquired. The static data includes at least one of ultrasonic detection data and X-ray image data. Then, based on the workpiece defect analysis network, the infrared image data, acoustic emission data and static data are semantically encoded to obtain infrared image features, acoustic emission features and static features. Then, the infrared image features and acoustic emission features are semantically correlated to obtain semantically correlated features. Based on the semantically correlated features, the static features are semantically enhanced to obtain semantically enhanced features. Finally, based on the semantically enhanced features, the workpiece defect analysis results are output. In this invention, at the data level, the limitation of a single data source is overcome by integrating dynamic process data and static detection data. Secondly, at the information processing level, deep, semantic-level information complementarity and synergistic enhancement between multimodal data are achieved through semantic encoding, association, and reinforcement, rather than simple data stacking. This enables the system to comprehensively consider the static performance and dynamic evolution of defects, thereby achieving a more comprehensive and accurate identification and evaluation of workpiece defects. This not only reduces the risk of missed detections and misjudgments but also enhances the early warning capability for early defects and potential failure risks, thus improving the overall intelligence level and reliability of industrial quality control.

[0086] As a further implementation of the method, the step of semantically encoding infrared image data, acoustic emission data, and static data to obtain infrared image features, acoustic emission features, and static features includes:

[0087] Step S21: The infrared image data, acoustic emission data and static data are loaded into the workpiece defect analysis network. The workpiece defect analysis network includes a semantic coding sub-network, which includes a first semantic coding unit, a second semantic coding unit and a third semantic coding unit with different network structures.

[0088] Step S22: The infrared image data is first semantically encoded by the first semantic encoding unit to obtain infrared image features.

[0089] Step S23: The acoustic emission data is semantically encoded using the second semantic encoding unit to obtain acoustic emission features.

[0090] Step S24: The static data is semantically encoded using the third semantic encoding unit to obtain static features.

[0091] It should be noted that from steps S21 to S24, in response to the essential differences in data structure, physical meaning, and inherent defect information patterns among infrared image data, acoustic emission data, and static data, this scheme adopts a dedicated coding unit strategy with non-shared weights. By independently designing and deploying first, second, and third semantic coding units with different network structures for each data type, it can be ensured that each encoder uses its most suitable architecture to deeply mine the most discriminative semantic information in its corresponding data. This clearly defined design aims to fully leverage the unique advantages of each modality of data, providing a high-quality and high-purity feature foundation for subsequent cross-modal information fusion, thereby ensuring the accuracy and effectiveness of the entire analysis process from the source.

[0092] In the above embodiments, in order to extract infrared image features, acoustic emission features, and static features, infrared image data, acoustic emission data, and static data are loaded into a workpiece defect analysis network. The workpiece defect analysis network includes a semantic coding sub-network, which includes a first semantic coding unit, a second semantic coding unit, and a third semantic coding unit with different network structures. Then, the infrared image data is firstly semantically encoded by the first semantic coding unit to obtain infrared image features. Then, the acoustic emission data is secondly semantically encoded by the second semantic coding unit to obtain acoustic emission features. Finally, the static data is thirdly semantically encoded by the third semantic coding unit to obtain static features.

[0093] As a further implementation of the method, the step of performing a first semantic encoding on the infrared image data to obtain infrared image features includes:

[0094] Step S31: Perform multi-level convolution processing on the infrared image data to obtain infrared multi-scale feature maps. The multi-level convolution processing includes parallel convolution processing of the same infrared image using convolution kernels of different sizes.

[0095] Step S32: Channel attention weighting is applied to the infrared multi-scale feature map to obtain infrared channel weighted features, wherein channel attention weighting is used to enhance the weight of feature channels related to thermal damage.

[0096] Step S33: Perform spatial self-attention mining on the weighted features of the infrared channel to obtain infrared self-attention features. The spatial self-attention mining is used to establish semantic associations between different temperature regions.

[0097] Step S34, update the first reference feature vector, wherein the first dynamic feature library is initialized or updated. The first dynamic feature library is used to store the infrared self-attention features corresponding to the workpieces that have been identified as qualified in other production batches. The capacity of the dynamic feature library is fixed. The first reference feature vector is obtained by averaging the first dynamic feature library.

[0098] It should be noted that the update of the first dynamic feature library is a dynamic rolling process following the "first-in, first-out" principle. Specifically, when a new qualified workpiece is confirmed, the system adds its extracted infrared self-attention features as a new data entry to the end of the dynamic feature library. Simultaneously, the system checks whether the current capacity of the feature library has reached a preset fixed upper limit. If it exceeds this limit, the system automatically removes or overwrites the oldest historical feature entry in the library. Through this mechanism, the feature library always maintains the latest and most representative set of qualified samples. Subsequently, the first reference feature vector is regenerated by immediately averaging all features in the updated feature library. This process ensures that the constructed qualification benchmark is not a static historical snapshot, but a continuously evolving dynamic standard that can absorb the latest production data, adapt to the slow drift of production conditions, and significantly improve the adaptability and accuracy of the defect detection system in long-term operation.

[0099] Step S35: Calculate the first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector, and based on the first difference vector, perform attention weighting on the infrared self-attention feature to generate infrared image features.

[0100] It should be noted that from step S31 to step S35, deep semantic information related to thermal damage is mined from infrared image data. Specifically, the system employs multi-level convolutional processing to capture diverse temperature field features extracted by convolutional kernels of different sizes in parallel, forming an infrared multi-scale feature map to meet the detection needs of defects at different scales. Next, a channel attention weighting mechanism is used to adaptively enhance the weights of feature channels strongly correlated with thermal damage phenomena, effectively focusing on key information. Based on this, spatial self-attention mining is used to establish long-range semantic dependencies between different temperature regions in the image, thereby understanding the association between the overall thermal distribution pattern and local abnormal regions, generating infrared self-attention features. To introduce an adaptive benchmark, the scheme dynamically maintains a first dynamic feature library composed of features of qualified workpieces, and obtains a first reference feature vector representing the normal state through averaging. Finally, by calculating the difference vector between the current workpiece features and this reference vector, and using this difference to perform secondary weighting on the self-attention features, the model can significantly enhance the response of abnormal features that deviate from the normal pattern, thereby generating infrared image features that are extremely sensitive to defects. This achieves a step-by-step refinement and enhancement from the original image to high-level semantic features with strong discriminative power.

[0101] In the above embodiments, in order to obtain infrared image features, the infrared image data is subjected to multi-level convolution processing to obtain an infrared multi-scale feature map. The multi-level convolution processing includes parallel convolution processing of the same infrared image using convolution kernels of different sizes. Then, the infrared multi-scale feature map is subjected to channel attention weighting to obtain infrared channel weighted features. Channel attention weighting is used to enhance the weight of feature channels related to thermal damage. Then, spatial self-attention mining is performed on the infrared channel weighted features to obtain infrared self-attention features. Spatial self-attention mining is used to establish semantic associations between different temperature regions. Then, the first reference feature vector is updated. The first dynamic feature library is initialized or updated. The first dynamic feature library is used to store the infrared self-attention features corresponding to workpieces that have been identified as qualified in other production batches. The capacity of the dynamic feature library is fixed. The first reference feature vector is obtained by averaging the first dynamic feature library. Then, the first difference vector between the infrared self-attention features of the current workpiece and the first reference feature vector is calculated. Based on the first difference vector, attention weighting is performed on the infrared self-attention features to generate infrared image features.

[0102] As a further implementation of the method, the step of performing a second semantic encoding on the acoustic emission data to obtain acoustic emission features includes:

[0103] Step S41: Perform time-frequency transformation processing on the acoustic emission data to obtain the acoustic emission time-frequency spectrum. The time-frequency transformation processing is used to convert the acoustic emission data from time domain representation to time-frequency domain representation in order to capture the timing and frequency characteristics of the acoustic emission event.

[0104] Step S42: Perform two-dimensional convolution processing on the acoustic emission spectrum to obtain acoustic emission convolution features. The two-dimensional convolution processing is used to extract waveform patterns and frequency distribution features in the acoustic emission event.

[0105] Step S43: Temporal attention weighting is applied to the acoustic emission convolution features to obtain temporally weighted acoustic emission features, wherein temporal attention weighting is used to enhance the weights of acoustic emission events related to thermal damage.

[0106] Step S44: Apply frequency attention weighting to the acoustic emission time-series weighted features to obtain acoustic emission radio frequency domain weighted features, wherein frequency attention weighting is used to enhance the weight of characteristic frequency components related to material damage.

[0107] Step S45, update the second reference feature vector, wherein the second dynamic feature library is initialized or updated. The second dynamic feature library is used to store the acoustic emission radio frequency domain weighted features corresponding to the workpieces that have been identified as qualified in other production batches. The capacity of the second dynamic feature library is fixed. The second reference feature vector is obtained by averaging the second dynamic feature library.

[0108] It should be noted that the working mechanism of the second dynamic feature library is the same as that of the first dynamic feature library, and step S34 can be referred to.

[0109] Step S46: Calculate the second difference vector between the current workpiece's acoustic emission radio frequency domain weighted features and the second reference feature vector, and based on the second difference vector, perform attention weighting on the acoustic emission radio frequency domain weighted features to generate acoustic emission features.

[0110] It should be noted that from steps S41 to S46, the one-dimensional time-series signal is converted into a two-dimensional time-spectrum image containing rich time-frequency information through time-frequency transformation, thereby preserving the temporal evolution law and frequency component characteristics of the signal simultaneously. Subsequently, the feature extraction stage is entered, and two-dimensional convolution processing is used to automatically learn and extract typical waveform patterns and frequency distribution features related to material damage from the time-spectrum image. On this basis, the process introduces a dual attention enhancement mechanism (steps S43 and S44): firstly, through temporal attention weighting, the acoustic emission events associated with key time points in the thermal damage process are focused on to suppress irrelevant background noise; then, through frequency attention weighting, the contribution of characteristic frequency components that are closely related to damage mechanisms such as crack initiation and propagation in a material science sense is further amplified. Finally, in order to achieve adaptive anomaly detection, the process establishes a dynamic benchmark adjustment and feature generation stage (steps S45 and S46). By maintaining a regularly updated second dynamic feature library composed of qualified workpiece features and calculating its average value to obtain a second reference feature vector, the system establishes a dynamic benchmark representing normal acoustic behavior. By calculating the second difference vector between the current feature and this benchmark, and using this difference to perform final attention weighting on the feature, the generated acoustic emission features can keenly highlight abnormal acoustic activities that deviate from the normal pattern, thereby providing highly sensitive and discriminative input features for defect identification.

[0111] In the above embodiments, to obtain acoustic emission characteristics, the acoustic emission data undergoes time-frequency transformation processing to obtain an acoustic emission time-frequency spectrum. The time-frequency transformation converts the acoustic emission data from a time-domain representation to a time-frequency domain representation to capture the temporal and frequency characteristics of the acoustic emission events. Then, the acoustic emission time-frequency spectrum is subjected to two-dimensional convolution processing to obtain acoustic emission convolutional features. This two-dimensional convolution is used to extract waveform patterns and frequency distribution features from the acoustic emission events. Finally, the acoustic emission convolutional features are subjected to temporal attention weighting to obtain temporally weighted acoustic emission features. Temporal attention weighting is used to enhance the weights of acoustic emission events related to thermal damage. Finally, frequency attention is applied to the temporally weighted acoustic emission features. Attention-weighted features in the acoustic emission radio frequency domain are obtained. Frequency attention weighting is used to enhance the weight of characteristic frequency components related to material damage. Then, the second reference feature vector is updated. The second dynamic feature library is initialized or updated. The second dynamic feature library is used to store the acoustic emission radio frequency domain weighted features corresponding to workpieces that have been identified as qualified in other production batches. The capacity of the second dynamic feature library is fixed. The second reference feature vector is obtained by averaging the second dynamic feature library. Then, the second difference vector between the acoustic emission radio frequency domain weighted features of the current workpiece and the second reference feature vector is calculated. Based on the second difference vector, attention weighting is applied to the acoustic emission radio frequency domain weighted features to generate acoustic emission features.

[0112] As a further implementation of the method, when the static data is ultrasound detection data and X-ray image data, the step of performing third semantic encoding on the static data through a third semantic encoding unit to obtain static features includes:

[0113] Step S51: Perform multimodal feature alignment processing on the static data to obtain static aligned features. The multimodal feature alignment processing is used to map the ultrasound detection data and X-ray image data to a unified feature space.

[0114] Step S52: Perform convolution processing on the static alignment features to obtain static convolution features, wherein the convolution processing is used to enhance structural features related to internal defects.

[0115] Step S53: Perform cross-modal attention weighting on the static convolutional features to obtain static weighted features. The cross-modal attention weighting calculates the mutual enhancement weights between different modal features through a cross-attention mechanism to achieve complementary correlation between ultrasound features and X-ray features.

[0116] Step S54: Instantiate defects on the static weighted features to obtain a set of defect entities. The defect instantiation process is used to identify defect individuals and extract the geometric parameters of the defect individuals.

[0117] Step S55: Based on the set of defective entities, construct a defect topology graph using a graph neural network.

[0118] Step S56: Extract graph features from the defect topology map to obtain static features.

[0119] It should be noted that from step S51 to step S56, a refined and structured processing flow for static data is constructed, which aims to deeply mine and correlate the defect information inside the workpiece. The process begins with feature alignment (step S51), which maps heterogeneous ultrasonic testing data and X-ray image data to a unified feature space through multimodal feature alignment processing. Next, feature enhancement (step S52) is performed, using convolution processing to strengthen the structural feature representation related to internal defects. Based on this, cross-modal attention weighting (step S53) is used to explore the complementarity between different modal features, and a cross-attention mechanism is used to calculate the mutual enhancement weights between ultrasonic and X-ray features, achieving information complementarity and semantic association. Subsequently, the process enters the defect structuring stage (steps S54 and S55), where defect instantiation processing accurately identifies and parameterizes each individual defect, forming a defect entity set. Based on this set, a graph neural network is used to construct a defect topology graph, modeling the spatial location and interaction relationships between different defects. Finally, graph feature extraction (step S56) is performed on the defect topology graph, transforming the abstract defect distribution and association patterns into highly structured static features that can be directly used by subsequent tasks, thereby achieving a comprehensive and in-depth representation of the internal defect state of the workpiece from individual attributes to the overall layout.

[0120] In the above implementation, to obtain static features, multimodal feature alignment processing is performed on the static data to obtain static aligned features. This multimodal feature alignment processing maps ultrasound detection data and X-ray image data to a unified feature space. Then, convolution processing is performed on the static aligned features to obtain static convolutional features. This convolution processing enhances structural features related to internal defects. Next, cross-modal attention weighting is applied to the static convolutional features to obtain static weighted features. This cross-modal attention weighting calculates the mutual enhancement weights between different modal features through a cross-attention mechanism to achieve complementary correlation between ultrasound and X-ray features. Finally, defect instantiation is performed on the static weighted features to obtain a defect entity set. This defect instantiation process identifies individual defects and extracts their geometric parameters. Based on the defect entity set, a defect topology graph is constructed using a graph neural network. Finally, graph feature extraction is performed on the defect topology graph to obtain the static features.

[0121] As a further implementation of the method, the step of semantically associating infrared image features and acoustic emission features to obtain semantically associated features includes:

[0122] Step S61: The infrared image features and acoustic emission features are loaded into the semantic association unit of the workpiece defect analysis network, wherein the semantic association unit has a built-in learnable first semantic space transformation matrix and second semantic space transformation matrix.

[0123] Step S62: The infrared image features are semantically transformed using the first semantic space transformation matrix to obtain the first transformed features.

[0124] Step S63: The acoustic emission features are semantically transformed using the second semantic space transformation matrix to obtain the second transformed features, wherein the first transformed features and the second transformed features are in the same semantic space.

[0125] Step S64: Based on the first transformation feature, the second transformation feature is attention-weighted to obtain the first weighted feature, wherein the first weighted feature is used to enhance the corresponding thermal anomaly region in the infrared feature by utilizing the damage time sequence information in the acoustic emission feature.

[0126] Step S65: Based on the second transformation feature, the first transformation feature is attention-weighted to obtain the second weighted feature, wherein the second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by utilizing the temperature distribution information in the infrared feature.

[0127] Step S66: The first weighted feature and the second weighted feature are fused to obtain the semantic association feature.

[0128] It should be noted that from steps S61 to S66, two learnable semantic space transformation matrices are used to map infrared image features and acoustic emission features from different physical domains into a unified and comparable common semantic space. Then, there is a bidirectional attention interaction (steps S64 and S65). This mechanism does not perform simple feature concatenation, but performs two attention weightings in different directions: first, the damage time sequence information in the acoustic emission features (such as stress wave events at a specific moment) is used as a guide to enhance the saliency of the corresponding thermal anomaly region in the infrared features; conversely, the spatial temperature distribution information in the infrared features is used as context to strengthen the weight of damage events in the acoustic emission features that are temporally associated with the thermal anomaly region. This bidirectional interaction enables the two modalities to corroborate and reinforce each other. Finally, the two weighted features after bidirectional enhancement are integrated through feature fusion (step S66) to generate a unified semantic association feature. This feature not only retains the original information of each modality, but also deeply embeds the strong semantic association between "where the heat is generated" and "when the sound is emitted", thereby constructing a joint representation that is more sensitive to thermal damage.

[0129] In the above embodiments, to obtain semantic association features, infrared image features and acoustic emission features are loaded into the semantic association unit of the workpiece defect analysis network. The semantic association unit has a built-in learnable first semantic space transformation matrix and second semantic space transformation matrix. Then, the infrared image features are semantically transformed using the first semantic space transformation matrix to obtain the first transformed feature. Then, the acoustic emission features are semantically transformed using the second semantic space transformation matrix to obtain the second transformed feature. The first and second transformed features are in the same semantic space. Then, attention weighting is applied to the second transformed feature based on the first transformed feature to obtain the first weighted feature. The first weighted feature is used to enhance the corresponding thermal anomaly region in the infrared feature by utilizing the damage time sequence information in the acoustic emission feature. Then, attention weighting is applied to the first transformed feature based on the second transformed feature to obtain the second weighted feature. The second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by utilizing the temperature distribution information in the infrared feature. Finally, the first weighted feature and the second weighted feature are fused to obtain the semantic association features.

[0130] As a further implementation of the method, the step of semantically enhancing static features based on semantic association features to obtain semantically enhanced features includes:

[0131] Step S71: Load semantic association features and static features into the semantic enhancement unit of the workpiece defect analysis network, wherein the semantic enhancement unit includes at least a first semantic enhancement subunit and a second semantic enhancement subunit.

[0132] Step S72: Using the first semantic enhancement subunit, the static features are enhanced semantically based on the semantic association features to obtain the first semantic enhancement features.

[0133] Step S73: Through the second semantic enhancement subunit, based on the semantic association features, the static features are enhanced in a second semantic way to obtain the second semantic enhancement features.

[0134] Step S74: Calculate the mean of the first semantic enhancement feature and the second semantic enhancement feature to obtain the semantic enhancement feature.

[0135] It should be noted that from steps S71 to S74, a dual-enhancement and fusion architecture is adopted: First, through two independent semantic enhancement subunits, the static features are semantically enhanced twice with different focuses based on a unified semantic association feature (which integrates dynamic infrared and acoustic emission information), generating the first and second semantic enhancement features. This parallel processing design enables the model to learn from different dimensions how to use dynamic process information to supplement and enrich the static detection data; subsequently, by averaging the two enhancement results, the final semantic enhancement feature is obtained. This fusion strategy not only integrates the advantages of different enhancement paths and improves the richness of features, but also effectively improves the stability and robustness of the final feature through integrated averaging, ensuring that the enhanced static features can more comprehensively and reliably reflect the true state of the workpiece.

[0136] In the above embodiments, in order to obtain semantic enhancement features, semantic association features and static features are loaded into the semantic enhancement unit of the workpiece defect analysis network. The semantic enhancement unit includes at least a first semantic enhancement subunit and a second semantic enhancement subunit. Then, the first semantic enhancement subunit performs first semantic enhancement on the static features based on the semantic association features to obtain the first semantic enhancement feature. Then, the second semantic enhancement subunit performs second semantic enhancement on the static features based on the semantic association features to obtain the second semantic enhancement feature. Finally, the first semantic enhancement feature and the second semantic enhancement feature are averaged to obtain the semantic enhancement feature.

[0137] As a further implementation of the method, the step of performing first semantic enhancement on static features based on semantic association features through a first semantic enhancement subunit to obtain first semantically enhanced features includes:

[0138] Step S81: The semantically associated features are transformed into third transformed features by the third semantic space transformation matrix built into the first semantic enhancement subunit. The third semantic space transformation matrix is ​​used to transform the semantically associated features into the semantic space where the static features are located.

[0139] Step S82: Determine the parameter mapping relationship between the third transformation feature and the static feature, and based on the parameter mapping relationship, determine the influence weight distribution of the semantic association feature on the static feature. The influence weight distribution is used to quantify the degree of influence of the semantic association feature on the static feature.

[0140] Step S83: Based on the influence weight distribution, the static features are weighted to obtain the first semantic enhancement feature.

[0141] It should be noted that steps S81 to S83 specifically illustrate the working mechanism of the first semantic enhancement subunit, the core of which is to achieve precise injection of cross-domain information through a guided weighting. This process begins with semantic space alignment (step S81), using a third semantic space transformation matrix to project the semantic association features representing dynamic processes onto the semantic space where static features reside, generating third transformed features. This transformation ensures that features from different domains can be directly compared and interacted in a unified context. Subsequently, the system analyzes and quantifies the impact of dynamic information through a learnable mapping function (step S82), accurately calculating the parameter mapping relationship between the third transformed features and the original static features, and generating a fine-grained impact weight distribution accordingly. This weight distribution quantifies the potential impact of dynamic process information (such as thermo-acoustic related events) on each component (i.e., different internal structural attributes) in the static features. Finally, in the feature enhancement execution stage (step S83), the system performs element-wise weighting on the original static features based on this weight distribution, thereby generating the first semantic enhancement feature. The essence of this operation is to use the clues revealed by dynamic monitoring to selectively amplify those components in static features that are more likely to be affected by thermo-acoustic processes and are related to potential damage, while suppressing irrelevant or interfering components. This allows the final features to more acutely highlight the defect patterns induced or aggravated by the dynamic process, significantly improving the sensitivity and accuracy of defect identification.

[0142] It should be further explained that the mechanism of the second semantic enhancement feature is basically the same as that of the first semantic enhancement feature.

[0143] In the above implementation, in order to obtain the first semantic enhancement feature, the semantic association feature is semantically transformed by the third semantic space transformation matrix built into the first semantic enhancement subunit to obtain the third transformed feature. The third semantic space transformation matrix is ​​used to transform the semantic association feature to the semantic space where the static feature is located. Then, the parameter mapping relationship between the third transformed feature and the static feature is determined. Based on the parameter mapping relationship, the influence weight distribution of the semantic association feature on the static feature is determined. The influence weight distribution is used to quantify the degree of influence of the semantic association feature on the static feature. Then, based on the influence weight distribution, the static feature is weighted to obtain the first semantic enhancement feature.

[0144] This application also discloses a sensor data analysis system based on the Industrial Internet of Things.

[0145] refer to Figure 2 A sensor data analysis system based on the Industrial Internet of Things (IIoT) includes a management platform, a sensor network platform, and an object platform that are sequentially connected in communication. The management platform is configured with:

[0146] The data acquisition module is used to acquire infrared image data and acoustic emission data of each workpiece in the same batch in a continuously changing temperature field through sensors, and to acquire static data of the workpiece under normal temperature conditions. The static data includes at least one of ultrasonic detection data and X-ray image data.

[0147] A single semantic encoding module is used to perform semantic encoding on infrared image data, acoustic emission data and static data based on the workpiece defect analysis network to obtain infrared image features, acoustic emission features and static features;

[0148] The multi-semantic fusion module is used to semantically correlate infrared image features and acoustic emission features to obtain semantically correlated features; and, based on the semantically correlated features, to semantically enhance static features to obtain semantically enhanced features.

[0149] The analysis module is used to output workpiece defect analysis results based on semantic enhancement features.

[0150] The overall framework of another application scenario of the sensor network monitoring system based on the Industrial Internet of Things of this invention is as follows: Figure 3 As shown, the system can include a user platform, a service platform, a management platform, a sensor network platform, and an object platform that interact sequentially, forming a five-platform architecture based on the Industrial Internet of Things (IIoT). The service platform consists of a main service database, multiple service sub-platforms, and multiple service sub-databases. The management platform includes a feature value generation module, a feature construction module, a feature decomposition module, an inspection quality index generation module, and an inspection personnel allocation module. The management platform can interact with the sensor network platform and the service platform. The sensor network platform can include a main sensor database, multiple sensor network sub-platforms, and multiple sensor sub-databases. In this embodiment, there are n sensor network sub-platforms and n sensor sub-databases. Each sensor network sub-platform has a corresponding sensor sub-database. The sensor network platform can interact with the object platform.

[0151] By leveraging the interaction between various functional platforms of the industrial IoT-based sensor network monitoring system, which is based on the aforementioned three or five platforms, a complete closed-loop information operation logic is established, ensuring the orderly operation of sensing and control information and realizing intelligent equipment management.

[0152] Specifically, the sensor network monitoring system based on the Industrial Internet of Things in this embodiment includes a management platform. The management platform is configured to: acquire infrared image data and acoustic emission data corresponding to each workpiece in the same batch in a continuously changing temperature field through sensors, and acquire static data of the workpiece under normal temperature conditions, wherein the static data includes at least one of ultrasonic detection data and X-ray image data; based on a workpiece defect analysis network, perform semantic encoding on the infrared image data, acoustic emission data and static data respectively to obtain infrared image features, acoustic emission features and static features; perform semantic association on the infrared image features and acoustic emission features to obtain semantic association features; and, based on the semantic association features, perform semantic enhancement on the static features to obtain semantic enhancement features; and output the workpiece defect analysis results based on the semantic enhancement features.

[0153] The sensor data analysis system based on the Industrial Internet of Things (IIoT) of the present invention can implement any of the sensor data analysis methods based on the Industrial Internet of Things, and the specific working process of the sensor data analysis system based on the Industrial Internet of Things of the present invention can refer to the corresponding process in the above-mentioned sensor data analysis methods based on the Industrial Internet of Things.

[0154] This application also discloses a computer device.

[0155] refer to Figure 4 A computer device includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement any of the above-described methods for sensor data analysis based on the Industrial Internet of Things.

[0156] This application also discloses a computer-readable storage medium.

[0157] A computer-readable storage medium storing a computer program that can be loaded by a processor and executed any of the above-described methods for sensor data analysis based on the Industrial Internet of Things.

[0158] The computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device; the program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0159] The above are all preferred embodiments of this application and are not intended to limit the scope of protection of this application. Any feature disclosed in this specification (including the abstract and drawings) may be replaced by other equivalent or similar features unless specifically stated otherwise. That is, unless specifically stated otherwise, each feature is only one example of a series of equivalent or similar features.

Claims

1. A method for analyzing sensor data based on industrial internet of things, characterized in that, Applied to an industrial Internet of Things (IIoT) system, the IIoT system includes a management platform, a sensor network platform, and an object platform that are sequentially and communicatively connected. The method is executed by the management platform and includes: The infrared image data and acoustic emission data of each workpiece in the same batch are acquired by the sensor in a continuously changing temperature field, and the static data of the workpiece at room temperature are also acquired. The static data includes at least one of ultrasonic detection data and X-ray image data. Based on the workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are semantically encoded to obtain infrared image features, acoustic emission features and static features; Semantic association is performed on the infrared image features and the acoustic emission features to obtain semantic association features; and semantic enhancement is performed on the static features based on the semantic association features to obtain semantic enhancement features. Based on the semantic enhancement features, output the workpiece defect analysis results; The step of semantically associating the infrared image features and the acoustic emission features to obtain semantically associated features includes: The infrared image features and the acoustic emission features are loaded into the semantic association unit of the workpiece defect analysis network, wherein the semantic association unit has a built-in learnable first semantic space transformation matrix and second semantic space transformation matrix; The first semantic space transformation matrix is ​​used to perform semantic space transformation on the infrared image features to obtain the first transformed features; The acoustic emission features are semantically transformed using the second semantic space transformation matrix to obtain the second transformed features, wherein the first transformed features and the second transformed features are in the same semantic space; Based on the first conversion feature, the second conversion feature is attention-weighted to obtain a first weighted feature, wherein the first weighted feature is used to enhance the corresponding thermal anomaly region in the infrared feature by utilizing the damage time sequence information in the acoustic emission feature. Based on the second conversion feature, the first conversion feature is attention-weighted to obtain the second weighted feature, wherein the second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by utilizing the temperature distribution information in the infrared feature; The first weighted feature and the second weighted feature are fused to obtain semantic association features; The step of semantically enhancing the static features based on the semantic association features to obtain semantically enhanced features includes: The semantic association features and the static features are loaded into the semantic enhancement unit of the workpiece defect analysis network, wherein the semantic enhancement unit includes at least a first semantic enhancement subunit and a second semantic enhancement subunit; The first semantic enhancement subunit performs first semantic enhancement on the static features based on the semantic association features to obtain the first semantic enhancement features; The second semantic enhancement subunit performs second semantic enhancement on the static features based on the semantic association features to obtain the second semantic enhancement features; The semantic enhancement features are obtained by averaging the first semantic enhancement features and the second semantic enhancement features. 2.The industrial Internet of things based sensor data analysis method according to claim 1, characterized in that, The step of semantically encoding the infrared image data, the acoustic emission data, and the static data respectively to obtain infrared image features, acoustic emission features, and static features includes: The infrared image data, the acoustic emission data, and the static data are loaded into the workpiece defect analysis network, wherein the workpiece defect analysis network includes a semantic coding sub-network, and the semantic coding sub-network includes a first semantic coding unit, a second semantic coding unit, and a third semantic coding unit, each with a different network structure. The infrared image data is semantically encoded by the first semantic encoding unit to obtain infrared image features; The acoustic emission data is semantically encoded using the second semantic encoding unit to obtain acoustic emission features; The static data is subjected to third semantic encoding by the third semantic encoding unit to obtain static features. 3.The industrial Internet of things based sensor data analysis method according to claim 2, characterized in that, The step of performing a first semantic encoding on the infrared image data to obtain infrared image features includes: The infrared image data is subjected to multi-level convolution processing to obtain infrared multi-scale feature maps. The multi-level convolution processing includes parallel convolution processing of the same infrared image using convolution kernels of different sizes. The infrared multi-scale feature map is subjected to channel attention weighting to obtain infrared channel weighted features, wherein the channel attention weighting is used to enhance the weight of feature channels related to thermal damage; Spatial self-attention mining is performed on the weighted features of the infrared channels to obtain infrared self-attention features, wherein the spatial self-attention mining is used to establish semantic associations between different temperature regions; Update the first reference feature vector, wherein the first dynamic feature library is initialized or updated. The first dynamic feature library is used to store the infrared self-attention features corresponding to the workpieces that have been identified as qualified in other production batches. The capacity of the dynamic feature library is fixed. The first reference feature vector is obtained by averaging the first dynamic feature library. Calculate the first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector, and based on the first difference vector, perform attention weighting on the infrared self-attention feature to generate infrared image features. 4.The industrial Internet of things based sensor data analysis method according to claim 2, wherein, The step of performing a second semantic encoding on the acoustic emission data to obtain acoustic emission features includes: The acoustic emission data is subjected to time-frequency transformation processing to obtain an acoustic emission time-frequency spectrum. The time-frequency transformation processing is used to convert the acoustic emission data from a time domain representation to a time-frequency domain representation in order to capture the temporal and frequency characteristics of the acoustic emission event. The acoustic emission time-spectrum is subjected to two-dimensional convolution processing to obtain acoustic emission convolution features, wherein the two-dimensional convolution processing is used to extract waveform patterns and frequency distribution features in the acoustic emission event; Temporal attention weighting is applied to the acoustic emission convolutional features to obtain temporally weighted acoustic emission features, wherein the temporal attention weighting is used to enhance the weights of acoustic emission events related to thermal damage; Frequency attention weighting is applied to the acoustic emission time-series weighted features to obtain acoustic emission radio frequency domain weighted features, wherein the frequency attention weighting is used to enhance the weight of characteristic frequency components related to material damage; Update the second reference feature vector, wherein the second dynamic feature library is initialized or updated. The second dynamic feature library is used to store the acoustic emission radio frequency domain weighted features corresponding to the workpieces that have been identified as qualified in other production batches. The capacity of the second dynamic feature library is fixed. The second reference feature vector is obtained by averaging the second dynamic feature library. Calculate the second difference vector between the acoustic emission radio frequency domain weighted features of the current workpiece and the second reference feature vector, and based on the second difference vector, perform attention weighting on the acoustic emission radio frequency domain weighted features to generate acoustic emission features. 5.The industrial Internet of things based sensor data analysis method according to claim 2, wherein, When the static data consists of the ultrasound detection data and the X-ray image data, the step of performing third semantic encoding on the static data through the third semantic encoding unit to obtain static features includes: The static data is subjected to multimodal feature alignment processing to obtain static aligned features, wherein the multimodal feature alignment processing is used to map ultrasound detection data and X-ray image data to a unified feature space; The static alignment features are convolved to obtain static convolution features, wherein the convolution process is used to enhance structural features related to internal defects; The static convolutional features are subjected to cross-modal attention weighting to obtain static weighted features. The cross-modal attention weighting calculates the mutual enhancement weights between different modal features through a cross-attention mechanism to achieve complementary correlation between ultrasound features and X-ray features. The static weighted features are instantiated to obtain a set of defect entities, wherein the defect instantiation process is used to identify defect individuals and extract the geometric parameters of the defect individuals; Based on the set of defective entities, a defect topology graph is constructed using a graph neural network; Graph feature extraction is performed on the defect topology graph to obtain static features. 6.The industrial Internet of things based sensor data analysis method according to claim 1, wherein, The step of performing first semantic enhancement on the static features based on the semantic association features through the first semantic enhancement subunit to obtain the first semantically enhanced features includes: The semantic association features are transformed into third transformed features by the third semantic space transformation matrix built into the first semantic enhancement subunit. The third semantic space transformation matrix is ​​used to transform the semantic association features into the semantic space where the static features are located. Determine the parameter mapping relationship between the third transformation feature and the static feature, and based on the parameter mapping relationship, determine the influence weight distribution of the semantic association feature on the static feature, wherein the influence weight distribution is used to quantify the degree of influence of the semantic association feature on the static feature; Based on the influence weight distribution, the static features are weighted to obtain the first semantic enhancement feature.

7. An industrial internet of things based sensor data analysis system characterized in that, It includes a management platform, a sensor network platform, and an object platform that are connected in sequence. The management platform is configured with: The data acquisition module is used to acquire infrared image data and acoustic emission data of each workpiece in the same batch in a continuously changing temperature field through sensors, and to acquire static data of the workpiece under normal temperature conditions, wherein the static data includes at least one of ultrasonic detection data and X-ray image data. A single semantic encoding module is used to perform semantic encoding on the infrared image data, the acoustic emission data and the static data respectively based on the workpiece defect analysis network to obtain infrared image features, acoustic emission features and static features; A multi-semantic fusion module is used to semantically correlate the infrared image features and the acoustic emission features to obtain semantically correlated features; and to semantically enhance the static features based on the semantically correlated features to obtain semantically enhanced features. The analysis module is used to output the workpiece defect analysis results based on the semantic enhancement features; The step of semantically associating the infrared image features and the acoustic emission features to obtain semantically associated features includes: The infrared image features and the acoustic emission features are loaded into the semantic association unit of the workpiece defect analysis network, wherein the semantic association unit has a built-in learnable first semantic space transformation matrix and second semantic space transformation matrix; The first semantic space transformation matrix is ​​used to perform semantic space transformation on the infrared image features to obtain the first transformed features; The acoustic emission features are semantically transformed using the second semantic space transformation matrix to obtain the second transformed features, wherein the first transformed features and the second transformed features are in the same semantic space; Based on the first conversion feature, the second conversion feature is attention-weighted to obtain a first weighted feature, wherein the first weighted feature is used to enhance the corresponding thermal anomaly region in the infrared feature by utilizing the damage time sequence information in the acoustic emission feature. Based on the second conversion feature, the first conversion feature is attention-weighted to obtain the second weighted feature, wherein the second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by utilizing the temperature distribution information in the infrared feature; The first weighted feature and the second weighted feature are fused to obtain semantic association features; The step of semantically enhancing the static features based on the semantic association features to obtain semantically enhanced features includes: The semantic association features and the static features are loaded into the semantic enhancement unit of the workpiece defect analysis network, wherein the semantic enhancement unit includes at least a first semantic enhancement subunit and a second semantic enhancement subunit; The first semantic enhancement subunit performs first semantic enhancement on the static features based on the semantic association features to obtain the first semantic enhancement features; The second semantic enhancement subunit performs second semantic enhancement on the static features based on the semantic association features to obtain the second semantic enhancement features; The semantic enhancement features are obtained by averaging the first semantic enhancement features and the second semantic enhancement features.

8. A computer device, comprising: The method includes a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method of any one of claims 1 to 6.

Images