Sensor data analysis method, system and equipment based on industrial Internet of Things

Abstract

The invention discloses a sensor data analysis method, system and device based on the industrial Internet of Things, and relates to the technical field of the industrial Internet of Things, and the method comprises the steps: obtaining the corresponding infrared image data and acoustic emission data of each workpiece in the same batch in a continuously changing temperature field through a sensor, obtaining static data of the workpiece in a normal temperature state; respectively performing semantic coding on the infrared image data, the acoustic emission data and the static data to obtain an infrared image feature, an acoustic emission feature and a static feature; performing semantic association on the infrared image features and the acoustic emission features to obtain semantic association features; based on the semantic association feature, performing semantic enhancement on the static feature to obtain a semantic enhancement feature; and outputting a workpiece defect analysis result based on the semantic enhancement features. The method has the effect of improving the accuracy of workpiece defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial Internet of Things, and in particular to a sensor data analysis method, system and device based on industrial Internet of Things. BACKGROUND

[0002] In modern industrial manufacturing, workpiece quality is the cornerstone of product safety and reliability. Non-destructive testing technologies, such as ultrasonic testing, X-ray testing, infrared thermal imaging and acoustic emission testing, have become key means of quality control. These technologies each have their own advantages: ultrasonic and X-ray are good at detecting internal defects, infrared thermal imaging can reflect surface temperature distribution anomalies, and acoustic emission can capture real-time stress wave activity in the material.

[0003] However, the existing technology generally has the following limitations: first, various detection methods usually operate independently, and the data analysis process is isolated from each other, forming an "information island". This single-modal analysis method is difficult to fully characterize the complete attributes of defects, for example, a micro-crack that occurs during thermal stress and hides after cooling may be missed by static detection methods; second, traditional detection schemes cannot effectively correlate the response data of the workpiece in the dynamic process with its internal quality data in the static state, making it impossible to trace the formation mechanism of defects. In addition, although industrial Internet of Things can achieve multi-source data collection, there is a lack of models and methods that can deeply integrate and intelligently analyze these heterogeneous data, limiting the further improvement of defect recognition accuracy.

[0004] Therefore, there is an urgent need in the art for a technical solution that can integrate industrial Internet of Things resources and cooperatively analyze and semantically correlate multi-modal sensor data, in order to achieve earlier, more comprehensive and more accurate identification of workpiece defects. SUMMARY

[0005] In order to improve the accuracy of workpiece defect detection, the present application provides a sensor data analysis method, system and device based on industrial Internet of Things.

[0006] In a first aspect, the present application provides a sensor data analysis method based on industrial Internet of Things, which adopts the following technical solution: The sensor data analysis method based on industrial Internet of Things is applied to an industrial Internet of Things system, the industrial Internet of Things system comprising a management platform, a sensor network platform and an object platform connected in sequence, the method being executed by the management platform and comprising: acquiring, by a sensor, infrared image data and acoustic emission data corresponding to each workpiece in a same batch in a continuously changing temperature field, and acquiring static data of the workpiece in a normal temperature state, wherein the static data comprises at least one of ultrasonic testing data and X-ray image data; based on the workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are respectively semantically encoded to obtain infrared image features, acoustic emission features and static features; The infrared image features and the acoustic emission features are semantically associated to obtain semantic association features, and the static features are semantically reinforced based on the semantic association features to obtain semantic reinforcement features. Based on the semantic reinforcement features, a workpiece defect analysis result is output.

[0007] By adopting the above technical solutions, the infrared image data and the acoustic emission data of each workpiece in the same batch in a continuously changing temperature field are obtained by a sensor, and the static data of the workpiece under normal temperature conditions is obtained, wherein the static data includes at least one of ultrasonic detection data and X-ray image data, then based on the workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are respectively semantically encoded to obtain infrared image features, acoustic emission features and static features, then the infrared image features and the acoustic emission features are semantically associated to obtain semantic association features, and based on the semantic association features, the static features are semantically reinforced to obtain semantic reinforcement features, and then based on the semantic reinforcement features, a workpiece defect analysis result is output. In the present application, at the data level, the fusion of dynamic process data and static detection data overcomes the one-sidedness of a single data source. Secondly, at the information processing level, through semantic encoding, association and reinforcement, deep-level and semantic-level information complementation and synergistic enhancement between multi-modal data are realized, rather than simple data stacking, so that the system can comprehensively consider the static performance and dynamic evolution of defects, thereby realizing more comprehensive and accurate identification and evaluation of workpiece defects, reducing the risk of missed detection and misjudgment, and further enhancing the early warning ability of early defects and potential failure risks, thereby improving the intelligent level and reliability of industrial quality control.

[0008] Optionally, the step of respectively semantically encoding the infrared image data, the acoustic emission data and the static data to obtain infrared image features, acoustic emission features and static features includes: loading the infrared image data, the acoustic emission data and the static data into a workpiece defect analysis network, wherein the workpiece defect analysis network includes a semantic encoding subnetwork, and the semantic encoding subnetwork includes first, second and third semantic encoding units with different network structures; first semantic encoding of the infrared image data by the first semantic encoding unit to obtain infrared image features; second semantic encoding of the acoustic emission data by the second semantic encoding unit to obtain acoustic emission features; The third semantic coding unit codes the static data to obtain static features.

[0009] By adopting the technical scheme, in order to realize extraction of infrared image features, acoustic emission features and static features, the infrared image data, the acoustic emission data and the static data are loaded into a workpiece defect analysis network, wherein the workpiece defect analysis network comprises a semantic coding subnetwork, the semantic coding subnetwork comprises a first semantic coding unit, a second semantic coding unit and a third semantic coding unit which have different network structures, then the first semantic coding unit is used to code the infrared image data to obtain infrared image features, the second semantic coding unit is used to code the acoustic emission data to obtain acoustic emission features, and the third semantic coding unit is used to code the static data to obtain static features.

[0010] Optionally, the step of coding the infrared image data to obtain infrared image features comprises: performing multi-level convolution processing on the infrared image data to obtain an infrared multi-scale feature map, wherein the multi-level convolution processing comprises parallel convolution processing on the same infrared image by using convolution kernels of different sizes; performing channel attention weighting on the infrared multi-scale feature map to obtain infrared channel weighted features, wherein the channel attention weighting is used to enhance the weight of a feature channel related to thermal damage; performing spatial self-attention mining on the infrared channel weighted features to obtain infrared self-attention features, wherein the spatial self-attention mining is used to establish a semantic association relationship between different temperature regions; updating a first reference feature vector, wherein a first dynamic feature library is initialized or updated, the first dynamic feature library is used to store infrared self-attention features corresponding to workpieces that have been identified as qualified in other production batches, the capacity of the dynamic feature library is fixed, and the first reference feature vector is obtained by averaging the first dynamic feature library; calculating a first difference vector between the infrared self-attention features of the current workpiece and the first reference feature vector, and performing attention weighting on the infrared self-attention features based on the first difference vector to generate infrared image features.

[0011] By adopting the technical scheme, in order to obtain the infrared image feature, the infrared image data is subjected to multi-level convolution processing to obtain an infrared multi-scale feature map, wherein the multi-level convolution processing includes performing parallel convolution processing on the same infrared image by using convolution kernels of different sizes, then performing channel attention weighting on the infrared multi-scale feature map to obtain an infrared channel weighted feature, wherein the channel attention weighting is used to enhance the weight of a feature channel related to thermal damage, then performing spatial self-attention mining on the infrared channel weighted feature to obtain an infrared self-attention feature, wherein the spatial self-attention mining is used to establish a semantic correlation between different temperature regions, then updating a first reference feature vector, wherein a first dynamic feature library is initialized or updated, the first dynamic feature library is used to store the infrared self-attention features corresponding to the workpieces in other production batches that have been identified as qualified, the capacity of the dynamic feature library is fixed, and the first reference feature vector is obtained by averaging the first dynamic feature library, then a first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector is calculated, and the infrared self-attention feature is subjected to attention weighting based on the first difference vector to generate the infrared image feature.

[0012] Optionally, the step of performing second semantic encoding on the acoustic emission data to obtain the acoustic emission feature comprises: performing time-frequency transform processing on the acoustic emission data to obtain an acoustic emission time-frequency spectrogram, wherein the time-frequency transform processing is used to convert the acoustic emission data from a time domain representation to a time-frequency domain representation to capture the timing characteristics and frequency characteristics of the acoustic emission events; performing two-dimensional convolution processing on the acoustic emission time-frequency spectrogram to obtain an acoustic emission convolution feature, wherein the two-dimensional convolution processing is used to extract waveform patterns and frequency distribution characteristics in the acoustic emission events; performing timing attention weighting on the acoustic emission convolution feature to obtain an acoustic emission timing weighted feature, wherein the timing attention weighting is used to enhance the weight of the acoustic emission events related to thermal damage; performing frequency attention weighting on the acoustic emission timing weighted feature to obtain an acoustic emission frequency domain weighted feature, wherein the frequency attention weighting is used to enhance the weight of the feature frequency components related to material damage; updating a second reference feature vector, wherein a second dynamic feature library is initialized or updated, the second dynamic feature library is used to store the acoustic emission frequency domain weighted features corresponding to the workpieces in other production batches that have been identified as qualified, the capacity of the second dynamic feature library is fixed, and the second reference feature vector is obtained by averaging the second dynamic feature library; a second difference vector between the acoustic emission frequency domain weighted feature of the current workpiece and the second reference feature vector is calculated, and the acoustic emission frequency domain weighted feature is attention weighted based on the second difference vector to generate an acoustic emission feature.

[0013] By adopting the technical solutions, in order to obtain the acoustic emission feature, the acoustic emission data is subjected to time-frequency transformation processing to obtain an acoustic emission time-frequency spectrum, wherein the time-frequency transformation processing is used to convert the acoustic emission data from a time domain representation to a time-frequency domain representation to capture the timing characteristics and frequency characteristics of the acoustic emission event, then the acoustic emission time-frequency spectrum is subjected to two-dimensional convolution processing to obtain an acoustic emission convolution feature, wherein the two-dimensional convolution processing is used to extract the waveform mode and frequency distribution characteristics in the acoustic emission event, then the acoustic emission convolution feature is subjected to timing attention weighting to obtain an acoustic emission timing weighted feature, wherein the timing attention weighting is used to enhance the weight of the acoustic emission event related to thermal damage, then the acoustic emission timing weighted feature is subjected to frequency attention weighting to obtain an acoustic emission frequency domain weighted feature, wherein the frequency attention weighting is used to enhance the weight of the feature frequency component related to material damage, then the second reference feature vector is updated, wherein the second dynamic feature library is initialized or updated, the second dynamic feature library is used to store the acoustic emission frequency domain weighted features corresponding to the workpieces in other production batches that have been identified as qualified, the capacity of the second dynamic feature library is fixed, and the second reference feature vector is obtained by averaging the second dynamic feature library, then a second difference vector between the acoustic emission frequency domain weighted feature of the current workpiece and the second reference feature vector is calculated, and the acoustic emission frequency domain weighted feature is attention weighted based on the second difference vector to generate an acoustic emission feature.

[0014] Optionally, when the static data is the ultrasonic detection data and the X-ray image data, the step of performing third semantic encoding on the static data by the third semantic encoding unit to obtain a static feature comprises: performing multi-modal feature alignment processing on the static data to obtain a static aligned feature, wherein the multi-modal feature alignment processing is used to map the ultrasonic detection data and the X-ray image data to a unified feature space; performing convolution processing on the static aligned feature to obtain a static convolution feature, wherein the convolution processing is used to enhance the structural features related to internal defects; performing cross-modal attention weighting on the static convolution feature to obtain a static weighted feature, wherein the cross-modal attention weighting calculates the mutual enhancement weight between different modal features by a cross-attention mechanism to realize the complementary correlation between the ultrasonic feature and the X-ray feature; performing defect instantiation on the static weighted feature to obtain a defect entity set, wherein the defect instantiation processing is used to identify defect individuals and extract the geometric parameters of the defect individuals; constructing a defect topology graph based on the defect entity set through a graph neural network; performing graph feature extraction on the defect topology graph to obtain static features.

[0015] By adopting the technical scheme, in order to obtain static features, multi-modal feature alignment processing is performed on static data to obtain static alignment features, wherein the multi-modal feature alignment processing is used to map ultrasonic detection data and X-ray image data to a unified feature space, then convolution processing is performed on the static alignment features to obtain static convolution features, wherein the convolution processing is used to enhance structural features related to internal defects, then cross-modal attention weighting is performed on the static convolution features to obtain static weighted features, wherein the cross-modal attention weighting calculates mutual enhancement weights between different modal features through a cross-attention mechanism to realize complementary correlation between ultrasonic features and X-ray features, then defect instantiation is performed on the static weighted features to obtain a defect entity set, wherein the defect instantiation processing is used to identify defect individuals and extract geometric parameters of the defect individuals, then a defect topology graph is constructed based on the defect entity set through a graph neural network, then graph feature extraction is performed on the defect topology graph to obtain static features.

[0016] Optionally, the step of performing semantic association on the infrared image features and the acoustic emission features to obtain semantic association features comprises: loading the infrared image features and the acoustic emission features into a semantic association unit of the workpiece defect analysis network, wherein the semantic association unit is internally provided with a first semantic space conversion matrix and a second semantic space conversion matrix that can be learned; performing semantic space conversion on the infrared image features through the first semantic space conversion matrix to obtain first conversion features; performing semantic space conversion on the acoustic emission features through the second semantic space conversion matrix to obtain second conversion features, wherein the first conversion features and the second conversion features are in the same semantic space; performing attention weighting on the second conversion features based on the first conversion features to obtain first weighted features, wherein the first weighted features are used to enhance corresponding thermal anomaly regions in the infrared features by using damage time sequence information in the acoustic emission features; performing attention weighting on the first conversion features based on the second conversion features to obtain second weighted features, wherein the second weighted features are used to enhance corresponding damage events in the acoustic emission features by using temperature distribution information in the infrared features; fusing the first weighted features and the second weighted features to obtain semantic association features.

[0017] By adopting the technical scheme, in order to obtain the semantic correlation feature, the infrared image feature and the acoustic emission feature are loaded into a semantic correlation unit of the workpiece defect analysis network, wherein the semantic correlation unit is internally provided with a first semantic space conversion matrix and a second semantic space conversion matrix which can be learned, then the infrared image feature is subjected to semantic space conversion through the first semantic space conversion matrix to obtain a first conversion feature, then the acoustic emission feature is subjected to semantic space conversion through the second semantic space conversion matrix to obtain a second conversion feature, wherein the first conversion feature and the second conversion feature are in the same semantic space, then the second conversion feature is subjected to attention weighting based on the first conversion feature to obtain a first weighted feature, wherein the first weighted feature is used to enhance a corresponding thermal abnormal region in the infrared feature by using damage time sequence information in the acoustic emission feature, then the first conversion feature is subjected to attention weighting based on the second conversion feature to obtain a second weighted feature, wherein the second weighted feature is used to enhance a corresponding damage event in the acoustic emission feature by using temperature distribution information in the infrared feature, and then the first weighted feature and the second weighted feature are fused to obtain the semantic correlation feature.

[0018] Optionally, the step of performing semantic enhancement on the static feature based on the semantic correlation feature to obtain a semantic enhancement feature comprises: loading the semantic correlation feature and the static feature into a semantic enhancement unit of the workpiece defect analysis network, wherein the semantic enhancement unit at least comprises a first semantic enhancement subunit and a second semantic enhancement subunit; performing first semantic enhancement on the static feature based on the semantic correlation feature through the first semantic enhancement subunit to obtain a first semantic enhancement feature; performing second semantic enhancement on the static feature based on the semantic correlation feature through the second semantic enhancement subunit to obtain a second semantic enhancement feature; performing mean value calculation on the first semantic enhancement feature and the second semantic enhancement feature to obtain a semantic enhancement feature.

[0019] By adopting the technical scheme, in order to obtain the semantic correlation feature, the infrared image feature and the acoustic emission feature are loaded into a semantic correlation unit of the workpiece defect analysis network, wherein the semantic correlation unit is internally provided with a first semantic space conversion matrix and a second semantic space conversion matrix which can be learned, then the infrared image feature is subjected to semantic space conversion through the first semantic space conversion matrix to obtain a first conversion feature, then the acoustic emission feature is subjected to semantic space conversion through the second semantic space conversion matrix to obtain a second conversion feature, wherein the first conversion feature and the second conversion feature are in the same semantic space, then the second conversion feature is subjected to attention weighting based on the first conversion feature to obtain a first weighted feature, wherein the first weighted feature is used to enhance a corresponding thermal abnormal region in the infrared feature by using damage time sequence information in the acoustic emission feature, then the first conversion feature is subjected to attention weighting based on the second conversion feature to obtain a second weighted feature, wherein the second weighted feature is used to enhance a corresponding damage event in the acoustic emission feature by using temperature distribution information in the infrared feature, and then the first weighted feature and the second weighted feature are fused to obtain the semantic correlation feature.

[0020] Optionally, the step of performing first semantic reinforcement on the static feature based on the semantic correlation feature by the first semantic reinforcement subunit to obtain a first semantic reinforcement feature comprises: performing semantic space conversion on the semantic correlation feature by a third semantic space conversion matrix built in the first semantic reinforcement subunit to obtain a third conversion feature, wherein the third semantic space conversion matrix is used to convert the semantic correlation feature to a semantic space where the static feature is located; determining a parameter mapping relationship between the third conversion feature and the static feature, and determining an influence weight distribution of the semantic correlation feature on the static feature based on the parameter mapping relationship, wherein the influence weight distribution is used to quantify an influence degree of the semantic correlation feature on the static feature; performing weighting on the static feature based on the influence weight distribution to obtain the first semantic reinforcement feature.

[0021] By adopting the above technical solution, in order to obtain the first semantic reinforcement feature, the semantic space conversion matrix built in the first semantic reinforcement subunit is used to perform semantic space conversion on the semantic correlation feature to obtain the third conversion feature, wherein the third semantic space conversion matrix is used to convert the semantic correlation feature to the semantic space where the static feature is located, then the parameter mapping relationship between the third conversion feature and the static feature is determined, and the influence weight distribution of the semantic correlation feature on the static feature is determined based on the parameter mapping relationship, wherein the influence weight distribution is used to quantify the influence degree of the semantic correlation feature on the static feature, and then the weighting is performed on the static feature based on the influence weight distribution to obtain the first semantic reinforcement feature.

[0022] In a second aspect, the present application also provides a sensor data analysis system based on industrial Internet of Things, which adopts the following technical solution: The sensor data analysis system based on industrial Internet of Things comprises a management platform, a sensing network platform and an object platform which are sequentially connected in communication, and the management platform is configured with: a data acquisition module configured to acquire infrared image data and acoustic emission data corresponding to each workpiece in a same batch in a continuously changing temperature field by a sensor, and acquire static data of the workpiece in a normal temperature state, wherein the static data comprises at least one of ultrasonic detection data and X-ray image data; a single semantic encoding module configured to perform semantic encoding on the infrared image data, the acoustic emission data and the static data respectively based on a workpiece defect analysis network to obtain infrared image features, acoustic emission features and static features; A multi-semantic fusion module is configured to perform semantic association on the infrared image features and the acoustic emission features to obtain semantic association features, and perform semantic reinforcement on the static features based on the semantic association features to obtain semantic reinforcement features. An analysis module is configured to output a workpiece defect analysis result based on the semantic reinforcement features.

[0023] In a third aspect, the present application also provides a computer device, which adopts the technical scheme as follows: A computer device includes a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements the method in the first aspect when executing the computer program.

[0024] To sum up, the present application at least has the following beneficial technical effects: through the sensor, the infrared image data and the acoustic emission data of each workpiece in the same batch in the continuously changing temperature field are obtained, and the static data of the workpiece in the normal temperature state is obtained, wherein the static data includes at least one of the ultrasonic detection data and the X-ray image data, then based on the workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are respectively subjected to semantic coding to obtain the infrared image features, the acoustic emission features and the static features, then the infrared image features and the acoustic emission features are subjected to semantic association to obtain the semantic association features, and based on the semantic association features, the static features are subjected to semantic reinforcement to obtain the semantic reinforcement features, and then based on the semantic reinforcement features, the workpiece defect analysis result is output. In the present application, at the data level, the dynamic process data and the static detection data are fused to overcome the one-sidedness of a single data source; secondly, at the information processing level, through semantic coding, association and reinforcement, deep-level and semantic-level information complementation and synergistic enhancement between multi-modal data are realized, rather than simple data stacking, so that the system can comprehensively consider the static performance and dynamic evolution of defects, thereby realizing more comprehensive and accurate identification and evaluation of workpiece defects, reducing the risk of missed detection and misjudgment, and further enhancing the early warning ability of early defects and potential failure risks, and overall improving the intelligent level and reliability of industrial quality control. BRIEF DESCRIPTION OF DRAWINGS

[0025] Figure 1 is a whole process schematic diagram of an embodiment of the present application.

[0026] Figure 2 is a structure schematic diagram of one of the application scenarios of the system of an embodiment of the present application.

[0027] Figure 3 is a structure schematic diagram of another application scenario of the system of an embodiment of the present application.

[0028] Figure 4is a structural block diagram of a computer device of the present application. DETAILED DESCRIPTION

[0029] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and examples. It should be understood that the specific examples described herein are only used to explain the present application and do not limit the present application.

[0030] The embodiments of the present application disclose a sensor data analysis method based on an industrial Internet of Things.

[0031] With reference to Figure 1 The sensor data analysis method based on the industrial Internet of Things is applied to an industrial Internet of Things system, the industrial Internet of Things system includes a management platform, a sensor network platform and an object platform which are sequentially communicatively connected, the method is executed by the management platform, and includes the following steps. In step S11, the sensor obtains corresponding infrared image data and acoustic emission data of each workpiece in the same batch in a continuously changing temperature field, and obtains static data of the workpiece under a normal temperature state, wherein the static data includes at least one of ultrasonic detection data and X-ray image data.

[0032] It should be noted that in step S11, the system collects two types of key data through various sensors deployed on the object platform: one is process response data of the workpiece under the action of a dynamic and continuously changing temperature field, i.e., infrared image data and acoustic emission data, which are used to capture the surface thermodynamic behavior and internal microactivity of the workpiece during the heating process; the other is static data obtained under a normal temperature stable state, which can reflect the inherent structure of the workpiece, such as ultrasonic detection data or X-ray image data, thereby forming a multi-modal and multi-dimensional data set covering "dynamic process monitoring" and "static quality evaluation".

[0033] In step S12, based on a workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are respectively subjected to semantic coding to obtain infrared image features, acoustic emission features and static features.

[0034] It should be noted that in step S12, the collected original data is heterogeneous and contains a large amount of redundant information, and through the introduction of the semantic coding module in the workpiece defect analysis network, the system can perform high-level abstraction and understanding on the infrared image, the acoustic emission signal and the static data. This process converts the original pixel points, waveforms or images into a series of dense feature vectors that can represent potential defect patterns, material properties and their physical meanings, i.e., infrared image features, acoustic emission features and static features.

[0035] Step S13, performing semantic correlation on the infrared image features and the acoustic emission features to obtain semantic correlation features; and performing semantic reinforcement on the static features based on the semantic correlation features to obtain semantic reinforcement features.

[0036] It should be noted that in step S13, the two types of features obtained in the dynamic monitoring, i.e., the infrared image features and the acoustic emission features, are correlated in a cross-modal semantic manner, which aims to reveal the internal relationship between the thermal behavior and the acoustic activity, thereby obtaining a semantic correlation feature that can more comprehensively describe the comprehensive state of the workpiece under dynamic stress; subsequently, the static features extracted from the static data are semantically reinforced using the correlation feature as context or guiding information, which aims to inject the clues revealed by the dynamic process into the understanding of the static structure, so that the reinforced features not only contain internal structure information, but also fuse the behavior traces of defects in the dynamic process.

[0037] Step S14, outputting a workpiece defect analysis result based on the semantic reinforcement features.

[0038] It should be noted that in step S14, the semantic reinforcement features obtained after the foregoing fusion and reinforcement are input into the final decision layer (e.g., a classifier or regressor) of the workpiece defect analysis network, which analyzes and judges the input features based on the learned knowledge, and finally outputs a structured workpiece defect analysis result, which can specifically include the type (such as cracks, pores), position, severity level of the defects, or a comprehensive health state evaluation index, thereby directly serving the quality judgment and decision.

[0039] In the above embodiment, the corresponding infrared image data and acoustic emission data of each workpiece in the same batch in the continuously changing temperature field are acquired by the sensor, and static data of the workpiece in a normal temperature state is acquired, wherein the static data includes at least one of ultrasonic detection data and X-ray image data, then based on a workpiece defect analysis network, the infrared image data, the acoustic emission data and the static data are respectively subjected to semantic encoding to obtain infrared image features, acoustic emission features and static features, then the infrared image features and the acoustic emission features are subjected to semantic association to obtain semantic association features, and based on the semantic association features, the static features are subjected to semantic reinforcement to obtain semantic reinforcement features, and then based on the semantic reinforcement features, a workpiece defect analysis result is output. In the present application, at the data level, the one-sidedness of a single data source is overcome by fusing dynamic process data and static detection data; secondly, at the information processing level, deep-level and semantic-level information complementation and synergistic enhancement between multi-modal data are realized through semantic encoding, association and reinforcement, rather than simple data stacking, so that the system can comprehensively consider the static performance and dynamic evolution of defects, thereby realizing more comprehensive and more accurate identification and evaluation of workpiece defects, reducing the risk of missed detection and misjudgment, and further enhancing the early warning ability of early defects and potential failure risks, thereby improving the intelligent level and reliability of industrial quality control.

[0040] As a further embodiment of the method, the step of respectively subjecting the infrared image data, the acoustic emission data and the static data to semantic encoding to obtain the infrared image features, the acoustic emission features and the static features comprises: Step S21, loading the infrared image data, the acoustic emission data and the static data into a workpiece defect analysis network, wherein the workpiece defect analysis network includes a semantic encoding sub-network, and the semantic encoding sub-network includes first, second and third semantic encoding units with different network structures.

[0041] Step S22, first semantic encoding of the infrared image data by the first semantic encoding unit to obtain the infrared image features.

[0042] Step S23, second semantic encoding of the acoustic emission data by the second semantic encoding unit to obtain the acoustic emission features.

[0043] Step S24, third semantic encoding of the static data by the third semantic encoding unit to obtain the static features.

[0044] It should be noted that from step S21 to step S24, in view of the essential differences in data structure, physical meaning and defect information mode implied by the infrared image data, acoustic emission data and static data, the present scheme adopts a special coding unit strategy with non-shared weight. By independently designing and deploying the first, second and third semantic coding units with different network structures for each data type, it can ensure that each encoder can deeply mine the most discriminative semantic information in its corresponding data with the most suitable architecture. This clear division of labor design aims to fully leverage the unique advantages of each modal data and provide high-quality and high-purity feature basis for subsequent cross-modal information fusion, thereby ensuring the accuracy and effectiveness of the entire analysis process at the source.

[0045] In the above embodiment, in order to realize the extraction of the infrared image feature, the acoustic emission feature and the static feature, the infrared image data, the acoustic emission data and the static data are loaded into the workpiece defect analysis network, wherein the workpiece defect analysis network comprises a semantic coding sub-network, the semantic coding sub-network comprises a first semantic coding unit, a second semantic coding unit and a third semantic coding unit which have different network structures, then the infrared image data is subjected to first semantic coding by the first semantic coding unit to obtain the infrared image feature, then the acoustic emission data is subjected to second semantic coding by the second semantic coding unit to obtain the acoustic emission feature, and then the static data is subjected to third semantic coding by the third semantic coding unit to obtain the static feature.

[0046] As a further embodiment of the method, the step of subjecting the infrared image data to first semantic coding to obtain the infrared image feature comprises: Step S31, performing multi-level convolution processing on the infrared image data to obtain an infrared multi-scale feature map, wherein the multi-level convolution processing comprises performing parallel convolution processing on the same infrared image using convolution kernels of different sizes.

[0047] Step S32, performing channel attention weighting on the infrared multi-scale feature map to obtain an infrared channel weighted feature, wherein the channel attention weighting is used to enhance the weight of the feature channel related to thermal damage.

[0048] Step S33, performing spatial self-attention mining on the infrared channel weighted feature to obtain an infrared self-attention feature, wherein the spatial self-attention mining is used to establish semantic association relationships between different temperature regions.

[0049] Step S34, updating the first reference feature vector, wherein a first dynamic feature library is initialized or updated, the first dynamic feature library is used to store the infrared self-attention features corresponding to the workpieces identified as qualified in other production batches, the capacity of the dynamic feature library is fixed, and the first reference feature vector is obtained by averaging the first dynamic feature library.

[0050] It should be noted that the updating of the first dynamic feature library is a dynamic rolling process following the principle of "first-in first-out". Specifically, when a new qualified workpiece is confirmed, the system adds the extracted infrared self-attention feature thereof as a new data entry to the end of the dynamic feature library. At the same time, the system checks whether the current capacity of the feature library has reached the preset fixed upper limit. If the upper limit is exceeded, the system automatically removes or covers the historical feature entry with the longest time in the library, i.e., the earliest stored one. Through this mechanism, the feature library always maintains the latest and representative qualified sample set. Subsequently, the first reference feature vector is regenerated by immediately averaging all features in the updated feature library. This process ensures that the constructed qualified benchmark is not a static historical snapshot, but a dynamic standard that can absorb the latest production data and adapt to the slow drift of production conditions, thus significantly improving the adaptability and accuracy of the defect detection system in long-term operation.

[0051] In step S35, a first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector is calculated, and the infrared self-attention feature is attention-weighted based on the first difference vector to generate an infrared image feature.

[0052] It should be noted that from step S31 to step S35, deep semantic information related to thermal damage is mined from the infrared image data. Specifically, through multi-level convolution processing, diversified temperature field features extracted by different size convolution kernels are captured in parallel to form an infrared multi-scale feature map to meet the detection needs of defects of different scales. Then, a channel attention weighting mechanism is used to adaptively enhance the weights of feature channels that are strongly related to thermal damage phenomena, effectively focusing on key information. On this basis, long-range semantic dependency relationships between different temperature regions in the image are established through spatial self-attention to understand the correlation between the overall thermal distribution pattern and the local abnormal region, generating an infrared self-attention feature. To introduce an adaptive benchmark, a first dynamic feature library composed of qualified workpiece features is dynamically maintained, and a first reference feature vector representing the normal state is obtained through averaging operation. Finally, by calculating the difference vector between the current workpiece feature and the reference vector, and using the difference to perform secondary weighting on the self-attention feature, the model can significantly strengthen the abnormal feature response deviating from the normal pattern, thereby generating an infrared image feature extremely sensitive to defects, thereby realizing step-by-step refinement and enhancement from the original image to high-level semantic features with strong discriminability.

[0053] In the above embodiment, in order to obtain the infrared image feature, the infrared image data is subjected to multi-level convolution processing to obtain an infrared multi-scale feature map, wherein the multi-level convolution processing includes parallel convolution processing of the same infrared image using convolution kernels of different sizes, then channel attention weighting is performed on the infrared multi-scale feature map to obtain an infrared channel weighted feature, wherein the channel attention weighting is used to enhance the weight of the feature channel related to thermal damage, then spatial self-attention mining is performed on the infrared channel weighted feature to obtain an infrared self-attention feature, wherein the spatial self-attention mining is used to establish the semantic association relationship between different temperature regions, then the first reference feature vector is updated, wherein the first dynamic feature library is initialized or updated, the first dynamic feature library is used to store the infrared self-attention features corresponding to the workpieces in other production batches that have been identified as qualified, the capacity of the dynamic feature library is fixed, and the first reference feature vector is obtained by averaging the first dynamic feature library, then the first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector is calculated, and the infrared self-attention feature is subjected to attention weighting based on the first difference vector to generate the infrared image feature.

[0054] As a further embodiment of the method, the step of performing second semantic encoding on the acoustic emission data to obtain the acoustic emission feature includes: In step S41, time-frequency transformation processing is performed on the acoustic emission data to obtain an acoustic emission time-frequency spectrum, wherein the time-frequency transformation processing is used to convert the acoustic emission data from time domain representation to time-frequency domain representation to capture the timing characteristics and frequency characteristics of the acoustic emission events.

[0055] In step S42, two-dimensional convolution processing is performed on the acoustic emission time-frequency spectrum to obtain an acoustic emission convolution feature, wherein the two-dimensional convolution processing is used to extract the waveform pattern and frequency distribution characteristics in the acoustic emission events.

[0056] In step S43, time sequence attention weighting is performed on the acoustic emission convolution feature to obtain an acoustic emission time sequence weighted feature, wherein the time sequence attention weighting is used to enhance the weight of the acoustic emission events related to thermal damage.

[0057] In step S44, frequency attention weighting is performed on the acoustic emission time sequence weighted feature to obtain an acoustic emission frequency domain weighted feature, wherein the frequency attention weighting is used to enhance the weight of the feature frequency components related to material damage.

[0058] In step S45, a second reference feature vector is updated, wherein the second dynamic feature library is initialized or updated, the second dynamic feature library is used to store the acoustic emission frequency domain weighted features corresponding to the workpieces in other production batches that have been identified as qualified, the capacity of the second dynamic feature library is fixed, and the second reference feature vector is obtained by averaging the second dynamic feature library.

[0059] It should be noted that the working mechanism of the second dynamic feature library is consistent with that of the first dynamic feature library, and reference can be made to step S34.

[0060] In step S46, a second difference vector between the acoustic emission frequency domain weighted feature of the current workpiece and the second reference feature vector is calculated, and the acoustic emission frequency domain weighted feature is attention weighted based on the second difference vector to generate an acoustic emission feature.

[0061] It should be noted that from step S41 to step S46, the one-dimensional time sequence signal is converted into a two-dimensional time-frequency spectrum containing rich time-frequency information through time-frequency transformation, so that the time sequence evolution law and frequency component characteristics of the signal are retained at the same time; then, the feature extraction stage is entered, and the two-dimensional convolution processing is used to automatically learn and extract the typical waveform mode and frequency distribution feature related to material damage from the time-frequency spectrum; on this basis, the process introduces a double attention enhancement mechanism (steps S43 and S44): first, through time sequence attention weighting, the acoustic emission events associated with the key time points of the thermal damage process are focused, and irrelevant background noise is suppressed, and then through frequency attention weighting, the contribution of those feature frequency components closely related to the damage mechanism such as crack generation and expansion in the sense of material science is further amplified; finally, in order to realize adaptive anomaly detection, the process sets up a dynamic reference adjustment and feature generation link (steps S45 and S46). By maintaining a second dynamic feature library composed of qualified workpiece features, which is updated regularly, and calculating the average value to obtain a second reference feature vector, the system establishes a dynamic reference representing normal acoustic behavior, and by calculating the second difference vector between the current feature and this reference, and using the difference to finally attention weight the feature, the generated acoustic emission feature can sensitively highlight the abnormal acoustic activity deviating from the normal mode, thereby providing highly sensitive and discriminative input features for defect recognition.

[0062] In the above embodiment, in order to obtain the acoustic emission feature, the acoustic emission data is subjected to time-frequency transformation processing to obtain an acoustic emission time-frequency spectrum, wherein the time-frequency transformation processing is used to convert the acoustic emission data from a time domain representation to a time-frequency domain representation to capture the timing characteristics and frequency characteristics of the acoustic emission events, then the acoustic emission time-frequency spectrum is subjected to two-dimensional convolution processing to obtain acoustic emission convolution features, wherein the two-dimensional convolution processing is used to extract waveform patterns and frequency distribution characteristics in the acoustic emission events, then the acoustic emission convolution features are subjected to timing attention weighting to obtain acoustic emission timing weighted features, wherein the timing attention weighting is used to enhance the weight of the acoustic emission events related to thermal damage, then the acoustic emission timing weighted features are subjected to frequency attention weighting to obtain acoustic emission frequency domain weighted features, wherein the frequency attention weighting is used to enhance the weight of the characteristic frequency components related to material damage, then the second reference feature vector is updated, wherein the second dynamic feature library is initialized or updated, the second dynamic feature library is used to store acoustic emission frequency domain weighted features corresponding to workpieces in other production batches that have been identified as qualified, the capacity of the second dynamic feature library is fixed, and the second reference feature vector is obtained by averaging the second dynamic feature library, then a second difference vector between the acoustic emission frequency domain weighted features of the current workpiece and the second reference feature vector is calculated, and the acoustic emission frequency domain weighted features are subjected to attention weighting based on the second difference vector to generate the acoustic emission features.

[0063] As a further embodiment of the method, when the static data is ultrasonic detection data and X-ray image data, the step of performing third semantic encoding on the static data by the third semantic encoding unit to obtain the static feature comprises: Step S51, performing multi-modal feature alignment processing on the static data to obtain static alignment features, wherein the multi-modal feature alignment processing is used to map the ultrasonic detection data and the X-ray image data to a unified feature space.

[0064] Step S52, performing convolution processing on the static alignment features to obtain static convolution features, wherein the convolution processing is used to enhance the structural features related to internal defects.

[0065] Step S53, performing cross-modal attention weighting on the static convolution features to obtain static weighted features, wherein the cross-modal attention weighting calculates the mutual enhancement weights between different modal features through a cross-attention mechanism to realize the complementary correlation between the ultrasonic features and the X-ray features.

[0066] Step S54, performing defect instantiation on the static weighted features to obtain a defect entity set, wherein the defect instantiation processing is used to identify defect individuals and extract the geometric parameters of the defect individuals.

[0067] Step S55, constructing a defect topology graph based on the defect entity set through a graph neural network.

[0068] In step S56, graph feature extraction is performed on the defect topology graph to obtain static features.

[0069] It should be noted that from step S51 to step S56, a fine and structured processing flow for static data is constructed, aiming to deeply mine and correlate the defect information inside the workpiece. The flow starts with feature alignment (step S51), which maps heterogeneous ultrasonic detection data and X-ray image data to a unified feature space through multi-modal feature alignment processing; then feature enhancement (step S52) is performed to strengthen the structural features related to internal defects through convolution processing; on this basis, cross-modal attention weighting (step S53) is used to explore the complementarity between different modal features, and the cross-attention mechanism is used to calculate the mutual enhancement weight between ultrasonic features and X-ray features, to realize information complementarity and semantic correlation; then, the flow enters the defect structuring stage (steps S54 and S55), in which each independent defect individual is accurately identified and parameterized through defect instantiation processing to form a defect entity set, and then based on this set, a defect topology graph is constructed using a graph neural network to model the spatial position and interaction relationship between different defects; finally, graph feature extraction (step S56) is performed on the defect topology graph to convert the abstract defect distribution and correlation pattern into highly structured static features that can be directly utilized by subsequent tasks, thereby realizing comprehensive and deep representation of the internal defect state of the workpiece from individual attributes to overall layout.

[0070] In the above embodiment, in order to obtain the static features, the multi-modal feature alignment processing is performed on the static data to obtain the static alignment features, wherein the multi-modal feature alignment processing is used to map the ultrasonic detection data and the X-ray image data to a unified feature space, then the convolution processing is performed on the static alignment features to obtain the static convolution features, wherein the convolution processing is used to enhance the structural features related to the internal defects, then the cross-modal attention weighting is performed on the static convolution features to obtain the static weighted features, wherein the cross-modal attention weighting calculates the mutual enhancement weight between different modal features through the cross-attention mechanism to realize the complementary correlation between the ultrasonic features and the X-ray features, then the defect instantiation is performed on the static weighted features to obtain the defect entity set, wherein the defect instantiation processing is used to identify the defect individuals and extract the geometric parameters of the defect individuals, then based on the defect entity set, the defect topology graph is constructed through the graph neural network, then the graph feature extraction is performed on the defect topology graph to obtain the static features.

[0071] As a further embodiment of the method, the step of performing semantic correlation on the infrared image features and the acoustic emission features to obtain semantic correlation features includes: Step S61, load the infrared image features and acoustic emission features into a semantic association unit of the workpiece defect analysis network, wherein the semantic association unit is built-in with a first learnable semantic space conversion matrix and a second learnable semantic space conversion matrix.

[0072] Step S62, perform semantic space conversion on the infrared image features by the first semantic space conversion matrix to obtain first converted features.

[0073] Step S63, perform semantic space conversion on the acoustic emission features by the second semantic space conversion matrix to obtain second converted features, wherein the first converted features and the second converted features are in the same semantic space.

[0074] Step S64, perform attention weighting on the second converted features based on the first converted features to obtain first weighted features, wherein the first weighted features are used to enhance the corresponding thermal abnormal region in the infrared features by using the damage time sequence information in the acoustic emission features.

[0075] Step S65, perform attention weighting on the first converted features based on the second converted features to obtain second weighted features, wherein the second weighted features are used to enhance the corresponding damage event in the acoustic emission features by using the temperature distribution information in the infrared features.

[0076] Step S66, fuse the first weighted features and the second weighted features to obtain semantic association features.

[0077] It should be noted that from step S61 to step S66, through the two learnable semantic space conversion matrices, the infrared image features and the acoustic emission features originating from different physical domains are mapped into a unified, comparable public semantic space, followed by bidirectional attention interaction (steps S64 and S65), which is not a simple feature splicing, but performs two times of attention weighting in different directions: first, the damage time sequence information (such as stress wave events at a specific time) in the acoustic emission features is used as a guide to enhance the saliency of the corresponding thermal abnormal region in the infrared features; in turn, the spatial temperature distribution information in the infrared features is used as a context to strengthen the weight of those damage events in the acoustic emission features that are associated with the thermal abnormal region in time, and this bidirectional interaction enables the two modalities to confirm and enhance each other. Finally, through feature fusion (step S66), the two weighted features after bidirectional enhancement are integrated to generate unified semantic association features, which not only retain the original information of each modality, but also deeply embed the strong association semantics between “where is hot” and “when is sound”, thereby constructing a more sensitive joint representation to thermal-induced damage.

[0078] In the above embodiment, in order to obtain the semantic correlation feature, the infrared image feature and the acoustic emission feature are loaded into a semantic correlation unit of the workpiece defect analysis network, wherein the semantic correlation unit is internally provided with a first semantic space conversion matrix and a second semantic space conversion matrix which can be learned, then the infrared image feature is subjected to semantic space conversion through the first semantic space conversion matrix to obtain a first conversion feature, and then the acoustic emission feature is subjected to semantic space conversion through the second semantic space conversion matrix to obtain a second conversion feature, wherein the first conversion feature and the second conversion feature are in the same semantic space, then the second conversion feature is subjected to attention weighting based on the first conversion feature to obtain a first weighted feature, wherein the first weighted feature is used to enhance the corresponding thermal anomaly area in the infrared feature by using the damage time sequence information in the acoustic emission feature, then the first conversion feature is subjected to attention weighting based on the second conversion feature to obtain a second weighted feature, wherein the second weighted feature is used to enhance the corresponding damage event in the acoustic emission feature by using the temperature distribution information in the infrared feature, and then the first weighted feature and the second weighted feature are fused to obtain the semantic correlation feature.

[0079] As a further embodiment of the method, the step of performing semantic enhancement on the static feature based on the semantic correlation feature to obtain a semantic enhancement feature comprises: Step S71, loading the semantic correlation feature and the static feature into a semantic enhancement unit of the workpiece defect analysis network, wherein the semantic enhancement unit at least includes a first semantic enhancement subunit and a second semantic enhancement subunit.

[0080] Step S72, performing first semantic enhancement on the static feature based on the semantic correlation feature through the first semantic enhancement subunit to obtain a first semantic enhancement feature.

[0081] Step S73, performing second semantic enhancement on the static feature based on the semantic correlation feature through the second semantic enhancement subunit to obtain a second semantic enhancement feature.

[0082] Step S74, performing mean value calculation on the first semantic enhancement feature and the second semantic enhancement feature to obtain a semantic enhancement feature.

[0083] It should be noted that from step S71 to step S74, a double reinforcement and fusion architecture is adopted: first, through two independent semantic reinforcement sub-units, the static features are subjected to two different semantic enhancements based on the unified semantic correlation features (which fuse dynamic infrared and acoustic emission information) respectively, to generate first and second semantic reinforcement features. This parallel processing design enables the model to learn from different dimensions how to utilize dynamic process information to supplement and enrich static detection data; subsequently, the two reinforcement results are averaged to obtain the final semantic reinforcement features. This fusion strategy not only integrates the advantages of different reinforcement paths, improves the richness of the features, but also effectively improves the stability and robustness of the final features through ensemble averaging, ensuring that the enhanced static features can more comprehensively and reliably reflect the true state of the workpiece.

[0084] In the above embodiment, in order to obtain the semantic reinforcement features, the semantic correlation features and the static features are loaded into the semantic reinforcement unit of the workpiece defect analysis network, wherein the semantic reinforcement unit at least includes a first semantic reinforcement sub-unit and a second semantic reinforcement sub-unit, then the first semantic reinforcement sub-unit is used to perform first semantic reinforcement on the static features based on the semantic correlation features to obtain first semantic reinforcement features, and then the second semantic reinforcement sub-unit is used to perform second semantic reinforcement on the static features based on the semantic correlation features to obtain second semantic reinforcement features, and then the first semantic reinforcement features and the second semantic reinforcement features are averaged to obtain the semantic reinforcement features.

[0085] As a further embodiment of the method, the step of performing first semantic reinforcement on the static features based on the semantic correlation features by the first semantic reinforcement sub-unit to obtain first semantic reinforcement features includes: Step S81, performing semantic space conversion on the semantic correlation features by the third semantic space conversion matrix built in the first semantic reinforcement sub-unit to obtain third conversion features, wherein the third semantic space conversion matrix is used to convert the semantic correlation features to the semantic space where the static features are located.

[0086] Step S82, determining the parameter mapping relationship between the third conversion features and the static features, and determining the influence weight distribution of the semantic correlation features on the static features based on the parameter mapping relationship, wherein the influence weight distribution is used to quantify the influence degree of the semantic correlation features on the static features.

[0087] Step S83, weighting the static features based on the influence weight distribution to obtain the first semantic reinforcement features.

[0088] It should be noted that from step S81 to step S83, the working mechanism of the first semantic enhancement subunit is specifically described, and the core is to realize the accurate injection of cross-domain information through a guided weighting. The process starts with semantic space alignment (step S81), which uses a third semantic space conversion matrix to project the semantic correlation features representing the dynamic process into the semantic space where the static features are located, generating third conversion features. This conversion ensures that features from different domains can be directly compared and interacted in a unified context; then, the system analyzes and quantizes the influence of dynamic information through a learnable mapping function (step S82), accurately calculates the parameter mapping relationship between the third conversion features and the original static features, and generates a fine-grained influence weight distribution based on this, which quantifies the potential influence degree of dynamic process information (such as thermal-acoustic correlation events) on each component (i.e. different internal structure attributes) in the static feature; finally, in the feature enhancement execution stage (step S83), the system weights the original static features element by element according to the weight distribution, thereby generating the first semantic enhancement features. The essence of this operation is to use the clues revealed by dynamic monitoring to selectively amplify the components in the static feature that are more likely to be affected by the thermal-acoustic process and are related to potential damage, while suppressing irrelevant or interfering components, so that the final feature can more sensitively highlight the defect patterns induced or exacerbated by the dynamic process, significantly improving the sensitivity and accuracy of defect recognition.

[0089] It should be further pointed out that the mechanism of the second semantic enhancement feature is basically the same as that of the first semantic enhancement feature.

[0090] In the above embodiment, in order to obtain the first semantic enhancement feature, the semantic correlation features are converted by the third semantic space conversion matrix built in the first semantic enhancement subunit to obtain the third conversion features, wherein the third semantic space conversion matrix is used to convert the semantic correlation features to the semantic space where the static features are located, then the parameter mapping relationship between the third conversion features and the static features is determined, and based on the parameter mapping relationship, the influence weight distribution of the semantic correlation features on the static features is determined, wherein the influence weight distribution is used to quantify the influence degree of the semantic correlation features on the static features, and then the static features are weighted based on the influence weight distribution to obtain the first semantic enhancement features.

[0091] The application also discloses an industrial Internet of Things-based sensor data analysis system.

[0092] Reference Figure 2 The industrial Internet of Things-based sensor data analysis system comprises a management platform, a sensor network platform and an object platform which are sequentially connected in communication, and the management platform is configured to have: The data acquisition module is configured to acquire, by the sensor, infrared image data and acoustic emission data corresponding to each workpiece in the same batch in a continuously changing temperature field, and acquire static data of the workpiece in a normal temperature state, wherein the static data includes at least one of ultrasonic detection data and X-ray image data. The single semantic encoding module is configured to perform semantic encoding on the infrared image data, the acoustic emission data and the static data respectively based on a workpiece defect analysis network, to obtain infrared image features, acoustic emission features and static features. The multi-semantics fusion module is configured to perform semantic association on the infrared image features and the acoustic emission features to obtain semantic association features, and perform semantic reinforcement on the static features based on the semantic association features to obtain semantic reinforcement features. The analysis module is configured to output a workpiece defect analysis result based on the semantic reinforcement features.

[0093] Another application scenario of the sensor network monitoring system based on the industrial Internet of Things has a whole framework as shown in the figure. Figure 3 The service platform is composed of a service total database, a plurality of service sub-platforms and a plurality of service sub-databases; the management platform includes an associated characteristic value generation module, a characteristic construction module, a characteristic decomposition module, a patrol quality index generation module and a patrol personnel distribution module, and the management platform can interact with the sensing network platform and the service platform; the sensing network platform can include a sensing total database, a plurality of sensing network sub-platforms and a plurality of sensing sub-databases, in this embodiment, n sensing network sub-platforms and n sensing sub-databases, each sensing network sub-platform is provided with a corresponding sensing sub-database, and the sensing network platform can interact with the object platform.

[0094] Through the interaction between the various functional platforms of the sensor network monitoring system based on the industrial Internet of Things and the three platforms or the five platforms, a perfect closed-loop information operation logic is established, the ordered operation of the sensing information and the control information is ensured, and the intelligent management of the equipment is realized.

[0095] Specifically, the sensor network monitoring system based on the industrial Internet of Things in the embodiment comprises a management platform, and the management platform is configured to: acquire, by a sensor, infrared image data and acoustic emission data corresponding to each workpiece in a same batch in a continuously changing temperature field, and acquire static data of the workpiece in a normal temperature state, wherein the static data comprises at least one of ultrasonic detection data and X-ray image data; based on a workpiece defect analysis network, perform semantic coding on the infrared image data, the acoustic emission data and the static data respectively to obtain infrared image features, acoustic emission features and static features; perform semantic association on the infrared image features and the acoustic emission features to obtain semantic association features; and based on the semantic association features, perform semantic reinforcement on the static features to obtain semantic reinforcement features; and based on the semantic reinforcement features, output a workpiece defect analysis result.

[0096] The sensor data analysis system based on the industrial Internet of Things can implement any one of the sensor data analysis methods based on the industrial Internet of Things, and the specific working process of the sensor data analysis system based on the industrial Internet of Things can refer to the corresponding process in the above sensor data analysis methods based on the industrial Internet of Things.

[0097] The embodiment of the present application further discloses a computer device.

[0098] Reference Figure 4 A computer device comprises a memory and a processor, the memory stores a computer program capable of running on the processor, and the processor implements any one of the above sensor data analysis methods based on the industrial Internet of Things when executing the computer program.

[0099] The embodiment of the present application further discloses a computer readable storage medium.

[0100] A computer readable storage medium stores a computer program capable of being loaded by a processor and executing any one of the above sensor data analysis methods based on the industrial Internet of Things.

[0101] The computer readable storage medium can be any tangible medium containing or storing a program, which can be used or combined with an instruction execution system, device or apparatus; the program code contained on the computer readable medium can be transmitted by any appropriate medium, including but not limited to wireless, wire, optical cable, RF, etc., or any appropriate combination of the above.

[0102] The above are preferred embodiments of the present application, and are not intended to limit the protection scope of the present application, any feature disclosed in the specification (including the abstract and the drawings) can be replaced by other equivalent or similar purpose replacement features, unless specifically described. That is, unless specifically described, each feature is only an example of a series of equivalent or similar features.

Claims

1. A method for analyzing sensor data based on an industrial internet of things, characterized in that, The method is applied to an industrial Internet of Things system, the industrial Internet of Things system comprising a management platform, a sensor network platform and an object platform which are sequentially communicatively connected, the method being executed by the management platform and comprising: Obtaining, by a sensor, infrared image data and acoustic emission data corresponding to each workpiece in a same batch in a continuously changing temperature field, and obtaining static data of the workpiece in a normal temperature state, wherein the static data comprises at least one of ultrasonic detection data and X-ray image data; Respectively performing semantic encoding on the infrared image data, the acoustic emission data and the static data based on a workpiece defect analysis network to obtain infrared image features, acoustic emission features and static features; Performing semantic association on the infrared image features and the acoustic emission features to obtain semantic association features, and performing semantic reinforcement on the static features based on the semantic association features to obtain semantic reinforcement features; Outputting a workpiece defect analysis result based on the semantic reinforcement features. 2.The industrial Internet of things based sensor data analysis method according to claim 1, characterized in that, The step of respectively performing semantic encoding on the infrared image data, the acoustic emission data and the static data to obtain infrared image features, acoustic emission features and static features comprises: Loading the infrared image data, the acoustic emission data and the static data into a workpiece defect analysis network, wherein the workpiece defect analysis network comprises a semantic encoding subnetwork, and the semantic encoding subnetwork comprises first, second and third semantic encoding units which have different network structures; Performing first semantic encoding on the infrared image data by the first semantic encoding unit to obtain infrared image features; Performing second semantic encoding on the acoustic emission data by the second semantic encoding unit to obtain acoustic emission features; Performing third semantic encoding on the static data by the third semantic encoding unit to obtain static features. 3.The industrial Internet of things based sensor data analysis method according to claim 2, characterized in that, The step of performing first semantic encoding on the infrared image data to obtain infrared image features comprises: Performing multi-level convolution processing on the infrared image data to obtain infrared multi-scale feature maps, wherein the multi-level convolution processing comprises performing parallel convolution processing on the same infrared image using convolution kernels of different sizes; Performing channel attention weighting on the infrared multi-scale feature maps to obtain infrared channel weighted features, wherein the channel attention weighting is used to enhance the weight of a feature channel related to thermal damage; Performing spatial self-attention mining on the infrared channel weighted features to obtain infrared self-attention features, wherein the spatial self-attention mining is used to establish a semantic association relationship between different temperature regions; Updating a first reference feature vector, wherein a first dynamic feature library is initialized or updated, the first dynamic feature library being used to store infrared self-attention features of workpieces in other production batches which have been identified as qualified, the capacity of the dynamic feature library being fixed, and the first reference feature vector being obtained by averaging the first dynamic feature library; The first difference vector between the infrared self-attention feature of the current workpiece and the first reference feature vector is calculated, and the infrared self-attention feature is attention weighted based on the first difference vector to generate an infrared image feature. 4.The industrial Internet of things based sensor data analysis method according to claim 2, wherein, The step of performing second semantic encoding on the acoustic emission data to obtain acoustic emission features comprises: performing time-frequency transformation processing on the acoustic emission data to obtain an acoustic emission time-frequency spectrum, wherein the time-frequency transformation processing is used to convert acoustic emission data from time domain representation to time-frequency domain representation to capture the timing characteristics and frequency characteristics of acoustic emission events; performing two-dimensional convolution processing on the acoustic emission time-frequency spectrum to obtain acoustic emission convolution features, wherein the two-dimensional convolution processing is used to extract waveform patterns and frequency distribution characteristics in acoustic emission events; performing time sequence attention weighting on the acoustic emission convolution features to obtain acoustic emission time sequence weighted features, wherein the time sequence attention weighting is used to enhance the weight of acoustic emission events related to thermal damage; performing frequency attention weighting on the acoustic emission time sequence weighted features to obtain acoustic emission frequency domain weighted features, wherein the frequency attention weighting is used to enhance the weight of feature frequency components related to material damage; updating the second reference feature vector, wherein a second dynamic feature library is initialized or updated, the second dynamic feature library is used to store acoustic emission frequency domain weighted features corresponding to workpieces identified as qualified in other production batches, the capacity of the second dynamic feature library is fixed, and the second reference feature vector is obtained by averaging the second dynamic feature library; calculating a second difference vector between the acoustic emission frequency domain weighted features of the current workpiece and the second reference feature vector, and performing attention weighting on the acoustic emission frequency domain weighted features based on the second difference vector to generate acoustic emission features. 5.The industrial Internet of things based sensor data analysis method according to claim 2, wherein, When the static data is the ultrasonic detection data and the X-ray image data, the step of performing third semantic encoding on the static data by the third semantic encoding unit to obtain static features comprises: performing multi-modal feature alignment processing on the static data to obtain static aligned features, wherein the multi-modal feature alignment processing is used to map ultrasonic detection data and X-ray image data to a unified feature space; performing convolution processing on the static aligned features to obtain static convolution features, wherein the convolution processing is used to enhance structure features related to internal defects; performing cross-modal attention weighting on the static convolution features to obtain static weighted features, wherein the cross-modal attention weighting calculates mutual enhancement weights between different modal features through a cross-attention mechanism to realize complementary correlation between ultrasonic features and X-ray features; performing defect instantiation on the static weighted features to obtain a defect entity set, wherein the defect instantiation processing is used to identify defect individuals and extract geometric parameters of the defect individuals; constructing a defect topology graph based on the defect entity set through a graph neural network; performing graph feature extraction on the defect topology graph to obtain static features. 6.The industrial Internet of things based sensor data analysis method according to claim 1, wherein, The step of performing semantic correlation on the infrared image features and the acoustic emission features to obtain semantic correlation features comprises: loading the infrared image features and the acoustic emission features into a semantic correlation unit of the workpiece defect analysis network, wherein the semantic correlation unit is internally provided with learnable first and second semantic space conversion matrices; performing semantic space conversion on the infrared image features by using the first semantic space conversion matrix to obtain first conversion features; performing semantic space conversion on the acoustic emission features by using the second semantic space conversion matrix to obtain second conversion features, wherein the first conversion features and the second conversion features are in the same semantic space; performing attention weighting on the second conversion features based on the first conversion features to obtain first weighted features, wherein the first weighted features are used to enhance corresponding thermal anomaly regions in the infrared features by using damage time sequence information in the acoustic emission features; performing attention weighting on the first conversion features based on the second conversion features to obtain second weighted features, wherein the second weighted features are used to enhance corresponding damage events in the acoustic emission features by using temperature distribution information in the infrared features; fusing the first weighted features and the second weighted features to obtain semantic correlation features. 7.The industrial Internet of things based sensor data analysis method according to claim 1, wherein, The step of performing semantic reinforcement on the static features based on the semantic correlation features to obtain semantic reinforcement features comprises: loading the semantic correlation features and the static features into a semantic reinforcement unit of the workpiece defect analysis network, wherein the semantic reinforcement unit at least includes first and second semantic reinforcement sub-units; performing first semantic reinforcement on the static features based on the semantic correlation features by using the first semantic reinforcement sub-unit to obtain first semantic reinforcement features; performing second semantic reinforcement on the static features based on the semantic correlation features by using the second semantic reinforcement sub-unit to obtain second semantic reinforcement features; performing mean value calculation on the first semantic reinforcement features and the second semantic reinforcement features to obtain semantic reinforcement features. 8.The industrial Internet of things based sensor data analysis method according to claim 7, characterized in that, The step of performing first semantic reinforcement on the static features based on the semantic correlation features by using the first semantic reinforcement sub-unit to obtain first semantic reinforcement features comprises: performing semantic space conversion on the semantic correlation features by using a third semantic space conversion matrix internally provided in the first semantic reinforcement sub-unit to obtain third conversion features, wherein the third semantic space conversion matrix is used to convert the semantic correlation features to a semantic space in which the static features are located; determining a parameter mapping relationship between the third conversion features and the static features, and determining an influence weight distribution of the semantic correlation features on the static features based on the parameter mapping relationship, wherein the influence weight distribution is used to quantify an influence degree of the semantic correlation features on the static features; performing weighting on the static features based on the influence weight distribution to obtain first semantic reinforcement features.

9. An industrial internet of things based sensor data analysis system, characterized in that, The management platform, the sensor network platform and the object platform are sequentially communicatively connected, and the management platform is configured to have: a data acquisition module configured to acquire, by a sensor, infrared image data and acoustic emission data corresponding to each workpiece in a same batch in a continuously changing temperature field, and acquire static data of the workpiece in a normal temperature state, wherein the static data includes at least one of ultrasonic detection data and X-ray image data; a single semantic encoding module configured to perform semantic encoding on the infrared image data, the acoustic emission data and the static data respectively based on a workpiece defect analysis network to obtain infrared image features, acoustic emission features and static features; a multi semantic fusion module configured to perform semantic association on the infrared image features and the acoustic emission features to obtain semantic association features, and perform semantic reinforcement on the static features based on the semantic association features to obtain semantic reinforcement features; an analysis module configured to output a workpiece defect analysis result based on the semantic reinforcement features.

10. A computer device, comprising: The memory and the processor, the memory has the computer program which can run on the processor, the processor executes the computer program and realizes the method in any one of claims 1 to 8.

Images