An industrial control data acquisition and monitoring platform driven by intelligent algorithms

Abstract

The present invention belongs to the technical field of data acquisition. The present invention discloses an industrial control data acquisition and monitoring platform driven by intelligent algorithms, including a visual dynamic focusing layer, which obtains multimodal visual information based on the fovea centralis mechanism of the retina, applies a feature pyramid network to the multimodal visual information to generate an external health heat map of the device, and divides the risk areas of the external health heat map of the device; a bionic ultrasonic positioning and mapping layer, which deploys a MEMS ultrasonic array according to the divided risk areas, emits coded pulses to penetrate the device shell, calculates the internal echo signal through the time-of-flight algorithm, and generates a three-dimensional stress distribution map inside the device; a cross-modal data fusion layer, which establishes a spatio-temporal mapping between vision and ultrasound, synchronizes clocks using the PTP protocol, and fuses the external health heat map of the device and the three-dimensional stress distribution map inside the device through a graph attention network to generate a confidence score; improving production efficiency and the overall level of industrial automation.

Description

Technical Field

[0001] The present invention relates to the technical field of data acquisition, and more specifically, to an industrial control data acquisition and monitoring platform driven by intelligent algorithms. Background Art

[0002] The patent with the publication number CN105589356A discloses an industrial data acquisition and monitoring system, which includes a single-chip microcomputer and a data acquisition module, a USB communication interface, an LED display, an input module, and an alarm module that are controlled and connected to the single-chip microcomputer. It also includes an extended memory that is data-connected to the single-chip microcomputer. The industrial data acquisition and monitoring system of the present invention has a simple structure, convenient interfaces, high integration, strong reliability, and rich functions. Secondly, connecting to a PC through a USB interface not only enables a large amount of data storage, but also makes it easier to perform data processing. It is economical and simple, and the sampling data has a relatively high accuracy, and has high practical value in practical applications.

[0003] The existing industrial control data acquisition and monitoring platform driven by intelligent algorithms mainly has the following problems:

[0004] The prior art does not perform differential resolution processing for key areas and non-critical areas, resulting in the average allocation of computing resources and the inability to effectively concentrate resources on the fine monitoring of key parts. The lack of a dynamic resolution adjustment mechanism may cause the device to face computational pressure when processing complex scenarios or high-speed movements, affecting the frame rate and real-time performance. A single sensor may not provide comprehensive visual information, and the lack of fusion of multi-sensor data may lead to the inability to accurately capture abnormal conditions on the device surface in complex working conditions. The lack of intelligent analysis technologies such as spatial attention networks may cause the system to be unable to effectively identify abnormal areas when facing a large amount of data, resulting in omissions or false alarms. The inability to adjust high and low resolution areas in real time means that the system may not be able to respond to newly emerging abnormalities in a timely manner, reducing the sensitivity and accuracy of monitoring. In the face of different working conditions and monitoring requirements, the prior art cannot provide sufficient flexibility and configurability to meet the customized needs of users;

[0005] The prior art does not deploy ultrasonic sensors with different densities according to the risk area division, resulting in unreasonable sensor distribution and inability to efficiently capture key stress information inside the device; the detection effect in different risk areas is not ideal, and the type, frequency, and period of the transmitted signal may not be optimized according to the device characteristics and monitoring requirements, thus limiting the detection depth and accuracy; the ultrasonic detection angle is not dynamically adjusted according to the risk area of the external health heat map of the device, resulting in detection blind spots or repeated detections and reducing the monitoring efficiency. The prior art has inaccurate sound speed calculation, which in turn affects the solution accuracy of stress distribution; the influence of factors such as stress, temperature, and material properties on the sound speed is not considered, so the true stress state inside the device cannot be accurately reflected; full-coverage ultrasonic detection of the entire device requires processing a large amount of data and it is difficult to achieve real-time monitoring. The prior art has poor real-time performance and unreasonable calculation resource allocation, resulting in delayed detection results in high-risk areas and possibly missing key warning opportunities.

[0006] In view of this, the present invention proposes an industrial control data acquisition and monitoring platform driven by an intelligent algorithm to solve the above problems. Summary of the Invention

[0007] To overcome the above defects of the prior art and to achieve the above object, the present invention provides the following technical solution: An industrial control data acquisition and monitoring platform driven by an intelligent algorithm, comprising:

[0008] A visual dynamic focusing layer, which obtains multi-modal visual information based on the fovea centralis mechanism of the retina, applies a feature pyramid network to the multi-modal visual information, generates an external health heat map of the device, and divides the risk area of the external health heat map of the device;

[0009] A bionic ultrasonic positioning and mapping layer, which deploys a MEMS ultrasonic array according to the divided risk area, emits coded pulses to penetrate the device shell, calculates the internal echo signal through the time-of-flight algorithm, and generates a three-dimensional stress distribution map inside the device;

[0010] A cross-modal data fusion layer, which establishes a spatio-temporal mapping between vision and ultrasound, synchronizes clocks using the PTP protocol, and fuses the external health heat map of the device and the three-dimensional stress distribution map inside the device through a graph attention network to generate a confidence score;

[0011] A dynamic control decision layer, which converts the confidence score into an executable industrial control instruction based on a multi-objective optimization algorithm and sends the executable industrial control instruction to the PLC in real time through the MQTT protocol;

[0012] An execution feedback layer, which monitors the device state changes after the PLC executes the industrial control instruction in real time, calculates the deviation between the device state changes and the preset device state change threshold, generates an acquisition strategy effectiveness index, and dynamically optimizes the industrial control data acquisition strategy until the acquisition strategy effectiveness index reaches the optimum.

[0013] Preferably, the method for obtaining multi-modal visual information based on the fovea centralis mechanism of the retina includes:

[0014] Deploy three visual sensors, namely an infrared thermal imager, an industrial camera, and a lidar, on the industrial control data acquisition and monitoring platform to collect multi-modal visual information on the surface of the device; the multi-modal visual information includes temperature distribution data on the surface of the device, surface visual data of the device, and three-dimensional depth data of the surface of the device and its surrounding environment;

[0015] Simulate the non-uniform perception mechanism of the fovea centralis of the human eye, and dynamically adjust the sensor resolution and sampling density; based on the CAD model of the device, mark the welds, bearing positions, and gear meshing points of the device as high-resolution focusing areas; mark the remaining areas on the surface of the device as low-resolution peripheral areas; dynamically define a resolution function to divide the surface of the device into high-resolution focusing areas and low-resolution peripheral areas; the high-resolution focusing areas simulate the fovea centralis area of the human eye, and the low-resolution peripheral areas simulate the peripheral area of the retina;

[0016] Analyze the abnormal features in the multi-modal visual information in real time through a spatial attention network. The input of the spatial attention network is the multi-modal visual information, and the output is the abnormal feature area; dynamically adjust the range of the focusing area according to the abnormal feature area. If a new abnormal feature area is detected, include the new abnormal feature area in the high-resolution focusing area; if no abnormal features are detected in the high-resolution focusing area, reduce the high-resolution focusing area to a low-resolution peripheral area; the high-resolution focusing area uses dense grid sampling, and the low-resolution peripheral area uses sparse grid sampling. Dynamically adjust the sensor parameters through control instructions to obtain the final multi-modal visual information.

[0017] Preferably, the method for applying a feature pyramid network to the multi-modal visual information to generate a thermal map of the external health of the device includes:

[0018] Perform standard deviation normalization on the multi-modal visual information, and stack the normalized multi-modal visual information into a four-dimensional tensor according to the channel dimension. The dimensions of the four-dimensional tensor include height, width, number of channels, and time series frames; construct a feature pyramid network structure, which includes a bottom-up path, a top-down path, and lateral connections;

[0019] Input the multi-modal visual information into the feature pyramid network structure. In the bottom-up path, use a convolutional neural network to extract features from the input multi-modal data, generate feature maps of different scales, and layer by layer extract higher-level semantic information. The feature maps of different scales include low-level feature maps, intermediate feature maps, and high-level feature maps; in the top-down path, gradually restore the spatial resolution of the high-level feature maps through upsampling; establish lateral connections between the bottom-up path and the top-down path to fuse the feature maps of the same scale;

[0020] Apply 1×1 convolution on each output layer of the feature pyramid network structure for feature channel adjustment; fuse the feature maps of different scales output by the feature pyramid network structure to generate an external device health heat map.

[0021] Preferably, the method for dividing risk regions of the external device health heat map includes:

[0022] Calculate the device health status score for each region in the external device health heat map. By performing weighted summation on the temperature distribution data of the device surface, the surface visual data of the device, and the three-dimensional depth data of the device surface and the surrounding environment included in the multi-modal visual information, obtain the device health status score;

[0023] Preset a first threshold for the device health status score and a second threshold for the device health status score. Compare the device health status score with the preset first threshold and second threshold for the device health status score respectively. If the device health status score is less than the first threshold for the device health status score, mark the region corresponding to this score as green; if the device health status score is greater than the first threshold for the device health status score and less than the second threshold for the device health status score, mark the region corresponding to this score as blue; if the device health status score is greater than the second threshold for the device health status score, mark the region corresponding to this score as red;

[0024] Use different colors to divide the risk regions of the external device health heat map. Different colors represent different risk regions. Define the green region as a low-risk region, the blue region as a medium-risk region, and the red region as a high-risk region.

[0025] Preferably, the method for generating a three-dimensional internal stress distribution map of the device includes:

[0026] According to the divided risk regions, deploy ultrasonic sensors with different densities on the surface of the device shell; the ultrasonic sensors are MEMS ultrasonic array sensors. Adopt a differentiated emission strategy for different risk regions. Transmit coded pulses into the device interior through the ultrasonic sensors for initial global detection. Dynamically adjust the ultrasonic detection angle according to the risk regions of the external device health heat map, where the ultrasonic detection angle interval in the high-risk region is less than that in the medium-risk region, and the ultrasonic detection angle interval in the medium-risk region is less than that in the low-risk region; apply an angle adjustment algorithm to dynamically adjust the ultrasonic detection angle, introduce a multi-physical-field-coupled sound speed model, and continuously correct the sound speed parameters in the sound speed model through an iterative time-of-flight algorithm. Stop when the maximum number of iterations is reached; establish an ultrasonic propagation path matrix based on the ray tracing algorithm, solve the stress distribution by the regularized least squares method, and finally generate a three-dimensional internal stress distribution map of the device with risk markings.

[0027] Preferably, the method for establishing the spatio-temporal mapping between vision and ultrasound includes:

[0028] Taking any point on the surface of the industrial equipment as the origin, The axis and The axis is parallel to the surface of the industrial equipment, The axis is perpendicular to the surface of the industrial equipment, and a unified global coordinate system is defined; calibrating the vision sensor using a calibration board to obtain the internal and external parameters of the vision sensor; converting the pixel coordinates of the external health thermal map of the equipment to the camera coordinates through the internal parameters of the vision sensor, and then converting the camera coordinates to the global coordinates through the external parameters of the vision sensor, so as to obtain the external health thermal map of the equipment in the global coordinate system; taking the origin of the global coordinate system as the reference point, calibrating the ultrasound sensor to obtain the external parameters of the ultrasound sensor;

[0029] Converting the internal three-dimensional stress distribution map of the equipment to the global coordinate system through the external parameters of the ultrasound sensor, so as to obtain the internal three-dimensional stress distribution map of the equipment in the global coordinate system; in the global coordinate system, aligning the spatial coordinates of the external health thermal map of the equipment and the internal three-dimensional stress distribution map of the equipment to establish the spatial mapping between vision and ultrasound; setting the time source connected to the vision sensor as the master clock and the time source connected to the ultrasound sensor as the slave clock, and performing time synchronization through the PTP protocol to establish the time mapping between vision and ultrasound.

[0030] Preferably, the method for fusing the external health thermal map of the equipment and the internal three-dimensional stress distribution map through the graph attention network includes:

[0031] According to the external health thermal map of the equipment and the internal three-dimensional stress distribution map in the global coordinate system, constructing an undirected graph, using a multi-layer GAT structure based on the multi-head attention mechanism to build a graph attention network, and fusing the external health thermal map of the equipment and the internal three-dimensional stress distribution map in the global coordinate system; the graph attention network includes an input layer, a feature projection layer, a graph attention layer, a cross-modal interaction layer and an output layer; taking the undirected graph as the input of the input layer of the graph attention network and generating a confidence score through the output layer.

[0032] Preferably, the method for constructing the undirected graph includes:

[0033] Taking each risk area in the external health thermal map of the equipment in the global coordinate system as a vision node, and extracting the feature vector of each risk area as the vision node feature; taking each stress concentration area in the internal three-dimensional stress distribution map of the equipment in the global coordinate system as an ultrasound node, and extracting the three-dimensional voxel feature vector of each stress concentration area as the ultrasound node feature; collecting all vision nodes and ultrasound nodes to obtain a node set;

[0034] Traverse all visual nodes and calculate the Euclidean distance between any two visual nodes in the global coordinate system; preset a visual distance threshold. If the Euclidean distance between any two visual nodes in the global coordinate system is less than the preset visual distance threshold, add a bidirectional edge between the corresponding two visual nodes; if the Euclidean distance between any two visual nodes in the global coordinate system is greater than or equal to the preset visual distance threshold, do not add.

[0035] Traverse all ultrasonic nodes and calculate the Euclidean distance between any two ultrasonic nodes in the global coordinate system; preset an ultrasonic distance threshold. If the Euclidean distance between any two ultrasonic nodes in the global coordinate system is less than the preset ultrasonic distance threshold, add a bidirectional edge between the corresponding two ultrasonic nodes; if the Euclidean distance between any two ultrasonic nodes in the global coordinate system is greater than or equal to the preset ultrasonic distance threshold, do not add.

[0036] Through nearest neighbor search, find the nearest ultrasonic node for each visual node and add a bidirectional edge between the visual node and the corresponding nearest ultrasonic node; collect all bidirectional edges to obtain an edge set; construct an undirected graph based on the obtained node set and edge set.

[0037] Preferably, the method for converting the confidence score into an executable industrial control instruction based on the multi-objective optimization algorithm includes:

[0038] Divide the internal space of the device into small cube units, and each unit records the current stress value and confidence score; at the same time, list all adjustable industrial control data parameters, and the industrial control data parameters include the adjustment range, energy consumption cost, and the strength of the historical corresponding stress impact of each parameter.

[0039] Establish three optimization objectives, including the first safety objective, the second safety objective, and the economic objective; set two types of constraint conditions, including hard constraints and elastic constraints.

[0040] Use the multi-objective optimization algorithm to randomly generate m sets of optimization schemes for industrial control data parameters, evaluate the completion of the three optimization objectives for each scheme, obtain the completion scores of the three optimization objectives, and add the completion scores of the three optimization objectives to obtain a comprehensive score. Select the scheme with the highest comprehensive score from the m sets of schemes as the final optimization scheme; convert the selected final optimization scheme into an actual executable industrial control instruction; the executable industrial control instruction includes the first instruction, the second instruction, and the safety monitoring instruction.

[0041] The method for obtaining the completion scores of the three optimization objectives includes:

[0042] Obtain the completion score of the first safety target by the percentage reduction of the maximum stress value in the observation device; calculate the standard deviation of the confidence scores in different regions as the completion score of the second safety target; calculate the total energy consumption caused by the adjustment of all industrial control data parameters as the completion score of the economic target.

[0043] Preferably, the method for generating the effectiveness index of the acquisition strategy includes:

[0044] Collect the industrial control instruction parameters executed by the PLC, synchronously monitor the changes in the device status, calculate the deviation between the device status change and the preset device status change threshold, and generate a 0-1 standardized acquisition strategy effectiveness index d; preset a first threshold d1 and a second threshold d2 for the standardized acquisition strategy effectiveness index; if d>d2, it is determined that the acquisition strategy is effective; if d1<d<d2, it is determined that the acquisition strategy still needs to be observed; if d<d1, the acquisition strategy is ineffective, optimize the industrial control data acquisition strategy, and immediately generate a warning message.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] By simulating the fovea mechanism of the human eye, the present invention performs high-resolution acquisition in key areas and reduces the resolution in non-critical areas, ensuring high detection accuracy in key parts while reducing the computational burden. The dynamic resolution function is adaptively adjusted according to the importance of different regions on the device surface, rather than a one-size-fits-all approach, improving the effectiveness of the data. The frame rate is dynamically increased to 1200fps, and the visual information acquisition in key areas is more delicate, suitable for fine monitoring of working conditions such as high-speed rotation and vibration. At the same time, the data of the infrared thermal imager, industrial camera, and lidar are fused to obtain temperature, visible light texture, and 3D depth information, forming a more complete visual information and avoiding the limitations of a single sensor. The spatial attention network intelligently analyzes abnormal areas. When temperature mutations, abnormal deformation gradients, etc. occur on the device surface, the system will automatically focus on these areas to reduce omissions. The high and low resolution areas are adjusted in real time. If new abnormalities are found, the resolution is automatically increased; if a certain area has no abnormalities for a long time, it is reduced to low resolution to reduce the data acquisition pressure.

[0047] By deploying ultrasonic sensors with different densities on the device housing according to risk areas, the stress changes inside the device can be captured more precisely. A denser sensor layout is adopted in high-risk areas, which helps to monitor potential high-stress concentration areas more carefully; by adopting differentiated emission strategies for different risk areas, the frequency and form of detection signals can be adjusted according to actual needs, thereby improving the detection efficiency and accuracy. By dynamically adjusting the ultrasonic detection angle and optimizing it in real time according to the risk areas of the external health thermal map of the device, different areas inside the device can be covered more effectively, reducing blind spots and improving the overall detection effect; introducing a multi-physical-field-coupled sound speed model, which comprehensively considers the effects of stress, temperature, and material properties on the sound speed, can more accurately reflect the true stress state inside the device; by continuously correcting the sound speed parameters in the sound speed model through the iterative time-of-flight algorithm and establishing an ultrasonic propagation path matrix using the ray tracing algorithm, and combining the regularized least squares method to solve the stress distribution, the stability and accuracy of the calculation results can be further improved; by real-time monitoring and warning potential high-stress areas, it helps to detect and handle device failures or safety hazards in a timely manner, reducing the probability and losses of accidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 FIG. is a schematic structural diagram of an industrial control data acquisition and monitoring platform driven by an intelligent algorithm according to the present invention;

[0049] Figure 2 FIG. is a flowchart of a method for generating an effectiveness index of an acquisition strategy provided by the present invention;

[0050] Figure 3 FIG. is a schematic flowchart of an industrial control data acquisition and monitoring method driven by an intelligent algorithm according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0052] Embodiment 1

[0053] Please refer to Figure 1 and Figure 2 shown in the figure. Embodiment 1 further describes an industrial control data acquisition and monitoring platform driven by an intelligent algorithm proposed by the present invention, including:

[0054] With the accelerating advancement of digitalization and the transformation of Industry 4.0, the manufacturing industry is undergoing profound changes. Traditional production processes need to improve efficiency, reduce energy consumption and emissions, and mitigate risks, and industrial intelligence can precisely meet these requirements. Against this backdrop, the industrial control data acquisition and monitoring platform, as an important part of industrial intelligence, is becoming increasingly prominent. Through real-time and accurate data acquisition and monitoring, enterprises can achieve refined management of the production process, improve production efficiency, and reduce operating costs;

[0055] Traditional data acquisition methods often rely on manual records or simple sensors. This approach is not only inefficient but also error-prone. The real-time and accuracy of data acquisition cannot be guaranteed, making it difficult to meet the needs of modern industrial production. Although existing monitoring systems can achieve a certain degree of data acquisition and monitoring, they often lack intelligence and adaptability. In the face of complex and changing industrial production environments, existing systems often struggle to provide comprehensive and accurate monitoring information.

[0056] With the continuous maturity of technologies such as artificial intelligence, big data, and cloud computing, intelligent algorithms have been widely applied in various fields. In the field of industrial control data acquisition and monitoring, the application of intelligent algorithms can significantly enhance the intelligence level and adaptability of the system.

[0057] Improvement in data processing capabilities:

[0058] Intelligent algorithms can efficiently process and analyze massive amounts of data, extract valuable information from it, and provide strong support for decision-making.

[0059] Improvement in adaptability:

[0060] Intelligent algorithms can automatically adjust data acquisition and monitoring strategies according to changes in the industrial production environment to ensure the stability and accuracy of the system.

[0061] Improvement in prediction and early warning capabilities:

[0062] Through technologies such as machine learning, intelligent algorithms can predict the occurrence trend of equipment failures, arrange maintenance plans in advance, effectively avoid unplanned downtime, and extend the service life of equipment.

[0063] To effectively solve the above problems, the present invention proposes an industrial control data acquisition and monitoring platform driven by intelligent algorithms, including:

[0064] A visual dynamic focusing layer that obtains multi-modal visual information based on the mechanism of the fovea centralis of the retina, applies a feature pyramid network to the multi-modal visual information, generates an external health heat map of the device, and divides the risk areas of the external health heat map of the device;

[0065] The bionic ultrasonic positioning and mapping layer deploys a MEMS ultrasonic array according to the divided risk areas, emits coded pulses to penetrate the device shell, calculates the internal echo signals through the time-of-flight algorithm, and generates a three-dimensional stress distribution map inside the device;

[0066] The cross-modal data fusion layer establishes the spatio-temporal mapping between vision and ultrasound, synchronizes clocks using the PTP protocol, and fuses the external health thermal map of the device and the three-dimensional stress distribution map inside the device through a graph attention network to generate a confidence score;

[0067] The dynamic control decision-making layer converts the confidence score into executable industrial control instructions based on a multi-objective optimization algorithm, and sends the executable industrial control instructions to the PLC in real time through the MQTT protocol;

[0068] The execution feedback layer monitors the changes in the device state after the PLC executes the industrial control instructions in real time, calculates the deviation between the device state change and the preset device state change threshold, generates an acquisition strategy effectiveness index, and dynamically optimizes the industrial control data acquisition strategy until the acquisition strategy effectiveness index reaches the optimum.

[0069] The method for obtaining multi-modal visual information based on the fovea centralis mechanism of the retina includes:

[0070] The fovea centralis region of the human eye retina is the most delicate perception region in the visual system. In this region, the density of cone cells is extremely high and they are arranged closely, providing the highest visual resolution and color sensitivity. As the distance from the fovea centralis region increases, the density of photoreceptor cells on the retina gradually decreases, and the visual resolution also decreases accordingly. This structure enables the human visual system to concentrate limited neural processing resources on the most important visual targets while maintaining a wide field of view. The industrial control monitoring platform draws on this biological principle to achieve efficient visual information acquisition. Deploy three visual sensors, namely an infrared thermal imager, an industrial camera, and a lidar, on the industrial control data acquisition and monitoring platform to collect multi-modal visual information on the device surface; the multi-modal visual information includes the temperature distribution data on the device surface, the surface visual data of the device, and the three-dimensional depth data of the device surface and the surrounding environment;

[0071] Simulate the non-uniform perception mechanism of the fovea centralis of the human eye and dynamically adjust the sensor resolution and sampling density; based on the CAD model of the device, mark the welds, bearing positions, and gear meshing points of the device as high-resolution focus areas; mark the remaining areas on the device surface as low-resolution peripheral areas; the division basis is determined according to the structural importance. Key areas such as the welds, bearing positions, and gear meshing points of the device have a greater impact on the device operation state, and the peripheral areas are usually support structures with a lower failure probability; dynamically define the resolution function, and the resolution function is ; where represents the coordinate of any point on the device surface; Indicates high resolution; Indicates low resolution; Indicates the focus area; Indicates the peripheral area; the device surface is divided into a high-resolution focus area and a low-resolution peripheral area; the high-resolution focus area simulates the fovea region of the human eye, and the low-resolution peripheral area simulates the peripheral region of the retina;

[0072] The following problems existing in the prior art are solved:

[0073] Problems of data redundancy and resource waste: Traditional uniform sampling over-collects non-critical areas, resulting in a large amount of data, high computing and storage costs; Problem of insufficient monitoring accuracy in critical areas: Fixed-resolution sensors cannot balance the global and local areas, and micron-level defects in critical areas (such as welds) are easily missed; Problem of poor adaptability: The preset monitoring area cannot respond to sudden failures;

[0074] The present invention introduces the non-uniform perception mechanism of the human eye fovea into industrial data acquisition, breaking through the traditional uniform sampling paradigm; combines the CAD model with the dynamic attention mechanism to achieve dual driving of structural prior knowledge and real-time anomaly response; according to the CAD model of the device, it is known in advance which are the critical parts prone to problems; In addition, a dynamic resolution function is proposed, and the resolution and sampling density are continuously adjustable through mathematical modeling, rather than the traditional fixed grading mode;

[0075] The beneficial effects compared with the prior art are: Optimized allocation of computing resources, improved overall monitoring efficiency, higher-precision monitoring in critical areas, enhanced early fault detection ability, reduced data processing burden, reduced storage requirements, faster system response speed, and better real-time performance;

[0076] For example, in the monitoring of wind turbine generators, bearings and gearboxes are the critical components most prone to failure. After applying this technology, the system will use high-resolution monitoring of Rhigh (such as a sampling rate of 1 kHz) for these areas, while using low-resolution monitoring of Rlow (such as 100 Hz) for non-critical areas such as the casing. In this way, this differential monitoring strategy reduces the data processing volume by about 80% while maintaining the monitoring accuracy of critical components, extends the service life of the equipment, and improves the accuracy of fault warning at the same time.

[0077] Analyze the abnormal features in multimodal visual information in real time through a spatial attention network. The input of the spatial attention network is multimodal visual information, and the output is the abnormal feature area (such as temperature mutation, deformation gradient); Dynamically adjust the range of the focused area according to the abnormal feature area. If a new abnormal feature area is detected, the new abnormal feature area will be included in the high-resolution focused area; If no abnormal features are detected in the high-resolution focused area, the high-resolution focused area will be reduced to a low-resolution peripheral area; The high-resolution focused area uses dense grid sampling, and the frame rate is dynamically increased to 1200fps. The low-resolution peripheral area uses sparse grid sampling, and the data volume compression rate > 65%. Dynamically adjust the sensor parameters through control instructions to obtain the final multimodal visual information;

[0078] The sensor parameters include infrared thermal imager parameters, industrial camera parameters, and lidar parameters; The infrared thermal imager parameters include sensitivity, sampling area, and integration time; The industrial camera parameters include optical focal length, frame rate, and exposure time; The lidar parameters include scan density, transmit power, and multi-echo detection.

[0079] The method of applying a feature pyramid network to multimodal visual information to generate an external health thermal map of the device includes:

[0080] Perform standard deviation normalization on the multimodal visual information to ensure that the output data of different sensors are within the same scale range, thereby improving the fusion effect. Stack the normalized multimodal visual information into a four-dimensional tensor according to the channel dimension. The dimensions of the four-dimensional tensor include height, width, number of channels, and time sequence frames; Construct a feature pyramid network structure, which includes a bottom-up path, a top-down path, and lateral connections;

[0081] Input the multimodal visual information into the feature pyramid network structure. In the bottom-up path, use a convolutional neural network to extract features from the input multimodal data, generate feature maps of different scales, and extract higher-level semantic information layer by layer. The feature maps of different scales include low-level feature maps (such as edges, textures), intermediate-level feature maps (such as shapes, objects), and high-level feature maps (such as the overall state of the device); In the top-down path, gradually restore the spatial resolution of the high-level feature maps through upsampling; Establish lateral connections between the bottom-up path and the top-down path to fuse the feature maps of the same scale and enhance the feature expression ability;

[0082] Apply 1×1 convolution on each output layer of the feature pyramid network structure to adjust the feature channels so that each layer of features has the same number of channels; Fuse the feature maps of different scales output by the feature pyramid network structure to generate an external health thermal map of the device, that is, the fused feature map contains all the information in the multimodal visual information.

[0083] The method for dividing risk regions of the external health heat map of a device includes:

[0084] Calculate the device health status score for each region in the external health heat map of the device. By performing a weighted sum on the temperature distribution data of the device surface, the surface visual data of the device, and the three-dimensional depth data of the device surface and the surrounding environment included in the multi-modal visual information, the device health status score is obtained. It should be noted that the method for setting the weight factors corresponding to the temperature distribution data of the device surface, the surface visual data of the device, and the three-dimensional depth data of the device surface and the surrounding environment is as follows:

[0085] According to the importance of the temperature distribution data of the device surface to the final device health status score, initially set the initial value range of the weight factor of the temperature distribution data of the device surface to be from 0.2 to 0.4, and the initial default value is 0.3. Use the historical temperature distribution data of the device surface to analyze the contribution degree of the temperature distribution data of the device surface to the device health status score. If the temperature distribution data of the device surface has a greater impact on the device health status score (for example, the proportion affecting the evaluation result exceeds 50%), then increase the value of the weight factor of the temperature distribution data of the device surface; otherwise, decrease its value. Gradually optimize the weight factor value according to the feedback of the temperature distribution data of the device surface. Similarly, set the weight factor of the surface visual data of the device and the weight factor corresponding to the three-dimensional depth data of the device surface and the surrounding environment, and ensure that the sum of the weight factors among them is 1.

[0086] Preset the first threshold of the device health status score and the second threshold of the device health status score. Compare the device health status score with the preset first threshold and the second threshold of the device health status score respectively. If the device health status score is less than the first threshold of the device health status score, then mark the region corresponding to this score as green; if the device health status score is greater than the first threshold of the device health status score and less than the second threshold of the device health status score, then mark the region corresponding to this score as blue; if the device health status score is greater than the second threshold of the device health status score, then mark the region corresponding to this score as red;

[0087] Use different colors to divide the risk regions of the external health heat map of the device. Different colors represent different risk regions. Define the green region as the low-risk region, the blue region as the medium-risk region, and the red region as the high-risk region.

[0088] The method for generating the three-dimensional stress distribution map inside the device includes:

[0089] Deploy ultrasonic sensors with different densities on the device housing according to the divided risk areas; the ultrasonic sensors are MEMS ultrasonic array sensors. For example, a dense array layout with a 0.5 mm pitch is adopted in the high-risk area (red), a 2 mm pitch is configured in the medium-risk area (blue), and a 5 mm pitch is set in the low-risk area (green).

[0090] Adopt differentiated transmission strategies for different risk areas. For example, a broadband linear frequency modulation signal (2 - 10 MHz) is transmitted in the high-risk area, a 5-cycle tone burst pulse is used in the medium-risk area, and a conventional pulse echo mode is adopted in the low-risk area. Encoded pulses are transmitted into the device through the ultrasonic sensors for initial global detection, and the ultrasonic detection angle is dynamically adjusted according to the risk areas of the external health thermal map of the device. The ultrasonic detection angle interval in the high-risk area is less than that in the medium-risk area, and the ultrasonic detection angle interval in the medium-risk area is less than that in the low-risk area; an angle adjustment algorithm is applied to dynamically adjust the ultrasonic detection angle, ; where, represents the adjustment weight; represents the adjusted ultrasonic detection angle; represents the reference ultrasonic detection angle; represents the position of any area in the high, medium, and low-risk areas where the risk coefficient is;

[0091] Introduce a multi-physical-field coupled sound speed model, ; where, represents the propagation speed of ultrasonic waves in the device material, which changes due to internal stress and temperature; represents the internal stress state of the device material; represents the temperature of the device material. Temperature changes will cause changes in material density and elastic modulus, thereby affecting the sound speed; represents the reference sound speed of the device material in a stress-free state; represents the acoustoelastic coefficient in the direction of the principal stress; represents the principal stress component; represents the influence coefficient of temperature on the sound speed; represents the change in the current temperature relative to the reference temperature; represents the direction index of the principal stress, , corresponding to the three principal stress directions inside the equipment material, i.e., the three orthogonal principal axis directions of the stress tensor; the definition of the principal stress here is: inside the equipment material at any point, there are three mutually perpendicular directions (principal axes); continuously correct the sound speed parameters in the sound speed model through the iterative time-of-flight algorithm, and stop when the maximum number of iterations is reached; establish an ultrasonic propagation path matrix based on the ray tracing algorithm, solve the stress distribution through the regularized least squares method, and finally generate a three-dimensional stress distribution map of the equipment interior with risk marks.

[0092] The following problems existing in the prior art are solved:

[0093] The problem that traditional ultrasonic detection is not fine enough and prone to missing key areas: Existing ultrasonic detection usually places sensors at fixed intervals. However, in this way, the resolution in high-risk areas may be insufficient, while resources are wasted in low-risk areas.

[0094] The problem that the signal emission method is single and cannot meet the requirements of different areas: Existing detection methods generally use only one ultrasonic signal, resulting in too weak signals in some areas and wasted power in others.

[0095] The problem of large errors caused by not considering the influence of stress and temperature inside the material: Traditional methods often assume that the speed of ultrasonic waves propagating in the equipment material is fixed. However, in fact, the internal stress and temperature will affect the sound speed, which easily leads to inaccurate stress distribution maps calculated. The problem of poor dynamic adaptability: Conventional detection strategies are fixed and cannot adjust the sensor layout and detection parameters according to the real-time state of the equipment, resulting in a high risk of missed detection.

[0096] The problem that the data processing algorithm is too simple and the accuracy is insufficient: Existing calculation methods are relatively rough and are easily interfered by noise, resulting in insufficiently fine stress distribution maps and large errors.

[0097] The innovation points of the present invention are:

[0098] The sensor layout is more intelligent. Instead of placing sensors randomly, they are arranged according to the level of risk. More sensors are placed where key attention is needed, and fewer are placed where the risk is low, achieving targeted deployment. Risk-region adaptive sensor deployment: Dynamically configure the MEMS array density according to the risk level (for example, high risk 0.5mm / medium risk 2mm / low risk 5mm), realizing a hybrid detection mode of "high-precision coverage of key areas + efficient scanning of ordinary areas";

[0099] The detection strategy is more flexible. By adopting different signal emission methods, the signals in high-risk areas are stronger and the detection is more detailed, while resources are saved and efficiency is improved in low-risk areas. Multimodal ultrasonic emission strategy: High-risk area: Wideband linear frequency modulation (2 - 10 MHz) to improve the signal-to-noise ratio; Medium-risk area: 5-cycle tone burst pulse to balance resolution and penetration depth; Low-risk area: Conventional pulse echo mode to reduce energy consumption;

[0100] Dynamic angle adjustment algorithm: Based on the risk coefficient of the heat map, the detection angle interval is optimized in real time. The angle interval in high-risk areas is denser to capture anisotropic stress;

[0101] Traditional methods do not consider the influence of stress and temperature, and the calculated results are inaccurate. Now, the multi-physical field coupled sound speed model: simultaneously integrates the influence of stress field and temperature field on the sound speed, and dynamically corrects the sound speed parameters through the iterative flight time algorithm;

[0102] More advanced calculation method, hierarchical calculation optimization: High-risk areas are calculated first, low-risk areas use simplified models, and the regularized least squares method adds regional confidence weights;

[0103] The beneficial effects compared with the existing technology are as follows: The detection accuracy is greatly improved. Since the sensor layout density, detection angle, and signal emission strategy are all dynamically optimized, the stress detection accuracy in high-risk areas is improved, reducing the possibility of misjudgment and missed detection; The calculation efficiency is optimized. Through differential signal emission and ultrasonic detection angle adjustment, the calculation burden in low-risk areas is reduced, making the overall calculation load of the system balanced and improving the real-time performance. It adapts to complex working conditions and improves engineering practicability. Since the influence of factors such as material stress and temperature is considered, the detection results are more in line with the actual working state of the equipment, enhancing the adaptability under complex working conditions. Efficiently solve the three-dimensional stress distribution. Through advanced mathematical models and optimization algorithms, a three-dimensional stress distribution map with risk marks can be generated faster and more accurately, providing more accurate data support for equipment condition monitoring and fault prediction.

[0104] The method for establishing the spatio-temporal mapping between vision and ultrasound includes:

[0105] Taking any point on the surface of the industrial equipment (such as a fixed corner point of the equipment shell) as the origin, The axis and The axis are parallel to the surface of the industrial equipment, The axis is perpendicular to the surface of the industrial equipment, defining a unified global coordinate system that can contain the three-dimensional coordinates of all vision sensors and ultrasound sensors;

[0106] Calibrate the vision sensor using a calibration board to obtain the internal parameters (such as focal length and principal point coordinates) and external parameters (such as rotation matrix and translation vector) of the vision sensor; convert the pixel coordinates of the external health heat map of the device to the camera coordinate system through the internal parameters of the vision sensor, and then convert the camera coordinate system to the global coordinate system through the external parameters of the vision sensor, so as to obtain the external health heat map of the device in the global coordinate system; use the origin of the global coordinate system as the reference point to calibrate the ultrasonic sensor and obtain the external parameters of the ultrasonic sensor;

[0107] Convert the internal three-dimensional stress distribution map of the device to the global coordinate system through the external parameters of the ultrasonic sensor, so as to obtain the internal three-dimensional stress distribution map of the device in the global coordinate system; in the global coordinate system, align the spatial coordinates of the external health heat map of the device and the internal three-dimensional stress distribution map of the device to establish a spatial mapping between vision and ultrasound; set the time source connected to the vision sensor as the master clock, set the time source connected to the ultrasonic sensor as the slave clock, and perform time synchronization through the PTP protocol to establish a time mapping between vision and ultrasound.

[0108] Configure the PTP protocol on the vision sensor and set the time source connected to the vision sensor as the master clock; configure the PTP protocol on the ultrasonic sensor and set the time source connected to the ultrasonic sensor as the slave clock; start the PTP service on the vision sensor to start propagating the time information of the master clock; start the PTP service on the ultrasonic sensor to start receiving the time information of the master clock;

[0109] The slave clock sends a synchronization message to the master clock and records the sending timestamp After receiving the synchronization message, the master clock records the receiving timestamp and immediately sends a follow-up message to the slave clock; the follow-up message contains the receiving timestamp of the master clock and the sending timestamp ; the slave clock receives the follow-up message and records the receiving timestamp and calculates the round-trip delay according to the sending timestamp and the receiving timestamp ; the slave clock adjusts the time according to the round-trip delay to be synchronized with the master clock.

[0110] The method of fusing the external health heat map of the device and the internal three-dimensional stress distribution map through a graph attention network includes:

[0111] Construct an undirected graph based on the external healthy thermal map of the device and the internal three-dimensional stress distribution map of the device in the global coordinate system. Build a graph attention network using a multi-layer GAT structure based on the multi-head attention mechanism to fuse the external healthy thermal map of the device and the internal three-dimensional stress distribution map of the device in the global coordinate system. The graph attention network includes an input layer, a feature projection layer, a graph attention layer, a cross-modal interaction layer, and an output layer. Take the undirected graph as the input of the input layer of the graph attention network, and generate a confidence score through the output layer.

[0112] The method for constructing the undirected graph includes:

[0113] Take each risk area in the external healthy thermal map of the device in the global coordinate system as a visual node, and extract the feature vector of each risk area as the visual node feature. The visual node features include temperature gradient, risk area area, shape factor, and texture feature (extracted by CNN). The temperature gradient is obtained by calculating the maximum temperature difference within the risk area. The shape factor is obtained by calculating the compactness of the risk area.

[0114] Take each stress concentration area in the internal three-dimensional stress distribution map of the device in the global coordinate system as an ultrasonic node, and extract the three-dimensional voxel feature vector of each stress concentration area as the ultrasonic node feature. The ultrasonic node features include stress amplitude, gradient direction, frequency feature, and spatial position. Take the maximum stress value within the stress concentration area as the stress amplitude. Use the Sobel operator to calculate the gradient direction of the stress concentration area. Perform frequency domain analysis on the stress values within the stress concentration area, and take the dominant frequency component as the frequency feature. Perform Min-Max normalization on the visual node features and ultrasonic node features respectively to eliminate the influence of dimensions. Collect all visual nodes and ultrasonic nodes to obtain a node set.

[0115] The undirected graph contains two types of edges: intra-modal edges and cross-modal edges. Construct intra-modal edges based on the Euclidean distance in the global coordinate system: traverse all visual nodes and calculate the Euclidean distance between any two visual nodes in the global coordinate system. Preset a visual distance threshold. If the Euclidean distance between any two visual nodes in the global coordinate system is less than the preset visual distance threshold, add a bidirectional edge between the corresponding two visual nodes. If the Euclidean distance between any two visual nodes in the global coordinate system is greater than or equal to the preset visual distance threshold, do not add. The preset visual distance threshold is set to 10mm according to the physical range of heat diffusion on the metal surface.

[0116] Traverse all ultrasonic nodes, and calculate the Euclidean distance between any two ultrasonic nodes in the global coordinate system; preset an ultrasonic distance threshold. If the Euclidean distance between any two ultrasonic nodes in the global coordinate system is less than the preset ultrasonic distance threshold, add a bidirectional edge between the corresponding two ultrasonic nodes; if the Euclidean distance between any two ultrasonic nodes in the global coordinate system is greater than or equal to the preset ultrasonic distance threshold, do not add; the preset ultrasonic distance threshold is set to 15 mm according to the attenuation characteristics of stress waves in steel materials.

[0117] Through nearest neighbor search, find the nearest ultrasonic node for each visual node, add a bidirectional edge between the visual node and the corresponding nearest ultrasonic node to construct a cross-modal edge; collect all bidirectional edges to obtain an edge set; construct an undirected graph based on the obtained node set and edge set.

[0118] For example: Scenario: Monitoring of wind turbine gearbox

[0119] Visual nodes: 3 high-risk areas (coordinates , , )

[0120] Ultrasonic nodes: 2 stress concentration areas (coordinates , )

[0121] Construction result:

[0122] Edges within the modality: (distance is about 7 mm), (distance is about 14 mm)

[0123] Cross-modal edges: , , .

[0124] The method for converting confidence scores into executable industrial control instructions based on a multi-objective optimization algorithm includes:

[0125] Divide the internal space of the device into small cubic units, and each unit records the current stress value and confidence score; at the same time, list all adjustable industrial control data parameters, and the industrial control data parameters include the adjustment range, energy consumption cost, and the strength degree of the historical corresponding stress influence of each parameter;

[0126] Three optimization objectives are established. The optimization objectives include the first safety objective, the second safety objective, and the economic objective. The first safety objective is to reduce the maximum stress value in the equipment, especially focusing on areas with low confidence. When calculating the stress values in these areas, they are weighted and amplified. The second safety objective is to make the confidence scores in different areas as uniform as possible and avoid sudden drops in confidence in certain areas. The economic objective is to control the overall energy consumption of all industrial control data parameter adjustments and prefer adjustment plans with small change ranges. Two types of constraint conditions are set. The constraint conditions include hard constraints and flexible constraints. The hard constraint is that all industrial control data parameter adjustments shall not exceed the maximum change speed allowed by the equipment. The flexible constraint is that for areas with low confidence, a slightly larger stress change range is allowed.

[0127] Use a multi-objective optimization algorithm to randomly generate m sets of industrial control data parameter optimization plans. Evaluate the completion of the three optimization objectives for each plan, obtain the completion scores of the three optimization objectives, and add up the completion scores of the three optimization objectives to obtain a comprehensive score. Select the plan with the highest comprehensive score from the m sets of plans as the final optimization plan. Convert the selected final optimization plan into an actual executable industrial control instruction. The executable industrial control instruction includes the first instruction, the second instruction, and the safety monitoring instruction.

[0128] The method for obtaining the completion scores of the three optimization objectives includes:

[0129] Obtain the completion score of the first safety objective by observing the percentage reduction in the maximum stress value in the equipment. Calculate the standard deviation of the confidence scores in different areas as the completion score of the second safety objective. Calculate the total energy consumption caused by all industrial control data parameter adjustments as the completion score of the economic objective.

[0130] The first instruction: For areas with high stress and low confidence, quickly reduce the relevant parameter values within 1 second. The second instruction: Other parameters are adjusted step by step in stages. The adjustment range of each stage is determined according to the confidence of the relevant area: The lower the confidence, the more stages are divided, and the smaller the single adjustment range; Areas with high confidence can be adjusted in one step with a larger range. The safety monitoring instruction: Check the confidence change every 2 seconds. If it is found that the confidence in any area suddenly drops by more than 30%, immediately stop the current adjustment and recalculate the plan.

[0131] The method for generating the effectiveness index of the acquisition strategy includes:

[0132] Collect the industrial control instruction parameters executed by the PLC, synchronously monitor the changes in the device status, calculate the deviation between the device status change and the preset device status change threshold, and generate a 0-1 standardized acquisition strategy effectiveness index d; preset a first threshold d1 and a second threshold d2 for the standardized acquisition strategy effectiveness index; if d>d2, it is determined that the acquisition strategy is effective; if d1<d<d2, it is determined that the acquisition strategy still needs to be observed; if d<d1, the acquisition strategy is ineffective, optimize the industrial control data acquisition strategy, and immediately generate a warning message.

[0133] The preset device status change threshold is set by the staff. Different device status changes are collected through the industrial control data acquisition and monitoring platform, and the average value of multiple device status changes is taken as the preset device status change threshold; similarly, set a first threshold for the preset device health status score, a second threshold for the device health status score, a preset visual distance threshold, and a preset ultrasonic distance threshold.

[0134] In this embodiment, by simulating the fovea mechanism of the human eye, high-resolution acquisition is performed in key areas, and the resolution is reduced in non-critical areas to ensure high detection accuracy in key parts while reducing the computational burden. The dynamic resolution function is adaptively adjusted according to the importance of different areas on the device surface, rather than a one-size-fits-all approach, to improve the effectiveness of the data. The frame rate is dynamically increased to 1200fps, and the visual information acquisition in key areas is more delicate, suitable for fine monitoring of working conditions such as high-speed rotation and vibration. At the same time, the data of the infrared thermal imager, industrial camera, and lidar are fused to obtain temperature, visible light texture, and 3D depth information, forming a more complete visual information to avoid the limitations of a single sensor. The spatial attention network intelligently analyzes abnormal areas. When temperature mutations, abnormal deformation gradients, etc. occur on the device surface, the system will automatically focus on these areas to reduce omissions. The high and low resolution areas are adjusted in real time. If a new abnormality is found, the resolution is automatically increased; if there is no abnormality in a certain area for a long time, it is reduced to low resolution to reduce the data acquisition pressure.

[0135] By deploying ultrasonic sensors with different densities on the device shell according to risk areas, the stress changes inside the device can be captured more precisely. A denser sensor layout is adopted in high-risk areas, which helps to monitor potential high-stress concentration areas more carefully. By adopting a differentiated transmission strategy for different risk areas, the frequency and form of detection signals can be adjusted according to actual needs, thereby improving the detection efficiency and accuracy. By dynamically adjusting the ultrasonic detection angle and optimizing it in real time according to the risk areas of the external health thermal map of the device, different areas inside the device can be covered more effectively, reducing blind spots and improving the overall detection effect. By introducing a multi-physical-field-coupled sound speed model, the effects of stress, temperature, and material properties on the sound speed are comprehensively considered, so that the true stress state inside the device can be reflected more accurately. By continuously correcting the sound speed parameters in the sound speed model through the iterative time-of-flight algorithm and establishing an ultrasonic propagation path matrix using the ray tracing algorithm, and combining the regularized least squares method to solve the stress distribution, the stability and accuracy of the calculation results can be further improved. By real-time monitoring and warning potential high-stress areas, it helps to detect and handle device failures or safety hazards in a timely manner, reducing the probability and losses of accidents.

[0136] Embodiment 2

[0137] Please refer to Figure 3 As shown, for the parts not described in detail in this embodiment, refer to the description content of Embodiment 1. Provide an industrial control data acquisition and monitoring method driven by an intelligent algorithm, including:

[0138] S1. Obtain multi-modal visual information based on the fovea centralis mechanism of the retina, apply a feature pyramid network to the multi-modal visual information to generate an external health thermal map of the device, and divide the risk areas of the external health thermal map of the device;

[0139] S2. Deploy a MEMS ultrasonic array according to the divided risk areas, emit coded pulses to penetrate the device shell, calculate the internal echo signal through the time-of-flight algorithm, and generate a three-dimensional stress distribution map inside the device;

[0140] S3. Establish a spatio-temporal mapping between vision and ultrasound, synchronize clocks using the PTP protocol, and fuse the external health thermal map of the device and the three-dimensional stress distribution map inside the device through a graph attention network to generate a confidence score;

[0141] S4. Based on a multi-objective optimization algorithm, convert the confidence score into an executable industrial control instruction, and send the executable industrial control instruction to the PLC in real time through the MQTT protocol;

[0142] S5. Real-time monitor the device state changes after the PLC executes the industrial control instruction, calculate the deviation between the device state changes and the preset device state change threshold, generate an acquisition strategy effectiveness index, and dynamically optimize the industrial control data acquisition strategy until the acquisition strategy effectiveness index reaches the optimum.

[0143] Since the electronic device introduced in this embodiment is the electronic device adopted by the industrial control data acquisition and monitoring platform driven by an intelligent algorithm in the embodiments of the present application, based on the industrial control data acquisition and monitoring platform driven by an intelligent algorithm introduced in the embodiments of the present application, those skilled in the art can understand the specific implementation manners and various variations of the electronic device in this embodiment. Therefore, the specific implementation of how this electronic device implements the method in the embodiments of the present application will not be described in detail here. As long as those skilled in the art implement the electronic device adopted by the industrial control data acquisition and monitoring platform driven by an intelligent algorithm in the embodiments of the present application, it falls within the scope of protection of the present application.

[0144] The above formulas are all dimensionless and take their numerical calculations. The formula is a formula obtained by collecting a large amount of data for software simulation to get the closest to the real situation. The preset parameters and threshold selection in the formula are set by those skilled in the art according to the actual situation.

[0145] The above are only the preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the concept of the present invention belong to the scope of protection of the present invention. It should be noted that for ordinary technical users in the technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the scope of protection of the present invention.

Claims

1. An intelligent algorithm driven industrial control data acquisition and monitoring platform, characterized in that: include: The visual dynamic focusing layer obtains multimodal visual information based on the retinal fovea-like mechanism, applies a feature pyramid network to the multimodal visual information, generates a device external health heat map, and divides the device external health heat map into risk areas; The bionic ultrasonic positioning mapping layer deploys the MEMS ultrasonic array according to the divided risk areas, transmits coded pulses to penetrate the device shell, calculates the internal echo signal through the time-of-flight algorithm, and generates a three-dimensional stress distribution map inside the device; The cross-modal data fusion layer establishes the spatiotemporal mapping of vision and ultrasound, uses the PTP protocol to synchronize the clock, and fuses the external health thermal map of the device and the internal three-dimensional stress distribution map of the device through the graph attention network to generate a confidence score; The dynamic control decision layer converts the confidence score into executable industrial control instructions based on the multi-objective optimization algorithm, and sends the executable industrial control instructions to the PLC in real time through the MQTT protocol; The execution feedback layer monitors the changes in device status after the PLC executes industrial control instructions in real time, calculates the deviation between the device status change and the preset device status change threshold, generates the collection strategy effectiveness index, and dynamically optimizes the industrial control data collection strategy until the collection strategy effectiveness index reaches the optimal value.

2. According to claim 1, an intelligent algorithm driven industrial control data acquisition and monitoring platform is characterized in that: The method for acquiring multimodal visual information based on the fovea-mimicking mechanism comprises: Deploy three types of visual sensors, infrared thermal imagers, industrial cameras and lidar, on the industrial control data acquisition and monitoring platform to collect multimodal visual information on the equipment surface. The multimodal visual information includes temperature distribution data on the equipment surface, surface visual data of the equipment, and three-dimensional depth data of the equipment surface and the surrounding environment. Simulate the non-uniform perception mechanism of the human eye's fovea, and dynamically adjust the sensor resolution and sampling density; based on the device's CAD model, record the device's welds, bearings, and gear meshing points as high-resolution focal areas; record the rest of the device's surface as low-resolution peripheral areas; dynamically define the resolution function, and divide the device surface into high-resolution focal areas and low-resolution peripheral areas; the high-resolution focal area simulates the human eye's fovea, and the low-resolution peripheral area simulates the peripheral area of ​​the retina; The abnormal features in multimodal visual information are analyzed in real time through the spatial attention network. The input of the spatial attention network is multimodal visual information, and the output is the abnormal feature area. The focus area range is dynamically adjusted according to the abnormal feature area. If a new abnormal feature area is detected, the new abnormal feature area is included in the high-resolution focus area. If no abnormal features are detected in the high-resolution focus area, the high-resolution focus area is reduced to a low-resolution peripheral area. Dense rasterization sampling is used in the high-resolution focus area, and sparse rasterization sampling is used in the low-resolution peripheral area. The sensor parameters are dynamically adjusted through control instructions to obtain the final multimodal visual information.

3. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 2 is characterized in that: The method for applying a feature pyramid network to multimodal visual information to generate an external health heat map of a device includes: Performing standard deviation normalization on the multimodal visual information, stacking the normalized multimodal visual information into a four-dimensional tensor according to the channel dimension, the dimensions of the four-dimensional tensor include height, width, number of channels and time series frame; constructing a feature pyramid network structure, which includes a bottom-up path, a top-down path and a lateral connection; The multimodal visual information is input into the feature pyramid network structure. In the bottom-up path, a convolutional neural network is used to extract features from the input multimodal data, generate feature maps of different scales, and extract higher-level semantic information layer by layer. Feature maps of different scales include low-level feature maps, mid-level feature maps, and high-level feature maps. In the top-down path, the high-level feature maps are gradually restored to spatial resolution through upsampling. A lateral connection is established between the bottom-up path and the top-down path to fuse feature maps of the same scale. A 1×1 convolution is applied to each output layer of the feature pyramid network structure to adjust the feature channels, and the feature maps of different scales output by the feature pyramid network structure are fused to generate the external health heat map of the equipment.

4. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 3 is characterized in that: The method for dividing the risk area of ​​the external health thermogram of the equipment includes: Calculate the device health status score of each area in the device external health thermogram by weighted summing the temperature distribution data of the device surface, the surface visual data of the device, and the three-dimensional depth data of the device surface and the surrounding environment included in the multimodal visual information to obtain the device health status score; A first threshold value for the device health status score and a second threshold value for the device health status score are preset, and the device health status score is compared with the preset first threshold value for the device health status score and the second threshold value for the device health status score respectively. If the device health status score is less than the first threshold value for the device health status score, the area corresponding to the score is marked as green; if the device health status score is greater than the first threshold value for the device health status score and less than the second threshold value for the device health status score, the area corresponding to the score is marked as blue; if the device health status score is greater than the second threshold value for the device health status score, the area corresponding to the score is marked as red; Use different colors to divide the risk areas of the equipment external health heat map. Different colors represent different risk areas. The green area is defined as a low-risk area, the blue area is a medium-risk area, and the red area is a high-risk area.

5. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 4 is characterized in that: The method for generating a three-dimensional stress distribution diagram inside a device comprises: According to the divided risk areas, ultrasonic sensors of different densities are deployed on the surface of the equipment shell; the ultrasonic sensors are MEMS ultrasonic array sensors, which adopt differentiated emission strategies for different risk areas. The ultrasonic sensors transmit coded pulses to the inside of the equipment for initial global detection, and dynamically adjust the ultrasonic detection angle according to the risk area of ​​the external health heat map of the equipment. The ultrasonic detection angle interval in the high-risk area is smaller than that in the medium-risk area, and the ultrasonic detection angle interval in the medium-risk area is smaller than that in the low-risk area; the angle adjustment algorithm is used to dynamically adjust the ultrasonic detection angle, and a multi-physical field coupled sound velocity model is introduced. The sound velocity parameters in the sound velocity model are continuously corrected through an iterative flight time algorithm, and the algorithm stops when the maximum number of iterations is reached; the ultrasonic propagation path matrix is ​​established based on the ray tracing algorithm, and the stress distribution is solved by the regularized least squares method, and finally a three-dimensional stress distribution map of the internal equipment with risk tags is generated.

6. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 5, characterized in that: The method for establishing the spatiotemporal mapping of vision and ultrasound comprises: Take any point on the surface of industrial equipment as the origin, Axis and The axis is parallel to the surface of the industrial equipment, The axis is perpendicular to the surface of the industrial equipment, and a unified global coordinate system is defined; the visual sensor is calibrated using a calibration plate to obtain the internal and external parameters of the visual sensor; the pixel coordinates of the external health thermogram of the equipment are converted to the camera coordinates through the internal parameters of the visual sensor, and then the camera coordinates are converted to the global coordinates through the external parameters of the visual sensor, thereby obtaining the external health thermogram of the equipment in the global coordinate system; the ultrasonic sensor is calibrated using the origin of the global coordinate system as a reference point to obtain the external parameters of the ultrasonic sensor; The three-dimensional stress distribution map inside the equipment is converted to the global coordinate system through the external parameters of the ultrasonic sensor, and then the three-dimensional stress distribution map inside the equipment in the global coordinate system is obtained; in the global coordinate system, the spatial coordinates of the external health thermal map of the equipment and the three-dimensional stress distribution map inside the equipment are aligned to establish a spatial mapping between vision and ultrasound; the time source connected to the visual sensor is set as the master clock, and the time source connected to the ultrasonic sensor is set as the slave clock, and time synchronization is performed through the PTP protocol to establish a time mapping between vision and ultrasound.

7. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 6, characterized in that: The method of fusing the external health thermal map of the device and the internal three-dimensional stress distribution map of the device through the graph attention network includes: According to the external health heat map of the device in the global coordinate system and the three-dimensional stress distribution map of the device inside the device, an undirected graph is constructed, and a multi-layer GAT structure based on the multi-head attention mechanism is used to build a graph attention network to fuse the external health heat map of the device in the global coordinate system and the three-dimensional stress distribution map of the device inside the device; the graph attention network includes an input layer, a feature projection layer, a graph attention layer, a cross-modal interaction layer and an output layer; the undirected graph is used as the input of the input layer of the graph attention network, and the confidence score is generated through the output layer.

8. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 7, characterized in that: The method for constructing an undirected graph comprises: Each risk area in the external health thermodynamic map of the equipment in the global coordinate system is taken as a visual node, and the feature vector of each risk area is extracted as the visual node feature; each stress concentration area in the internal three-dimensional stress distribution map of the equipment in the global coordinate system is taken as an ultrasonic node, and the three-dimensional voxel feature vector of each stress concentration area is extracted as the ultrasonic node feature; all visual nodes and ultrasonic nodes are collected to obtain a node set; Traverse all visual nodes and calculate the Euclidean distance between any two visual nodes in the global coordinate system; preset a visual distance threshold. If the Euclidean distance between any two visual nodes in the global coordinate system is less than the preset visual distance threshold, add a bidirectional edge between the corresponding two visual nodes; if the Euclidean distance between any two visual nodes in the global coordinate system is greater than or equal to the preset visual distance threshold, do not add it; Traverse all ultrasonic nodes and calculate the Euclidean distance between any two ultrasonic nodes in the global coordinate system; preset an ultrasonic distance threshold. If the Euclidean distance between any two ultrasonic nodes in the global coordinate system is less than the preset ultrasonic distance threshold, add a bidirectional edge between the corresponding two ultrasonic nodes; if the Euclidean distance between any two ultrasonic nodes in the global coordinate system is greater than or equal to the preset ultrasonic distance threshold, do not add; through nearest neighbor search, find the nearest ultrasonic node for each visual node and add a bidirectional edge between the visual node and the corresponding nearest ultrasonic node; collect all bidirectional edges to obtain an edge set; construct an undirected graph based on the obtained node set and edge set.

9. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 8, characterized in that: The method for converting the confidence score into an executable industrial control instruction based on the multi-objective optimization algorithm includes: Divide the internal space of the device into small cube units, and each unit records the current stress value and confidence score; at the same time, list all adjustable industrial control data parameters, and the industrial control data parameters include the adjustment range, energy consumption cost, and the strength of the historical corresponding stress impact of each parameter; Establish three optimization objectives, including the first safety objective, the second safety objective, and the economic objective; set two types of constraint conditions, including hard constraints and flexible constraints; Use the multi-objective optimization algorithm to randomly generate m sets of industrial control data parameter optimization schemes, evaluate the completion of the three optimization objectives for each scheme, obtain the completion score of the three optimization objectives, and add the completion scores of the three optimization objectives to obtain a comprehensive score. Select the scheme with the highest comprehensive score from the m sets of schemes as the final optimization scheme; convert the selected final optimization scheme into an actual executable industrial control instruction; the executable industrial control instruction includes the first instruction, the second instruction, and the safety monitoring instruction; The method for obtaining the completion scores of the three optimization objectives includes: By observing the percentage reduction in the maximum stress value in the device, it is used as the completion score of the first safety objective; by calculating the standard deviation of the confidence scores in different regions, it is used as the completion score of the second safety objective; by calculating the total energy consumption caused by the adjustment of all industrial control data parameters, it is used as the completion score of the economic objective.

10. The intelligent algorithm driven industrial control data acquisition and monitoring platform according to claim 9, characterized in that: The method for generating the effectiveness index of the acquisition strategy includes: Collect the industrial control instruction parameters executed by the PLC, synchronously monitor the changes in the device state, calculate the deviation between the device state change and the preset device state change threshold, and generate a 0-1 standardized acquisition strategy effectiveness index d; preset a first threshold d1 and a second threshold d2 for the standardized acquisition strategy effectiveness index; if d > d2, it is determined that the acquisition strategy is effective; if d1 < d < d2, it is determined that the acquisition strategy still needs to be observed; if d < d1, the acquisition strategy is ineffective, optimize the industrial control data acquisition strategy, and immediately generate a warning message.

Images