Engine multi-modal operation identification method and device based on multi-dimensional Internet of Things, and medium

Abstract

The invention discloses an engine multi-mode operation identification method and device based on a multi-dimensional internet of things, and a medium, and aims to solve the problem that the driving state cannot be obtained in time due to inaccurate engine monitoring data. According to the method, multiple key parameters in operation of the engine can be comprehensively covered through different types of sensors, rich information basis is provided for multi-modal operation identification of the engine through multi-dimensional data collection, rich feature data is extracted from the multiple sensors, time domain, frequency domain and time frequency features are covered, and the method is suitable for multi-modal operation identification of the engine. Through construction of a multi-modal operation identification model, data of a plurality of sensors can be comprehensively considered, a neural network model is used for training, and a cross validation method is used for model verification, so that the stability and generalization ability of the model are ensured; and the state of the engine is quickly judged according to the multi-mode operation identification model, so that real-time monitoring and early warning of the state of the engine are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of engine identification, and in particular to an engine multimodal operation identification method, device and medium based on a multidimensional Internet of Things. Background Art

[0002] Engine multimodal operation identification refers to the use of a variety of different measurement methods and data sources to identify and analyze the characteristics of the engine under different operating conditions.

[0003] The Chinese patent with publication number CN111209683A discloses a method for constructing an aircraft engine working state recognition model, a model and a recognition method. The model constructed mainly through the C4.5 decision tree can effectively identify the working state of the aircraft engine, thereby supporting the health of the aircraft engine under different working states; when constructing the working state recognition model, the acquired data is preprocessed, which can effectively simplify the calculation and save computing resources. Although the above patent solves the problem of state recognition, there are still the following problems in actual operation:

[0004] 1. When performing engine identification and monitoring, the functions of each sensor are not fully utilized for more comprehensive monitoring, resulting in inaccurate monitoring data.

[0005] 2. Before the sensor monitors the engine, the position of each sensor is not carefully deployed, resulting in ineffective sensor monitoring.

[0006] 3. The engine monitoring data is not effectively processed and analyzed, resulting in the inability to obtain the engine driving status in a timely manner. Summary of the invention

[0007] The purpose of the present invention is to provide an engine multimodal operation identification method, device and medium based on a multidimensional Internet of Things. Through different types of sensors, multiple key parameters in the engine operation can be fully covered. This multi-dimensional data collection provides a rich information basis for the engine's multimodal operation identification, and extracts rich feature data from multiple sensors, covering time domain, frequency domain and time-frequency features, ensuring the comprehensiveness and accuracy of the data. Through the construction of a multimodal operation identification model, it is possible to comprehensively consider the data of multiple sensors, use a neural network model for training, and use a cross-validation method to verify the model, ensuring the stability and generalization ability of the model, and quickly judge the state of the engine according to the multimodal operation identification model, thereby realizing real-time monitoring and early warning of the engine state, which can solve the problems in the prior art.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] The engine multimodal operation identification method based on multidimensional Internet of Things includes the following steps:

[0010] S1: Engine sensor confirmation: Different sensors used to monitor the working status of the engine are confirmed according to the working nature of the engine, and the confirmed sensors are marked as target sensor data;

[0011] S2: Engine sensor detection: Perform functional detection on the target sensors according to the type of each sensor. After the functional detection is completed and qualified, it will be marked as standard sensor data;

[0012] S3: Sensor deployment: deploy standard sensor data around the engine, and obtain standard deployed sensors after deployment;

[0013] S4: Sensor data monitoring: Standard deployment of sensors to collect real-time data from the engine, and mark the real-time collected data as engine monitoring data;

[0014] S5: Monitoring data processing: preprocessing the real-time collected engine monitoring data to obtain target monitoring data;

[0015] S6: Monitoring data model construction: feature information is extracted from the target monitoring data, and a multimodal operation recognition model is constructed based on the extracted feature information. After the multimodal operation recognition model is constructed, the engine multimodal model data is obtained.

[0016] Preferably, in S1, different sensors for monitoring the working state of the engine are confirmed according to the working properties of the engine, including:

[0017] Sensors include pressure sensors, flow sensors, temperature sensors, three-axis accelerometers, speed sensors, torque sensors, oil level sensors, oxygen sensors, position sensors, load sensors, sound sensors, GPS sensors, and tilt sensors;

[0018] Pressure sensors are used to monitor hydraulic system pressure; flow sensors are used to monitor the flow of hydraulic oil or fuel; temperature sensors are used to monitor the temperature of the engine and hydraulic system; three-axis accelerometers are used to detect vehicle and engine vibrations; speed sensors are used to monitor engine speed; torque sensors are used to measure the torque output of the engine and hydraulic system; oil level sensors are used to monitor engine oil level and hydraulic oil level; oxygen sensors are used to monitor the oxygen content in the exhaust; position sensors are used to monitor the position of engine and hydraulic system components; load sensors are used to monitor the weight of the load on the boom; sound sensors are used to capture the sound of the engine and hydraulic system; GPS sensors are used to locate the vehicle position; tilt sensors are used to monitor the vehicle tilt angle;

[0019] After all sensors are confirmed, the target sensor data is obtained.

[0020] Preferably, in S2, the target sensors are respectively subjected to functional testing according to the type of each sensor, including:

[0021] Perform functional testing on each confirmed target sensor data;

[0022] Pressure sensor overhaul calibration test and output signal inspection; flow sensor for flow meter inspection and signal stability test; temperature sensor for temperature difference test and response time test; three-axis accelerometer for static test and dynamic test; speed sensor for speed matching and signal stability test; torque sensor for torque standard test and output signal inspection; oil level sensor for oil level simulation test; oxygen sensor for oxygen concentration test and response time test; position sensor for position matching and repeatability test; load sensor for loading test; sound sensor for sound source test and signal analysis; GPS sensor for positioning accuracy test and signal stability test; tilt sensor for tilt angle test;

[0023] The sensors that fail the functional test are repaired or replaced until all the tests are passed. After all the tests are passed, the standard sensor data is obtained.

[0024] Preferably, in S3, the standard sensor data is deployed around the engine, including:

[0025] Pressure sensors are installed on hydraulic lines or pressure points; flow sensors are installed between the air filter and the throttle; temperature sensors are installed in the engine cooling circuit; three-axis accelerometers are installed on engine brackets or near the engine in the vehicle; speed sensors are installed near the crankshaft or camshaft; torque sensors are installed on the engine output shaft or the actuator of the hydraulic system; oil level sensors are installed in the oil tank or hydraulic oil tank; oxygen sensors are installed in the exhaust pipe; position sensors are installed on the throttle; load sensors are installed on the engine bracket or suspension; sound sensors are installed in the engine compartment or near the exhaust pipe; GPS sensors are installed on the top or inside the vehicle; tilt sensors are installed inside the vehicle;

[0026] Each sensor is installed and then receives a standard deployment sensor.

[0027] Preferably, the standard deployed sensors in S4 are used to collect real-time data from the engine, including:

[0028] The pressure sensor converts the pressure value into an electrical signal by sensing the pressure change in the hydraulic pipeline or pressure point. The collection method is to continuously monitor and record the changes in the pressure value;

[0029] The flow sensor measures the voltage or frequency changes caused by the fluid flowing through the sensor magnetic field or orifice plate, and the collection method is to record the flow data in real time;

[0030] The temperature sensor senses the temperature changes in the engine cooling circuit or other parts through thermistors or thermocouples, and the collection method is to continuously read the temperature value;

[0031] The three-axis accelerometer measures the linear acceleration of the engine bracket or the position near the engine in the vehicle. The acquisition method is to continuously collect the acceleration data of the X, Y, and Z axes;

[0032] The speed sensor senses the speed of the magnetic mark or gear on the crankshaft or camshaft, and the collection method is real-time counting to generate a pulse signal proportional to the speed;

[0033] The torque sensor measures the torque on the engine output shaft or hydraulic system actuator through strain gauges or other sensing technologies. The acquisition method is to convert the torque change into an electrical signal.

[0034] The oil level sensor monitors the oil level in the oil tank or hydraulic oil tank through a float or capacitive sensor, and the oil level is converted into an electrical signal;

[0035] The oxygen sensor monitors emissions by measuring changes in oxygen concentration in the exhaust pipe, and the collection method is to read the oxygen concentration signal in real time;

[0036] Position sensors monitor the position of the throttle or other movable parts by using a potentiometer or Hall effect sensor to convert the position change into an electrical signal.

[0037] The load sensor measures the force or pressure on the engine mount or suspension by converting the load into an electrical signal;

[0038] The sound sensor captures the sound inside the engine compartment or near the exhaust pipe through a microphone by converting the sound waves into electrical signals;

[0039] The GPS sensor determines the vehicle's location by receiving satellite signals. The collection method is to receive GPS data in real time, including longitude, latitude and altitude;

[0040] The tilt sensor monitors the tilt angle by detecting the change in the direction of gravity inside the vehicle. The acquisition method is to convert the tilt angle into an electrical signal.

[0041] The sensor will obtain engine monitoring data after completing real-time data collection.

[0042] Preferably, data preprocessing is performed on the engine monitoring data collected in real time in S5, including:

[0043] The engine monitoring data is received and the data is cleaned after receiving, wherein the data cleaning includes removing invalid data and filling missing data;

[0044] After data cleaning is completed, data standardization is performed. Data standardization is to scale the engine monitoring data to a fixed range, usually between 0 and 1;

[0045] After the data standardization is completed, data smoothing is performed, and the data smoothing is to filter the data using a low-pass filter;

[0046] After the data is smoothed, time series processing is performed. The data series processing is to perform time series analysis on the time data in the engine monitoring data;

[0047] After time series processing, the target monitoring data is obtained.

[0048] Preferably, in S6, feature information is extracted from the target monitoring data, and a multimodal recognition model is constructed based on the extracted feature information, including:

[0049] Extracting feature data from target monitoring data, the extraction methods include time domain feature extraction, frequency domain feature extraction and time-frequency feature extraction;

[0050] After extraction, the average pressure, pressure fluctuation range and pressure change rate in the pressure sensor monitoring data are obtained; the average flow, flow fluctuation and flow peak in the flow sensor monitoring data; the average temperature, temperature change rate and temperature fluctuation range in the temperature sensor monitoring data; the vibration amplitude, frequency component and energy spectrum in the three-axis accelerometer monitoring data; the average speed, speed fluctuation and speed change rate in the speed sensor monitoring data; the average torque, torque fluctuation and torque change rate in the torque sensor monitoring data; the oil level height and oil level change rate in the oil level sensor monitoring data; the average oxygen concentration value and oxygen concentration fluctuation in the oxygen sensor monitoring data; the position change range and position change frequency in the position sensor monitoring data; the average load, load fluctuation and load change rate in the load sensor monitoring data; the sound intensity and frequency component in the sound sensor monitoring data; the position coordinates, moving speed and moving direction in the GPS sensor monitoring data; the average tilt angle and tilt angle change rate in the tilt sensor monitoring data.

[0051] Preferably, in S6, extracting feature information from the target monitoring data and constructing a multimodal recognition model for the extracted feature information further includes:

[0052] The extracted feature data of each sensor is used to construct a multimodal recognition model;

[0053] Among them, the feature data of each sensor extracted is first subjected to dimensionality reduction processing, and after the dimensionality reduction processing, the neural network model is used for model training;

[0054] After the model training is completed, the cross-validation method is used to verify the model;

[0055] After the model verification is completed, a multi-modal transport recognition model of the extracted feature data of each sensor is obtained;

[0056] The engine status is confirmed according to the multi-modal operation recognition model, and the engine status includes stationary, idling, working and driving;

[0057] Match the characteristic data of each sensor in the multimodal operation recognition model with the engine state threshold, and judge the engine state according to the corresponding result;

[0058] After the engine status is confirmed, the engine model data is obtained;

[0059] The engine model data is subjected to deep learning processing. The deep learning processing automatically extracts high-level features from the engine model data through a recurrent neural network deep learning model. After the high-level feature extraction is completed, the engine physical model data is obtained.

[0060] Use the federated learning model to train the local model of each sensor data, and iterate the local model based on the amount of data. Then integrate the model of each sensor into a global model, and obtain the engine drive model data after the integration is completed.

[0061] The engine physical model data and the engine driving model data are fused by using the model stacking method, and the model fusion strategy and parameters are adjusted by the optimization algorithm. After the model fusion is completed, the engine fusion model is obtained;

[0062] And the engine fusion model is annotated as engine multimodal model data;

[0063] Collect and preprocess the time series data in the engine multimodal model data;

[0064] Retrieve historical data from the database to train the model data, and use the time series prediction model to train the engine multimodal model data;

[0065] Use the engine multimodal model data after model training to predict the remaining service life of equipment components, and use the predicted remaining service life of equipment components to perform equipment component prediction and generate maintenance reports;

[0066] Finally, the engine multimodal model data and equipment component prediction and maintenance reports are transmitted to the display terminal for display.

[0067] Engine multi-modal operation recognition equipment based on multi-dimensional Internet of Things, including:

[0068] Data acquisition equipment, data processing equipment and communication equipment;

[0069] The data acquisition equipment includes sensors and data acquisition cards;

[0070] Data processing equipment includes computers and storage devices;

[0071] Communication equipment includes network equipment and IoT gateways.

[0072] Engine multimodal transport identification medium based on multi-dimensional Internet of Things, including:

[0073] A storage medium stores a computer program, which, when read and executed, implements a method for calculating ship operation hours.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] 1. Various types of sensors can comprehensively monitor various parameters of the engine and its hydraulic system. This comprehensive monitoring helps to obtain multi-dimensional data of engine operation and provide a rich information basis for subsequent operation status identification. By conducting detailed functional testing on each sensor, it can be ensured that the collected sensor data is accurate and reliable. This helps to improve the overall accuracy of the engine multi-modal operation identification method, because data quality is a key factor in the effectiveness of the identification algorithm.

[0076] 2. Through different types of sensors, multiple key parameters in engine operation can be fully covered. This multi-dimensional data collection provides a rich information basis for the multi-modal operation identification of the engine. By installing a variety of different types of sensors, multi-dimensional data during the engine operation process can be collected. The fusion analysis of multi-dimensional data can more comprehensively reflect the working status of the engine, thereby improving the accuracy and reliability of identification.

[0077] 3. Rich feature data is extracted from multiple sensors, covering time domain, frequency domain and time-frequency features, ensuring the comprehensiveness and accuracy of the data. Through the construction of a multimodal motion recognition model, the data of multiple sensors can be comprehensively considered to improve the accuracy and reliability of engine state recognition. The neural network model is used for training and the cross-validation method is used for model verification to ensure the stability and generalization ability of the model. The feature data of the sensor can be extracted and analyzed in real time, and the engine state can be quickly judged based on the multimodal motion recognition model, thereby realizing real-time monitoring and early warning of the engine state. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1It is a schematic diagram of the engine multi-modal operation identification steps of the present invention;

[0079] Figure 2 It is a schematic diagram of the engine multi-modal operation identification process of the present invention. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0081] In order to solve the problem that in the prior art, when the engine is identified and monitored, the function of each sensor is not fully utilized to conduct a more comprehensive monitoring, resulting in inaccurate monitoring data, please refer to Figure 1 and Figure 2 , this embodiment provides the following technical solutions:

[0082] The engine multimodal operation identification method based on multidimensional Internet of Things includes the following steps:

[0083] S1: Engine sensor confirmation: Different sensors used to monitor the working status of the engine are confirmed according to the working nature of the engine, and the confirmed sensors are marked as target sensor data;

[0084] Among them, high-precision sensors can monitor various parameters of the engine in real time and transmit data quickly;

[0085] S2: Engine sensor detection: Perform functional detection on the target sensors according to the type of each sensor. After the functional detection is completed and qualified, it will be marked as standard sensor data;

[0086] Among them, through comprehensive functional testing and calibration, it can be ensured that the sensor can work normally under various working conditions;

[0087] S3: Sensor deployment: deploy standard sensor data around the engine, and obtain standard deployed sensors after deployment;

[0088] Among them, by installing a variety of different types of sensors, multi-dimensional data during the operation of the engine can be collected;

[0089] S4: Sensor data monitoring: Standard deployment of sensors to collect real-time data from the engine, and mark the real-time collected data as engine monitoring data;

[0090] Among them, multi-dimensional data collection provides a rich information basis for the multi-modal operation identification of the engine;

[0091] S5: Monitoring data processing: preprocessing the real-time collected engine monitoring data to obtain target monitoring data;

[0092] Among them, data cleaning can ensure the integrity of the data set, making subsequent data analysis and processing more reliable;

[0093] S6: Monitoring data model construction: feature information is extracted from the target monitoring data, and a multi-modal operation recognition model is constructed based on the extracted feature information. After the multi-modal operation recognition model is constructed, the engine multi-modal model data is obtained;

[0094] Among them, through the construction of a multimodal operation recognition model, the data of multiple sensors can be comprehensively considered to improve the accuracy and reliability of engine status recognition.

[0095] Confirm the different sensors used to monitor the working status of the engine in S1 according to the working nature of the engine, including:

[0096] Sensors include pressure sensors, flow sensors, temperature sensors, three-axis accelerometers, speed sensors, torque sensors, oil level sensors, oxygen sensors, position sensors, load sensors, sound sensors, GPS sensors, and tilt sensors;

[0097] Pressure sensors are used to monitor hydraulic system pressure; flow sensors are used to monitor the flow of hydraulic oil or fuel; temperature sensors are used to monitor the temperature of the engine and hydraulic system; three-axis accelerometers are used to detect vehicle and engine vibrations; speed sensors are used to monitor engine speed; torque sensors are used to measure the torque output of the engine and hydraulic system; oil level sensors are used to monitor engine oil level and hydraulic oil level; oxygen sensors are used to monitor the oxygen content in the exhaust; position sensors are used to monitor the position of engine and hydraulic system components; load sensors are used to monitor the weight of the load on the boom; sound sensors are used to capture the sound of the engine and hydraulic system; GPS sensors are used to locate the vehicle position; tilt sensors are used to monitor the vehicle tilt angle;

[0098] After all sensors are confirmed, the target sensor data is obtained.

[0099] Specifically, various types of sensors can comprehensively monitor various parameters of the engine and its hydraulic system, such as pressure, flow, temperature, vibration, speed, torque, oil level, oxygen content, position, load, sound, positioning, and tilt angle. This comprehensive monitoring helps to obtain multi-dimensional data of engine operation and provide a rich information basis for subsequent operation status identification. Through high-precision sensors, such as three-axis accelerometers and torque sensors, it is possible to accurately capture the tiny vibrations and torque changes of the engine and hydraulic system, thereby achieving high-precision identification of the engine operation status. This is of great significance for timely detection of potential faults and abnormal operating conditions. The sensor can monitor various parameters of the engine in real time and transmit data quickly. This real-time performance helps to timely detect and respond to abnormal conditions of the engine and improve the efficiency of fault warning and processing. Combined with multi-dimensional Internet of Things technology, the data collected by the sensor can be processed and analyzed in real time, and the engine operation status can be intelligently identified and predicted through the algorithm model. This intelligent method can not only improve the accuracy of identification, but also realize early warning and intelligent maintenance of faults. Through multi-dimensional Internet of Things technology, data from different sensors can be fused and processed to form a more comprehensive and accurate description of the engine operation status. This data fusion approach helps discover complex faults and anomalies that are difficult to capture with a single sensor.

[0100] In S2, the target sensors are tested for their functions according to their types, including:

[0101] Perform functional testing on each confirmed target sensor data;

[0102] Pressure sensor overhaul calibration test and output signal inspection; flow sensor for flow meter inspection and signal stability test; temperature sensor for temperature difference test and response time test; three-axis accelerometer for static test and dynamic test; speed sensor for speed matching and signal stability test; torque sensor for torque standard test and output signal inspection; oil level sensor for oil level simulation test; oxygen sensor for oxygen concentration test and response time test; position sensor for position matching and repeatability test; load sensor for loading test; sound sensor for sound source test and signal analysis; GPS sensor for positioning accuracy test and signal stability test; tilt sensor for tilt angle test;

[0103] The sensors that fail the functional test are repaired or replaced until all the tests are passed. After all the tests are passed, the standard sensor data is obtained.

[0104] Specifically, by performing detailed functional testing on each sensor, it is possible to ensure that the collected sensor data is accurate and reliable. This helps to improve the overall accuracy of the engine multimodal operation identification method, because data quality is a key factor in the effectiveness of the identification algorithm. Repairing or replacing sensors with unqualified functions can prevent system false alarms or missed alarms caused by sensor failures, thereby enhancing the reliability of the entire Internet of Things system. Different types of sensors (such as pressure, flow, temperature, acceleration, etc.) can be ensured to have stable output signals that meet standards after special calibration and testing, which helps to optimize the performance of the engine multimodal operation identification method and improve the accuracy and speed of identification. Regular sensor function testing can timely discover and solve potential problems, avoid small problems from gradually evolving into big problems, thereby reducing later maintenance costs, and ensure the standardization and consistency of sensor data, which helps to more easily integrate new sensors or improve existing identification algorithms when expanding the system in the future. Through comprehensive functional testing and calibration, it can be ensured that the sensors can work properly under various working conditions, thereby improving the robustness of the entire Internet of Things system and making it more adaptable to complex and changeable engine operating conditions.

[0105] In order to solve the problem in the prior art that the position of each sensor is not precisely deployed before the sensor monitors the engine, thus causing the sensor monitoring to be invalid, please refer to Figure 1 and Figure 2 , this embodiment provides the following technical solutions:

[0106] Standard sensor data is deployed around the engine in S3, including:

[0107] Pressure sensors are installed on hydraulic lines or pressure points; flow sensors are installed between the air filter and the throttle; temperature sensors are installed in the engine cooling circuit; three-axis accelerometers are installed on engine brackets or near the engine in the vehicle; speed sensors are installed near the crankshaft or camshaft; torque sensors are installed on the engine output shaft or the actuator of the hydraulic system; oil level sensors are installed in the oil tank or hydraulic oil tank; oxygen sensors are installed in the exhaust pipe; position sensors are installed on the throttle; load sensors are installed on the engine bracket or suspension; sound sensors are installed in the engine compartment or near the exhaust pipe; GPS sensors are installed on the top or inside the vehicle; tilt sensors are installed inside the vehicle;

[0108] Each sensor is installed and then receives a standard deployment sensor.

[0109] Specifically, by installing a variety of different types of sensors, multi-dimensional data can be collected during the operation of the engine, such as pressure, flow, temperature, acceleration, speed, torque, oil level, oxygen content, position, load, sound, etc. The fusion analysis of multi-dimensional data can more comprehensively reflect the working status of the engine, thereby improving the accuracy and reliability of identification. By analyzing the sensor data, potential engine failures such as wear, blockage, leakage, etc. can be predicted. The intelligent control system based on sensor data can be dynamically adjusted according to the actual working status of the engine to achieve more precise control. The sensor can monitor the operating status of the engine and promptly discover potential safety hazards, such as cylinder explosion, cylinder pulling and other serious failures. Some sensors support remote communication functions, such as torque sensors, GPS sensors, etc., which can realize remote monitoring and data transmission, which enables maintenance personnel to understand the working status of the engine remotely and formulate maintenance plans in time. The oxygen sensor can monitor the oxygen content in the exhaust pipe and provide feedback to the engine control system to optimize the combustion process and reduce emission pollution. Other sensors such as temperature sensors and flow sensors can also provide data support for environmental protection control strategies.

[0110] Standard sensors deployed in S4 collect real-time data from the engine, including:

[0111] The pressure sensor converts the pressure value into an electrical signal by sensing the pressure change in the hydraulic pipeline or pressure point. The collection method is to continuously monitor and record the changes in the pressure value;

[0112] The flow sensor measures the voltage or frequency changes caused by the fluid flowing through the sensor magnetic field or orifice plate, and the collection method is to record the flow data in real time;

[0113] The temperature sensor senses the temperature changes in the engine cooling circuit or other parts through thermistors or thermocouples, and the collection method is to continuously read the temperature value;

[0114] The three-axis accelerometer measures the linear acceleration of the engine bracket or the position near the engine in the vehicle. The acquisition method is to continuously collect the acceleration data of the X, Y, and Z axes;

[0115] The speed sensor senses the speed of the magnetic mark or gear on the crankshaft or camshaft, and the collection method is real-time counting to generate a pulse signal proportional to the speed;

[0116] The torque sensor measures the torque on the engine output shaft or hydraulic system actuator through strain gauges or other sensing technologies. The acquisition method is to convert the torque change into an electrical signal.

[0117] The oil level sensor monitors the oil level in the oil tank or hydraulic oil tank through a float or capacitive sensor, and the oil level is converted into an electrical signal;

[0118] The oxygen sensor monitors emissions by measuring changes in oxygen concentration in the exhaust pipe, and the collection method is to read the oxygen concentration signal in real time;

[0119] Position sensors monitor the position of the throttle or other movable parts by using a potentiometer or Hall effect sensor to convert the position change into an electrical signal.

[0120] The load sensor measures the force or pressure on the engine mount or suspension by converting the load into an electrical signal;

[0121] The sound sensor captures the sound inside the engine compartment or near the exhaust pipe through a microphone by converting the sound waves into electrical signals;

[0122] The GPS sensor determines the vehicle's location by receiving satellite signals. The collection method is to receive GPS data in real time, including longitude, latitude and altitude;

[0123] The tilt sensor monitors the tilt angle by detecting the change in the direction of gravity inside the vehicle. The acquisition method is to convert the tilt angle into an electrical signal;

[0124] The sensor will obtain engine monitoring data after completing real-time data collection.

[0125] Specifically, all sensors have real-time or continuous data collection capabilities, which can instantly reflect the operating status of the engine and its related systems. This is crucial for timely detection and handling of potential faults. Through different types of sensors, multiple key parameters in engine operation can be fully covered, such as pressure, flow, temperature, acceleration, speed, torque, oil level, oxygen concentration, position, load, sound, position and tilt angle. This multi-dimensional data collection provides a rich information basis for the multi-modal operation identification of the engine. After long-term development, sensor technology has already achieved high measurement accuracy and reliability. For example, pressure sensors, flow sensors, and temperature sensors can accurately reflect the actual status of the engine and its related systems. The multi-dimensional data collected in real time provides rich input for data analysis and machine learning algorithms, which helps to more accurately identify the engine's operating mode and potential faults. This helps to achieve data-based intelligent decision-making and predictive maintenance.

[0126] In order to solve the problem in the prior art that the engine monitoring data is not effectively processed and analyzed, resulting in the inability to obtain the engine driving status in a timely manner, please refer to Figure 1 and Figure 2 , this embodiment provides the following technical solutions:

[0127] Data preprocessing is performed on the engine monitoring data collected in real time in S5, including:

[0128] The engine monitoring data is received and the data is cleaned after receiving, wherein the data cleaning includes removing invalid data and filling missing data;

[0129] After data cleaning is completed, data standardization is performed. Data standardization is to scale the engine monitoring data to a fixed range, usually between 0 and 1;

[0130] After the data standardization is completed, data smoothing is performed, and the data smoothing is to filter the data using a low-pass filter;

[0131] After the data is smoothed, time series processing is performed. The data series processing is to perform time series analysis on the time data in the engine monitoring data;

[0132] After time series processing, the target monitoring data is obtained.

[0133] Specifically, by removing invalid data and filling missing data, data cleaning can significantly improve the quality of engine monitoring data. This helps to reduce misjudgments caused by data errors or inaccuracies, thereby improving the accuracy of multimodal operation recognition. Data cleaning can ensure the integrity of the data set, making subsequent data analysis and processing more reliable. In a multidimensional Internet of Things environment, engine monitoring data often comes from multiple different sources and formats. Data cleaning can integrate these data to form a consistent and complete data set. By eliminating redundant information and noise, data cleaning can reduce the size of the data set, thereby reducing the computing resource requirements for subsequent data analysis and processing. This helps to improve the operating efficiency of the entire system. Using a low-pass filter to filter the data can effectively eliminate high-frequency noise and interference in the engine monitoring data. This helps to improve the signal-to-noise ratio of the data, making multimodal operation recognition more accurate and reliable. Data smoothing can make the data curve smoother, which helps to identify trends and patterns in the data. This is particularly important for multimodal operation recognition methods based on time series analysis, which can capture the temporal characteristics of engine monitoring data, such as periodicity and trend. These characteristics are of great significance for identifying the operating status and failure mode of the engine. Through time series analysis, the future operating status of the engine can be predicted. This helps to discover potential failure risks in advance and take corresponding preventive measures.

[0134] In S6, feature information of target monitoring data is extracted, and a multimodal recognition model is constructed based on the extracted feature information, including:

[0135] Extracting feature data from target monitoring data, the extraction methods include time domain feature extraction, frequency domain feature extraction and time-frequency feature extraction;

[0136] After extraction, the average pressure, pressure fluctuation range and pressure change rate in the pressure sensor monitoring data are obtained; the average flow, flow fluctuation and flow peak in the flow sensor monitoring data; the average temperature, temperature change rate and temperature fluctuation range in the temperature sensor monitoring data; the vibration amplitude, frequency component and energy spectrum in the three-axis accelerometer monitoring data; the average speed, speed fluctuation and speed change rate in the speed sensor monitoring data; the average torque, torque fluctuation and torque change rate in the torque sensor monitoring data; the oil level height and oil level change rate in the oil level sensor monitoring data; the average oxygen concentration value and oxygen concentration fluctuation in the oxygen sensor monitoring data; the position change range and position change frequency in the position sensor monitoring data; the average load, load fluctuation and load change rate in the load sensor monitoring data; the sound intensity and frequency component in the sound sensor monitoring data; the position coordinates, moving speed and moving direction in the GPS sensor monitoring data; the average tilt angle and tilt angle change rate in the tilt sensor monitoring data.

[0137] The extracted feature data of each sensor is used to construct a multimodal recognition model;

[0138] Among them, the feature data of each sensor extracted is first subjected to dimensionality reduction processing, and after the dimensionality reduction processing, the neural network model is used for model training;

[0139] After the model training is completed, the cross-validation method is used to verify the model;

[0140] After the model verification is completed, a multi-modal transport recognition model of the extracted feature data of each sensor is obtained;

[0141] The engine status is confirmed according to the multi-modal operation recognition model, and the engine status includes stationary, idling, working and driving;

[0142] Match the characteristic data of each sensor in the multimodal operation recognition model with the engine state threshold, and judge the engine state according to the corresponding result;

[0143] After the engine status is confirmed, the engine model data is obtained;

[0144] The engine model data is subjected to deep learning processing. The deep learning processing automatically extracts high-level features from the engine model data through a recurrent neural network deep learning model. After the high-level feature extraction is completed, the engine physical model data is obtained.

[0145] Use the federated learning model to train the local model of each sensor data, and iterate the local model based on the amount of data. Then integrate the model of each sensor into a global model, and obtain the engine drive model data after the integration is completed.

[0146] The engine physical model data and the engine driving model data are fused by using the model stacking method, and the model fusion strategy and parameters are adjusted by the optimization algorithm. After the model fusion is completed, the engine fusion model is obtained;

[0147] And the engine fusion model is annotated as engine multimodal model data;

[0148] Collect and preprocess the time series data in the engine multimodal model data;

[0149] Retrieve historical data from the database to train the model data, and use the time series prediction model to train the engine multimodal model data;

[0150] Use the engine multimodal model data after model training to predict the remaining service life of equipment components, and use the predicted remaining service life of equipment components to perform equipment component prediction and generate maintenance reports;

[0151] Finally, the engine multimodal model data and equipment component prediction and maintenance reports are transmitted to the display terminal for display.

[0152] Specifically, rich feature data are extracted from multiple sensors (including pressure, flow, temperature, triaxial accelerometer, speed, torque, oil level, oxygen sensor, position, load, sound and GPS, etc.), covering time domain, frequency domain and time-frequency features, ensuring the comprehensiveness and accuracy of the data. Through the construction of the multimodal motion recognition model, the data of multiple sensors can be comprehensively considered to improve the accuracy and reliability of engine state recognition. The neural network model is used for training and the cross-validation method is used for model verification to ensure the stability and generalization ability of the model. The feature data of the sensor can be extracted and analyzed in real time, and the engine state (stationary, idling, working and driving) can be quickly judged according to the multimodal motion recognition model, so as to realize real-time monitoring and early warning of the engine state. Before model training, the feature data of each sensor extracted is subjected to dimensionality reduction processing, which can reduce the complexity of the data, reduce the amount of calculation, improve the training efficiency and recognition speed of the model, and can easily integrate more sensors and data sources to expand the monitoring range and recognition ability of the system. At the same time, the engine multimodal model data is transmitted to the display terminal for display, which is convenient for users to intuitively understand the state and performance of the engine. By corresponding the characteristic data of each sensor to the engine state threshold, it can be flexibly adjusted according to different engine types and working conditions, with good adaptability and versatility. The deep learning model of the recurrent neural network can automatically extract high-level features from the engine model data, avoiding the tedious process of manually designing and selecting features in traditional methods, and improving the accuracy and efficiency of feature extraction. Through the federated learning model, the data of each sensor can be trained locally to protect the privacy of the data. At the same time, with the iteration of the number of data, the local model is continuously updated, and the performance and accuracy of the overall model are improved through global model integration. The model stacking method is used to fuse the engine physical model data and the engine drive model data, and the fusion strategy and parameters are adjusted through the optimization algorithm, so that the fused engine fusion model has stronger prediction ability and robustness. The time series data in the engine multimodal model data is collected and preprocessed, providing a reliable data foundation for the subsequent time series prediction model. This helps capture the dynamic characteristics of the engine and improve the accuracy of prediction. The trained engine multimodal model data can accurately predict the remaining service life of equipment components, providing an important reference for equipment maintenance and management. It makes full use of multi-dimensional Internet of Things technology and integrates multiple sensors and data sources to achieve comprehensive monitoring and identification of engine operating status. This helps to detect potential faults in a timely manner and improve the safety and reliability of equipment.

[0153] Engine multi-modal operation recognition equipment based on multi-dimensional Internet of Things, including:

[0154] Data acquisition equipment, data processing equipment and communication equipment;

[0155] The data acquisition equipment includes sensors and data acquisition cards;

[0156] Data processing equipment includes computers and storage devices;

[0157] Communication equipment includes network equipment and IoT gateways.

[0158] A storage medium having a computer program stored thereon, wherein when the computer program is read and executed, a method for calculating ship operation hours is implemented:

[0159] Storage medium, the storage medium is used to store engine multimodal model data, real-time data monitored by sensors, and program instructions.

[0160] Specifically, through multi-dimensional Internet of Things technology, multiple sensors (such as temperature sensors, pressure sensors, vibration sensors, etc.) can be integrated to achieve comprehensive monitoring of the engine's operating status. These sensors can capture multiple modal information of the engine under different operating conditions, such as temperature, pressure, vibration, sound, etc., and provide a rich and accurate data source for subsequent data processing and identification. Data processing equipment includes high-performance computers and storage devices, which can efficiently process the collected multi-dimensional data. Through advanced algorithms and models, computers can quickly analyze, process and identify data to achieve real-time monitoring and early warning of the engine's operating status. Based on the engine multi-modal model data stored in the storage medium and the real-time data monitored by the sensor, the computer can use machine learning, deep learning and other artificial intelligence technologies to intelligently identify and diagnose the engine's operating status. This can not only improve the accuracy of identification, but also detect potential faults in a timely manner, providing strong support for engine maintenance and management.

[0161] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0162] While the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that many changes, modifications, substitutions and variations can be made to the embodiments without departing from the principles and spirit of the invention.

Claims

1. An engine multimodal operation identification method based on multidimensional Internet of Things, characterized in that: The steps include: S1: Engine sensor confirmation: Different sensors used to monitor the working status of the engine are confirmed according to the working nature of the engine, and the confirmed sensors are marked as target sensor data; S2: Engine sensor detection: Perform functional detection on the target sensors according to the type of each sensor. After the functional detection is completed and qualified, it will be marked as standard sensor data; S3: Sensor deployment: deploy standard sensor data around the engine, and obtain standard deployed sensors after deployment; S4: Sensor data monitoring: Standard deployment of sensors to collect real-time data from the engine, and mark the real-time collected data as engine monitoring data; S5: Monitoring data processing: preprocessing the real-time collected engine monitoring data to obtain target monitoring data; S6: Monitoring data model construction: feature information is extracted from the target monitoring data, and a multimodal operation recognition model is constructed based on the extracted feature information. After the multimodal operation recognition model is constructed, the engine multimodal model data is obtained.

2. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 1 is characterized in that: Confirm the different sensors used to monitor the working status of the engine in S1 according to the working nature of the engine, including: Sensors include pressure sensors, flow sensors, temperature sensors, three-axis accelerometers, speed sensors, torque sensors, oil level sensors, oxygen sensors, position sensors, load sensors, sound sensors, GPS sensors, and tilt sensors; Pressure sensors are used to monitor hydraulic system pressure; flow sensors are used to monitor the flow of hydraulic oil or fuel; temperature sensors are used to monitor the temperature of the engine and hydraulic system; three-axis accelerometers are used to detect vehicle and engine vibrations; speed sensors are used to monitor engine speed; torque sensors are used to measure the torque output of the engine and hydraulic system; oil level sensors are used to monitor engine oil level and hydraulic oil level; oxygen sensors are used to monitor the oxygen content in the exhaust; position sensors are used to monitor the position of engine and hydraulic system components; load sensors are used to monitor the weight of the load on the boom; sound sensors are used to capture the sound of the engine and hydraulic system; GPS sensors are used to locate the vehicle position; tilt sensors are used to monitor the vehicle tilt angle; After all sensors are confirmed, the target sensor data is obtained.

3. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 1 is characterized in that: In S2, the target sensors are tested for their functions according to their types, including: Perform functional testing on each confirmed target sensor data; Pressure sensor overhaul calibration test and output signal inspection; flow sensor for flow meter inspection and signal stability test; temperature sensor for temperature difference test and response time test; three-axis accelerometer for static test and dynamic test; speed sensor for speed matching and signal stability test; torque sensor for torque standard test and output signal inspection; oil level sensor for oil level simulation test; oxygen sensor for oxygen concentration test and response time test; position sensor for position matching and repeatability test; load sensor for loading test; sound sensor for sound source test and signal analysis; GPS sensor for positioning accuracy test and signal stability test; tilt sensor for tilt angle test; The sensors that fail the functional test are repaired or replaced until all the tests are passed. After all the tests are passed, the standard sensor data is obtained.

4. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 2 is characterized in that: Standard sensor data is deployed around the engine in S3, including: Pressure sensors are installed on hydraulic lines or pressure points; flow sensors are installed between the air filter and the throttle; temperature sensors are installed in the engine cooling circuit; three-axis accelerometers are installed on engine brackets or near the engine in the vehicle; speed sensors are installed near the crankshaft or camshaft; torque sensors are installed on the engine output shaft or the actuator of the hydraulic system; oil level sensors are installed in the oil tank or hydraulic oil tank; oxygen sensors are installed in the exhaust pipe; position sensors are installed on the throttle; load sensors are installed on the engine bracket or suspension; sound sensors are installed in the engine compartment or near the exhaust pipe; GPS sensors are installed on the top or inside the vehicle; tilt sensors are installed inside the vehicle; Each sensor is installed and then receives a standard deployment sensor.

5. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 3 is characterized in that: Standard sensors deployed in S4 collect real-time data from the engine, including: The pressure sensor converts the pressure value into an electrical signal by sensing the pressure change in the hydraulic pipeline or pressure point. The collection method is to continuously monitor and record the changes in the pressure value; The flow sensor measures the voltage or frequency changes caused by the fluid flowing through the sensor magnetic field or orifice plate, and the collection method is to record the flow data in real time; The temperature sensor senses the temperature changes in the engine cooling circuit or other parts through thermistors or thermocouples, and the collection method is to continuously read the temperature value; The three-axis accelerometer measures the linear acceleration of the engine bracket or the position near the engine in the vehicle. The acquisition method is to continuously collect the acceleration data of the X, Y, and Z axes; The speed sensor senses the speed of the magnetic mark or gear on the crankshaft or camshaft, and the collection method is real-time counting to generate a pulse signal proportional to the speed; The torque sensor measures the torque on the engine output shaft or hydraulic system actuator through strain gauges or other sensing technologies. The acquisition method is to convert the torque change into an electrical signal. The oil level sensor monitors the oil level in the oil tank or hydraulic oil tank through a float or capacitive sensor, and the oil level is converted into an electrical signal; The oxygen sensor monitors emissions by measuring changes in oxygen concentration in the exhaust pipe, and the collection method is to read the oxygen concentration signal in real time; Position sensors monitor the position of the throttle or other movable parts by using a potentiometer or Hall effect sensor to convert the position change into an electrical signal. The load sensor measures the force or pressure on the engine mount or suspension by converting the load into an electrical signal; The sound sensor captures the sound inside the engine compartment or near the exhaust pipe through a microphone by converting the sound waves into electrical signals; The GPS sensor determines the vehicle's location by receiving satellite signals. The collection method is to receive GPS data in real time, including longitude, latitude and altitude; The tilt sensor monitors the tilt angle by detecting the change in the direction of gravity inside the vehicle. The acquisition method is to convert the tilt angle into an electrical signal; The sensor will obtain engine monitoring data after completing real-time data collection.

6. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 5 is characterized in that: Data preprocessing is performed on the engine monitoring data collected in real time in S5, including: The engine monitoring data is received and the data is cleaned after receiving, wherein the data cleaning includes removing invalid data and filling missing data; After data cleaning is completed, data standardization is performed. Data standardization is to scale the engine monitoring data to a fixed range, usually between 0 and 1; After the data standardization is completed, data smoothing is performed, and the data smoothing is to filter the data using a low-pass filter; After the data is smoothed, time series processing is performed. The data series processing is to perform time series analysis on the time data in the engine monitoring data; After time series processing, the target monitoring data is obtained.

7. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 6 is characterized in that: In S6, feature information of target monitoring data is extracted, and a multimodal recognition model is constructed based on the extracted feature information, including: Extracting feature data from target monitoring data, the extraction methods include time domain feature extraction, frequency domain feature extraction and time-frequency feature extraction; After extraction, the average pressure, pressure fluctuation range and pressure change rate in the pressure sensor monitoring data are obtained; the average flow, flow fluctuation and flow peak in the flow sensor monitoring data; the average temperature, temperature change rate and temperature fluctuation range in the temperature sensor monitoring data; the vibration amplitude, frequency component and energy spectrum in the three-axis accelerometer monitoring data; the average speed, speed fluctuation and speed change rate in the speed sensor monitoring data; the average torque, torque fluctuation and torque change rate in the torque sensor monitoring data; the oil level height and oil level change rate in the oil level sensor monitoring data; the average oxygen concentration value and oxygen concentration fluctuation in the oxygen sensor monitoring data; the position change range and position change frequency in the position sensor monitoring data; the average load, load fluctuation and load change rate in the load sensor monitoring data; the sound intensity and frequency component in the sound sensor monitoring data; the position coordinates, moving speed and moving direction in the GPS sensor monitoring data; the average tilt angle and tilt angle change rate in the tilt sensor monitoring data.

8. The engine multimodal operation identification method based on multidimensional Internet of Things according to claim 6 is characterized in that: In S6, feature information of target monitoring data is extracted, and a multi-modal recognition model is constructed based on the extracted feature information, which also includes: The extracted feature data of each sensor is used to construct a multimodal recognition model; Among them, the feature data of each sensor extracted is first subjected to dimensionality reduction processing, and after the dimensionality reduction processing, the neural network model is used for model training; After the model training is completed, the cross-validation method is used to verify the model; After the model verification is completed, a multi-modal transport recognition model of the extracted feature data of each sensor is obtained; The engine status is confirmed according to the multi-modal operation recognition model, and the engine status includes stationary, idling, working and driving; Match the characteristic data of each sensor in the multimodal operation recognition model with the engine state threshold, and judge the engine state according to the corresponding result; After the engine status is confirmed, the engine model data is obtained; The engine model data is subjected to deep learning processing. The deep learning processing automatically extracts high-level features from the engine model data through a recurrent neural network deep learning model. After the high-level feature extraction is completed, the engine physical model data is obtained. Use the federated learning model to train the local model of each sensor data, and iterate the local model based on the amount of data. Then integrate the model of each sensor into a global model, and obtain the engine drive model data after the integration is completed. The engine physical model data and the engine driving model data are fused by using the model stacking method, and the model fusion strategy and parameters are adjusted by the optimization algorithm. After the model fusion is completed, the engine fusion model is obtained; And the engine fusion model is annotated as engine multimodal model data; Collect and preprocess the time series data in the engine multimodal model data; Retrieve historical data from the database to train the model data, and use the time series prediction model to train the engine multimodal model data; Use the engine multimodal model data after model training to predict the remaining service life of equipment components, and use the predicted remaining service life of equipment components to perform equipment component prediction and maintenance report generation; Finally, the engine multimodal model data and equipment component prediction and maintenance reports are transmitted to the display terminal for display.

9. An engine multimodal operation identification device based on a multidimensional Internet of Things, applied in an engine multimodal operation identification method based on a multidimensional Internet of Things as claimed in claim 8, comprising: Data acquisition equipment, data processing equipment and communication equipment; The data acquisition equipment includes sensors and data acquisition cards; Data processing equipment includes computers and storage devices; Communication equipment includes network equipment and IoT gateways.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is read and executed, the method for calculating the ship operation hours described in any one of claims 1 to 8 is implemented.

Images