Vision and vibration-based multi-dimensional train derailment monitoring method

Abstract

The invention discloses a multi-dimensional train derailment monitoring method based on vision and vibration, and belongs to the field of rail transit, the method comprises the following steps: respectively collecting a wheel-rail contact area image and vibration data of each wheel in different directions, and carrying out space-time alignment on the wheel-rail contact area image and the vibration data; combining an edge detection algorithm and a Hough transform technology to extract visual features from the wheel-rail contact area image, extracting vibration features from the vibration data, and associating the visual features with the vibration features to generate a wheel flange transverse position-vibration feature mapping table; and establishing a D-S evidence theory fusion model based on the wheel flange transverse position-vibration feature mapping table, fusing the visual features and the vibration features by using the D-S evidence theory fusion model, and judging the train derailment risk according to a feature fusion result. According to the invention, deep coupling analysis of visual images and vibration data is realized, and the defects that the prior art cannot adapt to working condition changes, false alarms are easily generated and operation is affected are overcome.

Description

Technical Field

[0001] The present invention relates to the field of rail transit. Specifically, it relates to a multi-dimensional train derailment monitoring method based on vision and vibration. Background Technique

[0002] With the development of driverless technology in rail transit, derailment detection technology has become a research hotspot. At present, a contact-type obstacle and derailment detection device suitable for driverless subways has been developed, which is of great significance for improving the safety of driverless subways. It can perform emergency braking when the train derails, reducing accident injuries. At the same time, based on the existing on-line monitoring system of the running gear, the vehicle derailment monitoring function has been further expanded, and through relevant derailment monitoring algorithms, the function of predicting vehicle derailment based on vibration shock has been realized.

[0003] Derailment detection: The working principle of derailment detection usually involves monitoring the running state of the vehicle, including the running gear, stability, and comfort of the vehicle. These monitoring systems can evaluate whether the vehicle has a derailment risk through different sensors and analysis models.

[0004] The existing technologies are mainly contact detection and vibration shock detection, which have been widely applied. However, the contact type has a high false alarm rate, and the vibration shock has a high missed alarm rate. The actual operation effects of both are not very ideal. The contact-type obstacle detection can only give an alarm after derailment occurs and cannot give an early warning. The vibration shock detection is affected by the vehicle running vibration, and the possibility of false alarms is relatively high, making it impossible to effectively prevent and affecting the actual operation.

[0005] For example, Chinese invention patent CN114368411B discloses a monitoring and early warning method for train derailment safety. It uses a monitoring and early warning method based on visual behavior to analyze the pictures of the wheel-rail contact area, calculate the relative lateral displacement of the wheel-rail, and evaluate the derailment safety level according to the relative lateral displacement of the wheel-rail in the world coordinate system. However, the running environment under the vehicle is relatively harsh, which will affect the accuracy of visual detection. There are sundries interference under the vehicle, and the visual environment is relatively complex (lighting, background, etc.), resulting in a relatively high false alarm rate.

[0006] Chinese invention patent CN107521511B discloses an alarm method and system for wheel derailment. After obtaining the vibration signal, the vibration signal is processed to obtain a sample to be analyzed; time-domain identification is performed on the sample to be analyzed to obtain a time-domain derailment identification result; the corresponding spectrum is obtained by performing a fast Fourier transform on the sample to be analyzed, and frequency-domain identification is performed on the spectrum to obtain a frequency-domain derailment identification result. However, derailment vibration information generally only occurs when derailment has already occurred. Although there is a warning function, it involves train control, resulting in a relatively conservative threshold setting and false negatives; moreover, the algorithm is not mature and complex enough, which is prone to false positives. Once a false positive occurs, it will cause emergency braking, seriously affecting actual operation.

[0007] In response to the problems in the related art, no effective solution has been proposed yet. Summary of the Invention

[0008] (I) Technical Problems to be Solved In view of the deficiencies of the prior art, the present invention provides a multi-dimensional train derailment monitoring method based on vision and vibration, which has the advantage of in-depth coupling analysis of visual images and vibration data, thereby solving the problems of the prior art that it cannot adapt to working condition changes, is extremely prone to false positives, and affects operation.

[0009] (II) Technical Solutions To achieve the above-mentioned advantage of in-depth coupling analysis of visual images and vibration data, the specific technical solutions adopted by the present invention are as follows: A multi-dimensional train derailment monitoring method based on vision and vibration, the method includes: S1. Respectively collect images of the wheel-rail contact area and vibration data of each wheel in different directions, and perform spatio-temporal alignment on the images of the wheel-rail contact area and the vibration data; S2. Combine the edge detection algorithm and the Hough transform technology to extract visual features from the images of the wheel-rail contact area, and extract vibration features from the vibration data, and associate the visual features with the vibration features to generate a wheel flange lateral position-vibration feature mapping table; S3. Establish a D-S evidence theory fusion model based on the wheel flange lateral position-vibration feature mapping table, and use the D-S evidence theory fusion model to fuse the visual features and the vibration features, and determine the train derailment risk according to the feature fusion result.

[0010] Preferably, respectively collecting images of the wheel-rail contact area and vibration data of each wheel in different directions, and performing spatio-temporal alignment on the images of the wheel-rail contact area and the vibration data includes: S11. Use a video device to collect images of the wheel-rail contact area, and use an acceleration sensor to collect the three-axis acceleration of the wheel to obtain vibration data, and perform time alignment on the images of the wheel-rail contact area and the vibration data through a clock protocol; S12. Establish a vision coordinate system based on the orbital plane with the orbital center reference point as the origin, and establish a vibration coordinate system with the bogie center as the origin; S13. Map the vibration data of the vibration coordinate system to the vision coordinate system through a rigid body transformation matrix for spatial alignment.

[0011] Preferably, mapping the vibration data of the vibration coordinate system to the vision coordinate system through a rigid body transformation matrix for spatial alignment includes: S131. Use a laser rangefinder to collect the installation status of the acceleration sensor and measure the Euler angles of the vibration coordinate system relative to the vision coordinate system; S132. Calculate the rotation matrix according to the Euler angles and in combination with the rotation matrix synthesis formula, measure the coordinates of the origin of the vibration coordinate system in the vision coordinate system and determine the translation vector; S133. Convert the vibration data of the vibration coordinate system into the components of the vision coordinate system through the rotation matrix and the translation vector respectively, and perform spatial correlation with the flange position in the vision coordinate system; S134. Compare the flange position in the actual wheel-rail contact area image with the spatial distribution of the vibration data, and optimize the rotation matrix and the translation vector.

[0012] Preferably, combining the edge detection algorithm and the Hough transform technology to extract visual features from the wheel-rail contact area image, and extracting vibration features from the vibration data, and associating the visual features with the vibration features to generate a flange lateral position - vibration feature mapping table includes: S21. Divide the vision area in the vision coordinate system based on predefined division conditions, and the vision area includes a safety area, a warning area and an alarm area; S22. Combine the Canny edge detection algorithm and the Hough transform technology to extract the wheel-rail profile features in the wheel-rail contact area image in real time, and calculate the lateral offset of the center line of the wheel tread; S23. Calculate the time domain features of the acceleration signal in the vibration data, and use the adaptive time-frequency fusion fast Fourier transform algorithm to extract the frequency domain features of the acceleration signal; S24. Based on the time domain features and frequency domain features of the acceleration signal, use the adaptive empirical mode decomposition algorithm to generate a transform marginal spectrum and analyze the time-frequency features of derailment precursors; S25. Combine the wheel-rail profile features and the time-frequency features to generate a flange lateral position - vibration feature mapping table.

[0013] Preferably, using the adaptive time-frequency fusion fast Fourier transform algorithm to extract the frequency domain features of the acceleration signal includes: Preprocess the acceleration signal, retain the wheel-rail coupling related frequency bands, and window the wheel-rail coupling related frequency bands using a Hanning window function; Dynamically adjust the Fourier transform parameters for the windowed acceleration signals in different visual regions, introduce visual evidence weights in the calculation of Fourier transform parameters, and obtain the amplitude spectrum; Convert the amplitude spectrum into a single-sided spectrum, search for the peaks and amplitudes within a preset range in the single-sided spectrum, and obtain the wheel-rail coupling characteristic frequencies based on the peaks and amplitudes at each frequency point.

[0014] Preferably, based on the time-domain characteristics and frequency-domain characteristics of the acceleration signals, use the adaptive empirical mode decomposition algorithm to generate the transform marginal spectrum, and analyze the time-frequency characteristics of derailment precursors, including: S241. Dynamically adjust the empirical mode decomposition parameters of the visual region using the empirical mode decomposition algorithm to obtain a number of intrinsic mode function components; S242. Perform Hilbert transform on each intrinsic mode function component to obtain the analytic signal, and calculate the instantaneous amplitude and instantaneous frequency of the analytic signal; S243. Superimpose the distributions of the instantaneous amplitudes and instantaneous frequencies of all intrinsic mode function components over time to form the Hilbert spectrum; S244. Integrate the Hilbert spectrum along the time axis to obtain the marginal spectrum, and identify the abnormal frequency components of derailment precursors based on the marginal spectrum.

[0015] Preferably, dynamically adjust the empirical mode decomposition parameters of the visual region using the empirical mode decomposition algorithm to obtain a number of intrinsic mode function components, including: S2411. Select the decomposition method of the empirical mode decomposition algorithm based on different visual regions; S2412. Obtain the local maxima and minima of the acceleration signal, connect the maximum points to form the upper envelope line, connect the minimum points to form the lower envelope line, and fit the upper and lower envelope lines by cubic spline interpolation; S2413. Calculate the preliminary intrinsic mode function based on the fitted upper and lower envelope lines, and repeat steps S2411 - S2412 for the preliminary intrinsic mode function until the standard deviation criterion is satisfied to obtain the first intrinsic mode function component; S2414. Subtract the first intrinsic mode function component from the original acceleration signal to obtain the residual signal, and repeat the decomposition on the residual signal until the residual signal is a monotonic function to obtain a number of intrinsic mode function components.

[0016] Preferably, select the decomposition method of the empirical mode decomposition algorithm based on different visual regions, including: S24111. In the safety zone, use the coarse-grained decomposition method to extract the energy distribution of the low-frequency fundamental frequency and its harmonics; S24112. In the warning zone and the alarm zone, use the fine-grained decomposition method and introduce a visual constraint term to decompose the high-frequency impact mode.

[0017] Preferably, a D-S evidence theory fusion model is established based on the wheel flange lateral position-vibration characteristic mapping table, and the visual features and vibration features are fused by using the D-S evidence theory fusion model. Judging the derailment risk of the train according to the feature fusion result includes: S31. Defining a set of derailment risk states based on the divided visual regions; S32. Calculating the confidence degrees of each risk state in the derailment risk state set according to the region where the center line of the wheel tread is located, and taking them as visual evidence; S33. Calculating the confidence degree according to the abnormal degree of acceleration and the proportion of high-frequency components, and taking it as vibration evidence; S34. Based on the predefined Dempster combination rule, fusing the visual evidence and the vibration evidence, and judging the derailment risk of the train according to the fused features.

[0018] Preferably, the rotation matrix synthesis formula is: ; In the formula, represents the rotation matrix; represents the basic rotation matrix around the axis; represents the basic rotation matrix around the axis; represents the basic rotation matrix around the axis; represents the pitch angle; represents the yaw angle; represents the roll angle.

[0019] (III) Beneficial effects Compared with the prior art, the present invention provides a multi-dimensional train derailment monitoring method based on vision and vibration, and has the following beneficial effects: 1. Through the multi-dimensional architecture of visual region quantization and classification combined with multi-axis analysis of vibration acceleration, the present invention breaks through the limitations of single vibration data, and realizes the in-depth coupling analysis of visual images and vibration data through a multi-sensor fusion algorithm with a three-layer architecture of spatio-temporal registration, feature-level fusion, and decision-level fusion, thereby solving the defects that the prior art cannot adapt to working condition changes, is prone to false alarms, and affects operation.

[0020] 2. The present invention adopts an integrated sensor layout on the bogie, does not require modification of track facilities, and realizes risk probability calculation through an evidence theory fusion algorithm, rather than a simple signal trigger logic, which belongs to a substantial improvement in detection logic and control strategy, forms an intelligent closed loop of "multi-dimensional perception-cross analysis-accurate decision-making", and thus significantly improves the non-obviousness and practical application value of the technical solution. Description of the drawings

[0021] To more clearly illustrate the configuration method of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0022] Figure 1 is a flowchart of a multi-dimensional train derailment monitoring method based on vision and vibration according to an embodiment of the present invention; Figure 2 is a flowchart of an empirical mode decomposition algorithm in a multi-dimensional train derailment monitoring method based on vision and vibration according to an embodiment of the present invention; Figure 3 is a principle block diagram of a multi-dimensional train derailment monitoring method based on vision and vibration according to an embodiment of the present invention; Figure 4 is a schematic diagram of the position of the wheel flange center line in a multi-dimensional train derailment monitoring method based on vision and vibration according to an embodiment of the present invention; Figure 5 is a pre-alarm area identification diagram in a multi-dimensional train derailment monitoring method based on vision and vibration according to an embodiment of the present invention. Detailed implementation manners

[0023] To further illustrate the embodiments, the present invention provides drawings. These drawings are part of the disclosure of the present invention. They are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, those of ordinary skill in the art should be able to understand other possible implementation manners and the advantages of the present invention. The components in the drawings are not drawn to scale, and similar component symbols are usually used to represent similar components.

[0024] According to an embodiment of the present invention, a multi-dimensional train derailment monitoring method based on vision and vibration is provided.

[0025] Now, the present invention will be further described in conjunction with the drawings and specific implementation manners. As Figure 1 shown, a multi-dimensional train derailment monitoring method based on vision and vibration according to an embodiment of the present invention includes: S1. Collect images of the wheel-rail contact area and vibration data of each wheel in different directions respectively, and perform spatio-temporal alignment on the images of the wheel-rail contact area and the vibration data.

[0026] Among them, collecting images of the wheel-rail contact area and vibration data of each wheel in different directions respectively, and performing spatio-temporal alignment on the images of the wheel-rail contact area and the vibration data includes: S11. Collect images of the wheel-rail contact area using a video device, collect vibration data by acquiring the three-axis acceleration of the wheel using an acceleration sensor, and align the images of the wheel-rail contact area and the vibration data in terms of time through a clock protocol. S12. Establish a vision coordinate system based on the track plane with the reference point of the track center as the origin, and establish a vibration coordinate system with the center of the bogie as the origin. S13. Map the vibration data in the vibration coordinate system to the vision coordinate system through a rigid body transformation matrix for spatial alignment.

[0027] Among them, mapping the vibration data in the vibration coordinate system to the vision coordinate system through a rigid body transformation matrix for spatial alignment includes: S131. Use a laser rangefinder to collect the installation state of the acceleration sensor and measure the Euler angles of the vibration coordinate system relative to the vision coordinate system. S132. Calculate the rotation matrix according to the Euler angles and in combination with the rotation matrix synthesis formula, measure the coordinates of the origin of the vibration coordinate system in the vision coordinate system and determine the translation vector. S133. Convert the vibration data in the vibration coordinate system into components in the vision coordinate system through the rotation matrix and the translation vector respectively, and perform spatial association with the wheel flange position in the vision coordinate system. S134. Compare the wheel flange position in the actual wheel-rail contact area image with the spatial distribution of the vibration data, and optimize the rotation matrix and the translation vector.

[0028] It should be noted that the following further elaborates on separately collecting images of the wheel-rail contact area and vibration data of each wheel in different directions, and performing spatio-temporal alignment of the images of the wheel-rail contact area and the vibration data in combination with specific implementation manners.

[0029] I. Spatio-temporal registration (establishing a unified data benchmark): 1. Hardware synchronization mechanism To meet the requirements of high-speed train operation, it is planned to use a video acquisition device with high refresh rate, high resolution, and strong anti-interference ability. The hardware vision acquisition device is installed between the first and second axles of each bogie, and two sets of devices are installed on each bogie.

[0030] Vision data: The laser binocular camera acquires images of the wheel-rail contact area at a frame rate of 500fps, and the built-in hardware clock accuracy reaches ±1μs. Vibration data: The three-axis acceleration sensor synchronously acquires the X (longitudinal), Y (lateral), and Z (vertical) axis accelerations at a sampling rate of 10kHz to achieve hardware trigger synchronization. Spatio-temporal alignment: Through the IEEE 1588 Precision Time Protocol, the vision frame timestamps and the vibration sampling point timestamps are unified to the nanosecond level to ensure the alignment of multi-source data at the same physical moment.

[0031] 2. Spatial Coordinate System Mapping Visual Coordinate System: Taking the orbital plane as the XY plane, define the lateral position (I / II / III zones) of the center line O of the wheel flange. The position of the center line O of the wheel flange is as shown in Figure 4 shown ( Figure 4 Point A in represents the center line O of the wheel flange); As shown in Figure 5 shown, the basis for dividing the I / II / III zones is as follows: (1) According to the requirements of Chinese railway standards, the lateral displacement of the wheel set should not exceed 70% of the rail head width.

[0032] (2) In the wheel-rail contact simulation, when using tools such as ANSYS to simulate the lateral displacement of the wheel set, when the lateral displacement reaches 67% of the rail head width (i.e., 46.67 mm), the contact stress at the root of the wheel flange reaches 80% of the material yield strength, and an early warning needs to be triggered; when it exceeds the rail head width, the contact stress exceeds the yield strength.

[0033] (3) Field test data: In railway tests, when the lateral displacement of the wheel set exceeds 67% of the rail head width, the wear depth of the wheel flange increases by 1.2 mm per month, which is 2.5 times the normal wear. Based on the above three points, it is divided that the I zone is safe, the II zone is for early warning, and the III zone is for alarm.

[0034] Among them, the width of the I zone a = 0.67(L / 2); the width of the II zone b = 0.33(L / 2); the width greater than L is the III zone. For example: when L = 70 mm, it means that the wheel set can operate safely within the range of ±23.45 mm.

[0035] II zone (early warning zone): It means that early warning is required when the lateral displacement of the wheel set exceeds 23.45 mm but does not reach 35 mm.

[0036] III zone (alarm zone): When the width exceeds L (i.e., the lateral displacement > 35 mm), an emergency brake is triggered to prevent the wheel flange from completely disengaging from the rail head and causing derailment.

[0037] Vibration Coordinate System: Taking the center of the bogie as the origin, the vibration coordinate system is mapped to the visual coordinate system through a rigid body transformation matrix (rotation matrix R, translation vector T) to ensure that the spatial physical meaning of the vibration data is consistent with the visual area determination.

[0038] It should be noted that the rigid body transformation matrix includes a rotation matrix and a translation vector, which are used to describe the spatial transformation relationship between two coordinate systems: Rotation Matrix: It is the rotation attitude of a 3×3 coordinate system relative to the visual coordinate system (such as the rotation angle around the axis), satisfying (identity matrix).

[0039] Translation vector: A 3×1 vector representing the translation distance of the origin of the vibration coordinate system relative to the origin of the visual coordinate system (the offsets along the x / y / z axes).

[0040] Transformation expression (vector form): Let the visual coordinate system be , and the vibration coordinate system be . The coordinates of a certain point in are: ; The coordinates in are: ; Then the transformation relationship is: ; The homogeneous coordinate form (for easy processing by the pre-processor or host) is: ; In the formula, represents a 1×3 zero vector; represents the rotation matrix, describing the rotation attitude of the vibration coordinate system relative to the visual coordinate system; represents the translation vector, describing the position displacement of the origin of the vibration coordinate system in the visual coordinate system; represents the point coordinate vector in the visual coordinate system; represents the point coordinate vector in the visual coordinate system; represents the three-dimensional coordinates of a certain point in the vibration coordinate system; represents the three-dimensional coordinates of a certain point in the visual coordinate system.

[0041] II. Specific implementation steps of the rigid body transformation matrix: 1. Coordinate system definition and hardware installation Visual coordinate system: Taking the track plane as the XY plane, the origin can be set as a certain reference point in the track center, the X-axis along the track longitudinal direction (the train running direction), the Y-axis along the track transverse direction (the wheel set lateral displacement direction), and the Z-axis perpendicular to the track plane (the vertical direction).

[0042] Vibration coordinate system: Taking the bogie center as the origin, the sensor is installed at the axle box position, and its coordinate axis directions are aligned with the bogie structure (such as the X-axis corresponding to the longitudinal acceleration , the Y-axis corresponding to the lateral acceleration , and the Z-axis corresponding to the vertical acceleration ).

[0043] 2. Determine the rotation matrix Step 1: Measure the sensor installation attitude. Through a laser rangefinder or an inertial navigation device, measure the Euler angles (pitch angle , yaw angle Rolling angle )

[0044] Step 2: Construct the rotation matrix. According to the Euler angle sequence (such as z-y-x rotation), calculate through the rotation matrix synthesis formula :[[]] ; In the formula, are the basic rotation matrices around the axis respectively. For example: ; It should be noted that the Euler angle sequence needs to be consistent with the actual installation attitude to avoid the gimbal lock problem.

[0045] 3. Determine the translation vector Measure the origin coordinates of the sensor. Through a high-precision positioning tool, measure the coordinates of the origin of the vibration coordinate system in the vision coordinate system , that is: ; For example, the sensor is installed on the bogie axle box, and its lateral offset relative to the track center is , the vertical height is , and the longitudinal position is .

[0046] 4. Data transformation and verification Real-time coordinate conversion. The acceleration data collected by the vibration sensor is first passed through and to be converted into components in the vision coordinate system, and then spatially associated with the wheel flange position in the vision coordinate system (such as ).

[0047] Calibration and error correction. Through on-site tests (such as simulating wheel set lateral movement), compare the spatial distribution of the wheel flange position detected visually with the vibration data, and finely adjust and parameters to ensure that the transformation accuracy reaches nanosecond-level time synchronization and millimeter-level spatial alignment.

[0048] S2. Combine the edge detection algorithm and the Hough transform technology to extract visual features from the wheel-rail contact area image, and extract vibration features from the vibration data, and associate the visual features with the vibration features to generate a wheel flange lateral position-vibration feature mapping table.

[0049] Among them, combining the edge detection algorithm and the Hough transform technology to extract visual features from the wheel-rail contact area image, and extracting vibration features from the vibration data, and associating the visual features with the vibration features to generate a wheel flange lateral position-vibration feature mapping table includes: S21. Divide the visual area in the visual coordinate system based on predefined division conditions, and the visual area includes a safety area, a warning area, and an alarm area; S22. Combine the Canny edge detection algorithm and the Hough transform technology to extract the wheel-rail profile features in the image of the wheel-rail contact area in real time, and calculate the lateral offset of the center line of the wheel tread; S23. Calculate the time-domain features of the acceleration signal in the vibration data, and use the adaptive time-frequency fusion fast Fourier transform algorithm to extract the frequency-domain features of the acceleration signal.

[0050] Among them, using the adaptive time-frequency fusion fast Fourier transform algorithm to extract the frequency-domain features of the acceleration signal includes: Preprocess the acceleration signal, retain the wheel-rail coupling related frequency bands, and window the wheel-rail coupling related frequency bands using the Hann window function; Dynamically adjust the Fourier transform parameters of the windowed acceleration signal in different visual areas, introduce visual evidence weights in the calculation of the Fourier transform parameters, and obtain the amplitude spectrum; Convert the amplitude spectrum to a single-sided spectrum, search for the peaks and amplitudes within a preset range in the single-sided spectrum, and obtain the wheel-rail coupling characteristic frequencies according to the peaks and amplitudes of each frequency point.

[0051] S24. Based on the time-domain features and frequency-domain features of the acceleration signal, use the adaptive empirical mode decomposition algorithm to generate a transform marginal spectrum and analyze the time-frequency features of derailment precursors.

[0052] Among them, based on the time-domain features and frequency-domain features of the acceleration signal, using the adaptive empirical mode decomposition algorithm to generate a transform marginal spectrum and analyze the time-frequency features of derailment precursors includes: S241. Dynamically adjust the empirical mode decomposition parameters of the visual area using the empirical mode decomposition algorithm to obtain several intrinsic mode function components.

[0053] As Figure 2 shown, dynamically adjusting the empirical mode decomposition parameters of the visual area using the empirical mode decomposition algorithm to obtain several intrinsic mode function components includes: S2411. Select the decomposition method of the empirical mode decomposition algorithm based on different visual areas.

[0054] Among them, selecting the decomposition method of the empirical mode decomposition algorithm based on different visual areas includes: S24111. In the safety area, use the coarse-grained decomposition method to extract the energy distribution of the low-frequency fundamental frequency and its harmonics; S24112. In the warning area and the alarm area, use the fine-grained decomposition method and introduce a visual constraint term to decompose the high-frequency impact mode.

[0055] S2412. Obtain the local maxima and minima of the acceleration signal, connect the maximum points to form an upper envelope, connect the minimum points to form a lower envelope, and fit the upper and lower envelopes through cubic spline interpolation; S2413. Calculate the preliminary intrinsic mode function based on the fitted upper and lower envelopes, and repeat steps S2411 - S2412 for the preliminary intrinsic mode function until the standard deviation criterion is met to obtain the first intrinsic mode function component; S2414. Subtract the first intrinsic mode function component from the original acceleration signal to obtain a residual signal, and repeat the decomposition for the residual signal until the residual signal is a monotonic function to obtain several intrinsic mode function components.

[0056] S242. Perform Hilbert transform on each intrinsic mode function component to obtain an analytic signal, and calculate the instantaneous amplitude and instantaneous frequency of the analytic signal; S243. Superimpose the distributions of the instantaneous amplitudes and instantaneous frequencies of all intrinsic mode function components over time to form a Hilbert spectrum; S244. Integrate the Hilbert spectrum along the time axis to obtain a marginal spectrum, and identify the abnormal frequency components of derailment precursors based on the marginal spectrum.

[0057] S25. Combine the wheel-rail profile features and time-frequency features to generate a wheel flange lateral position - vibration feature mapping table.

[0058] It should be noted that the following further elaborates on extracting visual features from the wheel-rail contact area image by combining the edge detection algorithm and the Hough transform technology, extracting vibration features from vibration data, and associating the visual features with the vibration features to generate a wheel flange lateral position - vibration feature mapping table in combination with specific implementation manners.

[0059] Feature-level fusion (multi-source data feature extraction and association): 1. Visual feature extraction Track surface zoning: Divide the track interface into Zone I (safe zone, width 0.67L / 2), Zone II (warning zone, width 0.33L / 2), and Zone III (alarm zone, width > L / 2). Real-time extract the wheel-rail profile through Canny edge detection + Hough transform, and calculate the lateral offset Δ of the center line O of the wheel tread x ; It should be noted that the steps of real-time extracting the wheel-rail profile through Canny edge detection + Hough transform.

[0060] (1) Image preprocessing Gaussian filtering: Add a "blur filter" to the image to reduce random noise interference; Adaptive threshold: Dynamically adjust the black and white boundary according to the local area brightness to make the wheel-rail edge clearer.

[0061] (2)Canny Edge Detection Calculate the gradient: Find the places where the brightness changes drastically in the image (such as the wheel-rail boundary); Non-maximum suppression: Only retain the brightest edge points and "shave" the wide edges into thin lines; Double-threshold screening: Use two threshold values to screen the edges. Strong edges are directly retained, and weak edges are only retained if they are connected to strong edges.

[0062] (3)Hough Transform Line detection: Find the positions in the points of the image that are most likely to be lines (rails); Circle detection: Similar to line detection, but looking for circles (wheels).

[0063] (4)Contour Fitting Least squares method: Find a straight line or curve that minimizes the sum of the distances from all data points to this line; RANSAC algorithm: Repeatedly sample randomly to find the most "reliable" model (such as the rail line).

[0064] 2. Vibration Feature Extraction Time-domain features: Calculate the root mean square value (RMS), peak factor, and kurtosis of the acceleration signal to characterize the vibration impact intensity; (1)The calculation formula for the root mean square value (RMS) is: ; In the formula, represents the value of the th sampling point of the acceleration signal (unit: , gravitational acceleration), represents the number of sampling points, and the root mean square value is used to characterize the energy intensity of the vibration signal and reflects the average energy level of the vibration.

[0065] (2)The calculation formula for the peak factor (Peak Factor) is: Peak factor ; In the formula: represents the absolute value peak of the acceleration signal (unit: ); represents the root mean square value; The peak factor is used to measure the relative intensity of the impact peak in the vibration signal. The larger the peak factor, the more intense the impact (such as at the moment of wheel-rail collision).

[0066] (3)The calculation formula for kurtosis is: Kurtosis ; In the formula, Denotes the mean value of the acceleration signal (unit: ) Denotes the standard deviation of the acceleration signal (unit: ). Kurtosis is used to describe the amplitude distribution pattern of the vibration signal. The kurtosis of a normal vibration signal is close to 3 (normal distribution), and sudden shocks will cause a significant increase in kurtosis, which is used to identify abnormal shock events (such as high-frequency micro-shocks in the precursor of derailment).

[0067] Frequency domain feature: Perform an adaptive time-frequency fusion FFT algorithm on the acceleration signal to extract the wheel-rail coupling characteristic frequency.

[0068] It should be noted that the implementation steps of FFT feature extraction include: (1) Acceleration signal preprocessing Sampling parameters: According to the document, the vibration sensor collects three-axis acceleration signals at a sampling rate of 10 kHz ( ), and each analysis takes a data segment of length N (such as N = 1024 points, corresponding to a time duration of 102.4 ms).

[0069] Denoise processing: Use mean filtering to remove the DC drift (such as , denotes the signal mean value); Retain the wheel-rail coupling related frequency band (fundamental frequency 25 Hz and its harmonics, high-frequency shocks > 500 Hz are processed by wavelet packets) through band-pass filtering (such as 5 - 500 Hz).

[0070] (2) Windowing processing Window function selection: Use the Hanning window to reduce spectral leakage. The expression of the window function is: ; In the formula, denotes the length of the window function, usually a power of 2 (such as 512, 1024); denotes the discrete sampling point sequence number of the time domain signal ( ); Windowing processing is used to reduce the spectral spread caused by signal truncation and improve the calculation accuracy of the frequency component amplitude.

[0071] (3) Adaptive time-frequency fusion FFT (Fast Fourier Transform) algorithm Perform an N-point FFT on the windowed signal to obtain the frequency domain complex sequence X ( k ); Space-time linkage preprocessing: According to the I / II / III regions divided by the visual coordinate system, dynamically adjust the FFT parameters for the vibration signals in different regions: Zone I (Safety Zone): Use long-window FFT (e.g., N = 4096) to improve the resolution in the low-frequency band (25 Hz fundamental frequency) and monitor the long-term vibration trend; Zones II / III (Warning / Alarm Zones): Switch to short-window FFT (e.g., N = 256) + overlapping sampling (50% overlap rate) to capture the instantaneous frequency changes of high-frequency impacts (>500 Hz).

[0072] Cross-modal weighting: Introduce the visual evidence weight in the FFT amplitude calculation (e.g., in Zone II = 0.8, in Zone I = 0.2), to enhance the frequency feature response in the abnormal area: ; In the formula, represents the visual area trigger factor (1 for Zones II / III, 0 for Zone I).

[0073] Calculate the amplitude spectrum: ; In the formula, represents the frequency-domain complex sequence adjusted by the visual evidence weight; Re represents taking the real part of the complex number; represents taking the imaginary part of the complex number.

[0074] Convert to single-sided spectrum: Take the first N / 2 points, and multiply the amplitude by 2 / N (keep the amplitude of the DC component unchanged).

[0075] Frequency resolution: ; In the formula, represents the frequency resolution; represents the sampling rate of the acceleration signal; represents the number of samples when performing the fast Fourier transform.

[0076] (4) Extraction of wheel-rail coupling characteristic frequencies Fundamental frequency identification: Search for the peak within the range of in the spectrum to confirm the fundamental frequency of the wheel set passing through the rail joint (e.g., 25 Hz mentioned in the document); Harmonic detection: Check the amplitudes of the integer multiple frequency points such as 50 Hz, 75 Hz in sequence to construct the harmonic sequence , where, represents the fundamental vibration frequency generated by the wheel set passing through the rail joint, etc.; represents the harmonic order, which is a positive integer ( = 1, 2, 3...), used to characterize the integer multiple frequencies of the fundamental frequency.

[0077] Characteristic parameters: Extract the amplitudes of each frequency point Sum phase , which is used for subsequent spatio-temporal association with visual data.

[0078] Time-frequency feature: Adaptive EMD (Empirical Mode Decomposition) decomposition based on the visual region is adopted to generate the "Hilbert-Huang transform" (HHT) marginal spectrum, and the abnormal time-frequency energy distribution of derailment precursors is analyzed.

[0079] It should be noted that the implementation steps of the Hilbert-Huang transform include: (1) Adaptive EMD decomposition based on the visual region Input signal: Uniaxial acceleration signal collected by a vibration sensor (such as longitudinal acceleration ), with a sampling rate of 10 kHz and a data length of (corresponding to 10,000 points).

[0080] Adaptive decomposition: According to the I / II / III regions (rim lateral displacement) divided visually in the document, the EMD decomposition parameters are dynamically adjusted: Region I (safe zone): Coarse-grained decomposition is adopted (limiting the number of IMF layers ≤ 3), and the energy distribution of low-frequency fundamental frequency (25 Hz) and its harmonics is mainly extracted; Regions II / III (warning / alarm zones): Fine-grained decomposition is triggered (IMF layers ≥ 5), and a visual constraint term ( represents the rim lateral offset, represents the weight coefficient) is introduced to force the decomposition of high-frequency impact modes (such as the micro-impact at the initial stage of climbing the rail corresponding to IMF4-IMF5).

[0081] Decomposition process: Determine extreme points: Find all local maxima and minima of .

[0082] Construct envelope: The upper and lower envelopes are fitted by cubic spline interpolation to ensure that the two envelopes are tangent everywhere within the data segment, and the envelope mean is .

[0083] Extract IMF: Calculate the preliminary intrinsic mode function (IMF): ; Screening criterion: Repeat steps 1-3 for the preliminary intrinsic mode function until the standard deviation criterion ( ) is satisfied to obtain the first IMF component .

[0084] Recursive decomposition: Subtract from the original signal to obtain the residual signal . For repeated decomposition until the residual signal is a monotonic function, obtaining IMF components and the residual term .

[0085] (2)Hilbert transform and marginal spectrum generation Hilbert transform: Perform the Hilbert transform on each IMF component to obtain the analytic signal: ; In the formula, represents the Hilbert transform operator. The amplitude of the analytic signal is the instantaneous amplitude , represents the phase, represents the th IMF component, and the formula for the instantaneous frequency is: ; In the formula, represents the instantaneous frequency; represents the change in phase over time (in radians); represents the time element (in seconds); represents the phase change rate (in radians per second).

[0086] Hilbert spectrum construction: Superimpose the distributions of the instantaneous amplitudes and instantaneous frequencies of all IMFs over time to form the Hilbert spectrum , which characterizes the time-frequency energy distribution of the signal.

[0087] Marginal spectrum calculation: Integrate the Hilbert spectrum over the time axis to obtain the marginal spectrum : ; The marginal spectrum reflects the total energy distribution of the frequency components in the signal and is used to identify abnormal frequency components of derailment precursors.

[0088] 3. Feature correlation modeling: Establish a mapping table of the lateral position of the wheel rim - vibration characteristics, as shown in Table 1.

[0089] Table 1: Mapping table of the lateral position of the wheel rim - vibration characteristics S3. Establish a D-S evidence theory fusion model based on the wheel flange lateral position-vibration feature mapping table, and use the D-S evidence theory fusion model to fuse visual features and vibration features, and determine the train derailment risk according to the feature fusion result.

[0090] Among them, establishing a D-S evidence theory fusion model based on the wheel flange lateral position-vibration feature mapping table, using the D-S evidence theory fusion model to fuse visual features and vibration features, and determining the train derailment risk according to the feature fusion result includes: S31. Define the set of derailment risk states based on the divided visual regions; S32. Calculate the confidence degrees of each risk state in the set of derailment risk states according to the region where the center line of the wheel tread is located, and use them as visual evidence; S33. Calculate the confidence degree according to the acceleration anomaly degree and the proportion of high-frequency components, and use it as vibration evidence; S34. Based on the predefined Dempster combination rule, fuse the visual evidence and the vibration evidence, and determine the train derailment risk according to the fused features.

[0091] It should be noted that the establishment of a D-S evidence theory fusion model based on the wheel flange lateral position-vibration feature mapping table, the fusion of visual features and vibration features using the D-S evidence theory fusion model, and the determination of the train derailment risk according to the feature fusion result will be further described below in combination with specific implementation manners.

[0092] Decision-level fusion (multi-dimensional evidence synthesis and risk determination): 1. D-S evidence theory fusion framework Frame of discernment: Define the set of derailment risk states ={Safe, Warning, Alarm}; Basic probability assignment (BPA): Visual evidence: Calculate the confidence degree of each risk state according to the region where the center line O is located: Area I: ; Area II: Area III: ; Vibration evidence: Calculate the confidence degree according to the acceleration anomaly degree (ax>60g or ay>30g) and the proportion of high-frequency components: No anomaly: ; Longitudinal anomaly: ; Transverse and longitudinal double anomaly: ; Calculate the conflict coefficient (The conflict coefficient is the sum of the products of the probabilities of all combinations with empty intersections): ; Taking "Visual Area II + Longitudinal Vibration Anomaly" as an example: Visual evidence: ; Vibration evidence: ; Calculate the conflict coefficient : Calculate the sum of the products of the probabilities of all combinations with empty intersections, ; Substitute the values to get: ; Calculate the basic probability assignment value after fusion: Warning state: ; Substitute the values: ; Alarm state: ; Substitute the values: ; Safe state: ; By using the Dempster combination rule, the evidence from two different sources, vision and vibration, is fused to obtain a more comprehensive and reliable risk judgment. In the above example, the confidence level of the warning state after fusion is significantly increased to 0.89, which is greater than the single vision warning confidence level of 0.8 and the single vibration warning confidence level of 0.7. This shows that after combining the information from the two sensors, the system is more confident in judging that the train is in a warning state. Compared with the judgment of a single sensor, the risk of misjudgment is reduced, providing a more accurate basis for subsequent warning and braking decisions, and enhancing the reliability and accuracy of the train derailment monitoring system.

[0093] Risk determination: Three - coordinate sensors are used. Three - coordinate sensors are installed at each axle box position of the bogie to collect the accelerations in three directions (mainly longitudinal and lateral) at each wheel position. Because when derailment is about to occur, there will definitely be a phenomenon of flange climbing on the rail, and at this time, there will definitely be abnormal longitudinal acceleration (ax), and even lateral acceleration (ay) may occur. However, the occurrence of longitudinal acceleration does not necessarily mean it is a precursor to derailment. It may also be the radial run - out of the flange. Therefore, it is also necessary to combine the vision system to determine the relative position of the wheel and the rail.

[0094] Such as Figure 3As shown in the figure, the overall multi-dimensional derailment detection method logic includes: The entire multi-dimensional derailment system of vision and vibration takes a bogie as a basic judgment unit. Each wheel has a vision and vibration monitoring unit (vision device and vibration sensor). When the sensor ax>60g and ay>30g, it is determined that the acceleration is abnormal.

[0095] a. If the vision determines that the center line O is in area I: If the longitudinal acceleration of the vibration unit is normal, there is no early warning alarm, and the train does not take any action. If the longitudinal acceleration of the vibration unit is abnormal (ax>60g), and the early warning confidence level of the composite judgment is greater than that of the single early warning, a level 1 early warning occurs, and the vehicle does not decelerate.

[0096] b. If the vision determines that the center line O is in area II: If the longitudinal acceleration of the vibration unit is normal, and the early warning confidence level of the composite judgment is greater than that of the single early warning, a level 1 early warning occurs, and the vehicle does not decelerate; If the longitudinal acceleration of the vibration unit is abnormal (ax>60g), and the early warning confidence level of the composite judgment is greater than that of the single early warning, a level 2 early warning occurs, and the vehicle decelerates.

[0097] c. If the vision determination unit is in area III: Regardless of any data situation of the vibration unit, as long as the alarm confidence level of the composite judgment is greater than that of the single alarm, an alarm will occur, and the vehicle will make an emergency brake.

[0098] d. If ax>60g and ay>30g, and the alarm confidence level of the composite judgment is greater than that of the single alarm, a direct alarm will occur, and the vehicle will make an emergency brake.

[0099] It should be noted that the present invention can be applied to a brand-new rail transit vehicle derailment monitoring system. It mainly configures four groups of binocular cameras and 4 three-coordinate sensors for each bogie, and is equipped with a pre-processor (installed on the bogie or underframe). The signals are pre-processed and then transmitted to the host, so as to accurately give early warnings and alarms for the derailment site, and effectively avoid the hazards caused by train derailment.

[0100] In the present invention, the adaptive time-frequency fusion FFT algorithm and the EMD decomposition of the adaptive vision area are used to achieve the adaptive decomposition design driven by the vision area, deeply couple the vibration time-frequency analysis with the spatial position information, and form a technical path of "vision guidance - vibration precise judgment".

[0101] Through the three-level architecture of "physical layer synchronization - feature layer association - decision layer intelligence", the present invention realizes the full-chain optimization from data acquisition to risk decision-making, significantly improves the reliability and intelligence level of train derailment monitoring, and reduces the false alarm rate of a single sensor through multi-source data cross-validation.

[0102] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A multi-dimensional train derailment monitoring method based on vision and vibration, characterized in that The method includes: Collecting images of the wheel-rail contact area and vibration data of each wheel in different directions respectively, and performing spatio-temporal alignment on the images of the wheel-rail contact area and the vibration data; Combining an edge detection algorithm and a Hough transform technique to extract visual features from the images of the wheel-rail contact area, and extracting vibration features from the vibration data, and associating the visual features with the vibration features to generate a wheel flange lateral position-vibration feature mapping table; Establishing a D-S evidence theory fusion model based on the wheel flange lateral position-vibration feature mapping table, and using the D-S evidence theory fusion model to fuse the visual features and the vibration features, and determining the train derailment risk according to the feature fusion result; The combining an edge detection algorithm and a Hough transform technique to extract visual features from the images of the wheel-rail contact area, and extracting vibration features from the vibration data includes: Combining an edge detection algorithm and a Hough transform technique to extract the wheel-rail profile features in the images of the wheel-rail contact area in real time, and calculating the lateral offset of the center line of the wheel tread; Calculating the time domain features of the acceleration signal in the vibration data, and using an adaptive time-frequency fusion fast Fourier transform algorithm to extract the frequency domain features of the acceleration signal; Based on the time domain features and frequency domain features of the acceleration signal, using an adaptive empirical mode decomposition algorithm to generate a transform marginal spectrum, and analyzing the time-frequency features of derailment precursors.

2. The multi-dimensional train derailment monitoring method based on vision and vibration according to claim 1, characterized in that The collecting images of the wheel-rail contact area and vibration data of each wheel in different directions respectively, and performing spatio-temporal alignment on the images of the wheel-rail contact area and the vibration data includes: Using a video device to collect images of the wheel-rail contact area, and using an acceleration sensor to collect the three-axis acceleration of the wheel to obtain vibration data, and performing time alignment on the images of the wheel-rail contact area and the vibration data through a clock protocol; Establishing a visual coordinate system based on the track plane with the track center reference point as the origin, and establishing a vibration coordinate system with the bogie center as the origin; Mapping the vibration data of the vibration coordinate system to the visual coordinate system through a rigid body transformation matrix for spatial alignment.

3. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 2, characterized in that The mapping the vibration data of the vibration coordinate system to the visual coordinate system through a rigid body transformation matrix for spatial alignment includes: Using a laser rangefinder to collect the installation state of the acceleration sensor, and measuring the Euler angles of the vibration coordinate system relative to the visual coordinate system; Calculating a rotation matrix according to the Euler angles and combining the rotation matrix synthesis formula, measuring the coordinates of the origin of the vibration coordinate system in the visual coordinate system and determining the translation vector; Converting the vibration data of the vibration coordinate system into components of the visual coordinate system through the rotation matrix and the translation vector respectively, and performing spatial association with the wheel flange position in the visual coordinate system; Comparing the wheel flange position of the actual wheel-rail contact area image with the spatial distribution of the vibration data, and optimizing the rotation matrix and the translation vector.

4. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 1, characterized in that Before the combining an edge detection algorithm and a Hough transform technique to extract the wheel-rail profile features in the images of the wheel-rail contact area in real time, and calculating the lateral offset of the center line of the wheel tread, it further includes: Dividing the visual area in the visual coordinate system based on predefined division conditions, and the visual area includes a safety area, a warning area and an alarm area; After analyzing the time-frequency characteristics of derailment precursors by generating a transformed marginal spectrum using the adaptive empirical mode decomposition algorithm based on the time-domain and frequency-domain characteristics of the acceleration signal, it further includes: Combining the wheel-rail profile characteristics and the time-frequency characteristics to generate a wheel flange lateral position-vibration characteristic mapping table.

5. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 4, characterized in that The extracting of the frequency-domain characteristics of the acceleration signal using the adaptive time-frequency fusion fast Fourier transform algorithm includes: Preprocessing the acceleration signal, retaining the wheel-rail coupling related frequency bands, and windowing the wheel-rail coupling related frequency bands using the Hanning window function; Dynamically adjusting the Fourier transform parameters for the windowed acceleration signals in different visual regions, introducing visual evidence weights in the calculation of the Fourier transform parameters to obtain the amplitude spectrum; Converting the amplitude spectrum to a single-sided spectrum, searching for peaks and amplitudes within a preset range in the single-sided spectrum, and obtaining the wheel-rail coupling characteristic frequencies based on the peaks and amplitudes at each frequency point.

6. The multi-dimensional train derailment monitoring method based on vision and vibration according to claim 5, characterized in that Analyzing the time-frequency characteristics of derailment precursors by generating a transformed marginal spectrum using the adaptive empirical mode decomposition algorithm based on the time-domain and frequency-domain characteristics of the acceleration signal includes: Dynamically adjusting the empirical mode decomposition parameters of the visual region using the empirical mode decomposition algorithm to obtain several intrinsic mode function components; Performing Hilbert transform on each intrinsic mode function component to obtain an analytic signal, and calculating the instantaneous amplitude and instantaneous frequency of the analytic signal; Superposing the distributions of the instantaneous amplitudes and instantaneous frequencies of all intrinsic mode function components over time to form a Hilbert spectrum; Integrating the Hilbert spectrum along the time axis to obtain a marginal spectrum, and identifying abnormal frequency components of derailment precursors based on the marginal spectrum.

7. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 6, characterized in that The dynamically adjusting the empirical mode decomposition parameters of the visual region using the empirical mode decomposition algorithm to obtain several intrinsic mode function components includes: Selecting the decomposition method of the empirical mode decomposition algorithm based on different visual regions; Obtaining the local maximum and minimum values of the acceleration signal, connecting the maximum value points to form an upper envelope, connecting the minimum value points to form a lower envelope, and fitting the upper and lower envelopes by cubic spline interpolation; Calculating a preliminary intrinsic mode function based on the fitted upper and lower envelopes, and repeatedly performing iteration on the preliminary intrinsic mode function until the standard deviation criterion is satisfied to obtain the first intrinsic mode function component; Subtracting the first intrinsic mode function component from the original acceleration signal to obtain a residual signal, and repeatedly decomposing the residual signal until the residual signal is a monotonic function to obtain several intrinsic mode function components.

8. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 7, characterized in that The selecting the decomposition method of the empirical mode decomposition algorithm based on different visual regions includes: In the safe area, using a coarse-grained decomposition method to extract the energy distribution of the low-frequency fundamental frequency and its harmonics; In the warning area and the alarm area, using a fine-grained decomposition method and introducing a visual constraint term to decompose the high-frequency impact mode.

9. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 1, characterized in that, Establishing a D-S evidence theory fusion model based on the wheel flange lateral position-vibration characteristic mapping table, and using the D-S evidence theory fusion model to fuse the visual characteristics and the vibration characteristics and determining the derailment risk of the train according to the feature fusion result includes: Defining a derailment risk state set based on the divided visual regions; Calculate the confidence of each risk state in the derailment risk state set based on the area where the center line of the wheel tread is located and use it as visual evidence; Calculate the confidence based on the degree of acceleration anomaly and the proportion of high-frequency components and use it as vibration evidence; Based on the predefined synthesis rule, fuse the visual evidence and the vibration evidence, and determine the derailment risk of the train according to the fused features.

10. A multi-dimensional train derailment monitoring method based on vision and vibration according to claim 3, characterized in that, The rotation matrix synthesis formula is as follows: ； In the formula, represents the rotation matrix; represents the basic rotation matrix about axis; represents the basic rotation matrix about axis; represents the basic rotation matrix about axis; represents the pitch angle; represents the yaw angle; represents the roll angle.

Images