Field unmanned equipment remote control method based on VR glasses

Abstract

The application relates to the technical field of remote control of unmanned equipment, and discloses a field unmanned equipment remote control method based on VR glasses, impact angle acceleration is extracted according to the measured combined linear acceleration and translational acceleration, and an inverse optical flow velocity field is generated through integral and outer product operation; a partial derivative is calculated according to the mapping relationship of light penetration through a lens to obtain a magnification Jacobian gradient, the magnification Jacobian gradient is multiplied by an optical flow field and an divergence operator is applied to obtain an optical flow deformation divergence field, and then reverse radial pre-distortion processing is performed on spherical pixels; subsequently, an energy dissipation ratio is constructed in combination with head and chassis angular velocities to extract command angular acceleration; finally, instantaneous slip rate is calculated, and a resistance driving torque is calculated in combination with an equivalent electromagnetic damping coefficient to control an execution motor. The scheme effectively suppresses the dizziness caused by the superposition of bumping and optical distortion, and smooth closed-loop remote control is realized.

Description

Technical Field

[0001] This invention relates to the field of unmanned equipment remote control technology, and more specifically, to a method for remote control of unmanned equipment in the field based on VR glasses. Background Technology

[0002] In remote operation scenarios for agricultural unmanned equipment, operators typically control the equipment by wearing virtual reality glasses and viewing real-time footage transmitted from a vehicle-mounted panoramic camera. Farmland terrain is often uneven, with numerous mud clods and furrows, causing unmanned equipment to experience high-frequency angular oscillations when navigating non-rigid terrain.

[0003] After the panoramic camera synchronously transmits these oscillating images to the operator, the light must penetrate a lens to enter the human eye. Because the optical angular magnification of such lenses is not uniform between the center and the edges, exhibiting a distortion that increases with the off-axis angle, the originally rigid global image is forcibly refracted and transformed into a non-uniform deformation field with slower flow velocity at the center and faster flow velocity at the edges as it passes through different radial regions of the lens. Simultaneously, the operator is in a static environment with stable gravity, and the angular acceleration perceived by the inner ear's vestibular system is zero. This temporally alternating spatial divergence field of the visual image creates a significant conflict with the static vestibular perception, easily interfering with the brain's ability to resolve three-dimensional depth and inducing dizziness.

[0004] Existing remote control solutions mostly rely on fixed dead zone parameters for conventional low-pass filtering, which makes it difficult to accurately separate the pure control intention from the complex interference of mechanical bumps and optical distortion. This leads to delays or deviations in the issuance of low-level commands, thus affecting the safety and smoothness of remote control of unmanned equipment in the field. Summary of the Invention

[0005] This invention provides a method for remotely controlling unmanned field equipment based on VR glasses, solving the technical problems mentioned in the background art.

[0006] This invention provides a method for remotely controlling unmanned field equipment based on VR glasses. It is applied to a system comprising an equipment chassis, an inertial measurement unit and wheel speedometer mounted on the chassis, a panoramic camera, an actuator, and virtual reality glasses including lenses, configured to execute: Based on the measured linear acceleration and true angular velocity of the equipment chassis obtained by the inertial measurement unit, and the translational acceleration calculated by the wheel speed meter, the impact angular acceleration is extracted. Within the hardware frame refresh cycle, the impact angular acceleration is integrated over time and cross-multiplied with the virtual spherical pixel direction vector to generate a reverse optical flow velocity field. Based on the mapping relationship of light passing through the lens, the analytical partial derivative with respect to the off-axis angle is calculated to obtain the Jacobian gradient of magnification; The optical flow velocity field is multiplied by the magnification Jacobian gradient, and the spherical divergence operator is applied to calculate the optical flow deformation divergence field. Based on the optical flow deformation divergence field, the original spherical pixel orientation vector is subjected to reverse radial pre-distortion processing, and the distortion mapping coordinates are output. The head angular velocity and initial angular acceleration of the human body are obtained. The energy dissipation ratio is constructed by combining the synchronized angular velocity of the device chassis after network latency alignment, and the command angular acceleration is extracted. The instantaneous slip ratio is calculated by acquiring the linear velocity of the wheel and the actual translational linear velocity of the chassis. The command angular acceleration is converted into the target angular velocity. The anti-resistance driving torque is calculated by combining the instantaneous slip ratio and the equivalent electromagnetic damping coefficient, and the actuator motor is controlled.

[0007] The beneficial effects of this invention are as follows: by extracting the impact angular acceleration, generating the reverse optical flow field, and analyzing the partial derivative of the lens off-axis angle, the optical flow deformation divergence field is calculated to actively pre-distort the rendered image. At the same time, the energy dissipation ratio is constructed to decouple the pure operation intention, and the slip ratio is combined to realize the closed loop of the underlying motor torque. This invention dynamically cancels the perception misalignment between the visual and vestibular systems caused by the superposition of field machinery bumps and lens nonlinear distortion, effectively alleviates the depth perception collapse phenomenon, and improves the accuracy of remote operation and the stability of field equipment operation. Attached Figure Description

[0008] Figure 1 This is a flowchart of a remote control method for unmanned field equipment based on VR glasses according to the present invention. Detailed Implementation

[0009] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed only to enable those skilled in the art to better understand and implement the subject matter described herein, and changes may be made to the function and arrangement of the elements discussed without departing from the scope of this specification. Various processes or components may be omitted, substituted, or added as needed in the examples. Furthermore, features described in some examples may be combined in other examples.

[0010] like Figure 1 As shown, a method for remotely controlling unmanned field equipment based on VR glasses is applied to a system comprising an equipment chassis, an inertial measurement unit and wheel speedometer mounted on the equipment chassis, a panoramic camera, an actuator motor, and virtual reality glasses including lenses, configured to execute: Based on the measured linear acceleration and true angular velocity of the equipment chassis obtained by the inertial measurement unit, and the translational acceleration calculated by the wheel speed meter, the impact angular acceleration is extracted. Within the hardware frame refresh cycle, the impact angular acceleration is integrated over time and cross-multiplied with the virtual spherical pixel direction vector to generate a reverse optical flow velocity field. Based on the mapping relationship of light passing through the lens, the analytical partial derivative with respect to the off-axis angle is calculated to obtain the Jacobian gradient of magnification; The optical flow velocity field is multiplied by the magnification Jacobian gradient, and the spherical divergence operator is applied to calculate the optical flow deformation divergence field. Based on the optical flow deformation divergence field, the original spherical pixel orientation vector is subjected to reverse radial pre-distortion processing, and the distortion mapping coordinates are output. The head angular velocity and initial angular acceleration of the human body are obtained. The energy dissipation ratio is constructed by combining the synchronized angular velocity of the device chassis after network latency alignment, and the command angular acceleration is extracted. The instantaneous slip ratio is calculated by acquiring the linear velocity of the wheel and the actual translational linear velocity of the chassis. The command angular acceleration is converted into the target angular velocity. The anti-resistance driving torque is calculated by combining the instantaneous slip ratio and the equivalent electromagnetic damping coefficient, and the actuator motor is controlled.

[0011] This solution is applied to a system that includes a chassis, an inertial measurement unit and wheel speedometer mounted on the chassis, a panoramic camera, actuators, and virtual reality glasses with lenses. The system includes an inertial measurement unit (IMU) mounted at the center of gravity of the equipment chassis, with a sampling frequency of at least 1 kHz, capable of synchronously outputting raw data of triaxial acceleration and triaxial angular velocity; wheel speed meters mounted at the hubs of each wheel on the equipment chassis, with a sampling frequency synchronized with the IMU, capable of outputting real-time rotational speed data for each wheel; a panoramic camera mounted at the top center of the equipment chassis, with a field of view of at least 360° × 180° and a frame rate of at least 60 fps, used to capture real-time panoramic images of the field operation scene; permanent magnet synchronous motors, corresponding to each drive wheel, equipped with torque closed-loop control capability, with a control frequency of at least 10 kHz; and virtual reality glasses, a head-mounted display device with a built-in head posture tracking module, a sampling frequency of at least 1 kHz, capable of outputting triaxial angular velocity and triaxial attitude angle data of the wearer's head, a hardware frame refresh cycle of at least 16.7 ms, corresponding to a refresh frequency of at least 60 Hz.

[0012] All vector operations in this scheme are performed based on a unified coordinate system, and all vectors involved in the operation must be converted to the same coordinate system beforehand.

[0013] The geographic coordinate system has its origin at the centroid of the equipment chassis at the initial power-on moment, with the X-axis along the geographic east direction, the Y-axis along the geographic north direction, and the Z-axis along the geographic vertical upward direction. It serves as the global reference coordinate system for all kinematic parameters.

[0014] The vehicle coordinate system has its origin at the real-time center of mass of the equipment chassis. The X-axis is along the forward direction of the equipment chassis, the Y-axis is horizontally to the left along the equipment chassis, and the Z-axis is vertically upward along the equipment chassis. It is rigidly connected to the equipment chassis and is used to describe the kinematic parameters of the equipment chassis.

[0015] The head coordinate system has its origin at the optical center of the virtual reality glasses. The X-axis is along the direction the wearer is looking straight ahead, the Y-axis is along the horizontal left direction the wearer is facing, and the Z-axis is along the vertical upward direction the wearer is facing. It is rigidly connected to the virtual reality glasses and is used to describe the kinematic parameters of the wearer's head.

[0016] The virtual spherical coordinate system has its origin at the optical center of the virtual reality glasses, coinciding with the origin of the head coordinate system. The radius of the sphere is a fixed value R in meters, used for rendering panoramic images and distortion correction. Each pixel on the sphere corresponds to a unique spatial direction vector.

[0017] The lens optical coordinate system, with its origin at the optical center of the virtual reality glasses lens and its optical axis coinciding with the X-axis of the head coordinate system, is used to describe the optical mapping relationship of light passing through the lens.

[0018] The transformation between the vehicle coordinate system and the geographic coordinate system is achieved through the direction cosine matrix obtained from real-time attitude calculation. The transformation formula is as follows: in, A vector in a geographic coordinate system. This is a vector in the vehicle coordinate system. The direction cosine matrix from the vehicle coordinate system to the geographic coordinate system at time t is obtained by calculating the attitude angles output by the inertial measurement unit.

[0019] The conversion between the head coordinate system and the geographic coordinate system is achieved through the pose angles output by the head pose tracking module of the virtual reality glasses. The conversion formula is as follows: in, This is a vector in the head coordinate system. Let be the direction cosine matrix from the head coordinate system to the geographic coordinate system at time t.

[0020] In this solution, all sensor data, computational processes, and control outputs are executed based on a unified clock reference. The PTP precision time protocol is used to achieve clock synchronization of all hardware modules, with a synchronization accuracy of no less than 1ms. All acquired data is accompanied by a unique hardware timestamp. Timing alignment is achieved based on the timestamps during data processing, and linear interpolation is used to achieve time dimension synchronization alignment for data with different sampling frequencies.

[0021] Network transmission delay Real-time measurement is performed via bidirectional heartbeat packet exchange between the device and the operator, with a measurement frequency of no less than 10Hz. The measurement formula is as follows: in, The timestamp of the first heartbeat packet sent by the operator. The timestamp for the device receiving the first heartbeat packet. The timestamp for the heartbeat response packet sent by the device. This is the timestamp for the heartbeat response packet received by the operator. The measured network latency is smoothed using a first-order low-pass filter, with the following filter formula: in, These are the filter coefficients, and their values ​​range from [value range missing]. , This is the heartbeat packet sampling period.

[0022] Wheel rolling radius calibration: Place the equipment chassis on a flat, hard surface, mark the initial position of the wheels, control the equipment chassis to travel a fixed straight distance L, record the number of wheel rotations N, and the formula for calculating the wheel rolling radius r is: For soft road surfaces in fields, a settlement correction factor is introduced. Corrected rolling radius , The range of values ​​is Adjustments are made based on actual measurements of soil hardness in the field.

[0023] Equipment chassis moment of inertia calibration: The moment of inertia of the equipment chassis about the vertical axis is calibrated using the trilinear pendulum method. The unit is kilogram-square meter. During calibration, the full-load condition of field operations is simulated to ensure that the calibration parameters match the actual working conditions. When the load changes during operation, the moment of inertia is corrected in real time based on the load mass and distribution location. The correction formula is: in, The mass of the i-th newly added load, Let be the horizontal distance of the i-th load relative to the centroid.

[0024] Virtual Reality Glasses Lens Optical Parameter Calibration: The Zhang calibration method is used to complete the calibration of the internal distortion coefficients of the lens, obtain the equivalent focal length, principal point coordinates, radial distortion coefficient and tangential distortion coefficient of the lens, and establish a mapping model of light passing through the lens.

[0025] Motor parameter calibration was performed: the back electromotive force constant of the motor was obtained by actual measurement through no-load and locked-rotor tests. Torque constant Coil internal resistance All parameters are in the International System of Units (SI) to ensure dimensional consistency.

[0026] The translational acceleration and actual translational linear velocity of the equipment chassis are calculated using a multi-sensor fusion algorithm via an extended Kalman filter. The state vector is defined as follows: in, This is the position vector of the equipment chassis in the geographic coordinate system. This represents the actual translational linear velocity vector of the equipment chassis in the geographic coordinate system. Let be the attitude angle vector of the equipment chassis. For the accelerometer zero bias vector, This is the zero bias vector of the gyroscope.

[0027] The process equations are established based on the kinematic model of the inertial measurement unit, and the observation equations are established based on the wheel speed data output by the wheel speed gauge. Through iterative calculations of prediction steps and update steps, the translational acceleration and actual translational linear velocity of the equipment chassis are calculated in real time, and the calculation frequency is consistent with the sampling frequency of the inertial measurement unit.

[0028] The measured linear acceleration of the equipment chassis output by the inertial measurement unit (IMU) and the translational acceleration calculated by the wheel speedometer are obtained. Gravity removal processing is then performed to obtain the gravity-removed linear acceleration vector. All vectors involved in the calculation are pre-converted to a geographic coordinate system. The calculation formula is as follows: in, Let t be the degravity-free linear acceleration vector in the geographic coordinate system at time t, in meters per second squared. The measured linear acceleration vector of the equipment chassis, collected by the inertial measurement unit at time t and transformed to the geographic coordinate system, is expressed in meters per second squared. The vector of translational acceleration of the equipment chassis in the geographic coordinate system is obtained by the fusion calculation at time t, and the unit is meters per second squared. The standard gravitational acceleration vector in the geographic coordinate system has a magnitude of 9.81 meters per second squared and is directed along the negative Z-axis of the geographic coordinate system; t is the time variable in seconds.

[0029] Calculate the magnitude of the acceleration vector away from the gravitational lines and divide it by the magnitude of the standard gravitational acceleration to obtain the dimensionless dynamic weight. The calculation formula is: in, is the dimensionless dynamic weight at time t, with a value range of non-negative real numbers, representing the degree of impact on the equipment chassis caused by bumps on the field road surface; To determine the magnitude of the acceleration vector along the lines of gravity, This represents the magnitude of standard gravitational acceleration. When the equipment chassis travels at a constant speed on a flat road, the magnitude of the degravitational acceleration vector approaches 0, and the dimensionless dynamic weight also approaches 0. When the equipment chassis experiences severe bumps or jolts after passing through ditches or mud, the magnitude of the degravitational acceleration vector increases, and the dimensionless dynamic weight increases simultaneously. (This is for...) In extreme operating conditions approaching 0, the dimensionless dynamic weight is directly set to 0 to avoid the error of dividing by 0.

[0030] The true angular acceleration is obtained by taking the first derivative of the true angular velocity with respect to time. The true angular velocity is the chassis angular velocity of the equipment, collected by the inertial measurement unit and transformed to a geographic coordinate system. The calculation formula is: in, Let t be the actual angular acceleration of the equipment chassis in the geographic coordinate system at time t, in radians per second squared. Let be the true angular velocity of the equipment chassis in the geographic coordinate system at time t, in radians per second. For discretely sampled time-series data, the true angular acceleration is approximated using the first-order backward difference method, while a moving average filter is introduced to suppress high-frequency noise. The calculation formula is as follows: in, At the current sampling time, For the previous sampling time, The sampling period of the inertial measurement unit is M, and the moving average window length has a value range of [value missing]. .

[0031] The impact angular acceleration is obtained by multiplying the true angular acceleration by the dimensionless dynamic weight. The calculation formula is: in, Let be the impact angular acceleration in the geographic coordinate system at time t, expressed in radians per second squared. This represents the angular impact component of the equipment chassis caused by bumps in the field road, which can lead to image oscillations. The impact angular acceleration only retains the angular acceleration component caused by road bumps, filtering out the angular acceleration component generated by the normal turning of the equipment chassis.

[0032] The impact angular acceleration is integrated over time within the current hardware frame refresh cycle to obtain the compensated angular velocity vector. The integration operation is performed synchronously with the hardware frame refresh of the virtual reality glasses, and the calculation formula is as follows: in, is the compensated angular velocity vector in the geographic coordinate system at time t, in radians per second; The hardware frame refresh cycle of virtual reality glasses, measured in seconds, is determined by the device's hardware parameters. Let be the integration time variable. For discrete sampled data, the definite integral is implemented using the trapezoidal integration method to ensure the accuracy of the integration operation. The calculation formula is as follows: Where N is the number of sampling points within a single hardware frame refresh cycle. It represents the i-th sampling moment within the period.

[0033] The compensated angular velocity vector is transformed to the head coordinate system and then subjected to a three-dimensional outer product operation with the virtual spherical pixel position vector to generate a reverse optical flow velocity field. The direction of the reverse optical flow velocity field is opposite to the direction of image oscillation caused by device chassis vibration, and is used to counteract angular jitter in the image. The calculation formula is as follows: in, Let t be the direction vector of the corresponding virtual sphere pixel. The reverse optical flow velocity field, measured in meters per second; Let be the direction cosine matrix from the geographic coordinate system to the head coordinate system at time t. The transpose of the matrix; This is the outer product operator for three-dimensional vectors, and the order of operations strictly follows the formula definition. The unit vector representing the pixel direction of the virtual sphere represents the spatial orientation of each pixel in the virtual sphere relative to the optical center of the lens. R is the position vector of the virtual sphere pixels, R is the radius of the virtual sphere in meters, and R is a fixed preset parameter.

[0034] Obtain the off-axis angle corresponding to the light ray penetrating the lens. The off-axis angle is the angle between the direction of light propagation and the optical axis of the lens when the light ray enters the lens, expressed in radians, and its value ranges from [value missing]. ,in The maximum field of view of the lens is determined by the lens's optical design parameters. For any pixel on the virtual sphere, the formula for calculating its corresponding off-axis angle is: in, It is the unit direction vector of the lens optical axis, which coincides with the X-axis of the head coordinate system.

[0035] Taking the partial derivative of the tangent of the off-axis angle with respect to the off-axis angle, the expression is as follows: The partial derivative is used as the Jacobian gradient of magnification. At the same time, a higher-order correction term for the radial distortion of the lens is introduced to ensure that the optical model matches the actual lens characteristics. The calculation formula is as follows: in, For the corresponding off-axis angle The magnification Jacobian gradient is a dimensionless scalar that characterizes the rate of change of angular magnification of the lens at that off-axis position. , , This is the radial distortion coefficient of the lens, obtained from the lens optical calibration process.

[0036] Substituting the off-axis angle corresponding to the virtual sphere pixel direction vector into the magnification Jacobian gradient, we obtain the magnification Jacobian gradient for each virtual sphere pixel, i.e. ,in The virtual spherical pixel direction vector The corresponding off-axis angle.

[0037] The scalar multiplication of the reverse optical flow velocity field with the Jacobian gradient of magnification after substituting the angle yields the distorted optical flow vector. The calculation formula is as follows: in, Let t be the direction vector of the corresponding virtual sphere pixel. The distorted optical flow vector, in meters per second, characterizes the optical flow field distribution after nonlinear amplification by the lens.

[0038] The spherical divergence operator is applied to the distorted optical flow vector to calculate the optical flow deformation divergence field. The spherical divergence operator employs a complete vector analysis expansion to ensure accurate results; the calculation formula is as follows: in, Let t be the direction vector of the corresponding virtual sphere pixel. The optical flow divergence field, in units of 1 second; This is a spherical divergence operator used to calculate the divergence value of a spherical vector field at the corresponding pixel, characterizing the spatial deformation of the optical flow field. For discrete pixel meshes, the spherical divergence operator uses the finite difference method for discretization calculation, and mirror boundary conditions are applied to pixels on the spherical boundary to ensure the accuracy of boundary operations.

[0039] Obtain the original spherical pixel orientation vector and the radial unit vector outward from the optical center of the lens. Original spherical pixel orientation vector As a unit vector, it represents the standard spatial orientation of the corresponding pixel on the virtual sphere during the original rendering of the panoramic image; radial unit vector. Let be the unit vector radially outward from the optical center of the lens along the original spherical pixel direction vector, and be... The direction is the same, that is .

[0040] The dimensionless deformation scalar is obtained by definite integral over the optical flow deformation divergence field within the hardware frame refresh cycle. The calculation formula is as follows: in, Let t be the direction vector of the original spherical pixel. The dimensionless deformation scalar represents the cumulative deformation of the image at a given pixel within a single hardware frame refresh cycle. For discrete sampled data, the definite integral is implemented using the trapezoidal integral method, and a limiting threshold is set for the dimensionless deformation scalar, with a limiting range of [-0.2, 0.2], to prevent excessively large deformation scalars from causing inverse pixel mapping distortion.

[0041] Multiplying the dimensionless deformation scalar by the radial unit vector yields the displacement deviation vector. The calculation formula is: in, Let t be the direction vector of the original spherical pixel. The displacement deviation vector, which is on the order of a unit vector, represents the reverse radial compensation displacement that needs to be performed on this pixel.

[0042] Subtract the displacement deviation vector from the original spherical pixel direction vector to obtain the difference direction vector. The calculation formula is: in, Let t be the direction vector of the original spherical pixel. The differential direction vector is the spatial direction vector of the pixel point after reverse radial compensation.

[0043] The distortion-mapped coordinates are obtained by normalizing the difference direction vector by dividing it by its magnitude. The calculation formula is as follows: in, Let be the distortion-mapped coordinates at time t, and be a unit vector representing the final pixel spatial orientation used for rendering. For the extreme case where the magnitude of the difference direction vector approaches 0, the distortion-mapped coordinates are directly taken from the original spherical pixel direction vector to avoid division by zero errors. Based on the distortion-mapped coordinates, the rendering engine performs pixel remapping on the original panoramic image. The remapping process uses a bicubic interpolation algorithm, and neighborhood filling is used to handle pixel holes that appear after mapping. The pre-distorted image is then output to the display panel of the virtual reality glasses.

[0044] The head angular velocity collected by the virtual reality glasses' head posture tracking module is acquired, converted to a geographic coordinate system, and the square of the head angular velocity modulus is calculated to obtain the active angular velocity square term. The calculation formula is as follows: in, Let be the square term of the active angular velocity at time t, in square radians per second squared. The angular velocity of the human head at time t, converted to the geographic coordinate system, is expressed in radians per second and represents the wearer's active head rotation.

[0045] Based on the measured network latency, the actual angular velocity of the device chassis is time-aligned to obtain the synchronized angular velocity of the device chassis after network latency alignment. The square of its modulus is then calculated to obtain the square term of the stimulated angular velocity. The calculation formula is as follows: in, Let be the square term of the excited angular velocity at time t, in square radians per second squared. The device chassis synchronization angular velocity after network latency alignment is expressed in radians per second. The value at the alignment time is obtained by linear interpolation of historical time-series data.

[0046] The energy dissipation ratio is obtained by dividing the squared term of the active angular velocity by the sum of the squared terms of the active and stimulated angular velocities. The calculation formula is as follows: in, Let be the energy dissipation ratio at time t, and be a dimensionless parameter with a value range of . ; It is a very small positive number, taking the value 1e-6, to avoid calculation errors where the denominator is 0. When the wearer actively rotates their head, the head angular velocity modulus increases, the active angular velocity square term dominates, and the energy dissipation ratio approaches 1, preserving the wearer's control intention. When the device chassis passively rotates due to road bumps, the device chassis synchronous angular velocity modulus increases, the stimulated angular velocity square term dominates, and the energy dissipation ratio approaches 0, filtering out non-controllable angular motion caused by road bumps and achieving decoupling of control intention from interference disturbances.

[0047] The initial angular acceleration is obtained by taking the first derivative of the head angular velocity with respect to time. The command angular acceleration is then obtained by multiplying the initial angular acceleration by the energy dissipation ratio. The calculation formula is as follows: in, The command angular acceleration in the geographic coordinate system at time t is expressed in radians per second squared, and represents the clean control command after decoupling. Let t be the initial angular acceleration in the geographic coordinate system at time t, in radians per second squared. For discrete sampled data, a first-order backward difference method combined with moving average filtering is used to calculate and suppress high-frequency noise.

[0048] The feedforward inertial torque is obtained by multiplying the real-time rotational moment of inertia of the equipment chassis by the commanded angular acceleration. The calculation formula is: in, The feedforward inertial torque at time t, measured in Newton-meters, is used to compensate for the inertial load when the equipment chassis rotates, enabling a rapid response to control commands.

[0049] Obtain the linear velocity of each drive wheel and the actual translational linear velocity of the equipment chassis, and calculate the instantaneous slip ratio. The wheel linear velocity is the linear velocity of the wheel center along the direction of travel of the equipment chassis, obtained by multiplying the wheel speed collected by the wheel speedometer by the corrected rolling radius; the actual translational linear velocity of the chassis is the linear velocity along the direction of travel of the equipment chassis obtained through fusion calculation, and the calculation formula is: in, Let be the instantaneous slip rate at time t, and be a dimensionless parameter with a value range of . This characterizes the degree of slippage between the wheel and the field surface. Let t be the linear velocity of the wheel in the vehicle coordinate system at time t, in meters per second; Let t be the actual translational linear velocity of the chassis in the vehicle coordinate system at time t, in meters per second; It is a very small positive number, with a value of 1e-6, used to avoid calculation errors with a denominator of 0. For multi-drive wheel structures, the average slip ratio of all drive wheels is taken as the final instantaneous slip ratio, and the slip ratio limit range is set to [0,1] to avoid interference from invalid values.

[0050] The target angular velocity is obtained by definite integral of the commanded angular acceleration from the starting point of time to the current time. Anti-integral saturation processing is introduced to limit the maximum amplitude of the target angular velocity. The calculation formula is: in, t represents the target angular velocity in the geographic coordinate system at time t, in radians per second, and t represents the required rotational angular velocity of the equipment chassis. For a saturation limiting function, when the magnitude of the integral result is greater than the maximum permissible angular velocity... At that time, the output modulus is A vector whose direction is consistent with the integral result. The integral is determined by the mechanical performance parameters of the equipment chassis. For discrete sampled data, the definite integral is implemented using the trapezoidal integral method, and an integral reset mechanism is introduced to reset the initial integral value when the equipment chassis stops moving, thus eliminating accumulated integral error.

[0051] The equivalent electromagnetic damping coefficient is obtained by multiplying the back electromotive force constant of the actuator by the torque constant and dividing by the coil internal resistance. The calculation formula is as follows: in, The equivalent electromagnetic damping coefficient, in Newton-meter-second-per-radian, characterizes the electromagnetic damping characteristics of the actuator itself. The back electromotive force constant of the motor is given in volt-seconds per radian. The torque constant of the motor is expressed in Newton-meters per ampere. The internal resistance of the motor coil is measured in ohms.

[0052] Transform the target angular velocity into the vehicle coordinate system, and then continuously multiply the instantaneous slip ratio, equivalent electromagnetic damping coefficient, target angular velocity, and yaw unit normal vector to obtain the back electromotive force drag torque. The yaw unit normal vector is the unit vector along the Z-axis in the vehicle coordinate system. The rule for continuous multiplication is as follows: scalar parameters are sequentially multiplied, then multiplied by the target angular velocity vector, and finally multiplied by the yaw unit normal vector to ensure the torque direction matches the rotation direction. The calculation formula is: in, Let be the back electromotive force resistance torque in the vehicle body coordinate system at time t, in Newton-meters; The target angular velocity at time t, converted to the vehicle coordinate system, is expressed in radians per second. This is the yaw unit normal vector in the vehicle coordinate system, along the positive direction of the Z-axis of the vehicle coordinate system; This is the outer product operator for three-dimensional vectors. The back electromotive force (EMF) resistance torque is used to compensate for the resistance caused by wheel slippage and motor electromagnetic damping, ensuring that the actual rotational angular velocity of the equipment chassis accurately follows the target angular velocity.

[0053] The feedforward inertial torque is transformed to the vehicle coordinate system and added to the back electromotive force drag torque to obtain the final drag-resistant driving torque. The calculation formula is as follows: in, is the resistance driving torque in the vehicle coordinate system at time t, in Newton-meters, and is the final control command output to the actuator motor; Let be the direction cosine matrix from the geographic coordinate system to the vehicle coordinate system at time t.

[0054] For the differential drive equipment chassis, the resistive drive torque is distributed to the drive motors on the left and right sides using the following formula: in, This is the output torque of the left-side drive motor. The output torque of the right-side drive motor is expressed in Newton-meters. The forward driving force of the equipment chassis is determined by the wearer's forward control commands; r is the corrected wheel rolling radius.

[0055] The motor driver executes closed-loop PID control based on the allocated torque command, with the control frequency matching the motor control frequency. The transfer function of the PID controller is: in, This is the proportionality coefficient. The integral coefficient is... These are the differential coefficients, and the parameters are tuned based on motor characteristics and field conditions. A saturation limit is set for the motor output torque, with the limit value being the motor's maximum rated output torque. Simultaneously, an anti-integral saturation algorithm is introduced to avoid integral windup issues, ensuring the stability and response speed of motor control.

[0056] For common abnormal working conditions during field operations, corresponding fault-tolerant handling mechanisms are set up.

[0057] Sensor data anomaly handling: When the data from the inertial measurement unit, wheel speedometer, and head attitude tracking module exceed the preset reasonable range, or when continuous packet loss occurs, the valid data from the previous frame is used to maintain the data, and a fault warning is triggered. When the anomaly lasts for more than 100ms, the control device chassis decelerates to a stop.

[0058] Network anomaly handling: When network latency exceeds the preset maximum threshold, or when network disconnection occurs, immediately stop all movement of the device chassis, maintain the current posture, and wait until the network returns to normal.

[0059] Handling of rendering anomalies: When the panoramic camera image stutters or is obstructed, the pre-distortion mapping parameters of the previous frame are maintained, while the image refresh rate is reduced until the image returns to normal.

[0060] Wheel slippage and lock-up handling: When the instantaneous slip rate exceeds the preset threshold of 0.8, anti-slip control is triggered, the motor output torque is reduced, and the torque distribution between the left and right wheels is adjusted until the slip rate returns to a reasonable range.

[0061] To address the limited computational precision and stringent real-time requirements of embedded systems, corresponding safeguards are implemented.

[0062] Numerical stability handling: All trigonometric function operations are implemented using polynomial approximation to adapt to the operation characteristics of embedded processors; all division operations are protected against extremely small positive numbers to avoid division by zero errors; all floating-point operations are in single-precision floating-point format, and a numerical overflow detection mechanism is set up to trigger amplitude limiting when overflow occurs.

[0063] Real-time performance guarantee: All algorithm operations are scheduled according to priority, with the image pre-distortion processing task having the highest priority, followed by the motor closed-loop control task, and the sensor data fusion task having the third priority, ensuring that the end-to-end image compensation delay does not exceed 10ms and the motor control command delay does not exceed 5ms; for pixel-by-pixel pre-distortion operations, a parallel computing architecture is adopted to improve computing efficiency and meet real-time requirements.

[0064] Noise suppression: All high-frequency sampled data are preprocessed using a first-order low-pass filter. The filter cutoff frequency is set according to the signal characteristics. The filter cutoff frequency for acceleration and angular velocity data is set to 50Hz to avoid high-frequency noise being amplified by differential operations and to ensure the stability of core parameter calculations.

[0065] The virtual sphere uses an equal rectangular projection model to realize the spherical unfolding of the panoramic image. A one-to-one mapping relationship is established between the pixel direction vector of the virtual sphere and the two-dimensional pixel coordinates of the virtual reality glasses display panel. The mapping process is divided into two stages: forward mapping and reverse mapping.

[0066] Forward mapping is used to convert the virtual sphere pixel direction vector into two-dimensional pixel coordinates of the display panel. The calculation formula is as follows: Where u is the horizontal coordinate of the display panel pixels and v is the vertical coordinate of the display panel pixels, both in pixels; The azimuth angle of the virtual sphere pixel direction vector, in radians, has a range of values ​​of 1. ; The polar angle, or off-axis angle, of the virtual sphere pixel direction vector, is expressed in radians and has a range of values ​​of [value missing]. W represents the horizontal resolution of the display panel, and H represents the vertical resolution of the display panel; both are in pixels.

[0067] Inverse mapping is used to convert the two-dimensional pixel coordinates of the display panel into virtual spherical pixel direction vectors, providing a basic mapping relationship for pre-distortion processing. The calculation formula is as follows: in, This is the original spherical pixel orientation unit vector, corresponding to the spatial orientation of the display panel coordinates (u, v). For invalid pixel areas at the edges of the display panel, an effective field of view clipping rule is set, only applying to pixels with off-axis angles smaller than the lens's maximum field of view. Pixels within the specified range undergo pre-distortion processing, while pixels outside the range retain their original mapping relationship.

[0068] For a equirectangular projected mesh of a virtual sphere, the spherical divergence operator is discretized using the finite difference method. The formula for calculating the spherical divergence value for any pixel (i,j) on the sphere is as follows: Where R is the radius of the virtual sphere, in meters; The polar angle corresponding to the pixel in the j-th row. The polar angle interval between adjacent rows of pixels, in radians; The azimuth interval between adjacent columns of pixels, in radians; Let be the azimuth component of the distorted optical flow vector at pixel (i,j). Let be the polar angular components of the distorted optical flow vector at pixel (i,j), all in meters per second. For pixels at the poles of the sphere, a neighborhood averaging method is used to handle singular values ​​to avoid distortion of the calculation results; for pixels at the grid boundary, a mirror boundary condition is used to supplement the neighboring pixel values.

[0069] For high-frequency boundary conditions during field operations, corresponding image compensation and control logic are added to ensure dizziness suppression and control smoothness under all conditions.

[0070] For the scenario of pure translational vertical bumps on the equipment chassis, i.e., the equipment chassis only experiences vertical linear acceleration impact without angular acceleration, a translational impact compensation logic is added. First, the vertical impact acceleration of the equipment chassis is extracted, and the calculation formula is: in, Let t be the vertical impact acceleration at time t, expressed in meters per second squared. This is the unit vector of the Z-axis in the geographic coordinate system. A compensated optical flow field in the vertical direction is generated based on vertical impact acceleration; the calculation formula is as follows: in, For vertical compensation optical flow field, the unit is meters per second; This is the unit vector along the Z-axis of the head coordinate system. The vertical compensation optical flow field and the angular compensation optical flow field are superimposed and jointly participate in the subsequent calculation of the distortion optical flow vector and divergence field to achieve image stabilization compensation under pure translational turbulence conditions.

[0071] For normal driving conditions with pure steering of the equipment chassis, i.e., scenarios where the equipment chassis has stable angular acceleration and no bumpy acceleration impacts, smoothing logic for the steering screen is added. A minimum trigger threshold is set for the energy dissipation ratio. The value is 0.1. When the energy dissipation ratio is less than the minimum trigger threshold, a first-order low-pass filter is applied to the command angular acceleration for smoothing. The filter formula is: in, These are the smoothing filter coefficients, and their values ​​range from [value range missing]. This avoids sudden changes in control commands during normal steering and improves driving smoothness.

[0072] Filtering-related parameters: moving average window length M ranges from 35; network delay filter coefficients. The range of values ​​is , steering smoothing filter coefficient The range of values ​​is The low-pass filter cutoff frequency range is: The parameter tuning principle is: the bumpier the field road, the larger the values ​​of the filter coefficient and window length, and the lower the cutoff frequency.

[0073] Minimum trigger threshold for energy dissipation ratio The value is 0.1, and the dimensionless deformation scalar limit range is... The slip ratio anti-slip threshold is set to 0.8. The parameter tuning principle is: the higher the steering sensitivity requirement of the equipment chassis, the smaller the minimum trigger threshold value; the larger the lens field of view, the larger the deformation scalar limit range value.

[0074] Motor control related parameters, PID controller proportional coefficient The range of values ​​is Integral coefficient The range of values ​​is Differential coefficients The range of values ​​is The parameter tuning principle is as follows: the larger the rated torque of the motor, the larger the values ​​of the proportional coefficient and integral coefficient; the lower the adhesion of the field road surface, the larger the value of the derivative coefficient.

[0075] The embodiments of this example have been described above. However, this example is not limited to the specific implementation methods described above. The specific implementation methods described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms based on the guidance of this example, and all of them are within the protection scope of this example.

Claims

1. A VR glasses based field unmanned device remote control method, applied to a system comprising a device chassis, an inertial measurement unit and a wheel speed meter arranged on the device chassis, a panoramic camera, an execution motor, and a virtual reality glasses comprising a lens, characterized in that, Configured for execution: Based on the measured linear acceleration and true angular velocity of the equipment chassis obtained by the inertial measurement unit, and the translational acceleration calculated by the wheel speed meter, the impact angular acceleration is extracted. Within the hardware frame refresh cycle, the impact angular acceleration is integrated over time and cross-multiplied with the virtual spherical pixel direction vector to generate a reverse optical flow velocity field. Based on the mapping relationship of light passing through the lens, the analytical partial derivative with respect to the off-axis angle is calculated to obtain the Jacobian gradient of magnification; The optical flow velocity field is multiplied by the magnification Jacobian gradient, and the spherical divergence operator is applied to calculate the optical flow deformation divergence field. Based on the optical flow deformation divergence field, the original spherical pixel orientation vector is subjected to reverse radial pre-distortion processing, and the distortion mapping coordinates are output. The head angular velocity and initial angular acceleration of the human body are obtained. The energy dissipation ratio is constructed by combining the synchronized angular velocity of the device chassis after network latency alignment, and the command angular acceleration is extracted. The instantaneous slip ratio is calculated by acquiring the linear velocity of the wheel and the actual translational linear velocity of the chassis. The command angular acceleration is converted into the target angular velocity. The anti-resistance driving torque is calculated by combining the instantaneous slip ratio and the equivalent electromagnetic damping coefficient, and the actuator motor is controlled.

2. The method for remote control of unmanned field equipment based on VR glasses according to claim 1, characterized in that, Based on the measured linear acceleration and true angular velocity of the equipment chassis obtained by the inertial measurement unit, and the translational acceleration calculated by the wheel speed meter, the impact angular acceleration is extracted, including: Subtract the translational acceleration and the standard gravitational acceleration from the measured linear acceleration to obtain the gravitational-free linear acceleration vector; Calculate the magnitude of the degravation acceleration vector and divide it by the magnitude of the standard gravitational acceleration to obtain the dimensionless dynamic weight. The true angular acceleration is obtained by taking the first derivative of the true angular velocity with respect to time. The impact angular acceleration is obtained by multiplying the true angular acceleration by the dimensionless dynamic weight.

3. The method for remote control of unmanned field equipment based on VR glasses according to claim 2, characterized in that, Within the hardware frame refresh cycle, the impact angular acceleration is integrated over time and cross-multiplied with the virtual spherical pixel direction vector to generate a reverse optical flow velocity field, including: The impact angular acceleration is integralized over time within the current hardware frame refresh cycle to obtain the compensation angular velocity vector. The reverse optical flow velocity field is generated by performing a three-dimensional outer product operation between the compensated angular velocity vector and the virtual spherical pixel direction vector.

4. The method for remote control of unmanned field equipment based on VR glasses according to claim 3, characterized in that, Based on the mapping relationship of light transmission through the lens, the analytical partial derivative with respect to the off-axis angle is calculated to obtain the Jacobian gradient of magnification, including: Obtain the off-axis angle corresponding to the light passing through the lens; Take the partial derivative of the tangent of the off-axis angle with respect to the off-axis angle; The partial derivative is used as the magnification Jacobian gradient, which is equal to the reciprocal of the square of the cosine of the off-axis angle.

5. The method for remote control of unmanned field equipment based on VR glasses according to claim 4, characterized in that, The optical flow velocity field is multiplied by the amplification Jacobian gradient, and the spherical divergence operator is applied to calculate the optical flow deformation divergence field, including: Substitute the off-axis angle corresponding to the virtual spherical pixel direction vector into the magnification Jacobian gradient; The distorted optical flow vector is obtained by performing a tensor dot product between the reverse optical flow velocity field and the magnification Jacobian gradient after substituting the angle. The spherical divergence operator is applied to the distorted optical flow vector to calculate the optical flow deformation divergence field.

6. The method for remote control of unmanned field equipment based on VR glasses according to claim 5, characterized in that, Based on the optical flow deformation divergence field, the original spherical pixel direction vector is subjected to inverse radial pre-distortion processing, and the distortion mapping coordinates are output, including: Obtain the original spherical pixel direction vector and the radial unit vector from the optical center of the lens outward; Within the hardware frame refresh cycle, the optical flow deformation divergence field is integrally integrated to obtain a dimensionless deformation scalar. Multiplying the dimensionless deformation scalar by the radial unit vector yields the displacement deviation vector; Subtract the displacement deviation vector from the original spherical pixel direction vector to obtain the differential direction vector; The distortion mapping coordinates are obtained by normalizing the differential direction vector by dividing it by its magnitude.

7. The method for remote control of unmanned field equipment based on VR glasses according to claim 6, characterized in that, The head angular velocity and initial angular acceleration of the human body are obtained. Combined with the network latency-aligned device chassis synchronous angular velocity, an energy dissipation ratio is constructed. The command angular acceleration is extracted, including: Calculate the square of the head angular velocity modulus to obtain the active angular velocity square term; Calculate the square of the synchronous angular velocity modulus of the device chassis after network delay alignment to obtain the excited angular velocity square term; The energy dissipation ratio is obtained by dividing the squared term of the active angular velocity by the sum of the squared term of the active angular velocity and the squared term of the stimulated angular velocity. The commanded angular acceleration is obtained by multiplying the initial angular acceleration by the energy dissipation ratio.

8. The method for remote control of unmanned field equipment based on VR glasses according to claim 7, characterized in that, The process includes: acquiring the wheel linear velocity and the actual translational linear velocity of the chassis to calculate the instantaneous slip ratio; converting the commanded angular acceleration into a target angular velocity; calculating the resistive driving torque by combining the instantaneous slip ratio with the equivalent electromagnetic damping coefficient; and controlling the actuator motor. Multiply the moment of inertia of the equipment chassis by the commanded angular acceleration to obtain the feedforward inertial torque; The difference vector is obtained by subtracting the actual translational linear velocity of the chassis from the linear velocity of the wheel. The magnitude of the difference vector is calculated and divided by the magnitude of the linear velocity of the wheel to obtain the instantaneous slip ratio. The target angular velocity is obtained by integrating the commanded angular acceleration from the time start point to the current time. The equivalent electromagnetic damping coefficient is obtained by multiplying the back electromotive force constant of the actuator motor by the torque constant and dividing by the coil internal resistance. The instantaneous slip ratio, the equivalent electromagnetic damping coefficient, the target angular velocity, and the yaw unit normal vector are continuously multiplied to obtain the back electromotive force drag torque; The feedforward inertial torque is added to the back electromotive force drag torque to obtain the final drag driving torque.

Images