Device control method, device, storage medium and electronic device

Abstract

Embodiments of the present invention provide a device control method, apparatus, storage medium, and electronic device. The method includes: determining first motion information collected by a first acquisition device connected to a motion controller and included in a target device; if the first motion information is determined to be invalid and the target device has a pan / tilt platform, obtaining second motion information collected by a second acquisition device connected to the pan / tilt platform and included in the target device; determining first posture information based on the second motion information; and controlling the target device to operate in accordance with the first posture information. This invention solves the problem, existing in related technologies, of being unable to precisely control a device due to inaccurate determination of the device's posture information, thereby achieving precise control of the device's operation.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of communications, and in particular, to a device control method, device, storage medium, and electronic device. Background Art

[0002] Controlling the attitude of a device plays a crucial role in ensuring operational stability, precise control, and mission execution. Conventional attitude control systems often rely on a single sensor, which can provide basic attitude information under ideal conditions. However, in complex flight environments, a single sensor is susceptible to external interference, such as electromagnetic interference and mechanical vibration, leading to deviations in attitude information and, in turn, impacting operational safety and mission efficiency.

[0003] It can be seen from this that the related art has a problem in which the device cannot be accurately controlled due to inaccurate posture information of the device.

[0004] Currently, no effective solution has been proposed to the above-mentioned problems existing in the related technologies. Summary of the Invention

[0005] Embodiments of the present invention provide a device control method, device, storage medium, and electronic device to at least solve the problem in the related art that the device cannot be accurately controlled due to inaccurate posture information of the device.

[0006] According to one embodiment of the present invention, a method for controlling a device is provided, comprising: determining first motion information collected by a first acquisition device connected to a motion controller in a target device; when it is determined that the first motion information is invalid information and a gimbal exists in the target device, obtaining second motion information collected by a second acquisition device connected to the gimbal included in the target device; determining first posture information based on the second motion information; and controlling the target device to operate according to the first posture information.

[0007] In an exemplary embodiment, determining the first posture information based on the second motion information includes: determining a first angular velocity and a first acceleration included in the second motion information, wherein the first angular velocity is the angular velocity of the gimbal coordinate system relative to the inertial coordinate system, and the first acceleration is the acceleration of the gimbal coordinate system relative to the inertial coordinate system; determining a first correction term based on the first acceleration; correcting the first angular velocity based on the first correction term to obtain a first corrected angular velocity; and determining the first posture information based on the first corrected angular velocity.

[0008] In an exemplary embodiment, determining the first correction term based on the first acceleration includes: converting the first acceleration to the inertial coordinate system to obtain a second acceleration; and determining a first product of the second acceleration and a first preset matrix as the first correction term.

[0009] In an exemplary embodiment, correcting the first angular velocity based on the first correction term to obtain the first corrected angular velocity includes: converting the first correction term from the inertial coordinate system to the gimbal coordinate system to obtain a second correction term; determining a second product of the second correction term and a first preset weight; and determining the sum of the second product and the first angular velocity as the first corrected angular velocity.

[0010] In an exemplary embodiment, after determining that the target device includes first motion information collected by a first collection device connected to a motion controller, the method further includes: when the first motion information is valid information, determining second posture information based on the first motion information; and controlling the target device to operate according to the second posture information.

[0011] In an exemplary embodiment, determining the second posture information based on the first motion information includes: determining a second angular velocity and a third acceleration included in the first motion information, wherein the second angular velocity is the angular velocity of a motion control coordinate system constructed with the motion controller included in the target device relative to an inertial coordinate system, and the third acceleration is the acceleration of the motion control coordinate system relative to the inertial coordinate system; determining a third correction term based on the third acceleration; correcting the second angular velocity based on the third correction term to obtain a second corrected angular velocity; and determining the second posture information based on the second corrected angular velocity.

[0012] In an exemplary embodiment, determining the third correction term based on the third acceleration includes: converting the third acceleration to the inertial coordinate system to obtain a fourth acceleration; and determining the third correction term as the second product of the fourth acceleration and a second preset matrix.

[0013] In an exemplary embodiment, converting the third acceleration to the inertial coordinate system to obtain the fourth acceleration includes: determining a conversion matrix, where when the third acceleration is converted to the inertial coordinate system for the first time, the conversion matrix is ​​a pre-set initial matrix; when the third acceleration is not converted to the inertial coordinate system for the first time, the conversion matrix is ​​a matrix determined based on the second posture information obtained last time; and determining a third product of the third acceleration and the conversion matrix as the fourth acceleration.

[0014] In an exemplary embodiment, correcting the second angular velocity based on the third correction term to obtain the second corrected angular velocity includes: converting the third correction term from the inertial coordinate system to the operation and control coordinate system to obtain a fourth correction term; determining a fourth product of the fourth correction term and a second preset weight; and determining the sum of the fourth product and the second angular velocity as the second corrected angular velocity.

[0015] In an exemplary embodiment, after determining the second posture information based on the first motion information, the method further includes: determining the posture quaternion included in the second posture information; and determining a transformation matrix based on the posture quaternion, wherein the transformation matrix is ​​a matrix used to implement coordinate system transformation.

[0016] According to another embodiment of the present invention, a control device for a device is provided, comprising: a first determination module for determining first motion information collected by a first acquisition device connected to a motion controller in a target device; an acquisition module for acquiring second motion information collected by a second acquisition device connected to a gimbal in the target device when it is determined that the first motion information is invalid information and a gimbal exists in the target device; a second determination module for determining first posture information based on the second motion information; and a control module for controlling the target device to operate according to the first posture information.

[0017] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when run.

[0018] According to another embodiment of the present invention, an electronic device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to perform the steps in any one of the above method embodiments.

[0019] According to yet another embodiment of the present invention, a computer program product is provided, including a computer program, which implements the steps of the method described in each embodiment of the present application when executed by a processor.

[0020] Through the control method of the device provided by the present invention, the first motion information collected by the first acquisition device connected to the motion controller included in the target device can be determined. When the first motion information is invalid information and there is a pan-tilt head in the target device, the second motion information collected by the second acquisition device connected to the pan-tilt head is obtained, the first posture information is determined based on the second motion information, and the target device is controlled to operate according to the first posture information. Since the motion information collected by the first acquisition device is invalid, the second motion information can be collected by the second acquisition device to further determine the first posture information. This achieves that when the first acquisition device fails, the first posture information can still be determined by the information collected by other acquisition devices, and the second acquisition device is a device connected to the pan-tilt head, without the need to set up an additional acquisition device in the target device. Therefore, the problem in the related art that the device cannot be accurately controlled due to inaccurate posture information of the determined device can be solved, and the effect of accurately controlling the operation of the device can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 This is a hardware structure block diagram of a mobile terminal according to a device control method according to an embodiment of the present invention;

[0022] Figure 2 is a flow chart of a method for controlling a device according to an embodiment of the present invention;

[0023] Figure 3 is a flow chart of a control method for a device according to a specific embodiment of the present invention;

[0024] Figure 4 is a schematic diagram of a redundancy design according to an embodiment of the present invention;

[0025] Figure 5 is a structural block diagram of a control device of a device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with embodiments.

[0027] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal according to a device control method of an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown) a processor 102 (the processor 102 may include but is not limited to a microprocessor MCU or a programmable logic device FPGA and other processing devices) and a memory 104 for storing data. The mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It will be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components than shown, or with Figure 1 Different configurations shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the control method of the device in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, implementing the above-mentioned method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely located relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0030] Transmission device 106 is used to receive or transmit data via a network. A specific example of the aforementioned network may include a wireless network provided by the mobile terminal's communications provider. In one embodiment, transmission device 106 includes a network interface controller (NIC), which can be connected to other network devices via a base station to enable communication with the Internet. In another embodiment, transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0031] The following is an explanation of the professional terms that appear in this application:

[0032] Flight Controller Board: The core hardware responsible for flight control in a drone, usually integrating components such as IMU, processor, and communication module.

[0033] Gimbal Attitude Board: Hardware used to control the gimbal's attitude, typically containing sensors such as accelerometers, gyroscopes, and magnetometers, used to stabilize cameras or other payloads.

[0034] Encoder Angle: The mechanical rotation angle measured by an encoder, often used to feedback the position information of a motor or robotic arm.

[0035] Inertial Measurement Unit (IMU): A device that measures the three-axis attitude angle (or angular rate) and acceleration of an object, usually including an accelerometer and a gyroscope.

[0036] Data Fusion: Data from multiple sensors is integrated and processed to improve the accuracy and reliability of information.

[0037] Kalman Filter: An algorithm used to estimate the state of a dynamic system. It can effectively handle noise and uncertainty and is widely used in the field of data fusion.

[0038] Redundancy Design: Designing additional components or functions into the system to improve the reliability and safety of the system so that the system can still operate normally when some components fail.

[0039] In this embodiment, a method for controlling a device is provided. Figure 2 is a flow chart of a control method of a device according to an embodiment of the present invention. Figure 2 As shown, the process includes the following steps:

[0040] Step S202, determining that the target device includes first motion information collected by a first collection device connected to the motion controller;

[0041] In this embodiment, the target device can be a drone or a mobile robot, such as an unmanned vehicle or an unmanned ship, etc. The target device can also be an aerospace vehicle, such as a satellite or a rocket. The motion controller can be the core hardware in the target device for controlling the movement of the target device, that is, the motion controller is the control core of the device, responsible for receiving and processing the motion information from the first acquisition device to control the movement of the device. For example, when the target device is a drone, the motion controller can be a flight control board. When the target device is a mobile robot, the motion controller can be a processor in the mobile robot. The first acquisition device can be a motion information acquisition device connected to the motion controller, and the first acquisition device can be an inertial measurement unit, etc. The first acquisition device can collect the first motion information of the target device, such as angular velocity, acceleration, rotation angle, etc.

[0042] Step S204: if it is determined that the first motion information is invalid and the target device has a pan / tilt platform, obtaining second motion information collected by a second collection device connected to the pan / tilt platform and included in the target device;

[0043] In this embodiment, when the target device is in motion, the first motion information changes in real time. It is possible to determine whether two consecutive pieces of acquired first motion information have been updated. If so, the first motion information is determined to be valid; if not, the first motion information is determined to be invalid. The range of the first acquisition device may also be predetermined. When the first motion information is within the range of the first acquisition device, the first motion information is determined to be valid; when it is outside the range of the first acquisition device, the first motion information is determined to be invalid.

[0044] In this embodiment, the gimbal can be a rotatable platform on a motion device, used to carry sensors, cameras, and other devices. The motion controller can control the rotation of the gimbal and, in turn, the devices mounted on it. Therefore, the target device includes a second acquisition device connected to the gimbal to collect motion information from the gimbal. This second acquisition device can be an inertial measurement unit (IMU). The second motion information can include acceleration, angular velocity, rotation angle, and other information. This second motion information can be obtained via a connection bus, such as a CAN bus.

[0045] Step S206, determining first posture information based on the second motion information;

[0046] In this embodiment, the first posture information can be a quaternion of the target device. Quaternions can be used to efficiently update and express posture changes because they only require four parameters to fully describe a 3D rotation, while matrices require nine parameters. This makes quaternions a highly efficient, compact, and easy-to-use choice.

[0047] In this embodiment, a quaternion can be initialized first. The quaternion (Q(t)) represents the attitude of the gimbal relative to the inertial coordinate system. During initialization, it can be assumed that the gimbal is in an ideal attitude, that is, (Q(0) = [1, 0, 0, 0]) or equivalently no rotation. The angular velocity included in the second motion information can be filtered or calibrated to obtain a more accurate angular velocity estimate. Using the angular velocity, the quaternion can be continuously updated through numerical integration to estimate attitude changes. The acceleration is used to estimate the representation of the gravity vector in the current coordinate system, and the updated quaternion is then corrected. Since the attitude estimation deviation caused by the accumulated error of the integration can be calibrated based on the acceleration, the quaternion attitude estimation is further optimized by converting the acceleration data into an attitude angle update and combining it with the gyroscope data using an extended Kalman filter (EKF) or a complementary filter. The attitude estimate, which changes rapidly but has long-term errors, obtained by integrating the angular velocity is fused with the slowly changing but relatively accurate gravity direction provided by the accelerometer to reduce the drift of the attitude estimate.

[0048] Step S208: Control the target device to operate according to the first posture information.

[0049] In this embodiment, after determining the first attitude information, the target device can be controlled to operate according to the first attitude information. When the data from the first acquisition device is invalid, switching to the data from the second acquisition device can ensure the accuracy and continuity of the device attitude information, thereby improving the stability and safety of the device. For example, during the flight of a drone, if the main IMU is interfered with or fails, the data from the backup IMU can be used for attitude control in a timely manner, avoiding loss of control or crashing, and ensuring the successful completion of the flight mission.

[0050] In the above embodiments, the control method of the device can be applied in different scenarios according to the usage scenario of the target device. For example, it can be applied in the UAV flight control scenario to provide more stable and accurate attitude information when the UAV performs aerial photography, mapping, inspection and other tasks to ensure flight safety and mission quality. It can also be applied in robot navigation and positioning scenarios to achieve high-precision attitude estimation in mobile robots such as unmanned vehicles and unmanned ships, thereby improving the accuracy of navigation and positioning. It can also be applied in virtual reality and augmented reality scenarios to provide low-latency, high-precision attitude information in VR / AR devices to enhance user experience. It can also be applied in aerospace scenarios to achieve precise attitude control in satellites, rockets and other aerospace vehicles to ensure the smooth completion of the mission.

[0051] Through the control method of the device provided by the present invention, the first motion information collected by the first acquisition device connected to the motion controller included in the target device can be determined. When the first motion information is invalid information and there is a pan-tilt head in the target device, the second motion information collected by the second acquisition device connected to the pan-tilt head is obtained, the first posture information is determined based on the second motion information, and the target device is controlled to operate according to the first posture information. Since the motion information collected by the first acquisition device is invalid, the second motion information can be collected by the second acquisition device to further determine the first posture information. This achieves that when the first acquisition device fails, the first posture information can still be determined by the information collected by other acquisition devices, and the second acquisition device is a device connected to the pan-tilt head, without the need to set up an additional acquisition device in the target device. Therefore, the problem in the related art that the device cannot be accurately controlled due to inaccurate posture information of the determined device can be solved, and the effect of accurately controlling the operation of the device can be achieved.

[0052] Optionally, the execution subject of the above steps may be a processor, but is not limited thereto.

[0053] In an exemplary embodiment, determining the first attitude information based on the second motion information includes: determining the first angular velocity and the first acceleration included in the second motion information, wherein the first angular velocity is the angular velocity of the gimbal coordinate system relative to the inertial coordinate system, and the first acceleration is the acceleration of the gimbal coordinate system relative to the inertial coordinate system; determining a first correction term based on the first acceleration; correcting the first angular velocity based on the first correction term to obtain a first corrected angular velocity; and determining the first attitude information based on the first corrected angular velocity. In this embodiment, the gimbal attitude board is fixedly connected to a second acquisition device, such as an inertial measurement unit, which can measure the first angular velocity and the first acceleration of the gimbal attitude board. The gimbal attitude board is relative to the inertial coordinate system.

[0054] The first angular velocity of the system can be expressed as This variable is read as the angular velocity of the gimbal attitude board coordinate system (i.e., the gimbal coordinate system) (b_gimbal) relative to the inertial system (i) (i.e., the inertial coordinate system), expressed in the gimbal attitude board coordinate system (b_gimbal). The first acceleration of the gimbal attitude board relative to the inertial system can be expressed as The acceleration here is the non-gravitational acceleration. This variable is read as the non-gravitational velocity of the gimbal attitude board coordinate system (b_gimbal) relative to the inertial system (i), expressed in the gimbal attitude board coordinate system (b_gimbal).

[0055] In this embodiment, the first attitude information can be determined by Kalman filtering and the second motion information. For example, a state space model can be first constructed, which includes state variables such as the gimbal attitude quaternion, the bias of the gyroscope and the accelerometer. Then, by predicting the state and uncertainty of the gimbal at the next moment, and combining the angular velocity of the gyroscope and the gravity vector measurement of the accelerometer, the measurement residual is calculated. Next, the Kalman gain is calculated based on the residual and noise characteristics, and the state estimate and uncertainty are adjusted with this gain to achieve the optimal attitude estimate update based on the latest sensor data. This process is iterated repeatedly. Even in the face of gyroscope drift or accelerometer deviation, it can continuously provide stable and accurate first attitude information by automatically adjusting the fusion ratio and comprehensively considering the system and measurement noise.

[0056] In this embodiment, the first posture information can be expressed as . Angular velocity and acceleration are key parameters for describing the motion state of a device. Through the conversion between the gimbal coordinate system and the inertial coordinate system, the data of the second acquisition device can be applied to the attitude control of the entire device. After determining the first acceleration, the correction term is calculated to compensate for the angular velocity error caused by factors such as gravity and acceleration, thereby obtaining a more accurate corrected angular velocity and further determining the attitude information of the device. This method of correcting angular velocity based on acceleration can improve the accuracy of attitude information, especially when the device is performing fast or complex movements, and can more accurately reflect the actual attitude of the device, solving the problem of inaccurate attitude control of the device in complex environments.

[0057] In an exemplary embodiment, determining a first correction term based on the first acceleration includes: converting the first acceleration to the inertial coordinate system to obtain a second acceleration; and determining a first product of the second acceleration and a first preset matrix as the first correction term. In this embodiment, the first acceleration can be converted to the inertial coordinate system according to a conversion matrix to obtain the second acceleration. The conversion matrix can be a predetermined fixed matrix. The conversion matrix can also be a continuously iterative matrix. When the conversion matrix is ​​a continuously iterative matrix, when the first acceleration is converted to the inertial coordinate system for the first time, the conversion matrix can be a predetermined initial matrix. In each subsequent conversion process, the conversion matrix can be determined based on the first posture information determined last.

[0058] In this embodiment, after determining the first posture information, the conversion matrix from the gimbal coordinate system to the inertial coordinate system can be determined according to the first posture information. The conversion matrix from the gimbal coordinate system to the inertial coordinate system can be expressed as The motion controller and the gimbal attitude board can be connected by three motors. The transformation matrix from the motion controller coordinate system to the gimbal attitude board coordinate system can be expressed as , the rotation order is zxy, and the angles are , and .but . in, , and It can be measured by the encoder. Therefore, the transformation matrix from the inertial system to the flight control board coordinate system can be expressed as ,in, Can be Transpose to get. Can be Transpose to get.

[0059] In this embodiment, the first preset matrix can be , the second acceleration can be cross-multiplied with the first preset matrix to obtain a first correction term.

[0060] In this embodiment, acceleration conversion is based on the principle of coordinate system transformation. By converting acceleration from the gimbal coordinate system to the inertial coordinate system, acceleration errors caused by device motion can be eliminated, resulting in an acceleration value closer to reality. The first preset matrix is ​​pre-defined based on the device's physical characteristics and motion model and is used to calculate the correction term. By multiplying this with the converted acceleration, the correction term required to compensate for angular velocity errors can be obtained, thereby improving the accuracy of attitude information.

[0061] In one exemplary embodiment, correcting the first angular velocity based on the first correction term to obtain the first corrected angular velocity includes: converting the first correction term from the inertial coordinate system to the gimbal coordinate system to obtain a second correction term; determining a second product of the second correction term and a first preset weight; and determining the sum of the second product and the first angular velocity as the first corrected angular velocity. In this embodiment, the obtained first correction term is a correction term in the inertial coordinate system. Therefore, the first correction term can be converted to the gimbal coordinate system to obtain the second correction term. The conversion and application of the correction term is based on the principles of coordinate system transformation and weighted fusion. By converting the correction term from the inertial coordinate system back to the gimbal coordinate system, it is ensured that the correction term and the angular velocity are integrated in the same coordinate system. The first preset weight is set based on factors such as the reliability of the correction term and the real-time nature of the angular velocity. By multiplying it with the correction term, the degree of influence of the correction term on the angular velocity can be adjusted, ultimately obtaining a corrected angular velocity, thereby improving the real-time nature and accuracy of the attitude information.

[0062] In an exemplary embodiment, after determining that the target device includes first motion information collected by a first acquisition device connected to a motion controller, the method further includes: when the first motion information is valid information, determining second posture information based on the first motion information; and controlling the target device to operate in accordance with the second posture information. In this embodiment, when the first motion information is valid information, the second posture information can be determined based on the first motion information, and then the target device can be controlled to move in accordance with the second posture information. It can be determined whether the first motion information obtained twice adjacently has been updated. If an update has occurred, the first motion information is determined to be valid information. The range of the first acquisition device can also be predetermined. When the first motion information is within the range of the first acquisition device, the first motion information is determined to be valid information.

[0063] In this embodiment, the second posture information can be a quaternion of the target device. Quaternions can be used to efficiently update and express posture changes because they only require four parameters to fully describe a 3D rotation, while matrices require nine parameters. This makes quaternions a highly computationally efficient, small storage space, and easy-to-use choice.

[0064] In this embodiment, a quaternion can be initialized first. The quaternion (Q(t)) represents the attitude of the motion controller's coordinate system relative to the inertial coordinate system. During initialization, it can be assumed that the gimbal is in an ideal attitude, that is, (Q(0) = [1,0, 0, 0]) or equivalently no rotation. The angular velocity included in the first motion information can be filtered or calibrated to obtain a more accurate angular velocity estimate. Using the angular velocity, the quaternion can be continuously updated through numerical integration to estimate attitude changes. The acceleration is used to estimate the behavior of the gravity vector in the current coordinate system, and the updated quaternion is then corrected. Since the attitude estimation deviation caused by the accumulated error of the integration can be calibrated based on the acceleration, the quaternion attitude estimation is further optimized by converting the acceleration data into an attitude angle update and combining it with the gyroscope data using an extended Kalman filter (EKF) or a complementary filter. The attitude estimate, which changes rapidly but has long-term errors, obtained by integrating the angular velocity is fused with the slowly changing but relatively accurate gravity direction provided by the accelerometer to reduce the drift of the attitude estimate.

[0065] In this embodiment, when the data from the first acquisition device is valid, its data is directly used to determine the posture information. This fully utilizes the high precision and real-time nature of the primary data source, improving device control efficiency. This design ensures that the device operates optimally under normal circumstances. In the event of data anomalies, a switch to the backup data source is made promptly, maintaining device stability and security.

[0066] In an exemplary embodiment, determining the second posture information based on the first motion information includes: determining the second angular velocity and the third acceleration included in the first motion information, wherein the second angular velocity is the angular velocity of the operation and control coordinate system constructed by the motion controller included in the target device relative to the inertial coordinate system, and the third acceleration is the acceleration of the operation and control coordinate system relative to the inertial coordinate system; determining a third correction term based on the third acceleration; correcting the second angular velocity based on the third correction term to obtain a second corrected angular velocity; and determining the second posture information based on the second corrected angular velocity. In this embodiment, the flight control board is fixedly connected to a first acquisition device, such as an inertial measurement unit, which can measure the angular velocity and acceleration of the aircraft. The second angular velocity of the operation and control coordinate system relative to the inertial coordinate system can be expressed as , the variable is read as the rotational angular velocity of the flight control board coordinate system (b_uav) (i.e. the motion control coordinate system) relative to the inertial system (i) (i.e. the inertial coordinate system), expressed in the flight control board coordinate system (b_uav). The third acceleration of the flight control board relative to the inertial system can be expressed as The acceleration here is the non-gravitational acceleration. This variable is read as the non-gravitational velocity of the flight control board coordinate system (b_uav) relative to the inertial system (i), expressed in the flight control board coordinate system (b_uav). The second corrected angular velocity can be expressed as , the second corrected angular velocity can be converted into the form of x, y and z and can be expressed as .

[0067] In this embodiment, the second attitude information can be determined by Kalman filtering and the first motion information. For example, a state space model can be first constructed, which includes state variables such as the control attitude quaternion, the bias of the gyroscope and the accelerometer. Then, by predicting the state and uncertainty of the control at the next moment, and combining the angular velocity of the gyroscope and the gravity vector measurement of the accelerometer, the measurement residual is calculated. Next, the Kalman gain is calculated based on the residual and noise characteristics, and the state estimate and uncertainty are adjusted with this gain to achieve the optimal attitude estimate update based on the latest sensor data. This process is iterated repeatedly. Even in the face of gyroscope drift or accelerometer deviation, it can automatically adjust the fusion ratio and comprehensively consider the system and measurement noise to continuously provide stable and accurate second attitude information.

[0068] In this embodiment, the second posture information can be expressed as .in, is the second posture information calculated last time, Indicates the time interval between two consecutive acquisitions of the first motion information by the first acquisition device. The flight control board's attitude fusion relies entirely on the inertial measurement unit (IMU) fixed to the flight control board. If the IMU fails, the aircraft's attitude quaternion will be abnormal, thus affecting aircraft attitude control.

[0069] In this embodiment, during normal device operation, the angular velocity and acceleration in the control coordinate system constructed by the motion controller serve as the primary basis for device attitude control. By correcting the angular velocity with acceleration, attitude information errors caused by changes in the device's motion state can be eliminated, improving the accuracy and stability of device control. This method of correcting angular velocity with acceleration can be applied to the precise control of devices such as drones and robots. It can more accurately reflect the device's actual attitude, especially during high-speed or complex motions, solving the difficult problem of attitude control in complex environments.

[0070] In an exemplary embodiment, determining the third correction term based on the third acceleration includes: converting the third acceleration to the inertial coordinate system to obtain a fourth acceleration; and determining the third correction term by multiplying the fourth acceleration by the second product of the second preset matrix. In this embodiment, the third acceleration can be converted from the flight control board coordinate system (b_uav) (i.e., the operation control coordinate system) to the inertial coordinate system (i) to obtain the fourth acceleration. The fourth acceleration can be expressed as ,second

[0071] The preset matrix can be expressed as , the fourth acceleration can be cross-multiplied with the second preset matrix to obtain the third correction term. The third correction term can be expressed as .

[0072] In this embodiment, acceleration conversion and correction term calculation are based on the principles of coordinate system transformation and motion modeling. By converting acceleration from the control coordinate system to the inertial coordinate system, acceleration errors caused by changes in the device's motion state can be eliminated, resulting in an acceleration value closer to reality. A second preset matrix is ​​pre-defined based on the device's physical characteristics and motion model and is used to calculate the correction term. By multiplying this matrix with the converted acceleration, the correction term required to compensate for angular velocity errors is obtained, thereby improving the accuracy of attitude information and ensuring stable operation of the device in complex environments.

[0073] In an exemplary embodiment, converting the third acceleration to the inertial coordinate system to obtain the fourth acceleration includes: determining a conversion matrix, where, when the third acceleration is converted to the inertial coordinate system for the first time, the conversion matrix is ​​a pre-set initial matrix; and, when the third acceleration is not converted to the inertial coordinate system for the first time, the conversion matrix is ​​a matrix determined based on the second posture information obtained last time; and determining the third product of the third acceleration and the conversion matrix as the fourth acceleration. In this embodiment, the third acceleration is converted from the flight control board coordinate system (b_uav) (i.e., the motion control coordinate system) to the inertial coordinate system (i) to obtain the fourth acceleration, which can be expressed as ,in, represents the third acceleration, Represents the transformation matrix. When the third acceleration is first transformed into the inertial coordinate system, the transformation matrix can be a predetermined initial matrix. In subsequent transformations, the transformation matrix can be continuously updated iteratively.

[0074] In this embodiment, the conversion matrix is ​​determined based on the principles of coordinate system transformation and posture information feedback. By using an initial matrix or a matrix updated based on the last posture information, the accuracy and continuity of acceleration conversion can be ensured. This method not only eliminates acceleration errors caused by changes in the device's motion state, but also dynamically adjusts the conversion matrix based on the device's current posture information, improving the real-time and accuracy of posture information and ensuring stable operation of the device in complex environments.

[0075] In an exemplary embodiment, correcting the second angular velocity based on the third correction term to obtain the second corrected angular velocity includes: converting the third correction term from the inertial coordinate system to the control coordinate system to obtain a fourth correction term; determining a fourth product of the fourth correction term and a second preset weight; and determining the sum of the fourth product and the second angular velocity as the second corrected angular velocity. In this embodiment, the angular velocity error correction term, i.e., the third correction term, can be converted from the inertial system (i) to the flight control board coordinate system (b_uav) to obtain a fourth correction term. The fourth correction term can be expressed as .in, The conversion matrix from the inertial coordinate system to the control coordinate system can be obtained by transposing the conversion matrix from the control coordinate system to the inertial coordinate system. After obtaining the fourth correction term, the second angular velocity can be corrected according to the fourth correction term. The original angular velocity is corrected using the angular velocity error correction term expressed in the flight control board coordinate system (b_uav). k is the angular velocity correction weight. The second corrected angular velocity can be expressed as .

[0076] In this embodiment, the conversion and application of the correction term is based on the principles of coordinate system transformation and weighted fusion. By converting the correction term from the inertial coordinate system back to the motion control coordinate system, the correction term and angular velocity are integrated in the same coordinate system. The second preset weight is set based on factors such as the reliability of the correction term and the real-time nature of the angular velocity. By multiplying it with the correction term, the degree of its impact on the angular velocity is adjusted, ultimately resulting in a corrected angular velocity. This improves the real-time and accuracy of attitude information, ensuring stable operation of the device in complex environments.

[0077] In an exemplary embodiment, after determining the second posture information based on the first motion information, the method further includes: determining a posture quaternion included in the second posture information; and determining a transformation matrix based on the posture quaternion, wherein the transformation matrix is ​​a matrix for implementing coordinate system transformation. In this embodiment, the transformation matrix can be expressed as .

[0078] In this embodiment, the attitude quaternion is a mathematical representation of the device's attitude. By determining the attitude quaternion in the second attitude information, the transformation matrix between the control coordinate system and the inertial coordinate system can be calculated for subsequent conversion of acceleration and angular velocity. This method not only improves the accuracy of coordinate system conversion but also dynamically adjusts the transformation matrix based on the device's current attitude information, improving the real-time and accuracy of attitude information and ensuring stable operation of the device in complex environments.

[0079] The control method of the device is described below in conjunction with specific implementation methods.

[0080] Take the target device as a drone and the motion controller as a flight control board as an example. Figure 3 is a flow chart of a control method for a device according to a specific embodiment of the present invention, such as Figure 3 As shown, the process includes:

[0081] Step S302, determine whether the flight control board IMU is valid, if it is valid, execute step S304, if it is invalid, execute step S306.

[0082] Step S304: Flight control board attitude fusion R1.

[0083] The flight control board is fixedly connected to an inertial measurement unit (corresponding to the first acquisition device mentioned above), which can measure the angular velocity (corresponding to the third angular velocity) and acceleration (corresponding to the third acceleration) of the aircraft. , the variable is read as the angular velocity of the flight control board coordinate system (b_uav) relative to the inertial system (i), expressed in the flight control board coordinate system (b_uav).

[0084] The acceleration of the flight control board relative to the inertial system The acceleration here is the non-gravitational acceleration. The variable is read as the non-gravitational velocity of the flight control board coordinate system (b_uav) relative to the inertial system (i), expressed in the flight control board coordinate system (b_uav).

[0085] Convert the angular velocity into x, y and z forms,

[0086] .

[0087] Convert the acceleration from the flight control board coordinate system (b_uav) to the inertial system (i) to obtain the fourth acceleration:

[0088] .

[0089] Will and (corresponding to the above second preset matrix) cross-product, to obtain the angular velocity error correction term (corresponding to the above third correction term),

[0090] .

[0091] The angular velocity error correction term is transferred from the inertial system (i) to the flight control board coordinate system (b_uav) to obtain the fourth correction term:

[0092] .

[0093] Use the angular velocity error correction term expressed in the flight control board coordinate system (b_uav) to correct the original angular velocity, k is the angular velocity correction weight, and the second corrected angular velocity is obtained:

[0094] .

[0095] Convert the corrected angular velocity into x, y and z forms,

[0096]

[0097] The quaternion is updated using the corrected angular velocity to obtain the aircraft attitude quaternion (corresponding to the second attitude information mentioned above).

[0098]

[0099] Update the attitude matrix according to the quaternion (corresponding to the above transformation matrix),

[0100] .

[0101] Step S306, obtaining the posture R2 from the CAN bus.

[0102] The gimbal attitude board is fixedly connected to an inertial measurement unit, which can measure the angular velocity and acceleration of the gimbal attitude board. The variable is read as the angular velocity of the gimbal attitude board coordinate system (b_gimbal) relative to the inertial system (i), expressed in the gimbal attitude board coordinate system (b_gimbal).

[0103] The first acceleration of the gimbal attitude plate relative to the inertial system The acceleration here is the non-gravitational acceleration. The variable is read as the non-gravitational velocity of the gimbal attitude board coordinate system (b_gimbal) relative to the inertial system (i), expressed in the gimbal attitude board coordinate system (b_gimbal).

[0104] The attitude fusion of the gimbal can be done in the same way as the flight control, or the Kalman filter can be used to simplify the fusion method into a formula. The first attitude information can be expressed as , the transformation matrix can be expressed as .

[0105] The flight control board and the gimbal attitude board are connected by three motors. The rotation matrix from the flight control board coordinate system to the gimbal attitude board coordinate system is , the rotation order is zxy, and the angles are , and .

[0106]

[0107] , and Can be measured by an encoder.

[0108] Therefore, the rotation matrix from the inertial system to the flight control board coordinate system is .

[0109] Step S308: output the rotated posture matrix R.

[0110] The inertial measurement unit is installed on the flight control board, and the flight control board can calculate the attitude matrix from the inertial system to the flight control board .

[0111] The inertial measurement unit is installed on the gimbal attitude board, and the gimbal attitude board can calculate the attitude matrix from the inertial system to the gimbal attitude board The gimbal attitude board and the flight control board are connected using three motors. Each motor has an encoder to measure the motor rotation angle, so the attitude matrix can be calculated. So we can get the rotation matrix from the inertial system to the flight control board coordinate system: .

[0112] because , so this formula can be used to design the posture fusion redundancy design.

[0113] Whether the angular velocity or acceleration measured by the flight control inertial measurement unit exceeds the threshold is used to determine whether the flight control inertial measurement unit is valid. If valid, the attitude matrix R1 obtained by the attitude fusion of the flight control board is used. If invalid, the gimbal attitude fusion attitude matrix R2 obtained from the CAN bus is used.

[0114] In the above embodiment, when the first motion information is valid, the second posture information can be determined based on the first operation information, and the target device can be controlled to operate based on the second posture information. This means that a redundancy strategy is implemented based on the first acquisition device connected to the motion controller and the second acquisition device connected to the gimbal. When the motion controller is a flight control board and the gimbal controller is a gimbal posture board, a redundancy design diagram can be found in the attached figure. Figure 4 ,like Figure 4 As shown in the figure, the gimbal attitude board is connected to the flight control board through three motors. The flight control board calculates R1 through the attitude fusion algorithm, and can obtain the attitude R2 determined by the gimbal attitude board through the CAN bus. R is determined from R1 and R2 through the redundancy strategy. According to R, the attitude information (first attitude information or second attitude information) can be determined, and then the movement of the target device can be controlled.

[0115] In the aforementioned embodiment, redundant design of the attitude fusion algorithm can be achieved without adding an additional inertial measurement unit to the flight control board, thus reducing costs. The hardware and software designs of the gimbal attitude board and the flight control board can be developed by different engineers using different approaches, thus ensuring that if the flight control board's IMU fails, switching to the gimbal attitude board will not trigger the same fault.

[0116] Through the description of the above embodiments, those skilled in the art will clearly understand that the methods according to the above embodiments can be implemented using software plus the necessary general-purpose hardware platform. Of course, hardware can also be used, but in many cases the former is a more preferred embodiment. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, or optical disk) and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0117] In this embodiment, a device control device is also provided, which is used to implement the above-mentioned embodiments and preferred embodiments. Details already described are omitted. As used below, the term "module" may refer to a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation using hardware, or a combination of software and hardware, is also possible and contemplated.

[0118] Figure 5 is a structural block diagram of a control device of an apparatus according to an embodiment of the present invention, such as Figure 5 As shown, the device includes:

[0119] A first determining module 52 is configured to determine that the target device includes first motion information collected by a first collecting device connected to the motion controller;

[0120] an acquisition module 54 configured to, when determining that the first motion information is invalid and the target device has a pan / tilt platform, acquire second motion information collected by a second acquisition device connected to the pan / tilt platform and included in the target device;

[0121] A second determining module 56 is configured to determine first posture information based on the second motion information;

[0122] The control module 58 is configured to control the target device to operate according to the first posture information.

[0123] In an exemplary embodiment, the second determination module 56 can determine the first posture information based on the second motion information in the following manner: determine the first angular velocity and the first acceleration included in the second motion information, wherein the first angular velocity is the angular velocity of the gimbal coordinate system relative to the inertial coordinate system, and the first acceleration is the acceleration of the gimbal coordinate system relative to the inertial coordinate system; determine a first correction term based on the first acceleration; correct the first angular velocity based on the first correction term to obtain a first corrected angular velocity; and determine the first posture information based on the first corrected angular velocity.

[0124] In an exemplary embodiment, the second determination module 56 can determine the first correction term based on the first acceleration in the following manner: convert the first acceleration to the inertial coordinate system to obtain a second acceleration; and determine the first product of the second acceleration and a first preset matrix as the first correction term.

[0125] In an exemplary embodiment, the second determination module 56 can correct the first angular velocity based on the first correction term to obtain a first corrected angular velocity in the following manner: convert the first correction term from the inertial coordinate system to the gimbal coordinate system to obtain a second correction term; determine a second product of the second correction term and a first preset weight; and determine the sum of the second product and the first angular velocity as the first corrected angular velocity.

[0126] In an exemplary embodiment, the device can also be used to determine second posture information based on the first motion information after determining that the target device includes first motion information collected by a first collection device connected to the motion controller, if the first motion information is valid information; and control the target device to operate according to the second posture information.

[0127] In an exemplary embodiment, the device can determine the second posture information based on the first motion information in the following manner: determine the second angular velocity and third acceleration included in the first motion information, wherein the second angular velocity is the angular velocity of the control coordinate system constructed by the motion controller included in the target device relative to the inertial coordinate system, and the third acceleration is the acceleration of the control coordinate system relative to the inertial coordinate system; determine a third correction term based on the third acceleration; correct the second angular velocity based on the third correction term to obtain a second corrected angular velocity; and determine the second posture information based on the second corrected angular velocity.

[0128] In an exemplary embodiment, the device can determine the third correction term based on the third acceleration in the following manner: convert the third acceleration into the inertial coordinate system to obtain a fourth acceleration; and determine the third correction term as the second product of the fourth acceleration and the second preset matrix.

[0129] In an exemplary embodiment, the device can convert the third acceleration into the inertial coordinate system to obtain the fourth acceleration in the following manner: determine a conversion matrix, where when the third acceleration is converted to the inertial coordinate system for the first time, the conversion matrix is ​​a pre-set initial matrix; when it is not the first time that the third acceleration is converted to the inertial coordinate system, the conversion matrix is ​​a matrix determined based on the second posture information obtained last time; and determine the third product of the third acceleration and the conversion matrix as the fourth acceleration.

[0130] In an exemplary embodiment, the device can correct the second angular velocity based on the third correction term to obtain a second corrected angular velocity in the following manner: convert the third correction term from the inertial coordinate system to the operation and control coordinate system to obtain a fourth correction term; determine a fourth product of the fourth correction term and a second preset weight; and determine the sum of the fourth product and the second angular velocity as the second corrected angular velocity.

[0131] In an exemplary embodiment, the device can also be used to determine the posture quaternion included in the second posture information after determining the second posture information based on the first motion information; determine the transformation matrix based on the posture quaternion, wherein the transformation matrix is ​​a matrix used to realize coordinate system transformation.

[0132] It should be noted that the above modules can be implemented through software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0133] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any one of the above method embodiments when running.

[0134] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0135] An embodiment of the present invention further provides an electronic device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0136] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0137] An embodiment of the present invention further provides a computer program product, including a computer program, which implements the steps of the method in each embodiment of the present application when the computer program is executed by a processor.

[0138] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail here.

[0139] Obviously, those skilled in the art will appreciate that the various modules or steps of the present invention described above can be implemented using a general-purpose computing device, can be centralized on a single computing device, or can be distributed across a network of multiple computing devices. They can be implemented using program code executable by the computing device, and thus, can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described herein can be performed in a different order than that shown, or can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, the present invention is not limited to any particular combination of hardware and software.

[0140] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for controlling a device, characterized in that: include: Determining that the target device includes first motion information collected by a first collection device connected to the motion controller; When it is determined that the first motion information is invalid and the target device has a pan / tilt platform, obtaining second motion information collected by a second collection device connected to the pan / tilt platform and included in the target device; determining first posture information based on the second motion information; controlling the target device to operate according to the first posture information; Determining the first posture information based on the second motion information includes: determining the first angular velocity and the first acceleration included in the second motion information, wherein the first angular velocity is the angular velocity of the gimbal coordinate system relative to the inertial coordinate system, and the first acceleration is the acceleration of the gimbal coordinate system relative to the inertial coordinate system; determining a first correction term based on the first acceleration; correcting the first angular velocity based on the first correction term to obtain a first corrected angular velocity; and determining the first posture information based on the first corrected angular velocity.

2. The method according to claim 1, characterized in that Determining a first correction term based on the first acceleration includes: Converting the first acceleration to the inertial coordinate system to obtain a second acceleration; A first product of the second acceleration and a first preset matrix is ​​determined as the first correction term.

3. The method according to claim 1, characterized in that Correcting the first angular velocity based on the first correction term to obtain a first corrected angular velocity includes: Converting the first correction term from the inertial coordinate system to the gimbal coordinate system to obtain a second correction term; determining a second product of the second correction term and the first preset weight; The sum of the second product and the first angular velocity is determined as the first corrected angular velocity.

4. The method according to claim 1, wherein After determining that the target device includes first motion information collected by a first collection device connected to the motion controller, the method further includes: If the first motion information is valid, determining second posture information based on the first motion information; The target device is controlled to operate according to the second posture information.

5. The method according to claim 4, characterized in that Determining second posture information based on the first motion information includes: Determining a second angular velocity and a third acceleration included in the first motion information, wherein the second angular velocity is an angular velocity of a motion control coordinate system constructed by a motion controller included in the target device relative to an inertial coordinate system, and the third acceleration is an acceleration of the motion control coordinate system relative to the inertial coordinate system; determining a third correction term based on the third acceleration; correcting the second angular velocity based on the third correction term to obtain a second corrected angular velocity; The second posture information is determined based on the second corrected angular velocity.

6. The method according to claim 5, characterized in that Determining a third correction term based on the third acceleration includes: Converting the third acceleration to the inertial coordinate system to obtain a fourth acceleration; A second product of the fourth acceleration and the second preset matrix is ​​determined as the third correction term.

7. The method according to claim 6, characterized in that Converting the third acceleration to the inertial coordinate system to obtain a fourth acceleration includes: determining a conversion matrix, where, when the third acceleration is converted to the inertial coordinate system for the first time, the conversion matrix is ​​a pre-set initial matrix; and, when the third acceleration is not converted to the inertial coordinate system for the first time, the conversion matrix is ​​a matrix determined based on the second posture information obtained last time; A third product of the third acceleration and the conversion matrix is ​​determined as the fourth acceleration.

8. The method according to claim 5, characterized in that Correcting the second angular velocity based on the third correction term to obtain the second corrected angular velocity includes: Converting the third correction term from the inertial coordinate system to the operation and control coordinate system to obtain a fourth correction term; determining a fourth product of the fourth correction term and the second preset weight; The sum of the fourth product and the second angular velocity is determined as the second corrected angular velocity.

9. The method according to claim 4, characterized in that After determining second posture information based on the first motion information, the method further includes: determining a posture quaternion included in the second posture information; A transformation matrix is ​​determined based on the posture quaternion, wherein the transformation matrix is ​​a matrix used to implement coordinate system transformation.

10. A control device for an equipment, characterized in that: include: A first determining module is used to determine that the target device includes first motion information collected by a first collecting device connected to the motion controller; an acquisition module, configured to, when determining that the first motion information is invalid and the target device has a pan / tilt platform, acquire second motion information collected by a second acquisition device connected to the pan / tilt platform and included in the target device; a second determining module, configured to determine first posture information based on the second motion information; a control module, configured to control the target device to operate according to the first posture information; The second determination module determines the first posture information based on the second motion information in the following manner: determining a first angular velocity and a first acceleration included in the second motion information, wherein the first angular velocity is the angular velocity of the gimbal coordinate system relative to the inertial coordinate system, and the first acceleration is the acceleration of the gimbal coordinate system relative to the inertial coordinate system; determining a first correction term based on the first acceleration; correcting the first angular velocity based on the first correction term to obtain a first corrected angular velocity; and determining the first posture information based on the first corrected angular velocity.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the method according to any one of claims 1 to 9 when executed.

12. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to run the computer program to perform the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 9 are implemented.

Images