Automatic following method, device and system for intelligent mobile terminal

Abstract

The invention discloses an automatic following method, device and system for an intelligent mobile terminal, and relates to the technical field of agent following, the method comprises the following steps: obtaining first inertial data of a user side, receiving second inertial data sent by an agent side, mapping the first inertial data and the second inertial data to a default inertial coordinate system, obtaining corresponding third inertial data and fourth inertial data, obtaining first motion trail information of the user equipment under the world coordinate system according to the third inertial data, and obtaining second motion trail information of the intelligent agent under the world coordinate system according to the fourth inertial data; and sending the first motion track information and the second motion track information to an intelligent agent end, so that the intelligent agent performs motion following according to the first motion track information and the second motion track information, and automatic following is realized by combining first inertial data of the user end and second inertial data of the intelligent agent end. The precision and the cost can be well balanced, and the application scene is wider.

Description

Technical Field

[0001] The present application relates to the field of following technology, and more specifically, to an automatic following method, device and system for a smart mobile terminal. Background Art

[0002] At present, the automatic following solution is generally based on ultra-wideband (UWB) and binocular technology, or based on radar detection. The radar solution has low cost but poor following accuracy, while the UWB solution has high following accuracy but high cost. Therefore, it is very important to reduce the cost while ensuring the following accuracy. Summary of the invention

[0003] The present application proposes a method, device and system for automatic following of a smart mobile terminal to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides an automatic following method for an intelligent mobile terminal, which is applied to a user end of an automatic following system for an intelligent mobile terminal, wherein the system also includes an intelligent body end, and the method includes: obtaining first inertial data of the user end, and receiving second inertial data sent by the intelligent body end, wherein the first inertial data is located in a current inertial coordinate system of a user device, and the second inertial data is located in a current inertial coordinate system of the intelligent body; mapping both the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data, wherein the default inertial coordinate system is an inertial coordinate system obtained by aligning an initial inertial coordinate system of the user device with an initial inertial coordinate system of the intelligent body; obtaining first motion trajectory information of the user device in a world coordinate system according to the third inertial data, and obtaining second motion trajectory information of the intelligent body in a world coordinate system according to the fourth inertial data; sending the first motion trajectory information and the second motion trajectory information to the intelligent body end, so that the intelligent body performs motion following according to the first motion trajectory information and the second motion trajectory information.

[0005] In the second aspect, an embodiment of the present application also provides an automatic following method for an intelligent mobile terminal, which is applied to an intelligent body end of an automatic following system for an intelligent mobile terminal, wherein the system also includes a user end, and the method includes: receiving first motion trajectory information and second motion trajectory information sent by the user end based on the above method; performing motion following according to the first motion trajectory information and the second motion trajectory information.

[0006] In a third aspect, an embodiment of the present application further provides an automatic following device for an intelligent mobile terminal, which is applied to a user end of an automatic following system for an intelligent mobile terminal, wherein the system further comprises an intelligent body end, and the device comprises: a data acquisition module, which is used to acquire first inertial data of the user end, and receive second inertial data sent by the intelligent body end, wherein the first inertial data is located in a current inertial coordinate system of the user device, and the second inertial data is located in a current inertial coordinate system of the intelligent body; a data restoration module, which is used to map both the first inertial data and the second inertial data to a default inertial coordinate system, and obtain corresponding third inertial data and fourth inertial data, wherein the default inertial coordinate system is an inertial coordinate system obtained by aligning an initial inertial coordinate system of the user device and an initial inertial coordinate system of the intelligent body; a data calculation module, which is used to obtain first motion trajectory information of the user device in a world coordinate system according to the third inertial data, and obtain second motion trajectory information of the intelligent body in a world coordinate system according to the fourth inertial data; and a sending module, which is used to send the first motion trajectory information and the second motion trajectory information to the intelligent body end, so that the intelligent body performs motion following according to the first motion trajectory information and the second motion trajectory information.

[0007] In a fourth aspect, an embodiment of the present application further provides an automatic following device for an intelligent mobile terminal, which is applied to an intelligent body end of an automatic following system for an intelligent mobile terminal, wherein the system also includes a user end, and the device includes: a receiving module for receiving first motion trajectory information and second motion trajectory information sent by the user end based on the above method; and a following module for performing motion following according to the first motion trajectory information and the second motion trajectory information.

[0008] In a fifth aspect, an embodiment of the present application further provides an automatic following system for an intelligent mobile terminal, the system comprising a user terminal and an intelligent agent terminal; the user terminal is used to obtain first inertial data of the user terminal and receive second inertial data sent by the intelligent agent terminal, the first inertial data being located in the current inertial coordinate system of the user device, and the second inertial data being located in the current inertial coordinate system of the intelligent agent; the user terminal is also used to map both the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data; the user terminal is also used to obtain first motion trajectory information of the user device in a world coordinate system according to the third inertial data, and obtain second motion trajectory information of the intelligent agent in a world coordinate system according to the fourth inertial data; the user terminal is also used to send the first motion trajectory information and the second motion trajectory information to the intelligent agent terminal, so that the intelligent agent performs motion following according to the first motion trajectory information and the second motion trajectory information; the intelligent agent terminal is used to receive the first motion trajectory information and the second motion trajectory information sent by the user terminal based on the above steps; the intelligent agent terminal is also used to perform motion following according to the first motion trajectory information and the second motion trajectory information.

[0009] Therefore, the present application provides a method, device and system for automatic following of an intelligent mobile terminal, the method is applied to the user end of the automatic following system of an intelligent mobile terminal, and the system also includes an intelligent body end. The present application obtains the first inertial data and the second inertial data at the user end, and performs track calculation based on the first inertial data and the second inertial data to obtain the first motion trajectory information of the user device and the second motion trajectory information of the intelligent body, so that the intelligent body can achieve automatic following based on the first motion trajectory information and the second motion trajectory information. Furthermore, it is precisely because the method requires the coordinated calculation of the IMU device data of the user end and the intelligent body end, and the inertial coordinate systems based on which the user end IMU data and the intelligent body IMU data are obtained are different, that it is necessary to restore the first inertial data and the second inertial data to a unified default inertial coordinate system, so as to complete the subsequent calculation steps in the unified coordinate system. Therefore, the present application realizes automatic following by combining the first inertial data of the user end and the second inertial data of the intelligent body end. It only needs to configure inertial measurement devices on the user device and the intelligent body, which can achieve a better balance between accuracy and cost. The applicable scenarios are wider, especially suitable for some open following scenarios that do not require particularly precise results. In addition, the present method uses the user end to execute all the calculation steps, and can also use the device computing power to open some functions that require high computing power, making the application more flexible.

[0010] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 A flow chart of an automatic following method for a smart mobile terminal proposed in an embodiment of the present application is shown.

[0013] Figure 2 A flow chart of another automatic following method for a smart mobile terminal proposed in an embodiment of the present application is shown.

[0014] Figure 3 A structural block diagram of an automatic following device for a smart mobile terminal proposed in an embodiment of the present application is shown.

[0015] Figure 4 A structural block diagram of another automatic following device for a smart mobile terminal proposed in one embodiment of the present application is shown.

[0016] Figure 5 A structural block diagram of an automatic following system for a smart mobile terminal proposed in one embodiment of the present application is shown. DETAILED DESCRIPTION

[0017] In order to make those skilled in the art better understand the present application scheme, the technical scheme in the present application embodiment will be clearly and completely described below in conjunction with the drawings in the present application embodiment. Obviously, the described embodiment is only a part of the present application embodiment, rather than all the embodiments. The components of the present application embodiment usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the present application for protection, but merely represents the selected embodiment of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0018] It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. At the same time, in the description of this application, the terms "first", "second", etc. are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0019] See also Figure 1 , Figure 1 A flowchart of an automatic following method for a smart mobile terminal proposed in an embodiment of the present application is shown. Specifically, the method is applied to Figure 5 The illustrated intelligent mobile terminal automatically follows the user terminal 510 in the system 500, and the system 500 also includes an intelligent agent terminal 520. Specifically, the method includes the following steps:

[0020] S110: Acquire first inertial data of the user end, and receive second inertial data sent by the agent end, wherein the first inertial data is located in a current inertial coordinate system of the user device, and the second inertial data is located in a current inertial coordinate system of the agent.

[0021] As an implementation method, the user end may be provided with a first inertial measurement unit (IMU), and the intelligent agent end may be provided with a second inertial measurement unit. Furthermore, the inertial measurement unit IMU generally includes an accelerometer and a gyroscope. The accelerometer can measure the basic linear motion (acceleration) of an object in three directions, and the gyroscope can measure the basic angular motion (angular velocity) of an object in three directions. Therefore, the user end can obtain the first inertial data of the user end based on the first inertial measurement unit, and the intelligent agent end can also obtain the second inertial data of the intelligent agent end based on the second inertial measurement unit.

[0022] As an implementation method, the user device may be an electronic device such as a mobile phone, a tablet, etc., and the user device and the agent establish communication through a wireless network or Bluetooth, so that the user end may obtain the second inertial data sent by the agent end based on the communication method. The agent may be, for example, a robot, etc.

[0023] Furthermore, the first inertial data is located in the current inertial coordinate system of the user device, and the second inertial data is located in the current inertial coordinate system of the intelligent body. The inertial coordinate system may refer to the coordinate system of the IMU device. Usually, the coordinate origin of the IMU coordinate system is located at the common center point of the gyroscope and the accelerometer. The x-axis, y-axis, and z-axis of the IMU coordinate system are parallel to the corresponding axes of the gyroscope and the accelerometer, respectively. Without considering the installation error angle, the carrier coordinate system (i.e., the coordinate system of the carrier on which the IMU is installed) is also equivalent to the IMU coordinate system. Therefore, the current inertial coordinate system of the user device refers to the coordinate system of the user IMU device corresponding to the moment of acquiring the first inertial data, and the current inertial coordinate system of the intelligent body refers to the coordinate system of the intelligent body IMU device corresponding to the moment of acquiring the second inertial data.

[0024] Exemplarily, the first inertial data includes first rotation angle data and first acceleration data, and the second inertial data includes second rotation angle data and second acceleration data, wherein the first rotation angle data is used to indicate the angular motion of the user device in the corresponding current inertial coordinate system, the first acceleration data is used to indicate the linear motion of the user device in the corresponding current inertial coordinate system, the second rotation angle data is used to indicate the angular motion of the intelligent body in the corresponding current inertial coordinate system, and the second acceleration data is used to indicate the linear motion of the intelligent body in the corresponding current inertial coordinate system.

[0025] As an implementation method, before obtaining the first inertial data and the second inertial data, the user device IMU and the intelligent body IMU can be calibrated first. Specifically, a static calibration or a dynamic calibration method can be adopted. Static calibration is used to determine parameters such as the zero bias and proportional factor of the IMU device, and dynamic calibration is used to determine parameters such as the error model and nonlinear factors of the IMU device, so as to be applied in the subsequent trajectory calculation process.

[0026] S120: Mapping the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data, wherein the default inertial coordinate system is an inertial coordinate system obtained by aligning an initial inertial coordinate system of the user device and an initial inertial coordinate system of the intelligent agent.

[0027] Specifically, the default inertial coordinate system is an inertial coordinate system obtained by aligning the initial inertial coordinate system of the user device with the initial inertial coordinate system of the intelligent body. As an implementation method, the user device can evoke the automatic following function of the intelligent mobile terminal, and then start the user device IMU and the intelligent body IMU. Therefore, the initial inertial coordinate system of the user device can refer to the initial inertial coordinate system when the user device evokes the automatic following function of the intelligent mobile terminal, that is, the moment when the user device IMU is started, and the initial inertial coordinate system of the intelligent body can refer to the initial inertial coordinate system when the intelligent body is evoked to automatically follow the function of the intelligent mobile terminal, that is, the moment when the intelligent body IMU is started. Further, the method of aligning the initial inertial coordinate system of the user device and the initial inertial coordinate system of the intelligent body can be: aligning the coordinate origin of the initial inertial coordinate system of the intelligent body to the coordinate origin of the initial inertial coordinate system of the user device, aligning the x-axis, y-axis, and z-axis of the initial inertial coordinate system of the intelligent body to the x-axis, y-axis, and z-axis of the initial inertial coordinate system of the user device, and then using the aligned initial inertial coordinate system as the default inertial coordinate system. To put it simply, the alignment operation of the inertial coordinate system is to ensure that the inertial data obtained by the aligned intelligent body-side IMU and the inertial data obtained by the aligned user-side IMU are in the same inertial coordinate system, so as to facilitate subsequent data processing in a unified coordinate system.

[0028] However, based on the above description, it can be known that as the user device and the intelligent body move, the center point and axis of the user device IMU and the intelligent body IMU will change, that is, the current inertial coordinate system of the user device and the current inertial coordinate system of the intelligent body IMU will change with time, and the acquired inertial data are all inertial data in the current inertial coordinate system. Therefore, it is necessary to map the currently collected first inertial data in the current inertial coordinate system of the user device and the second inertial data in the current inertial coordinate system of the intelligent body to the same coordinate system, so as to facilitate the correspondence of the speed between the user device and the intelligent body. In this embodiment, the first inertial data and the second inertial data are mapped to the default inertial coordinate system to obtain the corresponding third inertial data and fourth inertial data.

[0029] According to the relevant instructions of step S110, the first inertial data includes first rotation angle data and first acceleration data, and the second inertial data includes second rotation angle data and second acceleration data. Further, as an implementation method, the first inertial data is mapped to the default inertial coordinate system to obtain the corresponding third inertial data. The method may be: based on the first rotation angle data, a first posture of the user device is obtained, and according to the pre-acquired initial posture and the first posture of the user device, the first acceleration data is restored to the default inertial coordinate system, and the restored first acceleration data is used as the third inertial data.

[0030] Specifically, a 3*3 matrix can express the rotation of an object. For example, the rotation around the z-axis can be expressed according to the following formula (1):

[0031]

[0032] Here, θ refers to the z-axis rotation data in the first rotation angle data. Therefore, the posture transformation results of the x-axis, y-axis and z-axis can be obtained based on the three-axis data in the first rotation angle data according to the same principle, that is, the first posture can be obtained.

[0033] Furthermore, the initial posture of the user device refers to the posture information obtained based on the IMU inertial data in the initial inertial coordinate system (that is, the default inertial coordinate system) after the user device is aligned. As an implementation method, when the initial inertial coordinate system of the user device is aligned with the initial inertial coordinate system of the intelligent body, that is, the IMU inertial data of the user device is obtained in the default inertial coordinate system, and then based on the same implementation method as above, the initial posture of the user device can be converted.

[0034] Thus, according to the first posture and the pre-acquired initial posture of the user device, by obtaining the transformation relationship between the first posture and the initial posture of the user device, the transformation relationship between the current inertial coordinate system of the user device and the default inertial coordinate system can be obtained. Simply put, this transformation relationship refers to the rotation relationship between the two coordinate systems, or it can be understood as how the current inertial coordinate system of the user device should be rotated and restored to the default inertial coordinate system based on the angle difference between the first posture and the initial posture of the user device. Therefore, after obtaining the transformation relationship between the current inertial coordinate system of the user device and the default inertial coordinate system, the same transformation relationship can be used to restore the first acceleration data to the default inertial coordinate system, and the restored first acceleration data is used as the third inertial data. It can be understood that the restoration of the first acceleration data here is mainly to restore the acceleration direction to the default inertial coordinate system.

[0035] Similarly, as an implementation method, the second inertial data can be mapped to the default inertial coordinate system to obtain the corresponding fourth inertial data by: obtaining the second posture of the intelligent body based on the second rotation angle data; restoring the second acceleration data to the default inertial coordinate system according to the pre-acquired initial posture of the intelligent body and the second posture, and using the restored second acceleration data as the fourth inertial data. The specific implementation method can refer to the above implementation method of the inertial data of the user device, which will not be repeated here.

[0036] S130: Obtain first motion trajectory information of the user equipment in a world coordinate system according to the third inertial data, and obtain second motion trajectory information of the agent in a world coordinate system according to the fourth inertial data.

[0037] In this embodiment, after obtaining the third inertial data of the user end and the fourth inertial data of the agent end in the default inertial coordinate system, the first motion trajectory information of the user device in the world coordinate system can be obtained according to the third inertial data, and the second motion trajectory information of the agent in the world coordinate system can be obtained according to the fourth inertial data. Specifically, the world coordinate system refers to the absolute coordinate system of the objective world, which is a fixed reference frame that does not depend on the position of the observer or the camera. For example, the origin of the world coordinate system can be selected at a position that is easy to calculate and describe, and the positive direction of the world coordinate axis is determined according to the right-hand rule (in the right-hand coordinate system) or the left-hand rule (in the left-hand coordinate system, but the right-hand coordinate system is more commonly used in computer graphics). Further, when the world coordinate system is uniquely determined and known, the third inertial data and the fourth inertial data can be restored to the world coordinate system first. According to the relevant instructions in step S120, the third inertial data and the fourth inertial data are acceleration data reflecting the acceleration of the user device and the acceleration of the intelligent body in the default inertial coordinate system. Therefore, the track of the user device and the intelligent body can be calculated based on the third inertial data and the fourth inertial data in the world coordinate system and the IMU motion equation, so as to obtain the position information and acceleration information of the user device, and the position information and acceleration information of the intelligent body, wherein the position information is used to reflect the position (displacement from the initial position) of the user device and the intelligent body at the time of obtaining the corresponding first inertial data and the second inertial data, and the acceleration is used to reflect the acceleration size and direction of the user device and the intelligent body at the time of obtaining the corresponding first inertial data and the second inertial data.

[0038] As an implementation manner, the third inertial data and the fourth inertial data can be restored to the world coordinate system by: restoring the third inertial data to the world coordinate system based on a pre-acquired rotation matrix, and restoring the fourth inertial data to the world coordinate system based on a pre-acquired rotation matrix. Specifically, the above process can be obtained based on the following formula (2):

[0039]

[0040] Among them, R is used to represent the rotation transformation matrix from the IMU inertial coordinate system to the world coordinate system. Refers to the measured acceleration value in the IMU inertial coordinate system, such as the third inertial data or the fourth inertial data, and a refers to the converted acceleration value in the world coordinate system, such as the restored third inertial data or the restored fourth inertial data. Furthermore, the value of R can be obtained based on the default inertial coordinate system obtained after aligning the initial inertial coordinate system of the user device and the initial inertial coordinate system of the intelligent body combined with the transformation relationship determined by the world coordinate system. Furthermore, since the third inertial data and the fourth inertial data are both in the default inertial coordinate system, the third inertial data and the fourth inertial data can be restored to the world coordinate system through the same rotation matrix R.

[0041] As an implementation method, the method for obtaining the first motion trajectory information and the second motion trajectory information may be: integrating the third inertial data restored to the world coordinate system to obtain the first motion trajectory information; integrating the fourth inertial data restored to the world coordinate system to obtain the second motion trajectory information. In some embodiments, the restored third inertial data and fourth inertial data are acceleration data of the user device and the intelligent agent in the world coordinate system, respectively, and the speed information can be obtained after the acceleration is integrated once, and the position information can be obtained after the speed information is integrated again. Then, the obtained position information and acceleration information of the user device are used as the first motion trajectory information of the user device in the world coordinate system, and the obtained position information and acceleration information of the intelligent agent are used as the second motion trajectory information of the intelligent agent in the world coordinate system.

[0042] S140: Send the first motion trajectory information and the second motion trajectory information to the intelligent agent end, so that the intelligent agent performs motion following according to the first motion trajectory information and the second motion trajectory information.

[0043] In this embodiment, the user terminal sends the calculated first motion trajectory information and second motion trajectory information to the agent terminal, so that the agent terminal can perform motion tracking according to the first motion trajectory information and the second motion trajectory information.

[0044] Therefore, an automatic following method for an intelligent mobile terminal provided in an embodiment of the present application is applied to the user end of an automatic following system for an intelligent mobile terminal. The system also includes an intelligent body end. The method obtains first inertial data and second inertial data at the user end, and performs track calculation based on the first inertial data and the second inertial data to obtain first motion trajectory information of the user device and second motion trajectory information of the intelligent body, so that the intelligent body can automatically follow based on the first motion trajectory information and the second motion trajectory information. Furthermore, precisely because the method requires the coordinated calculation of IMU device data at the user end and the intelligent body end, and the inertial coordinate systems based on which the acquired user-end IMU data and the intelligent body IMU data are different, it is necessary to restore the first inertial data and the second inertial data to a unified default inertial coordinate system to complete subsequent calculation steps in the unified coordinate system. Therefore, the present application realizes automatic following by combining the first inertial data of the user end and the second inertial data of the intelligent body end. It only needs to configure inertial measurement devices on the user device and the intelligent body, which can achieve a good balance between accuracy and cost. It not only solves the problem of too low accuracy in low-cost radar-based following solutions, but also solves the problem of too high cost in high-precision UWB-based following solutions. The applicable scenarios are wider, especially suitable for some open following scenarios that do not require particularly precise results. In addition, the method uses the user end to execute all calculation steps, and can also use the device computing power to open some functions that require high computing power, and the application is more flexible.

[0045] See also Figure 2 , Figure 2 A flowchart of another automatic following method for a smart mobile terminal proposed in an embodiment of the present application is shown. Specifically, the method is applied to Figure 5 The intelligent mobile terminal automatically follows the intelligent agent terminal 520 in the system 500, and the system 500 also includes a user terminal 510. Specifically, the method includes the following steps:

[0046] S210: Receive first motion trajectory information and second motion trajectory information sent by the user terminal.

[0047] In some embodiments, the user device and the agent establish communication via a wireless network or Bluetooth, so the agent can receive the first motion trajectory information and the second motion trajectory information from the user based on the above communication method.

[0048] In some embodiments, the first motion trajectory information includes location information and acceleration information of the user device, and the second motion trajectory information includes location information and acceleration information of the agent.

[0049] S220: According to the position information of the first motion trajectory information and the position information of the second motion trajectory information, the agent moves toward the location of the user equipment.

[0050] Specifically, after obtaining the first motion trajectory information and the second motion trajectory information, the intelligent body will first obtain the location information of the user device and the location information of the intelligent body itself based on the first motion trajectory information and the second motion trajectory information. Then, after determining the location information of the two, the intelligent body will move toward the location of the user device to achieve motion following.

[0051] According to the description of step S130, when the user terminal obtains the corresponding trajectory information according to the restored third inertial data and fourth inertial data, the speed information of the user device and the speed information of the agent can also be obtained during the integration process. In some embodiments, the speed of the agent during the movement process can be further determined based on the movement speed information sent by the user terminal. For example, the speed of the agent can be adjusted to be consistent with the speed of the user device.

[0052] In some embodiments, the agent terminal further includes a radar, and illustratively, the radar may be an ultrasonic radar, which is used to detect obstacle information in the agent's surrounding environment, such as obstacle type and distance of the obstacle from the agent. In some embodiments, while the agent is moving toward the location of the user device based on the position information of the first motion trajectory information and the position information of the second motion trajectory information, the agent terminal may further obtain obstacle information in the agent's surrounding environment through the radar, so that the agent can perform obstacle avoidance operations according to the obstacle information during movement, bypass surrounding obstacles, and continue to follow.

[0053] S230: according to the acceleration information of the first motion trajectory information and the acceleration information of the second motion trajectory information, making the acceleration of the agent when moving consistent with the acceleration of the user equipment.

[0054] In this embodiment, the intelligent body will further obtain the acceleration information of the user device and the acceleration information of the intelligent body itself based on the first motion trajectory information and the second motion trajectory information, so as to keep the acceleration of the intelligent body consistent with the acceleration of the user device during the automatic following process.

[0055] In some embodiments, since the acquired acceleration information of the user device is actually the acceleration of the user device at the moment when the user terminal acquires the first inertial data, and the acceleration of the user device may change at any time during the following process of the intelligent body, setting a suitable IMU data acquisition interval on the user terminal and the intelligent body can effectively reduce the following error caused by the acceleration change while ensuring the computing power of the user terminal. For example, the IMU data acquisition interval can be adjusted according to the distance between the user device and the intelligent body, the longer the distance, the longer the acquisition interval, and the shorter the distance, the shorter the acquisition interval.

[0056] Therefore, this embodiment provides an automatic following method for an intelligent mobile terminal. The method is applied to an intelligent body end of an automatic following system for an intelligent mobile terminal. The system also includes a user end. The method receives first motion trajectory information and second motion trajectory information sent by a user end at the intelligent body end, and reasonably uses the first motion trajectory information and the second motion trajectory information for motion following, thereby achieving a better balance between the accuracy and cost of the automatic following solution of the intelligent mobile terminal, and having a wider range of applicable scenarios.

[0057] See also Figure 3 , Figure 3 The structure block diagram of an automatic following device for a smart mobile terminal proposed in one embodiment of the present application is shown. The device 300 is applied to Figure 5 The illustrated intelligent mobile terminal automatically follows the user terminal 510 in the system 500, and the system 500 also includes an intelligent agent terminal 520. Specifically, the device 300 includes:

[0058] The data acquisition module 301 is used to acquire first inertial data of the user terminal and receive second inertial data sent by the agent terminal, wherein the first inertial data is located in the current inertial coordinate system of the user device and the second inertial data is located in the current inertial coordinate system of the agent;

[0059] A data restoration module 302 is used to map the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data, wherein the default inertial coordinate system is an inertial coordinate system obtained by aligning an initial inertial coordinate system of the user device with an initial inertial coordinate system of the agent;

[0060] A data calculation module 303 is used to obtain first motion track information of the user equipment in the world coordinate system according to the third inertial data, and obtain second motion track information of the agent in the world coordinate system according to the fourth inertial data;

[0061] The sending module 304 is used to send the first motion trajectory information and the second motion trajectory information to the intelligent agent end, so that the intelligent agent performs motion following according to the first motion trajectory information and the second motion trajectory information.

[0062] Optionally, the first inertial data includes first rotation angle data and first acceleration data, the second inertial data includes second rotation angle data and second acceleration data, and the data restoration module 302 is further used to obtain a first posture of the user device based on the first rotation angle data; restore the first acceleration data to the default inertial coordinate system according to a pre-acquired initial posture of the user device and the first posture, and use the restored first acceleration data as the third inertial data; obtain the second posture of the intelligent body based on the second rotation angle data; restore the second acceleration data to the default inertial coordinate system according to the pre-acquired initial posture of the intelligent body and the second posture, and use the restored second acceleration data as the fourth inertial data.

[0063] Optionally, the data calculation module 303 is also used to restore the third inertial data to the world coordinate system based on the pre-acquired rotation matrix; integrate the restored third inertial data to obtain the position information of the user device, and use the position information and acceleration information of the user device as the first motion trajectory information of the user device in the world coordinate system; restore the fourth inertial data to the world coordinate system based on the pre-acquired rotation matrix; integrate the restored fourth inertial data to obtain the speed information and position information of the intelligent body as the second motion trajectory information of the intelligent body in the world coordinate system.

[0064] Optionally, the data acquisition module 301 is also used to obtain the user device initial posture of the user device when the inertial coordinate system where the user device is located is aligned with the inertial coordinate system where the intelligent agent is located; receive the intelligent agent initial posture of the intelligent agent, and the intelligent agent initial posture is obtained when the inertial coordinate system where the user device is located is aligned with the inertial coordinate system where the intelligent agent is located.

[0065] See also Figure 4 , Figure 4 A structural block diagram of another intelligent mobile terminal automatic following device proposed in an embodiment of the present application is shown. The device 400 is applied to Figure 5 The intelligent mobile terminal automatically follows the intelligent agent terminal 520 in the system 500, and the system 500 also includes a user terminal 510. Specifically, the device 400 includes:

[0066] The receiving module 401 is used to receive the first motion trajectory information and the second motion trajectory information sent by the user terminal based on the method of the above embodiment.

[0067] The following module 402 is configured to perform motion following according to the first motion trajectory information and the second motion trajectory information.

[0068] Optionally, the following module 402 is further configured to enable the agent to move toward the location of the user equipment according to the location information of the first motion trajectory information and the location information of the second motion trajectory information.

[0069] Optionally, the following module 402 is further configured to make the acceleration of the agent when it moves consistent with the acceleration of the user device according to the acceleration information of the first motion trajectory information and the acceleration information of the second motion trajectory information.

[0070] Optionally, the intelligent agent terminal 520 further includes a radar, and the device 400 further includes an obstacle avoidance module, which is used to obtain obstacle information of the environment around the intelligent agent through the radar; and perform obstacle avoidance operations according to the obstacle information.

[0071] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0072] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0073] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0074] See also Figure 5 , Figure 5 The structure block diagram of an automatic following system for an intelligent mobile terminal proposed in an embodiment of the present application is shown. Specifically, the system 500 includes a user terminal 510 and an intelligent agent terminal 520. The user terminal 510 is used to obtain first inertial data of the user terminal 510 and receive second inertial data sent by the intelligent agent terminal 520. The first inertial data is located in the current inertial coordinate system of the user device, and the second inertial data is located in the current inertial coordinate system of the intelligent agent. The user terminal 510 is also used to map both the first inertial data and the second inertial data to the default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data. The user terminal 510 is also used to obtain first motion trajectory information of the user device in the world coordinate system according to the third inertial data, and obtain second motion trajectory information of the intelligent agent in the world coordinate system according to the fourth inertial data. The user terminal 510 is also used to send the first motion trajectory information and the second motion trajectory information to the intelligent agent terminal 520, so that the intelligent agent performs motion following according to the first motion trajectory information and the second motion trajectory information.

[0075] The intelligent terminal 520 is used to receive the first motion trajectory information and the second motion trajectory information sent by the user terminal 510 based on the above steps; the intelligent terminal 520 is also used to perform motion tracking according to the first motion trajectory information and the second motion trajectory information.

[0076] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for automatic following of an intelligent mobile terminal, characterized in that: The method is applied to a user end of an automatic following system for an intelligent mobile terminal, wherein the system further comprises an intelligent body end, and the method comprises: Acquire first inertial data of the user end, and receive second inertial data sent by the agent end, wherein the first inertial data is located in the current inertial coordinate system of the user device, and the second inertial data is located in the current inertial coordinate system of the agent; Mapping the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data, wherein the default inertial coordinate system is an inertial coordinate system obtained by aligning an initial inertial coordinate system of the user device with an initial inertial coordinate system of the agent; Obtaining first motion track information of the user equipment in a world coordinate system according to the third inertial data, and obtaining second motion track information of the agent in a world coordinate system according to the fourth inertial data; The first motion trajectory information and the second motion trajectory information are sent to the intelligent agent end, so that the intelligent agent performs motion following according to the first motion trajectory information and the second motion trajectory information.

2. The method according to claim 1, characterized in that The first inertial data includes first rotation angle data and first acceleration data, the second inertial data includes second rotation angle data and second acceleration data, and the first inertial data and the second inertial data are mapped to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data, including: Acquire a first posture of the user equipment based on the first rotation angle data; Restoring the first acceleration data to the default inertial coordinate system according to the pre-acquired initial posture of the user equipment and the first posture, and using the restored first acceleration data as the third inertial data; Based on the second rotation angle data, obtaining a second posture of the intelligent agent; According to the pre-acquired initial posture of the intelligent body and the second posture, the second acceleration data is restored to the default inertial coordinate system, and the restored second acceleration data is used as the fourth inertial data.

3. The method according to claim 2, characterized in that The obtaining, according to the third inertial data, first motion track information of the user equipment in the world coordinate system, and obtaining, according to the fourth inertial data, second motion track information of the agent in the world coordinate system, comprises: Based on the pre-acquired rotation matrix, restoring the third inertial data to the world coordinate system; Integrate the restored third inertial data to obtain position information of the user equipment, and use the position information and acceleration information of the user equipment as first motion trajectory information of the user equipment in a world coordinate system; Based on the rotation matrix obtained in advance, restoring the fourth inertial data to the world coordinate system; The restored fourth inertial data is integrated to obtain the position information of the intelligent body, and the position information and acceleration information of the intelligent body are used as the second motion trajectory information of the intelligent body in the world coordinate system.

4. The method according to claim 1, characterized in that: Before acquiring the first inertial data of the user equipment and receiving the second inertial data of the agent sent by the agent terminal, the method further includes: Acquire a user device initial posture of the user device when an initial inertial coordinate system of the user device is aligned with an initial inertial coordinate system of the agent; An initial posture of the agent is received, where the initial posture of the agent is obtained when an inertial coordinate system where the user device is located is aligned with an inertial coordinate system where the agent is located.

5. A method for automatic following of an intelligent mobile terminal, characterized in that: The intelligent terminal applied to the automatic following system of the intelligent mobile terminal, the system also includes a user terminal, and the method includes: Receiving first motion trajectory information and second motion trajectory information sent by the user terminal based on the method according to any one of claims 1 to 4; Motion following is performed according to the first motion trajectory information and the second motion trajectory information.

6. The method according to claim 5, characterized in that The performing motion tracking according to the first motion trajectory information and the second motion trajectory information includes: According to the position information of the first motion trajectory information and the position information of the second motion trajectory information, the intelligent agent moves toward the position of the user equipment; According to the acceleration information of the first motion trajectory information and the acceleration information of the second motion trajectory information, the acceleration of the agent when moving is made consistent with the acceleration of the user equipment.

7. The method according to claim 6, characterized in that The intelligent agent terminal further includes a radar, and the intelligent agent is moved toward the location of the user equipment according to the location information of the first motion trajectory information and the location information of the second motion trajectory information, including: Obtaining obstacle information of the environment around the intelligent agent through the radar; An obstacle avoidance operation is performed according to the obstacle information.

8. An automatic following device for an intelligent mobile terminal, characterized in that: The system is applied to the user end of the automatic following system of the intelligent mobile terminal, and the system also includes an intelligent end. The device includes: A data acquisition module, used to acquire first inertial data of a user terminal and receive second inertial data sent by the agent terminal, wherein the first inertial data is located in a current inertial coordinate system of the user device and the second inertial data is located in a current inertial coordinate system of the agent; a data restoration module, configured to map the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data, wherein the default inertial coordinate system is an inertial coordinate system obtained by aligning an initial inertial coordinate system of the user device with an initial inertial coordinate system of the agent; A data calculation module, configured to obtain first motion track information of the user equipment in a world coordinate system according to the third inertial data, and obtain second motion track information of the agent in a world coordinate system according to the fourth inertial data; The sending module is used to send the first motion trajectory information and the second motion trajectory information to the intelligent agent end, so that the intelligent agent performs motion following according to the first motion trajectory information and the second motion trajectory information.

9. An automatic following device for an intelligent mobile terminal, characterized in that: The intelligent terminal applied to the automatic following system of the intelligent mobile terminal, the system also includes a user terminal, and the device includes: A receiving module, configured to receive the first motion trajectory information and the second motion trajectory information sent by the user terminal based on the method according to any one of claims 1 to 4; A following module is used to perform motion following according to the first motion trajectory information and the second motion trajectory information.

10. An automatic following system for an intelligent mobile terminal, characterized in that: The system includes a user end and an agent end; The user terminal is used to obtain first inertial data of the user terminal and receive second inertial data sent by the agent terminal, wherein the first inertial data is located in a current inertial coordinate system of the user device and the second inertial data is located in a current inertial coordinate system of the agent; The user terminal is further used to map the first inertial data and the second inertial data to a default inertial coordinate system to obtain corresponding third inertial data and fourth inertial data; The user terminal is further configured to obtain first motion track information of the user device in a world coordinate system according to the third inertial data, and obtain second motion track information of the agent in a world coordinate system according to the fourth inertial data; The user terminal is further configured to send the first motion trajectory information and the second motion trajectory information to the agent terminal, so that the agent performs motion following according to the first motion trajectory information and the second motion trajectory information; The intelligent agent terminal is used to receive the first motion trajectory information and the second motion trajectory information sent by the user terminal based on the method described in any one of claims 1 to 4; The intelligent body terminal is also used to perform motion tracking according to the first motion trajectory information and the second motion trajectory information.

Images