Gimbal control method and device, and electronic equipment

Abstract

The application discloses a gimbal control method and device and electronic equipment, and relates to the technical field of electronic equipment. The specific technical scheme is as follows: acquiring first interaction information, the first interaction information including at least one of the following: user emotion information, user behavior information; generating a first gimbal control instruction based on the first interaction information; and sending the first gimbal control instruction to a second electronic equipment, the first gimbal control instruction being used for controlling the movement of a second gimbal equipment.

Description

Technical Field

[0001] This application belongs to the field of electronic equipment technology, specifically relating to a gimbal control method, device, and electronic equipment. Background Technology

[0002] As video calls become increasingly prevalent in remote communication scenarios, users' demand for fully expressing their emotions during these calls is growing. However, current video call technologies, which only transmit information through visuals and sound, cannot meet users' needs for information interaction during video calls. Thus, in remote video call scenarios, existing devices suffer from a lack of flexibility and a limited range of information interaction methods. Summary of the Invention

[0003] The purpose of this application is to provide a PTZ control method, device, and electronic device that can improve the flexibility of information interaction between electronic devices in remote video call scenarios.

[0004] In a first aspect, embodiments of this application provide a gimbal control method, executed by a first electronic device, the first electronic device being communicatively connected to a first gimbal device, the first electronic device being communicatively connected to a second electronic device, and the second electronic device being communicatively connected to a second gimbal device. The method includes: acquiring first interaction information, the first interaction information including at least one of the following: user emotion information, user behavior information; generating a first gimbal control command based on the first interaction information; and sending the first gimbal control command to the second electronic device, the first gimbal control command being used to control the movement of the second gimbal device.

[0005] Secondly, embodiments of this application provide a gimbal control device, which is applied to a first electronic device. The first electronic device is communicatively connected to a first gimbal device, and the first electronic device is communicatively connected to a second electronic device. The second electronic device is communicatively connected to a second gimbal device. The device includes: a processing module and a sending module. The processing module is used to acquire first interactive information, which includes at least one of the following: user emotion information and user behavior information. The processing module is also used to generate a first gimbal control command based on the first interactive information. The sending module is used to send the first gimbal control command to the second electronic device, which is used to control the movement of the second gimbal device.

[0006] Thirdly, embodiments of this application provide an electronic device including a processor and a memory, wherein the memory stores programs or instructions executable on the processor, and the programs or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0007] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0008] Fifthly, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the method as described in the first aspect.

[0009] In a sixth aspect, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method described in the first aspect.

[0010] In this embodiment, when the first electronic device is communicatively connected to the second electronic device, it can acquire first interactive information, such as user emotion information or user behavior information. Then, the first electronic device can send a first gimbal control command to the second electronic device based on this first interactive information to control the movement of the second gimbal device. In this solution, the electronic device can convert user emotion information or user behavior information into corresponding control commands, and use these commands to enable the second electronic device to control the connected second gimbal device to perform corresponding movements. In other words, this solution, through the combination of the electronic device and the gimbal device, can intuitively reflect user emotions or user behavior through the physical movement of the gimbal, thus meeting the user's information interaction needs during video calls. This improves the flexibility of information interaction between electronic devices in remote video call scenarios. Attached Figure Description

[0011] Figure 1 This is a flowchart illustrating the gimbal control method provided in an embodiment of this application;

[0012] Figure 2 This is a schematic diagram of the physical interaction scenario of the PTZ during a video call provided in an embodiment of this application;

[0013] Figure 3 This is a flowchart illustrating the gimbal control method provided in an embodiment of this application;

[0014] Figure 4 This is a schematic diagram illustrating the user emotion collection method applied in the gimbal control method provided in this application embodiment;

[0015] Figure 5 This is a schematic diagram illustrating the user emotion collection method applied in the gimbal control method provided in this application embodiment;

[0016] Figure 6This is a schematic diagram illustrating the method for establishing a physical expression library in the gimbal control method provided in this application embodiment;

[0017] Figure 7 This is a flowchart illustrating the gimbal control method provided in an embodiment of this application;

[0018] Figure 8 This is a schematic diagram of the gimbal control method provided in the embodiments of this application;

[0019] Figure 9 This is a flowchart illustrating the gimbal control method provided in an embodiment of this application;

[0020] Figure 10 This is a schematic diagram of the motion mapping method applied in the gimbal control method provided in the embodiments of this application;

[0021] Figure 11 This is a schematic diagram of the gimbal interaction window used in the gimbal control method provided in this application embodiment;

[0022] Figure 12 This is a flowchart illustrating the gimbal control method provided in an embodiment of this application;

[0023] Figure 13 This is a flowchart illustrating the gimbal control method provided in an embodiment of this application;

[0024] Figure 14 This is a schematic diagram of the gimbal control device provided in the embodiments of this application;

[0025] Figure 15 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0026] Figure 16 This is a schematic diagram of the hardware structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0028] The terms "first," "second," etc., used in this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class, without limiting the number of objects; for example, a first object can be one or more. Furthermore, in this application, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects have an "or" relationship.

[0029] In this application, the terms "at least one," "at least one," etc., refer to any one, any two, or a combination of two or more of the included objects. For example, at least one of a, b, and c can mean: "a," "b," "c," "a and b," "a and c," "b and c," and "a, b, and c," where a, b, and c can be single or multiple. Similarly, "at least two" refers to two or more, and its meaning is similar to that of "at least one."

[0030] The gimbal control method, device, and electronic equipment provided in this application will be described in detail below with reference to the accompanying drawings and through specific embodiments and application scenarios.

[0031] The embodiments of this application can be applied to video call scenarios, specifically scenarios where information is exchanged during a video call.

[0032] The executor of the gimbal control method provided in this application embodiment can be a gimbal control device, which can be an electronic device, or a functional module or functional entity within an electronic device. The following uses an electronic device as an example to illustrate the technical solution provided in this application embodiment.

[0033] Figure 1 A flowchart of a gimbal control method provided in an embodiment of this application is shown, such as... Figure 1 As shown, the gimbal control method provided in this application embodiment may include the following steps 201 to 203.

[0034] Step 201: The first electronic device acquires the first interactive information.

[0035] In some embodiments of this application, the first electronic device is communicatively connected to the first gimbal device, the first electronic device is communicatively connected to the second electronic device, and the second electronic device is communicatively connected to the second gimbal device.

[0036] In some embodiments of this application, the first interaction information mentioned above includes at least one of the following: user emotion information and user behavior information.

[0037] In some embodiments of this application, during a video call between a first electronic device and a second electronic device, the first electronic device can obtain first interactive information.

[0038] In some embodiments of this application, after the first electronic device and the second electronic device establish a video call connection through any communication application, the first electronic device and the second electronic device can interact with each other through the video call data channel, such as the picture, sound, control commands, etc. during the video call.

[0039] In some embodiments of this application, the first electronic device and the first gimbal device are connected via Type-C or other interfaces or wirelessly, and the second electronic device and the second gimbal device are connected via Type-C or other interfaces or wirelessly. The first electronic device and the second electronic device can transmit first gimbal control commands, such as gimbal motion commands and haptic feedback commands, through WebRTC DataChannel or other video call data channels to achieve physical motion drive and vibration feedback control of the other party's gimbal device. It should be noted that the embodiments of this application use the example of the first electronic device controlling the gimbal device motion of the second electronic device to illustrate the gimbal control method. The gimbal control method of the embodiments of this application is also applicable to the implementation scheme of the second electronic device controlling the gimbal device motion of the first electronic device. That is, during a video call, both the first electronic device and the second electronic device can achieve information interaction by controlling the gimbal device of the other party.

[0040] For example, let's illustrate this using mobile phones as both the first and second electronic devices. Figure 2 The diagram shown illustrates the physical interaction scenario of the PTZ (pan-tilt unit) during a video call. The first electronic device 10, i.e. Figure 2 Mobile phone A (the local device) is connected to the first gimbal device 12 via Type-C interface 11, and the second electronic device 13, i.e. Figure 2 Mobile phone B (the peer device) is connected to the second gimbal device 15 via Type-C interface 14. During a video call, mobile phone A and mobile phone B can exchange information, such as images and sounds, as well as control commands for the first gimbal, through the video call data channel (which can be referred to as the video call channel). Thus, mobile phone A can display the image 16 captured by mobile phone B, and mobile phone B can display the image 17 captured by mobile phone A. In this way, while exchanging images and sounds in real time, the gimbal device connected to the other end can be controlled to move, thereby realizing information exchange.

[0041] In some embodiments of this application, the aforementioned user emotion information may include the user emotion type and the user emotion intensity.

[0042] In some embodiments of this application, the aforementioned user emotion type can be a user-defined emotion type or a default emotion type of the electronic device system.

[0043] For example, the aforementioned user emotion types may include, but are not limited to: anger, happiness, sadness, confusion, surprise, fear, disgust, etc.

[0044] In some embodiments of this application, the aforementioned user emotion intensity can be a user-defined emotion intensity or the default emotion intensity of the electronic device system.

[0045] For example, the intensity of the user's emotions mentioned above may include, but is not limited to, low intensity, medium intensity, and high intensity.

[0046] It should be noted that the explanation of how user emotion information is obtained can be found in the description of the embodiments below, and will not be repeated here.

[0047] In some embodiments of this application, the aforementioned user behavior information may include at least one of the following: user head movement information, user gaze information, and motion trajectory information when the user operates the first gimbal device.

[0048] In some embodiments of this application, the first electronic device can acquire user head movement information through a front-facing camera. The user head movement information may include the user head deflection angle and the user head deflection direction.

[0049] In some embodiments of this application, the first electronic device can capture a user's facial image using a front-facing camera to detect the user's head yaw angle. This yaw angle can be represented by three Euler angles: yaw, pitch, and roll. It should be noted that, to clearly define these angles, a three-dimensional Cartesian coordinate system is pre-constructed with the electronic device screen as the reference: the origin O is located at the center of the screen, the X-axis is horizontally to the right along the screen, the Y-axis is vertically upward along the screen, and the Z-axis is perpendicular to the screen plane and points towards the user. When the user's head is in an initial posture facing the screen, with the line connecting their eyes parallel to the X-axis and their face facing the positive Z-axis, all three angles are zero. Any head posture can be considered as the result of rotation around the Z-axis, Y-axis, and X-axis in sequence: the angle of rotation around the Z-axis is the Roll angle, used to represent the "head tilting" action of the user's head tilting towards the shoulder; the Roll angle is positive when the right ear is lowered towards the screen. The angle of rotation around the Y-axis is the Yaw angle, used to represent the "head shaking" action of the user's head turning left and right around the vertical axis; the Yaw angle is positive when the head turns to the left (the face turns to the user's left side). The angle of rotation around the X-axis is the Pitch angle, used to represent the "nodding" or "head tilting" action of the user's head swinging up and down; the Pitch angle is positive when the head is raised and the face tilts upward. The posture estimation module in the electronic device can extract feature points from the facial image and calculate the above three Euler angles, thereby uniquely determining the spatial deflection state of the user's head relative to the screen of the electronic device.

[0050] In some embodiments of this application, the first electronic device can acquire user gaze information, such as the user's gaze direction, through eye tracking.

[0051] It should be noted that the aforementioned user gaze information can be obtained through an infrared camera or a dedicated eye-tracking sensor when the first electronic device supports eye-tracking functionality. The gaze direction can be represented as a normalized direction vector, or as the horizontal and vertical offset angles relative to the center of the screen or the optical axis of the camera.

[0052] In some embodiments of this application, the first electronic device may also combine the attitude information of the first electronic device to generate the first gimbal control command.

[0053] In some embodiments of this application, the first electronic device can acquire attitude information of the first electronic device through sensors, and the attitude information may include tilt angle, angular velocity information, and angular acceleration information.

[0054] In some embodiments of this application, the attitude information of the first electronic device can be obtained by the first electronic device through its built-in inertial measurement unit. This attitude information includes at least the tilt angle of the device relative to the direction of gravity or the geomagnetic direction. For example, the tilt angle may include the pitch angle, roll angle, and yaw angle.

[0055] In some embodiments of this application, the motion trajectory information of the first gimbal device operated by the user can be the motion trajectory when the user directly pushes the first gimbal device to rotate manually. The motor encoder or angle sensor of the first gimbal device records the sequence of angle changes of each axis of the gimbal during the user's pushing process, forming motion trajectory information. For example, this motion trajectory information can be represented as a sequence of attitude angle key points with timestamps.

[0056] Step 202: The first electronic device generates a first gimbal control command based on the first interactive information.

[0057] In some embodiments of this application, the first gimbal control command includes first motion information.

[0058] In some embodiments of this application, the first electronic device may generate a first gimbal control command based on the first interactive information, and transmit the first motion information through the first gimbal control command.

[0059] In some embodiments of this application, the first motion information may include at least one of the following: motion trajectory information, haptic feedback instructions (including vibration mode information), and gimbal motion parameters.

[0060] In some embodiments of this application, the first interactive information includes user emotion information, which is associated with motion trajectory information and / or vibration pattern information. The first electronic device generates a first gimbal motion command based on the motion trajectory information and generates a haptic feedback command based on the vibration pattern information.

[0061] In some embodiments of this application, the first interactive information includes user behavior information, which is mapped to gimbal motion parameters, and the first electronic device generates a first gimbal control command based on the gimbal motion parameters.

[0062] It should be noted that the explanation of each piece of information included in the first motion information, as well as the specific implementation of step 202, are provided below and will not be repeated here.

[0063] Step 203: The first electronic device sends a first gimbal control command to the second electronic device.

[0064] In some embodiments of this application, the first gimbal control command is used to control the movement of the second gimbal device.

[0065] In some embodiments of this application, the first interactive information includes user emotion information, and the first gimbal control command is used to control the movement of the second gimbal device. That is, the second electronic device realizes physical motion control of the second gimbal device based on motion trajectory information, and realizes tactile feedback control of the vibration motor of the second gimbal device based on vibration mode information.

[0066] In some embodiments of this application, the first interactive information includes user behavior information, and the first gimbal control command is used to control the movement of the second gimbal device, that is, the second electronic device drives the second gimbal device to rotate synchronously based on the gimbal motion parameters.

[0067] In this way, when the first electronic device is connected to the second electronic device, it can acquire first interactive information, such as user emotion information or user behavior information. Then, the first electronic device can send a first gimbal control command to the second electronic device based on this first interactive information to control the movement of the second gimbal device. In this solution, the first electronic device can convert user emotion information or user behavior information into corresponding control commands, and use these commands to enable the second electronic device to control the connected second gimbal device to perform corresponding movements. In other words, this solution, through the combination of electronic devices and gimbal devices, can intuitively reflect user emotions or user behavior through the physical movement of the gimbal, thus meeting the user's information interaction needs during video calls. This improves the flexibility of information interaction between electronic devices in remote video call scenarios.

[0068] In some embodiments of this application, the aforementioned first interaction information includes user emotion information. For example, in conjunction with... Figure 1 ,like Figure 3 As shown, step 201 above can be implemented through step 201A or step 201B as described below.

[0069] Step 201A: The first electronic device collects user facial expression information and user posture information through its camera, and performs intent recognition based on the user facial expression information and user posture information to obtain user emotion information.

[0070] In some embodiments of this application, the aforementioned user emotion information includes the user emotion type and the user emotion intensity.

[0071] In some embodiments of this application, the first electronic device can recognize the user's facial expressions and head posture in real time through the front-facing camera of the first electronic device, and input the user's facial expressions and head posture into an AI model so as to determine the user's current intention category and intensity through the AI ​​model and obtain the user's emotional information.

[0072] Specifically, the first electronic device captures the user's facial images in real time through its front-facing camera and uses image processing algorithms to extract facial muscle movement features (such as eyebrow and eye opening and closing, and mouth corner curvature) and posture parameters representing changes in the three-dimensional spatial angle of the head. These two types of temporal feature vectors are fed into a pre-trained deep learning model as input data. The model is based on a cascaded architecture of convolutional neural networks and recurrent neural networks. It first encodes spatial features, then captures the dynamic evolution patterns in the temporal dimension, and outputs probability distributions corresponding to multiple preset emotion categories (such as happiness, surprise, anger, sadness, etc.) and a continuous intensity value in the range of 0 to 1 through a fully connected classification layer and a regression branch. Finally, the user's current intention category is determined based on the maximum probability value, and the intensity value is used to quantify the degree of emotional agitation, thereby achieving real-time semantic understanding and quantitative evaluation of the user's non-verbal behavior signals.

[0073] For example, in combination Figure 2 ,like Figure 4 As shown, the first electronic device 10, namely mobile phone A, is connected to the first gimbal device 12 through the Type-C interface 11. The first electronic device collects the user's facial expressions and head posture in real time through the front-facing camera, and inputs the user's facial expressions and head posture into the AI ​​model so that the AI ​​model can determine the user's current intention category and intensity, such as the intention category being "happy" and the intensity being "medium intensity".

[0074] In some embodiments of this application, the first electronic device may also collect user facial expression information and user posture information through the camera of the first gimbal device.

[0075] Step 201B: When the first electronic device acquires the first expression image, it acquires a first emotion tag associated with the first expression image based on the first expression image. The first emotion tag is used to represent the user's emotional information.

[0076] In some embodiments of this application, a user can send a first emoticon image in the message box of the video call interface, and the first electronic device can obtain the first emoticon image from the message box of the video call interface. The first emoticon image can be a user-defined emoticon image or a default emoticon image in the application corresponding to the video call interface.

[0077] In some embodiments of this application, the first facial expression image described above can be a static image or a dynamic image.

[0078] For example, in combination Figure 2 ,like Figure 5As shown, the first electronic device 10 is connected to the first gimbal device 12 via a Type-C interface 11. The user can select a first emoticon image, such as a happy emoticon image, in the video call interface of the first electronic device, i.e., mobile phone A, to trigger mobile phone A to send the first emoticon image to mobile phone B through the video call interface.

[0079] In some embodiments of this application, the aforementioned first emotion tag can be an emoticon image or emotional text content. Associating a first emoticon image with a first emotion tag means that the first emoticon image matches the emoticon image corresponding to the first emotion tag, or that the emotional text content corresponding to the first emoticon image matches the emotional text content corresponding to the first emotion tag.

[0080] In some embodiments of this application, when a user sends a first emoticon image in the message box of a video call interface, the first electronic device can read the emotion tag, motion trajectory information and vibration mode information bound to the first emoticon image from a preset physical emoticon library.

[0081] In some embodiments of this application, the aforementioned physical expression library includes at least one emotion tag, as well as motion trajectory information and vibration pattern information associated with each emotion tag. It should be noted that the at least one emotion tag can be an expression image or text content corresponding to the emotion.

[0082] In some embodiments of this application, after the first electronic device enters the gimbal motion recording mode, the user can select an emotion tag (such as angry, happy, confused, etc.). The user can then manually control the first gimbal device to execute the desired motion trajectory. For example, anger corresponds to rapid left-right vibration, and happiness corresponds to slow left-right swaying in a V-shape. The first electronic device can record this motion trajectory, recording the angular displacement sequence of each axis, motion speed, motion acceleration, and duration. It also binds a custom vibration mode (including vibration frequency, amplitude, vibration duration, etc.) to each motion or action, executed by the built-in vibration motor of the gimbal device. Thus, the first electronic device binds the motion trajectory information to the emotion tag and stores it in a preset physical expression library.

[0083] For example, such as Figure 6As shown, the first electronic device 10, namely mobile phone A, is connected to the first gimbal device 12 via a Type-C interface 11. After the mobile phone enters the gimbal motion recording mode, it can display multiple emotion labels, such as angry, sad, excited, etc. Users can customize or overwrite emotion labels. Users can manually control the gimbal device to execute the desired motion trajectory. For example, anger corresponds to rapid left and right vibration, happiness corresponds to slow left and right swing in a V shape, excitement corresponds to small left and right swings, etc. The mobile phone can record the motion trajectory, record the angular displacement sequence of each axis, motion speed, motion acceleration and duration, and bind a custom vibration mode (including vibration frequency, amplitude, vibration duration, etc.) to each motion or action. For example, rapid left and right vibration is bound to high frequency short amplitude, slow left and right swing in a V shape is bound to mid-frequency gradually increasing amplitude, and small left and right swings are bound to low-frequency micro-vibration, etc., thereby obtaining a physical expression library.

[0084] In some embodiments of this application, the aforementioned physical emoji library can be shared with the other party in a call and can be migrated between different devices synchronized with the user's account.

[0085] In this way, electronic devices can acquire user emotional information in two ways: through camera capture and recognition, and by searching in a physical expression database, which improves the flexibility of user emotion acquisition. Furthermore, by binding user emotions with corresponding gimbal movements, the information expression is made more human-like by extending from visual symbols to physical motion.

[0086] In some embodiments of this application, the aforementioned first interactive information includes user emotion information, and the aforementioned first gimbal control command includes at least one of the following: a first gimbal motion command and a haptic feedback command. For example, combined with... Figure 1 ,like Figure 7 As shown, step 202 above can be implemented through steps 202A and 202B as described below.

[0087] Step 202A: The first electronic device acquires motion trajectory information associated with the user's emotional information and generates a first gimbal motion command based on the motion trajectory information.

[0088] In some embodiments of this application, the first electronic device can generate the first gimbal motion command based on the aforementioned motion trajectory information. The first gimbal motion command is used to instruct the second electronic device to control the second gimbal to perform the motion corresponding to the motion trajectory information, such as the second gimbal rapidly vibrating left and right, or the second gimbal slowly swinging left and right in a V shape, or the second gimbal swinging slightly left and right.

[0089] In some embodiments of this application, the aforementioned motion trajectory information may further include motion amplitude information and motion time information. The motion amplitude information indicates a positive correlation between the motion amplitude and the intensity of the user's emotions.

[0090] It should be noted that the aforementioned motion trajectory information can consist of a sequence of spatial key points arranged in time. Each spatial key point contains at least one expected value of the gimbal attitude angle. The gimbal device can perform interpolation based on this sequence to drive smooth gimbal movement, and the motion trajectory information can also be represented as a set of parameters for a motion control function.

[0091] The aforementioned motion amplitude information is used to characterize the intensity of the gimbal's actions, and it is positively correlated with the emotion intensity value corresponding to the emotion tag. For example, when the detected user emotion is "happy" and the emotion intensity is 0.8 (intensity range 0-1), the motion amplitude information can be a scaling factor of 0.8. This factor will be multiplied by the range of change of each posture angle in the basic motion trajectory, so that the actual swing angle of the gimbal is 80% of the basic trajectory.

[0092] Understandably, the scaling factor mentioned above can also be positively correlated with the user's emotional intensity value. For example, when a low user emotional intensity is detected, the scaling factor can be 0.3; when a medium user emotional intensity is detected, the scaling factor can be 0.6; and when a high user emotional intensity is detected, the scaling factor can be 1.0.

[0093] The aforementioned motion timing information is used to characterize the time it takes for the gimbal device to execute the motion command. For example, the motion timing information may include a timestamp indicating the start time of the gimbal motion command. This timestamp may be generated based on the local clock of the first electronic device to ensure synchronization between the gimbal movement and other outputs (such as video and audio). The motion timing information may also further include the motion duration and an optional velocity profile curve. For example, the motion timing information indicates that the entire trajectory sequence is completed within 2 seconds, and uses easing-in / easing-out velocity control to present a more natural physical expression effect.

[0094] In some embodiments of this application, the exercise time information can be positively correlated with the intensity of the user's emotions. For example, the greater the intensity of the user's emotions, the longer the duration of the corresponding exercise, which can more realistically reflect the user's emotions.

[0095] In some embodiments of this application, after the first electronic device obtains the current user's emotion information, it can query a preset physical expression library based on the emotion information to determine whether there is a physical expression or emotion tag that matches the user's emotion information. If a matching physical expression is found, the motion trajectory information bound to the physical expression is read, and a first gimbal motion command is generated based on the motion trajectory information. If no physical expression is matched, a preset action template is invoked to generate the first gimbal motion command.

[0096] In some embodiments of this application, the "first electronic device acquires motion trajectory information associated with user emotion information" in step 202A above can be specifically implemented through step 202A1 or step 202A2 below.

[0097] Step 202A1: The first electronic device acquires a second emotion tag that matches the user's emotion information and determines the motion trajectory information associated with the second emotion tag.

[0098] In some embodiments of this application, the first electronic device may query a preset physical expression library to obtain a second emotion tag that matches the user's emotional information. Matching the user's emotional information with the second emotion tag means that the degree of matching or similarity between the user's emotional information and the emotional information represented by the expression image corresponding to the second emotion tag is greater than a threshold, or the difference between the two is less than a threshold.

[0099] It should be noted that a detailed explanation of the physical emoji library can be found in the description of the above embodiments, and will not be repeated here.

[0100] Step 202A2: The first electronic device generates motion trajectory information using an action template associated with the user's emotional information.

[0101] It should be noted that the aforementioned preset action templates refer to predefined and stored in the first electronic device or its associated cloud storage, which are independent of user-defined gimbal motion command generation rules or parameter sets. Each preset action template can correspond to one or more basic emotion categories (such as happiness, sadness, surprise). It can be understood that a preset action template can be a complete set of motion parameters or an action generation algorithm that can output deterministic gimbal motion command parameters based on the input emotion information.

[0102] In this way, electronic devices can obtain motion trajectory information in two ways: searching in the physical expression library and generating preset action templates, which improves the flexibility of motion trajectory acquisition. Furthermore, by driving the gimbal device to execute the actions corresponding to the motion trajectory information through gimbal motion commands, information interaction is achieved through the physical movement of the gimbal device, making the information expression more human-like.

[0103] Step 202B: The first electronic device acquires vibration pattern information associated with the user's emotional information and generates tactile feedback instructions based on the vibration pattern information.

[0104] In some embodiments of this application, the first electronic device can generate the tactile feedback instruction based on the vibration mode information. The tactile feedback instruction is used to instruct the second electronic device to control the second gimbal device to execute the tactile feedback corresponding to the vibration mode information, such as controlling the vibration motor of the second gimbal device to generate a medium-frequency gradually increasing vibration, or the vibration motor of the second gimbal device to generate a high-frequency short-amplitude vibration, or the vibration motor of the second gimbal device to generate a low-frequency long-amplitude vibration.

[0105] In some embodiments of this application, the first motion information includes motion trajectory information and vibration mode information. It is understood that when performing step 202A, the first motion information includes motion trajectory information, and when performing step 202B, the first motion information includes vibration mode information.

[0106] In some embodiments of this application, the vibration mode information described above is a set of parameters used to describe the vibration behavior characteristics of the haptic feedback device. For example, the haptic feedback device can be a linear motor or an eccentric rotor motor. Exemplarily, the vibration mode information may include at least one of the following:

[0107] Vibration frequency information, used to indicate the repetition rate of vibration, for example, expressed in Hertz;

[0108] Vibration amplitude information, used to indicate the intensity of vibration, for example, expressed as a percentage of maximum displacement or driving voltage;

[0109] Vibration duration information, used to indicate the duration of a single vibration or a set of vibration modes.

[0110] In some embodiments of this application, the haptic feedback instructions described above may include at least one of the following:

[0111] Vibration frequency parameter, used to set the drive signal frequency of the motor;

[0112] Vibration amplitude parameters are used to set the amplitude or duty cycle of the drive signal;

[0113] Vibration time parameter, used to specify the start time or duration of vibration.

[0114] In some embodiments of this application, the first electronic device can generate haptic feedback commands based on the user's emotion type and intensity, so that the second gimbal device can generate haptic feedback through its built-in vibration motor.

[0115] For example, in combination Figure 2 ,like Figure 8As shown, the first electronic device 10, namely mobile phone A (local device), is connected to the first gimbal device 12 via Type-C interface 11, and the second electronic device 13, namely mobile phone B (remote device), is connected to the second gimbal device 15 via Type-C interface 14. Based on the "happy" emotional information, mobile phone A obtains motion trajectory information associated with the "happy" emotional information, such as slowly swinging left and right in a V shape, and generates a first gimbal motion command based on the motion trajectory information, so that mobile phone B controls gimbal device B to perform slowly swinging left and right in a V shape. Furthermore, based on the "happy" emotional information, mobile phone A obtains vibration mode information associated with the "happy" emotional information, such as mid-frequency gradually increasing vibration, and generates a corresponding tactile feedback command based on the vibration mode information, so that mobile phone B controls gimbal device B to generate mid-frequency gradually increasing vibration tactile feedback through the built-in vibration motor. In addition, for users with the emotion "anger," the associated vibration pattern is high-frequency, short-amplitude vibration; for users with the emotion "sadness," the associated vibration pattern is low-frequency, long-amplitude vibration; for users with the emotion "excitement," the associated vibration pattern is mid-frequency, gradually increasing vibration; and for users with the emotion "calm," the associated vibration pattern is no vibration or extremely low-frequency micro-vibration. Of course, users can also bind custom vibration patterns (including vibration frequency, amplitude, duration, etc.) to each custom action in the physical expression library.

[0116] In this way, electronic devices can drive the remote gimbal device to complete corresponding actions through gimbal motion commands, and electronic devices can drive the remote gimbal device to output corresponding tactile feedback through tactile feedback commands. Therefore, by combining motion and tactile feedback, intuitive feedback of user emotions can be achieved, realizing the frequency-based physical expression of emotions and improving the flexibility of emotional information interaction during video calls.

[0117] In some embodiments of this application, the aforementioned first interaction information includes user behavior information. For example, in conjunction with... Figure 1 ,like Figure 9 As shown, step 202 above can be specifically implemented through step 202C below.

[0118] Step 202C: The first electronic device maps user behavior information into gimbal motion parameters and generates a first gimbal control command based on the gimbal motion parameters.

[0119] In some embodiments of this application, the aforementioned user behavior information may include at least one of the following: user head movement information, user gaze information, and motion trajectory information when the user operates the first gimbal device.

[0120] The aforementioned first gimbal control command is used to instruct the second electronic device to control the second gimbal device to perform the movement corresponding to the gimbal motion parameters, such as the second gimbal device turning 20 degrees to the left, or the second gimbal device turning to the right, or the second gimbal device synchronously performing the rotation corresponding to the user's manual push of the first gimbal device.

[0121] In some embodiments of this application, the first motion information mentioned above includes gimbal motion parameters. It is understood that, when performing step 202C above, the first motion information includes gimbal motion parameters.

[0122] In some embodiments of this application, the above-mentioned gimbal motion parameters may include at least one of the following: gimbal rotation direction, gimbal rotation angle, and motion trajectory information when the user operates the first gimbal device.

[0123] For example, in combination Figure 2 ,like Figure 10 As shown, the first electronic device 10, i.e., mobile phone A (the local device), is connected to the first gimbal device 12, and the second electronic device 13, i.e., mobile phone B (the remote device), is connected to the second gimbal device 15. When the user turns their head 20 degrees to the left, mobile phone A is mapped proportionally, and gimbal device B turns 20 degrees to the left (or scales according to a set ratio). When the user tilts their phone to the right, mobile phone A is mapped proportionally, and gimbal device B rotates to the right. When the user manually pushes the local gimbal, mobile phone A is mapped proportionally, and gimbal device B rotates synchronously. It is understood that the mapping range is limited by physical limit protection, such as maximum rotation angle limit, rotation speed limit, etc. In addition, the mapping range is also limited by user authorization restrictions. For an explanation of user authorization restrictions, please refer to the description of the embodiments below, which will not be repeated here.

[0124] In this way, electronic devices can drive the other end's gimbal device to complete corresponding actions through gimbal motion commands. Thus, intuitive feedback on user behavior is achieved through actions. That is, gimbal motion commands are transmitted through the video call data channel, and the other end drives the gimbal device to perform synchronous rotation, enabling users to actively look around the space where the other party is located, gain a sense of remote presence, and improve the flexibility of behavioral information interaction during video calls.

[0125] In some embodiments of this application, before "the first electronic device acquires the first interactive information" in step 201 above, the gimbal control method provided in this application embodiment further includes the following steps 301 to 303.

[0126] Step 301: The first electronic device displays the PTZ interactive window.

[0127] In some embodiments of this application, the above-mentioned gimbal interaction window includes at least one of the following: an emotion interaction authorization control and a behavior interaction authorization control.

[0128] In some embodiments of this application, during a video call between the first electronic device and the second electronic device, if both devices are detected to be connected to a PTZ device, PTZ interaction options, such as emotion interaction authorization controls and behavior interaction authorization controls, are displayed through a PTZ interaction window.

[0129] In some embodiments of this application, the aforementioned emotion interaction authorization control is used to trigger authorization for emotion interaction. The aforementioned behavior interaction authorization control is used to trigger authorization for behavior interaction.

[0130] Step 302: The first electronic device sends a PTZ interaction authorization request to the second electronic device based on the confirmation input of the target authorization control.

[0131] In some embodiments of this application, the target authorization control includes at least one of an emotion interaction authorization control and a behavior interaction authorization control.

[0132] In some embodiments of this application, the target authorization control includes an emotion interaction authorization control, and the PTZ interaction authorization request is used to request authorization for emotion interaction. Alternatively, the target authorization control may include a behavior interaction authorization control, and the PTZ interaction authorization request is used to request authorization for behavior interaction. In another embodiment, the target authorization control may include both an emotion interaction authorization control and a behavior interaction authorization control, and the PTZ interaction authorization request is used to request authorization for both emotion interaction and behavior interaction.

[0133] For example, such as Figure 11 As shown, the first electronic device 10, namely mobile phone A (the local device), is connected to the first gimbal device 12. Mobile phone A displays a gimbal interaction window, which includes emotion interaction authorization controls and behavior interaction authorization controls. Figure 11 The option for "emotional expression only (physical facial expression)" illustrates the emotion interaction authorization control, while the option for "direct guidance (gimbal control)" illustrates the behavior interaction authorization control. Users can select at least one of these controls to enable phone A to send a gimbal interaction authorization request to phone B.

[0134] Step 303: After receiving the authorization and consent message from the second electronic device, the first electronic device controls the establishment of a PTZ control channel between the first electronic device and the second electronic device.

[0135] In some embodiments of this application, the above-mentioned gimbal control channel is used to transmit a first gimbal control command.

[0136] Understandably, if the authorization message grants permission for emotional interaction, the aforementioned gimbal control channel can be used to transmit the first gimbal motion command and haptic feedback command. If the authorization message grants permission for behavioral interaction, the aforementioned gimbal control channel can be used to transmit the second gimbal motion command.

[0137] Thus, during video calls between the first and second electronic devices, the scope of interaction of the peer PTZ device is controlled based on the authorization mechanism, ensuring the security and controllability of the PTZ device.

[0138] In some embodiments of this application, the above-described gimbal control method further includes steps 304 and 305.

[0139] Step 304: The first electronic device receives the second PTZ control command sent by the second electronic device.

[0140] In some embodiments of this application, the second gimbal control command includes second motion information;

[0141] Step 305: The first electronic device controls the movement of the first gimbal device based on the second gimbal control command.

[0142] It should be noted that the implementation scheme of the second gimbal control command is similar to that of the first gimbal control command, and the implementation scheme of the second motion information is similar to that of the first motion information, and will not be described in detail here.

[0143] In this way, during a video call, the first and second electronic devices can instruct each other to control their respective PTZ devices to perform corresponding movements, thus enabling more flexible information interaction.

[0144] The execution flow of the gimbal control method provided in this application embodiment is illustrated below through specific implementation methods.

[0145] Implementation method 1: Emotional interaction. For example... Figure 12 As shown, the gimbal control method provided in this application embodiment includes the following steps 20 to 26.

[0146] Step 20: The first electronic device pre-records a physical expression library.

[0147] The process involves first selecting an emotion tag, then recording the gimbal's motion trajectory and binding it to a vibration mode, and finally storing the binding relationship in a physical expression library.

[0148] The aforementioned physical expression library can include: preset emotion tags, such as sadness, excitement, surprise, etc., and users can also customize expression tags, such as anger, happiness, etc.

[0149] It is understandable that the actions of the gimbal device can provide tactile feedback corresponding to the motion trajectory information and the vibration mode information.

[0150] Step 21: The first electronic device establishes video call and PTZ interaction authorization.

[0151] In this process, the first electronic device and the second electronic device establish a video call and detect whether both the first electronic device and the second electronic device are connected to the PTZ device. If both are connected to the PTZ device, the authorization option is displayed. The user can select the authorization range to request the other end device to confirm the authorization.

[0152] Step 22A: The first electronic device automatically identifies the user's emotional information through AI.

[0153] The first electronic device uses a front-facing camera and AI to recognize the user's facial expressions and head posture, and determines the type and intensity of the user's emotions based on the facial expressions and head posture.

[0154] Step 22B: The first electronic device obtains the user's emotional information based on the physical facial expressions sent by the user in the video call interface.

[0155] In this process, the first electronic device displays a physical emoji library on the video call interface. The user actively selects an emoji image, and then the first electronic device obtains the emotion tag corresponding to the emoji image, as well as the motion trajectory and vibration mode bound to the emotion tag.

[0156] Step 23: The first electronic device generates gimbal movement commands based on the user's emotional information.

[0157] Step 24: The first electronic device generates haptic feedback instructions based on the user's emotional information.

[0158] The haptic feedback commands are generated based on the user's emotion type and intensity. Additionally, the gimbal device generates haptic feedback through its vibration motor.

[0159] Step 25: The first electronic device sends gimbal motion commands and haptic feedback commands to the second electronic device to drive the second gimbal device to perform corresponding movements.

[0160] For example, the second gimbal device performs physical motion and provides haptic feedback through a vibration motor, with the two motions transitioning smoothly.

[0161] Step 26: During the video call, the first electronic device continuously collects the user's emotional information.

[0162] It is understood that the first electronic device may repeatedly execute steps 22A (or 22B) up to step 25.

[0163] It should be noted that the relevant solutions for steps 20 to 26 above can be found in the detailed description of the above embodiments. They will not be repeated here.

[0164] Implementation Method Two: Behavioral Interaction. For example... Figure 13As shown, the gimbal control method provided in this application embodiment includes the following steps 31 to 35.

[0165] Step 31: The first electronic device establishes video call and authorizes gimbal interaction.

[0166] In this process, the first electronic device and the second electronic device establish a video call and detect whether both the first electronic device and the second electronic device are connected to the PTZ device. If both are connected to the PTZ device, the authorization option is displayed. The user can select the authorization range to request the other end device to confirm the authorization.

[0167] Step 32: The first electronic device acquires user behavior information.

[0168] The first electronic device acquires the user's head posture through a front-facing camera, or obtains the user's gaze information through eye tracking, or detects the user's manual pushing of the first gimbal device. In addition to user behavior information, the tilt angle of the first electronic device can also be obtained through sensors.

[0169] Step 33: The first electronic device generates gimbal motion commands based on user behavior information.

[0170] The first electronic device can proportionally map user behavior information, such as when the user turns their head 20 degrees to the left, the PTZ device B turns 20 degrees to the left (or scales proportionally); when the user tilts their phone to the right, the PTZ device B rotates to the right; when the user manually pushes the local PTZ, the PTZ device B rotates synchronously.

[0171] Step 34: The first electronic device sends a gimbal motion command to the second electronic device to drive the second gimbal device to perform the corresponding motion.

[0172] The first electronic device can instruct the second electronic device to control the second gimbal device to perform synchronous rotation, thereby achieving a remote sense of presence by looking around the other party's space.

[0173] Step 35: During the video call, the first electronic device continuously collects user behavior information.

[0174] It is understandable that the first electronic device can repeatedly execute steps 32 to 34 above.

[0175] It should be noted that the relevant solutions for steps 31 to 35 above can be found in the detailed description of the above embodiments. They will not be repeated here.

[0176] It should be noted that each of the above method embodiments, or various possible implementations of each method embodiment, can be executed individually or in combination of any two or more. The specific implementation can be determined according to actual usage requirements, and this application embodiment does not impose any restrictions on this.

[0177] The gimbal control method provided in this application can be executed by a gimbal control device. This application uses the example of a gimbal control device executing the gimbal control method to illustrate the gimbal control device provided in this application.

[0178] Figure 14 This is a schematic diagram of a gimbal control device provided in an embodiment of this application. The gimbal control device is applied to a first electronic device, which is communicatively connected to a first gimbal device, a second electronic device, and a second gimbal device. Figure 14 As shown, the gimbal control device 500 includes a processing module 501 and a transmission module 502.

[0179] The aforementioned processing module 501 is used to acquire first interaction information, which includes at least one of the following: user emotion information and user behavior information.

[0180] Processing module 501 is also used to generate a first gimbal control command based on the first interactive information;

[0181] The sending module 502 is used to send a first gimbal control command to the second electronic device. The first gimbal control command is used to control the second gimbal device to perform movement.

[0182] In other embodiments, the first interaction information mentioned above includes user emotion information; the processing module 501 is specifically used for:

[0183] The camera of the control unit 500 collects user facial expression information and user posture information, and performs intent recognition based on the user facial expression information and user posture information to obtain user emotion information;

[0184] or,

[0185] If a first expression image is obtained, a first emotion tag associated with the first expression image is obtained based on the first expression image. The first emotion tag is used to represent the user's emotional information.

[0186] The user emotion information includes the user's emotion type and the intensity of the user's emotion.

[0187] In other embodiments, the first interactive information mentioned above includes user emotion information, and the first gimbal control command includes at least one of the following: a first gimbal motion command and a haptic feedback command;

[0188] Processing module 501 is specifically used to implement at least one of the following:

[0189] Obtain motion trajectory information associated with user emotional information, and generate the first gimbal motion command based on the motion trajectory information;

[0190] It acquires vibration pattern information associated with the user's emotional information and generates haptic feedback commands based on the vibration pattern information.

[0191] In other embodiments, the processing module 501 described above is specifically used to implement at least one of the following:

[0192] Obtain a second emotion tag that matches the user's emotional information, and determine the motion trajectory information associated with the second emotion tag;

[0193] Motion trajectory information is generated using action templates that are associated with the user's emotional information.

[0194] In other embodiments, the first interaction information mentioned above includes user behavior information; the processing module 501 is specifically used for:

[0195] User behavior information is mapped to gimbal motion parameters, and the first gimbal control command is generated based on the gimbal motion parameters.

[0196] The user behavior information includes at least one of the following: user head movement information, user gaze information, and motion trajectory information when the user operates the first gimbal device.

[0197] In some other embodiments, the gimbal control device 500 further includes: a display module; the display module is used to display a gimbal interaction window before the processing module 501 obtains the first interaction information, the gimbal interaction window including at least one of the following: an emotion interaction authorization control and a behavior interaction authorization control;

[0198] The sending module 502 is also used to send a gimbal interaction authorization request to the second electronic device based on the confirmation input of the target authorization control, wherein the target authorization control includes at least one of an emotion interaction authorization control and a behavior interaction authorization control;

[0199] The processing module 501 is also used to control the PTZ control device 500 to establish a PTZ control channel with the second electronic device after receiving the authorization consent message from the second electronic device. The PTZ control channel is used to transmit the aforementioned first PTZ control command.

[0200] In some other embodiments, the gimbal control device 500 further includes a receiving module;

[0201] The receiving module is used to receive the second gimbal control commands sent by the second electronic device;

[0202] The processing module 501 is also used to control the first gimbal device to perform motion based on the second gimbal control command.

[0203] This application provides a gimbal control device. When the gimbal control device is communicatively connected to a second electronic device, it can convert user emotional information or user behavior information into corresponding control commands. These commands then enable the second electronic device to control the connected second gimbal device to perform corresponding movements. In other words, this solution, through the combination of electronic devices and gimbal devices, can intuitively reflect user emotions or behaviors through the physical movement of the gimbal, thus meeting the user's information interaction needs during video calls. This improves the flexibility of information interaction between electronic devices in remote video call scenarios.

[0204] The gimbal control device in this application embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, handheld camera, aircraft, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application embodiment does not specifically limit the device.

[0205] The gimbal control device in this application embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0206] The gimbal control device provided in this application embodiment can realize all the processes implemented in the above gimbal control method embodiment, and will not be described again here to avoid repetition.

[0207] Optionally, such as Figure 15As shown, this application embodiment also provides an electronic device 1500, including a processor 1501 and a memory 1502. The memory 1502 stores a program or instructions that can run on the processor 1501. When the program or instructions are executed by the processor 1501, they implement the various steps of the above-described gimbal control method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0208] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0209] Figure 16 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0210] The electronic device 1600 includes, but is not limited to, components such as: radio frequency unit 1601, network module 1602, audio output unit 1603, input unit 1604, sensor 1605, display unit 1606, user input unit 1607, interface unit 1608, memory 1609, and processor 1610.

[0211] Those skilled in the art will understand that the electronic device 1600 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1610 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 16 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0212] The processor 1610 is used to acquire first interaction information, which includes at least one of the following: user emotion information and user behavior information.

[0213] The processor 1610 is also used to generate a first gimbal control command based on the first interactive information;

[0214] The radio frequency unit 1601 is used to send a first gimbal control command to the second electronic device. The first gimbal control command is used to control the second gimbal device to perform movement.

[0215] In other embodiments, the first interaction information mentioned above includes user emotion information; the processor 1610 is specifically used for:

[0216] The camera of the first electronic device is used to collect user facial expression information and user posture information, and intention recognition is performed based on the user facial expression information and user posture information to obtain user emotion information;

[0217] or,

[0218] If a first expression image is obtained, a first emotion tag associated with the first expression image is obtained based on the first expression image. The first emotion tag is used to represent the user's emotional information.

[0219] The user emotion information includes the user's emotion type and the intensity of the user's emotion.

[0220] In other embodiments, the first interactive information includes user emotion information, and the first gimbal control command includes at least one of the following: a first gimbal motion command and a haptic feedback command; the processor 1610 is specifically used to implement at least one of the following:

[0221] Obtain motion trajectory information associated with user emotional information, and generate the first gimbal motion command based on the motion trajectory information;

[0222] It acquires vibration pattern information associated with the user's emotional information and generates haptic feedback commands based on the vibration pattern information.

[0223] In other embodiments, the processor 1610 described above is specifically used to implement at least one of the following:

[0224] Obtain a second emotion tag that matches the user's emotional information, and determine the motion trajectory information associated with the second emotion tag;

[0225] Motion trajectory information is generated using action templates that are associated with the user's emotional information.

[0226] In other embodiments, the first interaction information mentioned above includes user behavior information; the processor 1610 is specifically used for:

[0227] User behavior information is mapped to gimbal motion parameters, and the first gimbal control command is generated based on the gimbal motion parameters.

[0228] The user behavior information includes at least one of the following: user head movement information, user gaze information, and motion trajectory information when the user operates the first gimbal device.

[0229] In other embodiments, the display unit 1606 is used to display a gimbal interaction window before the processor 1610 obtains the first interaction information. The gimbal interaction window includes at least one of the following: an emotion interaction authorization control and a behavior interaction authorization control.

[0230] The radio frequency unit 1601 is also used to send a gimbal interaction authorization request to a second electronic device based on the confirmation input of the target authorization control, wherein the target authorization control includes at least one of an emotion interaction authorization control and a behavior interaction authorization control;

[0231] The processor 1610 is also configured to, upon receiving an authorization consent message from the second electronic device, control the first electronic device to establish a gimbal control channel with the second electronic device, the gimbal control channel being used to transmit the aforementioned first gimbal control command.

[0232] In other embodiments, the radio frequency unit 1601 described above is also used to receive a second gimbal control command sent by a second electronic device;

[0233] The processor 1610 is also used to control the movement of the first gimbal device based on the second gimbal control commands.

[0234] This application provides an electronic device in which a first electronic device, when communicatively connected to a second electronic device, can convert user emotional information or user behavior information into corresponding control commands. These control commands then enable the second electronic device to control a second pan-tilt-zoom (PTZ) device connected to the device to perform the movement corresponding to the motion information. In other words, this solution, through the combination of an electronic device and a PTG device, can intuitively reflect user emotions or behavior through the physical movement of the PTG device, thus meeting the user's information interaction needs during video calls. This improves the flexibility of information interaction between electronic devices in remote video call scenarios.

[0235] The electronic device provided in this application embodiment can realize all the processes implemented in the above-described gimbal control method embodiment. To avoid repetition, it will not be described again here.

[0236] It should be understood that, in this embodiment, the input unit 1604 may include a graphics processing unit (GPU) 16041 and a microphone 16042. The GPU 16041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1606 may include a display panel 16061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1607 includes at least one of a touch panel 16071 and other input devices 16072. The touch panel 16071 is also called a touch screen. The touch panel 16071 may include a touch detection device and a touch controller. Other input devices 16072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0237] The memory 1609 can be used to store software programs and various data. The memory 1609 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1609 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1609 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0238] Processor 1610 may include one or more processing units; optionally, processor 1610 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 1610.

[0239] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described gimbal control method embodiments and achieve the same technical effects. To avoid repetition, these will not be described again here.

[0240] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0241] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described gimbal control method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0242] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0243] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described gimbal control method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0244] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0245] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0246] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. A gimbal control method, characterized in that, Performed by a first electronic device, the first electronic device being communicatively connected to a first gimbal device, the first electronic device being communicatively connected to a second electronic device, and the second electronic device being communicatively connected to a second gimbal device, the method includes: Obtain first interaction information, which includes at least one of the following: user emotion information, user behavior information; A first gimbal control command is generated based on the first interactive information; The first gimbal control command is sent to the second electronic device, and the first gimbal control command is used to control the movement of the second gimbal device.

2. The method according to claim 1, characterized in that, The first interactive information includes user emotion information; The acquisition of the first interaction information includes: The user's facial expression information and posture information are collected by the camera of the first electronic device, and the user's emotional information is obtained by performing intent recognition based on the user's facial expression information and posture information. or, If a first expression image is obtained, a first emotion tag associated with the first expression image is obtained based on the first expression image, and the first emotion tag is used to characterize the user's emotion information; The user emotion information includes the user emotion type and the user emotion intensity.

3. The method according to claim 1, characterized in that, The first interactive information includes user emotion information, and the first gimbal control command includes at least one of the following: first gimbal motion command, haptic feedback command; The generation of the first gimbal control command based on the first interaction information includes at least one of the following: Obtain motion trajectory information associated with the user's emotional information, and generate the first gimbal motion command based on the motion trajectory information; Obtain vibration pattern information associated with the user's emotional information, and generate the tactile feedback command based on the vibration pattern information.

4. The method according to claim 3, characterized in that, The acquisition of motion trajectory information associated with the user's emotional information includes at least one of the following: Obtain a second emotion tag that matches the user's emotion information, and determine the motion trajectory information associated with the second emotion tag; The motion trajectory information is generated using an action template associated with the user's emotional information.

5. The method according to claim 1, characterized in that, The first interaction information includes user behavior information; The step of generating the first gimbal control command based on the first interaction information includes: The user behavior information is mapped to gimbal motion parameters, and the first gimbal control command is generated based on the gimbal motion parameters. The user behavior information includes at least one of the following: user head movement information, user gaze information, and motion trajectory information when the user operates the first gimbal device.

6. The method according to claim 1, characterized in that, Before obtaining the first interaction information, the method further includes: Display a gimbal interaction window, which includes at least one of the following: an emotion interaction authorization control and a behavior interaction authorization control; Based on the confirmation input of the target authorization control, a gimbal interaction authorization request is sent to the second electronic device, wherein the target authorization control includes at least one of the emotion interaction authorization control and the behavior interaction authorization control; Upon receiving the authorization and consent message from the second electronic device, the system controls the first electronic device to establish a gimbal control channel with the second electronic device. The gimbal control channel is used to transmit the first gimbal control command.

7. The method according to any one of claims 1-6, characterized in that, The method further includes: Receive the second gimbal control command sent by the second electronic device; Based on the second gimbal control command, the movement of the first gimbal device is controlled.

8. A gimbal control device, characterized in that, The device is applied to a first electronic device, which is communicatively connected to a first gimbal device, and is also communicatively connected to a second electronic device and a second gimbal device. The device includes a processing module and a transmitting module. The processing module is used to acquire first interaction information, which includes at least one of the following: user emotion information and user behavior information; The processing module is also used to generate a first gimbal control command based on the first interactive information; The sending module is used to send the first gimbal control command to the second electronic device, and the first gimbal control command is used to control the movement of the second gimbal device.

9. The apparatus according to claim 8, characterized in that, The first interactive information includes user emotion information; The processing module is specifically used for: The camera of the gimbal control device is used to collect user facial expression information and user posture information, and intention recognition is performed based on the user facial expression information and user posture information to obtain the user emotion information; or, If a first expression image is obtained, a first emotion tag associated with the first expression image is obtained based on the first expression image, and the first emotion tag is used to characterize the user's emotion information; The user emotion information includes the user emotion type and the user emotion intensity.

10. The apparatus according to claim 8, characterized in that, The first interactive information includes user emotion information, and the first gimbal control command includes at least one of the following: first gimbal motion command, haptic feedback command; The processing module is specifically used to implement at least one of the following: Obtain motion trajectory information associated with the user's emotional information, and generate the first gimbal motion command based on the motion trajectory information; Obtain vibration pattern information associated with the user's emotional information, and generate the tactile feedback command based on the vibration pattern information.

11. The apparatus according to claim 10, characterized in that, The processing module is specifically used to implement at least one of the following: Obtain a second emotion tag that matches the user's emotion information, and determine the motion trajectory information associated with the second emotion tag; The motion trajectory information is generated using an action template associated with the user's emotional information.

12. The apparatus according to claim 8, characterized in that, The first interaction information includes user behavior information; The processing module is specifically used for: The user behavior information is mapped to gimbal motion parameters, and the first gimbal control command is generated based on the gimbal motion parameters. The user behavior information includes at least one of the following: user head movement information, user gaze information, and motion trajectory information when the user operates the first gimbal device.

13. The apparatus according to claim 8, characterized in that, The device further includes: a display module; The display module is used to display a gimbal interaction window before the processing module obtains the first interaction information. The gimbal interaction window includes at least one of the following: an emotion interaction authorization control and a behavior interaction authorization control. The sending module is further configured to send a gimbal interaction authorization request to the second electronic device based on the confirmation input of the target authorization control, wherein the target authorization control includes at least one of the emotion interaction authorization control and the behavior interaction authorization control; The processing module is further configured to, upon receiving the authorization consent message from the second electronic device, control the gimbal control device to establish a gimbal control channel with the second electronic device, the gimbal control channel being used to transmit the first gimbal control command.

14. The apparatus according to any one of claims 8-13, characterized in that, The device further includes: a receiving module; The receiving module is used to receive the second gimbal control command sent by the second electronic device; The processing module is also used to control the movement of the first gimbal device based on the second gimbal control command.

15. An electronic device, characterized in that, It includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the gimbal control method as described in any one of claims 1 to 7.

Images