Robot control method and device, mobile terminal and storage medium

Abstract

The application is suitable for the field of robot technology, and provides a robot control method, a mobile terminal and a computer readable storage medium. The robot control method is applied to the mobile terminal, the mobile terminal is in communication connection with a robot, the mobile terminal comprises a touch screen, a display interface of the touch screen comprises a robot control area, and the method comprises the following steps: detecting a touch gesture input through the touch screen; calculating a control parameter of the robot based on a type of the touch gesture, a position set of the touch gesture and a position relationship between the control area, the control parameter comprising at least one of a speed, a rotation angle, a target angle, a target position and a displacement; and sending a control instruction comprising the control parameter to the robot, the control instruction being used for instructing the robot to move according to the control parameter.

Description

Technical Field

[0001] This application belongs to the field of robotics technology, and in particular relates to a robot control method and device, a mobile terminal and a computer-readable storage medium. Background Technology

[0002] Robots can use navigation systems to locate themselves, plan their own paths, and move point-to-point. However, to achieve these functions, robots need to first acquire a location map of the work area, a process known as robot mapping.

[0003] Because robot movement is highly dependent on maps, but if robots were to move autonomously to create maps, they would need to navigate areas where maps have not yet been created, easily leading to a chicken-and-egg dilemma. Therefore, workers often push the robots around the work area to complete the mapping. While this method avoids the aforementioned dilemma, it is cumbersome and inefficient. Summary of the Invention

[0004] This application provides a robot control method and device, a mobile terminal and a computer-readable storage medium, which can solve the problem of cumbersome and inefficient manual robot mapping operations in related technologies.

[0005] In a first aspect, embodiments of this application provide a robot control method. This method is applied to a mobile terminal, which is communicatively connected to a robot. The mobile terminal includes a touchscreen, and the touchscreen's display interface includes a robot control area. The method includes: detecting a touch gesture input via the touchscreen; calculating robot control parameters based on the type of the touch gesture, its location set, and its positional relationship with the control area; the control parameters including at least one of speed, rotation angle, target angle, target position, and displacement; and sending a control command containing the control parameters to the robot, the control command instructing the robot to move according to the control parameters.

[0006] Secondly, embodiments of this application provide a robot control device. This device is communicatively connected to a robot and includes a touchscreen. The touchscreen's display interface includes a robot control area. The device includes: a detection module for detecting touch gestures input via the touchscreen; a calculation module for calculating robot control parameters based on the type of the touch gesture, its location set, and its positional relationship with the control area. The control parameters include at least one of speed, rotation angle, target angle, target position, and displacement; and a sending module for sending control instructions containing the control parameters to the robot. The control instructions instruct the robot to move according to the control parameters.

[0007] Thirdly, embodiments of this application provide a mobile terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the robot control method described in the first aspect above.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the robot control method described in the first aspect above.

[0009] Fifthly, embodiments of this application provide a computer program product that, when run on a mobile terminal, causes the mobile terminal to execute the robot control method described in the first aspect.

[0010] The beneficial effects of this application embodiment compared with the prior art are: during the mapping process, the robot is controlled by touch gestures input through the touch screen of the mobile terminal. On the one hand, it eliminates the process of personnel pushing the robot to move, saving effort and improving efficiency. On the other hand, compared with inputting the movement direction and clicking the up, down, left, and right buttons on the terminal to control the robot's movement, the use of touch gestures can control the robot's movement more efficiently and with greater flexibility and freedom. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the structure of a mobile phone to which the robot control method provided in this application embodiment is applicable;

[0013] Figure 2 This is a schematic diagram of the structure of the robot provided in the embodiments of this application;

[0014] Figure 3 This is a flowchart illustrating the robot control method provided in an embodiment of this application;

[0015] Figure 4 This is a schematic diagram of the display interface provided in an embodiment of this application;

[0016] Figure 5 This is a schematic diagram of the robot control device provided in the embodiments of this application. Detailed Implementation

[0017] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0018] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0019] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0020] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0021] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0022] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0023] The robot control method provided in this application can be applied to mobile terminals such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of mobile terminal.

[0024] For example, the mobile terminal may be a station (STAION, ST) in a WLAN, a cellular phone, a cordless phone, a Session Initiation Protocol (SIP) phone, a Wireless Local Loop (WLL) station, a Personal Digital Assistant (PDA) device, a handheld device with wireless communication capabilities, a computing device or other processing device connected to a wireless modem, an in-vehicle device, a vehicle networking terminal, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a wireless modem card, a set-top box (STB), customer premises equipment (CPE), and / or other devices used for communication over a wireless system, as well as next-generation communication systems, such as mobile terminals in 5G networks or mobile terminals in future evolved Public Land Mobile Network (PLMN) networks.

[0025] As an example, and not a limitation, when a mobile terminal is a wearable device, the term "wearable device" can also refer to any device that utilizes wearable technology to intelligently design and develop everyday wearables, such as glasses, gloves, watches, clothing, and shoes. Wearable devices are portable devices worn directly on the body or integrated into a user's clothing or accessories. Wearable devices are not merely hardware devices; they achieve powerful functions through software support, data interaction, and cloud interaction. Broadly defined, wearable smart devices include those with comprehensive functions, large sizes, and the ability to perform complete or partial functions without relying on a smartphone, such as smartwatches or smart glasses, as well as those focused on a specific application function that require interaction with other devices like smartphones, such as various smart bracelets and smart jewelry for vital sign monitoring.

[0026] Take a mobile phone as an example. Figure 1This is a block diagram illustrating a portion of the structure of a mobile phone according to an embodiment of this application. (Reference) Figure 1 The mobile phone includes components such as a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, a wireless fidelity (WiFi) module 170, a processor 180, and a power supply 190. Those skilled in the art will understand that... Figure 1 The mobile phone structure shown does not constitute a limitation on the mobile phone and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0027] The following is combined with Figure 1 A detailed introduction to each component of a mobile phone:

[0028] RF circuit 110 can be used for receiving and transmitting signals during information transmission or calls. Specifically, it receives downlink information from the base station and processes it with processor 180; additionally, it transmits uplink data to the base station. Typically, RF circuitry includes, but is not limited to, antennas, at least one amplifier, transceiver, coupler, low-noise amplifier (LNA), duplexer, etc. Furthermore, RF circuit 110 can also communicate wirelessly with networks and other devices. The aforementioned wireless communication can use any communication standard or protocol, including but not limited to Global System for Mobile Communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0029] The memory 120 can be used to store computer programs, and the processor 180 executes various functions and data processing of the mobile phone by running the computer programs stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the mobile phone (such as audio data, phonebook, etc.). In addition, the memory 120 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0030] The input unit 130 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the mobile phone 100. Specifically, the input unit 130 may include a touch screen 131 and other input devices 132. The touch screen 131, also known as a touchscreen, can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch screen 131), and drive the corresponding connection devices according to a pre-set program. Optionally, the touch screen 131 may include two parts: a touch detection device and a touch controller. The touch detection device detects the user's touch position and the signal generated by the touch operation, and transmits the signal to the touch controller; the touch controller receives touch information from the touch detection device, converts it into touch point coordinates, sends it to the processor 180, and can receive and execute commands sent by the processor 180. In addition, the touch screen 131 can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touchscreen 131, the input unit 130 may also include other input devices 132. Specifically, other input devices 132 may include, but are not limited to, one or more of the following: physical keyboard, function keys (such as volume control buttons, power buttons, etc.), trackball, mouse, joystick, etc.

[0031] Display unit 140 can be used to display information input by the user or information provided to the user, as well as various menus of the mobile phone. Display unit 140 may include a display panel 141, optionally configured as a Liquid Crystal Display (LCD), Organic Light-Emitting Diode (OLED), or similar display panel 141. Furthermore, touchscreen 131 may cover display panel 141. When touchscreen 131 detects a touch operation on or near it, it transmits the information to processor 180 to determine the type of touch event. Subsequently, processor 180 provides corresponding visual output on display panel 141 based on the type of touch event. Although in Figure 1 In this embodiment, the touch screen 131 and the display panel 141 are two separate components to realize the input and output functions of the mobile phone. However, in some embodiments, the touch screen 131 and the display panel 141 can be integrated to realize the input and output functions of the mobile phone.

[0032] The mobile phone 100 may also include at least one sensor 150, such as a light sensor, a motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display panel 141 according to the ambient light level, and the proximity sensor can turn off the display panel 141 and / or backlight when the phone is moved to the ear. As a type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in various directions (generally three axes). When stationary, it can detect the magnitude and direction of gravity and can be used for applications that recognize the phone's posture (such as landscape / portrait switching, related games, magnetometer posture calibration), vibration recognition-related functions (such as pedometer, tapping), etc. Other sensors that may be configured in the phone, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, will not be described in detail here.

[0033] Audio circuit 160, speaker 161, and microphone 162 provide an audio interface between the user and the mobile phone. Audio circuit 160 converts received audio data into electrical signals and transmits them to speaker 161, where speaker 161 converts them into sound signals for output. On the other hand, microphone 162 converts collected sound signals into electrical signals, which are received by audio circuit 160, converted into audio data, and then processed by processor 180 before being transmitted via RF circuit 110 to, for example, another mobile phone, or the audio data can be output to memory 120 for further processing.

[0034] WiFi is a short-range wireless transmission technology. Mobile phones, through their WiFi modules (170), can help users send and receive emails, browse web pages, and access streaming media, providing wireless broadband internet access. Although Figure 1 WiFi module 170 is shown, but it is understood that it is not a necessary component of mobile phone 100 and can be omitted as needed without changing the nature of the invention.

[0035] The processor 180 is the control center of the mobile phone, connecting various parts of the phone through various interfaces and lines. It performs various functions and processes data by running or executing computer programs stored in the memory 120 and calling data stored in the memory 120, thereby providing overall monitoring of the phone. Optionally, the processor 180 may include one or more processing units; preferably, the processor 180 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 180.

[0036] The mobile phone 100 also includes a power supply 190 (such as a battery) that supplies power to various components. Preferably, the power supply can be logically connected to the processor 180 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system.

[0037] Although not shown, the mobile phone 100 may also include a camera. Optionally, the camera may be positioned on the front or rear of the mobile phone 100, and this embodiment does not limit this.

[0038] Optionally, the mobile phone 100 may include a single camera, dual cameras, or triple cameras, etc., and this application embodiment does not limit this. For example, the mobile phone 100 may include three cameras, of which one is a main camera, one is a wide-angle camera, and one is a telephoto camera.

[0039] Optionally, when the mobile phone 100 includes multiple cameras, these multiple cameras can all be front-facing, all be rear-facing, or some can be front-facing and others rear-facing. This application embodiment does not limit this.

[0040] In addition, although not shown, the mobile phone 100 may also include a Bluetooth module, etc., which will not be described in detail here.

[0041] The mobile phone 100 can communicate with the robot to be controlled through at least one of the following communication circuits: radio frequency circuit 110, WiFi module 170, Bluetooth module, infrared module, etc.

[0042] Figure 2 The diagram shown is a block diagram of a portion of the structure of the robot provided in an embodiment of this application.

[0043] Please refer to Figure 2The robot includes: a shell 20 for carrying objects, a mobile chassis 10, a function controller for providing user operation, a low-level controller for map generation and path planning, and a component controller for controlling the movement unit and the environment detection unit; the mobile chassis is provided with at least two sets of drive wheels 121, each set of drive wheels 121 is located on one side of the mobile chassis 10; the component controller controls the travel speed of the drive wheels 121.

[0044] The chassis 10 has at least one turn signal unit (not shown in the figure) at the bottom, and each turn signal unit includes at least one turn signal 111; the moving unit is provided with at least two sets of drive wheels 121, each set of drive wheels is located on one side of the chassis 10; the component controller controls the traveling speed of the drive wheels 121; and controls the turn signals in the turn signal unit to light up in a preset manner when the robot turns.

[0045] Specifically, among the drive wheels 121 configured in the mobile unit, at least one set of drive wheels 121 serves as the left drive wheel, and at least one set of drive wheels 121 serves as the right drive wheel, with the left and right drive wheels located on opposite sides of the chassis 10. Optionally, the mobile unit may also include at least two sets of driven wheels, one set of drive wheels corresponding to one set of driven wheels, wherein at least one set of driven wheels serves as the left driven wheel, and at least one set of driven wheels serves as the right driven wheel. The left and right driven wheels assist the left and right drive wheels in driving the robot's housing 20 and chassis 10, reducing the load pressure on the drive wheels 121.

[0046] The robot provided in this embodiment of the invention illuminates the turn signal 111 via a component controller when the robot turns, in order to alert pedestrians.

[0047] Based on the above technical solution, optionally, when the speed difference between the drive wheels 121 on both sides of the chassis 10 is greater than a preset value, the component controller controls the turn signal 111 in the turn signal unit to light up in a preset manner.

[0048] Optionally, the robot also includes a voice module, which is electrically connected to the component controller; the component controller controls the voice module to issue voice prompts when the robot turns.

[0049] The lidar system can be installed in a robot. The lidar system can rotate along a set plane, so that the photoelectric receiving array of the lidar system forms a scanning cylinder.

[0050] The lidar system can be installed at the opening of the robot housing 20, making it easy to emit laser signals to detect surrounding objects. The lidar system includes a photoelectric receiving array and a laser emitting unit array. When the lidar system rotates along a set plane, the photoelectric receiving array forms a scanning cylinder, increasing the scanning area and facilitating the acquisition of detailed object shapes, thus preventing the robot from bumping into obstacles. If the lidar system only contains a single photoelectric receiving unit and a single laser emitting unit, it can only measure the shape of an object within a single circle after rotating along the set plane, failing to acquire the shapes of complex objects in a timely manner, increasing the risk of collisions and endangering personal safety and property. Optionally, the set plane can be a horizontal plane, facilitating object detection during robot movement. Furthermore, other set planes, such as a vertical plane, can be selected according to user needs; this embodiment does not limit this selection.

[0051] Figure 3 A schematic flowchart of a robot control method provided in an embodiment of this application is shown, which can be applied to the aforementioned mobile terminal.

[0052] S1: Detects touch gestures input via the touchscreen.

[0053] The touchscreen can be stacked vertically with the display panel, with the two largely or even completely overlapping, or the touchscreen can be integrated with the display panel. The touchscreen's display interface is the same as the display panel's display interface, including a robot control area. This control area can include at least one of an operation area and a map area. The map area is used to display reference maps for mapping, such as architectural drawings or floor plans. For example, ... Figure 4 As shown in the figure, the map portion displayed is map area 201, and the circle in the lower left corner is the operation area 202. In practical applications, the shape, size, and other parameters of the operation area 202 can be determined according to actual needs.

[0054] The touchscreen periodically detects and provides feedback on touch information, including: whether there is a touch point, and if so, the number, location, and pressure of the detected touch points. Each feedback of touch information can be called a frame of touch data. The coordinate system used for the touch point position in the touch data can be the touchscreen's own coordinate system. However, in practical applications, the touch point position often needs to be used in conjunction with the content on the display interface; therefore, the coordinate system used by the display panel is usually used to record the touch point position. Generally, touch precision is far lower than display precision; the resolution of touch points is typically at the millimeter level. The smallest area of ​​a single touch point may contain tens or even hundreds of pixels on the display panel. Therefore, when using the coordinate system used by the display panel to describe the touch point position, the position of a single touch point essentially corresponds to an area on the display panel, and the specific coordinates of this position can be selected from the corresponding area according to a set rule.

[0055] This embodiment mainly uses single-point touch gestures such as clicking, swiping, and dragging to control the robot. Clicking can include single-click and / or double-click. The main difference between swiping and dragging is whether the user presses during the initial stage. This is reflected in the touch data, which can be determined based on the duration and / or force of the initial position.

[0056] Touch gesture recognition requires multiple consecutive frames of touch point data. Many gestures are reflected in the touch point information after the touch ends; that is, the type of gesture can only be determined after the touch point has gone from being present to being absent. Therefore, the detected touch gesture actually includes the touch point positions of each frame throughout the process. The location of the touch gesture is usually described by a set of these touch point positions. The touch point positions in the location set are generally sorted by feedback time. Optionally, consecutive identical touch point positions in the location set can be merged. After merging, the set of clicked positions usually contains only a few, or even a single, touch point position.

[0057] S2: Calculate the robot's control parameters based on the type and location set of the touch gestures and their positional relationship with the control area.

[0058] Control parameters include at least one of speed, rotation angle, target angle, target position, and displacement. Optionally, the control mode can be determined based on the type of touch gesture, the set of positions, and the positional relationship between the gesture and the control area. Examples include speed control, displacement control, real-time position control, target position control, and rotation control. The positional relationship here generally includes the relative positional relationship between the starting position in the set of touch gesture positions and the control area. The specific control parameters are then calculated by further combining the set of positions.

[0059] If the touch gesture is drag or swipe, the control mode is generally selected based on whether the starting position is within the starting area of ​​the control area. Specifically, if the starting position is not within the starting area of ​​the control area, rotation control is generally selected. If the starting position is within the starting area of ​​the control area, speed control, displacement control, or real-time position control can be selected based on the type of control area.

[0060] For the operating area, its starting area is generally a small region including the center point, which can be highlighted in the display interface, for example... Figure 4 The solid circle 222 at the center of the operating area 202 represents the starting area of ​​the operating area 202. For the map area, the robot's position can be displayed in real time (i.e., the robot's icon indicates the actual operating robot), and its starting area is generally a small area including the robot's icon. For example, in... Figure 4 In the map area 201, marker 211 refers to the actual operating robot, and its position within the map area 201 indicates the robot's real-time position. The area 212 surrounding marker 211 is the starting area of ​​map area 201. The relative positional relationship between the edge of the starting area and marker 211 remains unchanged; that is, the starting area moves with marker 211, and marker 211 moves with the robot. The specific size of the starting area can be determined based on the detection accuracy of the touchscreen itself.

[0061] The following example illustrates the specific process of calculating robot control parameters based on drag or swipe gestures.

[0062] If the touch gesture is a drag or swipe, and the starting position in the touch gesture's position set is within the starting area of ​​the control region, real-time control of the robot's movement is generally chosen. Compared to related technologies that require inputting the movement direction and clicking up, down, left, or right buttons on the terminal, this improves the flexibility and freedom of robot control. Specifically, at least one of the robot's speed, displacement, and target position can be calculated based on the current position in the touch gesture's position set.

[0063] If the starting position is within the initial area of ​​the operating area, speed or displacement control can be selected, enabling real-time free control of the robot. Compared to related technologies that require inputting the movement direction and clicking up, down, left, and right buttons on the terminal, this method improves the flexibility and freedom of robot control. Here, speed and displacement are both vectors, including data in both direction and magnitude. The direction of the robot's speed or displacement can be determined based on the first direction pointing from the starting point of the operating area to the current position, and the magnitude of the robot's speed or displacement can be calculated based on the distance between the current position and the starting point of the operating area.

[0064] Specifically, if the coordinates of the starting point of the operation area are (x0, y0) and the coordinates of the current position are (x1, y1), then the direction of the vector (x1-x0, y1-y0) pointing from the starting point to the current position is the first direction, and the magnitude of this vector is the distance between the starting point and the current position.

[0065] You can set a maximum speed or a maximum single displacement for the robot. Calculate the product of the first ratio and the maximum speed as the robot's speed, or calculate the product of the first ratio and the maximum single displacement as the robot's displacement. The first ratio is the ratio of the distance between the starting point and the current position to the radius of the operating area. The radius of the operating area refers to the distance from the center point of the operating area to the boundary.

[0066] In some embodiments, the robot can be configured to move at a fixed step size / speed during the sliding or dragging process. In this case, the calculation process can be omitted, and the set value can be directly used as the speed or displacement of the robot.

[0067] Generally, the first direction can be represented by the angle of a vector (x1-x0, y1-y0). This first direction can be directly used as the direction of the robot's velocity or displacement. In practical applications, touch operations are often unstable, leading to operational errors. For example, when a user attempts to draw a straight line on a touchscreen with their finger, the resulting trajectory is often a multi-segmented broken line. If the first direction is directly used as the direction of the robot's velocity or displacement, the robot may make multiple unnecessary turns due to operational errors, which is detrimental to smooth robot control. To mitigate this, in some embodiments, a certain redundancy can be set for the direction of the robot's velocity or displacement to improve the stability of robot control. Two examples are given below to illustrate how redundancy works.

[0068] Optionally, the robot can receive the latest motion status information from the robot, including the robot's motion direction, calculate the directional deviation between the first direction and the motion direction, and if the absolute value of the directional deviation is greater than the set deviation threshold, then the first direction is used as the direction of the robot's velocity or displacement; otherwise, the motion direction is used as the direction of the robot's velocity or displacement, that is, the robot's current motion direction is not changed. Here, the deviation threshold serves as a redundancy.

[0069] Optionally, multiple direction intervals can be pre-defined, and the representative direction in each direction interval can be determined according to the pre-defined correspondence, such as selecting the center, upper limit, or lower limit of the direction interval. The direction interval to which the first direction belongs is determined, and the representative direction corresponding to the direction interval to which the first direction belongs replaces the first direction as the direction of the robot's velocity or displacement. For example, if the direction is represented by an angle, 0° to 360° can be divided into multiple direction intervals according to a set step size. The representative angle of each direction interval is the center of that direction interval, and the center angle of the direction interval to which the first direction belongs is used as the direction of the robot's velocity or displacement. During continuous sliding or dragging, as long as the first direction corresponding to the touch points of two adjacent frames belongs to the same direction interval, the robot's movement direction will not change; the direction interval here serves as redundancy.

[0070] If the starting position is within the initial area of ​​the map, real-time control of the robot to follow the touch gesture's trajectory is generally chosen. Compared to related technologies that require inputting movement direction and clicking up, down, left, and right buttons on the terminal, this improves the robot's control flexibility and freedom. Specifically, the current position can be transformed from the touchscreen coordinate system to the map coordinate system to obtain the robot's target position, and / or the first displacement from the previous position to the current position in the touch gesture's position can be calculated, and then the first displacement can be transformed from the touchscreen coordinate system to the map coordinate system to obtain the robot's displacement. This coordinate transformation includes not only common rotation and translation but also scaling. The specific scaling ratio can be determined by considering the map's scale, the map's scaling ratio on the display interface, and the resolution of the touchscreen's coordinate system.

[0071] After calculating the robot's target position, if the robot's current position can be obtained from the latest feedback motion state information, the robot's displacement can be calculated by subtracting the target position from the current position. After calculating the robot's displacement, if the robot's current position can be obtained from the latest feedback motion state information, the robot's target position can be calculated by summing the displacement and the current position.

[0072] If the touch gesture is a drag or swipe, and the starting position in the set of touch gesture positions is not within the starting area of ​​the control region, rotation control can be performed. This provides a control method different from common speed and position control, thereby improving the flexibility of robot control. Rotation control does not have to be real-time; instead, it waits for the touch gesture to end, and then calculates at least one of the robot's rotation angle and target angle based on the trajectory formed by the set of touch gesture positions. The object of rotation here can be the robot's chassis or its shell, which can be set according to actual needs.

[0073] Specifically, after the touch gesture ends—that is, after the touch point disappears and remains contactless for a period of time—a line segment can be fitted to the trajectory. If the fitted trajectory is a single line segment, the direction of the fitted result, i.e., the direction from the starting point to the ending point of the fitted line segment, is calculated as the robot's target angle. If the fitted trajectory is two line segments, the angle between the two line segments can be calculated as the robot's rotation angle, or the direction of the angle bisector can be calculated as the robot's target angle, i.e., the robot's direction is determined by drawing an arrow.

[0074] Based on the target angle calculated directly from the trajectory fitting results, if the current angle of the rotating object can be obtained from the robot's latest motion state information, the difference between the target angle and the current angle can be calculated as the rotation angle. Alternatively, based on the rotation angle calculated directly from the trajectory fitting results, if the current angle of the rotating object can be obtained from the robot's latest motion state information, the sum of the rotation angle and the current angle can be calculated as the target angle.

[0075] For click-type touch gestures, all touch point positions in the position set can be fitted to a single touch point position. The position set of touch gestures can be simply referred to as the touch gesture position. The position of click-type touch gestures can be used as the target position of the robot for position control or displacement control, thereby simplifying control operations while increasing the robot's degree of freedom.

[0076] If the touch gesture is a click and the location is within the map area, the position of the touch gesture can be transformed from the touchscreen coordinate system to the map coordinate system to obtain the robot's target position. Based on this, if the robot's current position can be obtained from the latest feedback motion state information, the difference between the target position and the current position can be calculated to obtain the robot's displacement.

[0077] If the touch gesture is a click and the location is within the operation area, the displacement from the starting point of the operation area to the current position can be calculated. This displacement is then scaled according to a set ratio to obtain the robot's displacement. This ratio can be the ratio of a set maximum displacement to the radius of the operation area. Based on this, if the robot's current position can be obtained from its latest motion status information, the sum of the displacement and the current position can be calculated to obtain the robot's target position.

[0078] S3: Send control commands containing control parameters to the robot.

[0079] Control commands are used to instruct the robot to move according to control parameters. The robot can move according to the control parameters and model the movement process.

[0080] Specifically, the robot is equipped with data acquisition sensors and a modeling processor. The modeling processor uses environmental data collected by the sensors to build an environmental map. In this embodiment, the data acquisition sensors include LiDAR, ultrasonic sensors, and infrared sensors. These sensors collect data on the robot's working area. The modeling processor uses this data to create a map. During map creation, different map layers are generated using different sensors, such as static layers, dynamic obstacle layers, ultrasonic layers, and visual layers. These layers are then fused to obtain a positioning map for the robot's localization and navigation.

[0081] Furthermore, the robot can send the completed location map to the mobile terminal. This action can be performed in real time during the mapping process or after the complete mapping is finished. The mobile terminal can display the completed location map, for example, by merging it with a map area, that is, replacing the portion of the map area corresponding to the completed map with the location map.

[0082] Optionally, after the robot completes the mapping, the mobile terminal can check the integrity of the completed positioning map. If a gap is found in the positioning map, it means that there are missed parts. In this case, the mobile terminal can automatically or notify the robot to give a prompt that it needs to be scanned again and mark the gap area on the map.

[0083] Based on this, the mobile terminal can respond to user commands, modifying the location map, adding points, and other operations, which are then synchronized to the robot. A point refers to a target location where the robot may stop during normal operation; for example, a food delivery robot's points could include the table location. Points can be added without controlling the robot's movement, or the robot can be controlled to move to a point and stop before adding the point.

[0084] Optionally, after receiving a motion command, the robot can first combine the data collected by its sensors to determine whether the control parameters in the motion command are reasonable. If they are unreasonable, such as moving towards an obstacle, the robot can refuse to execute the motion command and provide a prompt or notify the mobile terminal to provide a prompt. During actual operation, the robot can make appropriate adjustments to the motion commands to improve operational stability. For example, when making a sharp turn, the robot can reduce its speed to reduce the possibility of tipping over, and automatically avoid obstacles when it detects them.

[0085] Through the implementation of this embodiment, the robot is controlled by touch gestures input through the touch screen of the mobile terminal during the mapping process. On the one hand, it eliminates the need for personnel to push the robot, saving effort and improving efficiency. On the other hand, compared with inputting the movement direction and clicking the up, down, left, and right buttons on the terminal to control the robot's movement, touch gestures can control the robot's movement more efficiently and with greater flexibility and freedom.

[0086] Furthermore, the inventors of this application discovered during their long-term research and development that when workers push the robot to create a map, the robot often tilts relative to the ground, a situation that generally does not occur when the robot moves autonomously. This leads to deviations in the created positioning map, affecting the robot's subsequent positioning and path planning. The implementation of this embodiment eliminates the need for personnel to push the robot and avoids positioning map deviations caused by tilting.

[0087] Figure 5 A schematic diagram of the structure of a robot control device provided in an embodiment of this application is shown. The device is communicatively connected to a robot and includes a touch screen. The display interface of the touch screen includes a robot control area. The robot control device includes a detection module 11, a calculation module 12, and a transmission module 13.

[0088] The detection module 11 is used to detect touch gestures input via the touch screen.

[0089] The calculation module 12 is used to calculate the robot's control parameters based on the type of touch gesture, the set of positions, and the positional relationship between the gesture and the control area. The control parameters include at least one of speed, rotation angle, target angle, target position, and displacement.

[0090] The sending module 13 is used to send control commands containing control parameters to the robot. The control commands are used to instruct the robot to move according to the control parameters.

[0091] It should be noted that the information interaction and execution process between the above-mentioned devices / modules / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0092] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0093] The robot control method provided in this application can be implemented as a computer software program. For example, an embodiment of this application provides a computer program product including a computer program carried on a computer-readable medium, the computer program containing program code for performing the method shown in the flowchart.

[0094] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0095] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0096] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographic device / electronic device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0097] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0098] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0099] In the embodiments provided in this application, it should be understood that the disclosed apparatus / network devices and methods can be implemented in other ways. For example, the apparatus / network device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0100] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0101] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A robot control method, characterized in that, The method is applied to a mobile terminal, which is communicatively connected to the robot. The mobile terminal includes a touchscreen, and the touchscreen's display interface includes a robot control area. Detect touch gestures input via the touchscreen; The control parameters of the robot are calculated based on the type and location set of the touch gesture and its positional relationship with the control area. The control parameters include at least one of speed, rotation angle, target angle, target position, and displacement. Send a control command containing the control parameters to the robot, the control command being used to instruct the robot to move according to the control parameters; The calculation of the robot's control parameters based on the type and location set of the touch gesture and its positional relationship with the control area includes: If the type of the touch gesture is drag or swipe, and the starting position in the set of positions of the touch gesture is within the starting area of ​​the control area, then at least one of the robot's speed, displacement, and target position is calculated based on the current position in the set of positions of the touch gesture. If the type of the touch gesture is drag or swipe, and the starting position in the position set of the touch gesture is not within the starting area of ​​the control area, then after the touch gesture ends, at least one of the robot's rotation angle and target angle is calculated based on the trajectory formed by the position set of the touch gesture.

2. The method as described in claim 1, characterized in that, The control area includes at least one of an operation area and a map area, and the calculation of at least one of the robot's speed, displacement, and target position based on the current position of the touch gesture includes: If the starting position is within the starting area of ​​the operating area, the direction of the robot's speed or displacement is determined based on the first direction from the starting point of the operating area to the current position, and the magnitude of the robot's speed or displacement is calculated based on the distance between the current position and the starting point of the operating area. If the starting position is within the starting area of ​​the map region, the current position is transformed from the coordinate system of the touch screen to the map coordinate system to obtain the target position of the robot, and / or the first displacement from the previous position to the current position in the position of the touch gesture is calculated, and then the first displacement is transformed from the coordinate system of the touch screen to the map coordinate system to obtain the displacement of the robot.

3. The method as described in claim 2, characterized in that, The method of determining the direction of the robot's velocity or displacement based on the first direction pointing from the starting point of the operating area to the current position includes: Calculate the directional deviation between the first direction and the motion direction from the motion state information of the robot; If the absolute value of the directional deviation is greater than the set deviation threshold, then the first direction is taken as the direction of the robot's velocity or displacement; otherwise, the direction of motion is taken as the direction of the robot's velocity or displacement.

4. The method as described in claim 2, characterized in that, The method of determining the direction of the robot's velocity or displacement based on the first direction pointing from the starting point of the operating area to the current position includes: Determine the direction interval to which the first direction belongs; According to the preset direction interval and its corresponding representative direction, the representative direction corresponding to the direction interval to which the first direction belongs is determined as the direction of the robot's velocity or displacement.

5. The method as described in claim 1, characterized in that, The calculation of at least one of the robot's rotation angle and target angle based on the trajectory formed by the set of positions of the touch gestures includes: Perform line segment fitting on the trajectory; If the fitting result of the trajectory is a single line segment, then the direction of the fitting result is calculated as the target angle of the robot; If the fitting result of the trajectory is two line segments, then the included angle between the two line segments is calculated as the rotation angle of the robot, or the direction of the angle bisector is calculated as the target angle of the robot.

6. The method as described in claim 1, characterized in that, The control area includes at least one of an operation area and a map area, and the calculation of the robot's control parameters based on the type of the touch gesture, the set of locations, and the positional relationship with the control area includes: If the type of the touch gesture is a click and the location is within the map area, then the location of the touch gesture is transformed from the coordinate system of the touch screen to the map coordinate system to obtain the target location of the robot; If the type of the touch gesture is a click and the position is within the operation area, then the displacement from the starting point of the operation area to the position is calculated, and then the displacement of the robot is obtained by scaling according to a set ratio.

7. A robot control device, characterized in that, The device is communicatively connected to the robot and includes a touchscreen. The touchscreen's display interface includes a robot control area. The device includes: The detection module is used to detect touch gestures input via the touchscreen; The calculation module is used to calculate the robot's control parameters based on the type of the touch gesture, the set of positions, and the positional relationship between the touch gesture and the control area. The control parameters include at least one of speed, rotation angle, target angle, target position, and displacement. The calculation includes: if the type of the touch gesture is dragging or sliding, and the starting position in the set of positions of the touch gesture is within the starting area of ​​the control area, then calculating at least one of the robot's speed, displacement, and target position based on the current position in the set of positions of the touch gesture; if the type of the touch gesture is dragging or sliding, and the starting position in the set of positions of the touch gesture is not within the starting area of ​​the control area, then waiting for the touch gesture to end before calculating at least one of the robot's rotation angle and target angle based on the trajectory formed by the set of positions of the touch gesture. The sending module is used to send control instructions containing the control parameters to the robot, the control instructions being used to instruct the robot to move according to the control parameters.

8. A mobile terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images