Intelligent control method, mobile terminal, self-moving robot and intelligent control system

Abstract

The invention relates to an intelligent control method which is applied to a mobile terminal and comprises the following steps: displaying a first interface in response to a received fault signal sent by a self-moving robot; displaying a second interface in response to the received first operation of the user for the first interface; and in response to the received second operation of the user for the second interface, sending a control instruction to the self-moving robot, so that the self-moving robot moves based on the control instruction. According to the method, the user can remotely check the fault state and the surrounding environment of the self-moving robot through the mobile terminal, so that the self-moving robot can be further remotely controlled to remove the fault based on the fault state and the surrounding environment, the intelligent degree is high, and the user experience feeling is good.

Description

Technical Field

[0001] This invention relates to the field of self-moving robots, and more specifically, to an intelligent control method, a mobile terminal, a self-moving robot, and an intelligent control system. Background Technology

[0002] Self-moving robots (such as lawnmowers) are prone to malfunctions during operation, and when a self-moving robot malfunctions, it cannot move freely.

[0003] In existing technologies, users need to manually resolve the fault at the location of the self-moving robot; if users cannot reach the location of the self-moving robot in time to manually resolve the fault, the self-moving robot will remain unable to move, resulting in low work efficiency and a poor user experience. Summary of the Invention

[0004] Therefore, it is necessary to propose an intelligent control method to solve the problem of low work efficiency caused by the inability of users to reach the location of the self-moving robot in time to manually resolve the fault.

[0005] This invention provides an intelligent control method applied to a mobile terminal to control a self-moving robot. The mobile terminal is wirelessly connected to the self-moving robot. The control method includes the following steps:

[0006] In response to receiving a fault signal from the self-moving robot, a first interface is displayed; in response to receiving a first operation from the user on the first interface, a second interface is displayed, the second interface including at least a satellite image and remote control buttons; in response to receiving a second operation from the user on the second interface, a control command is sent to the self-moving robot so that the self-moving robot moves based on the control command.

[0007] Furthermore, the first interface is a fault prompt interface, which includes at least fault information, a first control, and a second control. If the first control is triggered, it indicates that the fault information will not be processed for the time being; if the second control is triggered, it indicates that the fault scenario corresponding to the fault information can be viewed.

[0008] Furthermore, the first operation is that the user triggers the second control.

[0009] Furthermore, the satellite imagery includes a satellite map and the real-time GPS location of the self-moving robot, with the real-time GPS location of the self-moving robot displayed on the satellite map.

[0010] Furthermore, the second operation is the user's operation on the remote control button when the mobile terminal displays the satellite image.

[0011] Furthermore, the second interface also includes a switching button, which, in response to receiving the user's operation on the switching button, switches the satellite image to the real-time image, which is a real-time environmental image within the field of view of the self-moving robot's camera.

[0012] This application also provides an intelligent control method applied to a self-moving robot to achieve control of the self-moving robot, wherein the self-moving robot is wirelessly connected to a mobile terminal, and the method includes the following steps:

[0013] When a fault is detected in the self-moving robot, a fault signal is sent to the mobile terminal; a first instruction is received from the mobile terminal, target data is acquired and sent to the mobile terminal, wherein the target data is the current GPS location of the self-moving robot or a real-time environmental image within the field of view of the self-moving robot's camera; a second instruction is received from the mobile terminal, and the self-moving robot is controlled to move based on the second instruction.

[0014] Furthermore, the second instruction is a remote control instruction.

[0015] This application also provides an intelligent control method applied to an intelligent control system including a self-moving robot and a mobile terminal, to realize the control of the self-moving robot, wherein the self-moving robot is wirelessly connected to the mobile terminal, and the method includes the following steps:

[0016] If the self-moving robot detects that it is in a fault state, it sends a fault signal to the mobile terminal; in response to receiving the fault signal sent by the self-moving robot, the mobile terminal displays a first interface; in response to receiving a first operation from the user on the first interface, the self-moving robot acquires target data and sends it to the mobile terminal, and the mobile terminal displays a second interface, wherein the target data is the current GPS location of the self-moving robot or a real-time environmental image within the field of view of the self-moving robot's camera, and the second interface includes at least satellite images and remote control buttons; in response to receiving a second operation from the user on the second interface, the mobile terminal sends a control command to the self-moving robot; the self-moving robot moves based on the received control command.

[0017] This application also provides a mobile terminal that is wirelessly connected to a self-moving robot. The mobile terminal includes: a memory for storing a computer program; and a processor for executing the computer program to implement the above-described intelligent control method.

[0018] This application also provides a self-moving robot, which includes: a memory for storing a computer program; and a processor for implementing the above-described intelligent control method when executing the computer program.

[0019] This application also provides an intelligent control system, which includes at least the aforementioned mobile terminal and self-moving robot.

[0020] Compared with existing technologies, the present invention has the following advantages: when a fault is detected in the self-moving robot, a fault signal is sent to the mobile terminal; a first instruction is received from the mobile terminal, target data is acquired and sent to the mobile terminal, wherein the target data is the current GPS location of the self-moving robot or a real-time environmental image within the field of view of the self-moving robot's camera; a second instruction is received from the mobile terminal, and the self-moving robot is controlled to move based on the second instruction. Users can remotely monitor whether the self-moving robot has malfunctioned through the mobile terminal, and can also remotely view the specific location of the malfunction or the real-time environmental information at the location of the malfunction. This allows for remote control of the self-moving robot to overcome the malfunction based on the specific location of the malfunction or the real-time environmental information at the location of the malfunction, improving the working efficiency of the self-moving robot; furthermore, since users do not need to physically go to the location of the self-moving robot to manually resolve the malfunction, the user experience is enhanced. Attached Figure Description

[0021] Figure 1 The flowchart of the intelligent control method in the embodiments of the present invention Figure 1 ;

[0022] Figure 2 This is a schematic diagram of the fault display interface in an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the second interface in an embodiment of the present invention. Figure 1 ;

[0024] Figure 4 This is a schematic diagram of the second interface in an embodiment of the present invention. Figure 2 ;

[0025] Figure 5 The flowchart of the intelligent control method in the embodiments of the present invention Figure 2 . Detailed Implementation

[0026] Self-propelled robots (such as lawnmowers) are prone to malfunctions during operation, and when a malfunction occurs, it cannot move freely. Current technology requires the user to physically reach the robot to manually resolve the malfunction; if the user cannot reach the robot in time to manually resolve the malfunction, the robot will remain immobile, resulting in low work efficiency and a poor user experience.

[0027] To address the aforementioned issues, this application provides an intelligent control method that allows users to remotely access fault information and resolve faults via mobile terminals, thereby improving the working efficiency of the self-moving robot and enhancing the user experience.

[0028] To make the technical solutions and advantages of the embodiments of this application clearer, the exemplary embodiments of this application will be described in further detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not an exhaustive list of all embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of this application can be combined with each other.

[0029] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or action from another, and do not necessarily require or imply any actual such relationship or order between such entities or actions. The terms "comprising," "including," or any other variations are intended to cover non-exclusive inclusion, thereby causing a process, method, article, or apparatus that comprises a list of elements to include not only those elements but also other elements not expressly listed or inherent to such process, method, article, or apparatus.

[0030] Please see Figure 1 The following embodiments use a mobile terminal as the execution subject, and the mobile terminal is wirelessly connected to the self-moving robot. The intelligent control method provided in this application includes the following steps 101-103.

[0031] Step 101: In response to receiving a fault signal sent by the self-moving robot, the first interface is displayed.

[0032] In optional embodiments, the mobile terminal includes, but is not limited to, mobile phones, smartwatches, tablets, local servers, etc. The methods by which the mobile terminal wirelessly connects to the self-propelled robot include, but are not limited to, Wi-Fi connection, 4G connection, 5G connection, and Bluetooth connection.

[0033] In an optional embodiment, the first interface is a fault prompt interface. When the mobile terminal receives a fault signal sent by the mobile robot, it displays the fault prompt interface on the user interface.

[0034] like Figure 2As shown, the fault prompt interface displays the fault type, fault code, and a first control and a second control that the user can select to handle the fault.

[0035] Different fault types correspond to different fault codes.

[0036] The first control could be, for example, the OK button. If the user triggers the first control, it means that although the user is aware that the mobile robot has malfunctioned, they choose to ignore the malfunction information and will not handle the malfunction remotely for the time being.

[0037] The second control could be a view button, for example. If the user triggers the second control, it means that the user has chosen to view more detailed information about the location where the fault occurred.

[0038] Step 102: In response to receiving the user's first operation on the first interface, display the second interface.

[0039] In an optional embodiment, the first operation is for the user to trigger the second control (i.e., the user selects to view more detailed information about the location where the fault occurred). After the user triggers the second control on the fault prompt interface, they are redirected to the second interface, which displays detailed information about the location where the fault occurred.

[0040] In alternative embodiments, such as Figure 3 As shown, the second interface may display a real-time environmental image of the self-propelled robot's front on the left and remote control buttons on the right. A camera is mounted on the front of the self-propelled robot, and the real-time environmental image is captured by this camera, which captures images within the camera's field of view. Users can view the surrounding environment at the location of the malfunction through the real-time environmental image, thereby determining the specific cause of the malfunction and taking further measures to resolve it.

[0041] In alternative embodiments, such as Figure 4 As shown, the second interface could also display satellite imagery on the left and remote control buttons on the right. The satellite imagery includes a satellite map (such as Google Maps) and the robot's real-time GPS location, which is displayed as a location point on the satellite map. Users can view the robot's real-time location information through the satellite imagery.

[0042] Displaying satellite maps on mobile devices is primarily achieved by calling the layer management API (Application Programming Interface) encapsulated in the SDK (Software Development Kit). Specifically, for example, if the mobile device integrates the Google Maps SDK, it creates a map view through the Google Maps SDK interface. In response to user actions, it generates a request containing remote sensing image layer identifiers and sends it to the map server. It then receives the raster image data stream for the corresponding area from the server, decodes and renders it using the SDK's rendering engine, and finally displays it as a satellite map on the mobile device interface.

[0043] In alternative embodiments, such as Figure 3 and Figure 4 As shown, the second interface also includes a switching button. In response to the user's operation on the switching button, the user can switch between the satellite image or the real-time environmental image on the left side of the second interface.

[0044] Specifically, assuming the user triggers the second control on the fault prompt interface, the initial second interface displays a real-time environmental image on the left and remote control buttons on the right. If the user clicks the switch button, the real-time environmental image on the left of the second interface switches to a 3D satellite map; if the user clicks the switch button again, the 3D satellite map on the left of the second interface switches to a 2D satellite map; if the user clicks the switch button a third time, the 2D satellite map on the left of the second interface switches back to the real-time environmental image. This cycle continues. Through the switch button settings, the user can select the desired display based on the actual situation, resulting in a higher level of intelligence and a stronger user experience.

[0045] Step 103: In response to receiving a second operation from the user on the second interface, send a control command to the self-moving robot so that the self-moving robot moves based on the control command.

[0046] In an optional embodiment, when the second operation is that the mobile terminal displays satellite images or real-time environmental images, the user operates the remote control buttons, for example, by manipulating the remote control buttons to control the self-moving robot to move forward, backward, left, or right.

[0047] Specifically, when the satellite image is displayed on the left side of the second interface, users can check the location of the self-moving robot by its real-time GPS location on the satellite image. For example, they can check whether the self-moving robot is in or outside the work area. If the self-moving robot is found to be outside the work area, users can control the robot to return to the work area by operating the remote control button.

[0048] When the real-time environmental image is displayed on the left side of the second interface, the user can view the environment in front of the self-moving robot through the real-time environmental image, such as whether there are obstacles or cliffs in front of the self-moving robot, so as to understand the actual cause of the failure (such as the self-moving robot being lifted by a protruding obstacle or falling off a cliff). Then, with the real-time environmental image as a reference, the user can gradually remotely control the self-moving robot to get rid of the failure.

[0049] Understandably, remotely controlling the self-propelled robot may not always resolve the current malfunction. If the malfunction cannot be resolved, the user can simply go to the robot's location and manually resolve the issue. If the malfunction can be resolved, the user can click the "Start Task" button on their mobile device to control the robot to continue its task. Once the robot receives the "Start Task" button from the mobile device, it can either resume its task directly from the location where the malfunction was resolved, or it can return to the charging station first and then proceed to the next station to continue its task.

[0050] Through the aforementioned intelligent control method, users can remotely monitor whether the self-moving robot has malfunctioned via mobile terminal. They can also remotely view the specific location of the malfunction or the real-time environmental information at the location of the malfunction, thereby making a preliminary judgment on the type of malfunction. Furthermore, based on the specific location of the malfunction or the real-time environmental information at the location of the malfunction, users can remotely control the self-moving robot to get rid of the malfunction, thus improving the working efficiency of the self-moving robot. In addition, since users do not need to go to the location of the self-moving robot to manually resolve the malfunction, the user experience is enhanced.

[0051] Please see Figure 5 The following embodiments use a self-moving robot as the execution subject, and the self-moving robot is wirelessly connected to a mobile terminal. The intelligent control method provided in this application includes the following steps 501-503.

[0052] Step 501: When the self-moving robot is detected to be in a fault state, a fault signal is sent to the mobile terminal.

[0053] In an optional embodiment, the fault state includes, but is not limited to, failure of the self-mobile robot to lift off obstacles, failure to recharge, or being outside the working area.

[0054] In an optional embodiment, when the self-moving robot malfunctions, it stops moving, and the display interface on the robot body displays a fault code (it is easy to understand that different fault states correspond to different fault codes, for example, the fault code corresponding to the failure of the self-moving robot to recharge is E61, and the fault code corresponding to the failure of the self-moving robot to lift off obstacles is E70, etc.). At the same time, the self-moving robot will send the corresponding fault signal to the mobile terminal for display, so that even if the user is not at the location of the self-moving robot, they can remotely and promptly understand the fault status of the self-moving robot.

[0055] Step 502: Receive the first instruction sent by the mobile terminal, acquire the target data and send it to the mobile terminal.

[0056] In an optional embodiment, the target data is the current GPS location of the self-moving robot or a real-time environmental image within the field of view of the self-moving robot's camera.

[0057] In an optional embodiment, the first instruction is issued by the user through a mobile terminal. For example, the first instruction may be a second control of the fault prompt interface on the mobile terminal triggered by the user.

[0058] In an optional embodiment, after the self-mobilizing robot receives the first instruction, the fault code on the self-mobilizing robot will disappear, and the self-mobilizing robot will acquire a real-time environmental image of the front through the camera on the front of the robot body and send the real-time environmental image to the mobile terminal.

[0059] In an optional embodiment, after the self-moving robot receives the first instruction, the fault code on the self-moving robot will disappear, and the self-moving robot will obtain the real-time GPS positioning coordinates of the self-moving robot through its own GPS positioning module and send them to the mobile terminal.

[0060] Step 503: Receive the second instruction sent by the mobile terminal, and control the self-moving robot to move based on the second instruction.

[0061] In an optional embodiment, the second instruction is a remote control instruction.

[0062] In an optional embodiment, after the self-moving robot acquires a real-time environmental image of the area in front of it through the camera and sends it to the mobile terminal, the mobile terminal will display the real-time environmental image and remote control buttons. The user can use the real-time environmental image as a reference and manipulate the remote control buttons to remotely control the self-moving robot to move in the appropriate direction in an attempt to resolve the current fault.

[0063] In an optional embodiment, after the self-moving robot obtains its real-time GPS location and sends it to the mobile terminal, the mobile terminal will display satellite images and remote control buttons. The satellite images are generated by displaying the real-time GPS location of the self-moving robot on a satellite map. Users can clearly view the location of the self-moving robot on the satellite map. Users can use the satellite images as a reference and manipulate the remote control buttons to remotely control the self-moving robot to move in a suitable direction in an attempt to resolve the current fault.

[0064] Through the aforementioned intelligent control methods, the self-moving robot can send its fault status to a mobile terminal for display in real time, allowing users to remotely check whether the self-moving robot has malfunctioned. The self-moving robot can also receive control commands from the mobile terminal in real time to acquire GPS location information or real-time environmental images in front of it and send them to the mobile terminal. This ensures that users can remotely control the self-moving robot based on GPS location information or real-time environmental images in front of it to attempt to resolve the current fault, avoiding the problem of low work efficiency caused by the self-moving robot being unable to move due to a malfunction. Furthermore, since users do not need to physically go to the self-moving robot's location to manually resolve the fault, it is more intelligent and enhances the user experience.

[0065] This application also provides an intelligent control method applied to an intelligent control system including a self-moving robot and a mobile terminal, wherein the self-moving robot is wirelessly connected to the mobile terminal, and the method includes the following steps:

[0066] If the self-moving robot detects that it is in a fault state, it sends a fault signal to the mobile terminal.

[0067] In response to receiving a fault signal sent by the self-moving robot, the mobile terminal displays a first interface.

[0068] In an optional embodiment, the first interface is a fault prompt interface. When the mobile terminal receives a fault signal sent by the mobile robot, the fault prompt interface is displayed on the user interface. The fault prompt interface displays the fault type, fault code, and a first control and a second control available to the user for handling the fault. If the first control is triggered, it indicates that the fault information will not be processed temporarily; if the second control is triggered, it indicates that the fault scenario corresponding to the fault information can be viewed.

[0069] In response to receiving a first operation from the user on the first interface, the self-moving robot acquires target data and sends it to the mobile terminal, and the mobile terminal displays a second interface.

[0070] In an optional embodiment, the first operation is for the user to trigger the second control.

[0071] In an optional embodiment, the target data is the current GPS location of the self-moving robot or a real-time environmental image within the field of view of the self-moving robot's camera.

[0072] In an optional embodiment, the second interface can be as follows: Figure 3 As shown, the left side displays a real-time environmental image from in front of the mobile robot, and the right side displays the remote control buttons; the second interface can also be displayed as follows. Figure 4 As shown, the left side displays satellite images, and the right side displays remote control buttons. The satellite images include satellite maps (such as Google Maps) and the real-time GPS location of the self-moving robot. The real-time GPS location of the self-moving robot is displayed as a location point on the satellite map.

[0073] When the user triggers the second control on the fault prompt interface, the self-moving robot can send the real-time environmental image captured by its camera to the mobile terminal, so that the mobile terminal's second interface displays the real-time environmental image. When the user triggers the second control on the fault prompt interface, the self-moving robot can also send its real-time GPS coordinates to the mobile terminal, so that the mobile terminal's second interface displays satellite imagery, including a satellite map and the self-moving robot's real-time GPS position mapped onto the satellite map.

[0074] In response to receiving a second operation from the user on the second interface, the mobile terminal sends a control command to the self-moving robot.

[0075] In an optional embodiment, when the second operation involves the mobile terminal displaying satellite images or real-time environmental images, the user operates the remote control buttons, for example, by manipulating the remote control buttons to control the self-propelled robot to move forward, backward, left, or right. The control command is a remote control command.

[0076] In response to receiving the control command sent by the mobile terminal, the self-moving robot moves based on the received control command.

[0077] Through the aforementioned intelligent control method, the self-moving robot can send its own fault status, the specific location of the fault, or real-time environmental information at the location of the fault to a mobile terminal. This allows users to remotely obtain detailed fault information of the self-moving robot in a timely manner and remotely control the self-moving robot to get rid of the fault based on this detailed fault information, thereby improving the working efficiency of the self-moving robot and enhancing the user experience.

[0078] This application also provides a self-moving robot, which includes: a memory for storing a computer program; and a processor for implementing the above-mentioned intelligent control method when executing the computer program. The specific control method will not be described in detail here.

[0079] This application also provides a mobile terminal that is wirelessly connected to a self-moving robot. The mobile terminal includes: a memory for storing a computer program; and a processor for executing the computer program to implement the above-mentioned intelligent control method. The specific control method will not be described in detail here.

[0080] This application also provides an intelligent control system, which includes at least the aforementioned mobile terminal and the aforementioned self-moving robot.

[0081] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this application. In practical applications, those skilled in the art can select some or all of it to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0082] This invention is not limited to the specific embodiments described above. Those skilled in the art will readily understand that many alternative solutions exist for the intelligent control method of this invention without departing from its principles and scope. The scope of protection of this invention is defined by the claims.

Claims

1. An intelligent control method applied to a mobile terminal to control a self-moving robot, wherein the mobile terminal is wirelessly connected to the self-moving robot, characterized in that, The control method includes: In response to receiving a fault signal sent by the self-moving robot, a first interface is displayed; In response to receiving a first operation from the user on the first interface, a second interface is displayed, the second interface including at least satellite images and remote control buttons; In response to receiving a second operation from the user on the second interface, a control command is sent to the self-moving robot so that the self-moving robot moves based on the control command.

2. The intelligent control method according to claim 1, characterized in that, The first interface is a fault prompt interface. The first interface includes at least fault information, a first control, and a second control. If the first control is triggered, it means that the fault information will not be processed for the time being; if the second control is triggered, it means that the fault scenario corresponding to the fault information can be viewed.

3. The intelligent control method according to claim 2, characterized in that, The first operation is that the user triggers the second control.

4. The intelligent control method according to claim 1, characterized in that, The satellite imagery includes a satellite map and the real-time GPS location of the self-moving robot, with the real-time GPS location of the self-moving robot displayed on the satellite map.

5. The intelligent control method according to claim 1, characterized in that, The second operation is the user's operation on the remote control buttons when the mobile terminal displays the satellite image.

6. The intelligent control method according to claim 1, characterized in that, The second interface also includes a switching button. In response to receiving the user's operation on the switching button, the satellite image is switched to the real-time image, which is a real-time environmental image within the field of view of the self-moving robot's camera.

7. An intelligent control method applied to a self-moving robot to achieve control of the self-moving robot, wherein the self-moving robot is wirelessly connected to a mobile terminal, characterized in that, The control method includes: When the self-moving robot is detected to be in a fault state, a fault signal is sent to the mobile terminal; The system receives a first instruction sent by the mobile terminal, acquires target data and sends it to the mobile terminal. The target data is the current GPS location of the self-mobilizing robot or a real-time environmental image within the field of view of the self-mobilizing robot's camera. The system receives a second instruction sent by the mobile terminal and controls the self-moving robot to move based on the second instruction.

8. The intelligent control method according to claim 7, characterized in that, The second command is a remote control command.

9. An intelligent control method applied to an intelligent control system including a self-moving robot and a mobile terminal, to realize the control of the self-moving robot, characterized in that, The self-moving robot is wirelessly connected to the mobile terminal, and the control method includes: If the self-moving robot detects that it is in a fault state, it sends a fault signal to the mobile terminal; In response to receiving a fault signal sent by the self-moving robot, the mobile terminal displays a first interface; In response to receiving a first operation from the user on the first interface, the self-moving robot acquires target data and sends it to the mobile terminal, and the mobile terminal displays a second interface, wherein the target data is the current GPS location of the self-moving robot or a real-time environmental image within the field of view of the self-moving robot's camera, and the second interface includes at least satellite images and remote control buttons. In response to receiving a second operation from the user on the second interface, the mobile terminal sends a control command to the self-moving robot; The self-moving robot moves based on the received control commands.

10. A mobile terminal, characterized in that, The mobile terminal and the self-moving robot are wirelessly connected, and the mobile terminal includes: Memory, used to store computer programs; A processor for implementing the intelligent control method as described in any one of claims 1 to 6 when executing the computer program.

11. A self-moving robot, characterized in that, The self-moving robot includes: Memory, used to store computer programs; A processor for implementing the intelligent control method as described in any one of claims 7 to 8 when executing the computer program.

12. An intelligent control system, characterized in that, The intelligent control system includes at least the mobile terminal as described in claim 10 and the self-moving robot as described in claim 11.

Images