Mobile equipment control method and device, storage medium and electronic device

Abstract

The invention provides a control method and device of a mobile device, a storage medium and an electronic device, and the method comprises the steps: constructing a panoramic image of an area where the mobile device is located according to an image collected by an image collection part on the mobile device; under the condition that the mobile equipment needs to be charged, a simulation track route of the mobile equipment from a current position to a target position is determined in the panoramic image, the target position is a charging stop position of a base station, and the base station is used for charging the mobile equipment; and controlling the mobile device to enter the charging parking position according to the simulated track route.

Description

[Technical field]

[0001] The present application relates to the field of smart home, and more specifically, to a control method and device for a mobile device, a storage medium, and an electronic device. [Background technology]

[0002] At present, the control of mobile devices usually relies on the combined control of the front camera, top LDS (Laser Direct Structuring, i.e., LiDAR) and collision switch on the mobile device to identify objects near the mobile device. Especially when the mobile device needs to return to charge due to insufficient power during the execution of a task, it usually relies on LDS to identify the base station to explore the recharging route.

[0003] However, the control method of the above-mentioned mobile device is greatly affected by the external environment. In a small space, it is easy for the signal to be reflected multiple times, which will lead to a longer LDS recharging exploration time. In addition, when controlling the mobile device for recharging, if there are straight walls or some materials that are easy to reflect signals near the base station, it is also easy to cause the recharging signal to be disordered, and then fail to identify the base station location.

[0004] It can be seen from this that the control method of the mobile device in the related art has the problem of low recharging efficiency of the mobile device. [Summary of the invention]

[0005] The purpose of the present application is to provide a control method and device for a mobile device, a storage medium and an electronic device, so as to at least solve the problem of low recharging efficiency of the mobile device in the control method for the mobile device in the related art.

[0006] The purpose of this application is to achieve the following technical solutions:

[0007] According to one aspect of an embodiment of the present application, a method for controlling a mobile device is provided, comprising: constructing a panoramic image of an area where the mobile device is located based on an image captured by an image acquisition component on the mobile device; when the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device; and controlling the mobile device to enter the charging docking position according to the simulated trajectory route.

[0008] In an exemplary embodiment, the number of the image acquisition components is at least three, and at least three of the image acquisition components are evenly distributed on the side wall of the mobile device, wherein constructing a panoramic image of the area where the mobile device is located based on the images captured by the image acquisition components on the mobile device includes: constructing a panoramic image of the area where the mobile device is located based on at least one group of images captured by at least three of the image acquisition components on the mobile device.

[0009] In an exemplary embodiment, when the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image includes: when the mobile device needs to be charged, obtaining the current position of the mobile device in the panoramic image; generating the simulated trajectory route of the mobile device from the current position to the charging docking position based on the current position and a track line on the base of the base station, wherein at least a portion of the charging docking position is defined by the track line, and a portion of the simulated trajectory route overlaps with the track line.

[0010] In an exemplary embodiment, after determining the simulated trajectory route of the mobile device from the current position to the target position in the panoramic image, the method further includes: when the distance between the position of the mobile device and the position of the obstacle shown in the panoramic image is less than a preset distance threshold, updating the simulated trajectory route in combination with the distance between the obstacle and the mobile device, and the target position, to obtain a target simulated trajectory route, wherein, in the target simulated trajectory route, the distance between the mobile device and the obstacle is greater than the preset distance threshold.

[0011] In an exemplary embodiment, the simulated trajectory route includes a first trajectory route and a second trajectory route, the first trajectory route is a route for guiding the mobile device to move from the current position to a preset position, and the second trajectory route is a route for guiding the mobile device from the preset position to the target position, the preset position is a position in front of the entrance of the charging docking position and the distance from the entrance is a preset distance, and controlling the mobile device to enter the charging docking position according to the simulated trajectory route includes: controlling the mobile device to move from the current position to the preset position according to the first trajectory route; when it is determined that the mobile device moves to the preset position, displaying the movement trajectory of the mobile device generated based on the location of the charging interface of the mobile device in the panoramic image; and controlling the mobile device to enter the charging docking position based on the movement trajectory in the panoramic image and the second trajectory route.

[0012] In an exemplary embodiment, controlling the mobile device to enter the charging docking position based on the moving trajectory in the panoramic image and the second trajectory route includes: when the degree of overlap between the moving trajectory in the panoramic image and the second trajectory route is greater than a preset threshold, controlling the mobile device to enter the charging docking position along the second trajectory route.

[0013] In an exemplary embodiment, controlling the mobile device to enter the charging docking position based on the moving trajectory in the panoramic image and the second trajectory route includes: determining a target angle between the moving trajectory and the second trajectory route when the degree of overlap between the moving trajectory in the panoramic image and the second trajectory route is less than or equal to a preset threshold; adjusting the posture of the mobile device according to the target angle until the degree of overlap between the moving trajectory of the mobile device and the second trajectory route is greater than the preset threshold; and controlling the mobile device to enter the charging docking position according to the second trajectory route when it is determined that the posture adjustment of the mobile device is completed.

[0014] According to another aspect of an embodiment of the present application, a control device for a mobile device is also provided, including: a construction unit, used to construct a panoramic image of the area where the mobile device is located based on the image captured by the image acquisition component on the mobile device; a determination unit, used to determine a simulated trajectory route of the mobile device from the current position to the target position in the panoramic image when the mobile device needs to be charged, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device; a control unit, used to control the mobile device to enter the charging docking position according to the simulated trajectory route.

[0015] According to another aspect of the embodiments of the present application, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the control method of the mobile device when running.

[0016] According to another aspect of an embodiment of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the control method of the mobile device through the computer program.

[0017] According to another aspect of the embodiment of the present application, a cleaning device is also provided, characterized in that it includes: a cleaning robot and a base station, the cleaning robot includes an image acquisition component, and at least one of the cleaning robot and the base station includes a processing component, wherein the image acquisition component is used to acquire images; the base station includes a charging docking position, wherein the base station is used to charge the cleaning robot; the processing component is used to construct a panoramic image of the area where the cleaning robot is located based on the image acquired by the image acquisition component; when the cleaning robot needs to be charged, determine a simulated trajectory route of the cleaning robot from the current position to the target position in the panoramic image, wherein the target position is the charging docking position; and control the cleaning robot to enter the charging docking position according to the simulated trajectory route.

[0018] In an embodiment of the present application, a panoramic image of the area where the mobile device is located is constructed based on the image captured by the image acquisition component on the mobile device, and the movement of the mobile device is controlled based on the panoramic image. According to the image captured by the image acquisition component on the mobile device, a panoramic image of the area where the mobile device is located is constructed; when the mobile device needs to be charged, a simulated trajectory route of the mobile device from the current position to the target position is determined in the panoramic image, wherein the target position is the charging docking position of the base station, and the base station is used to charge the mobile device; the mobile device is controlled to enter the charging docking position according to the simulated trajectory route. During the movement of the mobile device, the panoramic image can be updated in real time according to the captured image, thereby providing accurate route guidance for the movement of the mobile device. Since there is no need to explore the route based on the signal, it can avoid the situation where the mobile device cannot complete the recharging in a short time due to multiple reflections of the recharging signal or information disorder, thereby achieving the technical effect of improving the recharging efficiency of the mobile device.

Brief Description of the Drawings

[0019] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0020] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0021] Figure 1 is a schematic diagram of a hardware environment of an optional control method for a mobile device according to an embodiment of the present application;

[0022] Figure 2is a flow chart of an optional control method of a mobile device according to an embodiment of the present application;

[0023] Figure 3 is a schematic diagram of an optional mobile device according to an embodiment of the present application;

[0024] Figure 4 is a schematic diagram of an optional control method of a mobile device according to an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of another optional control method of a mobile device according to an embodiment of the present application;

[0026] Figure 6 is a flow chart of another optional control method of a mobile device according to an embodiment of the present application;

[0027] Figure 7 is a structural block diagram of an optional control device of a mobile device according to an embodiment of the present application;

[0028] Figure 8 It is a structural block diagram of an optional electronic device according to an embodiment of the present application. [Specific implementation method]

[0029] The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments. It should be noted that the embodiments and features in the embodiments of the present application can be combined with each other without conflict.

[0030] It should be noted that the terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0031] According to one aspect of an embodiment of the present application, a control method for a mobile device is provided. Optionally, in this embodiment, the control method for the mobile device can be applied to Figure 1 In the hardware environment shown in FIG. 1 , a mobile device 102, a base station 104 and a cloud platform 106 are formed. Figure 1 As shown, the mobile device 102 can be connected to the base station 104 and / or the cloud platform 106 (e.g., a voice cloud platform) through a network to enable interaction between the mobile device 102 and the base station 104 and / or the cloud platform 106.

[0032] The above-mentioned network may include but is not limited to at least one of the following: wired network, wireless network. The above-mentioned wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the above-mentioned wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth, infrared. The network used by the mobile device 102 to communicate with the base station 104 and / or the cloud platform 106 may be the same as or different from the network used by the base station 104 to communicate with the cloud platform 106. The mobile device 102 may include but is not limited to: cleaning robots, such as sweeping robots, floor washing robots, robots that combine washing and sweeping, etc., and delivery robots, such as food delivery robots, object delivery robots, etc.

[0033] The control method of the mobile device of the embodiment of the present application can be executed by the mobile device 102 and the cloud platform 106 alone, or by the mobile device 102 and the cloud platform 106 together. The mobile device 102 can also execute the control method of the mobile device of the embodiment of the present application by the client installed thereon.

[0034] Taking the mobile device 102 executing the mobile device control method in this embodiment as an example, Figure 2 is a flow chart of an optional control method of a mobile device according to an embodiment of the present application, such as Figure 2 As shown, the process of the method may include the following steps:

[0035] Step S202: construct a panoramic image of the area where the mobile device is located based on the images collected by the image collection component on the mobile device.

[0036] The control method of the mobile device in this embodiment can be applied to the scene of controlling the mobile device moving in the target area. The above-mentioned target area can be an indoor area, for example, a user's home room, an office, a factory workshop, etc. The mobile device can be a robot. From the functional point of view, the above-mentioned mobile device can be the aforementioned cleaning robot, delivery robot, etc. From the morphological point of view, the above-mentioned mobile device can be a bionic robot, for example, a quadruped robot, a crawling robot, etc. This is not limited in this embodiment.

[0037] In the prior art, the control of the mobile device mainly relies on a combination of a front camera, a top LDS radar and a striker switch installed on the mobile device to detect the surrounding environment and control the movement of the robot.

[0038] Take the sweeping robot as an example. When the sweeping robot is working, it relies on the combined control of the front camera, the top LDS radar and the collision plate switch to achieve the purpose of identifying and avoiding obstacles. However, in corners such as wall corners, piles of items, and the bottom of stools, obstacles are often not identified or recognized incorrectly and collisions occur. In addition, when the sweeping robot performs the recharging action, it mainly relies on the LDS radar to identify the base station to explore the recharging, dock with the base station, and rely on the transmission and reception of infrared signals between the base station and the sweeping robot. However, there are many problems in relying on the LDS radar to identify the base station to explore the recharging. For example, when the sweeping robot encounters a small space, due to the influence of the small space, the signal emitted by the sweeping robot is easily reflected multiple times, which leads to a longer LDS recharging exploration time; when there is a straight wall or some materials that are easy to reflect signals near the base station, it will cause the recharging signal to be disordered, which will lead to the failure of the sweeping robot to recharge.

[0039] In order to at least partially solve the above-mentioned problems, in this embodiment, a panoramic image of the area where the mobile device is located can be constructed based on the images captured by the image acquisition component installed on the mobile device, the position of the mobile device can be determined in the panoramic image, and the movement of the mobile device can be controlled according to the destination of the mobile device.

[0040] Optionally, the above-mentioned image acquisition component can acquire images of the surrounding environment of the mobile device in real time during the movement of the mobile device. Correspondingly, the panoramic image constructed based on the acquired images can also be a panoramic image updated in real time. The panoramic image can only represent the impact of a two-dimensional plane, or it can be a three-dimensional image constructed based on the captured image. In the panoramic image, the distance between the objects, walls, etc. around the mobile device and the mobile device can be displayed in real time, so as to avoid collision between the mobile device and obstacles.

[0041] Step S204, when the mobile device needs to be charged, determine a simulated trajectory route of the mobile device from the current position to the target position in the panoramic image, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device.

[0042] The power of the mobile device is limited. When the power is low, it is necessary to promptly control the mobile device to return to the base station to charge in the charging compartment of the base station. In addition, in order to ensure the working efficiency of the mobile device, after each mobile device completes a task, it is also necessary to promptly control the mobile device to return to the base station to charge so that it has enough power to perform the work next time.

[0043] In this embodiment, when the mobile device needs to be charged, the simulated trajectory route of the mobile device from the current position to the target position can be determined in the constructed panoramic image. The target position here can be the charging docking position of the base station of the mobile device, and the base station can be used to charge the mobile device.

[0044] The situation in which the mobile device needs to be charged may refer to a situation in which the power of the mobile device is insufficient, or may refer to a situation in which the mobile device has completed executing a corresponding task.

[0045] Optionally, the simulated trajectory route from the current position to the target position may be a real-time updated route, that is, in the process of the mobile device moving along the simulated trajectory route, the simulated trajectory route is updated in real time in the panoramic image constructed or updated based on the latest image captured by the image acquisition component on the mobile device.

[0046] Optionally, when the mobile device does not need to be charged, when the destination of the mobile device is determined, a simulated trajectory route from the current position to the destination position can also be determined in the panoramic image based on the current position of the mobile device and the destination position. Correspondingly, the simulated trajectory route from the current position to the destination position can also be updated in real time during the movement of the mobile device.

[0047] Step S206: Control the mobile device to enter a charging docking position according to the simulated trajectory route.

[0048] When the mobile device needs to be charged, the simulated trajectory route determined in the panoramic image may include a route for guiding the mobile device to enter the charging docking position. In this embodiment, the mobile device may be controlled to enter the charging docking position according to the determined simulated trajectory route.

[0049] Through the above steps S202 to S206, a panoramic image of the area where the mobile device is located is constructed based on the images collected by the image acquisition component on the mobile device; when the mobile device needs to be charged, a simulated trajectory route of the mobile device from the current position to the target position is determined in the panoramic image, wherein the target position is the charging docking position of the base station, and the base station is used to charge the mobile device; controlling the mobile device to enter the charging docking position according to the simulated trajectory route can solve the problem of low recharging efficiency of the mobile device in the mobile device control method in the related art.

[0050] In an exemplary embodiment, the number of image acquisition components is at least three, and the at least three image acquisition components are evenly distributed on the side wall of the mobile device, wherein a panoramic image of the area where the mobile device is located is constructed based on the images acquired by the image acquisition components on the mobile device, including:

[0051] S11, constructing a panoramic image of the area where the mobile device is located according to at least one group of images collected by at least three image collection components on the mobile device.

[0052] In order to improve the accuracy of the constructed panoramic image and reduce the production cost, in this embodiment, the number of image acquisition components on the mobile device can be at least three. The acquisition field of view of each image acquisition component can be 120 degrees.

[0053] The three image acquisition components can be binocular cameras or other cameras that can measure distances while acquiring images. The three image acquisition components can also be monocular cameras that can determine the positions of objects around the mobile device by acquiring the object images in combination with other distance measuring devices on the mobile device.

[0054] For example, taking the mobile device as a sweeping robot, Figure 3 As shown, the sweeping robot removes the front collision plate switch, regards the side wall structure of the machine as an integral structure, and the three image acquisition components are evenly distributed at 0°, 120° and 240° of the side wall structure of the sweeping robot.

[0055] When constructing a panoramic image of the area where the mobile device is located based on a group of images collected by at least three image acquisition components, the images collected by each acquisition component can be spliced ​​to obtain relevant information of each object in the environment around the mobile device, and then construct a panoramic image model.

[0056] Through this embodiment, a panoramic image is constructed based on the object images captured by three image acquisition components with a capture field of view of 120 degrees, which can reduce production costs while ensuring the accuracy of the position of each object in the constructed panoramic image.

[0057] In an exemplary embodiment, when the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in a panoramic image includes:

[0058] S21, when the mobile device needs to be charged, obtaining the current position of the mobile device in the panoramic image;

[0059] S22, generating a simulated trajectory route of the mobile device from the current position to the charging docking position according to the current position and the track line on the base of the base station, wherein at least a portion of the charging docking position is defined by the track line, and a portion of the simulated trajectory route overlaps with the track line.

[0060] When a mobile device needs to be charged, the current position of the mobile device can be determined in the constructed panoramic image based on the position of the mobile device. At the same time, based on the location of the base station in the panoramic image, a simulated trajectory route from the current position to the target position can be planned.

[0061] It should be noted that the location of the base station in the panoramic image can be the location determined by the mobile device when it departs from the base station, or it can be determined by the mobile device based on the collected image containing the base station during the execution of the task.

[0062] In this embodiment, in the panoramic image, there may be two track lines on the base station base, and the track lines correspond to the climbing track of the base station base, such as Figure 4 As shown, at least part of the charging docking position is defined by the track line. In the panoramic image, according to the current position and the track line on the base of the base station, a part of the route close to the base station base can overlap with the track line in the simulated trajectory route generated.

[0063] Through this embodiment, by planning a simulated trajectory route in which some route segments overlap with the track line of the base station base, the success rate of controlling the mobile device to enter the base station charging compartment according to the simulated trajectory route can be improved.

[0064] In an exemplary embodiment, after determining the simulated trajectory route of the mobile device from the current position to the target position in the panoramic image, the method further includes:

[0065] S31, when the distance between the position of the mobile device and the position of the obstacle shown in the panoramic image is less than a preset distance threshold, the simulated trajectory route is updated in combination with the distance between the obstacle and the mobile device, and the target position, to obtain a target simulated trajectory route, wherein in the target simulated trajectory route, the distance between the mobile device and the obstacle is greater than the preset distance threshold.

[0066] Taking into account that a mobile device may encounter sudden obstacles or moving obstacles during its movement, thereby causing a collision of the mobile device moving along a simulated trajectory route, in this embodiment, while the mobile device is moving along the simulated trajectory route, the simulated trajectory route can be updated based on the real-time collected images to improve the mobility flexibility of the mobile device.

[0067] In particular, when the distance between the location of the mobile device and the location of the obstacle in the panoramic image display is less than a preset distance threshold, in order to avoid a collision between the mobile device and the obstacle, the simulated trajectory route can be updated in the panoramic image so that the distance between the mobile device and the obstacle is greater than the preset distance threshold.

[0068] In addition, when updating the simulated trajectory route, in addition to considering increasing the distance between the mobile device and obstacles whose distance is less than the preset distance threshold, the distances between other obstacles around the mobile device and the mobile device, as well as the target position, need to be considered.

[0069] Through this embodiment, when the mobile device moves along the simulated trajectory route, the simulated trajectory route is updated, which can improve the safety of the movement of the mobile device.

[0070] In an exemplary embodiment, the simulated trajectory route includes a first trajectory route and a second trajectory route, the first trajectory route is a route for guiding the mobile device to move from a current position to a preset position, and the second trajectory route is a route for guiding the mobile device from the preset position to a target position, the preset position is a position where the front side of the entrance of the charging docking position is at a preset distance from the entrance, and controlling the mobile device to enter the charging docking position according to the simulated trajectory route includes:

[0071] S41, controlling the mobile device to move from a current position to a preset position according to a first trajectory route;

[0072] S42, when it is determined that the mobile device has moved to a preset position, displaying a movement track of the mobile device generated based on the location of the charging port of the mobile device in the panoramic image;

[0073] S43, based on the moving track in the panoramic image and the second track route, controlling the mobile device to enter a charging docking position.

[0074] Considering that the mobile device needs to be successfully docked with the charging interface on the base station before charging can be started, in this embodiment, the simulated trajectory route planned when the mobile device needs to be charged may include a first trajectory route and a second trajectory route.

[0075] In the process of controlling the movement of the mobile device according to the first trajectory route, it is only necessary to ensure that the movement direction of the mobile device moves according to the first trajectory route. In the process of controlling the movement of the mobile device according to the second trajectory route, it is necessary to pay attention to the posture of the mobile device. In other words, it is necessary to control the posture of the mobile device according to the location of the charging interface of the mobile device. When the charging interface of the mobile device is located at the front side of the mobile device, the mobile device can enter the charging docking position with a forward posture, and when the charging interface of the mobile device is located at the back side of the mobile device, it is necessary to control the mobile device to move in a backward posture according to the second trajectory route, so as to complete the docking of the charging interface after entering the charging docking position.

[0076] Therefore, the first track route can guide the mobile device to move from the current position to the preset position. The second track route can guide the mobile device from the preset position to the charging compartment, and the preset position is the position where the distance between the front side of the entrance of the charging docking position and the entrance is the preset distance.

[0077] In this embodiment, when it is determined that the mobile device needs to be charged, the mobile device is first controlled to move from the current position to the preset position according to the first track route. When it is determined that the mobile device moves to the preset position, the movement track of the mobile device generated based on the location of the charging interface of the mobile device is displayed in the panoramic image, and based on the movement track in the panoramic image and the second track route, the mobile device is controlled to enter the charging docking position.

[0078] Taking the above-mentioned moving trajectory as a backward trajectory as an example, the backward trajectory can be a trajectory generated behind the mobile device in the panoramic image based on the current posture of the mobile device, which is used to simulate the position to which the mobile device can move when moving backward based on the current posture. Figure 5 As shown, the backward trajectory is a trajectory with a width equal to the width of the sweeping robot.

[0079] Through this embodiment, the simulated trajectory route is divided into two routes. The mobile device is first controlled to move to a preset position, and then a moving trajectory is generated. According to the moving trajectory and the simulated trajectory route, the mobile device is controlled to recharge, which can improve the efficiency of successful recharging.

[0080] In an exemplary embodiment, based on the moving track in the panoramic image and the second track route, controlling the mobile device to enter the charging docking position includes:

[0081] S51, when the overlap between the moving track in the panoramic image and the second track route is greater than a preset threshold, control the mobile device to enter a charging docking position according to the second track route.

[0082] After the mobile device reaches the preset position, in order to ensure that the mobile device can maintain the corresponding moving posture according to the second trajectory route to enter the charging docking position to complete recharging, the moving trajectory can be compared with the second trajectory route based on the generated moving trajectory to see whether they overlap. If the moving trajectory overlaps with the second trajectory route, the mobile device can be controlled to enter the charging compartment in the base according to the second trajectory route.

[0083] In this embodiment, when the degree of overlap between the moving trajectory in the panoramic image and the second trajectory route is greater than a preset threshold, it can be determined that the moving trajectory and the second trajectory route are substantially overlapped.

[0084] Through this embodiment, when the overlap between the moving trajectory and the second trajectory route is greater than a preset threshold, controlling the mobile device to enter the charging compartment in the base can improve the success rate of docking the mobile device with the charging interface, thereby improving the recharging efficiency.

[0085] In an exemplary embodiment, based on the moving track in the panoramic image and the second track route, controlling the mobile device to enter the charging docking position includes:

[0086] S61, when the overlap between the moving track in the panoramic image and the second track route is less than or equal to a preset threshold, determining a target angle between the moving track and the second track route;

[0087] S62, adjusting the posture of the mobile device according to the target angle until the overlap between the moving trajectory of the mobile device and the second trajectory route is greater than a preset threshold;

[0088] S63, when it is determined that the posture adjustment of the mobile device is completed, controlling the mobile device to enter a charging docking position along a second trajectory route.

[0089] When the overlap degree between the moving track in the panoramic image and the second track route is less than or equal to the preset threshold, it can be determined that the moving track and the second track route do not overlap, and the posture of the mobile device needs to be adjusted.

[0090] In this embodiment, when adjusting the posture of the mobile device, the target angle between the moving trajectory and the second trajectory route can be determined first, and then the posture of the mobile device can be adjusted according to the target angle until the overlap between the moving trajectory of the mobile device and the second trajectory route is greater than a preset threshold.

[0091] The aforementioned adjusting the posture of the mobile device may refer to controlling the rotation of the mobile device.

[0092] Taking into account that the position of the mobile device may change during the process of adjusting the posture of the mobile device, optionally, during the process of adjusting the posture of the mobile device, it is possible to determine in real time whether the position of the mobile device is a preset position. If it is not the preset position, the mobile device is controlled to move until the moving trajectory of the mobile device at the preset position coincides with the second trajectory route, and then the mobile device is controlled to enter the charging compartment in the base along the second trajectory route.

[0093] For example, taking the mobile device as a sweeping robot and the moving trajectory as a reverse trajectory, after the sweeping robot reaches the designated position in front of the base station, the base station is used as the target value of the target preference algorithm, and the obstacles around the base station are judged by pure camera vision. The sweeping robot rotates the target angle so that the robot performs a "reverse" operation and determines the reverse position of the sweeping robot in real time. When the simulated return path of the sweeping robot deviates from the target value, the sweeping robot driving wheel is controlled to perform a "sweeping" action, and the return posture of the machine is dynamically adjusted so that when the sweeping robot reaches a certain position in front of the base station, the real-time simulated recharging path coincides with the climbing track of the base station base. After adjusting the posture of the machine, the sweeping robot can be controlled to recharge and enter the base station. If the sweeping robot is successfully charged, it can be determined that the sweeping machine is successfully recharged.

[0094] like Figure 6 As shown, after the host (i.e., the sweeping robot) reaches the preset position, the host visual algorithm simulates the "reverse" recharging path in real time to determine whether it deviates from the base station track line and the deviation angle is greater than a certain threshold. If not, the host is controlled to retreat and recharge. If so, the host is controlled to adjust the drive wheel, and the sweeping robot performs a "sweeping" operation, and again determines whether it deviates from the base station track line and the deviation angle is greater than a certain threshold. If so, the drive wheel is readjusted, and the sweeping robot performs a "sweeping" operation. If not, the host is controlled to retreat and recharge. Among them, when the sweeping robot performs a "sweeping" operation, according to the angle of the sweeping robot deviating from the base station track line, the sweeping robot is controlled to rotate to a target angle to complete the "sweeping" operation, and the target angle is determined according to the angle of the sweeping robot deviating from the base station track line.

[0095] Through this embodiment, when the mobile trajectory does not coincide with the base station trajectory line, the posture of the mobile device is adjusted to achieve coincidence of the mobile trajectory with the base station trajectory line, which can improve the recharging success rate of the mobile device.

[0096] The control method of the mobile device in the embodiment of the present application is explained below in conjunction with an optional example. In this optional example, the mobile device is a sweeping robot.

[0097] This optional example provides a design of a sweeping robot panoramic image obstacle avoidance and recharging optimization solution. The process of the control method of the mobile device in this optional example may include the following steps:

[0098] Step 1: Based on the images collected by three image collection components evenly distributed on the side wall of the sweeping robot, a real-time panoramic image is constructed during the movement of the sweeping robot.

[0099] Step 2: Control the robot vacuum cleaner to avoid obstacles according to the positions of obstacles in the panoramic image.

[0100] Step 3: When the sweeping robot is recharging, a return route is simulated according to the current position of the sweeping robot in the panoramic image and the position of the base station, and the sweeping robot is controlled to reach a designated position in front of the base station.

[0101] Step 4: after the sweeping robot reaches the designated position, the moving state of the sweeping robot is switched to the "reverse" posture, and the posture of the sweeping robot is adjusted so that the simulated trajectory of the sweeping robot coincides with the "reverse" trajectory.

[0102] Step 5: According to the simulated trajectory, control the sweeping robot to retreat to the charging compartment of the base station to complete the recharging.

[0103] Through this optional example, three evenly distributed image acquisition components are used to quickly map the area where the mobile device is located to complete the construction of a panoramic image without exploring routes based on signals. This can effectively solve the collision problem caused by the mobile device retreating in a small environment. At the same time, when the mobile device reaches the specified position near the base station, the posture of the sweeping robot is adjusted so that the simulated trajectory of the sweeping robot coincides with the "reversing" trajectory. This can improve the success rate and efficiency of recharging the mobile device.

[0104] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM (Read-Only Memory) / RAM (Random Access Memory), a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0106] According to another aspect of an embodiment of the present application, a control device of a mobile device for implementing the above detection method is also provided. Figure 7is a structural block diagram of an optional control device of a mobile device according to an embodiment of the present application, such as Figure 7 As shown, the device may include:

[0107] A construction unit 702, configured to construct a panoramic image of the area where the mobile device is located based on the images collected by the image collection component on the mobile device;

[0108] The first determining unit 704 is connected to the constructing unit 702 and is used to determine a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image when the mobile device needs to be charged, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device;

[0109] The control unit 706 is connected to the first determination unit 704 and is used to control the mobile device to enter the charging docking position according to the simulated trajectory route.

[0110] It should be noted that the construction unit 702 in this embodiment can be used to execute the above step S202, the first determination unit 704 in this embodiment can be used to execute the above step S204, and the control unit 706 in this embodiment can be used to execute the above step S206.

[0111] Through the above module, a panoramic image of the area where the mobile device is located is constructed based on the images collected by the image acquisition component on the mobile device; when the mobile device needs to be charged, a simulated trajectory route of the mobile device from the current position to the target position is determined in the panoramic image, wherein the target position is the charging docking position of the base station, and the base station is used to charge the mobile device; controlling the mobile device to enter the charging docking position according to the simulated trajectory route can effectively improve the recharging efficiency of the mobile device.

[0112] In an exemplary embodiment, the number of the image acquisition components is at least three, and the at least three image acquisition components are evenly distributed on the side wall of the mobile device, and the construction unit includes:

[0113] The construction module is used to construct a panoramic image of the area where the mobile device is located based on a group of images collected by at least three image collection components on the mobile device.

[0114] In an exemplary embodiment, the determining unit includes:

[0115] An acquisition module, used to acquire the current position of the mobile device in the panoramic image when the mobile device needs to be charged;

[0116] A generation module is used to generate a simulated trajectory route of a mobile device from a current position to a charging docking position according to a current position and a track line on a base of a base station, wherein at least a portion of the charging docking position is defined by the track line, and a portion of the route in the simulated trajectory route coincides with the track line.

[0117] In an exemplary embodiment, the above device further comprises:

[0118] The second determination unit is used to determine a simulated trajectory route of the mobile device from the current position to the target position in the panoramic image, and when the distance between the position of the mobile device and the position of the obstacle shown in the panoramic image is less than a preset distance threshold, the simulated trajectory route is updated in combination with the distance between the obstacle and the mobile device, and the target position to obtain a target simulated trajectory route, wherein in the target simulated trajectory route, the distance between the mobile device and the obstacle is greater than the preset distance threshold.

[0119] In an exemplary embodiment, the simulated trajectory route includes a first trajectory route and a second trajectory route, the first trajectory route is a route for guiding the mobile device to move from a current position to a preset position, and the second trajectory route is a route for guiding the mobile device from the preset position to a target position, the preset position is a position at a preset distance from the front side of the entrance of the charging docking position to the entrance,

[0120] The control unit includes:

[0121] A first control module, used for controlling the mobile device to move from a current position to a preset position according to a first trajectory route;

[0122] A display module, for displaying a movement track of the mobile device generated based on the location of the charging interface of the mobile device in the panoramic image when it is determined that the mobile device has moved to a preset location;

[0123] The second control module is used to control the mobile device to enter the charging docking position based on the moving track in the panoramic image and the second track route.

[0124] In an exemplary embodiment, the second control module includes:

[0125] The first control submodule is used to control the mobile device to enter the charging docking position according to the second trajectory route when the overlap between the moving trajectory in the panoramic image and the second trajectory route is greater than a preset threshold.

[0126] In an exemplary embodiment, the second control module includes:

[0127] A determination submodule, configured to determine a target angle between the moving trajectory and the second trajectory route when the overlap between the moving trajectory in the panoramic image and the second trajectory route is less than or equal to a preset threshold;

[0128] An adjustment submodule, used to adjust the posture of the mobile device according to the target angle until the overlap between the moving trajectory of the mobile device and the second trajectory route is greater than a preset threshold;

[0129] The second control submodule is used to control the mobile device to enter the charging docking position according to the second trajectory route when it is determined that the posture adjustment of the mobile device is completed.

[0130] It should be noted that the examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the contents disclosed in the above embodiments. It should be noted that the above modules as part of the device can be run in Figure 1 In the hardware environment shown, it can be implemented by software or by hardware, wherein the hardware environment includes a network environment.

[0131] According to another aspect of the embodiments of the present application, a storage medium is further provided. Optionally, in this embodiment, the storage medium can be used to execute the program code of any of the above-mentioned methods for controlling a mobile device in the embodiments of the present application.

[0132] Optionally, in this embodiment, the storage medium may be located on at least one network device among a plurality of network devices in the network shown in the above embodiment.

[0133] Optionally, in this embodiment, the storage medium is configured to store program codes for executing the following steps:

[0134] S1, constructing a panoramic image of the area where the mobile device is located based on the images collected by the image collection component on the mobile device;

[0135] S2, when the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device;

[0136] S3, controlling the mobile device to enter a charging docking position according to the simulated trajectory route.

[0137] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, which will not be described in detail in this embodiment.

[0138] Optionally, in this embodiment, the storage medium may include, but is not limited to, various media that can store program codes, such as a USB flash drive, a ROM, a RAM, a mobile hard disk, a magnetic disk, or an optical disk.

[0139] According to another aspect of an embodiment of the present application, an electronic device for implementing the control method of the above-mentioned mobile device is also provided. The electronic device may be a server, a terminal, or a combination thereof.

[0140] Figure 8 is a structural block diagram of an optional electronic device according to an embodiment of the present application, such as Figure 8 As shown, it includes a processor 802, a communication interface 804, a memory 806 and a communication bus 808, wherein the processor 802, the communication interface 804 and the memory 806 communicate with each other through the communication bus 808, wherein,

[0141] Memory 806, used for storing computer programs;

[0142] The processor 802 is used to implement the following steps when executing the computer program stored in the memory 806:

[0143] S1, constructing a panoramic image of the area where the mobile device is located based on the images collected by the image collection component on the mobile device;

[0144] S2, when the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device;

[0145] S3, controlling the mobile device to enter a charging docking position according to the simulated trajectory route.

[0146] Optionally, in this embodiment, the communication bus may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The communication bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 The communication interface is used for communication between the electronic device and other devices.

[0147] The above-mentioned memory may include RAM, or may include non-volatile memory (non-volatile memory), for example, at least one disk memory. Optionally, the memory may also be at least one storage device located far away from the above-mentioned processor.

[0148] As an example, the memory 806 may include, but is not limited to, the construction unit 702, the first determination unit 704, and the control unit 706 in the control device of the mobile device. In addition, it may also include, but is not limited to, other module units in the control device of the mobile device, which will not be repeated in this example.

[0149] The above-mentioned processor can be a general-purpose processor, which can include but not be limited to: CPU (Central Processing Unit), NP (Network Processor), etc.; it can also be DSP (Digital Signal Processing), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0150] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0151] According to another aspect of the embodiment of the present application, a cleaning device for implementing the control method of the mobile device is also provided. The cleaning device comprises: a cleaning robot and a base station, the cleaning robot comprises an image acquisition component, and at least one of the cleaning robot and the base station comprises a processing component, wherein:

[0152] An image acquisition component is used to acquire images; the base station includes a charging docking position, wherein the base station is used to charge the cleaning robot;

[0153] The processing component is used to construct a panoramic image of the area where the cleaning robot is located based on the images collected by the image acquisition component; when the cleaning robot needs to be charged, determine a simulated trajectory route of the cleaning robot from the current position to the target position in the panoramic image, wherein the target position is the charging docking position; and control the cleaning robot to enter the charging docking position according to the simulated trajectory route.

[0154] Optionally, the specific examples in this embodiment may refer to the examples described in the above embodiments, and this embodiment will not be described in detail here.

[0155] It can be understood by those skilled in the art that Figure 8The structure shown is for illustration only. The device for implementing the above-mentioned control method of the mobile device may be a terminal device, which may be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (Mobile Internet Devices, MID), a PAD, and other terminal devices. Figure 8 The structure of the electronic device is not limited. Figure 8 More or fewer components (such as network interfaces, display devices, etc.) shown in, or having Figure 8 Different configurations shown.

[0156] A person of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and the program can be stored in a computer-readable storage medium, which can include: a flash drive, ROM, RAM, a magnetic disk or an optical disk, etc.

[0157] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0158] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0159] In the above embodiments of the present application, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.

[0160] In the several embodiments provided in the present application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0161] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution provided in this embodiment.

[0162] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0163] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for controlling a mobile device, characterized in that: include: Constructing a panoramic image of the area where the mobile device is located based on the images collected by the image collection component on the mobile device; In the case where the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image, wherein the target position is a charging docking position of a base station, and the base station is used to charge the mobile device; The mobile device is controlled to enter the charging docking position according to the simulated trajectory route.

2. The method according to claim 1, characterized in that The number of the image acquisition components is at least three, and the at least three image acquisition components are evenly distributed on the side wall of the mobile device, wherein the panoramic image of the area where the mobile device is located is constructed based on the images acquired by the image acquisition components on the mobile device, including: A panoramic image of the area where the mobile device is located is constructed based on at least one group of images collected by at least three image collection components on the mobile device.

3. The method according to claim 1, characterized in that When the mobile device needs to be charged, determining a simulated trajectory route of the mobile device from a current position to a target position in the panoramic image includes: When the mobile device needs to be charged, obtaining the current position of the mobile device in the panoramic image; The simulated trajectory route of the mobile device from the current position to the charging docking position is generated based on the current position and the track line on the base of the base station, wherein at least part of the charging docking position is defined by the track line, and part of the route in the simulated trajectory route coincides with the track line.

4. The method according to claim 1, characterized in that After determining the simulated trajectory route of the mobile device from the current position to the target position in the panoramic image, the method further includes: When the distance between the position of the mobile device and the position of the obstacle shown in the panoramic image is less than a preset distance threshold, the simulated trajectory route is updated in combination with the distance between the obstacle and the mobile device and the target position to obtain a target simulated trajectory route, wherein, in the target simulated trajectory route, the distance between the mobile device and the obstacle is greater than the preset distance threshold.

5. The method according to claim 1, characterized in that: The simulated trajectory route includes a first trajectory route and a second trajectory route, wherein the first trajectory route is a route for guiding the mobile device to move from the current position to a preset position, and the second trajectory route is a route for guiding the mobile device from the preset position to the target position, wherein the preset position is a position at a preset distance from the front side of the entrance of the charging docking position to the entrance, The step of controlling the mobile device to enter the charging docking position according to the simulated trajectory route includes: Control the mobile device to move from the current position to the preset position according to the first trajectory route; When it is determined that the mobile device has moved to the preset position, displaying a movement track of the mobile device generated based on the position of the charging port of the mobile device in the panoramic image; Based on the moving track and the second track route in the panoramic image, the mobile device is controlled to enter the charging docking position.

6. The method according to claim 5, characterized in that The controlling the mobile device to enter the charging docking position based on the moving track and the second track route in the panoramic image includes: When the overlap between the moving track in the panoramic image and the second track route is greater than a preset threshold, the mobile device is controlled to enter the charging docking position according to the second track route.

7. The method according to claim 5, characterized in that The controlling the mobile device to enter the charging docking position based on the moving track and the second track route in the panoramic image includes: When the overlap between the moving track and the second track route in the panoramic image is less than or equal to a preset threshold, determining a target angle between the moving track and the second track route; According to the target angle, adjusting the posture of the mobile device until the overlap between the moving trajectory of the mobile device and the second trajectory route is greater than the preset threshold; When it is determined that the posture adjustment of the mobile device is completed, the mobile device is controlled to enter the charging docking position according to the second trajectory route.

8. A control device for a mobile device, characterized in that: include: A construction unit, configured to construct a panoramic image of the area where the mobile device is located based on the image captured by the image acquisition component on the mobile device; a determination unit, configured to determine, in the panoramic image, a simulated trajectory route of the mobile device from a current position to a target position when the mobile device needs to be charged, wherein the target position is a charging docking position of a base station, and the base station is configured to charge the mobile device; A control unit is used to control the mobile device to enter the charging docking position according to the simulated trajectory route.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 7 when executed.

10. An electronic device comprising a memory and a processor, characterized in that: A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 7 through the computer program.

11. A cleaning device, characterized in that: include: A cleaning robot and a base station, wherein the cleaning robot comprises an image acquisition component, and at least one of the cleaning robot and the base station comprises a processing component, wherein: The image acquisition component is used to acquire images; The base station includes a charging docking position, wherein the base station is used to charge the cleaning robot; The processing component is used to construct a panoramic image of the area where the cleaning robot is located based on the image captured by the image acquisition component; when the cleaning robot needs to be charged, determine a simulated trajectory route of the cleaning robot from a current position to a target position in the panoramic image, wherein the target position is the charging docking position; and control the cleaning robot to enter the charging docking position according to the simulated trajectory route.

Images