Control method of mobile robot, controller, mobile robot and storage medium

By integrating a single camera and lidar on the intelligent sweeping robot, a 360-degree panoramic image is generated, the problem of insufficient environmental perception ability in the existing technology is solved, and the provision of comprehensive home visual information and the integration of smart home systems is realized, and users are enhanced. Experience and equipment work together.

CN118938896APending Publication Date: 2024-11-12MIDEA ROBOZONE TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410916968.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing smart sweeping robots have limited capabilities in environmental perception and information collection and are unable to provide comprehensive and continuous visual information of the home environment, limiting their potential in home monitoring and remote management applications.

Method used

By integrating single cameras and high-precision lidar on mobile robots, using rotary capture of multi-angle images combined with image matching and position optimization technology, a 360-degree panoramic image is generated, and an immersive home environment browsing experience is provided through smart home system integration Work in conjunction with the equipment.

Benefits of technology

It improves the environmental perception ability of mobile robots, provides comprehensive and continuous visual information of the home environment, enhances user operation convenience and satisfaction, and realizes collaborative work and data sharing among smart devices.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118938896A_ABST
    Figure CN118938896A_ABST
Patent Text Reader

Abstract

The invention provides a control method of a mobile robot, a controller, the mobile robot and a storage medium, the mobile robot is provided with a camera and a laser radar, and the method comprises the steps that firstly, the mobile robot is controlled to drive the camera to rotate by a preset angle for multiple times in the axial direction of a horizontal plane; secondly, capturing a plurality of environment images at different angles through a camera at a rotation stop position; and finally, generating a panoramic image according to the pose data detected by the laser radar and the environment images at multiple different angles. The mobile robot can drive the camera to rotate by the preset angle for multiple times in the axial direction of the horizontal plane, so that the camera captures the environment images of multiple different angles, and the panoramic image is generated through the pose data detected by the laser radar and the environment images of multiple different angles, and therefore, the panoramic image processing efficiency is improved. The comprehensive environment perception capability of the mobile robot can be improved, and the panoramic image is generated, so that comprehensive and continuous family environment visual information is provided for a user.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of intelligent robots, and in particular to a control method, a controller, a mobile robot and a storage medium for a mobile robot. Background Art

[0002] In related technologies, the functionality and interoperability of smart home devices have been greatly improved with the development of artificial intelligence, machine learning, robotics, and communication technologies. In the field of smart sweeping robots, by introducing advanced navigation systems, multi-camera image capture, and deep learning optimized image processing technology, these devices can perform environmental production measurement and data collection while performing cleaning tasks. However, despite the significant progress made in technology, there are still several key technical and application challenges that need to be addressed.

[0003] First, existing smart sweeping robots are still limited in their ability to perceive the environment and collect information. Although some high-end models are equipped with cameras and other sensors, they are mainly used for navigation and obstacle avoidance rather than for comprehensive monitoring of the home environment. This means that users cannot obtain comprehensive visual information about the home space, limiting the robot's potential for application in home monitoring, remote home management, and so on.

[0004] Secondly, although some sweeping robot models allow users to view images captured by the robot at a specific moment through mobile applications, these images are usually snapshots of fixed angles and directions and cannot provide a complete spatial view; in addition, users are required to manually control the sweeping robot to cruise through video cruising, etc., which makes it difficult for users to quickly and completely understand the home environment, especially when remote monitoring of the home is required. Summary of the invention

[0005] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a control method of a mobile robot, a controller, a mobile robot and a storage medium, aiming to improve the comprehensive environmental perception capability of the mobile robot and generate a panoramic image, thereby providing users with comprehensive and continuous visual information of the home environment.

[0006] In a first aspect, an embodiment of the present application provides a control method for a mobile robot, wherein the mobile robot is provided with a camera and a laser radar, and the control method comprises:

[0007] Controlling the mobile robot to drive the camera to rotate along the axis of the horizontal plane for multiple times by a preset angle;

[0008] At the rotation stop position, the camera is used to capture multiple environmental images at different angles;

[0009] A panoramic image is generated based on the posture data detected by the laser radar and the environment images at multiple different angles.

[0010] According to some embodiments of the present application, before controlling the mobile robot to rotate at intervals at a preset angle, the control method further includes:

[0011] Acquire an environmental map, and control the mobile robot to cruise along a preset planned path according to the environmental map.

[0012] According to some embodiments of the present application, before acquiring the environment map, one of the following is included:

[0013] Controlling the laser radar to scan the indoor environment to obtain environmental perception information, generating the environmental map according to the environmental perception information and determining the current position information of the mobile robot;

[0014] The sensor module is controlled to scan the indoor environment to obtain environmental perception information, generate the environmental map according to the environmental perception information and determine the current position information of the mobile robot, wherein the sensor module includes a light sensor, an ultrasonic sensor and / or an infrared sensor.

[0015] According to some embodiments of the present application, the control method further includes:

[0016] The positioning position of the mobile robot in the environmental map is adjusted according to the plurality of environmental images continuously captured by the camera, wherein the number of the cameras is 1.

[0017] According to some embodiments of the present application, adjusting the positioning position of the mobile robot in the environmental map according to the plurality of environmental images continuously captured by the camera includes:

[0018] Acquire common feature points of adjacent environment images;

[0019] The position of the mobile robot is adjusted according to the common feature points and the position and posture data.

[0020] According to some embodiments of the present application, the common feature points are obtained by the following steps:

[0021] A feature point matching algorithm is used to determine the common feature points of two adjacent environment images.

[0022] According to some embodiments of the present application, the control method further includes:

[0023] When the panoramic image is generated, the panoramic image is adjusted and optimized.

[0024] According to some embodiments of the present application, the adjusting and optimizing the panoramic image includes at least one of the following:

[0025] Performing distortion correction on the panoramic image;

[0026] Performing seam optimization on the panoramic image;

[0027] performing dynamic range processing on the panoramic image;

[0028] A color balance adjustment is performed on the panoramic image.

[0029] According to some embodiments of the present application, the control method further includes:

[0030] Acquire the communication protocol of the mobile robot, and connect with the intelligent device according to the communication protocol;

[0031] Receive control instructions from a user terminal device, control the mobile robot to perform a corresponding target task according to the control instructions, and control the smart device to perform a corresponding linkage operation.

[0032] In a second aspect, an embodiment of the present application provides a controller comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the control method of the mobile robot as described in the first aspect above when running the computer program.

[0033] In a third aspect, an embodiment of the present application provides a mobile robot comprising a controller as described in the second aspect.

[0034] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the control method of a mobile robot as described in the first aspect above.

[0035] According to the technical solution of the embodiment of the present application, at least the following beneficial effects are achieved: the embodiment of the present application proposes a control method, a controller, a mobile robot and a storage medium for a mobile robot, wherein the mobile robot is provided with a camera, wherein the method comprises: first, controlling the mobile robot to drive the camera to rotate multiple times along the axis of the horizontal plane by a preset angle; second, capturing multiple environmental images at different angles through the camera at the rotation stop position; finally, generating a panoramic image based on the posture data detected by the laser radar and the environmental images at multiple different angles. Since the mobile robot of the present application can drive the camera to rotate multiple times along the axis of the horizontal plane by a preset angle, thereby capturing multiple environmental images at different angles through the camera, and then generating a panoramic image through the posture data detected by the laser radar and the environmental images at multiple different angles, the present application can improve the comprehensive environmental perception capability of the mobile robot and generate a panoramic image, thereby providing users with comprehensive and continuous visual information of the home environment.

[0036] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings are used to provide further understanding of the technical solution of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the technical solution of the present application and do not constitute a limitation on the technical solution of the present application.

[0038] Figure 1 is a flow chart of a control method for a mobile robot provided by an embodiment of the present application;

[0039] Figure 2 is a flow chart of a control method for a mobile robot provided by another embodiment of the present application;

[0040] Figure 3 is a flow chart of a control method for a mobile robot provided by another embodiment of the present application;

[0041] Figure 4 is an overall flow chart of a control method for a mobile robot provided by an embodiment of the present application;

[0042] Figure 5 is an overall flow chart of a control method for a mobile robot provided by another embodiment of the present application;

[0043] Figure 6 is a structural schematic diagram of a navigation and multi-angle image capture system provided by an embodiment of the present application;

[0044] Figure 7 is a structural diagram of an image matching and posture optimization module provided by an embodiment of the present application;

[0045] Figure 8 It is a structural schematic diagram of a panoramic image generation and optimization processing module provided by an embodiment of the present application;

[0046] Fig. 9 It is a structural diagram of a smart home system integration module provided by an embodiment of the present application;

[0047] Fig.10 It is a schematic diagram of a controller for executing a control method for a mobile robot provided in one embodiment of the present application. DETAILED DESCRIPTION

[0048] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0049] In the description of the present application, it should be understood that descriptions involving orientation, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0050] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0051] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0052] In some cases, the functionality and interoperability of smart home devices have been greatly improved with the development of artificial intelligence, machine learning, robotics and communication technologies. In the field of smart sweeping robots, by introducing advanced navigation systems, multi-camera image capture and deep learning optimized image processing technology, these devices can perform environmental production and data collection while performing cleaning tasks. However, despite the significant progress made in technology, there are still several key technical and application challenges that need to be solved.

[0053] First, existing smart sweeping robots are still limited in their ability to perceive the environment and collect information. Although some high-end models are equipped with cameras and other sensors, they are mainly used for navigation and obstacle avoidance rather than for comprehensive monitoring of the home environment. This means that users cannot obtain comprehensive visual information about the home space, limiting the robot's potential for application in home monitoring, remote home management, and so on.

[0054] Secondly, although some sweeping robot models allow users to view images captured by the robot at a specific moment through mobile applications, these images are usually snapshots of fixed angles and directions and cannot provide a complete spatial view; in addition, users are required to manually control the sweeping robot to cruise through video cruising, etc., which makes it difficult for users to quickly and completely understand the home environment, especially when remote monitoring of the home is required.

[0055] Secondly, current sweeping robots also fall short in providing an interactive home environment browsing experience. Although some products try to provide real-time video streaming capabilities through integrated cameras, there is a lack of an immersive, user-friendly way for users to explore and monitor the home environment in a more natural way.

[0056] Thirdly, existing sweeping robots are generally relatively basic in terms of integration with smart home systems, mainly limited to connecting to the home network via Wi-Fi and being controlled through a dedicated mobile phone application. This limited level of integration limits the potential role of the device in the smart home ecosystem, especially in achieving device collaboration, data sharing, and scene linkage.

[0057] Based on the above situation, the present application proposes a control method, controller, mobile robot and storage medium for a mobile robot, aiming to improve the comprehensive environmental perception capability of the mobile robot and generate panoramic images, thereby providing users with comprehensive and continuous visual information of the home environment.

[0058] The following is a further description of various embodiments of the control method of the mobile robot of the present application in conjunction with the accompanying drawings.

[0059] like Figure 1 As shown, Figure 1 It is a flowchart of a control method for a mobile robot provided by an embodiment of the present application; the control method includes but is not limited to step S110, step S120 and step S130.

[0060] Step S110, controlling the mobile robot to drive the camera to rotate along the axis of the horizontal plane for multiple times by a preset angle;

[0061] Step S120, capturing multiple environmental images at different angles through a camera at the rotation stop position;

[0062] Step S130: Generate a panoramic image based on the posture data detected by the laser radar and multiple environmental images at different angles.

[0063] In one embodiment, first, the embodiment of the present application will control the mobile robot to drive the camera to rotate multiple times along the axis of the horizontal plane to a preset angle; then, the embodiment of the present application will control the mobile robot to stop at the rotation stop position, thereby capturing environmental images at multiple different angles through the camera; finally, the embodiment of the present application will generate a panoramic image based on the posture data detected by the lidar and the environmental images at multiple different angles captured by the camera.

[0064] It should be noted that in the embodiment of the present application, the system can generate a panoramic image through the posture data detected by the lidar and the environmental images from multiple different angles captured by the camera, so that a more intuitive and comprehensive view of the environment can be obtained through the panoramic image, which in turn helps the user or mobile robot to better understand the environmental layout and obstacle distribution, and provide strong support for the mobile robot's navigation, obstacle avoidance and other tasks.

[0065] It should be noted that the embodiment of the present application can provide users with a more intuitive and vivid environment display by generating panoramic images. Users can understand the situation of the environment in which the mobile robot is located more clearly through the panoramic images, so as to better interact and collaborate with the mobile robot.

[0066] It should be noted that the embodiment of the present application controls the mobile robot to drive the camera to rotate multiple times along the axis of the horizontal plane to preset angles, and captures environmental images at multiple angles at the rotation stop position to generate a panoramic image, thereby being able to adapt to various complex environments. Whether it is an open or narrow space indoors, the system can effectively acquire environmental images and generate panoramic images, thereby providing strong support for the mobile robot's active navigation and obstacle avoidance.

[0067] It is worth noting that since the mobile robot of the embodiment of the present application can drive the camera to rotate multiple times along the axis of the horizontal plane to a preset angle, the camera can be used to capture environmental images at multiple different angles, and then a panoramic image can be generated through the posture data detected by the lidar and environmental images at multiple different angles. Therefore, the embodiment of the present application can improve the comprehensive environmental perception capability of the mobile robot and generate panoramic images, thereby providing users with comprehensive and continuous visual information of the home environment.

[0068] Exemplarily, a synchronous rotation control algorithm is used to ensure the stability of the mobile robot when it is in a rotation stop position to capture a clear image of the environment.

[0069] Exemplarily, an efficient stitching algorithm is used to automatically process and fuse multiple environmental images from different angles to generate a panoramic image.

[0070] It can be understood that the posture data monitored by the lidar can ensure that multiple environmental images from different angles can be accurately stitched together, thereby improving the quality and accuracy of the panoramic image.

[0071] It can be understood that the above-mentioned preset angle can be set by the user according to actual conditions, and can be set by a mobile robot. The embodiment of the present application does not specifically limit the size of the preset angle.

[0072] In one embodiment, before the above step S110, the following step S111 is further included:

[0073] Step S111: Obtain an environmental map, and control the mobile robot to cruise along a preset planned path according to the environmental map.

[0074] It should be noted that the embodiment of the present application controls the mobile robot to cruise along a preset planned path, and controls the mobile robot to drive the camera to rotate multiple times along the axis of the horizontal plane by preset angles, thereby capturing environmental images at multiple different angles at the rotation stop position. The system can then obtain more comprehensive and detailed environmental information, and help the mobile robot to more accurately understand its surrounding environment and improve its environmental perception capabilities.

[0075] It can be understood that the above-mentioned preset planned path can be set by the user or by the mobile robot itself, and the embodiment of the present application does not specifically limit the method for setting the preset planned path.

[0076] Illustratively, in the embodiments of the present application, the user sets a preset angle and a preset planned path according to actual conditions. Then, the mobile robot cruises along the preset planned path according to the environmental map. Then, the mobile robot drives the camera to rotate multiple times along the axis of the horizontal plane by a preset angle and stop at the rotation stop position, thereby capturing environmental images at multiple different angles through the camera, and then generating a panoramic image based on the posture data detected by the lidar and the environmental images at multiple different angles. Therefore, in the embodiments of the present application, the user sets a preset angle and a preset planned path, so that the mobile robot can autonomously determine the shooting angle and shooting position according to the preset angle and the preset planned path, and then obtain environmental images at multiple different angles, thereby generating a panoramic image.

[0077] Illustratively, the mobile robot of the embodiment of the present application can autonomously set a preset angle and a preset planned path according to the environmental map. Then, the mobile robot will cruise along the preset planned path according to the environmental map. Then, the mobile robot will drive the camera to rotate multiple times along the axis of the horizontal plane by a preset angle and stop at the rotation stop position, thereby capturing environmental images at multiple different angles through the camera, and then generating a panoramic image based on the posture data detected by the lidar and the environmental images at multiple different angles. Therefore, the mobile robot of the embodiment of the present application can autonomously set the preset angle and the preset planned path, so that it can autonomously determine the shooting angle and shooting position, and then obtain environmental images at multiple different angles, thereby generating a panoramic image.

[0078] In one embodiment, before obtaining the environment map in the above step S111, the following step S1111 is also included.

[0079] Step S1111: control the laser radar to scan the indoor environment to obtain environmental perception information, generate an environmental map based on the environmental perception information, and determine the current position information of the mobile robot.

[0080] Exemplarily, before acquiring the environmental map, the embodiment of the present application controls the laser radar to scan the indoor environment to obtain environmental perception information, thereby obtaining the environmental map based on the environmental perception information and determining the current position information of the mobile robot.

[0081] It should be noted that by scanning the indoor environment through LiDAR, the distance, angle and other characteristic quantities of the target object can be accurately measured, providing high-precision environmental perception information; in addition, LiDAR can cover a wide scanning range, conduct a comprehensive scan of the indoor environment, and capture detailed information on various obstacles, walls, doors, windows and other objects, thereby making the generated environmental map more comprehensive and detailed; in addition, based on the environmental perception information obtained by LiDAR, the mobile robot can more accurately determine its own position and direction in the environment, thereby providing strong support for the subsequent cruising and obstacle avoidance of the mobile robot.

[0082] In one embodiment, before obtaining the environment map in the above step S111, the following step S1112 is also included.

[0083] Step S1112: control the sensor module to scan the indoor environment to obtain environmental perception information, generate an environmental map according to the environmental perception information and determine the current position information of the mobile robot, wherein the sensor module includes a light sensor, an ultrasonic sensor and / or an infrared sensor.

[0084] Illustratively, before acquiring the environment map, the embodiment of the present application controls the sensor module to scan the indoor environment to obtain environment perception information, thereby obtaining the environment map based on the environment perception information and determining the current position information of the mobile robot.

[0085] It should be noted that by scanning the indoor environment through the sensor module, the distance, angle and other characteristic quantities of the target object can be accurately measured, providing high-precision environmental perception information; in addition, the sensor module can cover a wide scanning range, comprehensively scan the indoor environment, and capture detailed information of various obstacles, walls, doors, windows and other objects, thereby making the generated environmental map more comprehensive and detailed; in addition, based on the environmental perception information obtained by the lidar, the mobile robot can more accurately determine its own position and direction in the environment, thereby providing strong support for the subsequent cruising and obstacle avoidance of the mobile robot.

[0086] It can be understood that the number of the ultrasonic sensor and infrared sensor can be one, two, three, or more, and the embodiment of the present application does not specifically limit the number of ultrasonic sensors and infrared sensors.

[0087] In one embodiment, the control method of the mobile robot in the embodiment of the present application further includes but is not limited to step S210.

[0088] Step S210: adjusting the positioning position of the mobile robot in the environment map according to a plurality of environment images continuously captured by a camera, wherein the number of cameras is 1.

[0089] Illustratively, in the process of capturing multiple environmental images at different angles, the mobile robot of the embodiment of the present application adjusts the positioning position of the mobile robot in the environmental map according to the multiple environmental images captured continuously.

[0090] It should be noted that the embodiment of the present application can ensure that the mobile robot has more accurate viewing angle and position information when capturing continuous images by adjusting the positioning position of the mobile robot in the environmental map, which helps to reduce image misalignment and distortion caused by viewing angle offset or position error, thereby improving the quality and accuracy of image stitching.

[0091] like Figure 2 As shown, Figure 2 It is a flow chart of a control method for a mobile robot provided in another embodiment of the present application; regarding the above-mentioned step S210, it includes but is not limited to step S310 and step S320.

[0092] Step S310, obtaining common feature points of adjacent environment images;

[0093] Step S320: Adjust the position of the mobile robot according to the common feature points and posture data.

[0094] In one embodiment, the mobile robot of the embodiment of the present application obtains common feature points of adjacent environmental images during the process of capturing environmental images at multiple different angles, thereby adjusting the position of the mobile robot based on the common feature points and posture data.

[0095] It should be noted that the embodiment of the present application adjusts the positioning position of the mobile robot in the environmental map through the common feature points and posture data of adjacent environmental images, which can ensure that the mobile robot has more accurate viewing angle and position information when capturing continuous images, and helps to reduce image misalignment and distortion caused by viewing angle offset or position error, thereby improving the quality and accuracy of image stitching.

[0096] It should be noted that the embodiment of the present application can determine the relative position and angle relationship between images by comparing and analyzing the common feature points of adjacent images. Combined with the position data of the mobile robot, the position of the mobile robot can be adjusted more accurately to ensure that subsequently captured images can be accurately spliced ​​with existing images, thereby improving the quality and accuracy of the panoramic image.

[0097] In one embodiment, the common feature points are obtained through the following step S410.

[0098] Step S410: using a feature point matching algorithm to determine the common feature points of two adjacent environment images.

[0099] It should be noted that by adopting a feature point algorithm to obtain common feature points of two adjacent environmental images, seamless stitching between adjacent images can be ensured, thereby improving the integrity and accuracy of the panoramic image.

[0100] In one embodiment, the control method of the mobile robot in the embodiment of the present application further includes but is not limited to step S510.

[0101] Step S510: When a panoramic image is generated, the panoramic image is adjusted and optimized.

[0102] Illustratively, in the process of generating a panoramic image based on multiple environmental images at different angles, the embodiment of the present application adjusts and optimizes the panoramic image.

[0103] It should be noted that the embodiments of the present application can ensure a natural transition of the panoramic image and optimize the visual effect of the panoramic image by adjusting and optimizing the panoramic image.

[0104] Exemplarily, adjusting and optimizing the panoramic image may include at least one of the following steps: performing distortion correction on the panoramic image; performing seam optimization on the panoramic image; performing dynamic range processing on the panoramic image; and performing color balance adjustment on the panoramic image.

[0105] It should be noted that when the panoramic image is subjected to distortion correction and seam optimization, the natural transition of the panoramic image can be ensured; when the panoramic image is subjected to dynamic range processing and color balance adjustment, the visual effect of the panoramic image can be optimized.

[0106] like Figure 3 As shown, Figure 3 This is a flowchart of a mobile robot control method provided by another embodiment of the present application; the mobile robot control method also includes but is not limited to step S610 and step S620.

[0107] Step S610: Obtain the communication protocol of the mobile robot, and connect with the smart device according to the communication protocol;

[0108] Step S620: Receive a control instruction from the user terminal device, control the mobile robot to perform the corresponding target task according to the control instruction, and control the smart device to perform the corresponding linkage operation.

[0109] In one embodiment, first, the embodiment of the present application obtains the communication protocol of the mobile robot and connects with the smart device through the communication protocol; then, when the user performs operation control on the user terminal device, the embodiment of the present application receives the control instructions from the user terminal device, and executes the corresponding target task according to the control instructions, thereby controlling the smart device to perform the corresponding linkage operation.

[0110] For example, when the mobile robot performs cleaning tasks according to control instructions, the intelligent device is controlled to adjust the lighting, so that appropriate lighting can ensure that the mobile robot can more accurately identify and locate areas that need to be cleaned, thereby improving cleaning efficiency. In addition, proper lighting can ensure that the mobile robot can clearly see the surrounding environment and avoid collisions with obstacles or falls.

[0111] It should be noted that when the mobile robot is connected to the smart device, the user can monitor and control the mobile robot and the smart device on the user terminal device, thereby simplifying the management process and improving the user experience; in addition, by controlling the mobile robot to perform the corresponding target task and controlling the smart device to perform the corresponding linkage operation according to the control instructions, the automation level of the smart home can be improved; in addition, through the user terminal device, it is possible to browse and experience an immersive home environment, and realize collaborative work and data sharing between smart devices, which greatly improves the user's operational convenience and satisfaction.

[0112] It can be understood that the above-mentioned user terminal device can be a mobile phone or other devices. The panoramic image of the mobile robot can be viewed remotely through the user terminal device to provide the user with comprehensive and continuous visual information of the home environment; in addition, the mobile robot and smart devices can be monitored and controlled through the user terminal to enhance the user experience; the embodiment of the present application does not specifically limit the type of user terminal device.

[0113] It should be noted that the mobile robot may have a cleaning function, that is, the mobile robot may be a sweeping robot; or, the mobile robot may have a mopping function, that is, the mobile robot may be a mopping robot; or, the mobile robot may have both cleaning and mopping functions, that is, the mobile robot may be an integrated sweeping and mopping robot; or, the mobile robot may be an electronic pet, such as a movable electronic pet dog; the mobile robot may also be in other device forms, which are not specifically limited in the embodiments of the present application.

[0114] It can be understood that the above-mentioned communication protocol can adopt a variety of existing smart home standards and protocols, such as Zigbee, Z-Wave, Wi-Fi, etc. The most suitable communication protocol can be selected according to actual application requirements and device compatibility. The embodiment of the present application does not specifically limit the type of communication protocol.

[0115] like Figure 4 As shown, Figure 4 This is an overall flow chart of a mobile robot control method provided by an embodiment of the present application; the process includes but is not limited to the following steps:

[0116] Step S710: Control the laser radar to scan the indoor environment to obtain environmental perception information, generate an environmental map according to the environmental perception information, and determine the current position information of the mobile robot;

[0117] Step S720: Obtain an environment map, and cruise along a preset planned path according to the environment map;

[0118] Step S730, controlling the mobile robot to drive the camera to rotate along the axis of the horizontal plane for multiple times by a preset angle;

[0119] Step S740, capturing multiple environmental images at different angles through a camera at the rotation stop position;

[0120] Step S750, obtaining common feature points of adjacent environment images and position data detected by the laser radar;

[0121] Step S760, adjusting the position of the mobile robot according to the common feature points and the position and posture data;

[0122] Step S770: Generate a panoramic image based on multiple environment images at different angles.

[0123] Illustratively, in the process of generating a panoramic image based on multiple environmental images at different angles, the embodiment of the present application adjusts and optimizes the panoramic image.

[0124] It should be noted that the embodiments of the present application can ensure a natural transition of the panoramic image and optimize the visual effect of the panoramic image by adjusting and optimizing the panoramic image.

[0125] Exemplarily, adjusting and optimizing the panoramic image may include at least one of the following steps: performing distortion correction on the panoramic image; performing seam optimization on the panoramic image; performing dynamic range processing on the panoramic image; and performing color balance adjustment on the panoramic image.

[0126] It should be noted that when the panoramic image is subjected to distortion correction and seam optimization, the natural transition of the panoramic image can be ensured; when the panoramic image is subjected to dynamic range processing and color balance adjustment, the visual effect of the panoramic image can be optimized.

[0127] Exemplarily, the embodiment of the present application will also obtain the communication protocol of the mobile robot and connect with the smart device through the communication protocol; then, when the user performs operation control on the user terminal device, the embodiment of the present application will receive the control instructions from the user terminal device, and execute the corresponding target task according to the control instructions, thereby controlling the smart device to perform the corresponding linkage operation.

[0128] For example, when the mobile robot performs cleaning tasks according to control instructions, the intelligent device is controlled to adjust the lighting, so that appropriate lighting can ensure that the mobile robot can more accurately identify and locate areas that need to be cleaned, thereby improving cleaning efficiency. In addition, proper lighting can ensure that the mobile robot can clearly see the surrounding environment and avoid collisions with obstacles or falls.

[0129] It should be noted that when the mobile robot is connected to the smart device, the user can monitor and control the mobile robot and the smart device on the user terminal device, thereby simplifying the management process and improving the user experience; in addition, by controlling the mobile robot to perform the corresponding target task and controlling the smart device to perform the corresponding linkage operation according to the control instructions, the automation level of the smart home can be improved; in addition, through the user terminal device, it is possible to browse and experience an immersive home environment, and realize collaborative work and data sharing between smart devices, which greatly improves the user's operational convenience and satisfaction.

[0130] It should be noted that by scanning the indoor environment through lidar, the distance, angle and other characteristic quantities of the target object can be accurately measured, providing high-precision environmental perception information; in addition, by adjusting the positioning position of the mobile robot in the environmental map through the common feature points and posture data of adjacent environmental images, it can ensure that the mobile robot has more accurate viewing angle and position information when capturing continuous images, which helps to reduce image dislocation and distortion caused by viewing angle offset or position error, thereby improving the quality and accuracy of image stitching; in addition, through the combination of a single camera and lidar, the need for high-cost multi-camera systems is avoided while ensuring comprehensive environmental perception capabilities.

[0131] It is worth noting that since the embodiment of the present application can cruise along a preset planned path according to the environmental map, and can capture environmental images from multiple angles through a camera, and generate a panoramic image through environmental images from multiple angles, the embodiment of the present application can improve the comprehensive environmental perception capability of the mobile robot and generate panoramic images, thereby providing users with comprehensive and continuous visual information of the home environment.

[0132] like Figure 5 As shown, Figure 5 This is an overall flow chart of a mobile robot control method provided by another embodiment of the present application; the process includes but is not limited to the following steps:

[0133] Step S810: Control the sensor module to scan the indoor environment to obtain environmental perception information, generate an environmental map according to the environmental perception information, and determine the current position information of the mobile robot;

[0134] Step S820: Obtain an environment map, and cruise along a preset planned path according to the environment map;

[0135] Step S830, controlling the mobile robot to drive the camera to rotate along the axis of the horizontal plane for multiple times by a preset angle;

[0136] Step S840, capturing multiple environmental images at different angles through a camera at the rotation stop position;

[0137] Step S850, obtaining common feature points of adjacent environment images and position data detected by the laser radar;

[0138] Step S860: adjusting the position of the mobile robot according to the common feature points and the position and posture data;

[0139] Step S870: Generate a panoramic image based on multiple environment images at different angles.

[0140] Illustratively, in the process of generating a panoramic image based on multiple environmental images at different angles, the embodiment of the present application adjusts and optimizes the panoramic image.

[0141] It should be noted that the embodiments of the present application can ensure a natural transition of the panoramic image and optimize the visual effect of the panoramic image by adjusting and optimizing the panoramic image.

[0142] Exemplarily, adjusting and optimizing the panoramic image may include at least one of the following steps: performing distortion correction on the panoramic image; performing seam optimization on the panoramic image; performing dynamic range processing on the panoramic image; and performing color balance adjustment on the panoramic image.

[0143] It should be noted that when the panoramic image is subjected to distortion correction and seam optimization, the natural transition of the panoramic image can be ensured; when the panoramic image is subjected to dynamic range processing and color balance adjustment, the visual effect of the panoramic image can be optimized.

[0144] Exemplarily, the embodiment of the present application will also obtain the communication protocol of the mobile robot and connect with the smart device through the communication protocol; then, when the user performs operation control on the user terminal device, the embodiment of the present application will receive the control instructions from the user terminal device, and execute the corresponding target task according to the control instructions, thereby controlling the smart device to perform the corresponding linkage operation.

[0145] For example, when the mobile robot performs cleaning tasks according to control instructions, the intelligent device is controlled to adjust the lighting, so that appropriate lighting can ensure that the mobile robot can more accurately identify and locate areas that need to be cleaned, thereby improving cleaning efficiency. In addition, proper lighting can ensure that the mobile robot can clearly see the surrounding environment and avoid collisions with obstacles or falls.

[0146] It should be noted that when the mobile robot is connected to the smart device, the user can monitor and control the mobile robot and the smart device on the user terminal device, thereby simplifying the management process and improving the user experience; in addition, by controlling the mobile robot to perform the corresponding target task and controlling the smart device to perform the corresponding linkage operation according to the control instructions, the automation level of the smart home can be improved; in addition, through the user terminal device, it is possible to browse and experience an immersive home environment, and realize collaborative work and data sharing between smart devices, which greatly improves the user's operational convenience and satisfaction.

[0147] It should be noted that by scanning the indoor environment through the sensor module, the distance, angle and other characteristic quantities of the target object can be accurately measured, providing high-precision environmental perception information; in addition, by adjusting the positioning position of the mobile robot in the environmental map through the common feature points and posture data of adjacent environmental images, it can ensure that the mobile robot has more accurate viewing angle and position information when capturing continuous images, which helps to reduce image dislocation and distortion caused by viewing angle offset or position error, thereby improving the quality and accuracy of image stitching; in addition, through the combination of a single camera and lidar, the need for high-cost multi-camera systems is avoided while ensuring comprehensive environmental perception capabilities.

[0148] It is worth noting that since the embodiment of the present application can cruise along a preset planned path according to the environmental map, and can capture environmental images from multiple angles through a camera, and generate a panoramic image through environmental images from multiple angles, the embodiment of the present application can improve the comprehensive environmental perception capability of the mobile robot and generate panoramic images, thereby providing users with comprehensive and continuous visual information of the home environment.

[0149] Existing smart sweeping robots mainly rely on traditional navigation and obstacle avoidance technologies, such as infrared sensors or simple camera systems. Although these technologies can support the execution of basic cleaning tasks, they are limited in their ability to fully perceive the home environment. The robot lacks sufficient information collection capabilities and cannot provide detailed visual information about the home space, which limits its ability to perform advanced functions such as environmental monitoring and safety inspections.

[0150] Another issue closely related to limited environmental perception capabilities is that existing technologies cannot effectively capture and process images to generate a 360-degree panoramic view of the home environment. Existing robot models, even if equipped with cameras, are usually only used for navigation assistance rather than environmental monitoring. As a result, users cannot obtain a continuous and comprehensive visual experience of the home space, which limits the potential of robots in home monitoring and remote management applications.

[0151] Current smart sweeping robots also fall short in providing an immersive, interactive user experience. While some products allow users to view images captured from the robot’s camera through a mobile app, this experience is usually static, limited, and does not meet users’ needs for immersive environmental monitoring. There is a lack of an effective interface or platform that allows users to explore and monitor their home environment in a more intuitive and interactive way.

[0152] Existing smart sweeping robots usually operate as independent smart devices, with low integration with other smart home devices in the home. This limited level of system integration restricts the potential role of the devices in the smart home ecosystem, especially in achieving device collaboration, data sharing, and scene linkage.

[0153] In one embodiment, by integrating a single camera, a high-precision lidar, image matching optimization technology, and advanced image processing algorithms, a cost-effective, easy-to-operate, and powerful smart home device is provided.

[0154] 1. High-precision navigation and multi-angle image capture system

[0155] The present invention first uses a single high-performance camera, combined with a high-precision laser radar, to capture multi-angle environmental images through the robot's own rotational motion. The laser radar is used to achieve accurate positioning and obstacle avoidance of the robot in a complex home environment, while providing accurate position and posture information for image stitching. The robot captures images at certain intervals through preset rotation steps to ensure that the environment is fully recorded.

[0156] A synchronous rotation control algorithm is used to ensure the stability of the camera when it stops at each preset angle to capture clear environmental images. At the same time, LiDAR data is used for real-time pose estimation to provide accurate spatial reference for subsequent image processing.

[0157] 2. Image matching and pose optimization

[0158] After capturing multi-angle images, this solution uses image matching technology to further optimize the robot's pose estimation. By analyzing the common feature points between adjacent images and combining the precise distance information provided by the LiDAR, a more accurate determination of the robot's position and direction can be achieved.

[0159] Feature point matching algorithms (such as SIFT or ORB) are used to identify and match feature points between adjacent images, and the robot's pose estimation is optimized by calculating the spatial relationship between feature points. In addition, deep learning methods are used to further improve the accuracy and efficiency of image matching, especially under conditions of changing ambient lighting and complex textures.

[0160] 3. Panoramic image generation and optimization processing

[0161] The multi-angle images with optimized posture are efficiently fused to generate seamless 360-degree panoramic images. The present invention adopts advanced image stitching technology to automatically correct the distortion between images, optimize the transition at the seams, and generate high-quality panoramic images.

[0162] Implement an efficient stitching algorithm based on image fusion to automatically process and fuse multi-angle images, including distortion correction, seam optimization, and dynamic range adjustment. Use machine learning algorithms to automatically identify and correct potential problems in the stitching process, such as stitching errors or color inconsistencies, to ensure the visual quality of the final panorama.

[0163] 4. Smart home system integration

[0164] Finally, this solution deeply integrates the smart sweeping robot with other smart devices in the home to achieve data sharing and intelligent collaboration between devices. Through a unified smart home platform, users can view a panoramic image of the home in real time and control the robot and other smart devices at the same time.

[0165] Develop a universal smart home device communication protocol to support efficient data exchange and control between robots and various smart devices (such as smart lights, security cameras, etc.). Implement smart scene linkage strategies to automatically adjust home environment settings according to user-defined rules, such as automatically adjusting lighting conditions according to cleaning areas.

[0166] It is worth noting that the embodiment of the present application avoids the need for a high-cost multi-camera system by combining a single camera with a high-precision lidar, while ensuring comprehensive environmental perception capabilities; in addition, it utilizes image matching and pose optimization technology, combined with advanced image stitching algorithms, to generate high-quality 360-degree panoramic images, providing users with comprehensive and continuous visual information about the home environment; in addition, through the deep integration of virtual reality technology and smart home devices, it provides an immersive home environment browsing experience, and enables collaborative work and data sharing between smart devices, greatly improving user convenience and satisfaction.

[0167] like Figure 6 As shown, Figure 6 It is a structural diagram of a navigation and multi-angle image capture system provided by an embodiment of the present application.

[0168] In one embodiment, the navigation and multi-angle image capture system is the core of the intelligent sweeping robot, responsible for accurate navigation and capturing environmental images from multiple angles to generate a panoramic view. It includes the following submodules:

[0169] Single camera: As the primary device for image capture, its configuration is designed to optimize the field of view and image resolution for a clear view of the environment.

[0170] LiDAR: Provides precise location information of the robot and distance data of the environment, supporting high-precision navigation and obstacle avoidance functions.

[0171] Synchronous rotation control algorithm: controls the robot body to rotate at a preset angle to ensure accurate image capture at every angle.

[0172] Real-time pose estimation: Use lidar data to calculate the robot’s real-time position and pose, providing an accurate reference for image stitching.

[0173] like Figure 7 As shown, Figure 7 It is a structural diagram of an image matching and posture optimization module provided in one embodiment of the present application.

[0174] In one embodiment, the accuracy of pose estimation is further improved through image matching and pose optimization modules, and the quality of panorama generation is optimized:

[0175] Time synchronization and fusion: Ensure the time consistency of camera and lidar data, and provide a synchronized data source for pose estimation and image stitching.

[0176] Feature point matching algorithm: Identify feature points between adjacent images, optimize the robot’s posture data through matching, and enhance the accuracy of posture estimation.

[0177] Deep learning optimization: Apply deep learning algorithms to improve the accuracy and processing efficiency of image matching, especially under lighting changes and complex scenes.

[0178] like Figure 8 As shown, Figure 8 It is a structural diagram of a panoramic image generation and optimization processing module provided by an embodiment of the present application.

[0179] In one embodiment, the panoramic image generation and optimization processing module is responsible for processing and stitching images captured from multiple angles into a complete 360-degree panoramic image:

[0180] Image stitching algorithm: Efficiently fuse multi-angle images, automatically process distortion and seams between images, and generate continuous panoramas.

[0181] Distortion Correction and Seam Optimization: Adjust images to reduce stitching distortion, smooth the visual transition at seams, and improve panorama quality.

[0182] Dynamic range processing and color balance adjustment: optimize the display effect of panoramas under different lighting conditions to ensure the consistency and naturalness of image colors.

[0183] like Fig. 9 As shown, Fig. 9 It is a structural diagram of a smart home system integration module provided by an embodiment of the present application.

[0184] In one embodiment, the smart home system integration module deeply integrates the smart sweeping robot with other smart devices in the home to achieve data sharing and smart collaborative work:

[0185] Communication protocol development: Design protocols that support efficient and stable communication between multiple devices, providing a reliable data exchange foundation for smart home systems.

[0186] Smart scene linkage strategy: Automatically adjust the home environment according to the rules set by the user, such as adjusting the lighting according to the cleaning task, to improve the automation level of the smart home.

[0187] Unified device management platform: Provides a centralized platform for users to monitor and control all smart home devices, simplifying the management process and improving user experience.

[0188] Exemplarily, when started, the intelligent sweeping robot uses its lidar to self-position and draw a map of the environment.

[0189] In cleaning or monitoring mode, the robot uses a synchronous rotation control algorithm to rotate the robot body at fixed angle intervals based on the environment map and preset path planning, and a single camera captures the environment image at each stopping point.

[0190] After the image is captured, the system uses a feature point matching algorithm to analyze the common feature points between continuously captured images, combines the position data provided by the lidar, and fine-tunes the robot's precise position through deep learning optimization to optimize the accuracy of subsequent image stitching.

[0191] After pose optimization, the images captured from multiple angles are automatically fused into a continuous and seamless 360-degree panorama through an image stitching algorithm. During the stitching process, distortion correction and seam optimization ensure a natural image transition, and dynamic range processing and color balance adjustment optimize the visual effect of the panorama.

[0192] System integration: The smart sweeping robot connects with other smart devices at home through the developed communication protocol, providing users with a one-stop smart home control and monitoring solution through a unified device management platform. Users can view the home panorama generated by the panoramic image generation and optimization processing module in real time, and control the robot and other smart devices at the same time.

[0193] It is understandable that, regarding the above-mentioned self-positioning and drawing of environmental maps through its laser radar, self-positioning and drawing of environmental maps can also be achieved by combining a combination of multiple ultrasonic sensors and infrared sensors; the embodiments of the present application do not specifically limit this.

[0194] It can be understood that, regarding the above-mentioned analysis of common feature points between continuously captured images through feature point matching algorithm, the common feature points between continuously captured images can also be analyzed through FAST algorithm, and the common feature points between continuously captured images can also be analyzed through ORB algorithm; the embodiments of the present application do not make specific limitations on this.

[0195] It can be understood that, regarding the above-mentioned automatic fusion of images captured from multiple angles into a continuous and seamless 360-degree panorama through the image stitching algorithm, it is also possible to automatically fuse images captured from multiple angles into a continuous and seamless 360-degree panorama through a feature-based fusion algorithm, and it is also possible to automatically fuse images captured from multiple angles into a continuous and seamless 360-degree panorama through a deep learning-based fusion technology. The embodiments of the present application do not make specific limitations on this.

[0196] It is understandable that the communication protocol can adopt a variety of existing smart home standards and protocols, such as Zigbee, Z-Wave or Wi-Fi, etc. The most suitable communication technology can be selected according to actual application requirements and device compatibility. The embodiments of this application do not specifically limit it.

[0197] like Fig.10 As shown, Fig.10 700 is a schematic diagram of a controller for executing a control method for a mobile robot provided by an embodiment of the present application. The controller 700 implemented in the present application includes: a processor 710, a memory 720, and a computer program stored in the memory 720 and executable on the processor 710, wherein: Fig.10 In the figure, a processor 710 and a memory 720 are taken as an example.

[0198] The processor 710 and the memory 720 may be connected via a bus or other means. Fig.10 The example of connecting through bus is taken in the following.

[0199] The memory 720, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer executable programs. In addition, the memory 720 may include a high-speed random access memory, and may also include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage device. In some embodiments, the memory 720 optionally includes a memory 720 remotely arranged relative to the processor 710, and these remote memories 720 can be connected to the controller 700 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0200] Those skilled in the art will understand that Fig.10 The device structure shown in the figure does not constitute a limitation on the controller 700, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0201] exist Fig.10In the controller 700 shown, the processor 710 can be used to call the control program stored in the memory 720 to implement the above control method. Specifically, the non-transient software program and instructions required to implement the control method of the above embodiment are stored in the memory 720, and when executed by the processor 710, the control method of the above embodiment is executed.

[0202] It is worth noting that since the controller 700 of the embodiment of the present application can execute the control method of any of the above embodiments, the specific implementation methods and technical effects of the controller 700 of the embodiment of the present application can refer to the specific implementation methods and technical effects of the control method of any of the above embodiments.

[0203] In addition, an embodiment of the present application also provides a mobile robot, which includes the controller of the above embodiment.

[0204] It is worth noting that since the mobile robot of the embodiment of the present application includes the controller of the above-mentioned embodiment, and the controller of the above-mentioned embodiment can execute the control method of the mobile robot of any of the above-mentioned embodiments, the specific implementation manner and technical effects of the mobile robot of the embodiment of the present application can refer to the specific implementation manner and technical effects of the control method of the mobile robot of any of the above-mentioned embodiments.

[0205] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions, and the computer-executable instructions are used to execute the above-described control method. Figures 1 to 5 The method steps in .

[0206] It is worth noting that since the computer-readable storage medium of the embodiments of the present application can execute the control method of any of the above embodiments, the specific implementation methods and technical effects of the computer-readable storage medium of the embodiments of the present application can refer to the specific implementation methods and technical effects of the control method of any of the above embodiments.

[0207] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0208] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions under the shared conditions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A control method for a mobile robot, characterized in that: The mobile robot is provided with a camera and a laser radar, and the control method comprises: Controlling the mobile robot to drive the camera to rotate along the axis of the horizontal plane for multiple times by a preset angle; At the rotation stop position, the camera is used to capture multiple environmental images at different angles; A panoramic image is generated based on the posture data detected by the laser radar and the environment images at multiple different angles.

2. The control method of the mobile robot according to claim 1, characterized in that: Before controlling the mobile robot to rotate at intervals at a preset angle, the control method further includes: Acquire an environmental map, and control the mobile robot to cruise along a preset planned path according to the environmental map.

3. The control method of the mobile robot according to claim 1, characterized in that: Before obtaining the environment map, one of the following is included: Controlling the laser radar to scan the indoor environment to obtain environmental perception information, generating the environmental map according to the environmental perception information and determining the current position information of the mobile robot; The sensor module is controlled to scan the indoor environment to obtain environmental perception information, generate the environmental map according to the environmental perception information and determine the current position information of the mobile robot, wherein the sensor module includes a light sensor, an ultrasonic sensor and / or an infrared sensor.

4. The control method of the mobile robot according to claim 1, characterized in that: The control method further comprises: The positioning position of the mobile robot in the environmental map is adjusted according to the plurality of environmental images continuously captured by the camera, wherein the number of the cameras is 1.

5. The control method of the mobile robot according to claim 4, characterized in that: The step of adjusting the positioning position of the mobile robot in the environmental map according to the plurality of environmental images continuously captured by the camera comprises: Acquire common feature points of adjacent environment images; The position of the mobile robot is adjusted according to the common feature points and the position and posture data.

6. The control method of the mobile robot according to claim 5, characterized in that: The common feature points are obtained by the following steps: A feature point matching algorithm is used to determine the common feature points of two adjacent environment images.

7. The control method of the mobile robot according to claim 1, characterized in that: The control method further comprises: When the panoramic image is generated, the panoramic image is adjusted and optimized.

8. The control method of the mobile robot according to claim 7, characterized in that: The adjusting and optimizing the panoramic image includes at least one of the following: Performing distortion correction on the panoramic image; Performing seam optimization on the panoramic image; performing dynamic range processing on the panoramic image; A color balance adjustment is performed on the panoramic image.

9. The control method of the mobile robot according to claim 1, characterized in that: The control method further comprises: Acquire the communication protocol of the mobile robot, and connect with the intelligent device according to the communication protocol; Receive control instructions from a user terminal device, control the mobile robot to perform a corresponding target task according to the control instructions, and control the smart device to perform a corresponding linkage operation.

10. A controller, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the control method of the mobile robot as claimed in any one of claims 1 to 9 when executing the computer program.

11. A mobile robot, characterized in that: Comprising a controller as claimed in claim 10.

12. A computer-readable storage medium, characterized in that: Computer executable instructions are stored, and the computer executable instructions are used to execute the control method of the mobile robot according to any one of claims 1 to 9.