Robot, robot control method and device, electronic equipment, medium and product

Abstract

The invention provides a robot, a robot control method and device, electronic equipment, a medium and a product, and relates to the field of intelligent robotics.The robot comprises a laser ranging sensor LDS, an artificial intelligence AI camera module, an infrared camera module and a double-line laser sensor which are located on the side face of a machine body; the LDS is located in the center of the front side of the machine body, the AI camera module and the infrared camera module are located around the LDS in an up-and-down distribution mode, and the double-line laser sensors are located on the left side and the right side of the infrared camera module, so that the LDS is arranged in the center of the front side of the machine body, and cooperative work of all the components under the structural layout is combined. Compared with the prior art, the robot can perform environment modeling and path planning more quickly and accurately, and the upper and lower position distribution layout provides visual fields with different heights and angles, so that the robot can flexibly adjust the position and the path when facing a complex environment, and particularly, the robot has better performance in a cleaning task in a low area.

Description

Technical Field

[0001] This application relates to the field of intelligent robots, and particularly to a robot, a robot control method, a device, an electronic device, a medium, and a product. Background Art

[0002] In the field of robots, Laser Distance Sensors (LDS) are widely used for environmental mapping, obstacle avoidance, and navigation. LDS measures the distance to surrounding objects by emitting laser light and receiving the reflected signal, thereby constructing a two-dimensional or three-dimensional map of the environment. Especially when applied to floor cleaning robots, it can help the floor cleaning robot autonomously plan paths and avoid obstacles, thereby improving the cleaning efficiency.

[0003] In the related art, the LDS component is usually installed on the top of the floor cleaning robot, increasing the overall height of the floor cleaning robot. When the floor cleaning robot performs cleaning tasks, it cannot enter low areas, such as under furniture and under the sofa, for cleaning, which limits the application range of the robot.

[0004] However, the challenge of reducing the body height lies in how to redesign the layout and installation method of the LDS component while ensuring that its ranging and mapping capabilities are not affected. Summary of the Invention

[0005] This application provides a robot, a robot control method, a device, an electronic device, a medium, and a product. By placing the LDS at the center position on the front side of the body, arranging the AI camera module and the infrared camera module in an upper-lower distribution layout around the LDS, and installing the dual-line laser sensor on the left and right sides of the infrared camera module, when the robot faces a complex environment, while ensuring that its ranging and mapping capabilities are not affected, it can more flexibly adjust its own position and path, especially performing better in cleaning tasks in low areas. Moreover, the upper-lower distribution layout can provide different heights and angles of view, reducing the occlusion problem caused by a single perspective.

[0006] In a first aspect, this application provides a robot, which includes: a Laser Distance Sensor (LDS) located on the side of the body, an Artificial Intelligence (AI) camera module, an infrared camera module, and a dual-line laser sensor; the LDS is located at the center position on the front side of the body, the AI camera module and the infrared camera module are arranged in an upper-lower distribution layout around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module.

[0007] In this application, placing the LDS at the center of the front side of the fuselage instead of the top can effectively reduce the overall height of the robot, enabling it to more easily enter low areas such as under furniture and sofas for cleaning, thus expanding its application scope. It should be noted that the LDS is located at the center of the front side to obtain the maximum field of view angle, enabling it to directly perceive the environmental information in front, provide accurate distance measurement. The AI camera module and the infrared camera module are distributed above and below around the LDS, which can provide views at different heights and angles, enhancing the overall environmental perception ability while ensuring sufficient field of view angles, and thus capturing more environmental details, especially in complex or multi-level scenarios, to improve the measurement accuracy. Moreover, the above-and-below distribution layout can also reduce the occlusion problem caused by a single perspective. By providing views at different angles, it is possible to better detect and identify occluded or partially occluded objects. The dual-line laser sensors are located on the left and right sides of the infrared camera module, making it more conform to the principle of triangulation ranging, ensuring a relatively balanced perception field of view on both the left and right sides, and thus enhancing the lateral environmental perception ability, providing a wider field of view and higher measurement accuracy.

[0008] In addition, traditional LDSs are usually installed on the top of the robot and need to rotate 360 degrees to scan the surrounding environment. However, after moving the LDS to the center of the front side of the fuselage, a complex rotation structure is no longer required in the design. This not only simplifies the mechanical design, significantly saves the internal space of the robot, but also reduces the wear and maintenance requirements of moving parts.

[0009] Therefore, in this application, components such as the LDS, AI camera module, infrared camera module, and dual-line laser sensors are integrated together. While reducing the height of the robot, it also saves the internal space of the fuselage, enabling it to accommodate more other devices, improving the design redundancy. And this multi-module integration facilitates unified software management and resource allocation. Saving the fuselage space makes the software more flexible in hardware resource allocation, thereby improving the operation efficiency.

[0010] Optionally, the dual-line laser sensors are symmetrically located on the left and right sides of the LDS.

[0011] It can be understood that the symmetrically distributed dual-line laser sensors can provide a wider field of view coverage, ensuring effective distance measurement in the environments in front of and on the sides of the robot. And through reasonable interval settings, the overlapping field of view between the dual-line laser sensors can minimize the recognition blind area, ensuring that the robot can comprehensively perceive the surrounding obstacles in a complex environment.

[0012] Therefore, the symmetrically distributed layout makes the perception fields on both sides of the robot more balanced, ensuring balanced information input in the left - right direction for the robot. This enables the robot to detect and locate obstacles more accurately and perform more stable navigation, especially in narrow or complex paths.

[0013] In addition, the symmetric design not only provides advantages in functionality but also offers a sense of balance and coordination in appearance. Moreover, the symmetrically distributed dual - line laser sensors provide a certain degree of redundancy. Even if one sensor fails, the other sensor can still provide the necessary environmental information, enhancing the robot's fault - tolerance ability.

[0014] Optionally, the AI camera module and the infrared camera module are located on the same side around the LDS.

[0015] In this application, placing the AI camera module and the infrared camera module on the same side of the LDS can achieve a more compact design. This integrated layout helps reduce the overall volume and weight of the robot, improving its flexibility and mobility, and is also more convenient for assembly.

[0016] Furthermore, concentrating the modules on the same side can simplify the internal wiring, thereby reducing manufacturing complexity and cost. Also, concentrating the AI camera module and the infrared camera module on one side can ensure they have a consistent viewing direction, reducing the data - processing complexity caused by viewing - angle differences. Therefore, integrating the AI camera module and the infrared camera module on the same side makes it easier for the robot to perform data fusion. By combining visual and infrared information, it provides a more comprehensive environmental perception ability.

[0017] Optionally, the robot further includes: a fill light located between the infrared camera module and the dual - line laser sensor.

[0018] Since the fill light can provide additional illumination in low - light environments, significantly improving the image quality captured by the AI camera module and the infrared camera module. This helps improve the accuracy of object recognition and classification. By improving the lighting conditions, the fill light can also help the sensors measure distances and identify objects more precisely, especially in complex or dynamic environments. Therefore, the illumination of the fill light enables better cooperation between the sensors and the modules, providing a more comprehensive environmental perception ability. And better visual information and distance measurement help the robot perform more accurate path planning, avoid obstacles, and improve navigation efficiency.

[0019] In addition, in low - light conditions, the use of the fill light can reduce the risk of accidental collisions and improve the operational safety of the robot.

[0020] Optionally, the robot further includes: a voice module located on the top of the fuselage. The voice module is used to receive voice commands to control the robot to perform corresponding operations.

[0021] It should be noted that in the prior art, in addition to setting an LDS module on the top of the fuselage, a voice module is also provided. The LDS module originally generated noise when operating on the top of the fuselage. In this application, by removing the LDS module and setting the LDS at the central position on the front side of the fuselage, the noise originally generated by the original LDS module when operating on the top of the fuselage is greatly reduced, thereby reducing the interference with the voice module. The LDS module integrates a rotating mechanism, while the LDS in this application does not need to be provided with a rotating mechanism.

[0022] Therefore, by moving the LDS to the central position on the front side of the fuselage, away from the voice module, the noise interference with the voice module can be significantly reduced. Reducing noise interference helps improve the recognition accuracy of the voice module, enabling it to receive and process user voice commands more reliably. Furthermore, more accurate voice recognition can improve the response speed of the robot, enabling the robot to execute user commands more quickly.

[0023] In a second aspect, this application provides a robot control method. The robot includes: a laser distance sensor LDS located on the side of the fuselage, an artificial intelligence AI camera module, an infrared camera module, and a dual-line laser sensor; the LDS is located at the central position on the front side of the fuselage, the AI camera module and the infrared camera module are located around the LDS in an upper and lower distribution layout, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the method includes:

[0024] During the process of the robot executing a target task, based on the first sensor data collected by the infrared camera module and the dual-line laser sensor, determine the first height of the obstacle in the front area; the dual-line laser sensor is used to emit a laser beam to the obstacle, and the infrared camera module is used to capture the corresponding infrared image after the laser beam is reflected and determine the first sensor data;

[0025] Based on the first height, determine the first obstacle avoidance strategy, and control the robot to perform corresponding obstacle avoidance operations based on the first obstacle avoidance strategy.

[0026] In this way, by providing high-precision distance and height measurement by the dual-line laser sensor and combining the image capture ability of the infrared camera module, the height of the obstacle can be accurately identified, enabling the robot to effectively avoid potential collision risks, improving the obstacle avoidance efficiency while ensuring safety. Therefore, this method enables the robot to be more intelligent and efficient when executing tasks, especially performing well when dealing with complex environments and diverse obstacles, and has higher accuracy compared to the method of using the LDS or the AI camera module to identify obstacles for obstacle avoidance.

[0027] Optionally, the method further includes:

[0028] When it is detected that the first sensor data does not meet the preset conditions, determine the second height of the obstacle based on the second sensor data collected by the LDS, and / or determine the type of the obstacle based on the image data recognized by the AI camera module;

[0029] Determine a second obstacle avoidance strategy based on the second height and / or the type of the obstacle, and control the robot to perform corresponding obstacle avoidance operations based on the second obstacle avoidance strategy.

[0030] Therefore, when the robot detects an abnormality in the obstacle, it can quickly switch to a backup plan, that is, use the LDS or the AI camera module for obstacle avoidance. This redundant design improves the overall reliability of the robot to ensure the continuity and stability of the execution of the obstacle avoidance task.

[0031] Optionally, determining a first obstacle avoidance strategy based on the first height includes:

[0032] Determine the second height of the obstacle based on the second sensor data collected by the LDS, and determine the type of the obstacle based on the image data recognized by the AI camera module;

[0033] Determine a first obstacle avoidance strategy based on the first height, the second height, and the type of the obstacle.

[0034] In this way, by combining the data of multiple sensors, the height of the obstacle can be measured more accurately and its type can be recognized, improving the accuracy and reliability of detection. Moreover, the multi-sensor data provides comprehensive environmental information, enabling the robot to better understand and respond to complex scenarios. Therefore, by combining height and type information for obstacle avoidance, a more accurate obstacle avoidance strategy can be formulated, with higher accuracy, reducing unnecessary actions, improving efficiency, and more accurate obstacle recognition and strategy formulation help to avoid potential collision risks, and thus can improve the flexibility and safety of obstacle avoidance.

[0035] Optionally, the method further includes:

[0036] Determine the type of dirt in the front area based on the image data recognized by the AI camera module;

[0037] Determine a first cleaning strategy to be adopted during the execution of the obstacle avoidance operation based on the type of dirt, and control the robot to clean the surface to be cleaned in the front area based on the first cleaning strategy.

[0038] In this way, by identifying the type of dirt through the AI ​​camera module, the robot can adopt appropriate cleaning strategies to ensure the effective removal of different types of stains and improve the cleaning effect. It can also clean the surface to be cleaned in the front area while avoiding obstacles. The robot can plan the path more intelligently to ensure that all areas are effectively cleaned without repeatedly covering the cleaned areas, reducing unnecessary pauses and path adjustments of the robot during work, thereby improving the overall cleaning efficiency.

[0039] Optionally, the method further includes:

[0040] Determine the type of stain in the front area based on the infrared image recognized by the infrared camera module;

[0041] A second cleaning strategy to be adopted when performing the obstacle avoidance operation is determined based on the stain type, and the robot is controlled to clean the surface to be cleaned in the front area based on the second cleaning strategy.

[0042] Since infrared images can reveal temperature differences and surface features, the robot can more accurately identify the type of stain and ensure the appropriate cleaning strategy. Therefore, the robot combines infrared images to identify the type of stain in real time and then dynamically adjusts the cleaning strategy, allowing the robot to efficiently handle different types of stains while avoiding obstacles. While improving the overall cleaning effect, it can also improve the robot's ability to adapt to different environments.

[0043] In a third aspect, the present application provides a robot control method, the robot comprising: a laser ranging sensor LDS located on the side of a body, an artificial intelligence AI camera module, an infrared camera module and a dual-line laser sensor; the LDS is located at the front center of the body, the AI ​​camera module and the infrared camera module are arranged in an upper and lower distribution around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the method comprises:

[0044] During the robot's execution of the target task, a first positioning result is determined based on the second sensor data collected by the LDS, and a second positioning result is determined based on the image data recognized by the AI ​​camera module;

[0045] The first positioning result and the second positioning result are fused to construct a raster map.

[0046] Therefore, by combining the data of the LDS and the AI camera module, the present application can provide more accurate positioning results, reduce the errors that may be brought by a single sensor, and then through data fusion, correct the errors that may be generated by a single sensor, improve the accuracy of overall positioning and map construction. A more accurate grid map is helpful for optimizing path planning, reducing unnecessary detours and pauses, improving the task execution efficiency. Furthermore, through accurate environmental perception and map construction, the robot can perform tasks more safely, avoiding collisions and other potential risks.

[0047] In a fourth aspect, a robot control device of the present application, the robot includes: a laser distance sensor LDS located on the side of the fuselage, an artificial intelligence AI camera module, an infrared camera module, and a dual-line laser sensor; the LDS is located at the front center position of the fuselage, the AI camera module and the infrared camera module are arranged in an upper and lower distribution layout around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the device includes:

[0048] A first determination module, configured to determine a first height of an obstacle in the front area based on first sensor data collected by the infrared camera module and the dual-line laser sensor during the process of the robot executing a target task; the dual-line laser sensor is used to emit a laser beam, and the infrared camera module is used to capture an infrared image corresponding to the laser beam and determine the first sensor data;

[0049] A control module, configured to determine a first obstacle avoidance strategy based on the first height and control the robot to perform corresponding obstacle avoidance operations based on the first obstacle avoidance strategy.

[0050] In a fifth aspect, a robot control device of the present application, the robot includes: a laser distance sensor LDS located on the side of the fuselage, an artificial intelligence AI camera module, an infrared camera module, and a dual-line laser sensor; the LDS is located at the front center position of the fuselage, the AI camera module and the infrared camera module are arranged in an upper and lower distribution layout around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the device includes:

[0051] A second determination module, configured to determine a first positioning result based on second sensor data collected by the LDS and determine a second positioning result based on image data recognized by the AI camera module during the process of the robot executing a target task;

[0052] A construction module, configured to fuse and process the first positioning result and the second positioning result to construct a grid map.

[0053] In a sixth aspect, an electronic device of the present application includes: a processor, and a memory communicatively connected to the processor;

[0054] The memory stores computer execution instructions;

[0055] The processor executes computer-executable instructions stored in the memory to implement the method according to any one of the second aspect and the third aspect.

[0056] In a seventh aspect, a computer-readable storage medium of the present application stores computer-executable instructions, which are used to implement the method according to any one of the second aspect and the third aspect when executed by a processor.

[0057] In an eighth aspect, the present application provides a computer program product, including a computer program, which implements the method according to any one of the second aspect and the third aspect when executed by a processor.

[0058] It should be noted that the technical solutions of the fourth aspect to the eighth aspect of the present application respectively correspond to those of the first aspect, the second aspect and the third aspect of the present application. The beneficial effects obtained by each aspect and the corresponding feasible implementation manners are similar, and will not be elaborated here.

[0059] In summary, the present application provides a robot, a robot control method, a device, an electronic device, a medium and a product, aiming to solve the problem of increased height caused by the installation of the LDS on the top of the traditional sweeping robot. For this purpose, the LDS is redesigned and installed at the center position on the front side of the robot body, rather than on the top. This layout effectively reduces the overall height of the robot, enabling it to more easily enter low-lying areas. When the LDS is installed on the top, due to the body blocking some areas, there are easily visual blind spots. By installing the LDS at the center position on the front side of the body, the maximum field of view angle can be obtained, reducing visual blind spots. Further, in order to obtain the visual field ranges at different heights and angles, the AI camera module and the infrared camera module are arranged in a superior-inferior distribution layout around the LDS. In this way, the AI camera module and the infrared camera module can provide visual fields at different heights and angles, enhancing the overall environmental perception ability, reducing the occlusion problem caused by a single perspective, and making the accuracy of the collected data higher. Moreover, the dual-line laser sensor combined with the infrared camera module can provide more accurate distance measurement and obstacle detection functions. The laser emitted by the dual-line laser sensor is cross-shaped, that is, the left dual-line laser sensor emits laser to the right, and the right dual-line laser sensor emits laser to the left. The infrared camera module is used to capture the image of the laser reflection. Therefore, installing the dual-line laser sensors on the left and right sides of the infrared camera module not only minimizes the blind spots to the greatest extent, but also makes it more in line with the principle of triangulation ranging, thereby improving the measurement accuracy. Description of the Drawings

[0060] The drawings here are incorporated into the specification and constitute a part of this specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0061] Figure 1 Partial structural schematic diagram of a robot provided by an embodiment of the present application;

[0062] Figure 2 Position structure distribution diagram of a robot provided by an embodiment of the present application;

[0063] Figure 3 Structural schematic diagram of a robot provided by an embodiment of the present application;

[0064] Figure 4 Schematic diagram of an application scenario provided by an embodiment of the present application;

[0065] Figure 5 Flow schematic diagram of a robot control method provided by an embodiment of the present application;

[0066] Figure 6 Flow schematic diagram of another robot control method provided by an embodiment of the present application;

[0067] Figure 7 Structural schematic diagram of a robot control device provided by an embodiment of the present application;

[0068] Figure 8 Structural schematic diagram of another robot control device provided by an embodiment of the present application;

[0069] Figure 9 Structural schematic diagram of an electronic device provided by an embodiment of the present application.

[0070] Through the above-mentioned drawings, specific embodiments of the present application have been shown, and there will be more detailed descriptions hereinafter. These drawings and textual descriptions are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed implementation manners

[0071] In order to clearly describe the technical solutions of the embodiments of the present application, in the embodiments of the present application, terms such as "first" and "second" are used to distinguish identical or similar items with basically the same functions and effects. For example, the first device and the second device are only used to distinguish different devices, and do not limit their sequence. Those skilled in the art can understand that terms such as "first" and "second" do not limit the quantity and execution order, and "first", "second", etc. do not necessarily mean different.

[0072] It should be noted that in this application, words such as "exemplary" or "for example" are used to give examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or having more advantages than other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0073] In this application, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. "At least one (item)" or its similar expression refers to any combination of these items, including any combination of a single item or multiple items. For example, at least one (item) of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0074] In the related art, the LDS component is usually installed on the top of the floor - cleaning robot, increasing the overall height of the floor - cleaning robot. In this way, when the floor - cleaning robot performs cleaning tasks, it cannot enter and clean low - lying areas such as under furniture and under the sofa, which limits the application range of the robot.

[0075] However, the challenge of reducing the body height lies in how to redesign the layout and installation method of the LDS component while ensuring that its ranging and mapping capabilities are not affected.

[0076] In view of the above problems, the present application provides a robot, aiming to solve the problem of increased height caused by the LDS being installed on the top of traditional floor-sweeping robots. To this end, the LDS is redesigned and installed at the center position on the front side of the robot body, rather than on the top. This layout effectively reduces the overall height of the robot, enabling it to more easily enter low-lying areas. Moreover, when the LDS is installed on the top, due to the body blocking some areas, there are easily visual blind spots. By installing the LDS at the center position on the front side of the body, the maximum field of view angle can be obtained, reducing visual blind spots. Further, to obtain the visual field ranges at different heights and angles, the AI camera module and the infrared camera module are arranged in a top-bottom distribution layout around the LDS. In this way, the AI camera module and the infrared camera module can provide visual fields at different heights and angles, enhancing the overall environmental perception ability and reducing the occlusion problem caused by a single perspective, making the accuracy of the collected data higher. Furthermore, the dual-line laser sensor combined with the infrared camera module can provide more accurate distance measurement and obstacle detection functions. The laser emitted by this dual-line laser sensor is cross-shaped, that is, the left dual-line laser sensor emits laser to the right, and the right dual-line laser sensor emits laser to the left. The infrared camera module is used to capture the image of the laser reflection. Therefore, installing the dual-line laser sensors on the left and right sides of the infrared camera module not only minimizes the blind spots to the greatest extent but also makes it more in line with the principle of triangulation ranging, thereby improving the measurement accuracy.

[0077] In summary, by placing the LDS at the center position on the front side of the body and combining the collaborative work of the AI camera module, the infrared camera module, and the dual-line laser sensor under the above structural layout, the robot can perform environmental modeling and path planning more quickly and accurately. This layout enables the robot to more flexibly adjust its own position and path when facing a complex environment, especially showing better performance in the cleaning tasks in low-lying areas.

[0078] In addition, traditional LDSs are usually installed on the top of the robot and need to rotate 360 degrees to scan the surrounding environment. However, after moving the LDS to the center position on the front side of the body, a complex rotation structure is no longer required in the design. This not only simplifies the mechanical design, significantly saves the internal space of the robot, but also reduces the wear and maintenance requirements of the moving parts.

[0079] Therefore, the present application integrates these components such as the LDS, the AI camera module, the infrared camera module, and the dual-line laser sensor. While reducing the height of the robot, it also saves the internal space of the body, enabling it to accommodate more other devices, improving the design redundancy. And this integration of multiple modules facilitates the unified management and resource allocation of software. Saving the body space makes the software more flexible in hardware resource allocation, thereby improving the operation efficiency.

[0080] Exemplarily, Figure 1This is a partial structural schematic diagram of a robot provided by an embodiment of the present application, as Figure 1 shown. The robot 100 includes: a laser distance sensor LDS 102 on the side of the fuselage 101, an artificial intelligence (AI) camera module 103, an infrared camera module 104, and a dual-line laser sensor 105; the LDS 102 is located at the front central position of the fuselage 101, the AI camera module 103 and the infrared camera module 104 are arranged in an upper-lower distribution around the LDS 102, and the dual-line laser sensor 105 is located on the left and right sides of the infrared camera module 103.

[0081] Optionally, the AI camera module 103 and the infrared camera module 104 are arranged in an upper-lower distribution around the LDS 102. They can be located on the same side of the LDS 102 or on different sides respectively. For example, one is on the left side and the other is on the right side. However, they are distributed in an upper-lower position. The upper-lower position can refer to taking the horizontal line where the center point of the LDS 102 is located as the axis, one is located above the axis and the other is located below the axis, and this horizontal line is parallel to the surface to be cleaned.

[0082] Among them, the LDS 102 is used to measure the distance to surrounding objects. By emitting laser light and measuring its return time, the LDS 102 can accurately calculate the distance to the object. Therefore, the LDS 102 can be used for mapping, positioning, and navigation based on the collected second sensor data. Optionally, the LDS 102 can be a C91S lidar, a time-of-flight (ToF) sensor, a pulsed laser sensor, etc. The embodiment of the present application does not make a specific limitation on the type of the LDS 102.

[0083] The AI camera module 103 combines computer vision technology and is used to identify and analyze objects and scenes in the environment. Therefore, the AI camera module 103 can be used to determine the type of obstacles and the type of dirt in the front area based on the identified image data. Optionally, the AI camera module 103 can be an AI camera. The embodiment of the present application does not make a specific limitation on the type of the AI camera module 103.

[0084] The dual-line laser sensor 105 is composed of two laser sensors. The dual-line laser sensor 105 can provide a wider field of view and higher measurement accuracy. The dual-line laser sensor 105 is usually used in combination with the infrared camera module 104. The dual-line laser sensor 105 is used to emit a laser beam to an object such as an obstacle in the front area, and the infrared camera module 104 is used to capture the corresponding infrared image after the laser beam is reflected and determine the first sensor data.

[0085] In addition, the infrared imaging module 104 can also be used to determine the type of stain in the front area based on the recognized infrared image. Optionally, the infrared imaging module 104 can be an infrared camera, and the dual-line laser sensor 105 can be a dual-line triangulation sensor. The embodiments of the present application do not specifically limit the types of the infrared imaging module 104 and the dual-line laser sensor 105.

[0086] It should be noted that the LDS 102, the artificial intelligence AI imaging module 103, the infrared imaging module 104, and the dual-line laser sensor 105 are all located at the front position of the fuselage 101 and are relatively close to each other. Therefore, the integration is high and it is convenient for installation.

[0087] Exemplarily, Figure 2 is a position structure distribution diagram of a robot provided by an embodiment of the present application. As Figure 2 shown, taking the top view of the robot 100 as an example, the area C in the figure is the front center position of the fuselage 101, the area A is the left side position of the fuselage 101, and the area B is the right side position of the fuselage 101. Correspondingly, the dual-line laser sensor 105 is located on the left and right sides of the infrared imaging module 104, and the infrared imaging module 104 is located on any side close to the area C. For example, if the infrared imaging module 104 is located at a certain position in the area A, the dual-line laser sensor 105 can be located at any position in the area A and the area B respectively, or can be located in the area A at the same time. The embodiments of the present application do not limit the specific distribution position of the dual-line laser sensor 105.

[0088] Exemplarily, the AI imaging module 103 and the infrared imaging module 104 are arranged side by side above and below on the side of the LDS 102, and can be on the right side at the same time, on the left side at the same time, or separately arranged on the left and right sides. The embodiments of the present application do not limit the specific distribution position of the AI imaging module 103 and the infrared imaging module 104.

[0089] As can be seen from the above embodiments, placing the LDS102 at the front center position of the fuselage 101 instead of the top can effectively reduce the overall height of the robot 100, which enables the robot 100 to more easily enter low areas, such as under furniture and under sofas, for cleaning, expanding its application range. It should be noted that the LDS102 is located at the front center position to obtain the maximum field of view angle, enabling it to directly sense the environmental information in front, provide accurate distance measurement. The AI camera module 103 and the infrared camera module 104 are arranged in a superior-inferior distribution layout around the LDS102, which can provide views at different heights and angles, enhancing the overall environmental perception ability while ensuring a sufficient field of view angle, and thus capturing more environmental details, especially in complex or multi-level scenarios, to improve the measurement accuracy. Moreover, the superior-inferior distribution layout can also reduce the occlusion problem caused by a single perspective. By providing views at different angles, it is possible to better detect and identify occluded or partially occluded objects. The dual-line laser sensor 105 is located on the left and right sides of the infrared camera module 104, making it more in line with the principle of triangulation ranging, ensuring a relatively balanced sensing field of view on both the left and right sides, and thus enhancing the lateral environmental perception ability, providing a wider field of view and higher measurement accuracy.

[0090] It should also be noted that for components such as the LDS102, the artificial intelligence AI camera module 103, the infrared camera module 104, and the dual-line laser sensor 105, since their distances from each other are relatively close, material compatibility needs to be considered when selecting materials. Therefore, a supplier that can provide multiple sensors and modules can be selected, which can simplify the procurement process and make material preparation more convenient.

[0091] Optionally, Figure 3 The following is a schematic structural diagram of a robot provided by an embodiment of the present application. As Figure 3 shown, the dual-line laser sensors 105 are symmetrically distributed on the left and right sides of the LDS102.

[0092] In the present application, the dual-line laser sensors 105 are located on both sides of the center of the fuselage 101, symmetrically distributed and spaced at a certain distance to obtain the best ranging and minimum recognition blind area effects.

[0093] It can be understood that the symmetrically distributed dual-line laser sensors 105 can provide a wider field of view coverage, ensuring effective distance measurement in the environments in front of and on the sides of the robot 100. And through reasonable spacing settings, the overlapping field of view between the dual-line laser sensors 105 can minimize the recognition blind area, ensuring that the robot 100 can comprehensively perceive surrounding obstacles in a complex environment.

[0094] Therefore, the symmetrically distributed layout makes the perception fields on both sides of the robot 100 more balanced, ensuring balanced information input for the robot 100 in the left - right direction. As a result, it can detect and locate obstacles more accurately and perform more stable navigation, especially in narrow or complex paths.

[0095] In addition, the symmetric design not only provides advantages in function but also offers a sense of balance and coordination in appearance. Moreover, the symmetrically distributed dual - line laser sensors 105 provide a certain degree of redundancy. Even if one sensor fails, the other sensor can still provide the necessary environmental information, enhancing the fault - tolerance ability of the robot 100.

[0096] Optionally, as Figure 3 shown, the AI camera module 103 and the infrared camera module 104 are located on the same side around the LDS 102. Preferably, the AI camera module 103 and the infrared camera module 104 can be both on the left side or both on the right side at the same time. When located on the same side, they can be distributed vertically, which is convenient for installation and makes the robot 100 more symmetric and coordinated in the vertical direction, enhancing the visual appeal of the product.

[0097] In this application, placing the AI camera module 103 and the infrared camera module 104 on the same side of the LDS 102 can achieve a more compact design. This integrated layout helps reduce the overall volume and weight of the robot 100, improving its flexibility and mobility, and is also more convenient for assembly.

[0098] In addition, concentrating the modules on the same side can simplify the internal wiring, thereby reducing manufacturing complexity and cost. Also, concentrating the AI camera module 103 and the infrared camera module 104 on one side can ensure that they have a consistent viewing direction, reducing the data - processing complexity caused by viewing - angle differences. Therefore, integrating the AI camera module 103 and the infrared camera module 104 on the same side makes it easier for the robot 100 to perform data fusion. By combining visual and infrared information, it provides a more comprehensive environmental perception ability.

[0099] Optionally, as Figure 3 shown, the robot 100 further includes: a fill light 106 located between the infrared camera module 104 and the dual - line laser sensor 105.

[0100] In this application, two fill lights 106 can be provided. When the detection environment is relatively dark, the fill lights 106 can be turned on to illuminate the detection space, enabling other modules or sensors to better identify the detection object. For example, the dual - line laser sensor 105 can better identify obstacles.

[0101] It should be noted that the number of supplementary lights 106 provided in the embodiments of the present application is not specifically limited. For example, there may be 2 or even more supplementary lights 106 located between the infrared camera module 104 and the dual-line laser sensor 105.

[0102] Since the supplementary lights 106 can provide additional illumination in environments with insufficient light, significantly improving the image quality captured by the AI camera module 103 and the infrared camera module 104, which helps to improve the accuracy of object recognition and classification. By improving the lighting conditions, the supplementary lights 106 can also help the sensors measure distances and identify objects more precisely, especially in complex or dynamic environments. Therefore, the illumination of the supplementary lights 106 enables better cooperation between the sensors and the modules, providing a more comprehensive environmental perception ability. And better visual information and distance measurement help the robot 100 perform more accurate path planning, avoid obstacles, and improve navigation efficiency.

[0103] In addition, under low-light conditions, the use of the supplementary lights 106 can reduce the risk of accidental collisions and improve the operational safety of the robot 100.

[0104] Optionally, as Figure 3 shown, the robot 100 further includes: a voice module 107 located at the top of the fuselage 101, and the voice module 107 is used to receive voice commands to control the robot 100 to perform corresponding operations.

[0105] It should be noted that in the prior art, in addition to setting an LDS module on the top of the fuselage, a voice module is also provided. The LDS module originally generated noise when operating on the top of the fuselage. In the present application, by moving the LDS module away and setting the LDS 102 at the front center position of the fuselage 101, the noise originally generated by the original LDS module when operating on the top of the fuselage is greatly reduced, and thus the interference to the voice module 107 can be reduced. The LDS module integrates a rotating mechanism, while the LDS 102 in the present application does not need to be provided with a rotating mechanism. Therefore, the structural setting of the LDS 102 is simplified.

[0106] Exemplarily, the user interacts with the robot through voice commands, that is, the user issues a voice command, and after the voice module 107 receives the voice command, it controls the robot to perform corresponding actions, such as cleaning, moving, obstacle avoidance, etc.

[0107] Therefore, by moving the LDS 102 to the front center position of the fuselage 101, away from the voice module 107, the noise interference to the voice module 107 can be significantly reduced. Reducing the noise interference helps to improve the recognition accuracy of the voice module 107, enabling it to more reliably receive and process the user's voice commands. Furthermore, more accurate voice recognition can improve the response speed of the robot 100, enabling the robot 100 to execute the user's commands more quickly.

[0108] Exemplarily, Figure 4 FIG. 1 is a schematic diagram of an application scenario provided by an embodiment of the present application. As Figure 4 shown, taking the robot 100 performing a cleaning task as an example, the LDS of the robot 100 is disposed near or at the center of the fuselage. The outer side of the LDS is the AI camera and the infrared camera distributed up and down, and the outermost side is the double-line laser sensor. The application scenario includes the robot 100 and the TV cabinet 200.

[0109] During the process of the robot 100 performing the cleaning task, for the area under the TV cabinet 200, since the robot 100 disposes the LDS, the AI camera, the infrared camera, and the double-line laser sensor on the side of the fuselage, the robot 100 can enter the area under the TV cabinet 200 for cleaning, thereby improving the overall cleaning effect of the room.

[0110] It should be noted that the robot 100 can be a cleaning robot such as a sweeping robot, an industrial mobile robot such as a material handling robot, a service robot such as a navigation robot, a detection and rescue robot, etc. The embodiment of the present application does not specifically limit the type of the robot.

[0111] It can be understood that for different types of robots, different application scenarios can be applicable. The embodiment of the present application does not specifically limit the scenarios applicable to the robot. The above is only an example, and it can also be applied to mapping scenarios, navigation scenarios, rescue scenarios, etc.

[0112] The following will specifically describe the control method of the application robot with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below with reference to the drawings.

[0113] Figure 5 FIG. 2 is a schematic flow chart of a robot control method provided by an embodiment of the present application. As Figure 5 shown, the robot control method is applied to Figure 1 or Figure 3 the robot shown in FIG. 3; the robot control method includes the following steps:

[0114] S501. During the process of the robot performing the target task, determine the first height of the obstacle in the front area based on the first sensor data collected by the infrared imaging module and the double-line laser sensor; the double-line laser sensor is used to emit a laser beam to the obstacle, and the infrared imaging module is used to capture the infrared image corresponding to the reflected laser beam and determine the first sensor data.

[0115] Optionally, the target task may be any one of a cleaning task, a navigation task, a mapping task, a positioning task, etc., and the embodiments of the present application do not make specific limitations thereto.

[0116] In the present application, the infrared image obtained by the infrared camera module contains information on laser reflection. Therefore, by analyzing the infrared image, the position and shape of the laser line on the obstacle can be determined. Furthermore, based on the change in the position of the laser line in the image, combined with the known laser emission angle and the position of the dual-line laser sensor, the first height of the obstacle can be calculated.

[0117] S502. Determine a first obstacle avoidance strategy based on the first height, and control the robot to perform corresponding obstacle avoidance operations based on the first obstacle avoidance strategy.

[0118] Optionally, the height of the obstacle can be classified according to a preset height threshold. For example, it can be classified into categories such as low obstacles, medium-height obstacles, and tall obstacles. Based on the height classification of the obstacles, different obstacle avoidance strategies correspond to different heights of the obstacles. For example, the obstacle avoidance strategy corresponding to a low obstacle is to choose to step over or pass directly, the obstacle avoidance strategy corresponding to a medium-height obstacle is to choose to detour or adjust the path, and the obstacle avoidance strategy corresponding to a tall obstacle is to choose to stop and re-plan the path. The embodiments of the present application do not make specific limitations on the obstacle avoidance strategies corresponding to obstacles of different heights.

[0119] Exemplarily, determine the height category of the obstacle based on the first height of the obstacle, and then determine a suitable first obstacle avoidance strategy based on the determined height category. Further, the robot generates specific control instructions according to the selected first obstacle avoidance strategy, and controls the robot to perform corresponding obstacle avoidance operations.

[0120] Optionally, during the execution of the obstacle avoidance operation, the robot can continuously monitor the changes in the surrounding environment and adjust the obstacle avoidance operation in real time to adapt to the dynamic environment.

[0121] Optionally, when it is determined that the first height of the obstacle is greater than a preset threshold, for example, the preset threshold is 5 cm, then control the robot to bypass the obstacle. When it is determined that the first height of the obstacle is less than or equal to the preset threshold, then control the robot to step over the obstacle. The embodiments of the present application do not make specific limitations on the size of the preset threshold, and it can be set based on the actual application scenario requirements.

[0122] Optionally, the combination of the dual-line laser sensor and the infrared camera module can also identify the height and type of the steps. The type includes horizontal steps, multi-level steps, slide rails, and the height of each step, etc. Then, determine the first obstacle avoidance strategy based on the identified height and type of the steps.

[0123] In this way, by providing high-precision distance and height measurements through a dual-line laser sensor and combining the image capture ability of the infrared camera module, the height of obstacles can be accurately identified, enabling the robot to effectively avoid potential collision risks, improve the obstacle avoidance efficiency, and ensure safety. Therefore, this method enables the robot to be more intelligent and efficient when performing tasks, especially performing well in dealing with complex environments and diverse obstacles, and having higher accuracy compared to the methods of using an LDS or an AI camera module to identify obstacles for obstacle avoidance.

[0124] Optionally, the method further includes:

[0125] When it is detected that the first sensor data does not meet the preset conditions, determine the second height of the obstacle based on the second sensor data collected by the LDS, and / or determine the type of the obstacle based on the image data recognized by the AI camera module;

[0126] Determine a second obstacle avoidance strategy based on the second height and / or the type of the obstacle, and control the robot to perform corresponding obstacle avoidance operations based on the second obstacle avoidance strategy.

[0127] In the embodiments of the present application, the LDS can emit a light beam that is inclined downward. The angle between the downward light beam and the horizontal line of the fuselage parallel to the surface to be cleaned is α, and α can take a value of 10°. This light beam is used to detect obstacles, such as detecting the second height of the obstacle. The embodiments of the present application do not make specific limitations on the value of α, which can be determined based on the design requirements and product performance of the product.

[0128] Optionally, the viewing angle range of the light beam emitted by the LDS in the horizontal direction is β, and β can take a value of 120°. This horizontal direction is parallel to the surface to be cleaned, and the emitted light beam is symmetric about the center position of the fuselage. The embodiments of the present application do not make specific limitations on the value of β, which can be determined based on the design requirements and product performance of the product.

[0129] In the present application, the LDS can detect point cloud data based on the emitted light beam to use this point cloud data for mapping, positioning, and navigation.

[0130] Optionally, the AI camera module is used to identify the objects in front, such as determining the type of obstacles in the front area based on the recognized image data, or alternatively, determining the type of dirt on the surface to be cleaned in the front area based on the recognized image data. This image data can be a grayscale image, a color image, etc. The embodiments of the present application do not make specific limitations on this.

[0131] Different obstacle avoidance strategies can be set for different obstacle types, and different obstacles have different obstacle avoidance distances. For example, for living obstacles such as pets and wiring harnesses, or obstacles that are easily entangled, the corresponding obstacle avoidance strategy is to avoid the obstacle at a certain distance; if it is a fixed obstacle such as a table or chair leg, the corresponding obstacle avoidance strategy is to avoid the obstacle when it is close.

[0132] Optionally, the second obstacle avoidance strategy determined by the second height of the obstacle determined by the second sensor data collected by LDS is regarded as backup plan one, the second obstacle avoidance strategy determined by the type of obstacle determined by the image data recognized by the AI ​​camera module is regarded as backup plan two, and the combination of the above two plans is regarded as backup plan three. In this way, when problems arise in collecting the first sensor data using the infrared camera module and the dual-line laser sensor, it is possible to flexibly switch to backup plan one, backup plan two or backup plan three based on the application scenario requirements, thereby improving the flexibility of the application.

[0133] Exemplarily, the robot first uses the first sensor data collected by the infrared camera module and the dual-line laser sensor to detect the first height of the obstacle. If the first sensor data does not meet the preset conditions, such as incomplete or inaccurate data, the robot identifies possible abnormalities. In this case, some functions in the LDS are activated to determine the second height of the obstacle based on the collected second sensor data, and / or some functions in the AI ​​camera module analyze the image data of the surrounding environment to identify the type of obstacle. Further, a second obstacle avoidance strategy is determined based on the second height and / or the type of obstacle, and the robot is controlled to perform corresponding obstacle avoidance operations according to the second obstacle avoidance strategy to ensure safe and effective avoidance of obstacles.

[0134] It should be noted that the preset conditions are used to determine the conditions set when inaccurate first sensor data is detected. For example, when the first sensor data collected by the infrared camera module or the dual-line laser sensor fails or the first sensor data is not collected, it is determined that the first sensor data does not meet the preset conditions. The embodiments of the present application do not limit the specific content corresponding to the preset conditions.

[0135] Therefore, when the robot detects an abnormal obstacle, it can quickly switch to the backup plan, that is, use the LDS or AI camera module to avoid the obstacle. This redundant design improves the overall reliability of the robot to ensure the continuity and stability of the obstacle avoidance task.

[0136] Optionally, determining a first obstacle avoidance strategy based on the first height includes:

[0137] Determine a second height of the obstacle based on the second sensor data collected by the LDS, and determine the type of the obstacle based on the image data recognized by the AI ​​camera module;

[0138] Determine the first obstacle avoidance strategy based on the first height, the second height, and the type of the obstacle.

[0139] In this step, the AI camera module determines the type of the obstacle, the infrared camera module and the double-line laser sensor are combined to determine the height of the obstacle, the LDS determines the distance between the obstacle and the robot. By fusing the type, height, distance, etc. of the obstacle together, the detailed information of the obstacle can be determined. The detailed obstacle information helps the robot better understand the surrounding environment, and then use this detailed obstacle information to determine a more accurate first obstacle avoidance strategy.

[0140] In this way, by combining the data of multiple sensors, the height of the obstacle can be measured more accurately and its type can be identified, improving the accuracy and reliability of detection. Moreover, the multi-sensor data provides comprehensive environmental information, enabling the robot to better understand and respond to complex scenarios. Therefore, by combining the height and type information for obstacle avoidance, a more precise obstacle avoidance strategy can be formulated, with higher accuracy, reducing unnecessary actions, improving efficiency, and more accurate obstacle recognition and strategy formulation helping to avoid potential collision risks, and thus improving the flexibility and safety of obstacle avoidance.

[0141] Optionally, the method further includes:

[0142] Determine the type of dirt in the front area based on the image data recognized by the AI camera module;

[0143] Determine the first cleaning strategy to be adopted during the execution of the obstacle avoidance operation based on the type of dirt, and control the robot to clean the surface to be cleaned in the front area based on the first cleaning strategy.

[0144] Among them, different types of dirt may require different cleaning strategies, such as vacuuming, wiping, mopping and other cleaning strategies. Optionally, the types of dirt can include solid dirt, wet dirt, dust dirt, solid-liquid mixed dirt, etc. The embodiments of the present application do not make specific limitations on the classification of the types of dirt.

[0145] In the present application, the next action of the robot can be determined based on the image data recognized by the AI camera module, that is, to perform an over-obstacle action or to perform different cleaning strategies, such as directly cleaning or bypassing the stain, or cleaning while performing the obstacle avoidance action, so as to improve the cleaning effect.

[0146] Exemplarily, the AI camera module captures the image data of the front area and analyzes these data through an image recognition algorithm to identify the type of dirt on the surface to be cleaned, such as dust, liquid stains, solid garbage, etc. Further, based on the identified type of dirt, the corresponding first cleaning strategy is formulated, and then combined with the obstacle avoidance operation, the cleaning path and method are optimized, so that the robot can clean the surface to be cleaned in the front area according to the first cleaning strategy to improve the cleaning efficiency and effect.

[0147] Optionally, for the position where a pet or humanoid obstacle is located, a dynamic supplementary cleaning strategy can be adopted, that is, after the obstacle leaves the position, the robot is controlled to return to that position for supplementary cleaning.

[0148] In this way, by using the AI camera module to identify the type of dirt, the robot can adopt appropriate cleaning strategies to ensure the effective removal of different types of stains, improve the cleaning effect, and can clean the surface to be cleaned in the front area while avoiding obstacles. The robot can more intelligently plan the path to ensure that all areas are effectively cleaned without repeatedly covering the cleaned areas, reducing unnecessary pauses and path adjustments during the operation of the robot, thereby improving the overall cleaning efficiency.

[0149] In addition, through intelligent obstacle avoidance, the robot can better approach the edge and corner areas to ensure that these areas that are usually difficult to clean are also effectively covered. Especially in a dynamic environment, such as a family with pets or children moving around, the robot can flexibly respond to changing obstacles while continuing to perform the cleaning task, enhancing the adaptability and improving the obstacle avoidance flexibility. Moreover, by reducing unnecessary movements and path adjustments, the robot can also reduce energy consumption, thereby extending the battery life and working time.

[0150] Optionally, the method further includes:

[0151] Determine the type of stain in the front area based on the infrared image recognized by the infrared camera module;

[0152] Determine a second cleaning strategy to be adopted when performing the obstacle avoidance operation based on the type of stain, and control the robot to clean the surface to be cleaned in the front area based on the second cleaning strategy.

[0153] In the embodiment of the present application, the infrared camera module can capture the infrared image of the front area. The data in the infrared image can reveal the temperature differences and surface features that are invisible to the naked eye. Furthermore, by analyzing the temperature distribution and reflection characteristics in the infrared image, different types of stain types can be identified, such as liquid stains. Therefore, the infrared camera module can also be used to identify liquid stains to facilitate the robot to identify and determine the cleaning strategy.

[0154] Exemplarily, the robot uses the infrared camera module to scan the front area, capture the infrared image, and then by analyzing the temperature distribution and reflection characteristics in the infrared image, different types of stain types can be identified. Further, based on the identified stain type, the corresponding second cleaning strategy is selected. Then the robot can clean the surface to be cleaned in the front area according to the second cleaning strategy and perform the corresponding obstacle avoidance operation according to the first obstacle avoidance strategy during the cleaning process.

[0155] Since infrared images can reveal temperature differences and surface features, enabling the robot to more accurately identify the types of stains and ensure the adoption of appropriate cleaning strategies. Therefore, the robot combines infrared images to identify the types of stains in real time, and then dynamically adjusts the cleaning strategy, enabling the robot to efficiently handle different types of stains while avoiding obstacles. While improving the overall cleaning effect, it can also enhance the robot's adaptability in different environments.

[0156] However, in terms of robot mapping, it is also possible to create a map based on the above-mentioned robot structure. Exemplarily, Figure 6 This is a schematic flowchart of another robot control method provided by an embodiment of the present application. As Figure 6 shown, the robot control method is applied to Figure 1 or Figure 3 the robot shown; the robot control method includes the following steps:

[0157] S601. During the process of the robot executing the target task, determine the first positioning result based on the second sensor data collected by the LDS, and determine the second positioning result based on the image data recognized by the AI camera module.

[0158] In this step, the target task refers to the mapping task. In this way, the LDS emits laser beams, measures the distances to the surrounding environment, generates the second sensor data of the environment, and this second sensor data is used to determine the position and attitude of the robot to form the first positioning result. The AI camera module captures the image data of the environment, analyzes the image data through an image recognition algorithm, and identifies the feature points and markers in the environment to form the second positioning result.

[0159] Exemplarily, the LDS transmits the collected second sensor data to the Simultaneous Localization and Mapping (SLAM) node, so that the SLAM node processes the second sensor data to determine the first positioning result. Correspondingly, the AI camera module transmits the recognized image data to the SLAM node, so that the SLAM node processes the image data to determine the second positioning result.

[0160] It should be noted that the modules or nodes for determining the first positioning result and the second positioning result in the embodiments of the present application are not specifically limited. Optionally, the LDS includes a first SLAM module for determining the first positioning result based on the second sensor data, and the AI camera module includes a second SLAM module for determining the second positioning result based on the image data.

[0161] Among them, the determination process of the first positioning result includes: based on the point cloud data of the LDS, the robot can construct a map of the surrounding environment, and determine its position and orientation in the environment by matching the current point cloud with the known map, obtaining the first positioning result, and the second sensor data is point cloud data.

[0162] The determination process of the second positioning result includes: through the image data recognized by the AI camera module, the robot can recognize specific environmental features, further calibrate and confirm its position, obtaining the second positioning result.

[0163] Optionally, the robot can update the positioning result in real time to quickly respond to environmental changes and improve the adaptability of the robot in a dynamic environment.

[0164] S602. Fuse and process the first positioning result and the second positioning result to construct a grid map.

[0165] Optionally, the grid map is updated in real time according to the first positioning result and the second positioning result of different frames, thereby reflecting the dynamic changes of the environment and enabling the robot to quickly adapt to new situations.

[0166] Exemplarily, convert the first positioning result and the second positioning result into the same coordinate system to ensure that the two types of data can be compared and fused under the same spatial reference framework. Then, in the unified coordinate system, identify and match the common features in the first positioning result and the second positioning result, and use the feature information in the first positioning result to correct the error in the second positioning result, improving the positioning accuracy. Then, convert the corrected data into a grid map.

[0167] It should be noted that the specific process of fusing and processing the first positioning result and the second positioning result in the embodiments of the present application is not limited, and the above is only an example for illustration.

[0168] Optionally, determine the first positioning result based on the second sensor data collected by the LDS, construct a grid map based on the first positioning result. When the first positioning result does not meet the predefined conditions, determine the second positioning result using the image data recognized by the AI camera module, and construct a grid map based on the second positioning result.

[0169] Among them, the predefined conditions can be data loss, excessive noise, signal interference, or data unable to accurately determine the pose, etc. The embodiments of the present application do not make specific limitations on the predefined conditions.

[0170] Therefore, by combining the data of the LDS and the AI camera module, the embodiments of the present application can provide more accurate positioning results, reduce the errors that may be brought by a single sensor, and then through data fusion, the errors that may be generated by a single sensor can be corrected, improving the accuracy of overall positioning and map construction. A more accurate grid map is helpful for optimizing path planning, reducing unnecessary detours and pauses, improving the task execution efficiency. Furthermore, through accurate environmental perception and map construction, the robot can perform tasks more safely, avoiding collisions and other potential risks.

[0171] In the foregoing embodiments, the robot control method provided by the embodiments of the present application has been introduced. In order to implement the various functions in the method provided by the embodiments of the present application, the robot as the execution subject may include a hardware structure and / or software module, and implement the above various functions in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module. Whether a certain function among the above various functions is executed in the form of a hardware structure, a software module, or a combination of a hardware structure and a software module depends on the specific application and design constraints of the technical solution.

[0172] For example, Figure 7 is a schematic structural diagram of a robot control device provided by an embodiment of the present application. As Figure 7 shown, the robot includes: a laser distance sensor LDS located on the side of the fuselage, an artificial intelligence AI camera module, an infrared camera module, and a dual-line laser sensor; the LDS is located at the front center position of the fuselage, the AI camera module and the infrared camera module are located around the LDS in an upper and lower distribution layout, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the robot control device 700 includes:

[0173] A first determination module 701, configured to determine a first height of an obstacle in the front area based on first sensor data collected by the infrared camera module and the dual-line laser sensor during the process of the robot executing a target task; the dual-line laser sensor is used to emit a laser beam, and the infrared camera module is used to capture an infrared image corresponding to the laser beam and determine the first sensor data;

[0174] A control module 702, configured to determine a first obstacle avoidance strategy based on the first height, and control the robot to perform corresponding obstacle avoidance operations based on the first obstacle avoidance strategy.

[0175] Optionally, the robot control device 700 further includes a third determination module, and the third determination module is configured to:

[0176] When it is detected that the first sensor data does not meet the preset conditions, determine a second height of the obstacle based on second sensor data collected by the LDS, and / or determine the type of the obstacle based on image data recognized by the AI camera module;

[0177] Determine a second obstacle avoidance strategy based on the second height and / or the type of the obstacle, and control the robot to perform corresponding obstacle avoidance operations based on the second obstacle avoidance strategy.

[0178] Optionally, the control module 702 is specifically configured to:

[0179] Determine the second height of the obstacle based on the second sensor data collected by the LDS, and determine the type of the obstacle based on the image data recognized by the AI camera module;

[0180] Determine a first obstacle avoidance strategy based on the first height, the second height, and the type of the obstacle.

[0181] Optionally, the robot control device 700 further includes a first cleaning module, and the first cleaning module is configured to:

[0182] Determine the type of dirt in the front area based on the image data recognized by the AI camera module;

[0183] Determine a first cleaning strategy to be adopted during the execution of the obstacle avoidance operation based on the type of dirt, and control the robot to clean the surface to be cleaned in the front area based on the first cleaning strategy.

[0184] Optionally, the robot control device 700 further includes a second cleaning module, and the second cleaning module is configured to:

[0185] Determine the type of stain in the front area based on the infrared image recognized by the infrared camera module;

[0186] Determine a second cleaning strategy to be adopted during the execution of the obstacle avoidance operation based on the type of stain, and control the robot to clean the surface to be cleaned in the front area based on the second cleaning strategy.

[0187] It should be noted that for the specific implementation principles and effects of the above robot control device, reference can be made to the relevant descriptions and effects corresponding to the above embodiments, and details will not be elaborated here.

[0188] Exemplarily, the present application further provides a robot control device, Figure 8 As shown in the structural schematic diagram of another robot control device provided by the embodiment of the present application, Figure 8 the robot includes: a laser distance sensor LDS located on the side of the fuselage, an artificial intelligence AI camera module, an infrared camera module, and a dual-line laser sensor; the LDS is located at the front center position of the fuselage, the AI camera module and the infrared camera module are arranged in an upper and lower distribution layout around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the robot control device 800 includes:

[0189] The second determination module 801 is configured to determine a first positioning result based on second sensor data collected by the LDS and determine a second positioning result based on image data recognized by the AI camera module during the process of the robot executing the target task;

[0190] The construction module 802 is configured to fuse and process the first positioning result and the second positioning result to construct a grid map.

[0191] It should be noted that for the specific implementation principles and effects of the above robot control device, reference can be made to the relevant descriptions and effects corresponding to the above embodiments, and details will not be elaborated here.

[0192] An embodiment of the present application further provides a schematic structural diagram of an electronic device. Figure 9 A schematic structural diagram of an electronic device provided by an embodiment of the present application is shown in Figure 9 As shown, the electronic device may include: a processor 901 and a memory 902 communicatively connected to the processor; the memory 902 stores a computer program; the processor 901 executes the computer program stored in the memory 902, so that the processor 901 executes the method described in any of the above embodiments.

[0193] Among them, the memory 902 and the processor 901 may be connected through a bus 903.

[0194] An embodiment of the present application further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed by a processor, they are used to implement the method described in any of the foregoing embodiments of the present application.

[0195] An embodiment of the present application further provides a chip for running instructions. The chip is used to execute the method described in any of the foregoing embodiments executed by the robot in any of the foregoing embodiments of the present application.

[0196] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, it can implement the method described in any of the foregoing embodiments executed by the robot in any of the foregoing embodiments of the present application.

[0197] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules is only a logical function division. In actual implementation, there may be other division methods. For example, multiple modules or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or modules can be in electrical, mechanical or other forms.

[0198] The modules described as separate components may or may not be physically separated. The components shown as modules may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to implement the solution of this embodiment.

[0199] In addition, in each embodiment of the present application, the functional modules can be integrated in a processing unit, or each module can exist physically alone, or two or more modules can be integrated in one unit. The units formed by the above modules can be implemented in the form of hardware or in the form of a combination of hardware and software functional units.

[0200] The integrated modules implemented in the form of software functional modules can be stored in a computer-readable storage medium. The above software functional modules are stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute some steps of the methods described in various embodiments of the present application.

[0201] It should be understood that the above processor can be a Central Processing Unit (CPU for short), and can also be other general-purpose processors, Digital Signal Processors (DSP for short), Application Specific Integrated Circuits (ASIC for short), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method applied in combination with the application can be directly implemented by the execution of the hardware processor, or can be implemented by the combination of hardware and software modules in the processor.

[0202] The memory may include high-speed random access memory (RAM), and may also include non-volatile memory (NVM), such as at least one disk memory, and may also be a USB flash drive, a mobile hard disk, a read-only memory, a magnetic disk, or an optical disc, etc.

[0203] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0204] The above storage medium may be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic memory, flash memory, magnetic disk or optical disc. The storage medium may be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0205] An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be a component of the processor. The processor and the storage medium may be located in an Application Specific Integrated Circuits (ASIC). Of course, the processor and the storage medium may also exist as discrete components in a cleaning device or a main control device.

[0206] It should be noted that, for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that this application is not limited by the described action sequence, because according to this application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0207] Further, it should be noted that although the steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or sub-steps or stages of other steps.

[0208] In the above embodiments, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of concise description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as falling within the scope described in this specification.

[0209] After considering the specification and practicing the invention of this application, those skilled in the art will easily think of other implementation schemes of this application. This application aims to cover any variations, uses or adaptive changes of this application, and these variations, uses or adaptive changes follow the general principles of this application and include the common general knowledge or conventional technical means in the technical field not claimed in this application. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of this application are pointed out by the claims.

[0210] As mentioned above, the above is only the specific implementation manner of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be subject to the protection scope of the claims.

Claims

1. A robot, characterized in that: The robot includes: a laser ranging sensor LDS, an artificial intelligence AI camera module, an infrared camera module and a dual-line laser sensor located on the side of the fuselage; the LDS is located at the front center of the fuselage, the AI ​​camera module and the infrared camera module are arranged in an upper and lower distribution around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module.

2. The robot according to claim 1, characterized in that: The dual-line laser sensors are symmetrically distributed on the left and right sides of the LDS.

3. The robot according to claim 1, characterized in that: The AI ​​camera module and the infrared camera module are located on the same side around the LDS.

4. The robot according to claim 1, characterized in that: The robot further comprises: a fill light located between the infrared camera module and the dual-line laser sensor.

5. The robot according to claim 1, characterized in that: The robot also includes: a voice module located on the top of the body, and the voice module is used to receive voice commands to control the robot to perform corresponding operations.

6. A robot control method, characterized in that: The robot comprises: a laser distance measuring sensor LDS, an artificial intelligence AI camera module, an infrared camera module and a dual-line laser sensor located on the side of the fuselage; the LDS is located at the front center of the fuselage, the AI ​​camera module and the infrared camera module are arranged in an upper and lower distribution around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the method comprises: In the process of the robot performing the target task, a first height of an obstacle in the front area is determined based on the first sensor data collected by the infrared camera module and the dual-line laser sensor; the dual-line laser sensor is used to emit a laser beam to the obstacle, and the infrared camera module is used to capture an infrared image corresponding to the reflection of the laser beam, and determine the first sensor data; A first obstacle avoidance strategy is determined based on the first height, and the robot is controlled to perform a corresponding obstacle avoidance operation based on the first obstacle avoidance strategy.

7. The method according to claim 6, characterized in that The method further comprises: When it is detected that the first sensor data does not meet the preset condition, determining the second height of the obstacle based on the second sensor data collected by the LDS, and / or determining the type of the obstacle based on the image data recognized by the AI ​​camera module; A second obstacle avoidance strategy is determined based on the second height and / or the type of the obstacle, and the robot is controlled to perform a corresponding obstacle avoidance operation based on the second obstacle avoidance strategy.

8. The method according to claim 6, characterized in that Determining a first obstacle avoidance strategy based on the first height includes: Determine a second height of the obstacle based on the second sensor data collected by the LDS, and determine the type of the obstacle based on the image data recognized by the AI ​​camera module; A first obstacle avoidance strategy is determined based on the first height, the second height, and the type of the obstacle.

9. The method according to claim 6 or 7, characterized in that: The method further comprises: Determine the type of dirt in the front area based on the image data recognized by the AI ​​camera module; A first cleaning strategy to be adopted in the process of performing the obstacle avoidance operation is determined based on the dirt type, and the robot is controlled to clean the surface to be cleaned in the front area based on the first cleaning strategy.

10. The method according to claim 6 or 7, characterized in that: The method further comprises: Determining the type of stain in the front area based on the infrared image recognized by the infrared camera module; A second cleaning strategy to be adopted when performing the obstacle avoidance operation is determined based on the stain type, and the robot is controlled to clean the surface to be cleaned in the front area based on the second cleaning strategy.

11. A robot control method, characterized in that: The robot comprises: a laser distance measuring sensor LDS, an artificial intelligence AI camera module, an infrared camera module and a dual-line laser sensor located on the side of the fuselage; the LDS is located at the front center of the fuselage, the AI ​​camera module and the infrared camera module are arranged in an upper and lower distribution around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the method comprises: During the process of the robot performing the target task, a first positioning result is determined based on the second sensor data collected by the LDS, and a second positioning result is determined based on the image data recognized by the AI ​​camera module; The first positioning result and the second positioning result are fused to construct a grid map.

12. A robot control device, characterized in that: The robot comprises: a laser distance measuring sensor LDS, an artificial intelligence AI camera module, an infrared camera module and a dual-line laser sensor located on the side of the fuselage; the LDS is located at the front center of the fuselage, the AI ​​camera module and the infrared camera module are arranged in an upper and lower distribution around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the device comprises: A first determination module is used to determine a first height of an obstacle in a front area based on first sensor data collected by the infrared camera module and the dual-line laser sensor during the robot's execution of a target task; the dual-line laser sensor is used to emit a laser beam, and the infrared camera module is used to capture an infrared image corresponding to the laser beam, and determine the first sensor data; A control module is used to determine a first obstacle avoidance strategy based on the first height, and control the robot to perform a corresponding obstacle avoidance operation based on the first obstacle avoidance strategy.

13. A robot control device, characterized in that: The robot comprises: a laser distance measuring sensor LDS, an artificial intelligence AI camera module, an infrared camera module and a dual-line laser sensor located on the side of the fuselage; the LDS is located at the front center of the fuselage, the AI ​​camera module and the infrared camera module are arranged in an upper and lower distribution around the LDS, and the dual-line laser sensor is located on the left and right sides of the infrared camera module; the device comprises: A second determination module is used to determine a first positioning result based on the second sensor data collected by the LDS and to determine a second positioning result based on the image data recognized by the AI ​​camera module during the process of the robot performing the target task; A construction module is used to fuse the first positioning result and the second positioning result to construct a grid map.

14. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 6 to 11.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 6 to 11 when executed by a processor.

16. A computer program product, characterized in that The invention comprises a computer program, which implements the method according to any one of claims 6 to 11 when being executed by a processor.

Images