Intelligent robot multi-task full-autonomous management method supporting environment interaction

Abstract

The invention discloses an intelligent robot multi-task full-autonomous management method supporting environment interaction. The system is applied to an application software architecture of a robot, and the architecture comprises a main module, a data acquisition module, a monitoring module, an image sending module, a sensing planning module, a moving module, a view finding module, a visual navigation module and a data sending module. The method comprises the following steps: determining task types of a main module, a data acquisition module and a monitoring module as normal tasks, and determining task types of other modules as abnormal tasks; the data acquisition module is used for acquiring measurement data sensed by the sensor in real time and storing the measurement data; monitoring an instruction sent by the outside in real time by using a monitoring module; according to a preset scheduling cycle, judging whether the module of which the task type is an abnormal task meets a triggering condition or not by utilizing the main module; if yes, enabling each triggered module to execute a corresponding task; and if not, keeping each untriggered module in a blocked state and waiting for triggering. The execution efficiency of the robot can be improved.

Description

Technical Field

[0001] The present invention relates to the field of extraterrestrial exploration technology, and in particular to a multi-task fully autonomous management method of an intelligent robot supporting environmental interaction. Background Art

[0002] At present, when mobile robots perform tasks, they mostly follow the serial processing method of sensor data collection, environmental perception, navigation positioning, path planning and motion control. Although this method has clear execution logic and is easy to control and manage, the CPU computing power is not fully utilized because the tasks are executed sequentially, which easily leads to performance bottlenecks.

[0003] Especially in the field of extraterrestrial exploration, mobile robots used for extraterrestrial exploration are small in weight and size, and are micro-robots with limited computing power and power consumption. If the serial processing method is still used, it will lead to the risk of low execution efficiency or even failure to complete the task.

[0004] Based on this, there is an urgent need for a multi-task management method that supports environmental interaction for extraterrestrial exploration intelligent robots to solve the above problems and provide a method for intelligent robots operating in complex environments such as extraterrestrial environments to realize embodied intelligence. Summary of the invention

[0005] The present invention provides a multi-task fully autonomous management method for an intelligent robot supporting environmental interaction, which can support environmental interaction, external instructions and data interaction, and reasonably utilize computing resources to improve the execution efficiency of the robot. The technical solution is as follows:

[0006] On the one hand, a multi-task fully autonomous management method of an intelligent robot supporting environmental interaction is provided, which is applied to a processing system inside the robot, wherein the system includes a main module, a data acquisition module, a monitoring module, an image sending module, a perception planning module, a mobile module, a framing module, a navigation module and a data sending module; the main module is respectively connected to other modules in communication, and the data acquisition module is connected to the sensor of the robot in communication; the method includes:

[0007] The task types of the main module, the data acquisition module and the monitoring module are determined as normal tasks, and the task types of other modules are determined as abnormal tasks; the normal tasks are tasks that are always running, and the abnormal tasks are tasks that are not always running;

[0008] The data acquisition module is used to collect and save the environmental data sensed by the sensor in real time; the environmental data includes at least a grayscale image and an RGB image;

[0009] Using the monitoring module to monitor the external commands sent by the wireless communication device in real time, and sending the monitored external commands to the main module;

[0010] According to the preset scheduling cycle, the main module is used to determine whether the module with the task type of abnormal task meets the corresponding triggering condition; if so, each triggered module executes the corresponding task respectively; if not, each untriggered module remains in a blocked state and waits for triggering.

[0011] On the other hand, a multi-task fully autonomous management device of an intelligent robot supporting environmental interaction is provided, which is applied to the application software architecture of the robot, and the architecture includes a main module, a data acquisition module, a monitoring module, an image sending module, a perception planning module, a movement module, a framing module, a navigation module and a data sending module; the main module is respectively connected to other modules in communication, and the data acquisition module is connected to the sensor of the robot in communication; the device includes:

[0012] A determination unit, used to determine the task types of the main module, the data acquisition module and the monitoring module as normal tasks, and the task types of other modules as abnormal tasks; the normal tasks are tasks that are always running, and the abnormal tasks are tasks that are not always running;

[0013] A collection unit, used to collect and save the environmental data sensed by the sensor in real time using the data collection module; the environmental data at least includes a grayscale image and an RGB image;

[0014] A monitoring unit, used to monitor the remote control instructions sent by the outside world through the wireless communication device in real time using the monitoring module, and transmit the monitored instructions to the main module;

[0015] The scheduling unit is used to use the main module to determine whether the module with the task type of abnormal task meets the corresponding triggering condition according to the preset scheduling cycle; if so, each triggered module executes the corresponding task respectively; if not, each untriggered module remains in a blocked state and waits for triggering.

[0016] On the other hand, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method for fully autonomous management of multi-tasks of an intelligent robot supporting environmental interaction are implemented.

[0017] On the other hand, a computer program product is provided, comprising a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for fully autonomous management of multi-tasks of an intelligent robot supporting environmental interaction.

[0018] The embodiment of the present invention provides a multi-task fully autonomous management method for an intelligent robot supporting environmental interaction. First, based on the task requirements, the task type of each module is divided into normal tasks and abnormal tasks. Then, the modules with the task type of normal tasks are kept running to ensure the smooth progress of the necessary tasks. For the modules with the task type of abnormal tasks, the main module is responsible for their task scheduling, and the trigger condition is used as the standard for whether they are executed. In this way, multi-task parallel processing can be achieved, and the robot's data processing capability, resource utilization and response speed can be improved, thereby improving the overall performance of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 This is a flow chart of a multi-task fully autonomous management method of an intelligent robot supporting environmental interaction provided by an embodiment of the present invention;

[0021] Figure 2 This is a structural diagram of an intelligent robot multi-task fully autonomous management device supporting environmental interaction provided by an embodiment of the present invention;

[0022] Figure 3 is a hardware architecture diagram of a computer device provided by an embodiment of the present invention;

[0023] Figure 4 Schematic diagram of an internal processing system of a robot provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0025] The specific implementation of the above concept is described below.

[0026] Please refer to Figure 1 and Figure 4The embodiment of the present invention provides a multi-task fully autonomous management method of an intelligent robot supporting environmental interaction, which is applied to a robot application software architecture. The architecture includes a main module, a data acquisition module, a monitoring module, an image sending module, a perception planning module, a mobile module, a framing module, a navigation module and a data sending module; the main module is respectively connected to other modules in communication, and the data acquisition module is connected to the robot's sensor in communication; the method includes:

[0027] Step 100, determining the task types of the main module, the data acquisition module and the monitoring module as normal tasks, and determining the task types of other modules as abnormal tasks; normal tasks are tasks that are always running, and abnormal tasks are tasks that are not always running;

[0028] Step 102, using a data acquisition module to collect and save environmental data sensed by the sensor in real time; the environmental data includes at least a grayscale image and an RGB image;

[0029] Step 104, using the monitoring module to monitor the external commands received by the wireless communication device in real time, and sending the monitored commands to the main module;

[0030] Step 106, according to the preset scheduling cycle, use the main module to determine whether the module with the task type of abnormal task meets the corresponding triggering condition; if so, each triggered module executes the corresponding task respectively; if not, each untriggered module remains in a blocked state and waits for triggering.

[0031] In this embodiment, first, based on the task requirements, the task types of each module are divided into normal tasks and abnormal tasks. Then, the modules with normal task types are kept running to ensure the smooth progress of the necessary tasks. For the modules with abnormal task types, the main module is responsible for their task scheduling, and the trigger condition is used as the standard for whether they are executed. In this way, multi-task parallel processing can be achieved, the robot's data processing capability, resource utilization and response speed can be improved, thereby improving the overall performance of the robot.

[0032] This application adopts the above-mentioned application software architecture to support tasks such as environmental and self-state measurement data collection, environmental perception, path planning, navigation positioning, motion control, imaging framing, sleep charging, and wireless communication with the outside world (including personnel and other intelligent robots).

[0033] It should be noted that abnormal tasks can be further divided into periodic tasks, non-periodic tasks and short-term tasks. Among them, periodic tasks run according to the set cycle and are executed once every preset time; non-periodic tasks and short-term tasks are executed when specific trigger conditions are met. In addition, the execution frequency of short-term tasks is less than the execution frequency of non-periodic tasks. For example, the task type of each module can be determined according to Table 1:

[0034] Table 1 Project types of each module

[0035]

[0036]

[0037] The following describes the execution process of each module:

[0038] For the main module, after the processing system is started, the main module first assigns initial values ​​to the relevant global variables in the processing system and performs initialization settings. After the initialization is completed, the main module determines the trigger conditions of each module according to the preset scheduling cycle, and schedules the modules whose task types are abnormal tasks according to the trigger conditions.

[0039] For the data acquisition module, after the processing system is initialized, it establishes a connection with the sensor, monitors the corresponding environmental data according to the camera setting variable value, and saves the data in real time in the corresponding variables and system local files. Among them, the environmental data includes at least grayscale images and RGB images. In addition, the environmental data is used to characterize the robot's own state and the interaction information between the robot and the environment.

[0040] For the monitoring module, when the task is started, it establishes a connection with the wireless communication device and monitors external commands. When it receives an external command sent by the wireless communication device, it first determines whether the command is complete and valid according to the agreed communication protocol. If it is a valid command, it responds accordingly according to the command content. If it is an invalid command, it ignores it.

[0041] For the image sending module, its triggering condition is one of the following conditions: the robot reaches the preset initial position, the robot reaches the position where the image is expected to be sent, and receives an image sending instruction from the outside;

[0042] When the image sending module is triggered, it is used to perform the following tasks:

[0043] Based on the task requirements, the grayscale image or RGB image is converted into a data frame according to the communication protocol, and a connection with an external wireless communication device is established to send the converted data frame to the wireless communication device.

[0044] For the perception planning module, its triggering condition is one of the following conditions: the robot is in a stationary state and the waiting movement mileage is not zero, and a movement instruction is received;

[0045] When the perception planning module is triggered, it is used to perform the following tasks:

[0046] Based on the images or point cloud data collected by the data acquisition module, environmental perception calculations are performed to identify various targets or areas in the environment, and path planning is performed according to the mobility of the intelligent robot.

[0047] For the mobile module, the triggering conditions are: the robot is in a non-dormant charging state and has completed path planning and has a safe path;

[0048] When the mobile module is triggered, it is used to perform the following tasks:

[0049] According to the self-state measurement data output by the data acquisition module, including IMU data, solar position, star position and motion mechanism measurement values, the robot's current position and posture are estimated

[0050] Based on the robot's current position, posture and planning results output by the perception planning module, the robot's motion control instructions are obtained;

[0051] Based on motion control instructions, the robot is controlled to move along the planned path.

[0052] In this step, the planning results output by the perception planning module are macro results, such as moving forward 5 meters, while the movement module decomposes the macro results into specific motion control instructions, such as how each wheel moves to make the robot reach the final position.

[0053] For the framing module, the triggering conditions are: the robot reaches the preset imaging position;

[0054] When the viewfinder module is triggered, it is used to perform the following tasks:

[0055] S1, obtaining an image corresponding to the current posture of the mobile robot collected by the data acquisition module, and using the image as input, outputting a detection frame of the target based on a predetermined intelligent framing model, and executing S2;

[0056] S2, calculating the similarity between the detection frame of the target output by the intelligent framing model and the detection frame of the target simulated in advance, and determining whether the similarity reaches a similarity threshold; if so, executing S3, if not, executing S4;

[0057] S3, the robot's posture is not adjusted, and the current posture of the robot is used for framing;

[0058] S4, determine the adjustment result of the robot based on the similarity, and send the adjustment result to the mobile module, so as to use the mobile module to adjust the position and posture of the robot, obtain a new posture of the mobile robot, and return to execute S1 until the similarity reaches the similarity threshold.

[0059] In this step, by continuously adjusting the robot's posture, including moving straight forward, moving straight backward, turning left when moving forward, turning right when moving forward, turning left when moving backward, and turning right when moving backward, the robot can reach the optimal viewing location and meet the task requirements.

[0060] For the navigation module, the triggering conditions are: the robot reaches the preset position for running visual navigation;

[0061] When the navigation module is triggered, it is used to perform the following tasks:

[0062] Acquire the image collected by the data acquisition module;

[0063] Extract feature points based on the collected images and remove erroneous points to obtain the final key points;

[0064] Based on the 2D and 3D information of each final key point, the EPNP algorithm and RANSAC algorithm are used to calculate the position and posture of the robot relative to the target.

[0065] In this step, feature points are extracted based on the collected image and erroneous points are removed to obtain the final key points, including:

[0066] Input the collected image into a pre-trained key point detection model to output the initial key points in the image;

[0067] Determine the probability value of each initial key point based on the heat map output by the prediction model;

[0068] For each initial key point, it is determined whether the probability value of the initial key point is higher than the probability threshold; if not, the initial key point is eliminated; if so, the initial key point is used as the final key point; and so on, until all the final key points are obtained.

[0069] In this step, by eliminating points whose probability values ​​are lower than the probability threshold, it is possible to prevent erroneous key points from entering the subsequent pose estimation, thereby ensuring the correctness of the prediction results. In addition, the probability threshold can be 0.8. Of course, the user can also use other values, which are not specifically limited in this application. In addition, the use of the above two algorithms for pose estimation is a common method in this field, which will not be described here.

[0070] In some implementations, when training the key point detection model, the following loss function is used:

[0071]

[0072] Where L represents the loss function; H and W represent the height and width of the image respectively, i = 1, 2, ... H; j = 1, 2, ... W; Y(pre) ij Represents the predicted value of the position in the i-th row and j-th column of the output heat map; Y(gt) ij W represents the true value of the position of the i-th row and j-th column in the input image; ij represents the weight of the i-th row and j-th column position in the image, σ ij is the mask coefficient at the i-th row and j-th column in the image, σ ij Is 0 or 1.

[0073] In this step, the image of the target of the pre-trained key point detection model is used as input, and the output is a heat map. The heat map is used to represent the possible locations of key points. The larger the value of a point on the heat map, the greater the possibility that the key point appears at that point. In addition, for the weight W ij It is designed based on the heat map. The closer the position is to the key point, the greater the weight, and the farther away from the key point, the smaller the weight.

[0074] In addition, a mask mechanism is added to the loss function. By adding the mask mechanism to the loss function, the problem of the difference in the ratio of positive and negative samples that may occur when using high-resolution target images for key point prediction can be solved, thereby improving the training speed and accuracy of the prediction model.

[0075] Several methods for determining the mask coefficients are described in detail below.

[0076] The first method is:

[0077] Set the mask threshold;

[0078] For the loss function of the current round, the mask coefficients at the positions where the predicted values ​​in the heat map output from the previous round of training are higher than the mask threshold are set to 1, and the mask coefficients at other positions are set to 0.

[0079] This method focuses on those pixels whose scores in the prediction results are higher than the mask threshold. These high-probability pixels can reveal the tendency of the network in predicting key points. Therefore, by setting the mask coefficient in this way, the corresponding pixels can be involved in key point prediction, which can significantly improve the training effect of the prediction model. In addition, this application does not make specific restrictions on the value of the mask threshold, and users can determine it according to actual needs, such as 0.3.

[0080] The second method is:

[0081] A first number of pixel points is determined, and mask coefficients at positions corresponding to the first number of pixel points are set to 1.

[0082] This method selects the first number of pixels by a random method, which can ensure that the prediction model does not overly rely on a specific data pattern and improve the training accuracy of the model. In addition, the first number can be 100, etc., and this application does not make a specific limitation.

[0083] The third method is:

[0084] Taking each key point in the input image as the center, the mask coefficient of the pixel point within the preset range of each key point is set to 1.

[0085] This method focuses on the pixels around the real key points. After undergoing multi-layer convolution processing, the receptive field of these pixels is expanded, which can more effectively capture information related to the key points, thereby improving the training accuracy of the model. In addition, the user can select 8 or more pixels adjacent to the key points, which is not specifically limited in this application.

[0086] It should be noted that the above three methods can be used at the same time, or one or two of the second and third methods can be used at the same time based on the first method. Of course, using the three methods at the same time can maximize the training accuracy of the model.

[0087] In addition, the key point detection model is trained using image sample sets of different styles and brightness conditions, and key points are annotated in each sample image. The training process is a common method in this field and will not be repeated here.

[0088] In addition, this application can also use the following key point confidence to evaluate the overall accuracy of all detected key points:

[0089] acc=num value / num all

[0090] In the formula, acc is the confidence level; num value is the number of final key points; num all is the number of initial key points.

[0091] For the data sending module, the triggering conditions are: reaching the preset sending cycle;

[0092] When the data sending module is triggered, it is used to perform the following tasks:

[0093] Based on the task requirements, the corresponding engineering telemetry data is packaged and connected to the wireless communication equipment to send the packaged engineering telemetry data to the wireless communication equipment.

[0094] In this step, the preset sending period may be 3S, that is, the telemetry data is sent once every 3S. Of course, the user may also use other sending periods, which is not specifically limited in this application.

[0095] Based on the above scheme, this application can be independent of large models and, under effective resource constraints, enable the micro-mobile robot to have the ability to perceive and interact with the environment, and realize the functions of data collection, terrain perception, path planning, navigation and positioning, mobile control, framing imaging, wireless communication, and multi-task autonomous management decision-making of the micro-embodied intelligent mobile robot.

[0096] like Figure 2 , Figure 3 As shown, the embodiment of the present invention provides a multi-task fully autonomous management device for an intelligent robot supporting environmental interaction. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. From the hardware level, Figure 2 As shown in FIG. 1 , a hardware architecture diagram of a computing device in which a multi-task fully autonomous management device of an intelligent robot supporting environmental interaction is provided in an embodiment of the present invention is located. Figure 2 In addition to the processor, memory, network interface, and non-volatile memory shown in the figure, the computing device in which the device is located in the embodiment may also generally include other hardware, such as a forwarding chip responsible for processing messages, etc. Taking software implementation as an example, Figure 3 As shown, as a device in a logical sense, the CPU of the computing device in which it is located reads the corresponding computer program in the non-volatile memory into the internal memory and runs it.

[0097] Please refer to Figure 3 The embodiment of the present invention provides a multi-task fully autonomous management device of an intelligent robot supporting environmental interaction, which is applied to the application software architecture of the robot. The architecture includes a main module, a data acquisition module, a monitoring module, an image sending module, a perception planning module, a mobile module, a framing module, a navigation module and a data sending module; the main module is respectively connected to other modules in communication, and the data acquisition module is connected to the robot's sensor in communication; the device includes:

[0098] The determination unit 300 is used to determine the task types of the main module, the data acquisition module and the monitoring module as normal tasks, and the task types of other modules as abnormal tasks; the normal tasks are tasks that are always running, and the abnormal tasks are tasks that are not always running;

[0099] The acquisition unit 302 is used to use the data acquisition module to collect and save the environmental data sensed by the sensor in real time; the environmental data includes at least a grayscale image and an RGB image;

[0100] The monitoring unit 304 is used to monitor the external commands received by the external world through the wireless communication device in real time by using the monitoring module, and transmit the monitored commands to the main module;

[0101] The scheduling unit 306 is used to use the main module to determine whether the module with the task type of abnormal task meets the corresponding triggering condition according to the preset scheduling cycle; if so, each triggered module executes the corresponding task respectively; if not, each untriggered module remains in a blocked state and waits for triggering.

[0102] In some embodiments, the triggering condition of the image sending module is one of the following conditions: the robot reaches a preset initial position, the robot reaches a position where an image is expected to be sent, or an image sending instruction is received from the outside;

[0103] When the image sending module is triggered, it is used to perform the following tasks:

[0104] Based on the task requirements, the grayscale image or RGB image is converted into a data frame according to the communication protocol, and a connection with an external wireless communication device is established to send the converted data frame to the wireless communication device.

[0105] In some embodiments, the triggering condition of the perception planning module is one of the following conditions: the robot is in a stationary state and the waiting movement mileage is not zero, and a movement instruction is received;

[0106] When the perception planning module is triggered, it is used to perform the following tasks:

[0107] Based on the images or point cloud data collected by the data acquisition module, environmental perception calculations are performed to identify various targets or areas in the environment, and path planning is performed according to the mobility of the intelligent robot.

[0108] In some embodiments, the triggering conditions of the mobile module are: the robot is in a non-dormant charging state and has completed path planning and has a safe path;

[0109] When the mobile module is triggered, it is used to perform the following tasks:

[0110] According to the self-state measurement data output by the data acquisition module, including IMU data, solar position, star position and motion mechanism measurement values, the robot's current position and posture are estimated

[0111] Based on the robot's current position, posture and planning results output by the perception planning module, the robot's motion control instructions are obtained;

[0112] Based on motion control instructions, the robot is controlled to move along the planned path.

[0113] In some embodiments, the triggering condition of the framing module is: the robot reaches a preset imaging position;

[0114] When the viewfinder module is triggered, it is used to perform the following tasks:

[0115] S1, obtaining an image corresponding to the current posture of the mobile robot collected by the data acquisition module, and using the image as input, outputting a detection frame of the target based on a predetermined intelligent framing model, and executing S2;

[0116] S2, calculating the similarity between the detection frame of the target output by the intelligent framing model and the detection frame of the target simulated in advance, and determining whether the similarity reaches a similarity threshold; if so, executing S3, if not, executing S4;

[0117] S3, the robot's posture is not adjusted, and the current posture of the robot is used for framing;

[0118] S4, determine the adjustment result of the robot based on the similarity, and send the adjustment result to the mobile module, so as to use the mobile module to adjust the position and posture of the robot, obtain a new posture of the mobile robot, and return to execute S1 until the similarity reaches the similarity threshold.

[0119] In some embodiments, the triggering condition of the navigation module is: the robot reaches a preset position for running visual navigation;

[0120] When the navigation module is triggered, it is used to perform the following tasks:

[0121] Acquire the image collected by the data acquisition module;

[0122] Extract feature points based on the collected images and remove erroneous points to obtain the final key points;

[0123] Based on the 2D and 3D information of each final key point, the EPNP algorithm and RANSAC algorithm are used to calculate the position and posture of the robot relative to the target.

[0124] In some implementations, the triggering condition of the data sending module is: reaching a preset sending cycle;

[0125] When the data sending module is triggered, it is used to perform the following tasks:

[0126] Based on task requirements, the corresponding engineering data is converted into data frames according to the communication protocol, and a connection with an external wireless communication device is established, and the converted data frames are sent to the wireless communication device.

[0127] It should be noted that the multi-task fully autonomous management device of an intelligent robot supporting environmental interaction provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the multi-task fully autonomous management device of an intelligent robot supporting environmental interaction provided in the above embodiment and the multi-task fully autonomous management method embodiment of an intelligent robot supporting environmental interaction belong to the same concept. The specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0128] The embodiment of the present application also provides a computer device, please refer to Figure 3 The computer device includes a processor and a memory, in which at least one instruction, at least one program, a code set or an instruction set is stored, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the multi-task fully autonomous management method of an intelligent robot supporting environmental interaction provided by the above-mentioned method embodiments.

[0129] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the multi-task fully autonomous management method of an intelligent robot supporting environmental interaction provided by the above-mentioned method embodiments.

[0130] An embodiment of the present application also provides a computer program product, which includes a computer program. A processor of a computer device reads the computer program from a computer-readable storage medium, and the processor executes the computer program, so that the computer device executes the multi-task fully autonomous management method of an intelligent robot supporting environmental interaction as described in any of the above embodiments.

[0131] For the convenience of description, the above system or device is described by dividing it into various modules or units according to its functions. Of course, when implementing the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0132] It can be known from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly contributed to the prior art in the form of a software product, which can be stored in a storage medium such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0133] Finally, it should be noted that, in this article, relational terms such as first, second, third and fourth are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0134] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A multi-task fully autonomous management method for an intelligent robot supporting environmental interaction, characterized in that: Applied to a robot application software architecture, the architecture includes a main module, a data acquisition module, a monitoring module, an image sending module, a perception planning module, a movement module, a framing module, a navigation module and a data sending module; the main module is respectively connected to other modules in communication, and the data acquisition module is connected to the sensor of the robot in communication; the method includes: The task types of the main module, the data acquisition module and the monitoring module are determined as normal tasks, and the task types of other modules are determined as abnormal tasks; the normal tasks are tasks that are always running, and the abnormal tasks are tasks that are not always running; The data acquisition module is used to collect and save the environmental data sensed by the sensor in real time; the environmental data includes at least a grayscale image and an RGB image; Using the monitoring module to monitor the external commands received by the wireless communication device in real time, and sending the monitored commands to the main module; According to the preset scheduling cycle, the main module is used to determine whether the module with the task type of abnormal task meets the corresponding triggering condition; if so, each triggered module executes the corresponding task respectively; if not, each untriggered module remains in a blocked state and waits for triggering.

2. The method according to claim 1, characterized in that: The triggering condition of the image sending module is one of the following conditions: the robot reaches a preset initial position, the robot reaches a position where an image is expected to be sent, or an image sending instruction is received from the outside; When the image sending module is triggered, it is used to perform the following tasks: Based on the task requirements, the grayscale image or the RGB image is converted into a data frame according to the communication protocol, and a connection with an external wireless communication device is established to send the converted data frame to the wireless communication device.

3. The method according to claim 1, characterized in that The triggering condition of the perception planning module is one of the following conditions: the robot is in a stationary state and the waiting movement mileage is not zero, and a movement instruction is received; When the perception planning module is triggered, it is used to perform the following tasks: Based on the images or point cloud data collected by the data acquisition module, environmental perception calculations are performed to identify various targets or areas in the environment, and path planning is performed according to the mobility of the intelligent robot.

4. The method according to claim 3, characterized in that The triggering condition of the mobile module is: the robot is in a non-dormant charging state and has completed path planning and has a safe path; When the mobile module is triggered, it is used to perform the following tasks: According to the self-state measurement data output by the data acquisition module, including IMU data, solar position, star position and motion mechanism measurement values, the robot's current position and posture are estimated Obtaining a motion control instruction for the robot based on the current position and posture of the robot and the planning result output by the perception planning module; The robot is controlled to travel along a planned path based on the motion control instruction.

5. The method according to claim 1, characterized in that The triggering condition of the framing module is: the robot reaches a preset imaging position; When the framing module is triggered, it is used to perform the following tasks: S1, obtaining an image corresponding to the mobile robot in the current position and posture acquired by the data acquisition module, and using the image as input, outputting a detection frame of the target based on a predetermined intelligent framing model, and executing S2; S2, calculating the similarity between the detection frame of the target output by the intelligent framing model and the detection frame of the target simulated in advance, and determining whether the similarity reaches a similarity threshold; If yes, execute S3, if no, execute S4; S3, not adjusting the posture of the robot, and taking a picture with the current posture of the robot; S4, determining the adjustment result of the robot based on the similarity, and sending the adjustment result to the mobile module, so as to use the mobile module to adjust the position and posture of the robot to obtain a new posture of the mobile robot, and return to execute S1 until the similarity reaches a similarity threshold.

6. The method according to claim 1, characterized in that The triggering condition of the navigation module is: the robot reaches the preset position for running visual navigation; When the navigation module is triggered, it is used to perform the following tasks: Acquire the image collected by the data acquisition module; Extract feature points based on the collected images and remove erroneous points to obtain the final key points; Based on the two-dimensional information and three-dimensional information of each final key point, the EPNP algorithm and the RANSAC algorithm are used to calculate the position and posture of the robot relative to the target.

7. The method according to claim 1, characterized in that The triggering condition of the data sending module is: reaching a preset sending cycle; When the data sending module is triggered, it is used to perform the following tasks: Based on the task requirements, the corresponding engineering data is converted into data frames according to the communication protocol, and a connection with the external wireless communication device is established, and the converted data frames are sent to the wireless communication device.

8. An intelligent robot multi-task fully autonomous management device supporting environmental interaction, characterized in that: In the application software architecture applied to the robot, the architecture includes a main module, a data acquisition module, a monitoring module, an image sending module, a perception planning module, a movement module, a framing module, a navigation module and a data sending module; the main module is respectively connected to other modules in communication, and the data acquisition module is connected to the sensor of the robot in communication; the device includes: A determination unit, used to determine the task types of the main module, the data acquisition module and the monitoring module as normal tasks, and the task types of other modules as abnormal tasks; the normal tasks are tasks that are always running, and the abnormal tasks are tasks that are not always running; A collection unit, used to collect and save the environmental data sensed by the sensor in real time using the data collection module; the environmental data at least includes a grayscale image and an RGB image; A monitoring unit, used to monitor external commands received from the outside world through the wireless communication device in real time using the monitoring module, and transmit the monitored commands to the main module; The scheduling unit is used to use the main module to determine whether the module with the task type of abnormal task meets the corresponding triggering condition according to the preset scheduling cycle; if so, each triggered module executes the corresponding task respectively; if not, each untriggered module remains in a blocked state and waits for triggering.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory is used to store a computer program, and the processor is used to execute the computer program stored in the memory to implement the steps of any one of the methods described in claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.

Images