Intelligent robot multi-task full autonomous management method supporting environment interaction

Abstract

The application discloses a kind of intelligent robot multitask full autonomous management methods of supporting environmental interaction.The application software architecture applied to robot includes main module, data acquisition module, monitoring module, image sending module, perception planning module, mobile module, viewfinder module, visual navigation module and data sending module;Method includes: the task type of main module, data acquisition module and monitoring module is determined as normal task, the task type of other module is determined as abnormal task;Real-time acquisition measured data perceived by sensor using data acquisition module and save;Real-time monitoring instruction sent by outside using monitoring module;According to preset scheduling period, using main module to judge whether the module of task type abnormal task reaches trigger condition;If yes, make each triggered module execute corresponding task;If not, make each untriggered module keep blocked state and wait for trigger.The present application can improve the execution efficiency of robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of extraterrestrial exploration, and in particular to a multi-task full-autonomous management method for an intelligent robot supporting environmental interaction. BACKGROUND

[0002] At present, when a mobile robot executes a task, it usually adopts a serial processing mode of sensor data collection, environmental perception, navigation positioning, path planning and motion control. Although this mode has clear execution logic and is easy to control and manage, the CPU computing power is not fully utilized due to the sequential execution of tasks, which may lead to performance bottlenecks.

[0003] Especially in the field of extraterrestrial exploration, the mobile robot used for extraterrestrial exploration is small in weight and volume, belonging to a micro robot, and has limited computing power and power consumption. If the serial processing method is still used, it may lead to low execution efficiency and even the risk of failure to complete the task.

[0004] Therefore, there is an urgent need for a multi-task management method for an intelligent robot for extraterrestrial exploration to support environmental interaction to solve the above problems and provide a method for implementing embodied intelligence for intelligent robots operating in complex environments such as extraterrestrial environments. SUMMARY

[0005] The present application provides a multi-task full-autonomous management method for an intelligent robot supporting environmental interaction, which can support environmental interaction, external instruction and data interaction, and reasonably utilize computing resources to improve the execution efficiency of the robot. The technical solution is as follows:

[0006] On the one hand, a multi-task full-autonomous management method for an intelligent robot supporting environmental interaction is provided, which is applied to a processing system inside the robot, and the system includes a main module, a data collection module, a listening module, an image sending module, a perception planning module, a moving module, a viewfinder module, a navigation module and a data sending module; the main module is in communication connection with other modules, the data collection module is in communication connection with the sensors of the robot; the method comprises:

[0007] The task types of the main module, the data collection module and the listening module are determined as normal tasks, and the task types of other modules are determined as abnormal tasks; the normal task is a task that runs all the time, and the abnormal task is a task that does not run all the time;

[0008] The data collection module is used to collect and save the environmental data perceived by the sensors in real time; the environmental data at least includes grayscale images and RGB images;

[0009] The listening module is used to listen to the external instructions sent by the wireless communication equipment in real time, and send the listened external instructions to the main module;

[0010] The scheduling unit is configured to determine, by using the main module, whether a triggered condition is met for a module whose task type is an abnormal task according to a preset scheduling period; if yes, each triggered module performs a corresponding task; if no, each untriggered module remains in a blocked state and waits for a trigger.

[0011] In another aspect, a multi-task full-autonomous management device for an intelligent robot supporting environment interaction is provided, which is applied to an application software architecture of the robot, and the architecture includes a main module, a data acquisition module, a listening module, an image sending module, a perception planning module, a moving module, a viewfinder module, a navigation module, and a data sending module; the main module is in communication connection with other modules; the data acquisition module is in communication connection with a sensor of the robot; the device includes:

[0012] A determination unit is configured to determine a task type of the main module, the data acquisition module, and the listening module as a normal task, and determine a task type of other modules as an abnormal task; the normal task is a task that is always running, and the abnormal task is a task that is not always running;

[0013] An acquisition unit is configured to acquire environment data perceived by the sensor in real time by using the data acquisition module and save the environment data; the environment data at least includes a grayscale image and an RGB image;

[0014] A listening unit is configured to listen to a remote control instruction sent by a wireless communication device from an outside in real time by using the listening module, and transmit the listened instruction to the main module;

[0015] The scheduling unit is configured to determine, by using the main module, whether a triggered condition is met for a module whose task type is an abnormal task according to a preset scheduling period; if yes, each triggered module performs a corresponding task; if no, each untriggered module remains in a blocked state and waits for a trigger.

[0016] In another aspect, a computer readable storage medium is provided, and the storage medium stores a computer program; the computer program is executed by a processor to implement the steps of the multi-task full-autonomous management method for an intelligent robot supporting environment interaction.

[0017] In another aspect, a computer program product is provided, and the computer program product includes a computer program; the computer program is executed by a processor to implement the steps of the multi-task full-autonomous management method for an intelligent robot supporting environment interaction.

[0018] The embodiment of the present application provides a kind of intelligent robot multitask full autonomous management method of supporting environmental interaction.First, based on task requirement, the task type of each module is divided into normal task and abnormal task.Then, the module of task type is normal task always runs, to ensure the smooth progress of necessary task.And for the module of task type being abnormal task, the main module is responsible for its task scheduling, and trigger condition is used as its execution standard.So, multi-task parallel processing can be realized, and the data processing capacity, resource utilization and response speed of robot are improved, so as to improve the overall performance of robot. BRIEF DESCRIPTION OF DRAWINGS

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0020] Figure 1 It is a flow chart of a kind of intelligent robot multitask full autonomous management method of supporting environmental interaction provided by an embodiment of the present application;

[0021] Figure 2 It is a structure diagram of a kind of intelligent robot multitask full autonomous management device provided by an embodiment of the present application;

[0022] Figure 3 It is a hardware architecture diagram of a kind of computer equipment provided by an embodiment of the present application;

[0023] Figure 4 It is a schematic diagram of the internal processing system of robot provided by an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application will be described clearly and completely in the following with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor belong to the scope of protection of the present application.

[0025] The specific implementation of the above concept will be described below.

[0026] Please refer to Figure 1 and Figure 4The embodiment of the application provides a kind of support environment interaction's intelligent robot multi-task full autonomous management method, it is applied to robot application software architecture, architecture includes main module, data acquisition module, monitoring module, image sending module, perception planning module, mobile module, viewfinder module, navigation module and data sending module;Main module is respectively communicated with other module connection, data acquisition module is communicated with the sensor of robot;Method includes:

[0027] Step 100, the task type of main module, data acquisition module and monitoring module is determined as normal task, and the task type of other module is determined as abnormal task;Normal task is a task that is always running, and abnormal task is a task that is not always running;

[0028] Step 102, real-time acquisition sensor perceived environmental data is acquired by using data acquisition module and saved;Environment data at least includes gray image and RGB image;

[0029] Step 104, real-time monitoring external instruction received by wireless communication equipment is monitored by using monitoring module, and the instruction monitored is sent to main module;

[0030] Step 106, according to preset scheduling period, whether the module of which the task type is abnormal task reaches corresponding trigger condition is judged by using main module;If yes, each triggered module executes corresponding task respectively;If not, each untriggered module keeps blocking state and waits for triggering.

[0031] In the embodiment, first, based on task requirements, the task type of each module is divided into normal task and abnormal task.Then, the module of which the task type is normal task is always running, to ensure the smooth progress of necessary task.For the module of which the task type is abnormal task, the main module is responsible for its task scheduling, and the trigger condition is used as the standard whether it is executed.So, multi-task parallel processing can be realized, and the data processing capacity, resource utilization and response speed of robot are improved, so as to improve the overall performance of robot.

[0032] The application can realize environment and self-state measurement data acquisition, environment perception, path planning, navigation positioning and pose, motion control, imaging viewfinder, hibernation charging, wireless communication with external environment (including personnel and other intelligent robots) and other tasks by using the above application software architecture.

[0033] It should be noted that the abnormal task can be further divided into periodic task, non-periodic task and short-time task. Among them, the periodic task is executed once every preset time according to the set period; the non-periodic task and the short-time task are executed when a specific trigger condition is met. In addition, the execution frequency of the short-time task is less than that of the non-periodic task. For example, the task type of each module can be determined according to Table 1:

[0034] Table 1: Project type of each module

[0035]

[0036]

[0037] The execution process of each module will be described below:

[0038] For the main module, after the processing system is started, the main module first assigns initial values to related global variables in the processing system and performs initialization setting. After initialization, the main module judges the trigger condition of each module according to the preset scheduling period, and performs task scheduling on the module whose task type is abnormal task according to the trigger condition.

[0039] For the data acquisition module, after the processing system is initialized, the connection with the sensor is established, the corresponding environmental data is monitored according to the variable value set by the camera, and the data is saved in the corresponding variable and the local file of the system in real time. Among them, the environmental data at least includes gray image and RGB image. In addition, the environmental data is used to represent the state of the robot itself and the interaction information between the robot and the environment.

[0040] For the listening module, when the task is started, the connection with the wireless communication device is established and the external instruction is listened. When it receives the external instruction sent by the wireless communication device, it first judges whether the instruction is complete and valid according to the agreed communication protocol, if it is a valid instruction, it makes corresponding response according to the instruction content, if it is an invalid instruction, it is ignored.

[0041] For the image sending module, its trigger condition is one of the following conditions: the robot reaches the preset initial position, the robot reaches the position where the image is expected to be sent, the image sending instruction is received from the outside;

[0042] When the image sending module is triggered, the following tasks are executed:

[0043] Based on the task requirement, the gray image or RGB image is converted into a data frame according to the communication protocol, and the connection with the external wireless communication device is established to send the converted data frame to the wireless communication device.

[0044] For the perception planning module, the triggering condition is one of the following conditions: the robot is in a stationary state and the distance to be moved is not zero, a movement instruction is received;

[0045] When the perception planning module is triggered, the following tasks are performed:

[0046] Based on the image or point cloud data collected by the data acquisition module, environmental perception calculation is performed to identify various targets or regions in the environment, and path planning is performed according to the movement ability of the intelligent robot.

[0047] For the movement module, the triggering condition is: the robot is in a non-sleep charging state and has completed path planning and has a safe path;

[0048] When the movement module is triggered, the following tasks are performed:

[0049] According to the self-state measurement data output by the data acquisition module, including IMU data, solar orientation, star orientation, and motion mechanism measurement values, the current position and attitude of the robot are estimated

[0050] Based on the current position and attitude of the robot and the planning result output by the perception planning module, the motion control instruction of the robot is obtained;

[0051] Based on the motion control instruction, the robot is controlled to travel according to the planned path.

[0052] In this step, the planning result output by the perception planning module is a macro result, such as advancing 5 meters, and the movement module decomposes the macro result into specific motion control instructions, such as how each wheel moves to make the robot reach the final position.

[0053] For the framing module, the triggering condition is: the robot reaches the preset imaging position;

[0054] When the framing module is triggered, the following tasks are performed:

[0055] S1, obtains the image of the mobile robot corresponding to the current pose collected by the data acquisition module, and takes the image as input, outputs the detection box of the target based on the pre-determined intelligent framing model, and performs S2;

[0056] S2, calculates the similarity of the detection box of the target output by the intelligent framing model and the pre-simulated detection box of the target, and determines whether the similarity reaches the similarity threshold; if yes, perform S3, if no, perform S4;

[0057] S3, do not adjust the pose of the robot, and frame with the current pose of the robot;

[0058] S4, determining the adjustment result of the robot based on the similarity, and sending the adjustment result to the moving module to adjust the position and pose of the robot by using the moving module, obtaining a new pose of the mobile robot, and returning to perform S1 until the similarity reaches a similarity threshold.

[0059] In this step, by continuously adjusting the pose of the robot, including straight forward, straight backward, forward left turn, forward right turn, backward left turn and backward right turn, etc., the robot can reach the optimal shooting location to meet the task requirements.

[0060] For the navigation module, the triggering condition is that the robot reaches the preset position of the running visual navigation;

[0061] When the navigation module is triggered, the following tasks are performed:

[0062] Obtaining the image collected by the data collection module;

[0063] Based on the collected image, feature points are extracted and error points are removed to obtain final key points;

[0064] Based on the two-dimensional information and three-dimensional information of each final key point, the EPNP algorithm and the RANSAC algorithm are used to calculate the position and pose of the robot relative to the target.

[0065] In this step, based on the collected image, feature points are extracted and error points are removed to obtain final key points, including:

[0066] Inputting the collected image into a pre-trained key point detection model to output initial key points in the image;

[0067] Determining the probability value of each initial key point based on the heat map output by the prediction model;

[0068] For each initial key point, it is judged whether the probability value of the initial key point is higher than a probability threshold; if not, the initial key point is removed; if yes, the initial key point is taken as a final key point; and so on, until all final key points are obtained.

[0069] In this step, by removing points with probability values lower than the probability threshold, it can be prevented that error key points enter subsequent pose estimation, and the correctness of the prediction result is ensured. In addition, the probability threshold can be 0.8, of course, the user can also use other numerical values, which are not limited in the present application. In addition, using the above two algorithms for pose estimation is a common means in the art, which will not be described here.

[0070] In some embodiments, when training the key point detection model, the following loss function is used:

[0071]

[0072] In the formula, L represents a loss function; H and W represent the height and width of an image respectively, i = 1, 2, …, H; j = 1, 2, …, W; Y(pre) ij represents a predicted value of an i-th row and j-th column position in an output heat map; Y(gt) ij represents a true value of an i-th row and j-th column position in an input image; W ij represents a weight of an i-th row and j-th column position in an image, σ ij is a mask coefficient of an i-th row and j-th column position in an image, σ ij is 0 or 1.

[0073] In this step, an image in which a pre-trained key point detection model targets are input, and an output is a heat map. The heat map is used to represent the position where the key point may appear. The greater the value of a certain point on the heat map, the greater the possibility that the key point appears at the point position. In addition, the weight W ij is designed on the basis of the heat map. The closer to the position of the key point, the greater the weight, and the farther away from the position of the key point, the smaller the weight.

[0074] In addition, the mask mechanism is added to the loss function. By adding the mask mechanism to the loss function, the problem of the difference in the proportion of positive and negative samples when predicting key points by using a high-resolution target image can be solved, and the training speed and accuracy of the prediction model can be improved.

[0075] The following specifically describes several determination methods of the mask coefficient.

[0076] The first method is:

[0077] A mask threshold is set.

[0078] For the loss function of the current round, the mask coefficient of the position where the predicted value of the heat map output in the last round of training is higher than the mask threshold is set to 1, and the mask coefficient of other positions is set to 0.

[0079] This method focuses on those pixel points whose scores in the prediction result are higher than the mask threshold. These high-probability pixel points can reveal the tendency of the network when predicting key points. Therefore, by setting the mask coefficient through this method, the corresponding pixel points participate in key point prediction, which can significantly improve the training effect of the prediction model. In addition, the value of the mask threshold is not specifically limited in the present application, and the user can determine it according to actual needs, such as 0.3.

[0080] The second method is:

[0081] A first number of pixel points is determined, and the mask coefficient of the position corresponding to the first number of pixel points is set to 1.

[0082] This way, by randomly selecting the first number of pixel points, it can ensure that the prediction model does not rely too much on specific data patterns, improving the training accuracy of the model. In addition, the first number can be 100, etc., which is not limited in the present application.

[0083] The third way is:

[0084] Respectively, taking each key point in the input image as the center, the mask coefficient of the pixel point in the preset range of each key point is set to 1.

[0085] This way focuses on the pixel points around the real key points. After experiencing multiple convolution processing, the receptive field of these pixel points is expanded, which can more effectively capture the information related to the key points, thereby providing the training accuracy of the model. In addition, the user can take the pixel points in the range of 8 or more adjacent to the key points, which is not limited in the present application.

[0086] It should be noted that the above three ways can be used at the same time, or one or two of the second and third ways can be used at the same time based on the use of the first way. Of course, the simultaneous use of the three ways can maximize the training accuracy of the model.

[0087] In addition, the key point detection model is trained using image sample sets with different styles and different brightness conditions, and each sample image is labeled with key points. Its training process is a common way in the art, which will not be repeated here.

[0088] In addition, the present application can also use the following key point confidence to evaluate the overall accuracy of all detected key points:

[0089] acc=num value / num all

[0090] In the formula, acc is the confidence; num value is the number of final key points; num all is the number of initial key points.

[0091] For the data sending module, the triggering condition is that the preset sending period is reached;

[0092] When the data sending module is triggered, the following tasks are performed:

[0093] Based on the task requirements, the corresponding engineering telemetry data is packaged and connected with the wireless communication device to send the packaged engineering telemetry data to the wireless communication device.

[0094] In this step, the preset sending period can be 3S, that is, telemetry data is sent once every 3S, of course, the user can also use other sending periods, and the application does not make specific limitations.

[0095] Based on the above scheme, the application can not depend on a large model, and under effective resource constraints, a small mobile robot can have the ability to perceive and interact with the environment, and realize functions such as data acquisition, terrain perception, path planning, navigation positioning, mobile control, imaging, wireless communication, and multi-task autonomous management decision of a small embodied intelligent mobile robot.

[0096] As shown in Figure 2 , Figure 3 , the embodiment of the application provides a multi-task full-autonomous management device of an intelligent robot supporting environment interaction. The device embodiment can be realized by software, or realized by hardware or a combination of software and hardware. From the hardware layer, as shown in Figure 2 , a hardware architecture diagram of a computing device where the multi-task full-autonomous management device of an intelligent robot supporting environment interaction provided by the embodiment of the application is located, in addition to the processor, the memory, the network interface, and the non-volatile memory shown in Figure 2 , the computing device where the device in the embodiment is usually also composed of other hardware, such as a forwarding chip responsible for processing messages and the like. Taking the software implementation as an example, as shown in Figure 3 , as a logically meaningful device, it is formed by the CPU of the computing device where it is located reading the corresponding computer program in the non-volatile memory into the memory for running.

[0097] Please refer to Figure 3 , the embodiment of the application provides a multi-task full-autonomous management device of an intelligent robot supporting environment interaction, which is applied to the application software architecture of a robot, and the architecture includes a main module, a data acquisition module, a listening module, an image sending module, a perception planning module, a mobile module, a viewfinder module, a navigation module, and a data sending module. The main module is in communication connection with other modules, and the data acquisition module is in communication connection with a sensor of the robot. The device includes:

[0098] A determination unit 300 is configured to determine the task type of the main module, the data acquisition module, and the listening module as a normal task, and determine the task type of other modules as an abnormal task. The normal task is a task that is always running, and the abnormal task is a task that is not always running.

[0099] An acquisition unit 302 is configured to acquire and save environment data perceived by the sensor in real time by using the data acquisition module. The environment data at least includes a grayscale image and an RGB image.

[0100] The monitoring unit 304 is configured to monitor instructions received by the wireless communication device from the outside world in real time by using the monitoring module, and transmit the monitored instructions to the main module.

[0101] The scheduling unit 306 is configured to determine whether the modules whose task types are abnormal tasks reach corresponding trigger conditions according to a preset scheduling period by using the main module; if yes, each triggered module performs a corresponding task; if no, each untriggered module remains in a blocked state and waits for triggering.

[0102] In some embodiments, the trigger condition of the image sending module is one of the following conditions: the robot reaches a preset initial position, the robot reaches a position where an image is expected to be sent, and an image sending instruction from the outside world is received.

[0103] When the image sending module is triggered, the following tasks are performed:

[0104] The grayscale image or the RGB image is converted into a data frame according to a communication protocol based on a task requirement, and a connection with a wireless communication device in the outside world is established, so that the converted data frame is sent to the wireless communication device.

[0105] In some embodiments, the trigger condition of the perception planning module is one of the following conditions: the robot is in a stationary state and the distance to be moved is not zero, and a movement instruction is received.

[0106] When the perception planning module is triggered, the following tasks are performed:

[0107] Based on the image or the point cloud data collected by the data acquisition module, environment perception calculation is performed, various targets or regions in the environment are identified, and path planning is performed according to the movement ability of the intelligent robot.

[0108] In some embodiments, the trigger condition of the movement module is that the robot is in a non-sleep charging state, the path planning has been completed, and there is a safe path.

[0109] When the movement module is triggered, the following tasks are performed:

[0110] According to the self-state measurement data output by the data acquisition module, including IMU data, solar orientation, star orientation and movement mechanism measurement values, the position and attitude of the robot at the current time are estimated.

[0111] Based on the position and attitude of the robot at the current time and the planning result output by the perception planning module, a movement control instruction of the robot is obtained.

[0112] The robot is controlled to travel along the planned path based on the movement control instruction.

[0113] In some embodiments, the triggering condition of the framing module is that the robot reaches a preset imaging position.

[0114] When the framing module is triggered, the following tasks are performed:

[0115] S1, obtaining an image corresponding to the mobile robot at the current pose collected by the data collection module, taking the image as input, outputting a detection box of the target based on a predetermined intelligent framing model, and performing S2;

[0116] S2, calculating the similarity between the detection box of the target output by the intelligent framing model and the detection box of the target simulated in advance, and determining whether the similarity reaches a similarity threshold; if yes, performing S3, and if no, performing S4;

[0117] S3, without adjusting the pose of the robot, framing with the current pose of the robot;

[0118] S4, determining the adjustment result of the robot based on the similarity, and sending the adjustment result to the movement module to adjust the position and attitude of the robot using the movement module to obtain a new pose of the mobile robot, and returning to perform S1 until the similarity reaches the similarity threshold.

[0119] In some embodiments, the triggering condition of the navigation module is that the robot reaches a preset position for running visual navigation;

[0120] When the navigation module is triggered, the following tasks are performed:

[0121] Obtaining an image collected by the data collection module;

[0122] Based on the collected image, feature points are extracted and error points are removed to obtain final key points;

[0123] Based on the two-dimensional information and three-dimensional information of each final key point, the EPNP algorithm and the RANSAC algorithm are used to calculate the position and attitude of the robot relative to the target.

[0124] In some embodiments, the triggering condition of the data sending module is that a preset sending period is reached.

[0125] When the data sending module is triggered, the following tasks are performed:

[0126] Based on the task requirements, the corresponding engineering data is converted into a data frame according to the communication protocol, a connection with the external wireless communication device is established, and the converted data frame is sent to the wireless communication device.

[0127] It should be noted that the above-mentioned embodiments of the intelligent robot multi-task full-autonomous management device supporting environment interaction are only used as examples for the division of the above-mentioned functional modules, and in actual applications, the above-mentioned functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the above-mentioned embodiments of the intelligent robot multi-task full-autonomous management device supporting environment interaction and the method embodiments of the intelligent robot multi-task full-autonomous management device supporting environment interaction belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0128] Embodiments of the present application also provide a computer device, which comprises a processor and a memory, and the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the intelligent robot multi-task full-autonomous management method supporting environment interaction provided by the above-mentioned method embodiments. Figure 3

[0129] Embodiments of the present application also provide a computer readable storage medium, which stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the intelligent robot multi-task full-autonomous management method supporting environment interaction provided by the above-mentioned method embodiments.

[0130] Embodiments of the present application also provide a computer program product, which comprises a computer program, and the processor of the computer device reads the computer program from the computer readable storage medium, and the processor executes the computer program, so that the computer device executes the intelligent robot multi-task full-autonomous management method supporting environment interaction described in any of the above-mentioned embodiments.

[0131] For the convenience of description, the above system or device is described as various modules or units respectively described in terms of functions. Of course, in the implementation of the present application, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0132] ​Those skilled in the art can clearly understand the application by the description of the above embodiments. The technical solutions of the application can be implemented by means of software and necessary universal hardware platforms. Based on such an understanding, the technical solutions of the application can be embodied in the form of a software product, which can be stored in a storage medium, such as a ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the application.

[0133] Finally, it should be noted that the terms such as first, second, third, and fourth, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply that there is any such actual relationship or order between these entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0134] The above description is only the preferred embodiments of the application, and it should be pointed out that those skilled in the art can make some improvements and refinements without departing from the principles of the application, and these improvements and refinements should also be regarded as the protection scope of the application.

Claims

1. A method for intelligent robot multitask full autonomous management supporting environmental interaction, characterized in that, The application is applied to a robot application software architecture, the architecture includes a main module, a data acquisition module, a listening module, an image sending module, a perception planning module, a moving module, a viewfinder module, a navigation module and a data sending module; the main module is in communication connection with other modules respectively, the data acquisition module is in communication connection with a sensor of the robot; the method includes: The task type of the main module, the data acquisition module and the listening module is determined as a normal task, and the task type of other modules is determined as an abnormal task; the normal task is a task running all the time, and the abnormal task is a task not running all the time; The data acquisition module is used for collecting environment data perceived by the sensor in real time and saving; the environment data at least includes a grayscale image and an RGB image; The listening module is used for listening to external instructions received by a wireless communication device in real time, and sending the listened instructions to the main module; According to a preset scheduling period, the main module is used for judging whether a module with a task type of an abnormal task reaches a corresponding trigger condition; if yes, each triggered module executes a corresponding task respectively; if not, each untriggered module keeps a blocked state and waits for triggering.

2. The method of claim 1, wherein, The trigger condition of the image sending module is one of the following conditions: the robot reaches a preset initial position, the robot reaches a position where an image is expected to be sent, and an image sending instruction is received from outside; When the image sending module is triggered, the following tasks are executed: According to a task requirement, the grayscale image or the RGB image is converted into a data frame according to a communication protocol, and a connection with a wireless communication device outside is established, so as to send the converted data frame to the wireless communication device.

3. The method of claim 1, wherein, The trigger condition of the perception planning module is one of the following conditions: the robot is in a static state and a to-be-moved mileage is not zero, and a moving instruction is received; When the perception planning module is triggered, the following tasks are executed: Based on image or point cloud data collected by the data acquisition module, environment perception calculation is performed, various targets or regions in the environment are identified, and path planning is performed according to the moving ability of the intelligent robot.

4. The method of claim 3, wherein, The trigger condition of the moving module is that the robot is in a non-sleep charging state, path planning is completed, and there is a safe path; When the moving module is triggered, the following tasks are executed: According to self-state measurement data output by the data acquisition module, including IMU data, solar orientation, star orientation and motion mechanism measurement values, the position and attitude of the robot at the current time are estimated Based on the position and attitude of the robot at the current time and the planning result output by the perception planning module, a motion control instruction of the robot is obtained; The robot is controlled to travel according to the planning path based on the motion control instruction.

5. The method of claim 1, wherein, The trigger condition of the viewfinder module is that the robot reaches a preset imaging position; When the viewfinder module is triggered, the following tasks are executed: S1, acquiring an image corresponding to the mobile robot in the current pose collected by the data collection module, taking the image as an input, outputting a detection box of a target based on a pre-determined intelligent framing model, and performing S2; S2, calculating the similarity between the detection box of the target output by the intelligent framing model and the detection box of the target simulated in advance, and determining whether the similarity reaches a similarity threshold; if yes, performing S3, and if no, performing S4; S3, without adjusting the pose of the robot, framing with the current pose of the robot; S4, determining the adjustment result of the robot based on the similarity, and sending the adjustment result to the movement module to adjust the position and attitude of the robot by using the movement module, obtaining a new pose of the mobile robot, and returning to perform S1 until the similarity reaches the similarity threshold.

6. The method of claim 1, wherein, The triggering condition of the navigation module is that the robot reaches a pre-set position for running visual navigation. When the navigation module is triggered, the following tasks are performed: acquiring the image collected by the data collection module; performing feature point extraction and error point elimination based on the collected image to obtain final key points; based on the two-dimensional information and three-dimensional information of each final key point, using EPNP algorithm and RANSAC algorithm to calculate the position and attitude of the robot relative to the target.

7. The method of claim 1, wherein, The triggering condition of the data sending module is that a pre-set sending period is reached. When the data sending module is triggered, the following tasks are performed: based on the task requirements, converting the corresponding engineering data into a data frame according to the communication protocol, establishing a connection with the external wireless communication device, and sending the converted data frame to the wireless communication device.

8. A smart robot multi-task full autonomous management device supporting environmental interaction, characterized in that, The application software architecture applied to the robot includes a main module, a data collection module, a listening module, an image sending module, a perception planning module, a movement module, a framing module, a navigation module, and a data sending module; the main module is in communication connection with other modules, and the data collection module is in communication connection with the sensor of the robot; the device comprises: a determination unit configured to determine the task type of the main module, the data collection module, and the listening module as a normal task, and the task type of other modules as an abnormal task; the normal task is a task that runs all the time, and the abnormal task is a task that does not run all the time; an acquisition unit configured to acquire and save the environmental data sensed by the sensor in real time by using the data collection module; the environmental data at least includes a grayscale image and an RGB image; a listening unit configured to listen to the external instructions received by the wireless communication device in real time by using the listening module, and deliver the listened instructions to the main module; a scheduling unit configured to determine whether the modules whose task types are abnormal tasks reach corresponding triggering conditions according to a pre-set scheduling period by using the main module; if yes, each triggered module performs a corresponding task; and if no, each un-triggered module remains in a blocked state and waits for triggering.

9. A computer device, comprising: The computer device comprises a memory for storing a computer program and a processor for executing the computer program stored in the memory to implement the steps of the method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the method according to any one of claims 1-7.

Images