Real-time sensing and planning fusion method for actions and target poses of intelligent robot

Abstract

The invention relates to the technical field of intelligent robots, in particular to a real-time sensing and planning fusion method for actions and target poses of an intelligent robot, which comprises the following steps of: 1, selecting a target, acquiring the pose data of the target in real time, setting the period duration of an action planning period as Tbc, and after each action planning period, acquiring the pose data of the target; determining pose change real features and pose change prediction features of the target in the current motion planning period, and synchronously obtaining pose change prediction features of the target in the previous motion planning period; 2, determining a pose change matching index based on the pose change real characteristics of the target in the current motion planning period and the pose change prediction characteristics of the target in the previous motion planning period, and judging whether to start a first-level adjustment step or not based on the pose change matching index and a pose change matching standard index; 3, when it is judged that the first-level adjustment planning step is not started, whether a second-level adjustment planning step is started or not is judged, real-time tracking and dynamic analysis are conducted on the target pose through a second-level adjustment mechanism, instant deviation quick response and long-term stability guarantee of the action of the intelligent robot are guaranteed, the method is suitable for adaptability expansion of multiple scenes, and the application range is wide. The robustness, the real-time performance, the energy efficiency ratio and the adaptability of intelligent robot control can be ensured in a complex scene of a target.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and more specifically, it relates to a real-time perception and planning fusion method for the actions and target poses of intelligent robots. Background Art

[0002] With the wide application of intelligent robots in the fields of industrial automation, logistics sorting, service robots, etc., its core challenge lies in how to achieve high-precision and low-latency motion planning and execution in a dynamic environment. Traditional methods usually have the following limitations:

[0003] Insufficiency of static assumptions and offline planning: Existing technologies (such as methods based on pre-programmed paths or static environment models) rely on fixed scenario assumptions and cannot adapt to the real-time changes of target poses (such as moving obstacles, sudden target movements). For example, in a logistics sorting scenario, a traditional robotic arm needs to set a pre-grabbing path in advance. If the position of the package shifts due to the vibration of the conveyor belt, it is easy to cause the grabbing to fail.

[0004] Serial processing bottleneck of the perception and planning module: Most solutions adopt a serial process of "perception → planning → execution", resulting in system response delays. For example, in a lidar-based navigation system, environmental perception and path planning are executed step by step. When the target suddenly turns, the robot needs to re-scan the environment and globally re-plan, which significantly increases the time consumption and is difficult to meet the requirements of high-speed dynamic scenarios.

[0005] Multi-source data fusion and error accumulation problems: Existing methods have insufficient ability to fuse multi-modal sensor data (such as vision, IMU, force sense). The limitations of a single sensor (such as vision occlusion, IMU drift) will introduce pose estimation errors. For example, in a narrow space, the vision sensor is easily occluded, resulting in the loss of the target, and traditional methods lack a dynamic weight adjustment mechanism and cannot quickly switch to the mode dominated by lidar data.

[0006] Lack of dynamic adjustment mechanism: Existing motion planning algorithms (such as RRT*, MPC) need to globally re-plan when the environment suddenly changes, consuming a large amount of computing resources and lacking a hierarchical adjustment strategy. For example, when the target slightly deviates from the predicted trajectory, frequent global re-planning will reduce the energy efficiency ratio of the system; while under severe disturbances, local adjustment is not enough to quickly correct the deviation.

[0007] Insufficiency in long-term stability and adaptability to complex scenarios: Traditional methods lack effective utilization of historical data and are difficult to balance instantaneous response and long-term stability. For example, when continuously tracking a target, short-term trajectory fluctuations may trigger mis-adjustments, while long-term accumulated errors cannot be corrected in time.

[0008] Based on the above content, the present invention proposes a real-time perception and planning fusion method for the actions and target poses of intelligent robots. Summary of the Invention

[0009] Aiming at the deficiencies of the existing technology, the purpose of the present invention is to provide a real-time perception and planning fusion method for the actions and target poses of intelligent robots.

[0010] To achieve the above purpose, the present invention provides the following technical solutions:

[0011] A real-time perception and planning fusion method for the actions and target poses of intelligent robots, the steps are as follows:

[0012] Step 1: Select a target, collect the pose data of the target in real time, set the cycle duration of the motion planning cycle as Tbc, and every time an action planning cycle passes, determine the real characteristics and predicted characteristics of the pose change of the target in the current action planning cycle, and synchronously obtain the predicted characteristics of the pose change of the target in the previous action planning cycle;

[0013] Step 2: Determine the pose change matching index based on the real characteristics of the pose change of the target in the current action planning cycle and the predicted characteristics of the pose change of the target in the previous action planning cycle, and determine whether to start the first-level adjustment step of the plan based on the pose change matching index and the pose change matching standard index;

[0014] Step 3: When it is determined not to start the first-level adjustment step of the plan, determine whether to start the second-level adjustment step of the plan.

[0015] Further, determine the real characteristics of the pose change of the target in the current action planning cycle: collect all the pose data of the target in the previous action planning cycle, preprocess and extract the features of all the pose data, and extract the real characteristics of the pose change of the target.

[0016] Further, determine the predicted characteristics of the pose change of the target in the current action planning cycle: obtain the real characteristics of the pose change of the target in the current action planning cycle, obtain the pose change prediction model, use the real characteristics of the pose change of the target as the input data of the pose change prediction model, and output the predicted characteristics of the pose change of the target.

[0017] Further, determine the pose change matching index based on the real characteristics of the pose change of the target in the current action planning cycle and the predicted characteristics of the pose change of the target in the previous action planning cycle, specifically: combine the real characteristics of the pose change of the target in the current action planning cycle and the predicted characteristics of the pose change of the target in the previous action planning cycle into a pose change feature comparison set, use the pose change feature comparison set as the pose change matching analysis model, use the pose change feature comparison set as the input data of the pose change matching analysis model, and output the pose change matching index.

[0018] Further, after starting the first-level planning adjustment step, calculate the ratio of the pose change matching standard index to the pose change matching index to obtain the first-level planning adjustment index SCE, and adjust the cycle duration of the motion planning cycle to Tbc SCE 。

[0019] Further, determine whether to start the second-level planning adjustment step. Specifically: obtain the pose change matching indexes obtained in the previous i motion planning cycles before the current time, and then obtain the average pose change matching continuous fluctuation index and the matching trace swing times. Determine the second-level planning analysis index based on the average pose change matching continuous fluctuation index and the matching trace swing times, and determine whether to start the second-level planning adjustment step based on the comparison result between the second-level planning analysis index and the second-level planning analysis threshold index.

[0020] Further, the specific process of obtaining the average pose change matching continuous fluctuation index is as follows: sort all the pose change matching indexes in the order of output time, calculate the absolute difference between two adjacent pose change matching indexes after sorting to obtain the pose change matching continuous fluctuation index, and calculate the sum mean of all the pose change matching continuous fluctuation indexes to obtain the average pose change matching continuous fluctuation index.

[0021] Further, the specific process of obtaining the matching trace swing times is as follows: compare all the pose change matching indexes in pairs, calculate the absolute difference between the two compared pose change matching indexes to obtain the pose change matching trace swing index, set the pose change matching trace swing threshold index, and when the pose change matching trace swing index is greater than or equal to the pose change matching trace swing threshold index, increase the matching trace swing times by one.

[0022] Further, after starting the second-level planning adjustment step, calculate the ratio of the second-level planning analysis threshold index to the second-level planning analysis index to obtain the second-level planning adjustment index RSW, and adjust the cycle duration of the motion planning cycle to Tbc RSW 。

[0023] Compared with the prior art, the present invention has the following beneficial effects:

[0024] The method of the present invention uses a two-level adjustment mechanism to perform real-time tracking and dynamic analysis on the target pose, ensuring rapid response to instantaneous deviations and long-term stability of the intelligent robot's actions, and is suitable for adaptive expansion in multiple scenarios. It can ensure the robustness, real-time performance, energy efficiency ratio, and adaptability of the intelligent robot control in complex scenarios of the target. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 is a flowchart of the method of the present invention;

[0026] Figure 2 Flowchart for obtaining the pose change matching index;

[0027] Figure 3 Flowchart for determining the secondary adjustment steps for planning. Specific implementation manner

[0028] Refer to Figures 1 to 3 , the real-time perception and planning fusion method for the actions of an intelligent robot and the target pose, the steps are as follows:

[0029] Step 1: Select a target, collect the pose data of the target in real time, set the cycle duration of the motion planning cycle as Tbc (the cycle duration of the motion planning cycle is dynamically adjusted according to the acquisition requirements of the pose data), and every time an action planning cycle passes, determine the real characteristics of the pose change of the target and the predicted characteristics of the pose change under the current action planning cycle, and synchronously obtain the predicted characteristics of the pose change of the target in the previous action planning cycle;

[0030] Determine the real characteristics of the pose change of the target under the current action planning cycle: collect all the pose data of the target in the previous action planning cycle, preprocess and extract features from all the pose data (the preprocessing methods include data cleaning, data normalization, etc., and the feature extraction methods include time-domain feature extraction, frequency-domain feature extraction, etc.), and extract the real characteristics of the pose change of the target;

[0031] Determine the predicted characteristics of the pose change of the target under the current action planning cycle: obtain the real characteristics of the pose change of the target under the current action planning cycle, obtain the pose change prediction model, use the real characteristics of the pose change of the target as the input data of the pose change prediction model, and output the predicted characteristics of the pose change of the target;

[0032] Construction principle of the pose change prediction model: construct a neural network model, collect multiple joint feature vectors, train the neural network model with the real characteristics of the pose change as the training data, assign a predicted characteristic of the pose change to each real characteristic of the pose change, the predicted characteristic of the pose change is the real characteristic of the pose change of the predicted target in the next action planning cycle, divide multiple real characteristics of the pose change into a training set, a validation set and a test set, with a ratio of 70%:15%:15%, perform neural network iterative training on the training set, the validation set and the test set, and after training is completed, construct the pose change prediction model.

[0033] Step 2: Combine the real pose change features of the target in the current motion planning cycle with the predicted pose change features of the target in the previous motion planning cycle into a pose change feature comparison set. Use the pose change feature comparison set as a pose change matching analysis model and the input data of the pose change matching analysis model to output a pose change matching index. Set a pose change matching standard index (the pose change matching standard index is a preset index used to compare with the pose change matching index). When the pose change matching index is greater than the pose change matching standard index, start the first-level planning adjustment step;

[0034] Principle for constructing the pose change matching analysis model: Construct a deep learning model. Collect multiple pose change feature comparison sets, each of which contains a real pose change feature and a predicted pose change feature. Use the pose change feature comparison sets as training data to train the deep learning model. Assign a pose change matching index to each pose change feature comparison set. The value range of the pose change matching index is (0.1 - 3.0). The larger the pose change matching index, the more similar the real pose change feature and the predicted pose change feature in the pose change feature comparison set. Divide the multiple pose change feature comparison sets into a training set and a validation set with a ratio of 60%:40%. Train the training set and the validation set. After training is completed, construct the pose change matching analysis model;

[0035] After starting the first-level planning adjustment step, calculate the ratio of the pose change matching standard index to the pose change matching index to obtain the first-level planning adjustment index SCE, and adjust the cycle duration of the motion planning cycle to Tbc SCE 。

[0036] Step 3: When the pose change matching index is less than or equal to the pose change matching standard index, determine whether to start the second-level planning adjustment step;

[0037] Determine whether to start the secondary planning adjustment step. Specifically: Obtain the pose change matching index obtained in the i action planning cycles before the current time, sort all the pose change matching indexes in the order of the output time, calculate the absolute difference between two adjacent pose change matching indexes after sorting to obtain the pose change matching continuous fluctuation index, calculate the sum average of all the pose change matching continuous fluctuation indexes to obtain the average pose change matching continuous fluctuation index Pftzg, compare all the pose change matching indexes in pairs, calculate the absolute difference between the two compared pose change matching indexes to obtain the pose change matching trace swing index, set the pose change matching trace swing threshold index (the pose change matching trace swing threshold index is a preset index used to compare with the pose change matching trace swing index). When the pose change matching trace swing index is greater than or equal to the pose change matching trace swing threshold index, increase the matching trace swing count by one (when the pose change matching trace swing index is less than the pose change matching trace swing threshold index, no further processing is performed), mark the matching trace swing count as Lsed, calculate the secondary planning analysis index Sew through Sew = (Pftzg + v1) * (Lsed + v2), where v1 is the first coefficient and v2 is the second coefficient. The value of v1 is 1.28 and the value of v2 is 1.51. Set the secondary planning analysis threshold index (the secondary planning analysis threshold index is a preset index used to compare with the secondary planning analysis index). When the secondary planning analysis index is greater than the secondary planning analysis threshold index, start the secondary planning adjustment step (when the secondary planning analysis index is less than or equal to the secondary planning analysis threshold index, no further processing is performed);

[0038] After starting the secondary planning adjustment step, calculate the ratio of the secondary planning analysis threshold index to the secondary planning analysis index to obtain the secondary planning adjustment index RSW, and adjust the cycle duration of the action planning cycle to Tbc RSW 。

[0039] Through the above method, the present invention performs real-time tracking and dynamic analysis on the target pose through a secondary adjustment mechanism, ensuring the rapid response of the instantaneous deviation of the intelligent robot's actions and the long-term stability guarantee, and is applicable to the adaptive expansion of multiple scenarios. It can ensure the robustness, real-time performance, energy efficiency ratio, and adaptability of the intelligent robot control in complex scenarios of the target.

[0040] The above formulas are all dimensionless and take their numerical values for calculation. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0041] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer, or a data storage device such as a server or data center that includes one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0042] It should be understood that in various embodiments of the present application, the order numbers of the above processes do not indicate the order of execution, and the order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0043] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. A professional technician can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0044] Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0045] In several embodiments provided in the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the devices or units can be in electrical, mechanical, or other forms.

[0046] If the above-mentioned functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs and other various media that can store program codes.

[0047] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should all be covered by the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A real-time perception and planning fusion method for intelligent robot motion and target posture, characterized in that: Here are the steps: Step 1: Select a target, collect the target's posture data in real time, set the duration of the action planning cycle to Tbc, and determine the target's actual posture change characteristics and posture change prediction characteristics in the current action planning cycle after each action planning cycle, and simultaneously obtain the posture change prediction characteristics of the target in the previous action planning cycle; Step 2: Determine the posture change matching index based on the real posture change characteristics of the target in the current action planning cycle and the predicted posture change characteristics of the target in the previous action planning cycle, and determine whether to start the first-level adjustment step of the planning based on the posture change matching index and the posture change matching standard index; Step 3: After determining not to initiate the first-level adjustment step of the plan, determine whether to initiate the second-level adjustment step of the plan.

2. The real-time perception and planning fusion method of intelligent robot motion and target posture according to claim 1 is characterized in that: Determine the true characteristics of the target's posture change in the current action planning cycle: collect all the posture data of the target in the previous action planning cycle, preprocess all the posture data and extract features to obtain the true characteristics of the target's posture change.

3. The real-time perception and planning fusion method of intelligent robot motion and target posture according to claim 1 is characterized in that: Determine the target's posture change prediction features in the current action planning cycle: obtain the target's true posture change features in the current action planning cycle, obtain a posture change prediction model, use the target's true posture change features as input data of the posture change prediction model, and output the target's posture change prediction features.

4. The method for real-time perception and planning fusion of intelligent robot motion and target posture according to claim 1 is characterized in that: The pose change matching index is determined based on the real pose change features of the target in the current action planning cycle and the predicted pose change features of the target in the previous action planning cycle. Specifically, the real pose change features of the target in the current action planning cycle and the predicted pose change features of the target in the previous action planning cycle are combined into a pose change feature comparison set, the pose change feature comparison set is used as a pose change matching analysis model, the pose change feature comparison set is used as input data of the pose change matching analysis model, and the pose change matching index is output.

5. The real-time perception and planning fusion method of intelligent robot motion and target posture according to claim 1 is characterized in that: When the first-level adjustment step of planning is started, the ratio of the posture change matching standard index to the posture change matching index is calculated to obtain the first-level planning adjustment index SCE, and the cycle length of the action planning cycle is adjusted to Tbc SCE .

6. The real-time perception and planning fusion method of intelligent robot motion and target posture according to claim 1 is characterized in that: Determine whether to start the secondary adjustment step of the plan, specifically: obtain the posture change matching index obtained in the i action planning cycles before the current time, and then obtain the average posture change matching continuous fluctuation index and the matching retrospective swing number, determine the secondary planning analysis index based on the average posture change matching continuous fluctuation index and the matching retrospective swing number, and determine whether to start the secondary adjustment step of the plan based on the comparison result of the secondary planning analysis index and the secondary planning analysis threshold index.

7. The real-time perception and planning fusion method of intelligent robot motion and target posture according to claim 6 is characterized in that: The specific acquisition process of the average posture change matching continuous fluctuation index is as follows: sort all posture change matching indexes in the order of output time, calculate the absolute difference between the two adjacent posture change matching indexes after sorting, and calculate the posture change matching continuous fluctuation index; calculate the sum and average of all posture change matching continuous fluctuation indexes to obtain the average posture change matching continuous fluctuation index.

8. The method for real-time perception and planning fusion of intelligent robot motion and target posture according to claim 6 is characterized in that: The specific acquisition process of the matching tracing swing number is as follows: all posture change matching indexes are compared in pairs, and the absolute difference between the two compared posture change matching indexes is calculated to obtain the posture change matching tracing swing index, and the posture change matching tracing swing threshold index is set. When the posture change matching tracing swing index is greater than or equal to the posture change matching tracing swing threshold index, the matching tracing swing number is increased by one.

9. The real-time perception and planning fusion method of intelligent robot motion and target posture according to claim 6 is characterized in that: When the planning secondary adjustment step is started, the secondary planning analysis threshold index is calculated by ratio with the secondary planning analysis index to obtain the secondary planning adjustment index RSW, and the cycle length of the action planning cycle is adjusted to Tbc RSW .

Images