Robot autonomous navigation method and system based on multimode interaction

Abstract

The invention provides a robot autonomous navigation method and system based on multi-mode interaction, belongs to the technical field of computer data processing, and is used for constructing an ability circle taking the state and task demand of a robot as a core, and the ability circle is used for reversely guiding focusing perception of an environment and erosion cost evaluation of a forward constraint path. And the reliability guarantee of online self-optimized navigation decision and task execution is realized. According to the method, the sensing area is focused, all environment information is not passively received, the sensing resources are actively and selectively allocated according to the current most urgent decision-making demand to search for the environment characteristics which are most decisive for current path selection, adjustment from static obstacle avoidance to dynamic environment adaptation is achieved, and the method has the advantages of being simple in structure and convenient to use. And the adaptive capacity in a complex dynamic environment is greatly improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer data processing, and in particular to a robot autonomous navigation method based on multi-mode interaction and a system thereof. BACKGROUND

[0002] A mobile robot is an automated device capable of perceiving the environment through its own sensors and moving autonomously. One of its core technologies is autonomous navigation, which refers to the ability of a robot to autonomously plan a path from a starting point to a target point without human intervention. The effectiveness of autonomous navigation technology directly determines the application effect and intelligent level of mobile robots in fields such as warehouse logistics, security inspection, and special operations.

[0003] The mainstream autonomous navigation technology for mobile robots mainly relies on external environment perception. The robot carries devices such as laser radars or vision sensors to collect point cloud data or image information of the surrounding environment in real time. It uses the Simultaneous Localization and Mapping (SLAM) technology to generate a two-dimensional or three-dimensional grid map of the environment and identifies the positions of obstacles and passable areas on the map. It uses path search methods such as A* algorithm or Dijkstra algorithm to plan a geometric shortest path from the current position to the target point on the map, and finally drives the robot to travel along the path by a motion controller.

[0004] The existing technology has a single dimension of environment perception when dealing with complex and variable actual application scenarios. It usually only focuses on geometric obstacles and ignores physical environmental factors such as road bumps, slippery ground, and environmental noise that have a significant impact on robot energy consumption, safety, and task execution effectiveness. It often takes distance or time as a single optimization target and does not consider internal factors such as the remaining power and load status of the robot. Moreover, the decision weight is fixed and cannot be adjusted flexibly according to the changes in task type to focus on the navigation strategy. There is room for improvement. SUMMARY

[0005] The robot autonomous navigation method based on multi-mode interaction and the system thereof provided by the embodiments of the present application construct a capability circle centered on the state of the robot and the demand of the task. The capability circle reversely guides the focused perception of the environment and positively constrains the erosion cost evaluation of the path, realizes online self-optimization of the navigation decision, and guarantees the reliability of the task execution.

[0006] To achieve the above-mentioned purpose, the present application adopts the following technical solutions: In a first aspect, a robot autonomous navigation method based on multi-mode interaction is provided. The method is applied to a processing terminal and includes: obtaining current state parameters of the robot to generate a set of state parameters; Obtain real-time task requirements, and calculate the adaptive processing object group of the robot's current task capability boundary based on the robot's operation strategy characterization of real-time task requirements; Based on the adaptive processing object group and the real-time task requirements, a focused sensing area is generated to guide environmental data acquisition; Collect multimodal environmental data of the focused sensing area to generate a semantic map with time-varying attributes; With the group of adaptive processing objects as constraints, path search is performed in the semantic map with time-varying attributes, and a path erosion cost set is generated based on the estimated erosion cost of each candidate path for the group of adaptive processing objects. Based on the set of path erosion costs, the candidate path with the lowest expected erosion cost is selected as the optimal navigation path. Control the robot to move along the optimal navigation path, and acquire actual state change data reflecting actual capacity consumption and environmental interaction data reflecting the true attributes of the environment during the movement; Based on the actual state change data and the environmental interaction data, the computational model of the adaptive processing object group and the time-varying attributes in the semantic map with time-varying attributes are updated online.

[0007] Optionally, the step of generating a focused sensing region to guide environmental data acquisition based on the adaptive processing object group and the real-time task requirements specifically includes: Analyze the real-time task requirements to determine the core optimization objectives and constraints of the current task; Analyze the boundary tightness of the adaptive processing object group in each capability dimension to identify the capability dimension with the strongest current constraints; By combining the core optimization objective with the most restrictive capability dimension, configuration parameter adjustment instructions for each environmental perception sensor are generated, and these configuration parameter adjustment instructions are combined into the focused perception area.

[0008] Optionally, real-time task requirements are obtained, and the robot's operational strategy is characterized based on these requirements to calculate the adaptive processing object group that represents the robot's current operational capability boundary. Specifically, this includes: Extract the weight coefficients of each capability dimension from the real-time task requirements; The data in the state parameter set are normalized and converted into the current capability value under each capability dimension; The current capability value is weighted based on the weight coefficients of each capability dimension, and combined with the safety threshold, the adaptive processing object group is defined. The adaptive processing object group is a closed region in the multidimensional capability space enclosed by the safety thresholds of each dimension.

[0009] Optionally, the evaluation of the estimated erosion cost of each candidate path for the adaptive processing object group specifically includes: Extract the time-varying attributes of the semantic map rasters traversed by the candidate path, the time-varying attributes including instantaneous energy consumption coefficient and risk accumulation value; Based on the security thresholds of each dimension in the adaptive processing object group, the basic erosion of the candidate path on each capability dimension is calculated. When the cumulative risk value exceeds the tolerance threshold of the adaptive processing object group in the corresponding risk dimension, a non-linear penalty cost is generated and added to the base erosion amount to obtain the expected erosion cost.

[0010] Optionally, the step of updating the computational model of the adaptive processing object group and the time-varying attributes in the semantic map with time-varying attributes online based on the actual state change data and the environmental interaction data specifically includes: Based on the actual state change data, the actual erosion cost is calculated in reverse. The actual erosion cost is compared with the estimated erosion cost to generate a cost prediction deviation. Based on the cost prediction deviation, the internal parameters used to calculate the expected erosion cost in the adaptive processing object group calculation model are adjusted, and based on the environmental interaction data, the instantaneous energy consumption coefficient and the risk accumulation value of the semantic map grid traversed by the candidate path are corrected.

[0011] Optionally, after completing a full navigation task, the actual state change data and the environmental interaction data throughout the process are integrated to generate an experience data package for this navigation. Based on the aforementioned experience data package, calculate the task success score to evaluate the effectiveness of this task execution. Based on the performance error between the task success score and the expected score, the weight coefficient mapping relationship in the strategy library used to generate the real-time task requirements is iteratively optimized.

[0012] Optionally, after the controlled robot moves along the optimal navigation path and acquires actual state change data reflecting actual capacity consumption and environmental interaction data reflecting the true attributes of the environment during the movement, the system further includes: Based on the actual state change data, the actual erosion rate of each dimension of the adaptive treatment object group is calculated in real time; Determine whether the actual erosion rate exceeds the warning threshold dynamically calculated based on the remaining safety margin in this dimension; If so, the current path following is immediately interrupted, and a local replanning process starting from the robot's current position is triggered, returning to the step of generating the focused perception area to guide environmental data acquisition.

[0013] Optionally, the method for calculating the cumulative risk value includes: Identify the environmental risk type corresponding to the semantic map grid, and determine the current instantaneous risk value based on the intensity of environmental data collected in the focused sensing area; Obtain risk impact data recorded when the robot passes through the grid in the past, and adjust the calculation model of the current instantaneous risk value based on the risk impact data; The current instantaneous risk value is accumulated over time to obtain the updated cumulative risk value.

[0014] Optionally, before obtaining the robot's current state parameters, the method further includes: Receive external instructions containing the task strategy and determine the initial task mode; Based on the initial task mode, the initial optimization target and the initial capability dimension weight coefficient required to generate the real-time task requirements are loaded from the strategy library that stores navigation strategy parameters.

[0015] Secondly, a robot autonomous navigation system based on multimodal interaction is provided, the system being configured to include: The status monitoring module is used to acquire the robot's current status parameters and generate a set of status parameters; The capability circle calculation module is used to calculate the appropriate processing object group based on the real-time task requirements and the set of state parameters. The perception and focusing control module is used to generate a focusing perception area based on the adaptive processing object group and the real-time task requirements; The semantic map management module is used to process multimodal environmental data based on the focused perception area and maintain a semantic map with time-varying attributes. A path planner, constrained by the group of adaptive processing objects, performs path search and erosion cost evaluation in the semantic map to generate an optimal navigation path; The control execution module is used to control the robot to move along the optimal navigation path and coordinate the acquisition of actual state change data and environmental interaction data. The online learning module is used to update the internal parameters of the circle of competence calculation model and the time-varying attributes in the semantic map based on the actual state change data and the environmental interaction data. Thirdly, an electronic device is provided, comprising: a processor and a memory; the memory is used to store a computer program, which, when executed by the processor, causes the electronic device to perform the multimodal interaction-based autonomous robot navigation method described in the first aspect.

[0016] In one possible design, the electronic device described in the third aspect may further include a transceiver. This transceiver may be a transceiver circuit or an interface circuit. The transceiver can be used for communication between the electronic device described in the third aspect and other electronic devices.

[0017] In the embodiments of the present invention, the electronic device described in the third aspect may be a terminal, or a chip (system) or other component or assembly disposed in the terminal, or a system containing the terminal.

[0018] Fourthly, a computer-readable storage medium is provided, comprising: a computer program or instructions; when the computer program or instructions are executed on a computer, the computer causes the computer to perform the multimodal interaction-based autonomous navigation method and system for robots described in the first aspect.

[0019] In summary, the above methods and systems have the following technical effects: This invention achieves a qualitative improvement in navigation capabilities by constructing a dynamic decision-making closed loop centered on the robot's own state. It significantly enhances the reliability of robot navigation decisions and the task completion rate. By introducing an adaptive processing object group as a hard constraint and evaluating path cost as the erosion cost of that capability circle, the robot shifts from pursuing geometric optimality to ensuring survival and task sustainability. Under extreme conditions such as critical power levels or sudden environmental changes, it can make optimal compromises to avoid risks and ensure reaching the destination. This fundamentally solves the shortcomings of traditional methods that fail midway due to neglecting the robot's own state. By focusing on the perception area, the system no longer passively receives all environmental information but actively and selectively allocates perception resources based on the most pressing decision-making needs, searching for environmental features most decisive for the current path selection. This achieves an adjustment from static obstacle avoidance to dynamic adaptation, greatly improving adaptability in complex dynamic environments. Attached Figure Description

[0020] Figure 1 A flowchart illustrating the robot autonomous navigation method and system based on multimodal interaction provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will now be described with reference to the accompanying drawings.

[0022] In this embodiment of the invention, "instruction" can include direct and indirect instructions, as well as explicit and implicit instructions. The information indicated by a certain piece of information is called the information to be instructed. In specific implementation, there are many ways to instruct the information to be instructed, such as, but not limited to, directly instructing the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly instruct the information to be instructed by instructing other information, where there is a correlation between the other information and the information to be instructed. It can also instruct only a part of the information to be instructed, while the other parts are known or pre-agreed upon. For example, the instruction of specific information can be achieved by using a pre-agreed (e.g., protocol-defined) arrangement of various pieces of information, thereby reducing instruction overhead to some extent. Simultaneously, common parts of various pieces of information can be identified and uniformly indicated to reduce the instruction overhead caused by individually indicating the same information.

[0023] Furthermore, the specific indication method can also be any existing indication method, such as, but not limited to, the above-mentioned indication methods and their various combinations. Specific details of various indication methods can be found in existing technologies, and will not be elaborated upon here. As described above, for example, when multiple pieces of information of the same type need to be indicated, the indication methods for different pieces of information may differ. In specific implementation, the required indication method can be selected according to specific needs. This embodiment of the invention does not limit the selected indication method; therefore, the indication methods involved in this embodiment of the invention should be understood to cover various methods that enable the party to be indicated to obtain the information to be indicated.

[0024] It should be understood that the information to be indicated can be sent as a whole or divided into multiple sub-information messages and sent separately. Furthermore, the sending period and / or timing of these sub-information messages can be the same or different. The specific sending method is not limited in this embodiment of the invention. The sending period and / or timing of these sub-information messages can be predefined, for example, according to a protocol, or configured by the sending device by sending configuration information to the receiving device.

[0025] "Predefined" or "pre-configured" can be achieved by pre-saving corresponding codes, tables, or other means that can be used to indicate relevant information in the device. This embodiment of the invention does not limit the specific implementation method. "Saving" can refer to saving in one or more memories. These memories can be separate installations or integrated into the encoder, decoder, processor, or electronic device. Alternatively, some memories can be separate installations, while others are integrated into the decoder, processor, or electronic device. The type of memory can be any form of storage medium, and this embodiment of the invention does not limit this.

[0026] In the embodiments of this invention, "protocol" may refer to a protocol family in the field of communication, a standard protocol with a similar protocol family frame structure, or a related protocol applied to a future multimodal interactive robot autonomous navigation method and system. The embodiments of this invention do not specifically limit this.

[0027] In this embodiment of the invention, descriptions such as "when," "under the circumstances," "if," and "if" all refer to the device making corresponding processing under certain objective circumstances, and are not limited to a specific time. They do not require the device to make a judgment action during implementation, nor do they imply any other limitations.

[0028] In the description of the embodiments of the present invention, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. "And / or" in the embodiments of the present invention is merely a description of the relationship between the related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. Furthermore, in the description of the embodiments of the present invention, unless otherwise stated, "multiple" refers to two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple. Additionally, to facilitate a clear description of the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first," "second," etc., do not limit the quantity or order of execution, and that "first," "second," etc., are not necessarily different. Furthermore, in the embodiments of this invention, words such as "exemplary" or "for example" are used to indicate that something is being described as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this invention should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner for ease of understanding.

[0029] The network architecture and business scenarios described in the embodiments of this invention are for the purpose of more clearly illustrating the technical solutions of the embodiments of this invention, and do not constitute a limitation on the technical solutions provided by the embodiments of this invention. As those skilled in the art will know, with the evolution of network architecture and the emergence of new business scenarios, the technical solutions provided by the embodiments of this invention are also applicable to similar technical problems.

[0030] Figure 1 This is a flowchart illustrating the method provided in an embodiment of the present invention. This multimodal interaction-based autonomous robot navigation method can be applied to the aforementioned processing terminal. The specific process is as follows: Obtain the robot's current state parameters and generate a set of state parameters; Obtain real-time task requirements, and calculate the adaptive processing object group of the robot's current task capability boundary based on the robot's operation strategy characterization of real-time task requirements; Based on the adaptive processing object group and the real-time task requirements, a focused sensing area is generated to guide environmental data acquisition; Collect multimodal environmental data of the focused sensing area to generate a semantic map with time-varying attributes; With the group of adaptive processing objects as constraints, path search is performed in the semantic map with time-varying attributes, and a path erosion cost set is generated based on the estimated erosion cost of each candidate path for the group of adaptive processing objects. Based on the set of path erosion costs, the candidate path with the lowest expected erosion cost is selected as the optimal navigation path. Control the robot to move along the optimal navigation path, and acquire actual state change data reflecting actual capacity consumption and environmental interaction data reflecting the true attributes of the environment during the movement; Based on the actual state change data and the environmental interaction data, the computational model of the adaptive processing object group and the time-varying attributes in the semantic map with time-varying attributes are updated online.

[0031] Specifically: By using real-time perceived state parameters and task requirements, an adaptive processing object group is constructed as a quantitative boundary of the robot's current available resources and safety capabilities. The core is to use this adaptive processing object group to guide the environmental perception process, generate a focused perception area, and enable the sensing end to collect the most critical environmental information for the current decision. Based on this, a semantic map with time-varying attributes is generated. In the path decision-making stage, with the adaptive processing object group as the core constraint, the path with the lowest expected erosion cost for the capability circle is evaluated and selected. By comparing the actual state changes with the predicted results after the path is executed, a feedback loop is formed to update the computational model of the adaptive processing object group and the time-varying attributes of the semantic map online, realizing an intelligent closed loop in which the navigation model evolves with the accumulation of experience.

[0032] The step of generating a focused sensing region to guide environmental data acquisition based on the adaptive processing object group and the real-time task requirements specifically includes: Analyze the real-time task requirements to determine the core optimization objectives and constraints of the current task; Analyze the boundary tightness of the adaptive processing object group in each capability dimension to identify the capability dimension with the strongest current constraints; By combining the core optimization objective with the most restrictive capability dimension, configuration parameter adjustment instructions for each environmental perception sensor are generated, and these configuration parameter adjustment instructions are combined into the focused perception area. Specifically: the robot's perception system transforms from a passive data receiver to an information seeker, and by dynamically adjusting the focus of the sensors, it ensures that the collected environmental data can serve the most urgent navigation decisions to the greatest extent possible. Analyzing real-time task requirements is typically a structured data instruction that transforms abstract task patterns, such as "silent patrol mode" or "emergency transport mode," into specific, quantifiable core optimization objectives and constraints. For example, an emergency transportation mode would be interpreted as a set of instructions with the shortest travel time as the core optimization objective, while taking the degree of bumpiness as a secondary constraint.

[0033] Next, the adaptive processing object group will be analyzed to identify the current most restrictive capability dimension. The adaptive processing object group is a multi-dimensional state space, and each dimension is defined by the robot's current state value and a preset safety threshold. In order to identify the most restrictive capability dimension, the system will calculate the boundary tightness of each dimension.

[0034] Boundary tightness is a dimensionless normalized index used to represent how close the current state is to the safe boundary. Its calculation formula can be expressed as B. i = S i / L i Among them, B i S represents the boundary tightness of the i-th capability dimension. i This represents the robot's current state value in this dimension, such as the current real-time power consumption or accumulated vibration, while L... i This is a preset safety threshold for this dimension, such as the maximum allowable average power consumption calculated based on the remaining battery power and path distance. By comparing the boundary tightness B across all dimensions i The value of the value is used to identify the dimension with the largest value as the current most restrictive capability dimension. For example, when the remaining battery is less than 20%, the boundary tightness of the battery dimension will increase significantly, making it the most restrictive capability dimension. Based on the core optimization objectives and the most restrictive capability dimensions obtained in the first two steps, configuration parameter adjustment instructions for each environmental perception sensor are generated. This is a decision-making process based on rules or mapping tables. For example, when the core optimization objective is the shortest time and the most restrictive capability dimension is battery power, the following configuration parameter adjustment instructions are generated. For vision sensors, the instructions may be to reduce the sampling frequency of 3D reconstruction of distant environments to save computing power, while improving the selection of recognition feature types for nearby ground textures to quickly identify smooth, low rolling resistance road surfaces. For gyroscopes, the instructions aim to increase their sampling frequency to more sensitively capture minute gradient changes that could lead to increased power consumption. For sound sensors, the instructions might focus on identifying specific high-frequency acoustic signatures associated with motor loads. The specific set of instructions together constitutes the focused sensing area, which is ultimately sent down to the underlying sensor driver to achieve dynamic focusing of the sensing.

[0035] Obtain real-time task requirements, characterize the robot's operation strategy based on these requirements, and calculate the adaptive processing object group that represents the robot's current operational capability boundary. Specifically, this includes: Extract the weight coefficients of each capability dimension from the real-time task requirements; The data in the state parameter set are normalized and converted into the current capability value under each capability dimension; The current capability value is weighted based on the weight coefficients of each capability dimension, and combined with the safety threshold, the adaptive processing object group is defined. The adaptive processing object group is a closed region in the multidimensional capability space enclosed by the safety thresholds of each dimension.

[0036] Specifically: by combining the robot's abstract task requirements with its real-time physical state parameters, a multi-dimensional and dynamically changing safe operating boundary is quantified and constructed, namely the adaptive processing object group, providing a clear and quantifiable constraint framework for subsequent path planning. The data in the state parameter set are converted into the current capability value of each capability dimension according to the preset mapping rules. The data is normalized and the engineering quantity is converted, and the original sensor data with different physical dimensions are converted into a unified, dimensionless evaluation value. For example, the current capability value under the power availability dimension can be directly mapped to the robot's remaining power percentage in the range of 0 to 1; the current capability value under the structural stress resistance dimension is calculated by integrating and peak detecting the acceleration data collected by the gyroscope within the most recent time window, such as 3 seconds, to obtain an impact index representing the severity of the turbulence, and then normalizing it. Similarly, the current capability values ​​for the noise tolerance dimension and the task timeliness dimension are generated through similar data processing.

[0037] Based on the weight coefficients of each capability dimension and the defined safety threshold, the adaptive processing object group is generated. The key here is to dynamically calculate the safety threshold of each dimension, which itself constitutes the boundary of the capability circle.

[0038] For the i-th capability dimension, its dynamic security threshold Li can be calculated using the following formula. Li = L base i * f(Wi,P) Among them, L basei is the preset baseline safety limit for this dimension, such as the maximum allowable instantaneous power of the motor or the maximum impact load allowed by the robot's structural design; f is an adjustment function whose input is the weight coefficient Wi of this dimension and other relevant task parameters P, such as the expected total path length; the output is an adjustment factor.

[0039] For example, when the weighting coefficient W of the electricity dimension energy When the value is high, function f will output an adjustment factor less than 1, thereby reducing the safe threshold L for power consumption. power This makes robots more conservative in their planning.

[0040] The N-dimensional space, S, is jointly defined by the safety thresholds Li of all dimensions i. i Less than or equal to L i The resulting set forms the adaptive processing object group, which is a closed area enclosed by safety thresholds in various dimensions. Any potential action outside this area is considered high-risk or infeasible.

[0041] The assessment of the estimated erosion cost of each candidate path for the adaptive processing object group specifically includes: Extract the time-varying attributes of the semantic map rasters traversed by the candidate path, the time-varying attributes including instantaneous energy consumption coefficient and risk accumulation value; Based on the security thresholds of each dimension in the adaptive processing object group, the basic erosion of the candidate path on each capability dimension is calculated. When the cumulative risk value exceeds the tolerance threshold of the adaptive processing object group in the corresponding risk dimension, a non-linear penalty cost is generated and added to the base erosion amount to obtain the expected erosion cost.

[0042] Based on the current boundary of the adaptive processing object group, the basic erosion of the candidate path on each capability dimension is calculated. The purpose is to assess the normal consumption of the robot's core capabilities by the path without triggering extreme risks.

[0043] For each capability dimension i, such as the available power dimension, its base erosion amount E basici It is obtained by accumulating the consumption of all grids j along the path, and its calculation formula can be expressed as follows: Ebasici = Σ ( Ci(j) * dj ) / Li In the formula, Ci(j) is the consumption coefficient of grid j in dimension i, which is its instantaneous energy consumption coefficient for the power dimension; dj is the distance through grid j; and Li is the upper limit of the safety threshold for the current dimension i obtained from the adaptive processing object group. By dividing by the safety threshold Li, the consumption with different physical meanings is uniformly converted into a dimensionless basic erosion amount representing the percentage of capacity consumption in that dimension, ensuring the additivity of erosion amounts in different dimensions.

[0044] When the accumulated risk exceeds the tolerance threshold of the adaptive treatment object group in the corresponding risk dimension, a non-linear penalty cost is added to the base erosion amount to obtain the final estimated erosion cost. This is the core mechanism of risk avoidance in this method.

[0045] The base erosion values ​​across all dimensions are weighted and summed according to task weights to form a base total cost. Then, the system checks the cumulative risk values ​​for each grid cell along the path. Once the cumulative risk value Rk of a grid cell k exceeds its corresponding tolerance threshold Tk, a penalty term Pk is activated. This penalty term Pk is calculated using a non-linear function, for example, Pk = Gk * (Rk / Tk - 1)^α, where Gk is the gain coefficient for that risk type, and α is an exponent greater than 1, such as 2 or 3. This ensures that the penalty cost increases dramatically as the risk exceeds the threshold. Ultimately, the projected erosion cost Eproj equals the base total cost plus the sum of all activated penalty terms along the path. This ensures that even if a path has a low base energy cost, if a small segment carries a serious safety risk, its overall projected erosion cost will become extremely high, leading to its natural abandonment in path selection.

[0046] The step of updating the computational model of the adaptive processing object group and the time-varying attributes in the semantic map with time-varying attributes online based on the actual state change data and the environmental interaction data specifically includes: Based on the actual state change data, the actual erosion cost is calculated in reverse. The actual erosion cost is compared with the estimated erosion cost to generate a cost prediction deviation. Based on the cost prediction deviation, the internal parameters used to calculate the expected erosion cost in the adaptive processing object group calculation model are adjusted, and based on the environmental interaction data, the instantaneous energy consumption coefficient and the risk accumulation value of the semantic map grid traversed by the candidate path are corrected.

[0047] After completing a full navigation task, the actual state change data and environmental interaction data throughout the process are integrated to generate an experience data package for this navigation. Based on the aforementioned experience data package, calculate the task success score to evaluate the effectiveness of this task execution. Based on the performance error between the task success score and the expected score, the weight coefficient mapping relationship in the strategy library used to generate the real-time task requirements is iteratively optimized.

[0048] After the controlled robot moves along the optimal navigation path and acquires actual state change data reflecting actual capacity consumption and environmental interaction data reflecting the true attributes of the environment during the movement, the system further includes: Based on the actual state change data, the actual erosion rate of each dimension of the adaptive treatment object group is calculated in real time; Determine whether the actual erosion rate exceeds the warning threshold dynamically calculated based on the remaining safety margin in this dimension; If so, the current path following is immediately interrupted, and a local replanning process starting from the robot's current position is triggered, returning to the step of generating the focused perception area to guide environmental data acquisition.

[0049] The method for calculating the cumulative risk value includes: Identify the environmental risk type corresponding to the semantic map grid, and determine the current instantaneous risk value based on the intensity of environmental data collected in the focused sensing area; Obtain risk impact data recorded when the robot passes through the grid in the past, and adjust the calculation model of the current instantaneous risk value based on the risk impact data; The current instantaneous risk value is accumulated over time to obtain the updated cumulative risk value.

[0050] Before obtaining the robot's current state parameters, the process also includes: Receive external instructions containing the task strategy and determine the initial task mode; Based on the initial task mode, the initial optimization objective and the initial capability dimension weight coefficients required to generate the real-time task requirements are loaded from the strategy library storing navigation strategy parameters. In another embodiment, it further includes: a robot autonomous navigation system based on multimodal interaction, the system being configured to include: The status monitoring module is used to acquire the robot's current status parameters and generate a set of status parameters; The capability circle calculation module is used to calculate the appropriate processing object group based on the real-time task requirements and the set of state parameters. The perception and focusing control module is used to generate a focusing perception area based on the adaptive processing object group and the real-time task requirements; The semantic map management module is used to process multimodal environmental data based on the focused perception area and maintain a semantic map with time-varying attributes. A path planner, constrained by the group of adaptive processing objects, performs path search and erosion cost evaluation in the semantic map to generate an optimal navigation path; The control execution module is used to control the robot to move along the optimal navigation path and coordinate the acquisition of actual state change data and environmental interaction data. The online learning module is used to update the internal parameters of the circle of competence calculation model and the time-varying attributes in the semantic map based on the actual state change data and the environmental interaction data. Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Exemplarily, the electronic device may be a network device, or a chip (system) or other component or assembly that can be disposed in a network device. Figure 2 As shown, the electronic device 400 may include a processor 401. Optionally, the electronic device 400 may also include a memory 402 and / or a transceiver 403. The processor 401 is coupled to the memory 402 and the transceiver 403, for example, via a communication bus.

[0051] The following is combined with Figure 2 A detailed description of each component of the electronic device 400 is provided below: The processor 401 is the control center of the electronic device 400. It can be a processor or a collective term for multiple processing elements. For example, the processor 401 can be one or more central processing units (CPUs), or application-specific integrated circuits (ASICs), or one or more integrated circuits configured to implement the embodiments of the present invention, such as one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs).

[0052] Optionally, the processor 401 can perform various functions of the electronic device 400 by running or executing software programs stored in the memory 402 and calling data stored in the memory 402, such as performing the aforementioned functions. Figure 2 The robot autonomous navigation method and system based on multimodal interaction are shown.

[0053] In a specific implementation, as one example, processor 401 may include one or more CPUs, for example... Figure 2CPU0 and CPU1 are shown in the diagram.

[0054] In a specific implementation, as one embodiment, the electronic device 400 may also include multiple processors. Each of these processors may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). Here, a processor may refer to one or more devices, circuits, and / or processing cores used for processing data (e.g., computer program instructions).

[0055] The memory 402 is used to store the software program that executes the solution of the present invention, and is controlled by the processor 401 to execute it. The specific implementation method can be referred to the above method embodiment, and will not be repeated here.

[0056] Optionally, the memory 402 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. The memory 402 may be integrated with the processor 401 or exist independently, and may be accessed through the interface circuit of the electronic device 400. Figure 2 (Not shown in the image) is coupled to processor 401, but this embodiment of the invention does not specifically limit this.

[0057] Transceiver 403 is used for communication with other electronic devices. For example, if electronic device 400 is a terminal, transceiver 403 can be used to communicate with network devices or with other terminal devices. As another example, if electronic device 400 is a network device, transceiver 403 can be used to communicate with terminals or with other network devices.

[0058] Alternatively, transceiver 403 may include a receiver and a transmitter. Figure 2 (Not shown separately). The receiver is used to implement the receiving function, and the transmitter is used to implement the transmitting function.

[0059] Optionally, the transceiver 403 can be integrated with the processor 401, or it can exist independently and be connected via the interface circuit of the electronic device 400. Figure 2 (Not shown in the image) is coupled to processor 401, but this embodiment of the invention does not specifically limit this.

[0060] Understandable, Figure 2 The structure of the electronic device 400 shown does not constitute a limitation on the electronic device. Actual electronic devices may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0061] Furthermore, the technical effects of the electronic device 400 can be referred to the technical effects of the robot autonomous navigation method and system based on multimodal interaction described in the above method embodiments, and will not be repeated here.

[0062] It should be understood that the processor in the embodiments of the present invention can be a central processing unit (CPU), or it can be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0063] It should also be understood that the memory in the embodiments of the present invention can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate synchronous DRAM (DDR SDRAM), enhanced synchronous DRAM (ESDRAM), synchronous linked DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0064] The above embodiments can be implemented, in whole or in part, by software, hardware (such as circuits), firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more sets of available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. A semiconductor medium can be a solid-state drive.

[0065] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. A and B can be singular or plural. Additionally, the character " / " in this article generally indicates an "or" relationship between the preceding and following related objects, but it can also represent an "and / or" relationship. Please refer to the context for a more accurate understanding.

[0066] In this invention, "at least" means one or more, and "more than" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be a single item or multiple items.

[0067] It should be understood that, in various embodiments of the present invention, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0068] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0069] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0070] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0071] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0072] In addition, the functional units in the various embodiments of the present invention can be integrated into the processing unit, or each unit can exist physically separately, or two or more units can be integrated into the unit.

[0073] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0074] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A robot autonomous navigation method based on multimodal interaction, characterized in that, The method is applied to a processing terminal, and the method comprises: obtaining current state parameters of the robot, generating a state parameter set; obtaining real-time task requirements, and calculating an adaptive processing object group of the current job capability boundary of the robot based on the real-time task requirements; based on the adaptive processing object group and the real-time task requirements, generating a focused perception area for guiding environmental data collection; collecting multi-modal environmental data of the focused perception area, and generating a semantic map with time-varying attributes; with the adaptive processing object group as a constraint, performing path search in the semantic map with time-varying attributes, and generating a path erosion cost set based on the evaluation of the predicted erosion cost of each candidate path to the adaptive processing object group; according to the path erosion cost set, selecting a candidate path with the minimum predicted erosion cost as an optimal navigation path; controlling the robot to move along the optimal navigation path, and obtaining actual state change data reflecting actual capability consumption and environmental interaction data reflecting real environmental attributes during the movement; based on the actual state change data and the environmental interaction data, updating the calculation model of the adaptive processing object group and the time-varying attributes in the semantic map with time-varying attributes online.

2. The method of claim 1, wherein, The generation of the focused perception area for guiding environmental data collection based on the adaptive processing object group and the real-time task requirements comprises: analyzing the real-time task requirements, determining the core optimization target and constraint index of the current task; analyzing the boundary tightness of the adaptive processing object group in each capability dimension, and identifying the most restrictive capability dimension; combining the core optimization target and the most restrictive capability dimension, generating configuration parameter adjustment instructions for each environmental perception sensor, and combining the set of configuration parameter adjustment instructions into the focused perception area.

3. The method of claim 2, wherein, The adaptive processing object group of the current job capability boundary of the robot calculated based on the real-time task requirements and the robot job strategy comprises: extracting each capability dimension weight coefficient from the real-time task requirements; normalizing each data in the state parameter set to convert it into a current capability value in each capability dimension; based on the each capability dimension weight coefficient, weighting the current capability value, and combining a safety threshold to jointly define the adaptive processing object group, which is a closed area surrounded by each dimension safety threshold in a multi-dimensional capability space.

4. The method of claim 1, wherein, The evaluation of the predicted erosion cost of each candidate path to the adaptive processing object group comprises: extracting the time-varying attributes of the semantic map grid passed by the candidate path, the time-varying attributes including an instantaneous energy consumption coefficient and a risk accumulation value; based on the safety threshold of each dimension in the adaptive processing object group, calculating the basic erosion amount of each capability dimension of the candidate path; when the risk accumulation value exceeds the tolerance threshold of the adaptive processing object group in the corresponding risk dimension, generating a nonlinear penalty cost, and adding the nonlinear penalty cost to the basic erosion amount to obtain the predicted erosion cost.

5. The method of claim 1, wherein, The online updating of the time-varying attribute in the computational model of the adaptive processing object group and the semantic map with time-varying attribute according to the actual state change data and the environment interaction data specifically comprises: Based on the actual state change data, the actual erosion cost is reversely calculated; The actual erosion cost is compared with the predicted erosion cost to generate a cost prediction deviation; According to the cost prediction deviation, the internal parameters for calculating the predicted erosion cost in the adaptive processing object group computational model are adjusted, and the instantaneous energy consumption coefficient and the risk accumulation value of the semantic map grid passed by the candidate path are corrected according to the environment interaction data.

6. The method of claim 5, wherein, After completing a complete navigation task, the actual state change data and the environment interaction data of the whole process are integrated to generate an experience data package of this navigation; Based on the experience data package, a task success degree score evaluating the execution effect of this task is calculated; According to the performance error between the task success degree score and the expected score, the weight coefficient mapping relationship in the strategy library used to generate the real-time task demand is iteratively optimized.

7. The method of claim 5, wherein, The control robot moves along the optimal navigation path, and after obtaining the actual state change data reflecting the actual capacity consumption and the environment interaction data reflecting the real attribute of the environment during the movement, the method further comprises: According to the actual state change data, the actual erosion speed of each dimension of the adaptive processing object group is calculated in real time; It is judged whether the actual erosion speed exceeds the warning threshold dynamically calculated based on the remaining safety margin of the dimension; If yes, the current path following is immediately interrupted, and a local re-planning process with the current position of the robot as the starting point is triggered to return to the step of generating the focused perception area for guiding the environment data collection.

8. The method of claim 4, wherein, The calculation method of the risk accumulation value comprises: The environment risk type corresponding to the semantic map grid is identified, and the current risk instantaneous value is determined according to the environment data intensity collected in the focused perception area; The risk influence data recorded when the robot passes through the grid is obtained, and the calculation model of the current risk instantaneous value is adjusted according to the risk influence data; The current risk instantaneous value is calculated by time decay accumulation to obtain the updated risk accumulation value.

9. The method of claim 1, wherein, Before obtaining the current state parameters of the robot, the method further comprises: An external instruction containing a task strategy is received to determine an initial task mode; According to the initial task mode, the initial optimization target required for generating the real-time task demand and the initial ability dimension weight coefficient are loaded from the strategy library storing the navigation strategy parameters.

10. A multi-modal interaction based robot autonomous navigation system for use in the method of any one of claims 1-9, wherein, The system is configured to comprise: A state monitoring module for obtaining the current state parameters of the robot to generate a state parameter set; An ability circle calculation module for calculating the adaptive processing object group according to the real-time task demand and the state parameter set; A perception focusing control module for generating a focused perception area based on the adaptive processing object group and the real-time task demand; A semantic map management module for processing multi-modal environment data according to the focused perception area and maintaining a semantic map with time-varying attribute; a path planner configured to perform path search and erosion cost evaluation in the semantic map with the constraint of the adaptive handling object group to generate an optimal navigation path; a control execution module configured to control the robot to move along the optimal navigation path and to coordinate acquisition of actual state change data and environment interaction data; an online learning module configured to update internal parameters of the capability circle calculation model and time-varying attributes in the semantic map according to the actual state change data and the environment interaction data.

Images