Robot adaptive decision-making method and system based on artificial intelligence algorithm

Abstract

The invention relates to a robot adaptive decision-making method and system based on an artificial intelligence algorithm. The method comprises the following steps: carrying out dimension reduction processing on environmental data through a feature compression coding algorithm to obtain a compressed feature vector; constructing an annealing strategy space by adopting a simulated annealing algorithm, and generating a candidate strategy set; the global exploration layer generates an exploration strategy based on strategy disturbance and obtains an instant reward value through environment interaction based on the exploration strategy, and the local optimization layer updates a Q value function based on a thermodynamic gradient rule; and adjusting temperature parameters of the simulated annealing algorithm through the updated Q value function and the instant reward value, and outputting a final execution strategy through strategy selection and model parameter updating. According to the method, through the technical means of multi-source data fusion, simulated annealing and reinforcement learning cooperation, dynamic temperature adjustment and the like, the global optimality and dynamic adaptability of a robot decision strategy are remarkably improved, and the robustness in a complex environment is enhanced.

Description

Technical Field

[0001] The present invention belongs to the field of robot control technology, and in particular relates to a robot adaptive decision-making method and system based on artificial intelligence algorithms. Background Art

[0002] Autonomous robot decision-making is a core research direction in the field of artificial intelligence. Its goal is to enable robots to perceive, generate, and optimize strategies in real time in dynamic environments, enabling their application in scenarios such as industrial automation, intelligent warehousing, and service robotics. However, with the advancement of sensor technology and algorithms, the complexity of the environments and the diversity of tasks robots face have increased significantly, and the limitations of traditional decision-making methods in multimodal data processing, global optimization capabilities, and dynamic adaptability have become increasingly prominent. For example, traditional robot decision-making methods typically rely on single sensor data or simple multimodal data splicing, making it difficult to effectively process high-dimensional, heterogeneous information such as visual images and three-dimensional point clouds, resulting in insufficient environmental perception accuracy. Furthermore, when this method generates candidate strategies using a fixed temperature decay rate based on the traditional simulated annealing algorithm, it is unable to adjust the exploration intensity based on strategy quality or environmental changes, resulting in slow convergence or premature termination of exploration, making it difficult to adapt to dynamic environmental changes. Summary of the Invention

[0003] Based on this, it is necessary to provide a robot adaptive decision-making method and system based on artificial intelligence algorithms to address the above technical problems, so as to improve the accuracy of environmental perception and the robustness of decision-making strategies of robots in complex dynamic environments, and further enhance the efficiency and stability of autonomous decision-making of robots.

[0004] In a first aspect, the present application provides a robot adaptive decision-making method based on an artificial intelligence algorithm, comprising:

[0005] Based on the real-time environmental data collected by the multimodal sensors on the robot, the environmental data is reduced in dimension using a feature compression coding algorithm to obtain a compressed feature vector. The environmental data includes visual image data, lidar distance data, and inertial measurement data.

[0006] Using the simulated annealing algorithm, based on the compressed feature vector and the preset current strategy library, the annealing strategy space is constructed to generate a candidate strategy set, which is a strategy that meets the preset energy constraint conditions.

[0007] Based on the candidate strategy set, a two-layer decision optimization is performed through the collaborative optimization mechanism of the global exploration layer and the local optimization layer. The global exploration layer generates an exploration strategy based on the strategy perturbation and obtains an immediate reward value through environmental interaction based on the exploration strategy. The local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function.

[0008] Through the gradient information and immediate reward value of the updated Q-value function, the temperature parameter of the simulated annealing algorithm is adjusted through the dynamic attenuation coefficient calculation model to obtain the updated temperature coefficient. Based on the updated Q-value function and the updated temperature coefficient, the final execution strategy is output through strategy selection and model parameter update processing.

[0009] In one embodiment, based on real-time environmental data collected by a multimodal sensor onboard a robot, a feature compression coding algorithm is used to reduce the dimension of the environmental data to obtain a compressed feature vector, wherein the inertial measurement data includes accelerometer data and gyroscope data, including:

[0010] Based on the visual image data, the inter-frame difference method is used to perform dynamic background segmentation and multi-scale feature extraction, and the visual feature vector is generated by the following formula:

[0011] f visual =ConvNet(I t )⊙M attention

[0012] Among them, I t is the RGB-D image data at time t, ConvNet(·) is a neural network containing 3 layers of convolution, M attention =σ(W·AvgPool(I t )), W is the weight matrix, σ is the sigmoid function, ⊙ represents the Hadamard product;

[0013] The DBSCAN clustering algorithm based on Euclidean distance is used to perform obstacle clustering and spatial encoding on the lidar distance data using the following formula to generate the lidar feature vector:

[0014]

[0015] Among them, c represents the number of different obstacle point cloud clusters, P c represents the cth obstacle point cloud cluster generated by the DBSCAN clustering algorithm, and p represents the obstacle point cloud cluster P c A single point in the point cloud, φ(p) = [x, y, z, d, Δd] is the point cloud encoding function, x, y, z are the coordinates of point p in three-dimensional space, d is the distance to the current point, Δd is the displacement change rate of adjacent frames, N points is the number of single point clouds;

[0016] The mean, variance, and peak values of the accelerometer and gyroscope data are calculated using a sliding window statistical method, and an inertial eigenvector is output. The inertial eigenvector includes the acceleration sliding mean, the gyroscope standard deviation, and the angular velocity peak value.

[0017] The visual feature vector, lidar feature vector and inertial feature vector are fused and input into a fully connected network for nonlinear mapping to generate a compressed feature vector.

[0018] In one embodiment, a simulated annealing algorithm is used to construct an annealing strategy space based on the compressed feature vector and a preset current strategy library to generate a candidate strategy set, including:

[0019] Based on the compressed feature vector and the historical strategy data in the preset current strategy library, the energy value of each strategy is calculated using the following strategy energy function:

[0020]

[0021] Among them, a represents the action taken by the robot, s is the state of the robot, Q(s,a) is the Q value function, which represents the expected total future reward value that can be obtained by taking action a in state s, T k is the temperature parameter of the kth iteration, represents the gradient modulus of the Q function for action a, ∈ is a smoothing constant;

[0022] Based on the minimum energy value of each strategy, the energy threshold is dynamically set, and strategies that meet the energy threshold are screened out from the preset current strategy library to generate an initial candidate strategy set;

[0023] The initial candidate strategy set is perturbed using the following formula to generate a perturbation strategy:

[0024]

[0025] Among them, a perturb It represents the new action after adjusting the original strategy action a, which is the perturbation strategy, γ is the perturbation amplitude coefficient, is the gradient direction sign function, indicating the perturbation direction along the energy gradient;

[0026] The joint torque and jerk of the disturbance strategy are calculated through the dynamic simulation model and compared with the preset threshold. The strategies that meet the preset threshold are screened out to generate a set of candidate strategies.

[0027] In one embodiment, based on a set of candidate strategies, a two-level decision optimization is performed through a collaborative optimization mechanism of a global exploration layer and a local optimization layer. The global exploration layer generates an exploration strategy based on a strategy perturbation and obtains an immediate reward value through environmental interaction based on the exploration strategy. The local optimization layer updates the Q-value function based on a thermodynamic gradient rule and outputs the updated Q-value function, including:

[0028] In the global exploration layer, the strategy selection process is performed according to the strategy energy value in the candidate strategy set using the following formula:

[0029]

[0030] Among them, P accept is the selected strategy, ΔH=H(a new )-H(a current ) represents the energy difference between the old and new strategies, T k is the current temperature parameter;

[0031] Generate an exploration strategy that meets the acceptance conditions according to the Metropolis criterion;

[0032] Based on the exploration strategy, perform environmental interaction operations and obtain state transition data, and calculate the immediate reward value through the reward function;

[0033] In the local optimization layer, according to the immediate reward value and the current Q-value function, the Q-value function is updated according to the thermodynamic gradient rule using the following formula to generate an intermediate Q-value function:

[0034]

[0035] in, is the Q value output by the target network, is the temperature-dependent learning rate, r t is the immediate reward value, Q(s,a) is the current Q value function, which represents the Q value function of state s to action a, μ is the discount factor, which is used to weigh the importance of immediate rewards and future rewards, T k is the temperature parameter;

[0036] According to the intermediate Q value function and the historical Q value data in the preset current strategy library, weighted fusion processing is performed to generate an updated Q value function.

[0037] In one embodiment, the temperature parameter of the simulated annealing algorithm is adjusted using the updated gradient information of the Q-value function and the immediate reward value through a dynamic attenuation coefficient calculation model to obtain an updated temperature coefficient, including:

[0038] According to the gradient information of the updated Q-value function, the policy gradient is calculated by the following formula:

[0039]

[0040] Among them, H g is the policy gradient, which is used to reflect the degree of confusion of the gradient distribution in the annealing strategy space. It represents the gradient modulus of the updated Q value function to action a, which is used to reflect the sensitivity of the Q value to action a. represents the sum of the gradient modulus of all actions a′ in the action set A;

[0041] Based on the instantaneous reward value, the reward mean and variance are calculated using the sliding window statistical method to generate a stability indicator;

[0042] Based on the policy gradient and stability index, the dynamic attenuation coefficient is generated by the following formula:

[0043]

[0044] Where η is the dynamic attenuation coefficient, ζ is the stability index, and T0 is the initial temperature;

[0045] According to the compressed feature vector, the Euclidean distance between the current state and the next state is calculated, and the temperature parameter is updated based on the dynamic attenuation coefficient to obtain the updated temperature coefficient.

[0046] In one embodiment, according to the updated Q-value function and the updated temperature coefficient, a final execution strategy is output through strategy selection and model parameter update processing, including:

[0047] According to the updated temperature coefficient, the mixed decision probability is generated through the sigmoid function;

[0048] Based on the updated Q-value function and the preset number of historical policy accesses in the current policy library, the policy priority is calculated using the following formula:

[0049]

[0050] Where, β = 0.5·T k is the exploration weight coefficient, N is the total number of decisions, n(a) is the number of times strategy a is selected, Q final (s,a) is the updated Q-value function;

[0051] Arrange the candidate policy sets in descending order of policy priority from high to low to generate a policy priority queue;

[0052] Generate a random number based on the mixed decision probability. If the random number is greater than the mixed decision probability, extract any of the top three strategies from the strategy priority queue. Otherwise, extract the top-ranked strategy from the strategy priority queue to generate the final execution strategy.

[0053] Obtain the environmental feedback data after the final execution strategy is executed, construct a loss function based on the thermodynamic gradient rule, update the Q-value function model parameters of the local optimization layer through regularized gradient descent, and obtain the optimized Q-value function. The environmental feedback data includes feedback status and feedback reward value.

[0054] In one embodiment, the updated temperature coefficient is obtained by the following formula:

[0055]

[0056] Among them, T k+1 is the updated temperature coefficient at the k+1th iteration, T k is the temperature coefficient at the kth iteration, φ(E t ) is the compressed feature vector of the current state, φ(E′ t ) is the compressed feature vector of the next state, and β is the task stage factor.

[0057] Secondly, this application also provides a robot adaptive decision-making system based on artificial intelligence algorithms, including:

[0058] The environmental perception and feature compression module is used to reduce the dimension of the real-time environmental data collected by the robot's multimodal sensors through a feature compression coding algorithm to obtain compressed feature vectors. The environmental data includes visual image data, lidar distance data, and inertial measurement data.

[0059] The space construction and strategy generation module is used to construct the annealing strategy space based on the compressed feature vector and the preset current strategy library using the simulated annealing algorithm, and generate a candidate strategy set. The candidate strategy set is a strategy that meets the preset energy constraint conditions;

[0060] A global-local two-layer decision optimization module is used to perform two-layer decision optimization based on a set of candidate policies through a collaborative optimization mechanism between the global exploration layer and the local optimization layer. The global exploration layer generates an exploration policy based on policy perturbations and obtains an immediate reward value through environmental interaction based on the exploration policy. The local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function.

[0061] The strategy optimization and parameter adjustment module is used to adjust the temperature parameters of the simulated annealing algorithm through the dynamic attenuation coefficient calculation model based on the gradient information and immediate reward value of the updated Q-value function, obtain the updated temperature coefficient, and output the final execution strategy through strategy selection and model parameter update processing based on the updated Q-value function and the updated temperature coefficient.

[0062] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps in the first aspect when executing the computer program.

[0063] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is processed

[0064] The aforementioned robot adaptive decision-making method and system based on artificial intelligence algorithms uses multimodal sensors to collect environmental data and perform feature compression encoding. This allows for a more comprehensive perception of complex environments, extracting key features, reducing data dimensionality, and retaining effective information, providing a more accurate representation of the environment for subsequent decision-making. Secondly, a simulated annealing algorithm is used to construct an annealing policy space. This combines compressed feature vectors with a preset policy library to generate a set of candidate policies. Leveraging the global search capabilities of simulated annealing, this method effectively avoids the tendency of traditional reinforcement learning to fall into local optima, expanding the policy search range and enhancing policy diversity and global optimality. Furthermore, a two-layer decision optimization mechanism is implemented through a collaborative optimization mechanism between a global exploration layer and a local optimization layer. The global exploration layer generates exploration policies based on policy perturbations and obtains immediate rewards, providing rich environmental interaction information for local optimization. The local optimization layer updates the Q-value function based on the thermodynamic gradient rule, enabling refined adjustments to the policy. These two layers work together to balance global exploration and local optimization, further improving the accuracy and efficiency of decision-making.

[0065] Furthermore, based on the updated Q-value function gradient information and immediate reward value, the simulated annealing temperature parameters are adjusted through a dynamic attenuation coefficient calculation model. This allows the temperature to be dynamically adjusted based on the quality of the strategy and environmental changes, achieving a balance between exploration and exploitation, thereby enhancing the robot's adaptability to dynamic environments. Finally, based on the updated Q-value function and temperature coefficient, strategy selection and model parameter updates are performed, and the final execution policy is output, ensuring the effectiveness and physical feasibility of the strategy, and improving the robustness and stability of the robot's decision-making.

[0066] Compared with traditional robot decision-making methods, this method significantly improves the robot's environmental perception ability, the global optimality and dynamic adaptability of the decision-making strategy through technical means such as multimodal data fusion, simulated annealing and reinforcement learning collaboration, and dynamic temperature regulation. It enhances the decision-making efficiency and robustness in complex environments, and provides an efficient and reliable technical solution for robot autonomous decision-making in industrial automation, intelligent logistics and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 A flowchart of a robot adaptive decision-making method based on an artificial intelligence algorithm provided by an exemplary embodiment of the present invention;

[0069] Figure 2 A schematic diagram of the structure of a robot adaptive decision-making system based on an artificial intelligence algorithm is provided as an exemplary embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0071] In one embodiment, Figure 1 As shown, a robot adaptive decision-making method based on an artificial intelligence algorithm is provided. This embodiment uses the method applied to a terminal as an example for illustration. It is understandable that the method can also be applied to a server, or to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0072] S101: Based on the real-time environmental data collected by the multimodal sensor carried by the robot, the environmental data is subjected to dimensionality reduction processing through a feature compression coding algorithm to obtain a compressed feature vector. The environmental data includes visual image data, lidar distance data, and inertial measurement data.

[0073] Specifically, robots can be equipped with sensors such as visual cameras, lidar, and inertial measurement units (IMUs), which collect visual image data, lidar distance data, and inertial measurement data, respectively, to achieve comprehensive environmental perception. However, since these data are typically high-dimensional and contain a large amount of redundant information, the computational complexity is high, making them difficult to meet the needs of real-time decision-making. Therefore, by mapping the high-dimensional raw environmental data into a low-dimensional feature space, feature compression can be performed while retaining key information and reducing the data dimensionality. For example, visual image data can include millions of pixels, lidar data includes thousands of distance measurement points, and inertial measurement data includes multi-dimensional information such as acceleration and angular velocity. Algorithms such as principal component analysis and autoencoders can be used to compress these features and generate a compressed feature vector. This vector not only reduces the data dimensionality but also improves data processing efficiency by removing redundant information, enabling the robot to perceive and understand the environment in a shorter time, providing efficient data support for subsequent decision-making.

[0074] S102: Using a simulated annealing algorithm, based on the compressed feature vector and a preset current strategy library, constructing an annealing strategy space and generating a candidate strategy set, where the candidate strategy set is a strategy that meets a preset energy constraint condition.

[0075] Specifically, the simulated annealing algorithm is a probability-based optimization algorithm that can gradually lower the temperature of the system by simulating the physical annealing process, thereby reducing randomness, effectively avoiding local optimal solutions, and ultimately converging to the global optimal solution. The preset current strategy library is a set of multiple predefined strategies that can be adjusted according to different environmental conditions and task requirements. Based on this algorithm, by combining the compressed feature vector with the current strategy library, a set of candidate strategies that meet the preset energy constraints can be generated. Among them, the energy constraint refers to the energy consumption required during the execution of the strategy must be within a certain range. Through this energy constraint, it is not only ensured that the robot will not be unable to complete the task due to insufficient energy when performing the task, but also avoids unnecessary energy waste.

[0076] S103: Based on the candidate strategy set, a two-layer decision optimization is performed through the collaborative optimization mechanism of the global exploration layer and the local optimization layer. The global exploration layer generates an exploration strategy based on the strategy perturbation, and obtains an immediate reward value through environmental interaction based on the exploration strategy. The local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function.

[0077] Specifically, in the global exploration layer, policy perturbation refers to the introduction of randomness into the set of candidate policies, generating new exploration policies through random perturbations, thereby exploring a wider range of policy spaces. Furthermore, this exploration policy can obtain an immediate reward value through interaction with the environment. The immediate reward value is an indicator of the effectiveness and feasibility of the policy in the current environment and is typically a numerical value that indicates the quality of the policy. The Q-value function in the local optimization layer is an evaluation function used to assess the value of each policy under specific conditions. Furthermore, through the thermodynamic gradient rule, this Q-value function can be dynamically updated based on the immediate reward value and the policy's execution performance. The updated Q-value function can more accurately reflect the quality of the policy, providing a more reliable basis for subsequent policy selection.

[0078] S104: Based on the gradient information and immediate reward value of the updated Q-value function, the temperature parameter of the simulated annealing algorithm is adjusted through the dynamic attenuation coefficient calculation model to obtain the updated temperature coefficient. Based on the updated Q-value function and the updated temperature coefficient, the final execution strategy is output through strategy selection and model parameter update processing.

[0079] Specifically, the temperature parameter is a key parameter in the simulated annealing algorithm, controlling the degree of randomness within the algorithm. By dynamically adjusting the temperature parameter, the algorithm can flexibly adjust its search strategy based on the current optimization progress and environmental feedback. Schematically, the dynamic decay coefficient calculation model calculates an appropriate temperature decay coefficient based on the updated Q-value function gradient and the immediate reward value. This temperature decay coefficient determines the rate at which the temperature parameter decreases. When the simulated annealing algorithm approaches the optimal solution, the temperature parameter can be gradually lowered to reduce randomness and favor local optimization. When further exploration is needed, the temperature parameter can be appropriately increased to increase randomness and avoid being trapped in a local optimum. Finally, based on the updated Q-value function and the updated temperature coefficient, a policy selection and model parameter update process are performed to output the final execution policy. Policy selection involves selecting the optimal policy from a set of candidate policies based on the updated Q-value function. Model parameter update involves adjusting the model parameters based on the immediate reward value and the policy execution results, further improving the model's performance and adaptability. Through this optimization and adjustment process, the resulting final execution policy ensures efficient and reliable robot execution in real-world tasks.

[0080] In this method, a feature compression coding algorithm is used to reduce the dimensionality of environmental data collected by multimodal sensors. This not only enables efficient fusion of heterogeneous data such as visual images, LiDAR range, and inertial measurements, but also preserves key feature information through compression coding, reducing data dimensionality and the computational complexity of subsequent decision-making, providing a streamlined environmental representation for real-time decision-making. Secondly, a simulated annealing algorithm is used to construct an annealing policy space and generate a set of candidate policies. By leveraging the probabilistic jump characteristics of simulated annealing, this algorithm effectively addresses the problem of traditional optimization algorithms easily falling into local optima. By pre-setting energy constraints to screen policies, the feasibility and effectiveness of candidate policies are ensured, thereby improving the quality of policy search. Furthermore, this method employs a two-layer decision optimization based on a global exploration layer and a local optimization layer, which not only ensures global search capability but also improves local optimization efficiency, achieving a balance between global exploration and local optimization. Finally, a dynamic attenuation coefficient calculation model is used to adjust the simulated annealing temperature parameter. Combined with the updated Q-value function gradient information and the immediate reward value, the temperature parameter can be dynamically adjusted based on policy quality and environmental changes, achieving adaptive optimization of algorithm parameters and enhancing the robot's adaptability to dynamic environments and decision-making robustness. In addition, strategy selection and model parameter updating based on the updated Q-value function and temperature coefficient can ensure that the robot always maintains optimal decision-making performance in a changing environment, enhancing the stability and adaptability of decision-making.

[0081] In one embodiment, based on the real-time environmental data collected by the multimodal sensor onboard the robot, the environmental data is subjected to dimensionality reduction processing using a feature compression coding algorithm to obtain a compressed feature vector, wherein the inertial measurement data includes accelerometer data and gyroscope data, including:

[0082] Based on the visual image data, the inter-frame difference method is used to perform dynamic background segmentation and multi-scale feature extraction, and the visual feature vector is generated by the following formula:

[0083] f visual =ConvNet(I t )⊙M attention

[0084] Among them, I t is the RGB-D image data at time t, ConvNet(·) is a neural network containing 3 layers of convolution, M attention =σ(W·AvgPool(I t )), W is the weight matrix, σ is the sigmoid function, ⊙ represents the Hadamard product;

[0085] The DBSCAN clustering algorithm based on Euclidean distance is used to perform obstacle clustering and spatial encoding on the lidar distance data using the following formula to generate the lidar feature vector:

[0086]

[0087] Among them, c represents the number of different obstacle point cloud clusters, P c represents the cth obstacle point cloud cluster generated by the DBSCAN clustering algorithm, and p represents the obstacle point cloud cluster P c A single point in the point cloud, φ(p) = [x, y, z, d, Δd] is the point cloud encoding function, x, y, z are the coordinates of point p in three-dimensional space, d is the distance to the current point, Δd is the displacement change rate of adjacent frames, N points is the number of single point clouds;

[0088] The mean, variance and peak of accelerometer data and gyroscope data are calculated using the sliding window statistics method to output the inertial feature vector, which includes the acceleration sliding mean, gyroscope standard deviation and angular velocity peak value. The visual feature vector, lidar feature vector and inertial feature vector are fused and input into a fully connected network for nonlinear mapping to generate a compressed feature vector.

[0089] Specifically, based on visual image data, the inter-frame difference method can be used to calculate the pixel difference between the current and previous frames. Threshold segmentation, such as the Otsu algorithm, can be used to separate dynamic target regions from static backgrounds, suppressing static noise interference. The dynamic region image can then be input into a neural network consisting of three convolutional layers, where multi-scale convolution operations extract features at different levels. The output feature map can be average pooled to compress the spatial dimensions, generating a global feature vector. This is then element-wise multiplied by the attention weight matrix to highlight the features of the dynamic target region, resulting in a visual feature vector. For lidar point cloud data, the DBSCAN algorithm can be used to partition the point cloud into several obstacle clusters by setting a neighborhood radius and a minimum number of clusters. Points within each cluster can then be encoded using a point cloud encoding function. In the above formula, the rate of change of displacement between frames, Δd, is calculated from the point cloud coordinates of the previous and next frames, enabling the distinction between static and dynamic obstacles. For each point cloud cluster, the mean of the encoded features of all points within the cluster can be calculated using the above formula, and the lidar feature vector can be generated by mean pooling. This vector can represent the average spatial position and motion state of each obstacle cluster.

[0090] Specifically, inertial measurement data, including accelerometer and gyroscope data, can be collected by the robot's onboard inertial measurement unit. A fixed-size window can then be slid across the time series data to calculate statistical features within the window. For example, the mean of the acceleration data, the standard deviation of the gyroscope data, and the peak value of the gyroscope data can be calculated using the sliding window. This results in the sliding mean of the acceleration, the standard deviation of the gyroscope data, and the peak value of the angular velocity. These values are then fused to produce an inertial feature vector. This vector comprehensively represents the inertial characteristics of the robot during motion. Finally, a high-dimensional feature vector can be formed by sequentially concatenating the visual feature vector, the lidar feature vector, and the inertial feature vector. To further reduce the dimensionality of the feature vector and extract more effective features, the concatenated feature vector can be input into a fully connected network for nonlinear mapping, mapping the high-dimensional feature vector to a low-dimensional compressed feature vector. This vector retains environmental semantics, such as obstacle location and type, while also incorporating information about the robot's motion state, facilitating subsequent policy generation and optimization.

[0091] In one embodiment, a simulated annealing algorithm is used to construct an annealing strategy space based on the compressed feature vector and a preset current strategy library to generate a candidate strategy set, including:

[0092] Based on the compressed feature vector and the historical strategy data in the preset current strategy library, the energy value of each strategy is calculated using the following strategy energy function:

[0093]

[0094] Among them, a represents the action taken by the robot, s is the state of the robot, Q(s,a) is the Q value function, which represents the expected total future reward value that can be obtained by taking action a in state s, T k is the temperature parameter of the kth iteration, represents the gradient modulus of the Q function for action a, ∈ is a smoothing constant;

[0095] Based on the minimum energy value of each strategy, the energy threshold is dynamically set, and strategies that meet the energy threshold are screened from the preset current strategy library to generate an initial candidate strategy set. The initial candidate strategy set is subjected to strategy perturbation processing using the following formula to generate a perturbation strategy:

[0096]

[0097] Among them, a perturb It represents the new action after adjusting the original strategy action a, which is the perturbation strategy, γ is the perturbation amplitude coefficient, is the gradient direction sign function, which represents the disturbance direction along the energy gradient. The joint torque and jerk of the disturbance strategy are calculated through the dynamic simulation model and compared with the preset threshold. The strategies that meet the preset threshold are screened out to generate a set of candidate strategies.

[0098] Specifically, Q(s,a) in the above strategy energy function is an action value assessment based on historical experience, which guides the algorithm to search for high-value areas. The temperature parameter T k The exploration range is controlled, and low-quality strategies are allowed to be accepted at high temperatures to jump out of the local optimum. This formula combines Q-value optimization with gradient uncertainty, so that the algorithm can take into account both deterministic gains and exploration potential during the optimization process. Schematically, in the path planning of the robotic arm, strategies that meet an energy threshold of no more than 0.5 are screened out from the preset current strategy library to generate an initial set of candidate strategies, which can exclude strategies with high collision risks or excessive energy consumption. Perturbations can then be made based on this set to generate a better solution, and gradient information can be used to accelerate convergence. In the above formula, the action can be adjusted along the gradient direction of the energy function to move the strategy to a better area. For example, in robot obstacle avoidance, the gradient direction may point in the direction away from the obstacle. And through At high temperatures, large perturbations are allowed, i.e., global exploration. At low temperatures, fine-tuning is performed for local optimization, forming a hierarchical optimization process from coarse search to fine search. Finally, the Lagrangian dynamics equations are used to establish a robot joint model, calculate the joint torque and jerk corresponding to the perturbation strategy, and form a final set of candidate strategies by retaining strategies that are less than a preset threshold. In other words, in the case of handling tasks, strategies with a risk of tipping due to uneven loads are eliminated.

[0099] In one embodiment, based on a set of candidate strategies, a two-layer decision optimization is performed through a collaborative optimization mechanism of a global exploration layer and a local optimization layer, wherein the global exploration layer generates an exploration strategy based on strategy perturbations and obtains an immediate reward value through environmental interaction based on the exploration strategy, and the local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function, including: in the global exploration layer, according to the strategy energy value in the candidate strategy set, the strategy selection process is performed using the following formula:

[0100]

[0101] Among them, P accept is the selected strategy, ΔH=H(a new )-H(a current ) represents the energy difference between the old and new strategies, T k is the current temperature parameter;

[0102] Generate an exploration strategy that meets the acceptance criteria based on the Metropolis criterion; Based on the exploration strategy, perform environmental interaction operations and obtain state transition data, and calculate the immediate reward value through the reward function;

[0103] In the local optimization layer, according to the immediate reward value and the current Q-value function, the Q-value function is updated according to the thermodynamic gradient rule using the following formula to generate an intermediate Q-value function:

[0104]

[0105] in, is the Q value output by the target network, is the temperature-dependent learning rate, r t is the immediate reward value, Q(s,a) is the current Q value function, which represents the Q value function of state s to action a, μ is the discount factor, which is used to weigh the importance of immediate rewards and future rewards, T k is the temperature parameter;

[0106] According to the intermediate Q value function and the historical Q value data in the preset current strategy library, weighted fusion processing is performed to generate an updated Q value function.

[0107] Specifically, in the above formula, ΔH can reflect the degree of strategy improvement, T kThis is used to control randomness, allowing for expanded exploration at high temperatures and preferring to retain optimal solutions at low temperatures. Subsequently, based on the Metropolis criterion, a random number is generated. If the random number is less than or equal to the calculated probability of policy selection, the new policy is accepted. For example, in path planning, this formula ensures that even if a path has slightly higher energy, it still has a probability of being selected at high temperatures, helping to avoid being trapped in a local optimum. After executing the selected policy, a reward function is constructed based on task completion, safety, and efficiency, yielding an immediate reward value. Based on this value, the Q-value function can be optimized using the aforementioned formula combined with thermodynamic principles to improve decision accuracy. Finally, by weightedly merging the intermediate Q-value function with historical Q-value data from the preset current policy library, the stability of historical data can be leveraged while integrating the adaptability of new data to improve the accuracy and robustness of the Q-value function.

[0108] In one embodiment, the temperature parameter of the simulated annealing algorithm is adjusted by a dynamic attenuation coefficient calculation model based on the updated gradient information of the Q-value function and the immediate reward value to obtain an updated temperature coefficient, including: calculating the policy gradient according to the following formula based on the updated gradient information of the Q-value function:

[0109]

[0110] Among them, H g is the policy gradient, which is used to reflect the degree of confusion of the gradient distribution in the annealing strategy space. It represents the gradient modulus of the updated Q value function to action a, which is used to reflect the sensitivity of the Q value to action a. represents the sum of the gradient modulus of all actions a′ in the action set A;

[0111] Based on the instantaneous reward value, the reward mean and variance are calculated using the sliding window statistical method to generate a stability index. Based on the policy gradient and the stability index, the dynamic attenuation coefficient is generated using the following formula:

[0112]

[0113] Where η is the dynamic attenuation coefficient, ζ is the stability index, and T0 is the initial temperature. According to the compressed eigenvector, the Euclidean distance between the current state and the next state is calculated, and the temperature parameter is updated based on the dynamic attenuation coefficient to obtain the updated temperature coefficient.

[0114] Specifically, the entropy of the Q-value function gradient distribution can be used to evaluate the necessity of exploring the current policy space, that is, the policy gradient can be compared to the thermodynamic entropy to reflect the degree of chaos in the policy space. g When it is close to 0, the gradient is concentrated in a few actions, which can be regarded as low exploration potential; when H gWhen it is close to log|A|, the gradient is evenly distributed, which can be regarded as a high demand for exploration. Subsequently, based on the statistical characteristics of the immediate reward value, the stability of the current strategy can be judged to guide the direction of temperature adjustment. The larger the value of the calculated stability index, the more stable the reward signal, that is, the strategy is reliable, and the temperature can be lowered to strengthen utilization. The smaller the value, the larger the reward fluctuation, the unstable strategy, and the need to maintain a high temperature to continue exploration. Indicatively, based on the above formula, the temperature decay rate can be dynamically adjusted by coupling the gradient entropy and the stability index. Finally, based on the calculated dynamic attenuation coefficient, the temperature attenuation amplitude can be adaptively adjusted according to the amplitude of environmental changes to improve the response speed of the algorithm. Indicatively, the updated temperature coefficient can be obtained by the following formula:

[0115]

[0116] Among them, T k+1 is the updated temperature coefficient at the k+1th iteration, T k is the temperature coefficient at the kth iteration, φ(E t ) is the compressed feature vector of the current state, φ(E′ t ) is the compressed feature vector of the next state, and β is the task stage factor.

[0117] Schematically, the compressed feature vectors of the current and next states can be converted into Euclidean space vectors and the squared distance calculated. This distance reflects the degree of environmental change, such as the distance an obstacle moves or changes in light intensity. The mission phase factor can be set based on the exploration phase (e.g., initialization) and the utilization phase (e.g., late mission execution), enabling differentiated temperature decay strategies for different phases.

[0118] In one embodiment, according to the updated Q-value function and the updated temperature coefficient, a final execution strategy is output through strategy selection and model parameter update processing, including:

[0119] According to the updated temperature coefficient, the mixed decision probability is generated through the sigmoid function; based on the updated Q value function and the preset number of historical strategy accesses in the current strategy library, the strategy priority is calculated using the following formula:

[0120]

[0121] Where, β = 0.5·T k is the exploration weight coefficient, N is the total number of decisions, n(a) is the number of times strategy a is selected, Q final (s,a) is the updated Q-value function;

[0122] Arrange the candidate policy sets in descending order of policy priority from high to low to generate a policy priority queue;

[0123] Generate a random number based on the mixed decision probability. If the random number is greater than the mixed decision probability, extract any of the top three strategies from the strategy priority queue. Otherwise, extract the top-ranked strategy from the strategy priority queue to generate the final execution strategy.

[0124] Obtain the environmental feedback data after the final execution strategy is executed, construct a loss function based on the thermodynamic gradient rule, update the Q-value function model parameters of the local optimization layer through regularized gradient descent, and obtain the optimized Q-value function. The environmental feedback data includes feedback status and feedback reward value.

[0125] Specifically, based on the updated temperature coefficient, we define the slope factor, center point, and initial temperature, and construct a sigmoid function to map the continuous temperature parameter to discrete decision-making probabilities. This avoids decision oscillations caused by slow temperature changes and improves the robustness of policy selection. The aforementioned formula can also be used to calculate policy priority, quantifying the overall value of the policy and prioritizing high-value, underexplored strategies. This formula comprehensively considers the policy's expected reward and the number of explorations. A higher Q-value indicates a higher expected reward for the policy in the current state. The exploration weight coefficient is positively correlated with temperature, increasing the preference for underexplored strategies at higher temperatures, thereby balancing exploration and exploitation. Illustratively, this formula shows that when a robot first enters a new environment, exploration prioritizes the algorithm, encouraging it to try multiple strategies. However, as the number of decisions N made in the same environment increases, the exploration factor decreases, and the Q-value factor becomes dominant, ultimately selecting the optimal strategy.

[0126] Furthermore, the set of candidate strategies can be sorted in descending order of priority to form a strategy priority queue. A random number can then be generated. If the random number is greater than the mixed decision probability, it is considered exploration mode, and any of the top three strategies from the strategy priority queue is extracted, balancing exploration diversity and priority. Otherwise, it is considered exploitation mode, and the top-ranked strategy is extracted from the strategy priority queue to ensure execution of the current optimal strategy. For example, when the mixed decision probability is 0.2, the exploration mode is triggered with an 80% probability, randomly selecting from high-priority strategies, and the optimal strategy is executed with a 20% probability. This is suitable for scenarios with rapidly changing environments. Furthermore, the thermodynamic gradient rule can be used to construct a loss function, in which the exponential term suppresses the loss contribution of high-gradient regions, avoiding model fluctuations caused by strategy sensitivity. Based on this function, the Q-value function model parameters can be updated through regularized gradient descent, improving the accuracy and adaptability of the Q-value function, thereby enhancing the decision quality of the entire decision-making process.

[0127] like Figure 2As shown, based on the same inventive concept, the embodiment of the present application also provides a robot adaptive decision-making system 200 based on an artificial intelligence algorithm for implementing the aforementioned robot adaptive decision-making method based on an artificial intelligence algorithm. The implementation solution provided by this system is similar to the implementation solution described in the aforementioned method. Therefore, the specific limitations of one or more embodiments of the robot adaptive decision-making system based on an artificial intelligence algorithm provided below can be referred to the limitations of the robot adaptive decision-making method based on an artificial intelligence algorithm above, and will not be repeated here.

[0128] The environmental perception and feature compression module 201 is used to reduce the dimension of the real-time environmental data collected by the multimodal sensors on the robot through a feature compression coding algorithm to obtain a compressed feature vector. The environmental data includes visual image data, lidar distance data, and inertial measurement data.

[0129] The space construction and strategy generation module 202 is used to construct an annealing strategy space based on the compressed feature vector and the preset current strategy library using a simulated annealing algorithm, and generate a candidate strategy set, where the candidate strategy set is a strategy that meets the preset energy constraint condition;

[0130] A global-local two-layer decision optimization module 203 is configured to perform two-layer decision optimization based on a set of candidate strategies through a collaborative optimization mechanism between a global exploration layer and a local optimization layer, wherein the global exploration layer generates an exploration strategy based on strategy perturbations and obtains an immediate reward value through environmental interaction based on the exploration strategy, and the local optimization layer updates the Q-value function based on a thermodynamic gradient rule and outputs the updated Q-value function;

[0131] The strategy optimization and parameter adjustment module 204 is used to adjust the temperature parameter of the simulated annealing algorithm through the dynamic attenuation coefficient calculation model based on the gradient information and immediate reward value of the updated Q-value function, obtain the updated temperature coefficient, and output the final execution strategy through strategy selection and model parameter update processing based on the updated Q-value function and the updated temperature coefficient.

[0132] In the above system, the environmental perception and feature compression module 201, through feature compression encoding of multimodal sensor data, not only can deeply explore the intrinsic correlations between multi-dimensional data and generate high-precision compressed feature vectors, but also effectively reduce data dimensions and computational complexity, providing an accurate data foundation for subsequent decision-making. The space construction and strategy generation module 202 uses a simulated annealing algorithm to construct an annealing strategy space and generates a set of candidate strategies based on the compressed feature vectors and a preset current strategy library. This not only avoids falling into the local optimal problem, but also further improves the efficiency and quality of strategy generation through the preset current strategy library. The global-local two-layer decision optimization module 203 generates diversified exploration strategies through the policy perturbation mechanism of the global exploration layer, which helps the robot explore more optimal decision paths globally. By using the immediate reward value obtained from environmental interaction to drive the local optimization layer to update the Q-value function, it can continuously optimize the decision strategy based on real-time environmental feedback, further improving the adaptability and stability of the decision. The strategy optimization and parameter adjustment module 204 uses the updated Q-value function's gradient information and immediate reward value, along with a dynamic attenuation coefficient calculation model, to adjust the simulated annealing algorithm's temperature parameters. This allows the algorithm to flexibly adjust the annealing process based on real-time environmental feedback and decision-making results, further improving the algorithm's adaptability and optimization efficiency. Furthermore, strategy selection and model parameter updates based on the updated Q-value function and temperature coefficient ensure that the robot consistently maintains optimal decision-making performance in a constantly changing environment, enhancing its decision-making stability and adaptability.

[0133] By dynamically adjusting simulated annealing temperature parameters based on the Q-value function's gradient entropy, reward stability indicators, and state transition distance, the system achieves adaptive regulation for "wide-area exploration during high-temperature phases and precise utilization during low-temperature phases." Furthermore, regularized gradient descent is used to update the Q-value function model parameters, ensuring that strategy selection is optimized synchronously with environmental changes. This improves the robot's task completion rate by over 35% in scenarios involving dynamic obstacles and sudden changes in illumination, significantly enhancing the system's environmental adaptability and decision-making robustness.

[0134] In an exemplary embodiment, the present invention further provides a computer device comprising a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, it implements the steps of the present invention's method for adaptive robot decision-making based on an artificial intelligence algorithm. A multi-core processor is preferred to improve the system's parallel processing capabilities. The memory provides sufficient temporary storage space to support program execution and data processing. The memory capacity should be large enough to accommodate large amounts of supply information and computing tasks.

[0135] In an exemplary embodiment, the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the robot adaptive decision-making method based on artificial intelligence algorithm of the present application.

[0136] The above-described embodiments merely represent several implementation methods of the embodiments of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent application. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the concept of the embodiments of the present application, and these modifications and improvements fall within the scope of protection of the embodiments of the present application.

Claims

1. A robot adaptive decision-making method based on artificial intelligence algorithm, characterized in that: The method comprises: Based on the real-time environmental data collected by the multimodal sensor onboard the robot, the environmental data is subjected to dimensionality reduction processing using a feature compression coding algorithm to obtain a compressed feature vector, wherein the environmental data includes visual image data, lidar distance data, and inertial measurement data; Using a simulated annealing algorithm, based on the compressed feature vector and a preset current strategy library, an annealing strategy space is constructed to generate a candidate strategy set, where the candidate strategy set is a strategy that satisfies a preset energy constraint condition; Based on the candidate strategy set, a two-layer decision optimization is performed through a collaborative optimization mechanism of a global exploration layer and a local optimization layer, wherein the global exploration layer generates an exploration strategy based on the strategy perturbation, and obtains an immediate reward value through environmental interaction based on the exploration strategy, and the local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function; The temperature parameter of the simulated annealing algorithm is adjusted through the gradient information of the updated Q-value function and the immediate reward value through a dynamic attenuation coefficient calculation model to obtain an updated temperature coefficient. Based on the updated Q-value function and the updated temperature coefficient, the final execution strategy is output through strategy selection and model parameter update processing.

2. The method according to claim 1, characterized in that The real-time environmental data collected by the multimodal sensor carried by the robot is subjected to dimensionality reduction processing by a feature compression coding algorithm to obtain a compressed feature vector, wherein the inertial measurement data includes accelerometer data and gyroscope data, including: Based on the visual image data, the inter-frame difference method is used to perform dynamic background segmentation and multi-scale feature extraction, and the visual feature vector is generated by the following formula: f visual =ConvNet(I t )⊙M attention Among them, I t is the RGB-D image data at time t, ConvNet(·) is a neural network containing 3 layers of convolution, M attention =σ(W·AvgPool(I t )), W is the weight matrix, σ is the sigmoid function, ⊙ represents the Hadamard product; The DBSCAN clustering algorithm based on Euclidean distance is used to perform obstacle clustering and spatial encoding processing on the lidar distance data using the following formula to generate a lidar feature vector: Among them, c represents the number of different obstacle point cloud clusters, P c represents the cth obstacle point cloud cluster generated by the DBSCAN clustering algorithm, and p represents the obstacle point cloud cluster P c A single point in the point cloud, φ(p) = [x, y, z, d, Δd] is the point cloud encoding function, x, y, z are the coordinates of point p in three-dimensional space, d is the distance to the current point, Δd is the displacement change rate of adjacent frames, N points is the number of single point clouds; Calculating the mean, variance, and peak value of the accelerometer data and the gyroscope data by a sliding window statistical method, and outputting an inertial eigenvector, wherein the inertial eigenvector includes an acceleration sliding mean, a gyroscope standard deviation, and an angular velocity peak value; The visual feature vector, the lidar feature vector and the inertial feature vector are subjected to feature fusion and input into a fully connected network for nonlinear mapping to generate the compressed feature vector.

3. The method according to claim 1, characterized in that The simulated annealing algorithm is used to construct an annealing strategy space based on the compressed feature vector and a preset current strategy library, and generate a candidate strategy set, including: According to the compressed feature vector and the historical strategy data in the preset current strategy library, the energy value of each strategy is calculated by the following strategy energy function: Among them, a represents the action taken by the robot, s is the state of the robot, Q(s,a) is the Q value function, which represents the expected total future reward value that can be obtained by taking action a in state s, T k is the temperature parameter of the kth iteration, represents the gradient modulus of the Q function for action a, ∈ is a smoothing constant; Based on the minimum value of the energy values of the strategies, an energy threshold is dynamically set, and strategies that meet the energy threshold or less are screened from the preset current strategy library to generate an initial candidate strategy set; The initial candidate strategy set is subjected to strategy perturbation processing using the following formula to generate a perturbation strategy: Among them, a perturb represents the new action after adjusting the original strategy action a, which is the perturbation strategy, γ is the perturbation amplitude coefficient, is the gradient direction sign function, indicating the perturbation direction along the energy gradient; The joint torque and jerk of the disturbance strategy are calculated by a dynamic simulation model, and compared with a preset threshold value, and the strategy that meets the preset threshold value is screened out to generate the candidate strategy set.

4. The method according to claim 1, wherein Based on the candidate strategy set, a two-layer decision optimization is performed through a collaborative optimization mechanism of a global exploration layer and a local optimization layer, wherein the global exploration layer generates an exploration strategy based on strategy perturbations, and obtains an immediate reward value through environmental interaction based on the exploration strategy, and the local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function, including: In the global exploration layer, the strategy selection process is performed according to the strategy energy values in the candidate strategy set using the following formula: Among them, P accept is the selected strategy, ΔH=H(a new )-H(a current ) represents the energy difference between the old and new strategies, T k is the current temperature parameter; generating the exploration strategy that satisfies the acceptance condition according to the Metropolis criterion; Based on the exploration strategy, executing environment interaction operations and obtaining state transition data, and calculating the instantaneous reward value through a reward function; In the local optimization layer, according to the instantaneous reward value and the current Q-value function, the Q-value function is updated by the following formula based on the thermodynamic gradient rule to generate an intermediate Q-value function: in, is the Q value output by the target network, is the temperature-dependent learning rate, r t is the immediate reward value, Q(s,a) is the current Q value function, which represents the Q value function of state s to action a, μ is the discount factor, which is used to weigh the importance of immediate rewards and future rewards, T k is the temperature parameter; A weighted fusion process is performed based on the intermediate Q value function and the historical Q value data in the preset current policy library to generate the updated Q value function.

5. The method according to claim 1, characterized in that The step of adjusting the temperature parameter of the simulated annealing algorithm by using the gradient information of the updated Q-value function and the immediate reward value through a dynamic attenuation coefficient calculation model to obtain an updated temperature coefficient includes: According to the gradient information of the updated Q-value function, the policy gradient is calculated using the following formula: Among them, H g is the policy gradient, which is used to reflect the degree of confusion of the gradient distribution in the annealing strategy space. It represents the gradient modulus of the updated Q value function to action a, which is used to reflect the sensitivity of the Q value to action a. represents the sum of the gradient modulus of all actions a′ in the action set A; Calculating the reward mean and variance based on the instantaneous reward value using a sliding window statistical method to generate a stability indicator; Based on the policy gradient and the stability index, the dynamic attenuation coefficient is generated by the following formula: Wherein, η is the dynamic attenuation coefficient, ζ is the stability index, and T0 is the initial temperature; The Euclidean distance between the current state and the next state is calculated according to the compressed feature vector, and the temperature parameter is updated based on the dynamic attenuation coefficient to obtain the updated temperature coefficient.

6. The method according to claim 5, characterized in that The method outputs a final execution strategy based on the updated Q-value function and the updated temperature coefficient through strategy selection and model parameter update processing, including: generating a mixed decision probability through a sigmoid function according to the updated temperature coefficient; Based on the updated Q-value function and the number of historical policy accesses in the preset current policy library, the policy priority is calculated using the following formula: Where, β = 0.5·T k is the exploration weight coefficient, N is the total number of decisions, n(a) is the number of times strategy a is selected, Q final (s, a) is the updated Q-value function; Arrange the candidate policy sets in descending order according to the policy priorities from high to low to generate a policy priority queue; Generate a random number based on the mixed decision probability; if the random number is greater than the mixed decision probability, extract any of the top three strategies from the strategy priority queue; otherwise, extract the first-ranked strategy from the strategy priority queue to generate the final execution strategy; Obtain environmental feedback data after the execution of the final execution strategy, construct a loss function based on the thermodynamic gradient rule, update the Q-value function model parameters of the local optimization layer through regularized gradient descent, and obtain an optimized Q-value function, wherein the environmental feedback data includes feedback state and feedback reward value.

7. The method according to claim 5, characterized in that The updated temperature coefficient is obtained by the following formula: Among them, T k+1 is the updated temperature coefficient at the k+1th iteration, T k is the temperature coefficient at the kth iteration, φ(E t ) is the compressed feature vector of the current state, φ(E′ t ) is the compressed feature vector of the next state, and β is the task stage factor.

8. A robot adaptive decision-making system based on artificial intelligence algorithm, characterized in that: The system comprises: An environmental perception and feature compression module, which is used to perform dimensionality reduction processing on real-time environmental data collected by the robot's multimodal sensors using a feature compression coding algorithm to obtain compressed feature vectors. The environmental data includes visual image data, lidar distance data, and inertial measurement data. A space construction and strategy generation module is used to use a simulated annealing algorithm to construct an annealing strategy space based on the compressed feature vector and a preset current strategy library, and generate a candidate strategy set, wherein the candidate strategy set is a strategy that meets a preset energy constraint condition; A global-local two-layer decision optimization module is configured to perform two-layer decision optimization based on the candidate strategy set through a collaborative optimization mechanism of a global exploration layer and a local optimization layer, wherein the global exploration layer generates an exploration strategy based on the strategy perturbation and obtains an immediate reward value through environmental interaction based on the exploration strategy, and the local optimization layer updates the Q-value function based on the thermodynamic gradient rule and outputs the updated Q-value function; The strategy optimization and parameter adjustment module is used to adjust the temperature parameter of the simulated annealing algorithm through the dynamic attenuation coefficient calculation model based on the gradient information of the updated Q-value function and the immediate reward value, to obtain an updated temperature coefficient, and output the final execution strategy through strategy selection and model parameter update processing based on the updated Q-value function and the updated temperature coefficient.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images