Decision-making method and system for mobile devices based on knowledge acceleration

Abstract

The present application provides a decision-making method and system for a mobile device based on knowledge acceleration. In the initial stage of the mobile device's movement process, the mobile device mainly relies on the prior knowledge of experts controlling the mobile device to make action decisions, and the knowledge-oriented first action instructions occupy a dominant position, ensuring the basic operational safety and efficiency of the mobile device. As the reinforcement learning algorithm gradually accumulates experience through interaction with the environment, and continuously adjusts and optimizes the decision-making strategy, the weight of the second action instruction output by the reinforcement learning algorithm gradually increases, and the weight of the first action instruction that depends on prior knowledge decreases accordingly, until the decision-making process of the mobile device is completely driven autonomously by the reinforcement learning algorithm. Through this strategy of dynamically integrating prior knowledge and real-time learning, the present application not only significantly improves the autonomous decision-making ability of mobile devices in complex environments, but also effectively shortens the learning cycle and reduces the operational risks caused by trial and error.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a decision-making method and system for action devices based on knowledge acceleration. Background Art

[0002] In the field of artificial intelligence, especially in the research of autonomous decision-making of mobile devices, traditional decision-making methods mainly rely on a series of preset instructions or behavioral patterns based on simple reactions. Although these strategies ensure the basic reliability of mobile device operation to a certain extent, they often perform poorly when dealing with complex and dynamically changing environments. This is mainly because these traditional methods lack the necessary flexibility and adaptability, making it difficult for mobile devices to make effective autonomous responses when encountering unexpected challenges, such as sudden changes in weather conditions, unmapped terrain obstacles, or other unpredictable interference factors. In addition, with the advancement of mobile device technology, the acquisition and processing of real-time data has become particularly critical. Mobile devices must be able to analyze a large amount of data collected from sensors in real time and make accurate judgments quickly based on this. However, traditional decision-making frameworks are usually unable to cope with the processing and analysis of large amounts of real-time data, limiting the execution efficiency and accuracy of mobile devices in complex task environments. Therefore, there is an urgent need to develop a more advanced, flexible and efficient mobile device autonomous decision-making technology to meet these challenges.

[0003] As an advanced technology that enables machines to self-learn and optimize decision-making strategies through continuous trial and error, reinforcement learning has shown great application potential in the field of autonomous decision-making of mobile devices. This technology allows mobile devices to gradually adjust their behavior patterns through real-time interaction with the surrounding environment to more effectively achieve the set goals. This feedback-based learning mechanism enables mobile devices to adapt to various complex scenarios and improve the accuracy and efficiency of task execution. However, reinforcement learning methods require learning optimization from a large number of trial and error samples, which is unrealistic in actual mobile device work. In addition, although reinforcement learning enables mobile devices to adapt and optimize their behavior in a constantly changing environment, the learning process is often time-consuming and inefficient. This is because reinforcement learning models usually learn from scratch without effectively utilizing existing knowledge or experience, resulting in a slow learning process and difficulty in quickly adapting to new tasks or environments. Summary of the invention

[0004] To at least partially solve the above technical problems, an embodiment of the present application provides a decision-making method for a mobile device based on knowledge acceleration, including:

[0005] Step 1: obtaining a task target for the mobile device to move, and using the task target to match fuzzy rules from a knowledge base, wherein the fuzzy rules are generated based on prior knowledge of expert control of the mobile device;

[0006] Step 2: Use the matched fuzzy rules to split the travel task of the mobile device into multiple subtasks;

[0007] Step 3: Use the prior knowledge of the expert control action device to build a trainable sub-knowledge control model for each sub-task;

[0008] Step 4, obtaining the state information at time t during the movement of the mobile device simulated on the simulation platform, wherein the state information includes the environmental characteristics and the posture characteristics of the mobile device, and executing step 5a and step 5b respectively;

[0009] Step 5a: input the state information into the sub-knowledge control model corresponding to the current sub-task to obtain a first action instruction;

[0010] Step 5b, inputting the state information into a reinforcement learning algorithm model, using the fully connected layer of the reinforcement learning algorithm model to fuse the environmental features in the state information and the posture features of the action device to generate hidden variable features, and inputting the hidden variable features into the intelligent network of the reinforcement learning algorithm model to obtain a second action instruction;

[0011] Step 6: Perform weighted summation on the first action instruction and the second action instruction using the set weight to obtain an executable action instruction;

[0012] Step 7: execute the executable action instruction in the simulation platform and return to step 4.

[0013] The embodiment of the present application also provides a decision-making system for mobile devices based on knowledge acceleration, including:

[0014] The mobile device control module is used to obtain the simulated mobile device movement process on the simulation platform Status information at a given moment, including environmental characteristics and posture characteristics of the mobile device;

[0015] A knowledge control module, used for inputting the state information into a sub-knowledge control model corresponding to the current sub-task to obtain a first action instruction;

[0016] A reinforcement learning control module is used to input the state information into a reinforcement learning algorithm model; use the fully connected layer of the reinforcement learning algorithm to fuse the environmental features in the state information and the posture features of the action device to generate hidden variable features; input the hidden variable features into the intelligent network of the reinforcement learning algorithm model to obtain a second action instruction;

[0017] The mobile device control module is further used to perform weighted summation on the first action instruction and the second action instruction using a set weight to obtain an executable action instruction; and execute the executable action instruction in the simulation platform;

[0018] The knowledge control module is also used to obtain the task objectives of the mobile device and use the task objectives to match fuzzy rules from the knowledge base, where the fuzzy rules are generated based on the prior knowledge of expert control of the mobile device; use the matched fuzzy rules to split the mobile device's travel task into multiple subtasks; and use the prior knowledge of expert control of the mobile device to construct a trainable sub-knowledge control model for each subtask.

[0019] The embodiment of the present application introduces an innovative reinforcement learning method and system based on knowledge acceleration, which jointly guides the decision-making process of mobile devices by integrating the prior knowledge of domain experts and data-driven learning strategies. In the early stage of learning, mobile devices mainly rely on a preset knowledge base to make decisions. At this stage, knowledge-oriented action selection occupies a dominant position, ensuring the basic operation safety and efficiency of mobile devices. As time goes by, the reinforcement learning algorithm gradually accumulates experience through interaction with the environment, and continuously adjusts and optimizes the decision-making strategy. In this process, the action weight output by the reinforcement learning algorithm gradually increases, while the action weight that depends on prior knowledge decreases accordingly, until the decision-making process of the mobile device is completely driven autonomously by the reinforcement learning algorithm. Through this strategy of dynamically integrating prior knowledge and real-time learning, the present invention not only significantly improves the autonomous decision-making ability of mobile devices in complex environments, but also effectively shortens the learning cycle and reduces the operational risks caused by trial and error. This method provides an efficient and safe solution for the intelligent control and autonomous operation of mobile devices, broadening its application potential in various complex application scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A system information flow chart provided for an embodiment of the present application;

[0021] Figure 2 A method flow chart provided for one embodiment of the present application;

[0022] Figure 3 A schematic diagram of the system structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] In view of the problems of low learning efficiency, high risk and limited adaptability of existing reinforcement learning methods when dealing with autonomous decision-making tasks in complex and dynamic environments, the embodiments of the present application propose a reinforcement learning method and system based on knowledge acceleration. This technology constructs an efficient autonomous decision-making system framework by integrating the prior knowledge of domain experts with a data-driven learning mechanism. The core innovation of this system is that it introduces the experience and intuition of human experts into the reinforcement learning process, and combines it with the big data processing capabilities of the machine to achieve improved decision-making accuracy and accelerated learning efficiency.

[0025] The technical solution provided in the embodiment of the present application is applicable to the action decision-making of unmanned action devices such as drones. By integrating the prior knowledge of experts in the field and the data-driven learning strategy, the decision-making process of the action device is jointly guided. In the early stage of learning, the action device mainly relies on the preset knowledge base to make decisions. At this stage, the knowledge-oriented action selection occupies a dominant position, ensuring the basic operation safety and efficiency of the action device. As time goes by, the reinforcement learning algorithm gradually accumulates experience through interaction with the environment, and continuously adjusts and optimizes the decision-making strategy. In this process, the action weight output by the reinforcement learning algorithm gradually increases, and the action weight that depends on the prior knowledge decreases accordingly, until the decision-making process of the action device is completely driven autonomously by the reinforcement learning algorithm. Through this strategy of dynamically integrating prior knowledge and real-time learning, the present application not only significantly improves the autonomous decision-making ability of the action device in a complex environment, but also effectively shortens the learning cycle and reduces the operational risks caused by trial and error. This method provides an efficient and safe solution for the intelligent control and autonomous operation of action devices, broadening its application potential in various complex application scenarios.

[0026] In order to implement the method and system provided in the embodiments of the present application, it is necessary to construct a virtual simulation environment, an information interaction interface between the environment and the algorithm, a knowledge base, and a knowledge control model and a reinforcement learning algorithm model.

[0027] Among them, the construction of virtual simulation environment: First, a mobile device virtual simulation environment was developed to instantiate a specific planning task scenario. The scenario not only includes complex external environmental factors such as terrain and climate conditions, but also includes a virtual simulation mobile device model that follows the laws of real-world dynamics.

[0028] Information exchange interface between environment and algorithm: an interface is established for status information and reward information This enables the algorithm to receive feedback based on the current state of the environment and adjust the control strategy accordingly.

[0029] Construction of knowledge base: The experience and prior knowledge of human experts in controlling action devices are converted into fuzzy rules and stored in the knowledge base to provide guidance for subsequent decision-making.

[0030] Implementation of the knowledge control model: Using the rules in the knowledge base, a knowledge control model is designed, which can issue knowledge control instructions (i.e., the first action instructions) to the action devices in the simulation environment. , guiding it to perform preliminary flight operations in a virtual environment.

[0031] Reinforcement Learning Algorithm Model: A reinforcement learning model was developed that can Learn and output the second action command to control the mobile device .

[0032] Taking the drone application scenario as an example, the corresponding system information flow process is as follows: Figure 1 As shown in the figure, based on the UAV simulation environment, information transfer between models is realized through the information interaction interface. Specifically, the prior knowledge, map data, and UAV data (such as state information) are input into the knowledge control model through the information interaction interface, and the reward function (i.e., reward information), map data, and UAV data (such as state information) are input into the reinforcement learning algorithm model through the information interaction interface.

[0033] At the beginning of the action, the knowledge control model divides the UAV's action task into multiple subtasks, and creates a sub-knowledge control model for each subtask. During the UAV's movement, the sub-knowledge control model corresponding to the current subtask is used to determine the probability distribution of the knowledge action instruction, and then generate the knowledge action instruction (i.e., the first action instruction).

[0034] The reinforcement learning model performs feature fusion on the input data and then uses the agent network to generate reinforcement learning action instructions (i.e., the second action instructions). On the other hand, the output of the agent network is used for strategy optimization.

[0035] Furthermore, we introduce a trade-off parameter , which is used to balance the action instructions output by the knowledge control model and the reinforcement learning model. The final control instruction (i.e., executable action instruction) is , where the weight It will gradually decrease as the learning process progresses.

[0036] The embodiment of the present disclosure does not limit the weight adjustment method. As a distance but not a limitation, the dynamic weight adjustment strategy provided by this application is as follows: a linear attenuation strategy is adopted, and the training steps are preset. Within, the weight It decreases linearly from 1 to 0, and its attenuation formula is: ,in, Indicates the current training step number. In the initial stage, the weight The value is close to 1. The specific value can be set as needed. At this time, the mobile device is mainly controlled by the knowledge model. As the training progresses, the reinforcement learning strategy is gradually optimized, and the parameter weight Gradually decreases, and the weight of the reinforcement learning model on the control of the action device gradually increases, until the flight of the action device is finally completely controlled by the reinforcement learning model.

[0037] The embodiments of the present application utilize expert prior knowledge or domain expert experience to achieve rapid start-up of reinforcement learning, which can significantly improve learning efficiency and effectiveness. By integrating the knowledge and experience of domain experts, the reinforcement learning model no longer needs to start from complete ignorance, but can be optimized and adjusted based on existing knowledge. This knowledge-guided learning method can accelerate the adaptation process of mobile devices to specific tasks and environments, reduce the number of trial and error required, and thus reduce risks and costs. For example, by pre-setting some basic action rules and strategies, the mobile device can conduct preliminary exploration under the guidance of these rules, avoid some obvious errors or dangerous behaviors, and quickly converge to a better strategy.

[0038] based on Figure 1 The decision-making method process of the action device based on knowledge acceleration provided by the embodiment of the present application is as follows Figure 2 As shown, the following steps are included:

[0039] Step 1: Obtain the mission goal of the mobile device, and use the mission goal to match fuzzy rules from the knowledge base, where the fuzzy rules are generated based on the prior knowledge of the expert controlling the mobile device.

[0040] A fuzzy rule may correspond to one or more action targets, as well as action knowledge for the action targets, such as the action task splitting method. The embodiments of the present application do not limit the specific content of the fuzzy rule.

[0041] Step 2: Use the matched fuzzy rules to split the travel task of the mobile device into multiple subtasks.

[0042] Preferably, the moving task of the mobile device is split into multiple subtasks using fuzzy rules according to the initial conditions of the mobile device, and the initial conditions include at least one of the following: initial flight environment, target position, initial wind speed, and obstacle information. Taking the drone material delivery task as an example, according to the current flight environment, target position, wind speed, obstacle information, etc., fuzzy rules are used to decompose the task into multiple subtask stages, such as takeoff, cruising, obstacle avoidance, landing, etc.

[0043] Step 3: Use the prior knowledge of expert control action devices to build a trainable sub-knowledge control model for each sub-task.

[0044] After that, the movement of the mobile device can be simulated on the simulation platform and subsequent operations can be performed.

[0045] Step 4: Obtain the state information at time t during the movement of the mobile device simulated on the simulation platform, wherein the state information includes environmental features and posture features of the mobile device, and execute steps 5a and 5b respectively.

[0046] Step 5a: input the state information into the sub-knowledge control model corresponding to the current sub-task to obtain a first action instruction.

[0047] Step 5b: input the state information into the reinforcement learning algorithm model, use the fully connected layer of the reinforcement learning algorithm model to fuse the environmental features in the state information and the posture features of the action device to generate hidden variable features, input the hidden variable features into the intelligent network of the reinforcement learning algorithm model, and obtain a second action instruction.

[0048] Environmental features include terrain features and / or climate features, wherein the environmental features may be, but are not limited to, map data.

[0049] Optionally, in this step, the hidden variable features are input into the intelligent network of the reinforcement learning model to obtain the value .

[0050] Step 6: Perform weighted summation on the first action instruction and the second action instruction using the set weight to obtain an executable action instruction.

[0051] Step 7: Execute the executable action instruction in the simulation platform. If there are unfinished subtasks, return to step 4.

[0052] The first action instruction, the second action instruction, and the executable action instruction are used to indicate the moving speed of the mobile device and / or the moving direction of the mobile device.

[0053] As described above, a linear attenuation strategy is adopted to dynamically adjust the set weight, and the set weight of the first action instruction is The attenuation formula is as follows:

[0054]

[0055] Indicates the current training step number, Indicates the preset number of training steps.

[0056] In the embodiment of the present application, the collection and input of status information once corresponds to one training step.

[0057] Accordingly, while obtaining parameter information, it is also possible to obtain the information about the movement of the mobile device simulated on the simulation platform. The reward information at the moment, the reward information reflects the The result of the action command execution at the moment. Furthermore, in each training step, the reward information and value can also be used Back propagation optimizes the network parameters of the reinforcement learning algorithm model and the travel strategy of the action device, and the value The hidden variable features are input into the intelligent network of the reinforcement learning algorithm model; whether the reinforcement learning algorithm model is trained to convergence is determined, if so, the reinforcement learning algorithm model is maintained and executed in the environment, otherwise, samples are continued to be collected to train the reinforcement learning algorithm model.

[0058] In order to further understand the invention content, features and functions of the embodiments of the present application, the following takes the drone decision-making process as an example to describe in detail the method provided in the embodiments of the present application:

[0059] 1. Build a reinforcement learning UAV simulation platform, which provides key real-time status information (covering environmental characteristics and the spatial position of the UAV), as well as feedback on its control actions and corresponding reward mechanisms (including collision detection and goal achievement evaluation). The UAV control system can obtain status information and reward information from the simulation environment, and issue specific action instructions based on this information, such as adjusting speed and changing heading. Including environmental features and the position and attitude of the drone Action Instructions This may include acceleration, deceleration, steering, etc. Reward feedback (i.e. reward information) Based on the results of the drone's actions, such as whether an obstacle collision occurs or successfully reach the target , which can be expressed as a function ,in It is a reward function defined based on collision and goal achievement. Reward feedback The larger the value, the more it indicates an action instruction. The better, the reinforcement learning algorithm gradually learns and optimizes the control strategy according to the reward, so that the drone control system can choose the best action according to the reward maximization.

[0060] 2. Design an information interaction interface between the environment and the algorithm so that the algorithm can receive feedback based on the current environmental status and adjust the control strategy accordingly.

[0061] 3. Constructing a knowledge control model

[0062] 3.1. The mission objectives of the UAV are input into the knowledge control model, and the knowledge control model matches multiple fuzzy rules from the knowledge base according to the mission objectives.

[0063] 3.2 According to fuzzy rules, the UAV’s flight target mission is divided into multiple flight subtasks.

[0064] 3.3. Utilize expert knowledge to build a trainable sub-knowledge control model for each flight sub-task.

[0065] 3.4. During the simulated movement of the UAV, the current state information of the UAV (such as environmental state, posture, etc.) is input into the sub-knowledge control model corresponding to the current flight subtask, and the probability distribution of the action instructions that the UAV can execute in the current state is output, and the probability distribution of the action instructions that can be executed by the UAV is used. Select the action instruction with the maximum probability as the knowledge action instruction .

[0066] 4. Build a reinforcement learning algorithm model

[0067] 4.1. During the simulated movement of the UAV, the current state information of the UAV (such as environmental state, posture characteristics, etc.) is input into the reinforcement learning algorithm model.

[0068] 4.2. Use the fully connected layer to fuse the environment features and pose features and generate hidden variable features.

[0069] 4.3. Input hidden variable features into the intelligent network and output value and the agent’s reinforcement learning action instructions .

[0070] 4.4. Based on state-action value Guide the generation of agent action instructions and optimize network parameters and agent strategies through back propagation.

[0071] 4.5 Determine whether the reinforcement learning model has been trained to convergence. If not, continue to collect samples for training; if so, keep the reinforcement learning model and execute it in the environment.

[0072] 5. Set trainable weight parameters , weighted sum of knowledge action instructions and reinforcement learning action instructions , forming the final executable action instruction.

[0073] In the above process, the knowledge model is first used to divide the tasks in the simulation environment, and the knowledge action instructions are output. , and use the reinforcement learning algorithm model to output reinforcement learning action instructions according to state characteristics ; In the early learning process, the knowledge model actions are used to interact with the environment and collect data for reinforcement learning model training; as the algorithm model is optimized, the decision-making power is gradually transferred from the knowledge model to the reinforcement learning model until a high-performance decision-making model is learned.

[0074] Based on the same inventive concept, the decision-making system of the mobile device based on knowledge acceleration provided by the embodiment of the present application is as follows: Figure 3 As shown, including:

[0075] The mobile device control module 301 is used to obtain the mobile device moving process simulated on the simulation platform. Status information at a given moment, including environmental characteristics and posture characteristics of the mobile device;

[0076] The knowledge control module 302 is used to input the state information into the sub-knowledge control model corresponding to the current sub-task to obtain a first action instruction;

[0077] The reinforcement learning control module 303 is used to input the state information into the reinforcement learning algorithm model; use the fully connected layer of the reinforcement learning algorithm to fuse the environmental features in the state information and the posture features of the action device to generate hidden variable features; input the hidden variable features into the intelligent network of the reinforcement learning algorithm model to obtain a second action instruction;

[0078] The mobile device control module 301 is further used to perform weighted summation on the first action instruction and the second action instruction using a set weight to obtain an executable action instruction; and execute the executable action instruction in the simulation platform;

[0079] The knowledge control module 302 is also used to obtain the task objectives of the mobile device's movement, and use the task objectives to match fuzzy rules from the knowledge base, where the fuzzy rules are generated based on the prior knowledge of expert control of the mobile device; use the matched fuzzy rules to split the mobile device's movement task into multiple subtasks; and use the prior knowledge of expert control of the mobile device to construct a trainable sub-knowledge control model for each subtask.

[0080] The knowledge control module may be implemented, but is not limited to, through the above-mentioned knowledge control model.

[0081] Optionally, the mobile device control module is further used to obtain the mobile device moving process simulated on the simulation platform. The reward information at the moment, the reward information reflects the The execution result of the action instruction at the moment;

[0082] The reinforcement learning control module is also used to utilize reward information and value Back propagation optimizes the network parameters of the reinforcement learning algorithm model and the travel strategy of the action device, and the value The hidden variable features are input into the intelligent network of the reinforcement learning algorithm model; whether the reinforcement learning algorithm model is trained to convergence is determined, if so, the reinforcement learning algorithm model is maintained and executed in the environment, otherwise, samples are continued to be collected to train the reinforcement learning algorithm model.

[0083] Optionally, the action instruction is used to indicate the moving speed of the mobile device and / or the moving direction of the mobile device.

[0084] Optionally, the environmental characteristics include terrain characteristics and / or climate characteristics.

[0085] The technical solution provided by the embodiments of the present application has at least the following technical effects:

[0086] 1. This technical solution significantly accelerates the learning process of mobile devices by integrating the prior knowledge of domain experts and reinforcement learning algorithms. In the early stages of learning, using expert knowledge can effectively avoid ineffective trial and error and quickly reach a relatively optimized decision-making level. As time goes by and data accumulates, the reinforcement learning algorithm further refines and optimizes the decision-making strategy, improving the ability of mobile devices to make accurate decisions in complex environments.

[0087] 2. This technical solution effectively reduces the operational risk of mobile devices before they have fully learned by relying on domain knowledge to make decisions in the early stages of learning. This approach ensures that mobile devices can avoid potentially dangerous actions when exploring new environments and strategies, thereby maintaining a high safety standard throughout the learning process.

[0088] 3. This technical solution gives mobile devices excellent adaptability and flexibility by dynamically adjusting the weights between knowledge-driven and data-driven strategies. This design enables mobile devices to flexibly adjust their decision-making strategies according to the current environment and task requirements, effectively respond to changing environmental conditions and unknown challenges, and ensure efficient task execution in various complex scenarios.

[0089] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A decision-making method for an action device based on knowledge acceleration, characterized in that: include: Step 1: Obtain the task goal of the mobile device, and use the task goal to match fuzzy rules from the knowledge base, where the fuzzy rules are generated based on the prior knowledge of the expert controlling the mobile device; Step 2: Use the matched fuzzy rules to split the travel task of the mobile device into multiple subtasks; Step 3: Use the prior knowledge of the expert control action device to build a trainable sub-knowledge control model for each sub-task; Step 4, obtaining the state information at time t during the movement of the mobile device simulated on the simulation platform, wherein the state information includes the environmental characteristics and the posture characteristics of the mobile device, and executing step 5a and step 5b respectively; Step 5a: input the state information into the sub-knowledge control model corresponding to the current sub-task to obtain a first action instruction; Step 5b, inputting the state information into a reinforcement learning algorithm model, using the fully connected layer of the reinforcement learning algorithm model to fuse the environmental features in the state information and the posture features of the action device to generate hidden variable features, and inputting the hidden variable features into the intelligent network of the reinforcement learning algorithm model to obtain a second action instruction; Step 6: Perform weighted summation on the first action instruction and the second action instruction using the set weight to obtain an executable action instruction; Step 7, executing the executable action instruction in the simulation platform, and returning to step 4; The method further comprises: Adopt linear decay strategy to dynamically adjust the set weight , the setting weight of the first action instruction The attenuation formula is as follows: Indicates the current training step number, Indicates the preset number of training steps; In the initial stage, the weights are set The value is close to 1, and the specific value is set according to the needs. As the training progresses, the weight setting Gradually decreases until it decays to 0.

2. The method according to claim 1, characterized in that The fuzzy rules obtained by matching are used to split the moving task of the mobile device into multiple subtasks, including: According to the initial conditions of the mobile device, the travel task of the mobile device is split into multiple subtasks using fuzzy rules, and the initial conditions include at least one of the following: initial flight environment, target position, initial wind speed, and obstacle information.

3. The method according to claim 1, characterized in that The method further comprises: Get the information of the mobile device moving in the simulation platform The reward information at the moment, the reward information reflects the The execution result of the action instruction at the moment; Leverage reward information and value Back propagation optimizes the network parameters of the reinforcement learning algorithm model and the travel strategy of the action device, and the value It is obtained by inputting the hidden variable features into the intelligent network of the reinforcement learning algorithm model; Determine whether the reinforcement learning algorithm model has been trained to convergence. If so, keep the reinforcement learning algorithm model and execute it in the environment; otherwise, continue to collect samples to train the reinforcement learning algorithm model.

4. The method according to claim 1, characterized in that: The executable action instruction is used to indicate the moving speed of the mobile device and / or the moving direction of the mobile device.

5. The method according to claim 1, characterized in that The environmental characteristics include terrain characteristics and / or climate characteristics.

6. A decision-making system for mobile devices based on knowledge acceleration, characterized in that: include: The mobile device control module is used to obtain the simulated mobile device movement process on the simulation platform Status information at a given moment, including environmental characteristics and posture characteristics of the mobile device; A knowledge control module, used for inputting the state information into a sub-knowledge control model corresponding to the current sub-task to obtain a first action instruction; A reinforcement learning control module, used to input the state information into a reinforcement learning algorithm model; use the fully connected layer of the reinforcement learning algorithm to fuse the environmental features in the state information and the posture features of the action device to generate hidden variable features; Inputting the hidden variable feature into the intelligent network of the reinforcement learning algorithm model to obtain a second action instruction; The mobile device control module is further used to perform weighted summation on the first action instruction and the second action instruction using a set weight to obtain an executable action instruction; and execute the executable action instruction in the simulation platform; The knowledge control module is also used to obtain the task goal of the mobile device, and use the task goal to match fuzzy rules from the knowledge base, where the fuzzy rules are generated based on the prior knowledge of the expert controlling the mobile device; use the matched fuzzy rules to split the mobile device's travel task into multiple subtasks; use the prior knowledge of the expert controlling the mobile device to construct a trainable sub-knowledge control model for each subtask; The mobile device control module is also used to: adopt a linear attenuation strategy to dynamically adjust the set weight , the setting weight of the first action instruction The attenuation formula is as follows: Indicates the current training step number, Indicates the preset number of training steps; In the initial stage, the weights are set The value is close to 1, and the specific value is set according to the needs. As the training progresses, the weight setting Gradually decreases until it decays to 0.

7. The system according to claim 6, characterized in that The mobile device control module is also used to obtain the mobile device during the simulation platform. The reward information at the moment, the reward information reflects the The execution result of the action instruction at the moment; The reinforcement learning control module is also used to utilize reward information and value Back propagation optimizes the network parameters of the reinforcement learning algorithm model and the travel strategy of the action device, and the value The hidden variable features are input into the intelligent network of the reinforcement learning algorithm model; whether the reinforcement learning algorithm model is trained to convergence is determined, if so, the reinforcement learning algorithm model is maintained and executed in the environment, otherwise, samples are continued to be collected to train the reinforcement learning algorithm model.

8. The system according to claim 6, characterized in that The executable action instruction is used to indicate the moving speed of the mobile device and / or the moving direction of the mobile device.

9. The system according to claim 6, characterized in that The environmental characteristics include terrain characteristics and / or climate characteristics.

Images