Intelligent navigation method and system for mobile charging robot based on distillation strategy

Abstract

The invention discloses a mobile charging robot intelligent navigation method and system based on a distillation strategy, and relates to the technical field of robot navigation. The method comprises the steps that the use state of a charging pile and the use state of a mobile charging robot in a target area are monitored in real time, and nonlinear road network information data in the target area are acquired; processing the non-linear road network information data, and performing environment feature extraction on the processed linear structured data by using a convolutional neural network to construct a dynamic node road network; the dynamic node road network serves as input, a DQN algorithm is used for training a teacher model for path planning, and an optimal path strategy in the dynamic environment is generated; and judging whether a knowledge distillation process needs to be carried out or not, if so, training a student model based on a distillation strategy, and visualizing a navigation path. According to the method and the system, intelligent navigation of the mobile charging robot in a weak network signal nonlinear dynamic scene can be realized.

Description

Technical Field

[0001] The present invention relates to the field of robot navigation technology, and in particular to an intelligent navigation method and system for a mobile charging robot based on a distillation strategy. Background Art

[0002] A mobile charging robot is a device that combines robotics and charging technologies, and can automatically provide charging services for electric vehicles and other equipment. It is usually composed of a chassis, battery pack, charging pile, charging gun, machinery, sensor, control system, charging port sensing system, voice interaction system, etc., and its appearance mostly appears in the form of a "vehicle", similar to a large mobile "power bank".

[0003] Mobile charging robots usually need to have autonomous navigation capabilities, be able to move freely in indoor or outdoor environments, and find the location of the device to be charged. Once the target device is found, it can automatically connect to the charging port and charge the device without human intervention.

[0004] Existing intelligent navigation algorithms can realize path recommendations in linear or nonlinear scenarios, but the dynamic scenario adaptability of these path recommendations is mostly based on Internet-based information updates. For dynamic and complex environments (such as weak network signals in underground multi-story parking lots, some vehicles blocking the road, the destination being unreachable, the location of the charging equipment moving, etc.), it is impossible to update the navigation strategy in time. The intelligent navigation method of mobile charging robots in large-scale weak network signal nonlinear dynamic scenarios is currently still in a blank stage. Summary of the invention

[0005] In response to the problems of update delays caused by weak network signals, complexity of graph topology caused by nonlinear scenarios, and difficulty in real-time updating of navigation strategies caused by dynamic scenarios, the present invention proposes an intelligent navigation method and system for a mobile charging robot based on a distillation strategy to improve the navigation efficiency of the mobile charging robot in dynamic and complex environments.

[0006] The present invention achieves the above-mentioned purpose through the following technical solutions: An intelligent navigation method for a mobile charging robot based on a distillation strategy, the method comprising: Monitor the use status of charging piles and mobile charging robots in the target area in real time, and obtain nonlinear road network information data in the target area, wherein the nonlinear road network information data at least includes geographic coordinates, altitude, and network signal strength, wherein the network signal strength is described by Wi-Fi fingerprint; The nonlinear road network information data is processed and converted into a linear structure, and the environmental features of the processed linear structured data are extracted using a convolutional neural network, and a dynamic node road network is constructed in combination with the usage status of the charging piles and the usage status of the mobile charging robot in the target area; Taking the dynamic node road network as input, using the DQN algorithm to train the teacher model for path planning, and generating the optimal path strategy in the dynamic environment; Determine whether the knowledge distillation process is needed. If necessary, train the student model based on the distillation strategy, output the training results of the teacher model with soft labels, optimize the decision probability of the soft labels according to the usage status of the charging pile and the mobile charging robot, and re-rank them to obtain the optimal navigation strategy under the current state; The real-time location of the mobile charging robot and the target charging device is determined through Wi-Fi fingerprint technology, and the navigation path planned by the teacher model or student model is visualized.

[0007] A further improvement of the present invention is that: the nonlinear road network information data is a nonlinear data set including static environmental information and dynamic environmental information, the static environmental information is directly acquired through an open source database or collected on-site by a handheld geographic information acquisition device, and the dynamic environmental information is collected in real time by a visual sensor and a laser radar; the acquired static and dynamic environmental information is presented in the form of ordered geographic information points; The use status of the charging pile includes at least four types: in use, unused, damaged, and unused but the parking space is occupied; The use status of the mobile charging robot includes at least six types: charging, on the way, in use, not in use, damaged, and low on power and waiting to be charged.

[0008] A further improvement of the present invention is that the method for processing the nonlinear road network information data and converting it into a linear structure includes: Path linearization: decompose the locations in nonlinear roads that are disconnected by altitude into linear path segments; Parameter optimization: To address the parameter expansion problem caused by environmental complexity, we adjust path weights and node attributes to reduce computational complexity. Data conversion: Convert the original nonlinear data into linear structured data to meet the training requirements of reinforcement learning models.

[0009] A further improvement of the present invention is that the method for constructing a dynamic node road network comprises: Static framework: Based on geographic coordinates and altitude, a basic node network is constructed, and network signal strength is used as an additional attribute of the node; Dynamic update: Combine the multimodal data collected by sensors in real time with the usage status of charging piles and mobile charging robots monitored in real time to dynamically adjust the properties of nodes and edges.

[0010] A further improvement of the present invention is that when the DQN algorithm is used to train the teacher model for path planning, the following environment modeling steps are included: The state space S is defined to represent the position, moving speed and distance from obstacles of the mobile charging robot in the environment. The expression is: ; In the formula, for t The status of the mobile charging robot at all times; for t The position coordinates of the mobile charging robot at all times, for t The moving speed of the charging robot at all times, for t Keep the distance between the mobile charging robot and obstacles at all times; The action space A is defined to represent the set of actions that the mobile charging robot can perform. The expression is: ; In the formula, for t The actions performed by the mobile charging robot at all times; For the current node The set of adjacent nodes of Define state transition probability , indicating that t Always take action After that, the system changes from state Transfer to The state transition probability is determined by the road network topology and the power consumption model, and the power consumption model is expressed as: ; In the formula, for t The battery level at the moment, for t +1 moment of battery power; Indicates the power consumption per unit distance. Indicates the current node Select Action The length of the walk you need to take; Defining the reward function , the expression is: ; In the formula, for t Rewards received at all times; express t The shortest distance that the charging robot moves to the target charging device at any time. express t +1: The shortest distance to the target charging device when the mobile charging robot reaches the new position after executing the action at time 1; Indicates the safe power threshold.

[0011] A further improvement of the present invention is that the training step of the teacher model includes: Define Q value to represent the action taken in state s The future cumulative reward expectation is calculated by using the Q value function to evaluate the value of each state-action pair. The Q value update formula is: ; In the formula, express t Always in the state Next action Q value; is the learning rate; is the reward given according to the path selection; is the discount factor; Using neural network parameters Approximate optimal Q value function, the formula is: ; In the formula, Represents the Q value of the neural network output; is the optimal Q value; Neural Network Architecture It is expressed as: ; In the formula, , , is the weight matrix of the neural network; , , is the bias term of the neural network; is the activation function; Model training uses the experience replay method to Storage transferred to playback buffer Among them Navigation end flag, if it is 1, it means it is in the end state, if it is 0, it means it is not in the end state; Mean square error is used to describe the loss function in model training : ; Target value The calculated value of is: ; In the formula, Indicates that the relevant parameters of the state space Put back into the playback buffer The expected value of the loss function obtained under the condition of Update the policy network using gradient descent: ; In the formula, is the neural network learning rate, is the loss function Neural network parameters The gradient of Use the ε-greedy strategy for strategy exploration and probability decay: ; After the model training is completed, the greedy strategy is used to test the model, and the initialization of the state space is used to reset and initialize the test environment, and then the navigation behavior is cyclically decided: ; In the formula, Indicates the initial state; Indicates the position of the mobile charging robot when the navigation action starts; Indicates the initial power randomly given in the test environment; Indicates the shortest moving distance before formal navigation; represents the optimal Q-value function, Take the value of the optimal neural network parameters; At each decision, DQN calculates the Q value of each action based on the current state, uses a greedy strategy to select the action with the largest Q value, and completes path optimization by continuously updating the Q value function; After training, the teacher model can select the optimal navigation path given the starting point and the end point.

[0012] A further improvement of the present invention is that the evaluation index of the teacher model includes: Path length , the formula is: ; Travel time , the formula is: ; Smoothness , the formula is: ; Node error rate , the formula is: ; In the formula, represents the distance between nodes, Indicates the first The coordinates or positions of the nodes, is the node number, m is the number of nodes; AverageSpeed ​​represents the average moving speed of the mobile charging robot on the path; Representation Node The complementary angle of the steering angle; std means standard deviation.

[0013] A further improvement of the present invention is that the method for determining whether a knowledge distillation process is required includes: The current position of the mobile charging robot is obtained as the starting point, and the position of the target charging device is obtained as the end point. If the starting point and the end point are both vertices in the dynamic node road network, the teacher model is directly called for path recommendation, and the model is solved through the ε-greedy strategy to output the path navigation result. If at least one of the starting point and the end point is not a vertex in the dynamic node network, or the dynamic node network changes during navigation, knowledge distillation is performed and the distilled student model is used for navigation path planning.

[0014] A further improvement of the present invention is that the method for training the student model based on the distillation strategy includes: The path strategy of the teacher model is transferred to the student model using the distillation strategy. If at least one of the starting point and the end point is not a vertex in the dynamic node network, the graph topology structure is updated during the student model training. The updating method is: adding new nodes, updating the attributes and weights of the corresponding edges, and using the ε-greedy strategy to solve the shortest path. If the dynamic node network changes during navigation, the student model will update the graph topology in real time, including: When a node disappears during navigation, if the disappeared node is not the end point, the node and the edges adjacent to the node are deleted in the graph topology, and the navigation is replanned to solve the shortest path; if the disappeared node is the end point, the end point is reset and the navigation is planned after updating the graph topology according to the charging needs of the target charging device, and the shortest path is solved; When a road becomes impassable during navigation, the student model updates the graph topology by deleting the impassable road on the graph topology. If the deletion causes the destination to be unreachable, the destination is reset and navigation is planned to solve the shortest path. When training the student model, it is necessary to align the state encoding in the model. By introducing the graph encoder Implement soft label and feature vector learning for student model and teacher model: Teacher Coder It is expressed as: ; Student Coder It is expressed as: ; In the formula, is the topological graph in the teacher model, is the time-varying topological graph of the student model; Represents the neural network parameters of the teacher model after being processed by the encoder. This parameter needs to be retrained in the student model; Classify the soft labels in model training into and ,in represents the abnormal action distribution, represents the normal action distribution; Distillation loss evaluation is performed through KL divergence and cross entropy evaluation, KL divergence loss The formula is: ; ; Cross Entropy Loss The formula is: ; In the formula, E represents expectation; , are the action probability distributions of the teacher model and the student model in state s respectively; , are the strategy functions of the teacher model and the student model respectively, , The teacher model and the student model are respectively Description of the distribution of Indicates that the teacher model selects an action in state s The probability of Indicates that the student model selects an action in state s The probability of , the parameter is .

[0015] A mobile charging robot intelligent navigation system based on a distillation strategy is applied to the mobile charging robot intelligent navigation method based on a distillation strategy as described above, and the system includes: A data acquisition module is used to monitor the usage status of charging piles and mobile charging robots in the target area in real time, and obtain nonlinear road network information data in the target area; A node road network construction module is used to process the nonlinear road network information data, convert it into a linear structure, and use a convolutional neural network to extract environmental features from the processed linear structured data, and build a dynamic node road network based on the usage status of the charging piles and the usage status of the mobile charging robot in the target area; A teacher model training module is used to take the dynamic node road network as input, use the DQN algorithm to train the teacher model for path planning, and generate an optimal path strategy in a dynamic environment; The student model training module is used to train the student model based on the distillation strategy, output the training results of the teacher model as soft labels, optimize the decision probability of the soft labels according to the usage status of the charging pile and the mobile charging robot, and re-rank them to obtain the optimal navigation strategy under the current state; The positioning and navigation module is used to determine the real-time location of the mobile charging robot and the target charging device through Wi-Fi fingerprint technology, and to visualize the navigation path planned by the teacher model or student model.

[0016] By improving the intelligent navigation method of traditional linear static scenes, the navigation problem of mobile charging robots in weak network signals and nonlinear dynamic scenes can be solved. Reinforcement learning is combined with knowledge distillation strategy, and Wi-Fi fingerprint positioning technology is integrated to improve the adaptability and intelligence of navigation paths, enhance the efficiency and accuracy of path planning, optimize the generalization ability of decision-making strategies, and enhance the real-time response capability of navigation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them: Figure 1 is a flow chart of the method of the present invention; Figure 2 It is a schematic diagram of the modular structure of the system of the present invention; Figure 3 A distribution map of parking spaces and charging piles in the embodiment of the present invention; Figure 4 The point-line charging pile distribution of the parking lot in the embodiment of the present invention is linearized Figure 1 ; Figure 5The point-line charging pile distribution of the parking lot in the embodiment of the present invention is linearized Figure 2 ; Figure 6 The point-line charging pile distribution of the parking lot in the embodiment of the present invention is linearized Figure 3 ; Figure 7 It is a schematic diagram comparing the decision results of the teacher model, the A* algorithm and the Dijkstra algorithm in a static scenario in an embodiment of the present invention; Figure 8 Schematic diagram of teacher model decision results in a dynamic scenario in an embodiment of the present invention; Fig. 9 Schematic diagram of the decision result of the student model in a dynamic scenario in an embodiment of the present invention; Fig.10 Schematic diagram of the probability distribution of user charging waiting time under the teacher model in an embodiment of the present invention; Fig.11 Schematic diagram of the probability distribution of user charging waiting time under the student model in an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0019] like Figure 1 As shown, it is an embodiment of the present invention, which provides an intelligent navigation method for a mobile charging robot based on a distillation strategy. The main application scenarios include weak signal areas (underground parking lots, tunnels and other scenes where GPS signals are missing or network signals are unstable), dynamic obstacles (temporarily parked vehicles, moving pedestrians, construction facilities, etc.), multi-story structures (stereoscopic navigation requirements that need to consider floor height differences), etc.

[0020] The method mainly includes the following steps: S1: Monitor the usage status of charging piles and mobile charging robots in the target area in real time, and obtain nonlinear road network information data in the target area, including at least geographic coordinates (expressed in longitude and latitude), altitude, network signal strength, etc., where the network signal strength is described by Wi-Fi fingerprint.

[0021] Step S1 is mainly used to collect data, and the target area is the working environment that the mobile charging robot needs to navigate. The use status of the charging pile means whether the charging pile in the monitoring area is available, whether it is charging, and information such as power, which includes at least four states: in use, not in use, damaged, and not in use but the parking space is occupied. The use status of the mobile charging robot is mainly used to monitor the current charging status and location of the robot, which includes at least six states: charging, on the way, in use, not in use, damaged, and low power to be charged.

[0022] Nonlinear road network information data is a nonlinear data set that contains static environmental information (road nodes, fixed obstacles, etc.) and dynamic environmental information (temporary obstacles, moving targets, road state changes, real-time network signal fluctuations, etc.). The acquired environmental information is presented in the form of ordered geographic information points.

[0023] Static environmental information is directly obtained through open source databases (open street map, UJIIndoorLoc, etc.) or collected on-site using handheld geographic information (GIS) collection equipment. For example, basic geographic data including road layout and floor distribution can be downloaded from the open street map platform, or the UJIIndoorLoc dataset designed for indoor weak network signal positioning problems can be used. In the absence of an available dataset, a handheld geographic information data collector can also be used to collect data and obtain data through on-site road information collection.

[0024] This embodiment preferably uses the UJIIndoorLoc dataset, which contains the strength of the network signal and the user's location coordinates, longitude and latitude, and altitude, including multiple test areas, in which the network signal strength and the corresponding coordinate positions are measured and marked. These data provide a reliable verification basis for convolutional neural network and deep reinforcement learning model training, and path planning algorithms.

[0025] Dynamic environmental information is collected in real time through visual sensors, lidar, etc. Furthermore, visual sensors, lidar and other sensors are directly embedded in the mobile charging robot body to support low latency and high precision characteristics of real-time environmental perception and path planning.

[0026] Wi-Fi Fingerprint is an indoor positioning technology based on Wi-Fi signal characteristics. In the offline stage, the Wi-Fi signal strength information (such as RSSI) of different locations in the environment is collected in advance and stored as "fingerprint" data to establish a fingerprint database; in the online stage, the device collects the current Wi-Fi signal characteristics in real time and matches them with the fingerprints in the fingerprint database to determine its own position. It has the characteristics of low cost and strong applicability, but the fingerprint database needs to be updated regularly to cope with environmental changes. It is difficult to use GPS positioning in weak network signal scenarios. Wi-Fi fingerprint is an important solution for spatial positioning in weak network scenarios. Therefore, the present invention uses Wi-Fi fingerprint to describe the network signal strength in the scene.

[0027] S2: Process the nonlinear road network information data and convert it into a linear structure. Use the convolutional neural network to extract environmental features (such as longitude and latitude, altitude, network signal strength, fixed obstacle locations, and dynamic targets) from the processed linear structured data. Combine the usage status of the charging piles in the target area and the usage status of the mobile charging robot to build a dynamic node road network (graph topology).

[0028] Methods for processing nonlinear road network information data and converting them into linear structures include: Path linearization: Decompose the locations in nonlinear roads that are disconnected due to altitude (such as ramps and floor connections) into linear path segments to ensure that the topology structure is resolvable; Parameter optimization: To address the parameter expansion problem caused by environmental complexity (such as the increase in dynamic obstacles), we adjust path weights and node attributes to suppress computational complexity. Data conversion: Convert the original nonlinear data (such as curved roads and three-dimensional structures) into linear structured data to meet the training requirements of reinforcement learning models.

[0029] The environmental perception capability of the convolutional neural network is used to perform convolution operations on the converted linear structured data, extract spatial features from the high-dimensional perception data of the data set, and transform the environmental information into graph topology network data that can be processed by the intelligent agent. Node attributes include geographic coordinates (latitude and longitude), altitude, signal strength, etc., and edge attributes include road length, traffic status, etc.

[0030] The convolutional layer of the convolutional neural network is set up as follows: The first convolutional layer: contains 32 convolution kernels, size 3*3, stride 1, padding "same", extracting low-level features (such as road edges, obstacle contours); The second convolutional layer: contains 64 convolution kernels, size 3*3, stride 1, padding "same", extracting complex features (such as dynamic target location, passable area); The third volume base layer: contains 128 convolution kernels, size 3*3, stride 1, padding "same", extracting high-level features (such as road connectivity, network signal distribution).

[0031] After each convolution operation, the maximum pooling layer reduces the dimension of the feature map, further reducing the complexity of the agent's calculation. Through the operation of three convolutional layers, the convolutional neural network can effectively identify obstacles, dynamic targets and other environmental information, providing real-time perception support for path planning.

[0032] Methods for constructing a dynamic node road network include: Static framework: Based on geographic coordinates and altitude, a basic node network (such as road nodes and fixed obstacles) is constructed, and the network signal strength is used as an additional attribute of the node; Dynamic update: Combined with multimodal data collected by sensors in real time (such as temporary obstacles, moving vehicles, and road availability data), as well as the usage status of charging piles and mobile charging robots monitored in real time, the properties of nodes and edges are dynamically adjusted (such as deleting blocked roads and adding temporary nodes).

[0033] S3: Take the dynamic node road network as input, use the DQN (Deep Q-Network) algorithm to train the teacher model for path planning, and generate the optimal path strategy in the dynamic environment; DQN optimizes the path selection of the mobile charging robot by continuously interacting with the environment, enabling it to find the optimal path in a dynamic environment and use the model as a teacher model.

[0034] When using the DQN algorithm to train the teacher model for path planning, the following environment modeling steps are included: The state space S is defined to represent the position, moving speed and distance from obstacles of the mobile charging robot in the environment. The expression is: ; In the formula, for t The status of the mobile charging robot at all times; for t The position coordinates of the mobile charging robot at all times, for t The moving speed of the charging robot at all times, for t Keep the distance between the mobile charging robot and obstacles at all times; The action space A is defined to represent the set of actions that the mobile charging robot can perform. The expression is: ; In the formula, for t The actions performed by the mobile charging robot at all times; For the current node The set of adjacent nodes of Define state transition probability , indicating that t Always take action After that, the system changes from state Transfer to The state transition probability is determined by the road network topology and the power consumption model. The power consumption model is expressed as: ; In the formula, for t The battery level at the moment, for t +1 moment of battery life, given by t +1 moment of battery life Determine the action to be taken ; Indicates the power consumption per unit distance. Indicates the current node Select Action The length of the walk you need to take; Defining the reward function , the expression is: ; In the formula, express t The shortest distance that the charging robot moves to the target charging device at any time. express t +1: When the mobile charging robot reaches the new position after executing the action at time 1, the shortest distance to the target charging device; that is, when the robot shortens the distance to the target charging device through the action ( ), the reward increases, otherwise the reward decreases; Indicates the safety power threshold. If the power level is lower than the safety threshold, an alarm will be triggered or the robot will be unable to move. When the battery power is lower than the safety threshold at the next moment, a penalty of -5 will be imposed to ensure that the robot maintains a safe power level.

[0035] In this embodiment, the training steps of the teacher model include: Define Q value to represent the action taken in state s (s、 It is a general variable, which represents the expected future cumulative reward after all possible states of the mobile charging robot and any action in the set of all executable actions. The value of each state-action pair is evaluated by the Q-value function. The Q-value update formula is: ; In the formula, express t Always in the state Next action Q value; is the learning rate; is the immediate reward value given according to the path selection; is the discount factor.

[0036] The parameter settings during model training are shown in Table 1: Table 1 Parameters of the teacher model trained by the DQN algorithm

[0037] Using neural network parameters Approximate optimal Q value function, the formula is: ; In the formula, Represents the Q value of the neural network output; is the optimal Q value; Neural Network Architecture It is expressed as: ; In the formula, , , is the weight matrix of the neural network; , , is the bias term of the neural network; is the activation function; Model training uses the experience replay method to Storage transferred to playback buffer Among them Navigation end flag, if it is 1, it means it is in the end state, if it is 0, it means it is not in the end state; Mean square error is used to describe the loss function in model training : ; Target value The calculated value of is: ; In the formula, Indicates that the relevant parameters of the state space Put back into the playback buffer The expected value of the loss function obtained under the condition of , using the expected value to describe the loss function can reduce the impact of noise generated by random data in training; Update the policy network using gradient descent: ; In the formula, is the neural network learning rate, is the loss function Neural network parameters The gradient of Use the ε-greedy strategy for strategy exploration and probability decay: ; After the model training is completed, the greedy strategy is used to test the model, and the initialization of the state space is used to reset and initialize the test environment, and then the navigation behavior is cyclically decided: ; In the formula, Indicates the initial state; Indicates the position of the mobile charging robot when the navigation action starts; Indicates the initial power randomly given in the test environment; Indicates the shortest moving distance before formal navigation; represents the optimal Q-value function, Take the value of the optimal neural network parameters; At each decision, DQN calculates the Q value of each action based on the current state, uses a greedy strategy to select the action with the largest Q value, and completes path optimization by continuously updating the Q value function; After training, the teacher model can select the optimal navigation path given the starting point and the end point.

[0038] In order to improve the training stability, the DQN algorithm in this embodiment uses the experience replay mechanism. Each time it interacts with the environment, DQN will store each experience (i.e., the current state, the action taken, the reward obtained, the next state, etc.) in the experience pool. DQN randomly extracts a batch of experience from the experience pool for training and updates the Q value. This mechanism breaks the correlation between experiences, helps to avoid overfitting, and improves learning efficiency.

[0039] In this embodiment, the evaluation indicators of the teacher model include: Path length , the formula is: ; Travel time , the formula is: ; Smoothness , the formula is: ; Node error rate , the formula is: ; In the formula, represents the distance between nodes, Indicates the first The coordinates or positions of the nodes, is the node number, m is the number of nodes; AverageSpeed ​​represents the average moving speed of the mobile charging robot on the path; Representation Node The complementary angle of the steering angle; std means standard deviation.

[0040] S4: Determine whether the knowledge distillation process is necessary. If necessary, train the student model based on the distillation strategy, output the training results of the teacher model with soft labels, optimize the decision probability of the soft labels according to the usage status of the charging piles and mobile charging robots, and re-sort them to obtain the optimal navigation strategy under the current state and display it visually.

[0041] Specifically, the method for determining whether the knowledge distillation process is needed includes: Get the current position of the mobile charging robot as the starting point and the position of the target charging device as the end point. If the starting point and the end point are both vertices in the dynamic node network (that is, the starting point and the end point are both preset nodes), directly call the teacher model for path recommendation, solve the model through the ε-greedy strategy, and output the path navigation result; If at least one of the starting point and the end point is not a vertex in the dynamic node network, or the dynamic node network changes during the navigation process (such as node disappearance, road congestion), knowledge distillation is performed and the distilled student model is used for navigation path planning.

[0042] In one embodiment, a method for training a student model based on a distillation strategy includes: The path strategy of the teacher model is transferred to the student model using the distillation strategy. If at least one of the starting point and the end point is not a vertex in the dynamic node network, the graph topology structure is updated during the student model training. The updating method is: adding new nodes, updating the attributes and weights of the corresponding edges, and using the ε-greedy strategy to solve the shortest path. If the dynamic node network changes during navigation, the student model will update the graph topology in real time, including: When a node disappears during navigation, if the disappeared node is not the end point, the node and the edges adjacent to the node are deleted in the graph topology, and the navigation is replanned to solve the shortest path; if the disappeared node is the end point, the end point is reset and the navigation is planned after updating the graph topology according to the charging needs of the target charging device, and the shortest path is solved; When a road becomes impassable during navigation, the student model updates the graph topology by deleting the impassable road on the graph topology. If the deletion causes the destination to be unreachable, the destination is reset and navigation is planned to solve the shortest path.

[0043] When training the student model, it is necessary to align the state encoding in the model. By introducing the graph encoder Implement soft label and feature vector learning for student model and teacher model: Teacher Coder It is expressed as: ; Student Coder It is expressed as: ; In the formula, , are the states of the teacher model and the student model respectively; is the topological graph in the teacher model, is the time-varying topological graph of the student model; Represents the neural network parameters of the teacher model after being processed by the encoder. This parameter needs to be retrained in the student model, but the training process is simpler than that of the teacher model. Classify the soft labels in model training into and ,in represents the abnormal action distribution, represents the normal action distribution; Distillation loss evaluation is performed through KL divergence and cross entropy evaluation, KL divergence loss The formula is: ; ; Cross Entropy Loss The formula is: ; In the formula, E represents expectation; , are the action probability distributions of the teacher model and the student model in state s respectively; , are the strategy functions of the teacher model and the student model respectively, , The teacher model and the student model are respectively Description of the distribution of Indicates that the teacher model selects an action in state s The probability of Indicates that the student model selects an action in state s probability.

[0044] S5: The real-time location of the mobile charging robot and the target charging device is determined through Wi-Fi fingerprint technology, and the navigation path planned by the teacher model or the student model is visualized.

[0045] like Figure 2 FIG. 1 is another embodiment of the present invention, which provides a mobile charging robot intelligent navigation system based on a distillation strategy, and is applied to a mobile charging robot intelligent navigation method based on a distillation strategy as described above, including: A data acquisition module is used to monitor the usage status of charging piles and mobile charging robots in the target area in real time, and obtain nonlinear road network information data in the target area; The node road network construction module is used to process the nonlinear road network information data, convert it into a linear structure, and use the convolutional neural network to extract environmental features from the processed linear structured data, and build a dynamic node road network based on the usage status of the charging piles and the mobile charging robot in the target area; The teacher model training module is used to take the dynamic node road network as input, use the DQN algorithm to train the teacher model for path planning, and generate the optimal path strategy in the dynamic environment; The student model training module is used to train the student model based on the distillation strategy, output the training results of the teacher model as soft labels, optimize the decision probability of the soft labels according to the usage status of the charging piles and mobile charging robots, and re-rank them to obtain the optimal navigation strategy under the current status; The positioning and navigation module is used to determine the real-time location of the mobile charging robot and the target charging device through Wi-Fi fingerprint technology, and to visualize the navigation path planned by the teacher model or student model.

[0046] The working principle of the system in this embodiment is further explained below by taking an underground parking lot as an example.

[0047] Step 1: Collect data - draw a "parking lot map" Use cameras, lidar and other equipment to scan underground parking lots, record information such as road locations, obstacles, floor heights, network signal strength, etc., and generate .csv format data files containing latitude and longitude, altitude, network signals, etc. through sensor scanning, so that the robot can understand where there are roads, where there are traffic jams, where the signal is poor, etc.

[0048] For positioning in weak networks, Wi-Fi fingerprints are used instead of GPS to determine the robot's location. The Wi-Fi signal distribution at different locations is recorded in advance, and the robot compares the current signal in real time and matches it to the preset location to achieve accurate positioning.

[0049] like Figure 3 As shown, the underground parking lot has 4 floors, each with a transition platform between them. The platforms also have parking spaces and charging spaces for electric vehicles, that is, there are 8 layers of map data in total. The 8 layers of map data overlap in longitude and latitude, forming a relatively complex nonlinear road network structure.

[0050] Step 2: Data processing - turning complex roads into "point-line maps" Use convolutional neural networks (CNN) to automatically identify road features (such as intersections and obstacles), generate simplified maps consisting of points and lines, and convert the scanned complex roads into a "node road network diagram" that computers can understand, thereby simplifying environmental information and making it easier for robots to plan paths.

[0051] like Figure 4-Figure 6 As shown in the figure, the 8-layer graph data is disassembled according to the different altitudes and combined according to the platform layer, and the complex nonlinear road network data is split into 3 different plane data, that is, the road network structure is converted into point-line linear road network data that is easier for the intelligent agent to understand (0 means idle, 1 means damaged, and 2 means in use or occupied). Figure 4 , Figure 6 The parking spaces are distributed in a "king" shape on the main floors, with a large number of parking spaces and a relatively even distribution. Figure 4 There are more damaged charging piles on the indicated floors. Figure 5 The parking spaces in the building are distributed in a "hyperbolic" shape. This floor is a platform between the main floors. There are fewer parking spaces and charging piles. Because there are other requirements such as fire protection on the platform, the charging piles in most parking spaces are out of use. Figure 4-Figure 6 The complexly distributed parking spaces and charging stations are converted into two-dimensional data consisting of longitude and latitude, which facilitates model training for intelligent agents.

[0052] Step 3: Train the "teacher model" - teach the robot to find the optimal path A "teacher model" is trained using deep reinforcement learning (DQN) to allow the robot to learn how to avoid obstacles and choose the shortest path through trial and error. The robot keeps trying different routes, adjusting its strategy based on arrival time and path length, and eventually learns to navigate efficiently. The teacher model is complex but smart and can handle a variety of complex situations, but it requires a lot of computing resources.

[0053] like Figure 7 As shown in the figure, when computing resources are sufficient, the path decision given by the teacher model trained by CNN+DQN is smoother, that is, it can avoid taking detours, while the path decision given by the traditional model (A* algorithm and Dijkstra algorithm) is less smooth, and there are situations of spinning in place and turning around.

[0054] Step 4: Train the "student model" - a lightweight navigation assistant The knowledge of the teacher model is "compressed" into a lighter student model to adapt to dynamically changing environments. The student model is faster and more flexible, suitable for real-time route adjustment.

[0055] For example: when the starting point or end point is not at the preset node, the student model can automatically add a new node to the map and re-plan the path (such as a temporary parking space suddenly appears). When the road is blocked or the node disappears, the student model can delete the node of the blocked road and find a new way (such as bypassing illegally parked vehicles). When the vehicle to be charged moves, the student model can update the end point position in time and recalculate the path (follow the target vehicle).

[0056] like Figure 8 , Fig. 9 As shown in the figure, in a dynamic scenario, the linear road network is reorganized into a nonlinear road network. Directly using the teacher model for training will cause the model side parameter expansion due to the dynamic scenario, resulting in online delays on the user side. The dynamic changes of charging piles, mobile charging robots and road conditions are not updated in time. Although the target location is successfully reached in the end, the path smoothness is poor and the efficiency is low. Fig. 9 The navigation path provided by the middle school student model is smoother and more efficient.

[0057] like Fig.10 , Fig.11 As shown in Figure 2, the waiting time distribution from the time a user sends a charging request to the time the charging process is successfully completed under the teacher model and the student model is compared. The waiting time of the user under the teacher model is obviously longer than that under the student model.

[0058] The reinforcement learning model of the present invention is used to compare with the A* algorithm and the Dijkstra algorithm to evaluate the path planning effect of all algorithms on the mobile charging robot in the same test area. In the nonlinear scenario, the A* algorithm and the Dijkstra algorithm did not find an effective path within the specified time. The knowledge distillation method was used to successfully plan the path in the nonlinear scenario with a shorter path length, showing a higher path optimization capability. Table 2 shows the evaluation results of the three algorithms.

[0059] Table 2 Comparative experimental results

[0060] During the model training process, the distillation strategy of reinforcement learning is carried out with the help of the Wi-Fi Fingerprint of the scene, so it performs well in multiple weak signal areas. The test results show that once dynamic changes occur in the scene, the A* algorithm and the Dijkstra algorithm cannot make adaptive adjustments, causing the mobile charging robot to stay in place due to blocked roads or the movement or disappearance of the target node. Table 3 shows the test results in multiple weak signal areas.

[0061] Table 3 Test results of indoor weak network signal area

[0062] The experimental results prove that the intelligent navigation model based on the reinforcement learning distillation strategy proposed in the present invention can improve the path planning and obstacle avoidance capabilities of the mobile charging robot to the vehicle to be navigated in a dynamic and complex environment. Compared with the traditional A* algorithm and Dijkstra algorithm, the reinforcement learning algorithm reduces the path length by 15.9% compared with the traditional algorithm, and shows strong robustness in multiple weak signal areas. In addition, the method proposed in the present invention also shows better performance in real-time performance and path smoothness, can effectively cope with dynamically changing environments and sudden obstacles, and effectively reduce the probability of accidents of mobile charging robots during movement. The experimental results fully verify the adaptability and accuracy of the present invention in complex environments.

[0063] In summary, the present invention realizes more refined and dynamic navigation data acquisition by real-time monitoring of the usage status of charging piles and mobile charging robots in the target area and integrating nonlinear road network information (such as geographic coordinates, altitude and network signal strength), ensuring that the path planning process fully adapts to complex and changeable environmental factors.

[0064] By processing nonlinear road network information in a structured manner and using convolutional neural networks to extract environmental features, the accuracy of path modeling is effectively enhanced. The dynamic node road network built on this basis makes path planning more in line with actual dynamic scenarios and improves the overall intelligence level of the navigation system.

[0065] A DQN-based reinforcement learning path training mechanism is introduced. The teacher model generates the optimal path strategy in a dynamic environment, and the student model is trained in combination with the distillation strategy. This significantly reduces the computational complexity while ensuring the model's accuracy, and improves the model's generalization ability and deployment flexibility in different scenarios.

[0066] Based on Wi-Fi fingerprint technology, high-precision positioning of robots and target devices is achieved, and combined with teacher or student models, the optimal path is generated and visualized in real time, which effectively improves the system's response speed and path visibility in practical applications and enhances the user's operating experience.

[0067] The present invention effectively improves the navigation intelligence and execution efficiency of the mobile charging robot in complex environments through multi-source heterogeneous data fusion, deep learning path planning and knowledge distillation strategy, and has broad application prospects and industrial promotion value.

[0068] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or other arbitrary combinations. When implemented using software, it can be implemented in whole or in part in the form of a computer program product, which includes one or more computer instructions. When loading and executing computer program instructions on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0069] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0070] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A mobile charging robot intelligent navigation method based on distillation strategy, characterized in that: The method comprises: Monitor the use status of charging piles and mobile charging robots in the target area in real time, and obtain nonlinear road network information data in the target area, wherein the nonlinear road network information data at least includes geographic coordinates, altitude, and network signal strength, wherein the network signal strength is described by Wi-Fi fingerprint; The nonlinear road network information data is processed and converted into a linear structure, and the environmental features of the processed linear structured data are extracted using a convolutional neural network, and a dynamic node road network is constructed in combination with the usage status of the charging piles and the usage status of the mobile charging robot in the target area; Taking the dynamic node road network as input, using the DQN algorithm to train the teacher model for path planning, and generating the optimal path strategy in the dynamic environment; Determine whether the knowledge distillation process is needed. If necessary, train the student model based on the distillation strategy, output the training results of the teacher model with soft labels, optimize the decision probability of the soft labels according to the usage status of the charging pile and the mobile charging robot, and re-rank them to obtain the optimal navigation strategy under the current state; The real-time location of the mobile charging robot and the target charging device is determined through Wi-Fi fingerprint technology, and the navigation path planned by the teacher model or student model is visualized.

2. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 1, characterized in that: The nonlinear road network information data is a nonlinear data set including static environmental information and dynamic environmental information. The static environmental information is directly acquired through an open source database or collected on-site by a handheld geographic information acquisition device, and the dynamic environmental information is collected in real time by a visual sensor and a laser radar. The acquired static and dynamic environmental information is presented in the form of ordered geographic information points. The use status of the charging pile includes at least four types: in use, unused, damaged, and unused but the parking space is occupied; The use status of the mobile charging robot includes at least six types: charging, on the way, in use, not in use, damaged, and low on power and waiting to be charged.

3. The intelligent navigation method of a mobile charging robot based on a distillation strategy as claimed in claim 1, characterized in that: The method for processing the nonlinear road network information data and converting it into a linear structure includes: Path linearization: decompose the locations in nonlinear roads that are disconnected by altitude into linear path segments; Parameter optimization: To address the parameter expansion problem caused by environmental complexity, we adjust path weights and node attributes to reduce computational complexity. Data conversion: Convert the original nonlinear data into linear structured data to meet the training requirements of reinforcement learning models.

4. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 1, characterized in that: Methods for constructing a dynamic node road network include: Static framework: Based on geographic coordinates and altitude, a basic node network is constructed, and network signal strength is used as an additional attribute of the node; Dynamic update: Combine the multimodal data collected by sensors in real time with the usage status of charging piles and mobile charging robots monitored in real time to dynamically adjust the properties of nodes and edges.

5. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 1, characterized in that: When the DQN algorithm is used to train the teacher model for path planning, the following environment modeling steps are included: The state space S is defined to represent the position, moving speed and distance from obstacles of the mobile charging robot in the environment. The expression is: ； In the formula, for t The status of the mobile charging robot at all times; for t The position coordinates of the mobile charging robot at all times, for t The moving speed of the charging robot at all times, for t Keep the distance between the mobile charging robot and obstacles at all times; The action space A is defined to represent the set of actions that the mobile charging robot can perform. The expression is: ； In the formula, for t The actions performed by the mobile charging robot at all times; For the current node The set of adjacent nodes of Define state transition probability , indicating that t Always take action After that, the system changes from state Transfer to The state transition probability is determined by the road network topology and the power consumption model, and the power consumption model is expressed as: ； In the formula, for t The battery level at the moment, for t +1 moment of battery power; Indicates the power consumption per unit distance. Indicates the current node Select Action The length of the walk you need to take; Defining the reward function , the expression is: ； In the formula, for t Rewards received at all times; express t The shortest distance that the charging robot moves to the target charging device at any time. express t +1: The shortest distance to the target charging device when the mobile charging robot reaches the new position after executing the action at time 1; Indicates the safe power threshold.

6. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 5, characterized in that: The training steps of the teacher model include: Define Q value to represent the action taken in state s The future cumulative reward expectation is calculated by using the Q value function to evaluate the value of each state-action pair. The Q value update formula is: ； In the formula, express t Always in the state Next action Q value; is the learning rate; is the reward given according to the path selection; is the discount factor; Using neural network parameters Approximate optimal Q value function, the formula is: ； In the formula, Represents the Q value of the neural network output; is the optimal Q value; Neural Network Architecture It is expressed as: ； In the formula, , , is the weight matrix of the neural network; , , is the bias term of the neural network; is the activation function; Model training uses the experience replay method to Storage transferred to playback buffer Among them Navigation end flag, if it is 1, it means it is in the end state, if it is 0, it means it is not in the end state; Mean square error is used to describe the loss function in model training : ； Target value The calculated value of is: ； In the formula, Indicates that the relevant parameters of the state space Put back into the playback buffer The expected value of the loss function obtained under the condition of Update the policy network using gradient descent: ； In the formula, is the neural network learning rate, is the loss function Neural network parameters The gradient of Use the ε-greedy strategy for strategy exploration and probability decay: ； After the model training is completed, the greedy strategy is used to test the model, and the initialization of the state space is used to reset and initialize the test environment, and then the navigation behavior is cyclically decided: ； In the formula, Indicates the initial state; Indicates the position of the mobile charging robot when the navigation action starts; Indicates the initial power randomly given in the test environment; Indicates the shortest moving distance before formal navigation; represents the optimal Q-value function, Take the value of the optimal neural network parameters; At each decision, DQN calculates the Q value of each action based on the current state, uses a greedy strategy to select the action with the largest Q value, and completes path optimization by continuously updating the Q value function; After training, the teacher model can select the optimal navigation path given the starting point and the end point.

7. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 6, characterized in that: The evaluation indicators of the teacher model include: Path length , the formula is: ； Passing time , the formula is: ； Smoothness , the formula is: ； Node error rate , the formula is: ； In the formula, represents the distance between nodes, Indicates the first The coordinates or positions of the nodes, is the node number, m is the number of nodes; AverageSpeed ​​represents the average moving speed of the mobile charging robot on the path; Representation Node The complementary angle of the steering angle; std represents the standard deviation.

8. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 1, characterized in that: The method for determining whether a knowledge distillation process is required includes: The current position of the mobile charging robot is obtained as the starting point, and the position of the target charging device is obtained as the end point. If the starting point and the end point are both vertices in the dynamic node road network, the teacher model is directly called for path recommendation, and the model is solved through the ε-greedy strategy to output the path navigation result. If at least one of the starting point and the end point is not a vertex in the dynamic node network, or the dynamic node network changes during navigation, knowledge distillation is performed and the distilled student model is used for navigation path planning.

9. The mobile charging robot intelligent navigation method based on distillation strategy as claimed in claim 8, characterized in that: The method for training the student model based on the distillation strategy includes: The path strategy of the teacher model is transferred to the student model using the distillation strategy. If at least one of the starting point and the end point is not a vertex in the dynamic node network, the graph topology structure is updated during the student model training. The updating method is: adding new nodes, updating the attributes and weights of the corresponding edges, and using the ε-greedy strategy to solve the shortest path. If the dynamic node network changes during navigation, the student model will update the graph topology in real time, including: When a node disappears during navigation, if the disappeared node is not the end point, the node and the edges adjacent to the node are deleted in the graph topology, and the navigation is replanned to solve the shortest path; if the disappeared node is the end point, the end point is reset and the navigation is planned after updating the graph topology according to the charging needs of the target charging device, and the shortest path is solved; When a road becomes impassable during navigation, the student model updates the graph topology by deleting the impassable road on the graph topology. If the deletion causes the destination to be unreachable, the destination is reset and navigation is planned to solve the shortest path. When training the student model, it is necessary to align the state encoding in the model. By introducing the graph encoder Implement soft label and feature vector learning for student model and teacher model: Teacher Coder It is expressed as: ; Student Coder It is expressed as: ; In the formula, is the topological graph in the teacher model, is the time-varying topological graph of the student model; Represents the neural network parameters of the teacher model after being processed by the encoder. This parameter needs to be retrained in the student model; Classify the soft labels in model training into and ,in represents the abnormal action distribution, represents the normal action distribution; Distillation loss evaluation is performed through KL divergence and cross entropy evaluation, KL divergence loss The formula is: ； ； Cross Entropy Loss The formula is: ； In the formula, E represents expectation; , are the action probability distributions of the teacher model and the student model in state s respectively; , are the strategy functions of the teacher model and the student model respectively, , The teacher model and the student model are respectively Description of the distribution of Indicates that the teacher model selects an action in state s The probability of Indicates that the student model selects an action in state s The probability of , the parameter is .

10. A mobile charging robot intelligent navigation system based on a distillation strategy, applied to a mobile charging robot intelligent navigation method based on a distillation strategy as claimed in any one of claims 1 to 9, characterized in that: The system comprises: A data acquisition module is used to monitor the usage status of charging piles and mobile charging robots in the target area in real time, and obtain nonlinear road network information data in the target area; A node road network construction module is used to process the nonlinear road network information data, convert it into a linear structure, and use a convolutional neural network to extract environmental features from the processed linear structured data, and build a dynamic node road network based on the usage status of the charging piles and the usage status of the mobile charging robot in the target area; A teacher model training module is used to take the dynamic node road network as input, use the DQN algorithm to train the teacher model for path planning, and generate an optimal path strategy in a dynamic environment; The student model training module is used to train the student model based on the distillation strategy, output the training results of the teacher model as soft labels, optimize the decision probability of the soft labels according to the usage status of the charging pile and the mobile charging robot, and re-rank them to obtain the optimal navigation strategy under the current state; The positioning and navigation module is used to determine the real-time location of the mobile charging robot and the target charging device through Wi-Fi fingerprint technology, and to visualize the navigation path planned by the teacher model or student model.

Images