A mobile charging robot obstacle avoidance method and system based on multi-sensor fusion

Abstract

The present invention discloses a mobile charging robot obstacle avoidance method and system based on multi-sensor fusion, which belongs to the field of robot technology, and includes real-time acquisition of surrounding environment information to obtain first environment data; judging the obstacle avoidance level according to the first environment data; if the obstacle avoidance level is primary, then roughly avoid obstacles by emitting radio waves to detect surrounding obstacles; if the obstacle avoidance level is intermediate, then collect second environment data through multiple sensors, and plan obstacle avoidance paths according to the second environment data to predict the obstacle movement trajectory; if the obstacle avoidance level is advanced, obtain third environment data through real-time scanning by unmanned aerial vehicle and build a real-time dynamic environment prediction model for the next moment in combination with the second environment data, and dynamically plan the obstacle avoidance path after the next moment to avoid obstacles. The present invention adopts the strategy of real-time acquisition of environmental information and judgment of obstacle avoidance level, and can take obstacle avoidance measures of different precisions according to different levels, thereby improving the accuracy and efficiency of obstacle avoidance.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to an obstacle avoidance method and system for a mobile charging robot based on multi-sensor fusion. Background Art

[0002] With the rapid development of science and technology, mobile charging robots are increasingly used in various places such as shopping malls, airports, hospitals, etc. These robots not only provide customers with convenient charging services, but also improve the overall intelligence level of the place. However, in actual applications, mobile charging robots face complex obstacle avoidance challenges. Especially in environments with large flow of people and many obstacles, how to ensure that the robot can safely and efficiently shuttle between people and obstacles has become an urgent problem to be solved.

[0003] Traditional obstacle avoidance technology lacks the ability to flexibly adjust obstacle avoidance strategies according to the complexity of different environments. In low-complexity environments, simple sensors such as infrared sensors can provide sufficient environmental information. However, in high-complexity environments with multiple dynamic obstacles, such single sensors and corresponding obstacle avoidance algorithms cannot meet the requirements of safety and efficiency. To address the above problems, some studies have proposed using multi-sensor data for comprehensive judgment, but there are still the following problems: high computational complexity and insufficient real-time performance, especially in emergency situations, where rapid obstacle avoidance decisions may be limited; in addition, there is also a lack of a mechanism to accurately predict the path of obstacles in the future, which is particularly important in highly dynamic environments. Therefore, it is necessary to propose an obstacle avoidance method that can adjust obstacle avoidance strategies in real time and has high-precision path planning capabilities to ensure the efficient operation of mobile robots in complex environments. Summary of the invention

[0004] To solve the above problems, the present invention provides a mobile charging robot obstacle avoidance method and system based on multi-sensor fusion, which adopts the strategy of real-time collection of environmental information and judgment of obstacle avoidance level. It can take obstacle avoidance measures with different precisions according to different levels, thereby improving the accuracy and efficiency of obstacle avoidance.

[0005] The above objectives can be achieved through the following solutions:

[0006] A mobile charging robot obstacle avoidance method based on multi-sensor fusion, the method comprising: real-time collection of surrounding environment information to obtain first environment data; judging the obstacle avoidance level according to the first environment data; if the obstacle avoidance level is primary, detecting surrounding obstacles by emitting radio waves to perform rough obstacle avoidance; if the obstacle avoidance level is intermediate, collecting second environment data through multiple sensors, predicting obstacle movement trajectories according to the second environment data, and planning obstacle avoidance paths to avoid obstacles; if the obstacle avoidance level is advanced, obtaining third environment data through real-time scanning by an unmanned aerial vehicle, building a real-time dynamic environment prediction model for the next moment according to the second environment data and the third environment data, and dynamically planning obstacle avoidance paths after the next moment to avoid obstacles in combination with the real-time dynamic environment prediction model for the next moment.

[0007] Optionally, determining the obstacle avoidance level according to the first environmental data includes: determining whether there are obstacles around according to the first environmental data; if there are no obstacles around, not starting the obstacle avoidance function; if there are obstacles around, determining the obstacle avoidance level.

[0008] Optionally, if there are obstacles around, determining the obstacle avoidance level also includes: determining the state of the obstacle based on the first environmental data; if the obstacle is static, determining that the obstacle avoidance level is primary; if the obstacle is dynamic, determining the obstacle avoidance level based on the number, quantity, and speed of direction changes of the obstacle.

[0009] Optionally, the determining of the obstacle avoidance level according to the number, quantity and speed of direction changes of the obstacles comprises: setting a first weight factor for the number of direction changes of the obstacles , set the second weight factor for obstacle speed ; Score the number, quantity and speed of obstacles’ direction changes, and construct a scoring matrix for the number and speed of obstacles’ direction changes , where each row represents an obstacle and each column represents an evaluation condition (including the number of direction changes and speed). The scoring matrix The structure is:

[0010] ,

[0011] In the formula, For the The score of the number of direction changes of each obstacle; For the The speed score of each obstacle is calculated; the weighted score of the number of direction changes and the speed weighted score of each obstacle are calculated, and the weighted total score of each obstacle is obtained by adding the weighted score of the number of direction changes and the speed weighted score of each obstacle. ,have

[0012] ;

[0013] Add the weighted total score of all obstacles to the score of the number of obstacles to get the actual score , for the actual score value have

[0014] ,

[0015] In the formula, is the number of obstacles; is a scoring value for the number of obstacles; and the obstacle avoidance level is determined according to the actual scoring value and the first threshold.

[0016] Optionally, scoring the number, quantity and speed of direction changes of obstacles includes: setting a second threshold for the number of obstacle direction changes, setting a third threshold for the obstacle speed, and setting a fourth threshold for the number of obstacles; if the number of obstacle direction changes is less than the second threshold, a first score is obtained, otherwise a second score is obtained; if the obstacle speed is less than the third threshold, a third score is obtained, otherwise a fourth score is obtained; if the number of obstacles is less than the fourth threshold, a fifth score is obtained, otherwise a sixth score is obtained.

[0017] Optionally, determining the obstacle avoidance level according to the actual score value and the first threshold includes: determining the size of the actual score value and the first threshold; if the actual score value is less than the first threshold, determining that the obstacle avoidance level is intermediate; if the actual score value is greater than or equal to the first threshold, determining that the obstacle avoidance level is advanced.

[0018] Optionally, predicting the obstacle trajectory according to the second environment data and planning the obstacle avoidance path to avoid the obstacle includes: establishing an environment perception function, obtaining a first environment perception result according to the second environment data and the environment perception function, and ,have

[0019] ,

[0020] In the formula, is the environment perception function, for The second environment data collected at all times; establish a motion prediction function, collect the historical motion data of the obstacle and combine it with the environmental perception result, obtain the first obstacle motion trajectory prediction result through the motion prediction function, and for the first obstacle motion trajectory prediction result ,have

[0021] ,

[0022] In the formula, express The prediction result of the first obstacle motion trajectory at time , represents the motion prediction function, express The obstacle historical motion data at the moment; establish a first path planning function, and obtain a first path planning result according to the obstacle motion trajectory prediction result and the first path planning function, for the first path planning result ,have

[0023] ,

[0024] In the formula, represents the first path planning result, Represents a first path planning function; and plans the robot's motion path according to the first path planning result to achieve obstacle avoidance.

[0025] Optionally, the constructing of a real-time dynamic environment prediction model for the next moment according to the second environment data and the third environment data, and dynamically planning an obstacle avoidance path after the next moment in combination with the real-time dynamic environment prediction model for the next moment to avoid obstacles comprises: calculating the environment perception related parameters for the next moment according to the environment perception function, the second environment data and the third environment data to obtain a second environment perception result ; According to the second environment perception result, the motion prediction function and the first obstacle motion trajectory prediction result, the obstacle motion trajectory after the next moment is calculated to obtain the second obstacle motion trajectory prediction result, and the second obstacle motion trajectory prediction result ,have

[0026] ,

[0027] In the formula, express The second obstacle motion trajectory prediction result at the moment; according to the second environment perception result and the second obstacle motion trajectory prediction result, a real-time dynamic environment prediction model at the next moment is established, and the real-time dynamic environment prediction model at the next moment is ,have

[0028] ,

[0029] In the formula, A function representing the construction of a real-time dynamic environment prediction model for the next moment; obtaining the obstacle position coordinate information according to the third environment data; establishing a second path planning function, and calculating the planned path after the next moment to obtain the second path planning result according to the second path planning function, the obstacle position coordinate information, the real-time dynamic environment model and the second obstacle motion trajectory prediction result. ,have

[0030] ,

[0031] In the formula, is the second path planning function, It is the obstacle position coordinate information.

[0032] Optionally, the step of calculating the environmental perception related parameters at the next moment according to the environmental perception function, the second environmental data and the third environmental data to obtain the second environmental perception result includes: performing target matching and spatiotemporal alignment on the second environmental data and the third environmental data to obtain fourth environmental data; establishing a prediction model for describing the change of the fourth environmental data over time; obtaining a predicted value of the fourth environmental data at the current moment by calculating the prediction model based on the fourth environmental data at the previous moment; training the prediction model based on the fourth environmental data at the current moment and the predicted value to obtain a stable prediction model; calculating the fourth environmental data at the next moment according to the prediction of the stable prediction model and the fourth environmental data to obtain fifth environmental data; calculating the environmental perception related parameters at the next moment according to the environmental perception function and the fifth environmental data to obtain the second environmental perception result, and for the second environmental perception result ,have;

[0033] ,

[0034] In the formula, It is the fifth environment data.

[0035] Based on the same inventive concept, the present invention also provides a mobile charging robot obstacle avoidance system based on multi-sensor fusion, the system comprising: an environmental data acquisition module, used to collect surrounding environmental information in real time to obtain first environmental data; an obstacle avoidance level decision module, used to judge the obstacle avoidance level according to the first environmental data; an obstacle avoidance strategy execution module, used to detect surrounding obstacles through a radio wave transmission module if the obstacle avoidance level is primary; the obstacle avoidance strategy execution module is also used to collect second environmental data through multiple sensors if the obstacle avoidance level is intermediate, and predict the obstacle movement trajectory according to the second environmental data, and plan the obstacle avoidance path; the obstacle avoidance strategy execution module is also used to obtain third environmental data through real-time scanning by a drone if the obstacle avoidance level is advanced, and construct a real-time dynamic environmental prediction model according to the second environmental data and the third environmental data, and dynamically plan the obstacle avoidance path in combination with the real-time dynamic environmental prediction model; a motion control module, used to control the movement of the robot.

[0036] Compared with the prior art, the present invention has the following advantages:

[0037] 1. The present invention collects surrounding environment information in real time and determines the obstacle avoidance level according to the complexity of the information and the dynamics of the obstacles. The present invention can adopt obstacle avoidance strategies with different precisions. In the primary obstacle avoidance stage, it transmits radio waves for rough detection. In the intermediate stage, it uses multi-sensor data to predict the movement trajectory of obstacles to achieve more accurate path planning. In the advanced stage, it combines drone scanning data to build a real-time dynamic environment prediction model, which further improves the accuracy and efficiency of obstacle avoidance.

[0038] 2. The method of the present invention can adapt to environments of different complexity. In an environment with fewer obstacles or slower movement speed, the robot can adopt a simpler obstacle avoidance strategy to save computing resources. In a scene with dense obstacles or faster movement speed, the robot can quickly switch to a more advanced obstacle avoidance mode to ensure safe passage.

[0039] 3. The multi-sensor fusion obstacle avoidance method of the present invention predicts the motion state and surrounding environment state of the current moment in advance, and predicts the motion state of the obstacle and the surrounding environment information at the next moment of the current moment, so that the robot can avoid obstacles more intelligently, reduce collision and stagnation time, and achieve efficient obstacle avoidance in complex environments;

[0040] 4. The present invention adopts multi-sensor fusion, which can make full use of the complementary advantages of different sensors and improve the system's perception and adaptability to environmental changes; even if some sensors fail or data is abnormal, the system can still rely on the information provided by other sensors to perform reliable obstacle avoidance operations.

[0041] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 It is a flow chart of a mobile charging robot obstacle avoidance method based on multi-sensor fusion according to an embodiment of the present invention.

[0044] Figure 2 It is an execution flow chart of a mobile charging robot obstacle avoidance method based on multi-sensor fusion according to an embodiment of the present invention.

[0045] Figure 3 It is a structural schematic diagram of a mobile charging robot obstacle avoidance system based on multi-sensor fusion according to an embodiment of the present invention. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0047] Reference Figure 1 An embodiment of the present invention proposes a mobile charging robot obstacle avoidance method based on multi-sensor fusion, which adopts a strategy of real-time collection of environmental information and judgment of obstacle avoidance levels, and can take obstacle avoidance measures of different precisions according to different levels, thereby improving the accuracy and efficiency of obstacle avoidance.

[0048] The method of this embodiment specifically includes:

[0049] Collect surrounding environment information in real time to obtain first environment data;

[0050] determining an obstacle avoidance level according to the first environmental data;

[0051] If the obstacle avoidance level is primary, it will detect surrounding obstacles by transmitting radio waves and perform rough obstacle avoidance;

[0052] If the obstacle avoidance level is intermediate, the second environment data is collected through multiple sensors, and the movement trajectory of the obstacle is predicted based on the second environment data, and the obstacle avoidance path is planned to avoid the obstacle;

[0053] If the obstacle avoidance level is high, the third environmental data is obtained by real-time scanning by the drone, and a real-time dynamic environmental prediction model for the next moment is constructed based on the second environmental data and the third environmental data. Combined with the real-time dynamic environmental prediction model for the next moment, the obstacle avoidance path after the next moment is dynamically planned to avoid obstacles.

[0054] For example, Figure 2 As shown in the figure, a large shopping mall has introduced a mobile charging robot to improve the customer experience, so that customers can charge whenever they need it. However, the internal environment of the shopping mall is complex, with many pedestrians, product display racks, elevators and other obstacles, so the mobile charging robot needs to have efficient obstacle avoidance capabilities. When the mobile charging robot is running, it collects environmental information in the shopping mall in real time through its various sensors such as laser radar, camera, ultrasonic sensor, etc., including the location, shape, size and dynamic information of obstacles (such as the movement speed of pedestrians), which are integrated into the first environmental data. According to the first environmental data, the obstacle avoidance difficulty faced by the robot is evaluated according to the algorithm. For example, if it is detected that there are few obstacles around and the moving speed is slow, the robot will determine the obstacle avoidance level as primary; if there are many obstacles but the movement pattern can be followed, such as fixed product display racks and slowly moving pedestrians, the robot will determine the obstacle avoidance level as intermediate; if the environment is extremely complex, such as a large number of fast-moving pedestrians, sudden temporary obstacles, etc., the robot will determine the obstacle avoidance level as advanced.

[0055] Exemplarily, when the obstacle avoidance level is primary, the mobile charging robot will use a simple radio wave detection method, such as infrared or ultrasonic waves, to detect surrounding obstacles; once an obstacle is detected, the robot will immediately adjust its driving direction to roughly avoid the obstacle; the radio wave transmitting device can be an improved transmitting end of the robot's wireless charging device; when the obstacle avoidance level is intermediate, the robot will start the multi-sensor fusion system, and use the second environment data collected by sensors such as lidar and cameras to predict the movement trajectory of the obstacle; for example, the robot can predict the future movement path of pedestrians by analyzing their walking speed and direction, and plan the optimal obstacle avoidance path accordingly; when the obstacle avoidance level is advanced, the mobile charging robot will start the drone it carries for auxiliary scanning; the drone can be equipped with high-resolution cameras and sensors, and can scan the environment inside the shopping mall in real time, and transmit the collected third environment data back to the robot; the robot combines the second environment data and the third environment data to build a real-time dynamic environment prediction model; this model can update the predicted environment information in real time, including the position, speed and acceleration of the obstacle, so as to help the robot dynamically plan the optimal obstacle avoidance path.

[0056] Optionally, determining an obstacle avoidance level according to the first environmental data includes:

[0057] Determine whether there are obstacles around according to the first environmental data;

[0058] If there are no obstacles around, the obstacle avoidance function will not be activated;

[0059] If there are obstacles around, the obstacle avoidance level is determined.

[0060] Exemplarily, the mobile charging robot is equipped with a variety of sensors, such as cameras, lidars and millimeter-wave radars, which can collect surrounding environmental information in real time, including the number, shape, distance and speed of objects, etc. This information is summarized into the first environmental data; based on the first environmental data, the robot first determines whether there are obvious obstacles around; if the environment is open and there are no objects hindering its progress, the robot will not activate the obstacle avoidance function, but continue to drive along the preset path; if obstacles are detected around, the robot will further determine the obstacle avoidance level based on factors such as the type, number, distance and moving speed of the obstacles.

[0061] Optionally, if there are obstacles around, determining the obstacle avoidance level further includes:

[0062] Determining a state of an obstacle according to the first environmental data;

[0063] If the obstacle is static, the obstacle avoidance level is determined to be primary;

[0064] If the obstacle is dynamic, the obstacle avoidance level is determined based on the number, quantity, and speed of the obstacle's direction changes.

[0065] Exemplarily, when the mobile charging robot is moving, its sensor network starts working to collect real-time environmental information in the shopping mall, such as the location, shape, size and dynamic information of obstacles (such as the movement speed of pedestrians). This information is integrated into the first environmental data to provide a basis for subsequent obstacle avoidance decisions. The robot first determines whether there are obstacles around based on the first environmental data. For example, the robot detects a static merchandise display rack in front of it. Due to the existence of the obstacle, the robot continues to judge the state of the obstacle. The static merchandise display rack is determined to be a static obstacle. According to the setting, the obstacle avoidance level corresponding to the static obstacle is primary. If the robot detects a moving pedestrian in front of it, the robot determines that it is a dynamic obstacle and needs further analysis. The robot needs to evaluate the number, quantity and speed of the direction changes of the dynamic obstacle (pedestrian) to determine the obstacle avoidance level.

[0066] Optionally, judging the obstacle avoidance level according to the number, quantity and speed of direction changes of the obstacle includes:

[0067] Set the first weight factor for the number of direction changes of the obstacle , set the second weight factor for obstacle speed ;

[0068] Specifically, the first weight factor and the second weighting factor It reflects the importance of the number of direction changes and speed in judging the obstacle avoidance level.

[0069] Scoring the number, quantity and speed of obstacles’ direction changes, and constructing a scoring matrix for the number and speed of obstacles’ direction changes , where each row represents an obstacle and each column represents an evaluation condition (including the number of direction changes and speed). The scoring matrix The structure is:

[0070] ,

[0071] In the formula, For the The score of the number of direction changes of each obstacle; For the The speed rating of each obstacle;

[0072] Specifically, the number of direction changes and speed of each obstacle are scored, and a scoring matrix is ​​constructed , the scoring matrix Each row represents an obstacle, and each column represents an evaluation condition (including the number of direction changes and speed ).

[0073] Calculate the weighted score of the number of direction changes and the speed of each obstacle, and add the weighted score of the number of direction changes and the speed of each obstacle to get the weighted total score of each obstacle. For any weighted total score of an obstacle ,have

[0074] ;

[0075] Add the weighted total score of all obstacles to the score of the number of obstacles to get the actual score , for the actual score value have

[0076] ,

[0077] In the formula, is the number of obstacles; A scoring value for the number of obstacles;

[0078] An obstacle avoidance level is determined according to the actual score value and the first threshold.

[0079] Optionally, scoring the number, quantity and speed of direction changes of obstacles includes:

[0080] Setting a second threshold for the number of obstacle direction changes, setting a third threshold for the obstacle speed, and setting a fourth threshold for the number of obstacles;

[0081] If the number of obstacle direction changes is less than the second threshold, a first score is obtained, otherwise a second score is obtained;

[0082] If the obstacle speed is less than the third threshold, a third score is obtained, otherwise a fourth score is obtained;

[0083] If the number of obstacles is less than the fourth threshold, a fifth score is obtained, otherwise a sixth score is obtained.

[0084] Optionally, determining the obstacle avoidance level according to the actual score value and the first threshold value includes:

[0085] Determine the difference between the actual score value and the first threshold value;

[0086] If the actual score value is less than the first threshold, the obstacle avoidance level is determined to be intermediate;

[0087] If the actual score value is greater than or equal to the first threshold, the obstacle avoidance level is determined to be high.

[0088] Specifically, according to the actual score and the preset first threshold value, determine the obstacle avoidance level; if is less than or equal to the first threshold, the obstacle avoidance level is determined to be primary; if If the value is greater than or equal to the first threshold, the obstacle avoidance level is determined to be high.

[0089] Exemplarily, a first weight factor is set for the number of direction changes of the obstacle , set the second weight factor for obstacle speed , according to the actual situation, set scoring standards for the number of direction changes and speed of the obstacle; for example, the fewer the number of direction changes, the lower the score; the slower the speed, the lower the score; for example, the robot detects 3 obstacles (pedestrians) at a certain point in time, which are recorded as obstacle 1, obstacle 2 and obstacle 3 respectively. According to the scoring standard, the first threshold is set to 7, the second threshold is set to 1, the third threshold is set to 2, and the fourth threshold is set to 4. The first score, the third score, and the fourth score are set to 1, and the second score, the fourth score, and the sixth score are set to 2; the number of direction changes of obstacle 1 is 0 times and the speed is 3, the number of direction changes of obstacle 2 is 0 times and the speed is 1, and the number of direction changes of obstacle 3 is 0 times and the speed is 4. Score the number of direction changes and speed of each obstacle to get the score of obstacle 1. , obstacle 2 , obstacle 3 , construct the scoring matrix , calculate the weighted score of the number of direction changes and the speed weighted score of each obstacle. The weighted score of obstacle 1 is , the weighted score of obstacle 2 is , the weighted score of obstacle 3 is , thereby calculating the actual score value ; The actual score value at this time If it is less than the first threshold value 7, the obstacle avoidance level is determined to be medium.

[0090] Optionally, predicting the obstacle movement trajectory according to the second environment data and planning an obstacle avoidance path to avoid the obstacle includes:

[0091] Establish an environment perception function, obtain a first environment perception result according to the second environment data and the environment perception function, and for the first environment perception result ,have

[0092] ,

[0093] In the formula, is the environment perception function, for Second environment data collected at all times;

[0094] For example, first, define an environment perception function , this function can receive Second environment data collected at all times , and output the first environmental perception result ;Environmental perception function By processing and analyzing the second environment data, information related to the obstacle position, speed, acceleration, etc. is extracted to form a real-time perception of the environment.

[0095] Establish a motion prediction function, collect historical obstacle motion data and combine it with the environmental perception result, obtain the first obstacle motion trajectory prediction result through the motion prediction function, and ,have

[0096] ,

[0097] In the formula, express The prediction result of the first obstacle motion trajectory at time , represents the motion prediction function, express Historical obstacle movement data at all times;

[0098] For example, before predicting the obstacle's motion trajectory, it is necessary to collect the obstacle's historical motion data. ; These data include the position, speed, acceleration and other information of the obstacle at the past moment; The method of collecting historical motion data can be achieved by continuously recording and analyzing sensor data; Define a motion prediction function , this function can receive the environmental perception results and obstacle historical motion data , and output The prediction result of the first obstacle motion trajectory at time ; Motion prediction function By comprehensively considering the environmental perception results and the historical motion data of obstacles, mathematical models or machine learning algorithms are used to predict the movement trajectory of obstacles in the future.

[0099] Establish a first path planning function, and obtain a first path planning result according to the obstacle motion trajectory prediction result and the first path planning function. ,have

[0100] ,

[0101] In the formula, represents the first path planning result, represents the first path planning function;

[0102] Exemplarily, a first path planning function is defined , which can receive the motion prediction function The output of , and output the first path planning result ; Path planning function An algorithm is used to find the optimal path from the current position to the target position while avoiding collisions with predicted obstacle trajectories.

[0103] The robot's motion path is planned according to the first path planning result to achieve obstacle avoidance.

[0104] For example, after obtaining the first path planning result After that, the mobile charging robot will adjust its own movement path according to the result; the robot will drive along the planned path while continuously monitoring changes in the surrounding environment so as to re-plan the path when necessary.

[0105] For example, in the obstacle avoidance system of the mobile charging robot, when the obstacle avoidance level is judged to be intermediate, the robot will start the multi-sensor fusion system, use the second environment data collected in real time by sensors such as lidar and cameras to predict the movement trajectory of the obstacle, and plan the optimal obstacle avoidance path accordingly.

[0106] Optionally, constructing a real-time dynamic environment prediction model for the next moment according to the second environment data and the third environment data, and dynamically planning an obstacle avoidance path after the next moment to avoid obstacles in combination with the real-time dynamic environment prediction model for the next moment includes:

[0107] According to the environmental perception function, the second environmental data and the third environmental data, the environmental perception related parameters of the next moment are calculated to obtain the second environmental perception result ;

[0108] According to the second environment perception result, the motion prediction function and the first obstacle motion trajectory prediction result, the obstacle motion trajectory after the next moment is calculated to obtain a second obstacle motion trajectory prediction result. ,have

[0109] ,

[0110] In the formula, express The prediction result of the motion trajectory of the second obstacle at time t;

[0111] According to the second environment perception result and the second obstacle motion trajectory prediction result, a real-time dynamic environment prediction model for the next moment is established. ,have

[0112] ,

[0113] In the formula, A function representing the construction of a real-time dynamic environment prediction model for the next moment;

[0114] Obtaining obstacle position coordinate information according to the third environment data;

[0115] Establish a second path planning function, calculate the planned path after the next moment according to the second path planning function, the obstacle position coordinate information, the real-time dynamic environment model and the second obstacle motion trajectory prediction result, and obtain the second path planning result. ,have

[0116] ,

[0117] In the formula, is the second path planning function, It is the obstacle position coordinate information.

[0118] Exemplarily, the environment state information at the next moment is predicted according to the environment perception function, the second environment data and the third environment data to obtain the second environment perception result , the robot uses the second environment to perceive the results And the previous obstacle trajectory prediction results As input, through the motion prediction function To calculate the obstacle trajectory prediction result after the next moment ; The robot perceives the second environment And obstacle trajectory prediction results As input, a function is constructed to predict the real-time dynamic environment model at the next moment. To build a real-time dynamic environment prediction model for the next moment ; The robot obtains the location coordinate information of the obstacle from the third environment data , this information is crucial for path planning. The robot establishes a second path planning function , and based on the real-time dynamic environment prediction model , obstacle location coordinate information , and obstacle trajectory prediction results ; To calculate the second path planning result ; This path planning result is the obstacle avoidance path that the robot should follow after the next moment.

[0119] Optionally, the calculating, according to the environment perception function, the second environment data and the third environment data, the environment perception related parameters at the next moment to obtain the second environment perception result includes:

[0120] Performing target matching and spatiotemporal alignment on the second environment data and the third environment data to obtain fourth environment data;

[0121] Exemplarily, the robot first performs target matching and spatiotemporal alignment on the second environment data it collects (such as the location and speed of obstacles) and the known third environment data (such as the location of obstacles on the map, road information, etc.) to ensure the consistency and accuracy of the data, thereby obtaining the fourth environment data.

[0122] Establishing a prediction model for describing the change of the fourth environmental data over time;

[0123] Exemplarily, the robot establishes a prediction model for describing the change of the fourth environment data over time; this model can be based on machine learning, such as a time series analysis model, a neural network model, etc.

[0124] According to the fourth environmental data at a previous moment, a predicted value of the fourth environmental data at a current moment is calculated by the prediction model;

[0125] Training the prediction model according to the fourth environmental data at the current moment and the prediction value to obtain a stable prediction model;

[0126] Calculate the fourth environmental data at the next moment according to the stable prediction model prediction and the fourth environmental data to obtain fifth environmental data;

[0127] Exemplarily, the robot uses the fourth environmental data of the previous moment to calculate the predicted value of the current moment through the prediction model, and compares it with the actually collected fourth environmental data of the current moment to train and optimize the prediction model; once the prediction model is stabilized, the robot can use it to predict the fourth environmental data of the next moment, which we call the fifth environmental data.

[0128] According to the environmental perception function and the fifth environmental data, the environmental perception related parameters at the next moment are calculated to obtain the second environmental perception result. ,have;

[0129] ,

[0130] In the formula, It is the fifth environment data.

[0131] For example, the robot perceives the environment according to the function and the fifth environmental data to calculate the environmental perception related parameters at the next moment, and obtain the second environmental perception result = ; By using the environmental information data of the next moment, the motion trajectory after the next moment can be predicted, and advance predictive control can be achieved, thereby improving the efficiency of the robot's obstacle avoidance.

[0132] For example, Figure 2 As shown, the robot collects information about the surrounding environment in real time through its equipped sensors (such as infrared sensors, ultrasonic sensors, laser radars, etc.), including the location, size, shape and motion state of obstacles. If the environmental information collected by the robot indicates that there is no obstacle ahead, or the obstacle is far away and will not affect the robot's driving, then the robot will not activate the obstacle avoidance function, but continue to drive along the preset path. The robot determines whether to activate the obstacle avoidance function by judging whether there is an obstacle in the collected environmental information. If an obstacle is detected, the robot also needs to determine whether the obstacle is static or dynamic. For static obstacles, it means that the obstacle has little impact on the robot's driving, and the robot will perform primary obstacle avoidance operations. Using a simple radio wave detection method, the robot can plan a detour path. For dynamic obstacles, the robot needs to plan an obstacle avoidance path based on the motion state of the obstacle. The robot calculates an actual score value based on factors such as the type, size, distance and driving speed of the obstacle. This score value is used to measure the degree of impact of the obstacle on the robot's driving. The robot compares the actual score value with the preset first threshold value. If the actual score value is less than the first threshold value, it means that the obstacle has a greater impact on the robot's driving and needs to enter the middle In the intermediate obstacle avoidance stage, the robot will activate the multi-sensor fusion system, use the second environment data collected in real time by sensors such as lidar and cameras to predict the movement trajectory of the obstacle, and plan the optimal obstacle avoidance path accordingly; if the actual score value is greater than or equal to the preset first threshold, it means that the obstacle has a great impact on the robot's driving, and the robot will enter the advanced obstacle avoidance stage; in the advanced obstacle avoidance stage, the robot uses the drone it carries for auxiliary scanning, combined with multiple sensors, and collects data to predict the environmental information data at the next moment, thereby predicting the movement trajectory after the next moment, realizing advance predictive control, and improving the efficiency of the robot's obstacle avoidance.

[0133] Based on the same inventive concept, Figure 3 As shown, the present invention also provides a mobile charging robot obstacle avoidance system based on multi-sensor fusion, the system comprising:

[0134] An environmental data collection module is used to collect surrounding environmental information in real time to obtain first environmental data;

[0135] an obstacle avoidance level decision module, configured to determine an obstacle avoidance level according to the first environmental data;

[0136] The obstacle avoidance strategy execution module is used to detect surrounding obstacles through the radio wave transmission module when the obstacle avoidance level is primary; the obstacle avoidance strategy execution module is also used to collect second environmental data through multiple sensors when the obstacle avoidance level is intermediate, and predict the obstacle movement trajectory according to the second environmental data to plan the obstacle avoidance path; the obstacle avoidance strategy execution module is also used to obtain third environmental data through real-time scanning by the drone when the obstacle avoidance level is advanced, and construct a real-time dynamic environmental prediction model according to the second environmental data and the third environmental data, and dynamically plan the obstacle avoidance path in combination with the real-time dynamic environmental prediction model;

[0137] Motion control module, used to control the movement of the robot.

[0138] It should be noted that the electrical connection between the above-mentioned units does not necessarily mean the connection between the lines. The indirect connection method can be applied to the embodiments of the present invention as long as the purpose of the present invention is achieved. The above is only an exemplary embodiment of the present invention and cannot be used to limit the scope of the present invention.

[0139] That is, any equivalent changes and modifications made according to the teachings of the present invention are still within the scope of the present invention. After considering the disclosure of the specification and the truth of practice, those skilled in the art will easily think of other embodiments of the present invention. This application is intended to cover any variation, use or adaptive change of the present invention, which follows the general principles of the present invention and includes common knowledge or customary technical means in the art that are not described in the present invention.

Claims

1. A mobile charging robot obstacle avoidance method based on multi-sensor fusion, characterized in that: The method comprises: Collect surrounding environment information in real time to obtain first environment data; determining an obstacle avoidance level according to the first environmental data; If the obstacle avoidance level is primary, it will detect surrounding obstacles by transmitting radio waves and perform rough obstacle avoidance; If the obstacle avoidance level is intermediate, the second environment data is collected through multiple sensors, and the movement trajectory of the obstacle is predicted based on the second environment data, and the obstacle avoidance path is planned to avoid the obstacle; If the obstacle avoidance level is high, the third environment data is obtained by real-time scanning by the drone, and a real-time dynamic environment prediction model for the next moment is constructed according to the second environment data and the third environment data. In combination with the real-time dynamic environment prediction model for the next moment, an obstacle avoidance path after the next moment is dynamically planned to avoid obstacles; Wherein, determining the obstacle avoidance level according to the first environmental data includes: Determine whether there are obstacles around according to the first environmental data; If there are no obstacles around, the obstacle avoidance function will not be activated; If there are obstacles around, determine the state of the obstacles according to the first environmental data; If the obstacle is static, the obstacle avoidance level is determined to be primary; If the obstacle is dynamic, the obstacle avoidance level is determined based on the number, quantity, and speed of the obstacle’s direction changes; Set the first weight factor for the number of direction changes of the obstacle , set the second weight factor for obstacle speed ; Scoring the number, quantity and speed of obstacles’ direction changes, and constructing a scoring matrix for the number and speed of obstacles’ direction changes , where each row represents an obstacle and each column represents an evaluation condition. The scoring matrix The structure is: ， In the formula, For the The score of the number of direction changes of each obstacle; For the The speed rating of each obstacle; Calculate the weighted score of the number of direction changes and the speed of each obstacle, and add the weighted score of the number of direction changes and the speed of each obstacle to get the weighted total score of each obstacle. For any weighted total score of an obstacle ,have ； Add the weighted total score of all obstacles to the score of the number of obstacles to get the actual score , for the actual score value have ， In the formula, is the number of obstacles; A scoring value for the number of obstacles; An obstacle avoidance level is determined according to the actual score value and the first threshold.

2. According to claim 1, a mobile charging robot obstacle avoidance method based on multi-sensor fusion is characterized in that: Scoring the number, quantity and speed of direction changes of obstacles includes: Setting a second threshold for the number of obstacle direction changes, setting a third threshold for the obstacle speed, and setting a fourth threshold for the number of obstacles; If the number of obstacle direction changes is less than the second threshold, a first score is obtained, otherwise a second score is obtained; If the obstacle speed is less than the third threshold, a third score is obtained, otherwise a fourth score is obtained; If the number of obstacles is less than the fourth threshold, a fifth score is obtained, otherwise a sixth score is obtained.

3. The obstacle avoidance method for a mobile charging robot based on multi-sensor fusion according to claim 2 is characterized in that: The determining the obstacle avoidance level according to the actual score value and the first threshold value comprises: Determine the difference between the actual score value and the first threshold value; If the actual score value is less than the first threshold, the obstacle avoidance level is determined to be intermediate; If the actual score value is greater than or equal to the first threshold, the obstacle avoidance level is determined to be high.

4. The obstacle avoidance method for a mobile charging robot based on multi-sensor fusion according to claim 1 is characterized in that: Predicting the obstacle movement trajectory according to the second environment data and planning an obstacle avoidance path to avoid the obstacle includes: Establish an environment perception function, obtain a first environment perception result according to the second environment data and the environment perception function, and ,have ， In the formula, is the environment perception function, for Second environment data collected at all times; Establish a motion prediction function, collect historical obstacle motion data and combine it with the environmental perception result, obtain the first obstacle motion trajectory prediction result through the motion prediction function, and ,have ， In the formula, express The prediction result of the first obstacle motion trajectory at time , represents the motion prediction function, express Historical obstacle movement data at all times; Establish a first path planning function, and obtain a first path planning result according to the obstacle motion trajectory prediction result and the first path planning function. ,have , In the formula, represents the first path planning result, represents the first path planning function; The robot's motion path is planned according to the first path planning result to achieve obstacle avoidance.

5. The obstacle avoidance method for a mobile charging robot based on multi-sensor fusion according to claim 4 is characterized in that: The step of constructing a real-time dynamic environment prediction model for the next moment according to the second environment data and the third environment data, and dynamically planning an obstacle avoidance path after the next moment in combination with the real-time dynamic environment prediction model for the next moment to avoid obstacles comprises: According to the environmental perception function, the second environmental data and the third environmental data, the environmental perception related parameters of the next moment are calculated to obtain the second environmental perception result ; According to the second environment perception result, the motion prediction function and the first obstacle motion trajectory prediction result, the obstacle motion trajectory after the next moment is calculated to obtain a second obstacle motion trajectory prediction result. ,have ， In the formula, express The prediction result of the motion trajectory of the second obstacle at time t; According to the second environment perception result and the second obstacle motion trajectory prediction result, a real-time dynamic environment prediction model for the next moment is established. ,have ， In the formula, A function representing the construction of a real-time dynamic environment prediction model for the next moment; Obtaining obstacle position coordinate information according to the third environment data; Establish a second path planning function, calculate the planned path after the next moment according to the second path planning function, the obstacle position coordinate information, the real-time dynamic environment prediction model and the second obstacle motion trajectory prediction result, and obtain the second path planning result. ,have , In the formula, is the second path planning function, It is the obstacle position coordinate information.

6. The obstacle avoidance method for a mobile charging robot based on multi-sensor fusion according to claim 5 is characterized in that: The calculating, according to the environment perception function, the second environment data and the third environment data, the environment perception related parameters at the next moment to obtain the second environment perception result comprises: Performing target matching and spatiotemporal alignment on the second environment data and the third environment data to obtain fourth environment data; Establishing a prediction model for describing the change of the fourth environmental data over time; According to the fourth environmental data at a previous moment, a predicted value of the fourth environmental data at a current moment is calculated by the prediction model; Training the prediction model according to the fourth environmental data at the current moment and the prediction value to obtain a stable prediction model; Calculate the fourth environmental data at the next moment according to the stable prediction model prediction and the fourth environmental data to obtain fifth environmental data; According to the environmental perception function and the fifth environmental data, the environmental perception related parameters at the next moment are calculated to obtain the second environmental perception result. ,have; ， In the formula, It is the fifth environment data.

7. A mobile charging robot obstacle avoidance system based on multi-sensor fusion, applied to the obstacle avoidance method according to any one of claims 1 to 6, characterized in that: The system comprises: An environmental data collection module is used to collect surrounding environmental information in real time to obtain first environmental data; an obstacle avoidance level decision module, configured to determine an obstacle avoidance level according to the first environmental data; The obstacle avoidance level decision module is further used to determine whether there are obstacles around according to the first environmental data. If there are no obstacles around, the obstacle avoidance function is not activated. If there are obstacles around, the state of the obstacle is determined according to the first environmental data. If the obstacle is in a static state, the obstacle avoidance level is determined to be primary. The obstacle avoidance level decision module is also used to calculate the actual score value according to the number of direction changes, quantity, and speed of the obstacle if the obstacle is in a dynamic state, and determine whether the obstacle avoidance level is intermediate or advanced according to the actual score value and the size of the first threshold; The obstacle avoidance strategy execution module is used to detect surrounding obstacles through the radio wave transmission module when the obstacle avoidance level is primary; the obstacle avoidance strategy execution module is also used to collect second environmental data through multiple sensors when the obstacle avoidance level is intermediate, and predict the obstacle movement trajectory according to the second environmental data to plan the obstacle avoidance path; the obstacle avoidance strategy execution module is also used to obtain third environmental data through real-time scanning by the drone when the obstacle avoidance level is advanced, and construct a real-time dynamic environmental prediction model according to the second environmental data and the third environmental data, and dynamically plan the obstacle avoidance path in combination with the real-time dynamic environmental prediction model; The motion control module is used to control the movement of the robot according to the obstacle avoidance level.

Images