Dynamic path planning method and system based on multi-modal environment perception

Abstract

The application discloses a dynamic path planning method and system based on multi-modal environment perception, and belongs to the technical field of intelligent mobile robots, and comprises the following steps: S1: obtaining expected parameters based on first multi-modal environment perception data; S2: obtaining a safety coefficient by quantifying the safety of path planning using the expected parameters based on a correlation function representing the correlation between the expected parameters and the working equipment state; S3: obtaining planning parameters based on the safety coefficient and the expected parameters, and obtaining a planning path using the planning parameters; and S4: during the working process based on the planning path, if second multi-modal environment perception data does not satisfy a preset condition, the second multi-modal environment perception data is taken as the first multi-modal perception data, and S1 is executed, otherwise, it is judged whether the working equipment state changes, if the working equipment state changes, the planning parameters are taken as the expected parameters, and S2 is executed. The technical problem that the prior art is difficult to improve the coverage rate while having high safety is overcome.

Description

Technical Field

[0001] This invention relates to the field of intelligent mobile robot technology, specifically to a dynamic path planning method and system based on multimodal environment perception. Background Technology

[0002] With the rapid development of intelligent technology, the application scenarios of outdoor automated equipment are constantly expanding, and the types of equipment are becoming increasingly diversified. However, facing complex and ever-changing outdoor working environments, the working efficiency of these devices faces severe challenges. Existing technologies mainly improve the working efficiency by planning the movement paths of outdoor automated equipment to adapt to the operational needs of corresponding environments. For example, a walking path planning method (patent number CN117472045A) includes: acquiring feature information of the target working area, including at least slope information; and planning the walking path of the intelligent device within the target working area based on the slope information. This method can not only solve the coverage planning problem in a single slope scenario, but also achieve reasonable coverage planning for multiple slopes with different orientations, thereby improving the working efficiency and mowing effect of the lawnmower. However, the above methods are only applicable to static environments. When the environment changes, these solutions become difficult to use, especially in snowy or rainy conditions where the coefficient of friction is low. When outdoor automated equipment operates according to the planned path, there is a risk of slippage, leading to insufficient coverage. Existing technologies mainly improve coverage through fixed rules (such as bow-shaped coverage), but this can easily result in a large number of repetitive paths in complex scenarios, significantly increasing energy consumption. Especially when the equipment is low on power, the lack of a dynamic adjustment mechanism may lead to task interruption due to unreasonable path planning, affecting not only work efficiency but also causing operational safety issues due to the outdoor automated equipment running out of power. Therefore, how to improve coverage while maintaining high safety is a currently difficult technical problem to solve. Summary of the Invention

[0003] To address the technical challenge of achieving both high coverage and robust security in existing technologies, this invention provides a dynamic path planning method and system based on multimodal environment perception. By establishing a correlation function representing the relationship between desired parameters and the status of working equipment, the safety of path planning using desired parameters is quantified to obtain a safety coefficient. This achieves quantification and dynamic evaluation of safety. Furthermore, path planning using the safety coefficient and desired parameters significantly improves safety. Moreover, by judging the second multimodal environment perception data and the status of working equipment, and executing corresponding steps based on the judgment results, the path can be updated according to the multimodal data representing the complex environment and the status of working equipment, achieving dynamic path planning and thus significantly improving coverage. This overcomes the technical problem of existing technologies failing to achieve both high coverage and robust security.

[0004] To address the aforementioned technical problems, this invention provides a dynamic path planning method based on multimodal environment perception, comprising the following steps: S1: Obtain the desired parameters based on the first multimodal environment perception data; S2: Based on the established correlation function representing the correlation between expected parameters and working equipment status, the safety of using expected parameters for path planning is quantified to obtain the safety coefficient; S3: Obtain planning parameters based on safety factor and expected parameters, and use planning parameters to obtain planning path; S4: During the operation based on the planned path, if the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data, and S1 is executed. Otherwise, it is determined whether the status of the working equipment has changed. If it has changed, the planning parameters are used as the expected parameters, and S2 is executed.

[0005] By adopting the above technical solution, the present invention has the following advantages: By establishing a correlation function representing the correlation between expected parameters and working equipment status, the safety of path planning using expected parameters is quantified to obtain a safety coefficient, thus realizing the quantification and dynamic evaluation of safety. Furthermore, path planning using the safety coefficient and expected parameters significantly improves safety. Moreover, by judging the second multimodal environmental perception data and working equipment status, and executing corresponding steps based on the judgment results, the path can be updated based on multimodal data representing complex environments and working equipment status, realizing dynamic path planning. This significantly improves coverage while further enhancing safety, overcoming the technical problem of existing technologies that struggle to achieve high safety while improving coverage. By acquiring desired parameters through first-mode multimodal environmental perception data, the limitations of a single sensor in a specific scenario are overcome, thereby improving the accuracy of the desired parameters; By using the accurate expected parameters obtained from the first multimodal environmental perception data, a safety coefficient is obtained by using a relevant function to quantify the safety of path planning using the accurate expected parameters. This also effectively solves the technical problem of the difficulty in quantifying the impact of the first multimodal environmental perception data on driving stability, thereby improving the driving stability of the working equipment and further enhancing the safety of the work.

[0006] Preferably, S1 includes: S11: Based on the characteristics of historical multimodal environmental perception data, the historical multimodal environmental perception data is divided into primary category data and secondary category data, and a scene recognition model is constructed based on gradient-driven learning; S12: Train the scene recognition model once using the first-level category data. If the training is successful, execute S13. Otherwise, use the data in the first-level category data that caused the training to fail as the first-level category data and execute S12 again. S13: Perform secondary training on the scene recognition model using secondary category data. If the secondary training is successful, execute S14; otherwise, use the data in the secondary category data that caused the secondary training to fail as secondary category data and execute S13 again. S14: Based on the first multimodal environment perception data, use the scene recognition model to obtain the expected parameters.

[0007] Preferably, the first multimodal environmental perception data in S1 and the second multimodal environmental perception data in S4 are respectively the environmental data and working data of the working equipment in different time periods.

[0008] Preferably, S2 further includes: The weights of the desired parameters are obtained based on the working equipment status, and a correlation function is established based on the weights of the desired parameters to characterize the correlation between the desired parameters and the working equipment status.

[0009] Preferably, the correlation function is: S = k1*μ + k2*sin(θ) + k3*D; where S is the safety factor, μ, θ and D are the expected parameters, and k1, k2 and k3 are the weights of μ, θ and D, respectively.

[0010] Preferably, S3 includes: S31: Obtain the work spacing in the planning parameters based on the safety factor and the work spacing in the expected parameters; S32: Obtain the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters, and use the work spacing in the planning parameters and the yaw angle in the planning parameters to obtain the planned path.

[0011] Preferably, S31 includes: d = n * Q * R; where d is the work spacing in the planning parameters, n represents the weight of the deviation between the actual position and the expected position when using the work spacing in the expected parameters for work, Q is the safety factor, and R is the work spacing in the expected parameters.

[0012] Preferably, in S32, obtaining the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters includes: yaw=α*m*Q*R+yan; where yaw is the yaw angle in the planning parameters, α is the yaw angle in the expected parameters, m represents the weight of the deviation between the actual position and the expected position when using the yaw angle in the expected parameters for operation, and yan represents the yaw angle calculated using GPS positioning and IMU inertial navigation.

[0013] The beneficial effects of this plan are: By establishing a correlation function representing the correlation between expected parameters and working equipment status, the safety of path planning using expected parameters is quantified to obtain a safety coefficient, thus realizing the quantification and dynamic evaluation of safety. Furthermore, path planning using the safety coefficient and expected parameters significantly improves safety. Moreover, by judging the second multimodal environmental perception data and working equipment status, and executing corresponding steps based on the judgment results, the path can be updated based on multimodal data representing complex environments and working equipment status, realizing dynamic path planning. This significantly improves coverage while further enhancing safety, overcoming the technical problem of existing technologies that struggle to achieve high safety while improving coverage. By acquiring desired parameters from first multimodal environmental perception data, the limitations of a single sensor in a specific scenario are overcome, thereby improving the accuracy of the desired parameters; Specifically, by dividing historical multimodal environmental perception data into primary and secondary categories, the scene recognition model is trained once using the primary category data, enabling it to identify grass, snow, and paving stones. After successful training, the scene recognition model is trained a second time using the secondary category data, enabling it to identify grass with dew, grass color, etc. This facilitates the acquisition of expected parameters based on the identified conditions, improves the hierarchical accuracy of scene recognition, further enhances the accuracy of expected parameters, and also improves the training efficiency of the model. By obtaining accurate expected parameters from the first multimodal environmental perception data, and using relevant functions to quantify the safety of path planning using the expected parameters to obtain a safety coefficient, this method effectively solves the technical problem of difficulty in quantifying the impact of the first multimodal environmental perception data on driving stability, thereby improving the driving stability of the working equipment and further enhancing work safety.

[0014] The present invention also provides a dynamic path planning system based on multimodal environment perception, which is applicable to the dynamic path planning method based on multimodal environment perception, including a desired parameter acquisition module, a safety factor acquisition module, a planned path acquisition module and a dynamic update module; The expected parameter acquisition module is used to acquire expected parameters based on the first multimodal environment perception data; The safety factor acquisition module is used to quantify the safety of path planning using the expected parameters based on the established correlation function representing the correlation between the expected parameters and the working equipment status to obtain the safety factor. The planning path acquisition module is used to obtain planning parameters based on the safety factor and expected parameters, and then use the planning parameters to obtain the planning path. During the operation based on the planned path, the dynamic update module is used to determine whether the second multimodal environment perception data meets the preset conditions. If the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data, which prompts the expected parameter acquisition module to work. If the second multimodal environment perception data meets the preset conditions, it is determined whether the status of the working equipment has changed. If the status of the working equipment has changed, the planned parameters are used as expected parameters, which prompts the safety factor acquisition module to work.

[0015] The beneficial effects of this plan are: By establishing a correlation function representing the correlation between expected parameters and working equipment status, the safety of path planning using expected parameters is quantified to obtain a safety coefficient, thus realizing the quantification and dynamic evaluation of safety. Furthermore, path planning using the safety coefficient and expected parameters significantly improves safety. Moreover, by judging the second multimodal environmental perception data and working equipment status, and executing corresponding steps based on the judgment results, the path can be updated based on multimodal data representing complex environments and working equipment status, realizing dynamic path planning. This significantly improves coverage while further enhancing safety, overcoming the technical problem of existing technologies that struggle to achieve high safety while improving coverage. By acquiring desired parameters through first-mode multimodal environmental perception data, the limitations of a single sensor in a specific scenario are overcome, thereby improving the accuracy of the desired parameters; By using the accurate expected parameters obtained from the first multimodal environmental perception data, a safety coefficient is obtained by using a relevant function to quantify the safety of path planning using the accurate expected parameters. This also effectively solves the technical problem of the difficulty in quantifying the impact of the first multimodal environmental perception data on driving stability, thereby improving the driving stability of the working equipment and further enhancing the safety of the work.

[0016] The present invention also provides a storage medium storing computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the dynamic path planning method based on multimodal environment awareness. Attached Figure Description

[0017] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings.

[0018] Figure 1 This is a flowchart of the dynamic path planning method based on multimodal environment perception according to the present invention; Figure 2 This is a diagram of the first downward trajectory of the working device in the dynamic path planning method based on multimodal environment perception of the present invention. Figure 3 This is the second downward trajectory diagram of the working device in the dynamic path planning method based on multimodal environment perception of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only one preferred embodiment of this invention and are only used to explain this invention. They do not limit the scope of protection of this invention. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0020] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations (or steps) can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the figures; the process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0021] Example 1: like Figure 1 As shown, the dynamic path planning method based on multimodal environment perception includes the following steps: S1: Obtain the desired parameters based on the first multimodal environment perception data.

[0022] S1 includes: S11: Based on the characteristics of historical multimodal environmental perception data, the historical multimodal environmental perception data is divided into primary category data and secondary category data, and a scene recognition model is constructed based on gradient-driven learning; S12: Train the scene recognition model once using the first-level category data. If the training is successful, execute S13. Otherwise, use the data in the first-level category data that caused the training to fail as the first-level category data and execute S12 again. S13: Perform secondary training on the scene recognition model using secondary category data. If the secondary training is successful, execute S14; otherwise, use the data in the secondary category data that caused the secondary training to fail as secondary category data and execute S13 again. S14: Based on the first multimodal environment perception data, use the scene recognition model to obtain the expected parameters.

[0023] In S1, the first multimodal environmental perception data and in S4, the second multimodal environmental perception data are respectively the environmental data and working data of the working equipment in different time periods.

[0024] In this embodiment, the first multimodal environmental perception data consists of the environmental data and operational data of the working device obtained before the path of the working device is optimized. The first-level category data can identify complex scenes such as lawns, snowfields, and the presence of obstacles. Taking lawns as an example, the second-level category data can identify whether the lawn is wet with dew, dry, and its color. In this embodiment, machine learning and gradient descent are combined to obtain gradient-driven learning. Machine learning integrates the first-level category data, thereby improving the accuracy of complex scene recognition. By combining gradient descent, micro-level differences can be identified, such as "wet lawn" or "dry lawn." By combining machine learning and gradient descent, the hierarchical accuracy of scene recognition is improved, and the accuracy of the expected parameters is also enhanced.

[0025] In this embodiment, primary category data is input into the scene recognition model. If a preset number of the first outputs of the scene recognition model match the actual situation corresponding to the primary category data, the first training iteration is considered successful. After successful training, secondary category data is input into the scene recognition model. If a preset number of the second outputs of the scene recognition model match the actual situation corresponding to the secondary category data, the second training iteration is considered successful. The preset number can be flexibly set according to user needs. This mode of performing secondary training only after successful training avoids wasting time and computing resources, thereby improving training efficiency. If either training or training fails, the data from the primary category data that caused the failure of the first training iteration is used as primary category data, and the data from the secondary category data that caused the failure of the second training iteration is used as secondary category data. Corresponding operations are then performed, avoiding the need to re-input all primary or secondary category data into the scene recognition model for training, further avoiding wasting time and computing resources and further improving model training efficiency.

[0026] In this embodiment, first multimodal environmental perception data is acquired through data acquisition devices, including sensors and cameras. The environmental data of the working device can be outdoor data such as lawns or snowfields, and the working data of the working device includes positioning coordinates, altitude, device heading, and inertial navigation attitude. In this embodiment, based on the first multimodal environmental perception data, obtaining desired parameters using a scene recognition model includes: inputting the environmental data of the working device into the scene recognition model to obtain the working condition coefficient μ; fusing the positioning coordinates (i.e., the position posx, poxy, posz under GPS parameters) with the inertial navigation attitude to obtain the slope θ; and obtaining the desired working distance through the width of the main components of the working device. For example, if the working device is a smart lawnmower, the main component is the blade; if the blade width is D1, then D1 > D, where D is the desired working distance. Obtaining desired parameters through the first multimodal environmental perception data overcomes the limitations of a single sensor or camera in a specific scenario, thereby improving the accuracy of the desired parameters. Obtaining the desired working distance through the width of the main components of the working device improves the working coverage of the working device.

[0027] S2: Based on the established correlation function representing the correlation between the expected parameters and the working equipment status, the safety factor is obtained by quantifying the safety of path planning using the expected parameters.

[0028] S2 further includes: The weights of the desired parameters are obtained based on the working equipment status, and a correlation function is established based on the weights of the desired parameters to characterize the correlation between the desired parameters and the working equipment status.

[0029] The relevant function is: S = k1*μ + k2*sin(θ) + k3*D; where S is the safety factor, μ, θ and D are the expected parameters, and k1, k2 and k3 are the weights of μ, θ and D respectively.

[0030] In this embodiment, the weights for obtaining the desired parameters based on the working device status include: when the working device's battery level is greater than 80%, k1, k2, and k3 are 0.2, 0.5, and 0.3 respectively, prioritizing work efficiency; when the working device's battery level is greater than 20% but less than or equal to 80%, it is in a balanced mode; and when the working device's battery level is less than or equal to 20%, energy-saving recharging is initiated. Dynamically adjusting the weights based on the battery level to obtain a safety factor enables the rational and full use of the working device, improving its utilization rate. It also allows for the quantification and dynamic evaluation of safety, facilitating path planning using the safety factor and desired parameters, avoiding task interruptions, and significantly improving safety. By setting weights for the desired parameters, the lack of collaborative decision-making among multiple environmental factors is also compensated for, thereby improving the comprehensiveness and adaptability of the decision-making process.

[0031] S3: Obtain planning parameters based on safety factor and expected parameters, and use planning parameters to obtain planning path.

[0032] S3 includes: S31: Obtain the work spacing in the planning parameters based on the safety factor and the work spacing in the expected parameters; S32: Obtain the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters, and use the work spacing in the planning parameters and the yaw angle in the planning parameters to obtain the planned path.

[0033] S31 includes: d = n * Q * R; where d is the work spacing in the planning parameters, n represents the weight of the deviation between the actual position and the expected position when using the work spacing in the expected parameters for work, Q is the safety factor, and R is the work spacing in the expected parameters.

[0034] In S32, the step of obtaining the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters includes: yaw=α*m*Q*R+yan; where yaw is the yaw angle in the planning parameters, α is the yaw angle in the expected parameters, m represents the weight of the deviation between the actual position and the expected position when using the yaw angle in the expected parameters for operation, and yan represents the yaw angle calculated using GPS positioning and IMU inertial navigation.

[0035] In this embodiment, when the safety factor is greater than 0.5, planning parameters are obtained based on the safety factor and expected parameters, and the planning path is obtained using the planning parameters. When the safety factor is less than or equal to 0.5, the planning path is obtained using the expected parameters. By setting a reasonable safety factor, both the working efficiency and safety of the equipment are improved. The weight of the deviation between the actual position and the expected position when using the yaw angle in the expected parameters can be obtained periodically or at a fixed distance. For working conditions with different friction coefficients, if horizontal operation is planned, the horizontal operation plan adjusts the drive wheel torque distribution strategy based on the longitudinal gravity component (slope θ) and the GPS positioning of the equipment. The force of the lower drive wheel needs to be slightly greater than the upper drive force to form an overall upward drive force, which counteracts the downward component of gravity. After counteracting, the operation can proceed along a horizontal trajectory. From the machine's perspective, this will form a control quantity yaw angle α (horizontal theoretical direction and actual planned trajectory). This also counteracts the vertical control deviation, ensuring the working distance. Figure 2 As shown, Figure 2 The text describes the deviation parameters in the planning operation, where the yaw angle α is the planning direction under the slip mode.

[0036] In this embodiment, obtaining the planned path using the work spacing and yaw angle in the planning parameters includes: when the slope is small, the planned path is obtained only through the yaw angle in the planning parameters; when the slope is large or the descent is severe, the planned path is obtained through both the yaw angle and the work spacing in the planning parameters. Figure 3 As shown, the blue line represents a deviation between the actual travel trajectory and the theoretically planned trajectory (green). If a smaller working spacing is used, although the machine slides down a certain distance due to gravity, the narrow spacing still achieves a high coverage rate. When the user is not concerned about the cutting direction, the dynamic optimization of the working mode on the slope to vertical operation minimizes the impact of gravity on yaw, significantly improving both safety and coverage.

[0037] S4: During the operation based on the planned path, if the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data, and S1 is executed. Otherwise, it is determined whether the status of the working equipment has changed. If it has changed, the planning parameters are used as the expected parameters, and S2 is executed.

[0038] In this embodiment, the second multimodal environmental perception data consists of environmental data of the working equipment and working data of the working equipment acquired during the process of working based on the planned path. The second multimodal environmental perception data is matched with the first multimodal environmental perception data. If the match is successful, it indicates that the preset conditions are met. For example, if the second multimodal environmental perception data and the first multimodal environmental perception data are analyzed separately, and the analysis results are "lawn with dew at 30%" and "lawn with dew at 40%", then according to the matching rules, the match is considered successful. Users can flexibly set the matching rules according to actual needs. Determining whether the working equipment status has changed includes: dividing the working equipment status into three categories, including the working equipment's battery level being greater than 80%, the working equipment's battery level being greater than 20% and less than or equal to 80%, and the working equipment's battery level being less than or equal to 20%. If the battery level before and after the change is in the same category, it indicates that there has been no change; otherwise, it indicates a change. By judging the second multimodal environmental perception data and the status of working equipment, and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data representing the complex environment and the status of working equipment, realizing dynamic path planning. This significantly improves coverage and further enhances security.

[0039] In specific scenarios, the coverage of the present invention is compared with that of traditional solutions. For example, in a scenario with wet and slippery grass in the morning with dew (slope angle θ of about 10°, working condition coefficient μ = 0.6), when the working equipment performs horizontal operations according to the new working spacing d and yaw angle yaw, the coverage is about 94% without working spacing d compensation and about 97% with spacing d compensation. The coverage of the traditional solution is about 82%. The comparison shows that the coverage of the present invention is significantly better than that of the traditional solution.

[0040] Example 2: This embodiment also provides a dynamic path planning system based on multimodal environment perception, which is applicable to the dynamic path planning method based on multimodal environment perception, including a desired parameter acquisition module, a safety factor acquisition module, a planned path acquisition module, and a dynamic update module; The expected parameter acquisition module is used to acquire expected parameters based on the first multimodal environment perception data; The safety factor acquisition module is used to quantify the safety of path planning using the expected parameters based on the established correlation function representing the correlation between the expected parameters and the working equipment status to obtain the safety factor. The planning path acquisition module is used to obtain planning parameters based on the safety factor and expected parameters, and then use the planning parameters to obtain the planning path. During the operation based on the planned path, the dynamic update module is used to determine whether the second multimodal environment perception data meets the preset conditions. If the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data, which prompts the expected parameter acquisition module to work. If the second multimodal environment perception data meets the preset conditions, it is determined whether the status of the working equipment has changed. If the status of the working equipment has changed, the planned parameters are used as expected parameters, which prompts the safety factor acquisition module to work.

[0041] In this embodiment, the first multimodal environmental perception data consists of the environmental data and operational data of the working equipment obtained before the path of the working equipment is optimized. Expected parameters include the working condition coefficient, slope gradient, and expected work spacing. The second multimodal environmental perception data consists of the environmental data and operational data of the working equipment obtained during the work process based on the planned path. The second multimodal environmental perception data is matched with the first multimodal environmental perception data. If the match is successful, it indicates that a preset condition is met. For example, if the second and first multimodal environmental perception data are analyzed separately, and the analysis results are "lawn with 30% dew" and "lawn with 40% dew," then according to the matching rules, the match is considered successful. Users can flexibly set the matching rules according to actual needs. Determining whether the working equipment status has changed includes classifying the working equipment status into three categories: working equipment battery level greater than 80%, working equipment battery level greater than 20% and less than or equal to 80%, and working equipment battery level less than or equal to 20%. If the battery level before and after the change belongs to the same category, it indicates no change; otherwise, it indicates a change. By judging the second multimodal environmental perception data and the status of working equipment, and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data representing the complex environment and the status of working equipment, realizing dynamic path planning. This significantly improves coverage and further enhances security.

[0042] Example 3: This embodiment also provides a storage medium storing computer-executable instructions. When the computer-executable instructions are loaded and executed by a processor, they implement the steps of the dynamic path planning method based on multimodal environment awareness.

[0043] The specific embodiments described above are preferred embodiments of the dynamic path planning method and system based on multimodal environment perception of the present invention, and are not intended to limit the specific scope of the present invention. The scope of the present invention includes but is not limited to the specific embodiments described above. All equivalent changes made in accordance with the shape and structure of the present invention are within the protection scope of the present invention.

Claims

1. A dynamic path planning method based on multimodal environment perception, characterized in that, Includes the following steps: S1: Obtain the desired parameters based on the first multimodal environment perception data; S2: Based on the established correlation function representing the correlation between expected parameters and working equipment status, the safety of using expected parameters for path planning is quantified to obtain the safety coefficient; S3: Obtain planning parameters based on safety factor and expected parameters, and use planning parameters to obtain planning path; S4: During the work based on the planned path, if the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data, and S1 is executed; otherwise, it is determined whether the status of the working equipment has changed. If it has changed, the planning parameters are used as the expected parameters, and S2 is executed. S3 includes: S31: Obtain the work spacing in the planning parameters based on the safety factor and the work spacing in the expected parameters; S32: Obtain the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters, and use the work spacing in the planning parameters and the yaw angle in the planning parameters to obtain the planned path; S31 includes: In the formula, The work spacing in the planning parameters, This represents the weight obtained by comparing the actual position with the desired position when performing operations using the work spacing in the desired parameters. The safety factor is... The work spacing is the desired parameter. In S32, obtaining the yaw angle in the planning parameters based on the safety factor and the expected yaw angle includes: In the formula, The yaw angle in the planning parameters. The yaw angle is the desired parameter. This represents the weight obtained from the deviation between the actual position and the desired position when using the yaw angle from the desired parameters for operation. This represents the yaw angle calculated using GPS positioning and IMU inertial navigation.

2. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that, S1 includes: S11: Based on the characteristics of historical multimodal environmental perception data, the historical multimodal environmental perception data is divided into primary category data and secondary category data, and a scene recognition model is constructed based on gradient-driven learning; S12: Train the scene recognition model once using the first-level category data. If the training is successful, execute S13. Otherwise, use the data in the first-level category data that caused the training to fail as the first-level category data and execute S12 again. S13: Perform secondary training on the scene recognition model using secondary category data. If the secondary training is successful, execute S14; otherwise, use the data in the secondary category data that caused the secondary training to fail as secondary category data and execute S13 again. S14: Based on the first multimodal environment perception data, use the scene recognition model to obtain the expected parameters.

3. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that, In S1, the first multimodal environmental perception data and in S4, the second multimodal environmental perception data are respectively the environmental data and working data of the working equipment in different time periods.

4. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that, S2 further includes: The weights of the desired parameters are obtained based on the working equipment status, and a correlation function is established based on the weights of the desired parameters to characterize the correlation between the desired parameters and the working equipment status.

5. The dynamic path planning method based on multimodal environment perception according to claim 4, characterized in that, The relevant function is: In the formula, The safety factor is... , and All of these are the desired parameters. , and 3 are respectively , and The weight.

6. A dynamic path planning system based on multimodal environment perception, applicable to the dynamic path planning method based on multimodal environment perception as described in any one of claims 1-5, characterized in that, It includes a module for obtaining expected parameters, a module for obtaining safety factors, a module for obtaining planned paths, and a dynamic update module; The expected parameter acquisition module is used to acquire expected parameters based on the first multimodal environment perception data; The safety factor acquisition module is used to quantify the safety of path planning using the expected parameters based on the established correlation function representing the correlation between the expected parameters and the working equipment status to obtain the safety factor. The planning path acquisition module is used to obtain planning parameters based on the safety factor and expected parameters, and then use the planning parameters to obtain the planning path. During the operation based on the planned path, the dynamic update module is used to determine whether the second multimodal environment perception data meets the preset conditions. If the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data, which prompts the expected parameter acquisition module to work. If the second multimodal environment perception data meets the preset conditions, it is determined whether the status of the working equipment has changed. If the status of the working equipment has changed, the planned parameters are used as expected parameters, which prompts the safety factor acquisition module to work.

7. A storage medium, characterized in that, The storage medium stores computer-executable instructions, which, when loaded and executed by a processor, implement the steps of the dynamic path planning method based on multimodal environment awareness as described in any one of claims 1 to 5.

Images