Dynamic path planning method and system based on multi-modal environment perception

Abstract

The invention discloses a dynamic path planning method and system based on multi-modal environment perception, and belongs to the technical field of intelligent mobile robots, and the method comprises the steps: S1, obtaining an expected parameter based on first multi-modal environment perception data; s2, quantifying the safety of path planning by using the expected parameter based on the established correlation function representing the correlation between the expected parameter and the working equipment state to obtain a safety coefficient; s3, acquiring planning parameters based on the safety coefficient and the expected parameters, and acquiring a planning path by using the planning parameters; and S4, in the working process based on the planned path, if the second multi-mode environment sensing data does not meet the preset condition, taking the second multi-mode environment sensing data as the first multi-mode sensing data, and executing S1, otherwise, judging whether the state of the working equipment is changed or not, and if the state is changed, taking the planning parameter as an expected parameter, and executing S2. The technical problem that in the prior art, it is difficult to have high safety while the coverage rate is increased is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent mobile robots, and in particular to a dynamic path planning method and system based on multimodal environment perception. Background Art

[0002] With the rapid development of intelligent technology, the application scenarios of outdoor automation equipment are constantly expanding, and the types of equipment are becoming more and more diverse. However, faced with the complex and changeable outdoor working environment, the working efficiency of the equipment faces severe challenges. The existing technology mainly plans the movement path of outdoor automation equipment so that it can adapt to the working requirements in the corresponding environment, thereby improving its working efficiency. For example, a walking path planning method with patent number CN117472045A includes: obtaining characteristic information of the target working area, the characteristic information at least includes ramp information; planning the walking path of the intelligent device in the target working area based on the ramp information. The method can not only solve the coverage planning problem of a single slope scene, but also can achieve reasonable coverage planning for the situation where there are multiple slopes with different directions, thereby improving the working efficiency and mowing effect of the lawn mower. However, the above method is only applicable to static environments. When the environment changes, the above solution is difficult to use, especially in snowy and rainy environments with a low friction coefficient. When outdoor automation equipment works according to the planned path, there is a risk of slipping, which leads to insufficient work coverage. Existing technologies mainly use fixed rules (such as bow-shaped coverage) to improve coverage, but in complex scenarios, it is easy to cause a large number of repeated paths, which significantly increases energy consumption. Especially when the device is in a low-battery state, there is a lack of a dynamic adjustment mechanism. Tasks may be interrupted due to unreasonable path planning, which not only affects work efficiency, but also causes operational safety issues of outdoor automation equipment due to power exhaustion. Therefore, how to improve coverage while maintaining high safety is a technical problem that is difficult to solve at present. Summary of the Invention

[0003] In response to the technical problem that existing technologies have difficulty in improving coverage while maintaining high safety, the present invention provides a dynamic path planning method and system based on multimodal environmental perception. By establishing a correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of path planning using the expected parameters is quantified to obtain a safety factor, thereby achieving quantification and dynamic evaluation of safety. Furthermore, path planning based on the safety factor and the expected parameters significantly improves safety. Furthermore, by judging the second multimodal environmental perception data and the working equipment status and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data characterizing the complex environment and the working equipment status, achieving dynamic path planning and significantly improving coverage. This overcomes the technical problem that existing technologies have difficulty in improving coverage while maintaining high safety.

[0004] To solve the above technical problems, the present invention provides a dynamic path planning method based on multimodal environment perception, comprising the following steps: S1: Acquire desired parameters based on first multimodal environment perception data; S2: Based on the established correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of path planning using the expected parameters is quantified to obtain a safety factor; S3: Obtain planning parameters based on the safety factor and the expected parameters, and use the planning parameters to obtain the planned path; S4: During the process of working based on the planned path, if the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data and S1 is executed. Otherwise, it is determined whether the working equipment status has changed. If it has changed, the planning parameters are used as the expected parameters and S2 is executed.

[0005] After adopting the above technical solution, the present invention has the following advantages: By establishing a correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of path planning using the expected parameters is quantified to obtain a safety factor, thereby achieving quantification and dynamic evaluation of safety. Furthermore, path planning is performed using the safety factor and the expected parameters, which significantly improves safety. Furthermore, by judging the second multimodal environmental perception data and the working equipment status and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data characterizing the complex environment and the working equipment status, thereby achieving dynamic planning of the path. While significantly improving coverage, it also further improves safety, overcoming the technical problem of the existing technology that it is difficult to achieve high safety while improving coverage. Obtaining the desired parameters through the first multimodal environmental perception data overcomes the limitations of a single sensor in a specific scenario, thereby improving the accuracy of the desired parameters; By obtaining accurate expected parameters through the first multimodal environmental perception data, the safety of path planning using the accurate expected parameters is quantified using relevant functions to obtain a safety factor. This also effectively solves the technical problem of the difficulty in quantifying the impact of the first multimodal environmental perception data on driving stability, thereby improving the driving stability of the working equipment and further improving work safety.

[0006] Preferably, the S1 includes: S11: Based on the characteristics of historical multimodal environmental perception data, the historical multimodal environmental perception data is divided into primary and secondary categories, and a scene recognition model is constructed based on gradient-driven learning. S12: The scene recognition model is trained once using the first-level category data. When the training is successful, S13 is executed. Otherwise, the data in the first-level category data that caused the training to fail is used as the first-level category data and S12 is executed again. S13: Perform secondary training on the scene recognition model using the secondary category data. If the secondary training succeeds, execute S14. Otherwise, execute S13 again using the data in the secondary category data that caused the secondary training to fail as the secondary category data. S14: Based on the first multimodal environmental perception data, use a scene recognition model to obtain expected parameters.

[0007] Preferably, the first multimodal environment perception data in S1 and the second multimodal environment perception data in S4 are respectively the environment data of the working equipment and the working data of the working equipment in different time periods.

[0008] Preferably, the S2 further includes: The weight of the expected parameter is obtained based on the working device state, and a correlation function representing the correlation between the expected parameter and the working device state is established based on the weight of the expected parameter.

[0009] Preferably, the correlation function is: S=k1*μ+k2*sin(θ)+k3*D; wherein S is the safety factor, μ, θ and D are the expected parameters, and k1, k2 and k3 are the weights of μ, θ and D respectively.

[0010] Preferably, the S3 includes: S31: Obtaining the operation spacing in the planning parameters based on the safety factor and the operation spacing in the expected parameters; S32: Obtaining the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters, and obtaining the planned path using the operation spacing in the planning parameters and the yaw angle in the planning parameters.

[0011] Preferably, the S31 includes: d=n*Q*R; wherein d is the operation spacing in the planning parameters, n represents the weight obtained by the deviation between the actual position and the expected position when the operation is performed using the operation spacing in the expected parameters, Q is the safety factor, and R is the operation spacing in the expected parameters.

[0012] Preferably, in S32, acquiring the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters includes: yaw = α*m*Q*R+yan; where yaw is the yaw angle in the planning parameters, α is the yaw angle in the expected parameters, m represents the weight obtained by the deviation between the actual position and the expected position when the yaw angle in the expected parameters is used for operation, and yan represents the yaw angle calculated using GPS positioning and IMU inertial navigation.

[0013] Beneficial effects of this program: By establishing a correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of path planning using the expected parameters is quantified to obtain a safety factor, thereby achieving quantification and dynamic evaluation of safety. Furthermore, path planning is performed using the safety factor and the expected parameters, which significantly improves safety. Furthermore, by judging the second multimodal environmental perception data and the working equipment status and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data characterizing the complex environment and the working equipment status, thereby achieving dynamic planning of the path. While significantly improving coverage, it also further improves safety, overcoming the technical problem of the existing technology that it is difficult to achieve high safety while improving coverage. Obtaining desired parameters through the first multimodal environmental perception data overcomes the limitations of a single sensor in a specific scenario, thereby improving the accuracy of the desired parameters; Specifically, by dividing historical multimodal environmental perception data into first-level category data and second-level category data, the scene recognition model is trained once with the first-level category data, enabling the scene recognition model to identify lawns, snow, floor tiles, etc. After successful training, the scene recognition model is trained again with the second-level category data, enabling the scene recognition model to identify dew on lawns and lawn colors, etc., thereby facilitating the acquisition of expected parameters based on the identified conditions, thereby improving the hierarchical accuracy of scene recognition, further improving the accuracy of expected parameters, and improving the training efficiency of the model; Accurate expected parameters are obtained through the first multimodal environmental perception data, and the safety of path planning using the expected parameters is quantified using relevant functions to obtain a safety factor. This also effectively solves the technical problem of the difficulty in quantifying the impact of the first multimodal environmental perception data on driving stability, thereby improving the driving stability of the working equipment and further improving work safety.

[0014] The present invention also provides a dynamic path planning system based on multimodal environment perception, which is applicable to the dynamic path planning method based on multimodal environment perception, and includes an expected parameter acquisition module, a safety factor acquisition module, a planned path acquisition module, and a dynamic update module; The expected parameter acquisition module is used to acquire the expected parameters based on the first multimodal environment perception data; The safety factor acquisition module is used to quantify the safety of path planning using the expected parameters based on the established correlation function that characterizes the correlation between the expected parameters and the working equipment state to obtain the safety factor; The planned path acquisition module is used to obtain planning parameters based on the safety factor and the expected parameters, and obtain the planned path using the planning parameters; In the process of working based on the planned path, the dynamic update module is used to determine whether the second multimodal environmental perception data meets the preset conditions. When it is determined that the second multimodal environmental perception data does not meet the preset conditions, the second multimodal environmental perception data is used as the first multimodal perception data to prompt the expected parameter acquisition module to work. When it is determined that the second multimodal environmental perception data meets the preset conditions, it is determined whether the working equipment status has changed. When it is determined that the working equipment status has changed, the planning parameters are used as the expected parameters to prompt the safety factor acquisition module to work.

[0015] Beneficial effects of this program: By establishing a correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of path planning using the expected parameters is quantified to obtain a safety factor, thereby achieving quantification and dynamic evaluation of safety. Furthermore, path planning is performed using the safety factor and the expected parameters, which significantly improves safety. Furthermore, by judging the second multimodal environmental perception data and the working equipment status and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data characterizing the complex environment and the working equipment status, thereby achieving dynamic planning of the path. While significantly improving coverage, it also further improves safety, overcoming the technical problem of the existing technology that it is difficult to achieve high safety while improving coverage. Obtaining the desired parameters through the first multimodal environmental perception data overcomes the limitations of a single sensor in a specific scenario, thereby improving the accuracy of the desired parameters; By obtaining accurate expected parameters through the first multimodal environmental perception data, the safety of path planning using the accurate expected parameters is quantified using relevant functions to obtain a safety factor. This also effectively solves the technical problem of the difficulty in quantifying the impact of the first multimodal environmental perception data on driving stability, thereby improving the driving stability of the working equipment and further improving work safety.

[0016] The present invention also provides a storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the steps of the dynamic path planning method based on multimodal environment perception are implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Other features, objects, and advantages of the present invention will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings. The drawings are provided for the purpose of illustrating preferred embodiments only and are not to be construed as limiting the present invention. Like reference characters are used throughout the drawings to designate like parts.

[0018] Figure 1 This is a flow chart of a dynamic path planning method based on multimodal environment perception according to the present invention; Figure 2 A first glide path diagram of the working equipment in the dynamic path planning method based on multimodal environment perception of the present invention; Figure 3 This is a second glide path diagram of the working equipment in the dynamic path planning method based on multimodal environment perception of the present invention. DETAILED DESCRIPTION

[0019] In order to make the objectives, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific implementation method described herein is only an optimal embodiment of the present invention, which is only used to explain the present invention and does not limit the scope of protection of the present invention. All other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0020] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe the operations (or steps) as sequential processes, many of the operations (or steps) therein can be performed in parallel, concurrently, or simultaneously. In addition, the order of the operations can be rearranged. The process can be terminated when its operations are completed, but can also have additional steps not included in the figures; the process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0021] Example 1: like Figure 1 As shown, the dynamic path planning method based on multimodal environment perception includes the following steps: S1: Acquire expected parameters based on first multimodal environmental perception data.

[0022] Said S1 comprises: S11: Based on the characteristics of historical multimodal environmental perception data, the historical multimodal environmental perception data is divided into primary and secondary categories, and a scene recognition model is constructed based on gradient-driven learning. S12: The scene recognition model is trained once using the first-level category data. When the training is successful, S13 is executed. Otherwise, the data in the first-level category data that caused the training to fail is used as the first-level category data and S12 is executed again. S13: Perform secondary training on the scene recognition model using the secondary category data. If the secondary training succeeds, execute S14. Otherwise, execute S13 again using the data in the secondary category data that caused the secondary training to fail as the secondary category data. S14: Based on the first multimodal environmental perception data, use a scene recognition model to obtain expected parameters.

[0023] The first multimodal environment perception data in S1 and the second multimodal environment perception data in S4 are respectively the environment data of the working equipment and the working data of the working equipment in different time periods.

[0024] In this embodiment, the first multimodal environmental perception data is the environmental data and working data of the working equipment acquired before the working equipment's path is optimized. The primary category data can be used to identify whether a complex scene is lawn, snow, or whether there are obstacles. Taking lawn as an example, the secondary category data can be used to identify whether the lawn is dewy, dry, and its color. In this embodiment, gradient-driven learning is achieved by combining machine learning with gradient descent algorithms. Machine learning integrates the primary category data, thereby improving the recognition accuracy of complex scenes. Combined with gradient descent algorithms, microscopic differences, such as "dewy lawn" or "dry lawn," can be identified. By combining machine learning with gradient descent algorithms, the hierarchical accuracy of scene recognition is improved, as well as the accuracy of expected parameters.

[0025] In this embodiment, the first-level category data is input into the scene recognition model. When a preset number of first outputs of the scene recognition model match the actual situation corresponding to the first-level category data, it indicates that the first training is successful. After the first training is successful, the second-level category data is input into the scene recognition model. When a preset number of second outputs of the scene recognition model match the actual situation corresponding to the second-level category data, it indicates that the second training is successful. The preset number is flexibly set according to user needs. The mode of performing the second training only after the first training is successful avoids the waste of time and computing resources, thereby improving the training efficiency. When the first training or the second training is unsuccessful, the data in the first-level category data that caused the first training to fail is used as the first-level category data, and the data in the second-level category data that caused the second training to fail is used as the second-level category data, and the corresponding operations are performed respectively, avoiding the need to input all the first-level category data or the second-level category data into the scene recognition model again to train the scene recognition model, further avoiding the waste of time and computing resources, and further improving the model training efficiency.

[0026] In this embodiment, first multimodal environmental perception data is acquired via a data acquisition device, which includes sensors, cameras, and the like. The environmental data of the working device can be data from an outdoor environment, such as a lawn or snowy field. The working device operating data includes positioning coordinates, altitude, device heading, inertial navigation attitude, and other data. In this embodiment, using a scene recognition model to acquire desired parameters based on the first multimodal environmental perception data includes: inputting the environmental data of the working device into the scene recognition model to acquire a working condition coefficient μ; fusing the positioning coordinates (i.e., GPS positions posx, poxy, and posz) with the inertial navigation attitude to acquire the slope gradient θ; and acquiring the desired working distance based on the width of the main components of the working device. For example, if the working device is a smart lawn mower, the main component is the blade. If the blade width is D1, then D1 > D, where D is the desired working distance. Acquiring the desired parameters via the first multimodal environmental perception data overcomes the limitations of a single sensor or camera in specific scenarios, thereby improving the accuracy of the desired parameters. Acquiring the desired working distance based on the width of the main components of the working device improves the working coverage of the working device.

[0027] S2: Based on the established correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of using the expected parameters for path planning is quantified to obtain a safety factor.

[0028] Said S2 further comprises: The weight of the expected parameter is obtained based on the working device state, and a correlation function representing the correlation between the expected parameter and the working device state is established based on the weight of the expected parameter.

[0029] The correlation function is: S=k1*μ+k2*sin(θ)+k3*D; where S is the safety factor, μ, θ and D are the expected parameters, and k1, k2 and k3 are the weights of μ, θ and D respectively.

[0030] In this embodiment, the weights of the expected parameters obtained based on the working equipment status include: when the power of the working equipment is greater than 80%, k1, k2 and k3 are 0.2, 0.5 and 0.3 respectively, and work efficiency is prioritized at this time; when the power of the working equipment is greater than 20% and less than or equal to 80%, it is in balance mode; when the power of the working equipment is less than or equal to 20%, energy-saving recharging is performed. The safety factor is obtained by dynamically adjusting the weight according to the power, which realizes the reasonable and sufficient use of the working equipment, improves the reasonable utilization rate of the working equipment, and realizes the quantification and dynamic evaluation of safety, thereby facilitating path planning through the safety factor and the expected parameters, avoiding task interruption, and significantly improving safety. By setting weights for the expected parameters, the lack of collaborative decision-making of multiple environmental factors is also compensated, thereby improving the comprehensiveness and adaptability of the decision.

[0031] S3: Obtain planning parameters based on the safety factor and the expected parameters, and use the planning parameters to obtain a planned path.

[0032] The S3 includes: S31: Obtaining the operation spacing in the planning parameters based on the safety factor and the operation spacing in the expected parameters; S32: Obtaining the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters, and obtaining the planned path using the operation spacing in the planning parameters and the yaw angle in the planning parameters.

[0033] The S31 includes: d=n*Q*R; where d is the operation spacing in the planning parameters, n represents the weight obtained by the deviation between the actual position and the expected position when the operation is performed using the operation spacing in the expected parameters, Q is the safety factor, and R is the operation spacing in the expected parameters.

[0034] In S32, the yaw angle in the planning parameters obtained based on the safety factor and the yaw angle in the expected parameters includes: yaw = α*m*Q*R+yan; wherein, yaw is the yaw angle in the planning parameters, α is the yaw angle in the expected parameters, m represents the weight obtained by the deviation between the actual position and the expected position when the yaw angle in the expected parameters is used for operation, and yan represents the yaw angle calculated using GPS positioning and IMU inertial navigation.

[0035] In this embodiment, when the safety factor is greater than 0.5, the planning parameters are obtained based on the safety factor and the expected parameters, and the planning path is obtained using the planning parameters. When the safety factor is less than or equal to 0.5, the expected parameters are used to obtain the planning path. By setting a reasonable safety factor, the working efficiency of the working equipment is improved while the work safety is also improved. The weight obtained by the deviation between the actual position and the expected position when the yaw angle in the expected parameter is used for operation can be obtained at a fixed time or at a fixed distance. For working conditions with different friction coefficients, if the horizontal operation is planned, the horizontal operation plan adjusts the drive wheel torque distribution strategy based on the longitudinal gravity component (the slope θ of the slope) and the GPS positioning of the equipment. The force of the lower drive wheel needs to be slightly greater than the upper drive force to form an overall upward drive force to offset the downward component of gravity. After offsetting, the operation can be carried out according to the horizontal trajectory. From the perspective of the machine form, a yaw angle α of the control quantity will be formed (the horizontal theoretical direction and the actual planned trajectory). Thereby, the control deviation in the vertical direction is offset, so that the operation spacing is guaranteed. As Figure 2 As shown, Figure 2 The deviation parameters in the planning operation are described in , and the yaw angle α is the planned direction in the slip mode.

[0036] In this embodiment, the use of the operation spacing in the planning parameters and the yaw angle in the planning parameters to obtain the planned path includes: when the slope is small, only the yaw angle in the planning parameters is used to obtain the planned path; when the slope is large or the slope is severely declining, the yaw angle in the planning parameters and the operation spacing in the planning parameters are used to obtain the planned path, such as Figure 3 As shown, the blue line shows a deviation between the actual trajectory and the theoretically planned trajectory (green). If a smaller working distance is used, although gravity causes the machine to slide down a distance, the narrower spacing still allows for a high coverage rate. When the user does not care about the cutting direction, the working mode on the slope is dynamically optimized to vertical operation, minimizing the impact of gravity on yaw. This significantly improves safety and coverage.

[0037] S4: During the process of working based on the planned path, if the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data and S1 is executed. Otherwise, it is determined whether the working equipment status has changed. If it has changed, the planning parameters are used as the expected parameters and S2 is executed.

[0038] In this embodiment, the second multimodal environmental perception data comprises environmental data and working data of the working device acquired during operation based on the planned path. The second multimodal environmental perception data is matched with the first multimodal environmental perception data. A successful match indicates that a preset condition has been satisfied. For example, if the second multimodal environmental perception data and the first multimodal environmental perception data are analyzed separately and the analysis results are respectively lawn with a dew content of 30% and lawn with a dew content of 40%, then the match is considered successful according to the matching rules. The user can flexibly set the matching rules based on actual needs. Determining whether the working device status has changed includes classifying the working device status into three categories: when the working device battery level is greater than 80%, when the working device battery level is greater than 20% and less than or equal to 80%, and when the working device battery level is less than or equal to 20%. If the battery levels before and after the change are the same, it indicates no change; otherwise, it indicates a change. By judging the second multimodal environmental perception data and the working equipment status and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data representing the complex environment and the working equipment status, thereby realizing dynamic planning of the path, which not only significantly improves the coverage but also further improves safety.

[0039] The coverage rates of the present invention and the traditional solution are compared in specific scenarios. For example, in the morning on slippery grass with dew (slope gradient θ is about 10°, working condition coefficient μ=0.6), when the working equipment performs horizontal operations according to the new working spacing d and yaw angle yaw, the coverage rate is about 94% without using working spacing d compensation, and about 97% with using spacing d compensation. The coverage rate of the traditional solution is about 82%. It can be seen from the comparison that the coverage rate of the present invention is significantly better than that of the traditional solution.

[0040] Example 2: This embodiment also provides a dynamic path planning system based on multimodal environment perception, which is applicable to the dynamic path planning method based on multimodal environment perception, including an expected parameter acquisition module, a safety factor acquisition module, a planned path acquisition module and a dynamic update module; The expected parameter acquisition module is used to acquire the expected parameters based on the first multimodal environment perception data; The safety factor acquisition module is used to quantify the safety of path planning using the expected parameters based on the established correlation function that characterizes the correlation between the expected parameters and the working equipment state to obtain the safety factor; The planned path acquisition module is used to obtain planning parameters based on the safety factor and the expected parameters, and obtain the planned path using the planning parameters; In the process of working based on the planned path, the dynamic update module is used to determine whether the second multimodal environmental perception data meets the preset conditions. When it is determined that the second multimodal environmental perception data does not meet the preset conditions, the second multimodal environmental perception data is used as the first multimodal perception data to prompt the expected parameter acquisition module to work. When it is determined that the second multimodal environmental perception data meets the preset conditions, it is determined whether the working equipment status has changed. When it is determined that the working equipment status has changed, the planning parameters are used as the expected parameters to prompt the safety factor acquisition module to work.

[0041] In this embodiment, the first multimodal environmental perception data comprises environmental data and operating data of the working equipment acquired before the working equipment path is optimized. Expected parameters include a working condition coefficient, slope gradient, and expected working distance. The second multimodal environmental perception data comprises environmental data and operating data of the working equipment acquired during operation based on the planned path. The second multimodal environmental perception data is matched with the first multimodal environmental perception data. A successful match indicates that a preset condition has been met. For example, if the second multimodal environmental perception data and the first multimodal environmental perception data are analyzed and the analysis results are respectively lawn with a dew content of 30% and lawn with a dew content of 40%, then the match is considered successful according to the matching rules. The user can flexibly configure the matching rules based on actual needs. Determining whether the working equipment status has changed includes classifying the working equipment status into three categories: when the working equipment battery level is greater than 80%, when the working equipment battery level is greater than 20% but less than or equal to 80%, and when the working equipment battery level is less than or equal to 20%. If the battery level before and after the change is the same, it indicates no change; otherwise, it indicates a change. By judging the second multimodal environmental perception data and the working equipment status and executing corresponding steps based on the judgment results, the path can be updated based on the multimodal data representing the complex environment and the working equipment status, thereby realizing dynamic planning of the path, which not only significantly improves the coverage but also further improves safety.

[0042] Example 3: This embodiment also provides a storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are loaded and executed by a processor, the steps of the dynamic path planning method based on multimodal environment perception are implemented.

[0043] The specific implementation described above is a preferred implementation of the dynamic path planning method and system based on multimodal environment perception of the present invention, and is not intended to limit the specific implementation scope of the present invention. The scope of the present invention includes but is not limited to this specific implementation. Any equivalent changes made in accordance with the shape and structure of the present invention are within the scope of protection of the present invention.

Claims

1. A dynamic path planning method based on multimodal environment perception, characterized in that: The following steps are involved: S1: Acquire desired parameters based on first multimodal environment perception data; S2: Based on the established correlation function that characterizes the correlation between the expected parameters and the working equipment status, the safety of path planning using the expected parameters is quantified to obtain a safety factor; S3: Obtain planning parameters based on the safety factor and the expected parameters, and use the planning parameters to obtain the planned path; S4: During the process of working based on the planned path, if the second multimodal environment perception data does not meet the preset conditions, the second multimodal environment perception data is used as the first multimodal perception data and S1 is executed. Otherwise, it is determined whether the working equipment status has changed. If it has changed, the planning parameters are used as the expected parameters and S2 is executed.

2. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that: Said S1 comprises: S11: Based on the characteristics of historical multimodal environmental perception data, the historical multimodal environmental perception data is divided into primary and secondary categories, and a scene recognition model is constructed based on gradient-driven learning. S12: The scene recognition model is trained once using the first-level category data. When the training is successful, S13 is executed. Otherwise, the data in the first-level category data that caused the training to fail is used as the first-level category data and S12 is executed again. S13: Perform secondary training on the scene recognition model using the secondary category data. If the secondary training succeeds, execute S14. Otherwise, execute S13 again using the data in the secondary category data that caused the secondary training to fail as the secondary category data. S14: Based on the first multimodal environmental perception data, use a scene recognition model to obtain expected parameters.

3. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that: The first multimodal environment perception data in S1 and the second multimodal environment perception data in S4 are respectively the environment data of the working equipment and the working data of the working equipment in different time periods.

4. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that: Said S2 further comprises: The weight of the expected parameter is obtained based on the working device state, and a correlation function representing the correlation between the expected parameter and the working device state is established based on the weight of the expected parameter.

5. The dynamic path planning method based on multimodal environment perception according to claim 4 is characterized in that: The correlation function is: S=k1*μ+k2*sin(θ)+k3*D; where S is the safety factor, μ, θ and D are the expected parameters, and k1, k2 and k3 are the weights of μ, θ and D respectively.

6. The dynamic path planning method based on multimodal environment perception according to claim 1, characterized in that: The S3 includes: S31: Obtaining the operation spacing in the planning parameters based on the safety factor and the operation spacing in the expected parameters; S32: Obtaining the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters, and obtaining the planned path using the operation spacing in the planning parameters and the yaw angle in the planning parameters.

7. The dynamic path planning method based on multimodal environment perception according to claim 6, characterized in that: The S31 includes: d=n*Q*R; where d is the operation spacing in the planning parameters, n represents the weight obtained by the deviation between the actual position and the expected position when the operation is performed using the operation spacing in the expected parameters, Q is the safety factor, and R is the operation spacing in the expected parameters.

8. The dynamic path planning method based on multimodal environment perception according to claim 7, characterized in that: In S32, obtaining the yaw angle in the planning parameters based on the safety factor and the yaw angle in the expected parameters includes: yaw = α*m*Q*R+yan; where yaw is the yaw angle in the planning parameters, α is the yaw angle in the expected parameters, m represents the weight obtained by the deviation between the actual position and the expected position when the yaw angle in the expected parameters is used for operation, and yan represents the yaw angle calculated using GPS positioning and IMU inertial navigation.

9. A dynamic path planning system based on multimodal environment perception, applicable to the dynamic path planning method based on multimodal environment perception according to any one of claims 1 to 8, characterized in that: It includes expected parameter acquisition module, safety factor acquisition module, planned path acquisition module and dynamic update module; The expected parameter acquisition module is used to acquire the expected parameters based on the first multimodal environment perception data; The safety factor acquisition module is used to quantify the safety of path planning using the expected parameters based on the established correlation function that characterizes the correlation between the expected parameters and the working equipment state to obtain the safety factor; The planned path acquisition module is used to obtain planning parameters based on the safety factor and the expected parameters, and obtain the planned path using the planning parameters; In the process of working based on the planned path, the dynamic update module is used to determine whether the second multimodal environmental perception data meets the preset conditions. When it is determined that the second multimodal environmental perception data does not meet the preset conditions, the second multimodal environmental perception data is used as the first multimodal perception data to prompt the expected parameter acquisition module to work. When it is determined that the second multimodal environmental perception data meets the preset conditions, it is determined whether the working equipment status has changed. When it is determined that the working equipment status has changed, the planning parameters are used as the expected parameters to prompt the safety factor acquisition module to work.

10. A storage medium, characterized in that: The storage medium stores computer-executable instructions, which, when loaded and executed by the processor, implement the steps of the dynamic path planning method based on multimodal environment perception as described in any one of claims 1 to 8.

Images