Vehicle control method and device, electronic equipment and computer readable storage medium

Abstract

The embodiment of the invention provides a vehicle control method and device, electronic equipment and a computer readable storage medium, and the method comprises the steps: obtaining first navigation information of a vehicle in a driving process, the first navigation information being used for at least determining a preset driving path of the vehicle; determining second navigation information corresponding to the preset driving path, and collecting a road image corresponding to the preset driving path; determining a global driving path of the vehicle based on the first navigation information, the second navigation information and the road image; based on obstacle information in the global driving path and the global driving path, a local driving path of the vehicle in the driving direction is determined, and the obstacle information is used for determining obstacles in a moving state in the global driving path; and controlling the vehicle to run according to the local driving path. The technical problem that the safety of the vehicle planning path is low is solved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of data processing, and in particular, to a control method, device, electronic device, and computer-readable storage medium for a vehicle. Background Art

[0002] Currently, the path planning of vehicles mainly relies on pre-set static map data. However, this method requires a long time to identify and evaluate obstacles to adjust the driving path. Therefore, this method may lead to a delay in the driving path planning decision and increase the potential collision risk. In addition, it is difficult for this method to immediately reflect the moving obstacles on the road. When encountering dynamic obstacles (such as pedestrians, suddenly appearing obstacles, etc.), it is often unable to immediately adjust the path, resulting in a low safety of the driving path planning. Therefore, the above method still has the technical problem of low safety of the path planned by the vehicle. Summary of the Invention

[0003] The embodiments of the present application provide a control method, device, electronic device, and computer-readable storage medium for a vehicle, aiming to improve the technical problem of low safety of the path planned by the vehicle.

[0004] According to one embodiment of the present application, a control method for a vehicle is provided. The method may include: obtaining first navigation information during the driving process of the vehicle, where the first navigation information is used to at least determine the preset driving path of the vehicle; determining second navigation information corresponding to the preset driving path, and collecting a road image corresponding to the preset driving path, where the second navigation information is used to at least represent the driving instruction information corresponding to the vehicle when driving in the preset driving path; determining the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image; determining the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path, where the obstacle information is used to determine the obstacles in the global driving path that are in a moving state; controlling the vehicle to drive according to the local driving path.

[0005] The above optional embodiments of the present application can achieve the following beneficial effects: determining a preset driving path using the obtained first navigation information, and determining second navigation information corresponding to the preset driving path. Combining the first navigation information, the second navigation information, and the road image, a global driving path can be determined. Through this method, the road environment and navigation information can be considered more comprehensively, thereby improving the accuracy and reliability of path planning. And determining a local driving path according to the global driving path and obstacle information, and controlling the vehicle to drive according to the local driving path can enable the vehicle to make more reasonable and rapid decisions in the case of encountering obstacle information (such as dynamic obstacles) or different traffic conditions, ensuring the smoothness and safety of the vehicle under complex road conditions, thereby solving the technical problem of low safety of the path planned by the vehicle and achieving the technical effect of improving the safety of the path planned by the vehicle.

[0006] Optionally, based on the first navigation information, the second navigation information, and the road image, determining the global driving path of the vehicle includes: identifying the first navigation information to obtain a driving instruction text and auxiliary road data of the preset driving path; based on the road image, adjusting the auxiliary road data to obtain target road data; planning a global driving path of the vehicle based on the driving instruction text, the second navigation information, and the target road data.

[0007] The above optional embodiments of the present application can achieve the following beneficial effects: by identifying the first navigation information, a driving instruction text and auxiliary road data of the preset driving path can be obtained, and then combining the road image to adjust the auxiliary road data, thereby forming more accurate target road data, which can effectively make up for the problem of inaccurate target road data caused by the deficiency of a single information source. After obtaining the target road data, planning the global driving path of the vehicle by combining the driving instruction text and the second navigation information can consider various factors such as road attributes and traffic rules more carefully and comprehensively, generate a more detailed and optimized global driving path, which is beneficial to avoiding the situation where the planned global driving path does not match the actual road conditions, and improving the vehicle driving experience and safety.

[0008] Optionally, based on the road image, adjusting the auxiliary road data to obtain target road data includes: identifying the road image to obtain the road topology, lane information, the first confidence corresponding to the road topology, and the second confidence corresponding to the lane information in the road image, where the first confidence is used to characterize the accuracy of the road topology, and the second confidence is used to characterize the accuracy of the lane information; based on the road topology and the first confidence, adjusting the auxiliary road topology in the auxiliary road data to obtain the target road topology, and based on the lane information and the second confidence, adjusting the auxiliary lane information in the auxiliary road data to obtain the target lane information; constructing the target road data based on the target lane information and the target road topology.

[0009] The above optional embodiments of the present application can achieve the following beneficial effects: By identifying road images to obtain road topologies, lane information, and their corresponding confidence levels, the accuracy of auxiliary road data can be evaluated and adjusted, effectively improving the authenticity and reliability of target road data. Based on the first confidence level corresponding to the road topology and the second confidence level corresponding to the lane information, the auxiliary road topology and auxiliary lane information can be adjusted in a timely manner to obtain accurate target road topologies and target lane information. Thus, based on the target lane information and target road topology, more accurate target road data can be obtained, providing a basis for determining the subsequent global driving path.

[0010] Optionally, based on the road topology and the first confidence level, the auxiliary road topology in the auxiliary road data is adjusted to obtain the target road topology, and based on the lane information and the second confidence level, the auxiliary lane information in the auxiliary road data is adjusted to obtain the target lane information, including: inputting the auxiliary road topology, road topology, and the first confidence level into the global path planning model to obtain the target road topology, and inputting the auxiliary lane information, lane information, and the second confidence level into the global path planning model to obtain the target lane information.

[0011] The above optional embodiments of the present application can achieve the following beneficial effects: The first confidence level and the second confidence level can adjust the auxiliary road topology and auxiliary lane information. By inputting the auxiliary road topology, road topology, the first confidence level, auxiliary lane information, lane information, and the second confidence level into the pre-trained global path planning model, accurate target lane information can be obtained. Based on the accurate target lane information, an accurate global driving path can be obtained. Especially in the case where the road topology changes or the lane information is unclear, a more accurate global driving path can be planned based on information with a higher confidence level, thereby improving the vehicle's global driving path planning ability and ensuring the safety of vehicle driving.

[0012] Optionally, based on the driving instruction text, the second navigation information, and the target road data, the global driving path of the vehicle is planned, including: determining the stationary objects in the preset driving path based on the road image; inputting the second navigation information, driving instruction text, stationary objects, and target road data into the global path planning model to obtain the global driving path.

[0013] The above optional embodiments of the present application can achieve the following beneficial effects: Determine the stationary objects in the preset driving path, and input the stationary objects, the second navigation information, the driving instruction text, and the target road data into the global path planning model. By using the pre-trained global path planning model, the global driving path can be more accurately evaluated and planned, avoiding path planning errors caused by insufficient or biased information, ensuring that road condition elements are fully considered during the path planning process, and improving the comprehensiveness of path planning.

[0014] Optionally, based on the obstacle information in the global driving path and the global driving path, determine the local driving path of the vehicle in the driving direction, including: based on the obstacle information, the global driving path, and the stationary objects, determine the local driving path of the vehicle and the driving speed of the vehicle when driving on the local driving path.

[0015] The above optional embodiments of the present application can achieve the following beneficial effects: By analyzing the obstacle information in the global driving path and combining the position distribution of the stationary objects, determine the local driving path that can effectively avoid obstacles, thereby improving the driving safety of the vehicle when encountering obstacles and the rationality of path planning. That is to say, after determining the global driving path, the method can further optimize the global driving path based on the obstacle information (which can be used to determine dynamic obstacles) and the stationary objects to determine the local driving path of the vehicle within a certain range ahead. Through the precise analysis of the obstacles in the driving path (which can include stationary obstacles and dynamic obstacles), the local driving path can be optimized and planned, and the driving speed in the local driving path can be determined to reduce unnecessary deceleration or waiting time, thereby improving the driving efficiency of the vehicle.

[0016] Optionally, based on the obstacle information, the global driving path, and the stationary objects, determine the local driving path of the vehicle and the driving speed of the vehicle when driving on the local driving path, including: based on the obstacle information, the global driving path, the stationary objects, and the target lane information, construct an initial local driving path; based on the road topology and lane information, adjust the initial local driving path to obtain the local driving path and the driving speed.

[0017] The above optional embodiments of the present application can achieve the following beneficial effects: By considering the obstacle information and stationary objects on the global driving path, the changes in the surrounding environment can be more accurately understood and predicted to obtain an initial local driving path; then, based on the road topology and lane information collected from the road image, the initial local driving path is optimized and adjusted, thereby generating a safer and more reasonable local driving path and driving speed to ensure that the vehicle can continue to drive safely. That is, after obtaining the initial local driving path, it is possible to further optimize the initial local driving path based on the road topology and lane information to avoid abnormal situations in the planned initial local driving path due to inaccurate recognition, thereby obtaining an accurate local driving path and driving speed, achieving the purpose of improving the accuracy of driving path planning.

[0018] Optionally, the method further includes: obtaining image data in the global driving path collected by an image acquisition device; identifying the image data to obtain obstacle information.

[0019] The above optional embodiments of the present application can achieve the following beneficial effects: By using the image acquisition device to real-time obtain the image data in the global driving path, the obstacle information on the road can be quickly identified using the image data. The obstacle information can be used to determine the dynamic obstacles on the road, thereby providing timely road condition information for the vehicle and enhancing the response ability to emergencies. Identifying the image data to obtain obstacle information can plan the driving path to avoid obstacles in advance, reduce the collision risk, and improve the driving safety of the vehicle.

[0020] Optionally, obtaining the first navigation information during the driving process of the vehicle includes: calling the voice broadcast interface and the visualization display interface in the vehicle; using the voice broadcast interface to collect the voice information of the driving instruction, and using the visualization display interface to collect the display content of the visualization interface in the vehicle; converting the voice information of the driving instruction to obtain the driving instruction text, and converting the display content to a navigation image including auxiliary road data; constructing the first navigation information based on the navigation image and the driving instruction text.

[0021] The above optional embodiments of the present application can achieve the following beneficial effects: By calling the voice broadcast interface and the visualization display interface, the voice information and the display information of the visualization interface can be collected simultaneously. Converting the above information to the driving instruction text and the navigation image can construct a more comprehensive first navigation information, which not only improves the accuracy and timeliness of the first navigation information, but also enhances the driving experience of the user. Especially in the case of complex navigation instructions or emergencies, the provision of multi-modal information can reduce the time for the user to understand the instructions and increase the driving safety of the vehicle.

[0022] Optionally, determining the second navigation information corresponding to the preset driving path includes: determining the second navigation information corresponding to the preset driving path based on the navigation map including the preset driving path.

[0023] The above optional embodiments of the present application can achieve the following beneficial effects: Since the navigation map can include information related to the preset driving path, such as road attributes, speed limit information, navigation actions, etc., based on the navigation map including the preset driving path, the second navigation information corresponding to the preset driving path can be determined. This second navigation information can provide a detailed data basis for driving path planning, thereby improving the accuracy and comprehensiveness of the vehicle's planned driving path.

[0024] According to one embodiment of the present application, there is also provided a control device for a vehicle, including: an acquisition unit for acquiring first navigation information during the driving of the vehicle, where the first navigation information is used to at least determine the preset driving path of the vehicle; a processing unit for determining the second navigation information corresponding to the preset driving path and collecting road images corresponding to the preset driving path, where the second navigation information is used to at least represent the driving instruction information corresponding to the vehicle when driving on the preset driving path; a first determination unit for determining the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road images; a second determination unit for determining the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path, where the obstacle information is used to determine the obstacles in the global driving path that are in a moving state; and a control unit for controlling the vehicle to drive according to the local driving path.

[0025] Optionally, the first determination unit may further include: an identification module for identifying the first navigation information to obtain the driving instruction text and the auxiliary road data of the preset driving path; an adjustment module for adjusting the auxiliary road data based on the road images to obtain the target road data; and a planning module for planning the global driving path of the vehicle based on the driving instruction text, the second navigation information, and the target road data.

[0026] Optionally, the adjustment module may further include: an identification subunit, configured to identify a road image to obtain road topology, lane information, a first confidence level corresponding to the road topology, and a second confidence level corresponding to the lane information in the road image, where the first confidence level is used to represent the accuracy of the road topology, and the second confidence level is used to represent the accuracy of the lane information; a first adjustment subunit, configured to adjust the auxiliary road topology in the auxiliary road data based on the road topology and the first confidence level to obtain a target road topology, and adjust the auxiliary lane information in the auxiliary road data based on the lane information and the second confidence level to obtain target lane information; a first construction subunit, configured to construct target road data based on the target lane information and the target road topology.

[0027] Optionally, the first adjustment subunit may include: an input sub-module, configured to input the auxiliary road topology, the road topology, and the first confidence level into a global path planning model to obtain a target road topology, and input the auxiliary lane information, the lane information, and the second confidence level into the global path planning model to obtain target lane information.

[0028] Optionally, the second determination unit may include: a first determination module, configured to determine a local driving path of the vehicle and a driving speed of the vehicle on the local driving path based on obstacle information, a global driving path, and stationary objects.

[0029] Optionally, the first determination module may include: a second construction subunit, configured to construct an initial local driving path based on obstacle information, the global driving path, stationary objects, and target lane information; a second adjustment subunit, configured to adjust the initial local driving path based on the road topology and the lane information to obtain a local driving path and a driving speed.

[0030] Optionally, the apparatus may further include: a second acquisition unit, configured to acquire image data in the global driving path acquired by an image acquisition device; an identification unit, configured to identify the image data to obtain obstacle information.

[0031] Optionally, the acquisition unit may further include: a call module, configured to call a voice broadcast interface and a visualization display interface in the vehicle; a collection module, configured to collect driving instruction voice information by using the voice broadcast interface and collect the display content of a visualization interface in the vehicle by using the visualization display interface; a conversion module, configured to convert the driving instruction voice information to obtain a driving instruction text, and convert the display content into a navigation image including auxiliary road data; a construction module, configured to construct first navigation information based on the navigation image and the driving instruction text.

[0032] Optionally, the processing unit may further include: a second determination module, configured to determine second navigation information corresponding to a preset driving route based on a navigation map including the preset driving route.

[0033] According to another aspect of the embodiments of the present application, there is provided an electronic device, including a processor; a memory for storing instructions executable by the processor; wherein, the processor is configured to execute the instructions to implement the above method.

[0034] According to another aspect of the embodiments of the present application, there is provided a computer-readable storage medium, in which a computer program is stored, wherein the computer program is set to execute the above method when being run by a processor.

[0035] According to another aspect of the embodiments of the present application, there is provided a computer program product, including a computer program, which implements the above method when being executed by a processor.

[0036] According to another aspect of the embodiments of the present application, there is provided a vehicle, including an in-vehicle processor and an in-vehicle memory, the in-vehicle memory for storing a computer program; the in-vehicle processor for executing the computer program stored on the memory to implement the above method.

[0037] It should be noted that the above general description and the following detailed description are only for exemplifying and explaining the present application, and do not constitute a limitation to the present application. Description of the Drawings

[0038] Figure 1 is a flowchart of a vehicle control method provided by an embodiment of the present application;

[0039] Figure 2 is a schematic diagram of a navigation visualization provided by an embodiment of the present application;

[0040] Figure 3 is a schematic diagram of an autonomous driving system integrating navigation voice broadcast and intersection visibility provided by an embodiment of the present application;

[0041] Figure 4 is a structural diagram of a vehicle control device provided by an embodiment of the present application;

[0042] Figure 5 is a structural diagram of an electronic device provided by an embodiment of the present application;

[0043] Figure 6 is a structural diagram of a vehicle control device provided by an embodiment of the present application;

[0044] Figure 7 is a structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0045] In order to make the technical problems, technical solutions and beneficial effects solved by the present application clearer, the present application will be further described in detail below with reference to the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0046] Currently, navigation maps are usually used to implement functions such as vehicle positioning, route planning, and navigation. Among them, the navigation map contains basic information of roads, such as the location, direction, number of lanes, traffic signs, speed limits, and important geographical features of the roads. Based on the above basic information, the vehicle can determine its own position and plan a suitable route from the starting point to the ending point. However, on the one hand, the information update of the navigation map usually lags behind the changes in the actual road conditions, such as road construction, closure, newly opened roads, etc. The above situations may lead to inaccurate planned routes or even potential safety hazards. On the other hand, the information provided by the navigation map is usually limited to the visible road network, and its ability to perceive dynamic targets such as temporary obstacles, pedestrians, and non-motor vehicles on the road is limited, especially in intersections, tunnels, or areas with blocked visibility. Therefore, in the face of a rapidly changing traffic environment and complex urban road conditions, the accuracy of the vehicle's driving route planning is low, resulting in low safety of the vehicle's planned route.

[0047] In the related art, a real-time dynamic intelligent path planning method based on multi-sensor information fusion is proposed. This method can perceive the environment based on multi-sensor information fusion, optimize the global feasible path search and trajectory, dynamically adjust the planned path according to the situation perceived by the multi-sensors in real time, and perform real-time navigation and maneuver guidance according to the planned path, so as to quickly and accurately extract dynamic real-time information such as real-time weather, geology, and other natural disasters and emergencies in the environment, and can quickly process these dynamic information to obtain unknown static obstacles and unknown dynamic obstacles in the map. This method realizes the detection ability between unknown obstacles all day long through the learning mapping between the dynamic information in the environment obtained by multi-modal sensors such as vision, lidar, inertial, millimeter-wave radar, and satellite signals. However, the above method focuses on detecting unknown obstacles through multi-sensor information fusion. The design and implementation of the information fusion algorithm are very complex and require a highly accurate calibration and synchronization mechanism to ensure that the information from different sensors can be correctly interpreted and integrated. If the information obtained by different sensors is incorrect, it will affect the accuracy of the path planning, resulting in the technical problem of low safety of the vehicle's planned route.

[0048] In view of the above problems, an embodiment of the present application provides a vehicle control method, which may include: obtaining first navigation information of the vehicle during driving, where the first navigation information is used to at least determine a preset driving route of the vehicle; determining second navigation information corresponding to the preset driving route, and collecting road images corresponding to the preset driving route, where the second navigation information is used to at least represent driving instruction information corresponding to the vehicle when driving in the preset driving route; determining a global driving route of the vehicle based on the first navigation information, the second navigation information, and the road images; determining a local driving route of the vehicle in the driving direction based on the obstacle information in the global driving route and the global driving route, where the obstacle information is used to determine obstacles in a moving state in the global driving route; and controlling the vehicle to drive according to the local driving route.

[0049] The vehicle control method provided by the embodiment of the present application achieves the following technical effects: By obtaining the first navigation information of the vehicle during driving, a preset driving route can be determined. After determining the preset driving route, in combination with the first navigation information, the second navigation information corresponding to the preset driving route, and the actual road images, a global driving route can be determined more comprehensively and accurately. According to the global driving route and the obstacle information in the global driving route, a local driving route can be determined, so as to control the vehicle to drive according to the local driving route, enabling the vehicle to effectively avoid obstacles in a moving state (dynamic obstacles) and obstacles in a stationary state (static obstacles) in the case of obstacle information or different road condition information, reducing potential collision risks, thus solving the technical problem of low safety of the vehicle-planned route and achieving the technical effect of improving the safety of the vehicle-planned route.

[0050] Embodiment 1

[0051] An embodiment of the present application provides a vehicle control method. Please refer to Figure 1 , Figure 1 which is a flowchart of a vehicle control method provided by an embodiment of the present application, including the following steps:

[0052] S110: Obtain first navigation information of the vehicle during driving.

[0053] In step S110, the above first navigation information may be enhanced navigation information, also known as navigation enhancement information, which can be used to at least determine the preset driving path of the vehicle and may include navigation voice broadcast information and navigation visualization information. The above navigation voice broadcast information may be voice command information conveyed to the driver in voice form, which can be used to provide detailed descriptions of instructions such as the road conditions ahead, turning prompts, road attributes, speed limits, navigation actions, etc. during the vehicle's driving process. For example, the navigation voice broadcast information may be "There is a long solid line ahead. Please shift gears in advance". The above navigation voice broadcast information can be used to determine the upcoming road changes of the vehicle, so as to prepare for controlling the vehicle. The above navigation visualization information can be presented in the form of a graphical interface, such as a dynamic map interface, to display the preset driving path of the vehicle. The dynamic map interface may be a visualization interface containing the preset driving path, which can be used to display the actual situation of the road ahead, including the number of lanes, road attributes, and traffic signals and signs, etc., to provide visual navigation assistance to the driver. Through the navigation voice broadcast information and the navigation visualization information, the vehicle's perception ability of the environment can be enhanced, providing a basis for more accurate driving path planning.

[0054] Optionally, a pre-planned preset driving path can be generated by the in-vehicle navigation system in the vehicle. Therefore, the above first navigation information generated based on the preset driving path can be obtained.

[0055] For example, when the vehicle enters a complex urban section, the in-vehicle navigation system analyzes the high-precision map data and identifies that there will be a long solid line area ahead. Considering that lane changes are not allowed in the solid line area, the driver can be notified in advance through the navigation voice broadcast information: "There is a long solid line ahead. Please change lanes in advance". The dynamic map interface of the driving path can also be displayed through the navigation visualization information, such as visual display through a large intersection map. This is only an example for illustration, and there are no specific restrictions on the acquisition method and content of the first navigation information. As long as it is navigation enhancement information that can be used to determine the preset driving path of the vehicle, it is within the protection scope of the embodiments of the present application.

[0056] S120: Determine the second navigation information corresponding to the preset driving path, and collect the road image corresponding to the preset driving path.

[0057] In step S120, the above second navigation information can be used to at least represent the driving instruction information corresponding to the vehicle when driving on a preset driving path. It can be basic navigation information, which is the basic data when the vehicle's automatic driving system plans a path. In addition to the driving instruction information, the above second navigation information can also include information in multiple dimensions. For example, it can include road attribute information, intersection lane attribute information, etc. The above driving instruction information can be navigation instruction information, which can be used to determine information such as the driving direction and distance of the vehicle. The image content of the above road image can at least include at least one static object on the road corresponding to the preset driving path. For example, images of intersection layouts, road signs, traffic lights, lane markings, markings, building outlines, etc. can be collected by collection devices such as radars and cameras.

[0058] Optionally, the above road attribute information can be used to determine the road type (such as highway, urban road, rural road), road width, road surface material, etc. By determining the road attribute information corresponding to the preset driving path, it can provide a basis for the subsequent determination of the global driving path. The above navigation instruction information can include text or voice instructions issued by the vehicle navigation system, such as driving instruction information like "turn left", "go straight", "exit the highway", etc. The above intersection lane attribute information can be used to determine the lane attributes of the vehicle at the intersection, such as the number of lanes, the direction of the lanes (such as straight, left turn, right turn), the type of lane lines, and whether there are dedicated lanes (such as bus lanes, bicycle lanes), etc.

[0059] It should be noted that the road attribute information, navigation instruction information, and intersection lane attribute information in the above second navigation information are only for illustrative purposes, and the content of the above second navigation information is not specifically limited here. As long as it can be used to determine the navigation information corresponding to the preset driving path, it is within the protection scope of the embodiments of the present application.

[0060] Optionally, the above road image can be obtained through an in-vehicle camera, or through a high-precision map image obtained by pre-downloading or through an online service, or through image collection devices (such as cameras, radars, etc.) on the road. It should be noted that this is only for illustrative purposes, and the acquisition method of the road image is not specifically limited.

[0061] Optionally, the above second navigation information may be information pre-planned by the navigation system based on the starting point and the ending point of the vehicle. For example, when the vehicle is about to drive, the user inputs the starting point and the ending point, and the navigation system in the vehicle can plan the preset driving path of the vehicle, as well as driving instruction information on the preset driving path and other information. Based on the map data, information such as road attribute information and intersection lane attribute information corresponding to the preset driving path can be obtained. In advance, information such as road attribute information, navigation instruction information, and intersection lane attribute information corresponding to the preset driving path can be determined as the second navigation information. After determining the preset driving path based on the first navigation information, the second navigation information corresponding to the preset driving path can be obtained.

[0062] Optionally, the second navigation information may include navigation instruction information missing in the first navigation information. That is, the first navigation information only includes navigation voice broadcast information (i.e., voice instruction information), but in addition to the navigation voice broadcast information, the second navigation information also includes other non-voice instruction information. For example, the instruction information presented only in text form can be text information such as "There is a long solid line ahead. Please change lanes to the right in advance" displayed on the display interface.

[0063] Optionally, after determining the preset driving path, the second navigation information corresponding to the preset driving path can be determined according to the preset driving path based on the navigation map. The second navigation information can be used to determine the basic data when the vehicle is driving on the preset driving path, such as road attribute information, navigation instruction information, intersection lane attribute information, etc.

[0064] In this embodiment, through the second navigation information, information such as the road topology and lane topology in the preset driving path, as well as the driving process of the preset driving path, can be determined. By collecting road images, the road conditions of the preset driving path can be obtained more accurately, providing a basis for determining the global driving path of the subsequent vehicle. Obtaining the second navigation information and road images is an important support for the vehicle's automatic driving system to perform in-depth planning and decision-making based on the preset driving path, jointly ensuring that the vehicle can drive safely and efficiently on complex and changeable roads.

[0065] S130: Determine the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image.

[0066] In step S130, the above global driving path refers to the driving path planned throughout the entire journey from the starting point to the ending point of the vehicle, which may include the driving path that the vehicle should follow (for example, which road and which direction to drive).

[0067] Optionally, in addition to determining the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image, this embodiment can also determine the driving behavior of the vehicle on the global driving path based on the first navigation information, the second navigation information, and the road image. For example, information such as when to decelerate, when to accelerate, when to change lanes, and the expected driving sequence and time.

[0068] Optionally, by recognizing the road image collected by an in-vehicle camera or other visual sensors, perception information such as road topology, lane topology, and static targets can be obtained. Through navigation voice broadcast information and navigation visualization information, a dynamic map interface regarding the road conditions ahead during the vehicle's driving process and the vehicle's preset driving path can be provided. Based on the first navigation information, the second navigation information, and the road image, the global driving path of the vehicle can be determined. In the above process of planning the global path, not only information such as road topology and lane topology is considered, but also the first navigation information (such as navigation voice broadcast, visualization information) and the second navigation information (such as road attribute information, navigation instruction information, intersection lane attribute information) are combined to plan the global driving path. The planning of the global driving path based on comprehensive information can thus more effectively avoid congested sections, construction areas, and roads affected by emergencies.

[0069] In the related art, usually only simply relying on the map, the driving path of the vehicle is constructed based on the starting position and the ending position. However, the urban scene has a wider coverage range and a faster update speed than the highway scene. For lanes with changes, this method has the technical problem of low safety of the planned path. To solve this technical problem, in this application, the first navigation information of the preset driving path planned in advance is obtained, and the second navigation information and the road image of the preset driving path are obtained. Based on the first navigation information and the road image, a virtual intersection corresponding to the preset driving path is constructed. This virtual intersection can determine the preset driving path and the road topology, lane topology, and static targets associated with the preset driving path. Further, based on the second navigation information and the obtained road topology, an accurate global driving path can be planned. Since this embodiment can optimize the road topology corresponding to the preset driving path in a timely manner based on the first navigation information and the road image, the problem of abnormal driving path planning caused by inaccurate road information is avoided, and thus the technical problem of low safety of the vehicle-planned path is solved, achieving the technical effect of improving the safety of the vehicle-planned path.

[0070] S140: Determine the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path.

[0071] In step S140, the above obstacle information may be dynamic obstacles, such as other vehicles, pedestrians, bicycles, animals during driving, and suddenly appearing obstacles (such as fallen objects, roadblocks for temporary road closures), etc., which can be used to determine the obstacles in a moving state in the global driving path. This is only for illustrative purposes, and there is no specific limitation on the type of obstacle information. The above local driving path is a short-term goal or subset path when the vehicle executes the global driving path planning. For example, it may include the driving path of the vehicle within the next short journey. In addition to determining the local driving path of the vehicle in the driving direction based on the obstacle information and the global driving path in the global driving path, the driving operations for determining the local driving path of the vehicle in the driving direction can also be determined based on the obstacle information and the global driving path in the global driving path. The driving operations may include acceleration operations, deceleration operations, lane-changing driving, etc. For example, the driving speed of the vehicle can be adjusted according to the distance and driving state of the obstacle in front of the vehicle (such as whether to decelerate, whether to stop), and lane changes can be made in advance or within the local driving path to avoid obstacles. This is only for illustrative purposes, and there is no specific limitation on the driving operations. As long as the driving operations can be used to determine the local driving path of the vehicle in the driving direction to avoid obstacle information, they are within the protection scope of the embodiments of the present application.

[0072] In this embodiment, during the driving process of the vehicle, according to the obstacle information obtained in real time and the global driving path, a local driving path that can avoid obstacles in the current driving direction can be obtained. The local driving path can guide the driving direction, speed, acceleration and deceleration control, and necessary lane-changing or obstacle avoidance actions of the vehicle in the future for a period of time. The local driving path is dynamic and can be updated in real time according to the changes in the road section during the driving process of the vehicle, so as to ensure the safe and smooth driving of the vehicle.

[0073] Optionally, the obstacle information can be obtained through on-vehicle sensors, such as lidar, millimeter-wave radar, cameras, etc. This is only for illustrative purposes, and there is no specific limitation on the acquisition method of the obstacle information.

[0074] Optionally, the future behavior of the dynamic obstacle is predicted, such as predicting the movement path, speed change, etc. of the dynamic obstacle. Based on the future behavior of the dynamic obstacle, the global driving path of the vehicle can be adjusted to obtain the local driving path, so that the vehicle can react in advance and avoid potential collision risks.

[0075] In an embodiment of the present application, on the basis of the second navigation information, navigation voice broadcast information and navigation visualization information are added. The navigation visualization information can detail the road topology and lane topology of a preset driving route. Based on the navigation voice broadcast information, navigation visualization information, and obstacle information, the road topology and lane topology in the navigation visualization information can be recognized. Fusing and matching the recognized road topology, lane topology, and the road image corresponding to the collected preset driving route can improve the perception ability of the road environment. For example, in places with recognition blind spots or errors such as intersections, complex intersections, and out-of-sight scenarios, the road can be accurately recognized. The road image can also be used as a comparison to improve the accuracy of the recognized road topology, solve the problems of occlusion in visual perception or limitations in perception performance, and thus improve the passing ability of the vehicle at intersections. At the same time, the global driving route planning ability is improved through the navigation voice broadcast information and navigation visualization information.

[0076] S150: Control the vehicle to travel according to the local driving route.

[0077] In step S150, the above local driving route can be a driving route planned based on the current position of the vehicle and a short-range target. For example, it can be the driving route within a few minutes or a few hundred meters ahead of the vehicle. Local driving route planning can control the vehicle to avoid dynamic obstacles such as pedestrians, other vehicles, and unexpected roadblocks; it can ensure that the vehicle complies with traffic rules when the vehicle encounters traffic lights, stop signs, speed limit signs, etc.; it can control the vehicle to adapt to road conditions, such as driving strategies under specific road conditions like lane changes, turns, uphill and downhill.

[0078] For example, an autonomous vehicle is at an urban intersection. According to the global driving route, the vehicle needs to turn left after passing through the intersection. However, the traffic at the intersection is complex, with pedestrians, bicycles, and other vehicles passing by, and at the same time, the traffic light shows a yellow light and is about to turn red. At this time, due to the yellow light about to turn red, according to the local driving route, the vehicle can be controlled to decelerate and prepare to stop before the red light comes on; because it is detected that there is a pedestrian crossing the road ahead, according to the local driving route, the safe distance to avoid the pedestrian can be calculated, and the driving direction and speed of the vehicle can be adjusted to ensure that no collision occurs; when the traffic signal turns green, the vehicle needs to turn left. According to the local driving route, the vehicle can be controlled to change lanes to the left lane in advance and safely turn at an appropriate speed during the green light. After the vehicle turns, the vehicle can be controlled to keep driving in the left lane until the next obstacle appears.

[0079] Optionally, the vehicle is controlled to travel according to the local travel path. Through real-time control and adjustment, the safety and travel efficiency of the vehicle in a complex environment are ensured. Through the above process, the vehicle can flexibly respond to traffic signals, obstacles, and the behaviors of other vehicles, make timely responses, and achieve smooth and safe travel goals.

[0080] Based on the above steps S110 to S150, by obtaining the first navigation information during the vehicle's travel, where the first navigation information is used to at least determine the preset travel path of the vehicle; the second navigation information corresponding to the preset travel path can be determined, and the road image corresponding to the preset travel path is collected, where the second navigation information is used to at least represent the travel instruction information corresponding to the vehicle when traveling in the preset travel path; based on the first navigation information, the second navigation information, and the road image, the global travel path of the vehicle can be determined; based on the obstacle information in the global travel path and the global travel path, the local travel path of the vehicle in the travel direction can be determined, where the obstacle information is used to determine the obstacles in the global travel path that are in a moving state; the vehicle can be controlled to travel according to the local travel path. That is, in the embodiment of the present application, the preset travel path is determined by obtaining the first navigation information. For the preset travel path, the second navigation information and the corresponding road image are collected, and the first navigation information is fused to determine the global travel path. After determining the global travel path, the local travel path is determined according to the real-time obstacle information, and the vehicle is made to move forward along the established travel path following the local travel path planning. Through the high degree of intelligence and flexibility in information fusion, path planning, and dynamic control in the present application, the safe travel of the vehicle can be ensured, thus solving the technical problem of the low safety of the path planned by the vehicle and achieving the technical effect of improving the safety of the path planned by the vehicle.

[0081] Next, the above method of this embodiment will be further introduced.

[0082] As an optional embodiment, step S130 includes: identifying the first navigation information to obtain the travel instruction text and the auxiliary road data of the preset travel path; adjusting the auxiliary road data based on the road image to obtain the target road data; planning the global travel path of the vehicle based on the travel instruction text, the second navigation information, and the target road data.

[0083] In this embodiment, by identifying the first navigation information, the driving instruction text and the auxiliary road data of the preset driving path can be obtained. Among them, the above-mentioned driving instruction text can be instruction information and can be used to determine the driving direction, such as "go straight", "turn left", "turn right", etc.; the above-mentioned auxiliary road data can include road topology, lane information of the current road, and global navigation guidance information. Based on the road image, the auxiliary road data can be adjusted to obtain the target road data. Based on the driving instruction text, the second navigation information, and the target road data, the global driving path of the vehicle can be planned.

[0084] Optionally, receive the first navigation information, which can include navigation voice broadcast information in the text file (abbreviated as txt) format and navigation visualization information in the Joint Photographic Experts Group (abbreviated as JPG) format. The driving instruction text can be extracted from the navigation voice broadcast information in the txt format, such as clear instruction information like "go straight and then turn right ahead" and "drive in the second lane on the left". At the same time, by analyzing the navigation visualization information in the JPG format, the auxiliary road data of the preset driving path can be identified, including road topology, lane information of the current road, and global navigation guidance information. The global navigation guidance information can be used for the detailed guidance of the entire driving path of the vehicle from the starting point to the ending point, including the driving route (for example, the recommended route from the current location to the destination, including the use of main roads and secondary roads), navigation actions (such as action instructions like going straight, turning, and U-turning, as well as the execution timing and location of each action), and road attributes (such as speed limit, road type, special traffic signs, and real-time traffic conditions), etc. It should be noted that only examples are given here, and the content of the global navigation guidance information is not specifically limited.

[0085] Optionally, after obtaining the driving instruction text and the auxiliary road data of the preset driving path, the auxiliary road data can be further corrected and refined by using the road image. For example, through image recognition technology, verify or adjust the accuracy of the road topology, lane information of the current road, and global navigation guidance information, so as to obtain more real and reliable target road data.

[0086] Optionally, based on the driving instruction text, the second navigation information (such as road attributes, navigation instruction information, intersection lane attribute information, etc.) and the target road data, a global driving path of the vehicle can be generated through a path planning algorithm. For example, if the first navigation information indicates "turn left into Street B 300 meters ahead", and the road image shows that the intersection is under construction, the construction area can be identified from the road image, the road topology in the auxiliary road data can be adjusted, and then combined with the road attributes in the second navigation information, the driving path can be re-planned to choose a detour to Street C, and finally a global driving path that avoids the construction area and follows traffic rules can be generated.

[0087] In the embodiment of the present application, by identifying the first navigation information, the driving instruction text and the auxiliary road data of the preset driving path are obtained. Based on the road image, the auxiliary road data can be adjusted to obtain accurate target road data. Thus, even if the target road data is adjusted, it can be accurately identified. Based on the accurate target road data, an accurate global driving path can be planned, thereby effectively improving the driving safety of the vehicle under complex road conditions.

[0088] As an optional embodiment, based on the road image, adjusting the auxiliary road data to obtain target road data includes: identifying the road image to obtain the road topology, lane information, the first confidence level corresponding to the road topology, and the second confidence level corresponding to the lane information in the road image, where the first confidence level is used to represent the accuracy of the road topology, and the second confidence level is used to represent the accuracy of the lane information; adjusting the auxiliary road topology in the auxiliary road data based on the road topology and the first confidence level to obtain the target road topology, and adjusting the auxiliary lane information in the auxiliary road data based on the lane information and the second confidence level to obtain the target lane information; constructing the target road data based on the target lane information and the target road topology.

[0089] In this embodiment, the road topology can be the structural layout of the road network, which may include the connection mode, bifurcation, merging, intersection points, etc. of the roads. The lane information can be the specific data about the lanes on the road, which may include lane topology and lane attribute information. For example, the number of lanes, width, lane markings (such as solid lines, dashed lines), driving directions of the lanes, etc. The first confidence level can be used to represent the reliability of the road topology recognized from the road image, and it can be a value between 0 and 1. 1 can represent complete certainty, and 0 can represent complete uncertainty. The higher the first confidence level, the more consistent the road topology in the road image obtained by recognizing the road image is with the actual situation; conversely, if the first confidence level is low, there may be problems and it is necessary to cross-verify or adjust with other information sources. The second confidence level is similar to the first confidence level and is also a value reflecting the reliability of the lane information recognized from the road image, which can be used to represent the accuracy of the lane information recognition. A high second confidence level means that the probability of correctly recognizing the lane information is very high, while a low second confidence level indicates that there may be errors in the lane information in the road image obtained by recognizing the road image, and further confirmation or correction is required.

[0090] Optionally, road images in a preset driving path are collected through image acquisition devices deployed on the road or in the vehicle. The collected road images can be recognized through a static perception model (which can be a neural network model), and perception information (also known as the recognition result) such as the road topology, lane topology, and lane attribute information in the road image is output. After the recognition stage ends, in order to evaluate the accuracy of the recognition result, the first confidence level corresponding to the road topology (the accuracy of the road topology) and the second confidence level corresponding to the lane information (the accuracy of the lane information) can be given.

[0091] Optionally, based on the road image recognition result and the corresponding confidence level, the auxiliary road data can be dynamically adjusted. For example, if the first confidence level recognized from the road image is higher than the confidence level threshold, the road topology information in the auxiliary road data can be adjusted accordingly; if the second confidence level is also higher than the confidence level threshold, the auxiliary lane information can be updated. Only when the recognition result is determined to be reliable will the auxiliary road data be modified, thus avoiding mistakes in the driving path planning caused by incorrect real-time data.

[0092] Optionally, through the above adjustment process, a target road topology and target lane information that are more in line with the actual conditions of the current road can be obtained compared with the original auxiliary road data. Based on the target road topology and target lane information, target road data can be obtained. The target road data can not only include the geometric shape of the road and the distribution of the lanes, but also contain information such as real-time traffic conditions and road attributes, providing a more accurate and detailed basis for subsequent driving path planning and vehicle control.

[0093] Optionally, the recognition result of the static perception model and the virtual intersection are fused. When the virtual intersection is consistent with the road topology recognized by perception, direct fusion can be performed. When the virtual intersection is inconsistent with the road topology recognized by perception, it can be judged in combination with the confidence of the road topology output by perception (the first confidence). If the confidence of the perceived road topology is high, the perceived road topology can be adopted without fusion. When the confidence of the road topology output by perception is low, fusion can be performed with reference to the virtual intersection. When the virtual intersection is consistent with the road topology recognized by perception but the number of lanes or lane attributes are inconsistent, it can be judged according to the lane attribute confidence (the second confidence). Based on the fused road topology and lane information, more accurate target road data can be constructed.

[0094] Optionally, the adjustment of the above first confidence and second confidence can be reasonably optimized through model training.

[0095] In the embodiments of the present application, through real-time road image recognition and different confidence information, the auxiliary road data is enhanced and supplemented, so that based on the constructed target road data, it can provide a basis for improving the accuracy and reliability of the path planning of the subsequent autonomous driving system.

[0096] As an optional embodiment, based on the road topology and the first confidence, the auxiliary road topology in the auxiliary road data is adjusted to obtain the target road topology, and based on the lane information and the second confidence, the auxiliary lane information in the auxiliary road data is adjusted to obtain the target lane information, including: inputting the auxiliary road topology, road topology, and the first confidence into the global path planning model to obtain the target road topology, and inputting the auxiliary lane information, lane information, and the second confidence into the global path planning model to obtain the target lane information.

[0097] In this embodiment, the auxiliary road topology, road topology, and the first confidence can be input into the global path planning model to obtain the target road topology, and the auxiliary lane information, lane information, and the second confidence can be input into the global path planning model to obtain the target lane information. The above global path planning model can be called a navigation module, which can include a fusion module and a navigation sub-module, and can be a model constructed based on a Transformer, or a model constructed based on other neural network models. The above fusion module can fuse the auxiliary road data with the perception information. Based on the fusion module, the navigation sub-module can further perform driving path planning, and generate specific navigation instructions and driving paths based on the target road topology and target lane information.

[0098] It should be noted that the above global path planning model is only for illustration, and no specific restrictions are imposed on the type of the global path planning model here. As long as it can use the auxiliary road topology, road topology, auxiliary lane information, lane information, first confidence level, and second confidence level as inputs to obtain the target road topology and target lane information, it is within the protection scope of the embodiments of the present application.

[0099] Optionally, by considering the auxiliary road topology, the real-time recognized road topology, and the corresponding first confidence level, the auxiliary road topology can be intelligently adjusted to obtain a more accurate target road topology. Similarly, the global path planning model can also use the auxiliary lane information, lane information, and second confidence level as inputs to optimize the auxiliary lane information, thereby generating the target lane information.

[0100] Optionally, by inputting the road attribute information, navigation instruction information, and intersection lane attribute information in the navigation basic information, the navigation voice broadcast and navigation visualization information in the navigation enhancement information, as well as the road topology, lane topology, and static targets obtained through the static perception model into the navigation module, an optimized road topology and an optimized lane topology can be obtained.

[0101] In the embodiments of the present application, the auxiliary road topology, road topology, and first confidence level are packaged into an input sequence, and the auxiliary lane information, lane information, and second confidence level are processed in the same way. Each of the above sequences is respectively input into the global path planning model based on Transformer. The global path planning model can perform complex sequence-to-sequence conversion according to the characteristics and confidence level information of the input data. After being processed by the global path planning model, the output target road topology and target lane information will include the latest predictions of the road attribute information and lane attribute information, as well as the consideration of the first confidence level and the second confidence level. Based on the obtained target road topology and target lane information, global driving path planning can be performed to ensure that the global driving path not only takes into account the physical attributes of the road but also incorporates the confidence evaluation of the current situation to obtain a safer and more effective driving path.

[0102] As an optional embodiment, based on the driving instruction text, the second navigation information, and the target road data, a global driving path of the vehicle is planned, including: determining the stationary objects in the preset driving path based on the road image; inputting the second navigation information, the driving instruction text, the stationary objects, and the target road data into the global path planning model to obtain the global driving path.

[0103] In this embodiment, based on the collected real-time road images, stationary objects (static obstacles) in the preset driving path can be determined, such as road signs, traffic signals, street lights, parked vehicles by the roadside, etc.; the second navigation information, driving instruction text, stationary objects, and target road data are input into the global path planning model, and the global path planning model can output a global driving path. This global driving path not only considers the basic navigation information but also incorporates the navigation voice broadcast information and navigation visualization information provided by the enhanced navigation information, as well as the static obstacles on the real-time road, thus ensuring the safety of the global driving of the path.

[0104] Optionally, in an intersection scenario, based on the result of fusing the basic navigation information and the enhanced navigation information, the global navigation guidance information, navigation instruction information, and navigation voice broadcast information of the virtual intersection can be used simultaneously to generate a better global navigation planning path. For example, through voice broadcast: Go straight at the intersection ahead and then turn right immediately. Execute the global planning of the intersection and preferably choose the right lane. Before the intersection scenario, the navigation voice broadcast information in txt format can be received, and the road information beyond the line of sight can be extracted to adjust the global planning path in advance.

[0105] In the embodiment of the present application, by determining the stationary objects in the preset driving path and inputting the stationary objects, the second navigation information, the driving instruction text, and the target road data into the global path planning model, due to the fusion of various road data and navigation information, the global path planning model can accurately evaluate and plan the global driving path, avoiding the driving path planning errors caused by insufficient or biased road data and navigation information, ensuring that the road conditions elements are fully considered in the vehicle driving path planning process, and improving the comprehensiveness of the driving path planning.

[0106] As an optional embodiment, step S140 includes: based on the obstacle information in the global driving path and the global driving path, determining the local driving path of the vehicle in the driving direction, including: based on the obstacle information, the global driving path, and the stationary objects, determining the local driving path of the vehicle and the driving speed of the vehicle when driving on the local driving path.

[0107] In this embodiment, based on the obstacle information, the global driving path, and the stationary objects, the local driving path of the vehicle and the driving speed of the vehicle when driving on the local driving path can be determined. That is, based on the static environment information (stationary objects) after road topology optimization, the global driving path, and the dynamic obstacles (obstacle information), the real-time local driving path planning of the vehicle can be carried out.

[0108] Optionally, by fusing information from different sources (obstacle information, global driving path, stationary objects), a comprehensive assessment of the vehicle's surrounding environment can be carried out, including static obstacles (such as buildings, fixed facilities) and dynamic obstacles (such as pedestrians, other vehicles). The fused information mentioned above, along with the optimized road topology and optimized lane topology output by the global path planning model, is input into the local planning model. The local planning model can calculate a safe and effective driving path, which is the result of adjusting based on the immediate obstacles and surrounding environmental factors on the basis of the vehicle's current driving direction. In addition to determining the vehicle's local driving path, the driving speed of the vehicle on the local driving path can also be determined to ensure that the vehicle neither affects traffic efficiency due to too slow speed nor increases the accident risk due to too fast speed.

[0109] For example, an autonomous vehicle is approaching a busy intersection, and the global driving path has been planned, indicating that the vehicle should go straight through the intersection. At this time, through the obstacle information, it is found that there is a bus that suddenly decelerates ahead, and at the same time, based on the road image recognition, there are parked vehicles and some construction fences on the roadside. In this case, the global path planning model can be used to correct the auxiliary road data first to generate the target road data, ensuring that the planned driving path takes into account the newly emerged obstacles mentioned above. Subsequently, based on the corrected global driving path, real-time obstacle information, and known stationary objects, the local planning model can calculate a new local driving path. For example, a driving path that needs to bypass the bus and avoid the construction area, and at the same time, the vehicle driving speed needs to be reduced to ensure safe passage through this temporary complex section.

[0110] In the embodiment of the present application, by fusing the global driving path, obstacle information, and stationary objects, the local driving path and driving speed of the vehicle can be dynamically adjusted, effectively avoiding the vehicle collision risk, realizing the refined control of the vehicle under complex road conditions, and ensuring the safe driving of the vehicle.

[0111] As an optional embodiment, based on the obstacle information, global driving path, and stationary objects, determine the local driving path of the vehicle and the driving speed of the vehicle on the local driving path, including: constructing an initial local driving path based on the obstacle information, global driving path, stationary objects, and target lane information; adjusting the initial local driving path based on the road topology and lane information to obtain the local driving path and driving speed.

[0112] In this embodiment, based on the obstacle information obtained in real time (dynamic obstacles, such as other vehicles, pedestrians, etc.), combined with the global driving path and target lane information, as well as static obstacles (such as parked vehicles on the roadside, construction areas, etc.), an initial local driving path can be constructed. Since the above initial local driving path takes into account the distribution of obstacles in the surrounding environment of the current vehicle, as well as the preset driving direction and lane requirements, it can ensure the safe driving of the vehicle.

[0113] Optionally, after obtaining the initial local driving path, the perception information (such as road topology, lane topology) can be compared with the initial local driving path. If there is no abnormality, there is no need to adjust the initial local driving path and driving speed. If an abnormality is found during the comparison of the perception information and the initial local driving path, for example, the traffic signal in the perception information shows a red light while the red light is not shown in the initial local driving path, the initial local driving path can be adjusted based on the perception information to obtain the local driving path and driving speed.

[0114] In the embodiment of the present application, after obtaining the initial local driving path, in order to improve the accuracy of the local driving path, the local driving path can be secondarily verified and adjusted based on the road topology and lane information determined from the image data. For example, use the road topology and lane information to determine whether there are any unreasonable places in the local driving path. If so, the path can be re-planned, or the unreasonable places can be adjusted based on the road topology and lane information. Or, if it is determined that there are unreasonable places, the target lane information can be re-determined, and a global driving path can be constructed based on the newly determined target lane information to determine the local driving path.

[0115] It should be noted that the present application can directly determine the initial local driving path constructed based on the obstacle information, global driving path, stationary objects, and target lane information as the local driving path. At the same time, after obtaining the initial local driving path, the constructed initial local driving path can also be adjusted based on the perception information (such as road topology and lane topology) to obtain the local driving path and driving speed. The method for determining the local driving path here is only an example and is not specifically limited. As long as the method for determining the local driving path is based on the obstacle information, global driving path, stationary objects, and target lane information, it should be within the protection scope of the present application.

[0116] As an optional embodiment, the method further includes: obtaining the image data in the global driving path collected by the image acquisition device; identifying the image data to obtain the obstacle information.

[0117] In this embodiment, image data in the global driving path is collected by an image acquisition device, such as a camera, a radar, etc.; by identifying the image data, obstacle information can be obtained.

[0118] Optionally, a dynamic perception model can be constructed using a neural network model (such as a convolutional neural network, a Transformer-based model, etc.). The dynamic perception model can perform real-time analysis on the image data collected in the global driving path to identify the obstacle information in the image data, and the obstacle information can be used to determine dynamic obstacles. Dynamic obstacles can include, but are not limited to, moving objects such as pedestrians, other vehicles, animals, etc. that may affect driving safety.

[0119] In the embodiment of the present application, through the collection and analysis of real-time image data, various obstacles appearing on the vehicle driving road can be quickly identified and dealt with, reducing the accident risk and improving the driving safety of the vehicle.

[0120] As an optional embodiment, step S110 includes: obtaining first navigation information during the driving process of the vehicle, including: calling the voice broadcast interface and the visualization display interface in the vehicle; collecting driving instruction voice information using the voice broadcast interface, and collecting the display content of the visualization interface in the vehicle using the visualization display interface; converting the driving instruction voice information to obtain a driving instruction text, and converting the display content into a navigation image including auxiliary road data; based on the navigation image and the driving instruction text, first navigation information is constructed.

[0121] In this embodiment, during the driving process of the vehicle, the voice broadcast interface and the visualization display interface in the vehicle can be called. Among them, the voice broadcast interface can be used to output driving instruction voices in the vehicle navigation system, such as driving instruction voices like "turn left 500 meters ahead", "keep going straight and prepare to exit at the next exit", etc. The visualization display interface can be used to obtain the display content on the visualization interface in the vehicle, including but not limited to auxiliary road data such as maps, road topologies, lane information, etc.; collecting driving instruction voice information using the voice broadcast interface, and collecting the display content of the visualization interface in the vehicle using the visualization display interface; converting the driving instruction voice information can obtain a driving instruction text, and converting the display content can be converted into a navigation image including auxiliary road data; based on the navigation image and the driving instruction text, first navigation information can be constructed.

[0122] Optionally, the voice broadcast content obtained from the voice broadcast interface is converted into a driving instruction text and stored in a txt format for subsequent parsing and use. At the same time, the display content of the vehicle's visualization interface obtained from the visualization display interface is converted into a navigation image containing auxiliary road data and stored in a JPG format to ensure the integrity of information and the clarity of the graph. After obtaining the navigation image and the driving instruction text, the first navigation information constructed based on the navigation image and the driving instruction text can be transmitted to the intelligent driving domain controller through the communication protocol of the vehicle network (Scalable Open Middleware solution for In-Vehicle Networking over IP, abbreviated as SOMEIP) for further processing and analysis of the first navigation information.

[0123] Optionally, the navigation voice broadcast information in the above txt format can be: There is a long solid line 500 meters ahead. Please change lanes in advance. Or the navigation voice broadcast information in the txt format can be: There are multiple forks ahead. Please take the second lane on the left. The navigation voice broadcast information in the txt format can also be: After going straight, turn right immediately ahead. This is only an example for illustration, and the content of the navigation voice broadcast information is not specifically limited.

[0124] Figure 2 It is a schematic diagram of navigation visualization provided by an embodiment of the present application. As Figure 2 shown, navigation visualization can refer to the display of the intersections of vehicle driving in the form of a visualization graph in JPG format.

[0125] In the embodiment of the present application, by calling the built-in voice broadcast interface and visualization display interface of the vehicle, the driving instruction voice information can be converted into a driving instruction text, which can quickly and accurately understand the intention of the driver or the navigation system, avoid the errors that may be brought by voice recognition, and ensure the precise execution of the driving instruction. The navigation image is constructed from the display content of the visualization interface, which intuitively shows the road topology, the lane information of the current road, and the global navigation guidance information, provides a richer information source for subsequent road data adjustment, and improves the accuracy and reliability of the road data.

[0126] As an optional embodiment, determining the second navigation information corresponding to the preset driving path includes: determining the second navigation information corresponding to the preset driving path based on the navigation map including the preset driving path.

[0127] In this embodiment, based on the navigation map including the preset driving path, the second navigation information corresponding to the preset driving path can be determined, such as road attribute information, navigation instruction information, intersection lane attribute information, etc.

[0128] Optionally, the above navigation map may include basic geographical information (such as roads, buildings, landmarks), and may also include driving-related data, such as road network information, traffic rule data, dynamic traffic information, surrounding environment information, etc. The road network information may include road types (arterial roads, branch roads, highways), the orientation of the roads, width, number of lanes, turning radius, etc. The traffic rule data may include speed limits, no U-turns, right-of-way rules, traffic light positions, etc. The dynamic traffic information may include real-time traffic flow, accident reports, road construction conditions, weather conditions, etc. The surrounding environment information may include the locations of facilities such as parking lots, gas stations, hospitals, schools, etc. This is only an example, and there is no specific limitation on the content included in the navigation map.

[0129] Optionally, after determining the preset driving path, the second navigation information corresponding to the preset driving path may be collected from the navigation map according to the preset driving path.

[0130] In the embodiment of the present application, by extracting from the navigation map, the second navigation information related to the preset driving path can be accurately obtained. The second navigation information can provide more accurate and detailed road environment information for the vehicle, thereby improving the vehicle's decision-making ability and driving safety in complex scenarios. For example, in a complex urban road environment, potential traffic risks can be avoided, navigation accuracy can be improved, and driving safety can be ensured.

[0131] Figure 3 is a schematic diagram of an autonomous driving system that integrates navigation voice broadcast and intersection visibility, as Figure 3 shown, including: a static perception model 301, a navigation module 302, a dynamic perception model 303, a local planning model 304, and a control module 305.

[0132] The static perception model 301, the road topology, lane topology, and static targets obtained after being processed by the static perception model 301 can be input into the navigation module 302.

[0133] The navigation module 302 can receive the road topology, lane topology, and static targets input from the static perception model 301, and at the same time obtain the road attribute information, navigation instruction information, and intersection lane attribute information from the navigation basic information, as well as the navigation voice broadcast and navigation visualization information from the navigation enhanced information.

[0134] Optionally, the above navigation module 302 may process the road attribute information, navigation instruction information, and intersection lane attribute information in the received basic navigation information to obtain optimized road attribute information, navigation instruction information, and intersection lane attribute information. Based on the optimized road topology, optimized lane topology, static targets, and global driving path obtained from the fusion module and the navigation sub-module. Then, the perception information, optimized road topology, optimized lane topology, static targets, and global driving path are input into the local planning model 304.

[0135] The dynamic perception model 303 can input dynamic obstacles into the local planning model 304.

[0136] The local planning model 304 can receive the optimized road topology, optimized lane topology, and static targets from the navigation module 302, and the dynamic obstacles input from the dynamic perception model 303.

[0137] The control module 305 can use the local driving path planning and local speed planning obtained by processing through the local planning model 304 to control the vehicle, thereby improving the safety of the path planned by the vehicle.

[0138] For example, Figure 4 is a structural diagram of a vehicle control device provided by an embodiment of the present application. As Figure 4 shown, the basic navigation information may include (1) navigation actions, such as right front; (2) the distance from the navigation action; (3) the information of the front lane group. The information of the front lane group may include the background lane and the foreground lane. The background lane may be straight, straight, right front. The foreground lane may be empty, empty, right front (the lane on the navigation route). In this case, the enhanced navigation information can be obtained: (1) Voice broadcast: There is a long solid line ahead, please shift gears in advance; (2) A large map of the intersection.

[0139] Figure 5 is a structural diagram of an electronic device provided by an embodiment of the present application. As Figure 5As shown, the perception information, basic navigation information, and enhanced navigation information are input into the navigation module 501. The navigation module 501 may include a fusion module 5011 and a navigation sub-module 5012. After processing the perception information and the enhanced navigation information using the fusion module 5011, an optimized road topology and an optimized lane topology can be obtained. The navigation sub-module 5012 processes the optimized road topology and the optimized lane topology obtained through the fusion module 5011 to obtain a global driving path. The navigation module 501 inputs the basic navigation information (including road attribute information, navigation instruction information, etc.), the output optimized road topology, the optimized lane topology, and the global driving path into the local planning model 502, and a local driving path can be obtained; at the same time, the perception information can also be input into the local planning model 502 as perception information and compared with the local driving path output through the local planning model 502. When the comparison results are different, the local driving path is adjusted to make the local driving path more accurate.

[0140] Optionally, by recognizing the road images collected by an in-vehicle camera or other visual sensors, perception information such as road topology, lane topology, and static targets can be obtained.

[0141] In this embodiment, the fusion module receives the perception information and the enhanced navigation information (such as navigation voice broadcast information and navigation visualization information), and performs in-depth processing and matching fusion on the above information. Through comparison and analysis, an optimized road topology and an optimized lane topology can be generated. The above optimized road topology and optimized lane topology will take into account the accuracy of the real-time perception information and the consistency of the enhanced navigation information, so as to provide a road topology and a lane topology close to the actual situation. Based on the basic navigation information (including road attribute information, navigation instruction information, etc.) provided by the fusion module and the optimized road topology and lane topology, the navigation sub-module can plan a global driving path and generate a more refined global driving path. The global driving path may include necessary turning points, lane change points, and preferred driving routes. The optimized road topology, lane topology, and global driving path output by the navigation module can be sent to the local planning model. The local planning model can combine the received information with the perception information (such as road topology, lane topology) to generate a local driving path, that is, the specific path and speed arrangement that the vehicle needs to follow next. In addition, the local planning model can simultaneously use the received perception information as a reference and compare it with the local driving path output through the local planning model in real time. In the case of a difference between the perception information and the local driving path output through the local planning model, the local driving path can be immediately corrected to ensure that the local driving path fits the road conditions better and improve driving safety and comfort.

[0142] Embodiments of the present application can be applied to a navigation-assisted driving system. By receiving first navigation information (such as enhanced navigation information), the preset driving path of the vehicle is obtained, the second navigation information (such as basic navigation information) corresponding to the preset driving path is determined, and the perception information is obtained by recognizing the road image. Thus, the perception information, the first navigation information, and the second navigation information are fused to improve the perception ability of the road, obtain a better driving path planning, improve the functional pain points of the current navigation-assisted driving, solve the technical problem of low safety of the path planned by the vehicle, and achieve the technical effect of improving the safety of the path planned by the vehicle.

[0143] Embodiment 2

[0144] According to an embodiment of the present application, a control device for a vehicle is further provided. It should be noted that the control device for the vehicle can be used to execute the control method for the vehicle in Embodiment 1.

[0145] Embodiments of the present application further provide a control device 60 for a vehicle. Please refer to Figure 6 , Figure 6 which is a structural diagram of a control device for a vehicle provided in an embodiment of the present application. The device may include: an acquisition unit 602, configured to acquire first navigation information during the driving process of the vehicle, where the first navigation information is used to at least determine the preset driving path of the vehicle; a processing unit 604, configured to determine second navigation information corresponding to the preset driving path and collect a road image corresponding to the preset driving path, where the second navigation information is used to at least represent the driving instruction information corresponding to the vehicle when driving in the preset driving path; a first determination unit 606, configured to determine the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image; a second determination unit 608, configured to determine the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path, where the obstacle information is used to determine the obstacles in the global driving path that are in a moving state; and a control unit 610, configured to control the vehicle to drive according to the local driving path.

[0146] The above-mentioned control device for a vehicle provided by the embodiments of the present application achieves the following technical effects: By acquiring the first navigation information to determine the preset driving path, for the preset driving path, the second navigation information and the corresponding road image are collected, and the first navigation information is fused to determine the global driving path. After determining the global driving path, the local driving path is determined according to the real-time obstacle information, and following the local driving path planning, the vehicle moves forward along the established driving path. Through the high degree of intelligence and flexibility in information fusion, path planning, and dynamic control, the present application can ensure the safe driving of the vehicle, thus solving the technical problem of low safety of the path planned by the vehicle and achieving the technical effect of improving the safety of the path planned by the vehicle.

[0147] It should be noted that the above-mentioned units can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to this: the above-mentioned modules are all located in the same processor; or, the above-mentioned modules are respectively located in different processors in any combination form.

[0148] Embodiment III

[0149] The embodiment of the present application further provides an electronic device 70. Please refer to Figure 7 , Figure 7 which is a structural diagram of an electronic device provided by an embodiment of the present application, including a processor 710 and a memory 720. Among them, the memory 710 is used to store a computer program; the processor 720 is used to execute the program stored on the memory 710 to implement the vehicle control method introduced in any embodiment of the present application.

[0150] Optionally, in this embodiment, the above-mentioned processor can be set to execute the following steps through a computer program:

[0151] Step S1: Obtain the first navigation information of the vehicle during driving, where the first navigation information is used to at least determine the preset driving path of the vehicle;

[0152] Step S2: Determine the second navigation information corresponding to the preset driving path, and collect the road image corresponding to the preset driving path, where the second navigation information is used to at least represent the driving instruction information corresponding to the vehicle when driving in the preset driving path;

[0153] Step S3: Determine the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image;

[0154] Step S4: Determine the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path, where the obstacle information is used to determine the obstacles in the global driving path that are in a moving state;

[0155] Step S5: Control the vehicle to drive according to the local driving path.

[0156] The above electronic device provided by the embodiments of the present application achieves the following technical effects: By obtaining the first navigation information to determine the preset driving path, for the preset driving path, collect the second navigation information and the corresponding road images, and fuse the first navigation information, the global driving path can be determined. After determining the global driving path, determine the local driving path according to the real-time obstacle information, and follow the local driving path planning to make the vehicle move forward along the established driving path. Through the high intelligence and flexibility in information fusion, path planning, and dynamic control, the present application can ensure the safe driving of the vehicle, thus solving the technical problem of the low safety of the path planned by the vehicle and achieving the technical effect of improving the safety of the path planned by the vehicle.

[0157] Those of ordinary skill in the art can understand that Figure 7 the structure shown is only for illustration, and the electronic device can also be a terminal device such as a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a handheld computer, and a Mobile Internet Device (MID for short). Figure 7 It does not limit the structure of the above electronic device. For example, the electronic device 70 may further include more or fewer components (such as a network interface, a display device, etc.) than those shown in Figure 7 or have a different configuration from that shown in Figure 7 the illustration.

[0158] Embodiment 4

[0159] The embodiments of the present application further provide a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the vehicle control method described in any embodiment of the present application.

[0160] Optionally, in this embodiment, the above storage medium may be set to store a computer program for executing the following steps:

[0161] Step S1, obtain the first navigation information of the vehicle during driving, where the first navigation information is used to at least determine the preset driving path of the vehicle;

[0162] Step S2, determine the second navigation information corresponding to the preset driving path, and collect the road images corresponding to the preset driving path, where the second navigation information is used to at least represent the driving instruction information corresponding to the vehicle when driving in the preset driving path;

[0163] Step S3, based on the first navigation information, the second navigation information, and the road images, determine the global driving path of the vehicle;

[0164] Step S4: Based on the obstacle information in the global driving path and the global driving path, determine the local driving path of the vehicle in the driving direction, where the obstacle information is used to determine the obstacles in the moving state in the global driving path.

[0165] Step S5: Control the vehicle to drive according to the local driving path.

[0166] The above electronic device provided by the embodiment of the present application achieves the following technical effects: By obtaining the first navigation information to determine the preset driving path, for the preset driving path, collect the second navigation information and the corresponding road images, and fuse the first navigation information, the global driving path can be determined. After determining the global driving path, determine the local driving path according to the real-time obstacle information, and follow the local driving path planning to make the vehicle move forward along the established driving path. Through the high intelligence and flexibility in information fusion, path planning, and dynamic control, the present application can ensure the safe driving of the vehicle, thus solving the technical problem of low safety of the path planned by the vehicle and achieving the technical effect of improving the safety of the path planned by the vehicle.

[0167] Optionally, in this embodiment, the above storage medium may include, but is not limited to: USB flash drive, read-only memory (ROM), random access memory (RAM), mobile hard disk, magnetic disk, or optical disc, etc., various media that can store computer programs.

[0168] The above electronic device provided by the embodiment of the present application achieves the following technical effects:

[0169] In the present application, "a plurality of" means two or more.

[0170] In the present application, unless otherwise clearly defined, the terms "install", "connect", and "couple" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, and it can be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0171] The terms "first", "second", "third", "fourth", etc. (if any) in the present application are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence.

[0172] In this application, the term "and / or" is merely a description of the associated relationship of associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, in this application, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0173] If there is no special instruction, all steps of this application can be carried out sequentially or randomly. For example, the method includes steps A and B, indicating that the method may include steps A and B carried out sequentially, or may also include steps B and A carried out sequentially. For example, it is mentioned that the method may further include step C, indicating that step C can be added to the method in any order. For example, the method may include steps A, B, and C, or may also include steps A, C, and B, or may also include steps C, A, and B, etc.

[0174] The above are only the preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent replacements, and improvements made within the spirit and principles of this application shall be included within the protection scope of this application.

Claims

1. A control method for a vehicle, characterized in that, Including: Obtain first navigation information of a vehicle during driving, where the first navigation information is used to at least determine a preset driving path of the vehicle; Determine second navigation information corresponding to the preset driving path, and collect a road image corresponding to the preset driving path, where the second navigation information is used to at least represent driving instruction information corresponding to the vehicle when driving in the preset driving path; Based on the first navigation information, the second navigation information, and the road image, determine a global driving path of the vehicle; Based on obstacle information in the global driving path and the global driving path, determine a local driving path of the vehicle in the driving direction, where the obstacle information is used to determine obstacles in a moving state in the global driving path; Control the vehicle to drive according to the local driving path.

2. The method according to claim 1, wherein The determining the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image includes: Identify the first navigation information to obtain a driving instruction text and auxiliary road data of the preset driving path; Based on the road image, adjust the auxiliary road data to obtain target road data; Based on the driving instruction text, the second navigation information, and the target road data, plan to obtain the global driving path of the vehicle.

3. The method according to claim 2, wherein The adjusting the auxiliary road data to obtain target road data based on the road image includes: Identify the road image to obtain a road topology, lane information, a first confidence level corresponding to the road topology, and a second confidence level corresponding to the lane information in the road image, where the first confidence level is used to characterize the accuracy of the road topology, and the second confidence level is used to characterize the accuracy of the lane information; Based on the road topology and the first confidence level, adjust the auxiliary road topology in the auxiliary road data to obtain a target road topology, and based on the lane information and the second confidence level, adjust the auxiliary lane information in the auxiliary road data to obtain target lane information; Based on the target lane information and the target road topology, construct the target road data.

4. The method according to claim 3, characterized in that, The adjusting the auxiliary road topology in the auxiliary road data to obtain a target road topology based on the road topology and the first confidence level, and adjusting the auxiliary lane information in the auxiliary road data to obtain target lane information based on the lane information and the second confidence level includes: Input the auxiliary road topology, the road topology, and the first confidence level into a global path planning model to obtain the target road topology, and input the auxiliary lane information, the lane information, and the second confidence level into the global path planning model to obtain the target lane information.

5. The method according to claim 2, wherein The planning to obtain the global driving path of the vehicle based on the driving instruction text, the second navigation information, and the target road data includes: Based on the road image, determine the stationary objects in the preset driving path; Input the second navigation information, the driving instruction text, the stationary objects, and the target road data into the global path planning model to obtain the global driving path.

6. The method according to claim 5, wherein The determining the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path includes: Based on the obstacle information, the global driving path, and the stationary objects, determine the local driving path of the vehicle and the driving speed of the vehicle when driving on the local driving path.

7. The method according to claim 6, wherein The determining the local driving path of the vehicle and the driving speed of the vehicle when driving on the local driving path based on the obstacle information, the global driving path, and the stationary objects includes: Based on the obstacle information, the global driving path, the stationary objects, and the target lane information, construct an initial local driving path; Based on the road topology and the lane information, adjust the initial local driving path to obtain the local driving path and the driving speed.

8. The method according to claim 1, characterized in that The method further includes: Obtain the image data in the global driving path collected by the image acquisition device; Identify the image data to obtain the obstacle information.

9. The method according to claim 1, characterized in that, The obtaining the first navigation information of the vehicle during driving includes: Call the voice broadcast interface and the visualization display interface in the vehicle; Use the voice broadcast interface to collect the driving instruction voice information, and use the visualization display interface to collect the display content of the visualization interface in the vehicle; Convert the driving instruction voice information to obtain the driving instruction text, and convert the display content into a navigation image including auxiliary road data; Based on the navigation image and the driving instruction text, construct the first navigation information.

10. The method according to claim 1, wherein The determining the second navigation information corresponding to the preset driving path includes: Based on the navigation map including the preset driving path, determine the second navigation information corresponding to the preset driving path.

11. A control device for a vehicle, characterized in that, Includes: An obtaining unit, configured to obtain the first navigation information of the vehicle during driving, where the first navigation information is used to at least determine the preset driving path of the vehicle; A processing unit, configured to determine the second navigation information corresponding to the preset driving path and collect the road image corresponding to the preset driving path, where the second navigation information is used to at least represent the driving instruction information corresponding to the vehicle when driving in the preset driving path; A first determining unit, configured to determine the global driving path of the vehicle based on the first navigation information, the second navigation information, and the road image; A second determining unit, configured to determine the local driving path of the vehicle in the driving direction based on the obstacle information in the global driving path and the global driving path, where the obstacle information is used to determine the obstacles in the global driving path that are in a moving state; A control unit, configured to control the vehicle to drive according to the local driving path.

12. A vehicle, characterized in that, It includes an in-vehicle processor and an in-vehicle memory, wherein, the in-vehicle memory is used to store a computer program; the in-vehicle processor is used to execute the computer program stored on the memory to implement the method according to any one of claims 1 to 10.

13. An electronic device, characterized in that, It includes a processor and a memory, wherein, the memory is used to store a computer program; the processor is used to execute the program stored on the memory to implement the method according to any one of claims 1 to 10.

14. A computer-readable storage medium, characterized in that, A computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images