Positioning and navigation method, device, electronic device and storage medium
By acquiring and analyzing the road images ahead of the vehicle in real time, and using deep neural network to extract positioning auxiliary information, the problem of insufficient positioning accuracy of the navigation system in parallel roads and overpass scenarios is solved, achieving more accurate navigation and higher driving safety.
Patent Information
- Application Number
- CN202010610307.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-06-30
- Publication Date
- 2024-11-12
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing navigation system is difficult to accurately locate in parallel roads and overpass scenarios, resulting in navigation errors and increasing the time cost of users and the risk of traffic accidents.
By obtaining the road image ahead of the vehicle in real time, semantic segmentation is used using deep neural networks, positioning auxiliary information of traffic scenes is extracted, target roads currently driving on the vehicle, and navigation prompts are provided based on the destination and target roads.
It improves the positioning accuracy of the navigation system in parallel roads and overpass scenarios, reduces the risks of navigation errors and traffic violations, and improves user experience and driving safety.
Smart Images

Figure CN111967301B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of image processing, specifically to the field of artificial intelligence, deep learning, and intelligent transportation, and in particular to a positioning and navigation method, device, electronic device, and storage medium. Background Art
[0002] Navigation electronic products are combined with multi-platform devices such as mobile phones, driving recorders, and car computers, and are widely used in driving. Affected by the accuracy of GPS (Global Positioning System) and navigation data, in parallel roads and overpass scenarios, it is difficult for navigation products to accurately locate which road the user is on, which leads to incorrect navigation routes for users, greatly increasing the user's time cost and even increasing the risk of traffic accidents. Summary of the invention
[0003] The present application provides a positioning and navigation method, device, electronic device and storage medium.
[0004] According to a first aspect of the present application, a positioning and navigation method is provided, comprising:
[0005] Determining whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle;
[0006] If the vehicle is about to enter the traffic scene, acquiring a road image ahead of the vehicle;
[0007] Generating positioning auxiliary information of the traffic scene according to the front road image;
[0008] Determining a target road on which the vehicle is currently traveling according to the positioning assistance information;
[0009] Obtain navigation information and provide navigation prompts based on a preset destination and the target road.
[0010] According to a second aspect of the present application, a positioning and navigation device is provided, comprising:
[0011] A judgment module, used to judge whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle;
[0012] A first acquisition module, used for acquiring a road image ahead of the vehicle when the vehicle is about to enter the traffic scene;
[0013] A generating module, used for generating positioning auxiliary information of the traffic scene according to the front road image;
[0014] A determination module, used to determine the target road that the vehicle is currently traveling on according to the positioning auxiliary information;
[0015] The navigation prompt module is used to obtain navigation information and provide navigation prompts according to a preset destination and the target road.
[0016] According to a third aspect of the present application, an electronic device is provided, including:
[0017] at least one processor; and
[0018] a memory communicatively connected to the at least one processor; wherein,
[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the positioning and navigation method described in the first aspect of the present application.
[0020] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the positioning and navigation method described in the first aspect of the present application.
[0021] According to a fifth aspect of the present application, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the positioning and navigation method described in the first aspect is implemented.
[0022] According to the technical solution of the present application, it is possible to determine whether the vehicle is about to enter a preset traffic scene based on the real-time position of the vehicle. If the vehicle is about to enter the traffic scene, the image of the road ahead of the vehicle is obtained, and positioning auxiliary information of the traffic scene is generated based on the image of the road ahead. The target road that the vehicle is currently traveling on is determined based on the positioning auxiliary information, and then navigation information is obtained based on the preset destination and target road for navigation prompts. This can solve the problem in the prior art that the positioning accuracy is insufficient in parallel roads and overpass scenarios during navigation, which leads to navigation errors.
[0023] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.
[0025] Figure 1 is a flowchart of a positioning and navigation method according to an embodiment of the present application;
[0026] Figure 2is an example diagram of a road ahead image according to an embodiment of the present application;
[0027] Figure 3 Yes Figure 2 Schematic diagram of semantic segmentation obtained after semantic segmentation;
[0028] Figure 4 is from Figure 3 An example diagram of positioning auxiliary information extracted from the semantic segmentation map shown;
[0029] Figure 5 is a flow chart of a positioning and navigation method according to a specific embodiment of the present application;
[0030] Figure 6 is a flow chart of a positioning and navigation method according to another specific embodiment of the present application;
[0031] Figure 7 is a structural block diagram of a positioning and navigation device according to an embodiment of the present application;
[0032] Figure 8 is a structural block diagram of a positioning and navigation device according to another embodiment of the present application;
[0033] Fig. 9 It is a block diagram of an electronic device used to implement the positioning and navigation method of an embodiment of the present application. DETAILED DESCRIPTION
[0034] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.
[0035] It should be noted that navigation electronic products are combined with multi-platform devices such as mobile phones, driving recorders, and car computers, and are widely used in driving. Affected by the accuracy of GPS and navigation data, in parallel roads and overpass scenarios, it is difficult for navigation products to accurately locate which road the user is on, which leads to the wrong route for the user, greatly increasing the user's time cost and even increasing the risk of traffic accidents.
[0036] In other words, current navigation products cannot accurately determine which road the user is on, parallel road or above or below the overpass. Navigation products provide a user selection interface on the navigation interface and determine the road the user is traveling on based on the user's selection. However, this method relies on the user's judgment and the operational feedback of the navigation interface, which leads to safety risks when the user is driving. In addition, if the user does not make a selection, the navigation cannot provide correct navigation prompts, which may cause the user to deviate from the route and increase travel costs.
[0037] In order to solve the above technical problems, the present application proposes a positioning and navigation method, device, electronic device and storage medium.
[0038] Figure 1 1 is a flow chart of a positioning and navigation method according to an embodiment of the present application. It should be noted that the positioning and navigation method of the embodiment of the present application can be applied to the positioning and navigation device of the embodiment of the present application. The device can be configured on an electronic device. The electronic device can be a terminal device on a vehicle to implement the positioning and navigation of the vehicle.
[0039] like Figure 1 As shown, the positioning and navigation method may include:
[0040] Step 101, judging whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle.
[0041] It should be noted that the positioning and navigation method of the embodiment of the present application can be applied to an electronic device, and the electronic device has a navigation electronic map. The navigation electronic map may pre-store navigation data of each road, wherein the navigation data may include but is not limited to the coordinates of the road, the number of lanes contained in the road, etc. The electronic device can realize the positioning and navigation of the vehicle based on the navigation electronic map.
[0042] In the embodiment of the present application, the real-time position of the vehicle can be obtained in real time, and the real-time position of the vehicle can be used in combination with the navigation electronic map on the electronic device to determine whether the vehicle is about to enter the preset traffic scene. In other words, the characteristic information of the preset traffic scene area can be extracted in advance, and the extracted characteristic information can be stored in the navigation electronic map as a basis for judging whether to enter the traffic scene area. Among them, the electronic device can be an on-board terminal device (such as a driving recorder, a car computer, etc.) used to realize positioning and navigation on the vehicle, or it can also be other electronic devices such as mobile phones that can have positioning and navigation.
[0043] It should be noted that in some embodiments of the present application, the above-mentioned traffic scene can be a parallel road or an overpass scene. That is to say, due to the influence of GPS accuracy, navigation data accuracy, etc., in the parallel road and overpass scene, it is difficult for the navigation product to accurately locate which road the user is on above or below the parallel road or overpass. For this reason, the present application proposes a positioning and navigation method for the parallel road and overpass scene, so as to accurately locate the specific road that the vehicle is currently traveling on in the parallel road and overpass scene.
[0044] It should also be noted that in the embodiment of the present application, if the vehicle does not enter the preset traffic scene, there is no need to collect the image of the road ahead to assist in locating the specific road the vehicle is currently traveling on. In other words, when the vehicle is about to enter the preset traffic scene, the image of the road ahead can be collected to assist in locating the specific road the vehicle is currently traveling on, so that the vehicle's driving road in the preset traffic scene can be accurately located.
[0045] Step 102: If the vehicle is about to enter a traffic scene, an image of the road ahead of the vehicle is acquired.
[0046] That is to say, when it is determined that the vehicle is about to enter the traffic scene, auxiliary positioning will be performed based on the image of the road ahead, so that the current specific road on which the vehicle is traveling in the preset traffic scene can be accurately located.
[0047] In an embodiment of the present application, when it is determined that the vehicle is about to enter the preset traffic scene, the image of the road ahead of the vehicle can be collected by a front-view camera on the vehicle. The front-view camera can be installed on the front windshield. Alternatively, the image of the road ahead of the vehicle can be collected by a driving recorder on the vehicle. The present application does not specifically limit the tool and method for collecting the image of the road ahead of the vehicle.
[0048] Step 103: Generate positioning auxiliary information of the traffic scene according to the front road image.
[0049] Optionally, the image content is analyzed to obtain positioning assistance information of the preset traffic scene, wherein the positioning assistance information can be understood as information used to assist positioning.
[0050] In some embodiments of the present application, semantic segmentation of the road ahead image can be performed based on a pre-trained deep neural network model to obtain traffic element information; and positioning auxiliary information for the traffic scene can be extracted from the traffic element information. Among them, the above-mentioned deep neural network model is a pre-trained model based on artificial intelligence technology and deep learning technology. The deep neural network model has learned the function of semantic segmentation of images. Therefore, semantic segmentation of the road ahead image can be performed based on the deep neural network model to obtain the traffic element information contained in the image. Among them, the traffic element information may include but is not limited to information such as moving vehicles, non-motor vehicles, the vehicle itself, lane lines, and road surfaces. Afterwards, positioning auxiliary information for the preset traffic scene can be extracted from the traffic element information.
[0051] For example, Figure 2 As shown in the figure, it is the current road image in front of the vehicle. The deep neural network model can be used to perform semantic segmentation on the image to obtain the traffic element information in the image, such as Figure 3 As shown, the traffic element information may include information such as traveling vehicles, non-motor vehicles, the vehicle itself, lane lines, and road surfaces. Afterwards, positioning assistance information for the preset traffic scene can be extracted from the traffic element information. Among them, due to different traffic scenes, the positioning assistance information will also be different. For example, assuming that the traffic scene includes a first road surface and a second road surface set on the left and right by an isolation device, that is, a parallel road scene, the positioning assistance information may include the vehicle mapping road surface, the road surface area surface, and the number of lanes. For another example, assuming that the traffic scene includes a first road surface and a second road surface set up and down by a bridge device, that is, an overpass scene, the positioning assistance information may include information on the presence or absence of a bridge body.
[0052] It should be noted that, due to the difference in positioning auxiliary information, the methods used to extract different positioning auxiliary information will also be different. For example, for the recognition of road surface areas, such as Figure 4 As shown in , the road surface area can be extracted based on the result of image semantic segmentation, and then the road surface can be divided into two road surface areas A1 and A2 in order from left to right. Figure 4 As shown in , the position of the moving vehicle can be extracted from the image semantic segmentation image, and then the relationship between the moving vehicle and the road area extracted in the previous step can be determined, so as to determine which road area the vehicle is traveling on, and the vehicle mapping road surface can be obtained. Figure 4As shown in the figure, based on the image semantic segmentation results, the segmentation map can be preprocessed to extract lane lines. Specifically, binary image extraction, morphological operations, and Hough fitting can be used. Then, based on the extracted lane line results, the intersection with the road area where the vehicle is traveling is calculated to calculate the number of lane lines on the road where the vehicle is traveling, and then the number of lanes on the road is determined based on the number of lane lines. For overpass identification, the bridge body information in the traffic scene can be extracted based on the image semantic segmentation results, where the bridge body information can be expressed as whether there is a bridge body or not.
[0053] Step 104: determine the target road on which the vehicle is currently traveling based on the positioning assistance information.
[0054] That is to say, the target road that the vehicle is currently traveling on can be determined based on the positioning auxiliary information identified from the road image ahead, that is, which specific road in the traffic scene the vehicle is currently traveling on can be determined. For example, taking the traffic scene as a parallel road as an example, the positioning auxiliary information can be used to determine whether the target road that the vehicle is currently traveling on is the main road or the auxiliary road of the parallel road. For another example, taking the traffic scene as an overpass as an example, the positioning auxiliary information can be used to determine whether the target road that the vehicle is currently traveling on is on or under the overpass.
[0055] Step 105, obtaining navigation information according to the preset destination and target road to provide navigation prompts.
[0056] Specifically, when determining the target road that the vehicle is currently traveling on, the corresponding navigation information can be obtained based on the target location and the target road, combined with the navigation electronic map, and then navigation prompts can be given based on the navigation information. For example, for a scene where a vehicle enters an overpass, when the target road that the vehicle is currently traveling on is accurately located, if the vehicle is under the bridge, the user can be prompted to "please continue on the xx bridge", and if the vehicle is under the bridge, the user can be prompted to "please continue along the xx road". For another example, for a scene where a vehicle enters a parallel road, when the target road that the vehicle is currently traveling on is accurately located, if the vehicle is on the left side of the parallel road, the user can be prompted to "please continue along the left road / main road", and if the vehicle is on the right side of the parallel road, the user can be prompted to "please continue along the right road / auxiliary road".
[0057] According to the positioning and navigation method of the embodiment of the present application, it is possible to determine whether the vehicle is about to enter a preset traffic scene based on the real-time position of the vehicle. If the vehicle is about to enter the traffic scene, the image of the road ahead of the vehicle is obtained, and positioning auxiliary information of the traffic scene is generated based on the image of the road ahead, and the target road currently traveled by the vehicle is determined based on the positioning auxiliary information, and then navigation information is obtained according to the preset destination and target road for navigation prompts. It can be seen that the present application comprehensively utilizes image processing and other technologies to identify information used to assist positioning, and then uses this information to assist in positioning the specific road currently traveled by the vehicle, which can accurately locate the specific road on which the vehicle is currently traveling in the traffic scene, thereby improving the positioning accuracy. In addition, combined with the navigation electronic map data information, the navigation positioning accuracy in the preset traffic scene can be improved, thereby improving the navigation user experience, which has practical significance and value in reducing traffic violations and ensuring driving safety. It also has auxiliary decision-making value in emerging fields such as AR (Augmented Reality) navigation and intelligent transportation.
[0058] In order to accurately locate the road on which the vehicle is traveling in a parallel road scenario, positioning auxiliary information such as the vehicle-mapped road surface and the road surface area can be obtained, and then the specific road on which the vehicle is currently traveling can be accurately located based on the positioning auxiliary information. Specifically, in some embodiments of the present application, such as Figure 5 As shown, the positioning and navigation method may include:
[0059] Step 501, judging whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle.
[0060] Among them, in the embodiment of the present application, the traffic scene includes: a first road surface and a second road surface set on the left and right by an isolation device, the first road surface and the second road surface are roads running in the same direction, that is, the traffic scene can be understood as a parallel road scene. The parallel road scene can be understood as the existence of two parallel roads running in the same direction, such as the main road and the auxiliary road of a certain road, the main road and the auxiliary road running in the same direction and parallel.
[0061] It should be noted that, in the embodiments of the present application, the above-mentioned isolation device may include but is not limited to isolation fences, double yellow lines, etc.
[0062] Step 502: If the vehicle is about to enter a traffic scene, an image of the road ahead of the vehicle is acquired.
[0063] Step 503: Generate a vehicle mapping road surface and a road surface area surface of the traffic scene according to the front road image.
[0064] Step 504 , judging whether there is a road surface on the right side of the vehicle based on the vehicle-mapped road surface and road surface area surface.
[0065] Step 505 , if there is a road surface on the right side of the vehicle, determine that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene, and execute step 508 .
[0066] For example, Figure 4 As shown in the figure, assuming that the vehicle mapped road surface (i.e., the road surface where the vehicle is located) is road surface 1, it can be determined whether there is a road surface on the right side of the vehicle based on the vehicle mapped road surface and the road surface area (including road surface 1 and road surface 2), such as Figure 4 As shown, at this time, there is a road surface on the right side of the vehicle, and it can be determined that the target road currently traveled by the vehicle is the first road surface in the traffic scene. For example, the target road currently traveled by the vehicle is the main road of the parallel road.
[0067] Step 506: If there is no road surface on the right side of the vehicle, determine whether there is a road surface on the left side of the vehicle.
[0068] That is, if it is determined that there is no road on the right side of the vehicle based on the vehicle-mapped road surface and road surface area surface, then it is further determined whether there is a road on the left side of the vehicle based on the vehicle-mapped road surface and road surface area surface.
[0069] Step 507 , if there is a road surface on the left side of the vehicle, determine that the target road the vehicle is currently traveling on is the second road surface in the traffic scene, and execute step 508 .
[0070] That is, when it is determined that there is a road surface on the left side of the vehicle based on the vehicle-mapped road surface and the road surface area surface, it is determined that the target road on which the vehicle is currently traveling is the auxiliary road of the parallel road.
[0071] Step 508, obtaining navigation information according to the preset destination and target road to provide navigation prompts.
[0072] For example, in a scenario where a vehicle enters a parallel road, when the vehicle's current target road is accurately located, if the vehicle is on the left side of the parallel road, the user may be prompted "Please continue along the left road / main road"; if the vehicle is on the right side of the parallel road, the user may be prompted "Please continue along the right road / auxiliary road".
[0073] If a road in one of the parallel roads in the image is too narrow due to a problem with camera image acquisition, resulting in a failure in road recognition, the positioning auxiliary information for the road area is missing. In order to assist in positioning and improve positioning accuracy, the number of lanes in the image of the road ahead can be identified, and the number of lanes can be used to assist in determining whether the target road the vehicle is currently traveling on is the main road or the secondary road among the parallel roads. Specifically, in some embodiments of the present application, the positioning auxiliary information also includes the number of lanes. Among them, in an embodiment of the present application, if there is no road on the left side of the vehicle, the navigation data of the traffic scene is obtained; based on the number of lanes in the positioning auxiliary information and the number of lanes in the navigation data, it is identified whether the target road the vehicle is currently traveling on is the first road or the second road in the traffic scene.
[0074] That is to say, when the specific road that the vehicle is currently traveling on cannot be determined based on the vehicle-mapped road surface and road surface area, the navigation data of the parallel road scene can be obtained from the navigation electronic map, and the number of lanes in the positioning auxiliary information is matched with the number of lanes in the navigation data to determine whether the target road that the vehicle is currently traveling on is the main road or the auxiliary road among the parallel roads. It can be understood that, regardless of whether the parallel road is the main road or the auxiliary road, it has its own corresponding number of lanes, so it can be determined based on the number of lanes whether the road that the vehicle is currently traveling on is the main road or the auxiliary road of the parallel road.
[0075] According to the positioning and navigation method of the embodiment of the present application, when the vehicle is about to enter a parallel road scene, an image of the road in front of the vehicle can be obtained, and then image semantic segmentation, semantic understanding and other technologies can be comprehensively utilized to identify lane areas, lane numbers and other positioning auxiliary information, and combined with navigation electronic map data information, the navigation positioning accuracy under parallel roads can be improved, thereby improving the navigation user experience. While improving the navigation experience, driving safety can also be guaranteed.
[0076] In order to accurately locate the road on which the vehicle is traveling in the overpass scene, positioning auxiliary information such as bridge information can be obtained, and then the specific road on which the vehicle is currently traveling can be accurately located based on the bridge information. Specifically, in some embodiments of the present application, such as Figure 6 As shown, the positioning and navigation method may include:
[0077] Step 601, judging whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle.
[0078] Among them, in an embodiment of the present application, the traffic scene includes: a first road surface and a second road surface arranged up and down by a bridge device, the first road surface and the second road surface are up and down same-direction driving roads; the positioning auxiliary information includes whether there is a bridge body information.
[0079] Step 602: If the vehicle is about to enter a traffic scene, an image of the road ahead of the vehicle is acquired.
[0080] Step 603: Generate positioning auxiliary information of the traffic scene according to the front road image.
[0081] Step 604: If the positioning auxiliary information contains bridge information, it is determined that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene.
[0082] That is to say, when a bridge is identified on the road ahead, it can be determined that the target road the vehicle is currently traveling on is the first road surface in the overpass field, which means that the vehicle is currently traveling under the bridge in the overpass scene, and the road corresponding to the bridge under the bridge is the target road the vehicle is currently traveling on.
[0083] Step 605: If the positioning auxiliary information indicates that there is no bridge information, it is determined that the target road on which the vehicle is currently traveling is the second road surface in the traffic scene.
[0084] That is to say, when it is identified that there is no bridge on the road ahead, it can be determined that the target road the vehicle is currently traveling on is the second road surface in the traffic scene, which means that the vehicle is currently traveling on the bridge of an overpass, and the road name corresponding to the overpass is the name of the target road the vehicle is currently traveling on.
[0085] Step 606, obtaining navigation information according to the preset destination and target road to provide navigation prompts.
[0086] For example, in a scenario where a vehicle enters an overpass, when the vehicle's current target road is accurately located, if the vehicle is under the bridge, the user may be prompted "Please continue driving on xx bridge"; if the vehicle is under the bridge, the user may be prompted "Please continue driving along xx road".
[0087] According to the positioning and navigation method of the embodiment of the present application, when a vehicle is about to enter an overpass scene, an image of the road ahead of the vehicle can be obtained, and then image semantic segmentation, semantic understanding and other technologies can be comprehensively utilized to identify the presence or absence of bridge information and other positioning auxiliary information, and combined with navigation electronic map data information, the navigation positioning accuracy in the overpass scene can be improved, thereby improving the navigation user experience. While improving the navigation experience, driving safety can also be guaranteed.
[0088] Figure 7 FIG. 1 is a structural block diagram of a positioning and navigation device according to an embodiment of the present application. Figure 7 As shown, the positioning and navigation device 700 may include: a judgment module 710 , a first acquisition module 720 , a generation module 730 , a determination module 740 and a navigation prompt module 750 .
[0089] Specifically, the judgment module 710 is used to judge whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle.
[0090] The first acquisition module 720 is used to acquire the road image ahead of the vehicle when the vehicle is about to enter a traffic scene.
[0091] The generation module 730 is used to generate positioning auxiliary information of the traffic scene based on the road image ahead. In some embodiments of the present application, the generation module 730 performs semantic segmentation on the road image ahead based on a pre-trained deep neural network model to obtain traffic element information; and extracts positioning auxiliary information for the traffic scene from the traffic element information.
[0092] The determination module 740 is used to determine the target road that the vehicle is currently traveling on based on the positioning assistance information. In some embodiments of the present application, the traffic scene includes: a first road surface and a second road surface set on the left and right by an isolation device, the first road surface and the second road surface are roads for traveling in the same direction, and the positioning assistance information includes a vehicle-mapped road surface and a road surface area surface; wherein, in the embodiment of the present application, the determination module 740 is specifically used to: determine whether there is a road surface on the right side of the vehicle based on the vehicle-mapped road surface and the road surface area surface; if there is a road surface on the right side of the vehicle, determine that the target road that the vehicle is currently traveling on is the first road surface in the traffic scene; if there is no road surface on the right side of the vehicle, determine whether there is a road surface on the left side of the vehicle; if there is a road surface on the left side of the vehicle, determine that the target road that the vehicle is currently traveling on is the second road surface in the traffic scene.
[0093] In some embodiments of the present application, the traffic scene includes: a first road surface and a second road surface arranged up and down by a bridge device, the first road surface and the second road surface are road surfaces for traveling in the same direction up and down; the positioning auxiliary information includes information about whether there is a bridge body; wherein, in an embodiment of the present application, the determination module 740 is specifically used for: if the positioning auxiliary information is information about the presence of a bridge body, then determining that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene; if the positioning auxiliary information is information about the absence of a bridge body, then determining that the target road on which the vehicle is currently traveling is the second road surface in the traffic scene.
[0094] The navigation prompt module 750 is used to obtain navigation information and provide navigation prompts according to a preset destination and target road.
[0095] In some embodiments of the present application, the positioning assistance information also includes the number of lanes, where Figure 8 As shown, the positioning and navigation device 700 may further include: a second acquisition module 760 and an identification module 770. The second acquisition module 760 is used to acquire navigation data of the traffic scene when there is no road surface on the left side of the vehicle; the identification module 770 is used to identify whether the target road currently traveled by the vehicle is the first road surface or the second road surface in the traffic scene according to the number of lanes in the positioning auxiliary information and the number of lanes in the navigation data.
[0096] According to the positioning and navigation device of the embodiment of the present application, it is possible to determine whether the vehicle is about to enter a preset traffic scene based on the real-time position of the vehicle. If the vehicle is about to enter the traffic scene, the image of the road ahead of the vehicle is obtained, and positioning auxiliary information of the traffic scene is generated based on the image of the road ahead, and the target road on which the vehicle is currently traveling is determined based on the positioning auxiliary information, and then navigation information is obtained based on the preset destination and target road for navigation prompts. It can be seen that the present application comprehensively utilizes image processing and other technologies to identify information used to assist positioning, and then uses this information to assist in positioning the specific road on which the vehicle is currently traveling, which can accurately locate the specific road on which the vehicle is currently traveling in the traffic scene, thereby improving positioning accuracy. In addition, combined with navigation electronic map data information, the navigation positioning accuracy in the preset traffic scene can be improved, thereby improving the navigation user experience, which has practical significance and value in reducing traffic violations and ensuring driving safety. It also has auxiliary decision-making value in emerging fields such as AR navigation and intelligent transportation.
[0097] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.
[0098] like Fig. 9 , is a block diagram of an electronic device for implementing a positioning navigation method according to an embodiment of the present application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present application described and / or required herein.
[0099] like Fig. 9 As shown, the electronic device includes: one or more processors 901, a memory 902, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Fig. 9A processor 901 is taken as an example.
[0100] The memory 902 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the positioning and navigation method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the positioning and navigation method provided in the present application. The computer program product of the present application includes a computer program, which implements the positioning and navigation method provided in the present application when executed by the processor 901.
[0101] The memory 902 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the positioning and navigation method in the embodiment of the present application (for example, the attached Figure 7 The processor 901 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 902, that is, implements the positioning and navigation method in the above method embodiment.
[0102] The memory 902 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of an electronic device for implementing positioning navigation, etc. In addition, the memory 902 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 902 may optionally include a memory remotely arranged relative to the processor 901, and these remote memories may be connected to the electronic device for implementing positioning navigation via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.
[0103] The electronic device for implementing the positioning and navigation method may further include: an input device 903 and an output device 904. The processor 901, the memory 902, the input device 903 and the output device 904 may be connected via a bus or other means. Fig. 9 The example of connecting through bus is taken in the following.
[0104] The input device 903 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device used to realize positioning navigation, such as input devices such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick, etc. The output device 904 may include a display device, an auxiliary lighting device (e.g., an LED), and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device may be a touch screen.
[0105] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0106] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.
[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0108] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0109] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0110] According to the technical solution of the embodiment of the present application, by comprehensively utilizing image processing and other technologies to identify information used for auxiliary positioning, and then using this information to assist in positioning the specific road on which the vehicle is currently traveling, it is possible to accurately locate the specific road on which the vehicle is currently traveling in the traffic scene, thereby improving positioning accuracy. In addition, combined with navigation electronic map data information, the navigation positioning accuracy in the preset traffic scene can be improved, thereby improving the navigation user experience, which has practical significance and value in reducing traffic violations and ensuring driving safety. It also has auxiliary decision-making value in emerging fields such as AR navigation and intelligent transportation.
[0111] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.
[0112] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.
Claims
1. A positioning and navigation method, include: Determining whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle; If the vehicle is about to enter the traffic scene, acquiring a road image ahead of the vehicle; Generating positioning auxiliary information of the traffic scene according to the front road image; Determining a target road on which the vehicle is currently traveling according to the positioning assistance information; Obtaining navigation information and providing navigation prompts according to a preset destination and the target road; The generating the positioning auxiliary information of the traffic scene according to the front road image includes: Based on a pre-trained deep neural network model, semantic segmentation is performed on the road ahead image to obtain traffic element information; Extracting positioning auxiliary information for the traffic scene from the traffic element information; The positioning auxiliary information includes a vehicle-mapped road surface, a road surface area, and the number of lanes. The extracting positioning auxiliary information for the traffic scene from the traffic element information includes: According to the results of semantic segmentation, the road surface area is extracted; According to the order from left to right, the road surface area is divided into a first road surface and a second road surface; Extracting the position of the vehicle from the semantic segmentation image; Determining the association relationship between the vehicle and the first road surface and the second road surface, and determining the vehicle mapping road surface; According to the image semantic segmentation result, preprocessing the semantic segmentation image to extract lane lines; According to the lane line extraction result, the intersection is calculated with the vehicle mapped road surface to determine the number of lane lines on the vehicle mapped road surface; Determining the number of lanes of the road surface mapped by the vehicle based on the number of lane lines; The traffic scene includes: a first road surface and a second road surface arranged on the left and right by an isolation device, the first road surface and the second road surface being roads for traveling in the same direction; The step of determining the target road on which the vehicle is currently traveling according to the positioning auxiliary information includes: Determining whether there is a road surface on the right side of the vehicle according to the vehicle mapped road surface and the road surface area surface; If there is a road surface on the right side of the vehicle, determining that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene; If there is no road surface on the right side of the vehicle, determining whether there is a road surface on the left side of the vehicle; If there is a road surface on the left side of the vehicle, determining that the target road on which the vehicle is currently traveling is a second road surface in the traffic scene; If there is no road surface on the left side of the vehicle, obtaining navigation data of the traffic scene; According to the number of lanes in the positioning auxiliary information and the number of lanes in the navigation data, it is identified whether the target road on which the vehicle is currently traveling is the first road surface or the second road surface in the traffic scene.
2. The positioning and navigation method according to claim 1, in, The traffic scene includes: a first road surface and a second road surface arranged up and down by a bridge device, the first road surface and the second road surface are up and down same-direction driving roads; the positioning auxiliary information includes information on whether there is a bridge body; The step of determining the target road on which the vehicle is currently traveling according to the positioning auxiliary information includes: If the positioning auxiliary information contains bridge information, determining that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene; If the positioning auxiliary information is no bridge information, it is determined that the target road on which the vehicle is currently traveling is the second road surface in the traffic scene.
3. A positioning and navigation device, include: A judgment module, used to judge whether the vehicle is about to enter a preset traffic scene according to the real-time position of the vehicle; A first acquisition module, used for acquiring a road image ahead of the vehicle when the vehicle is about to enter the traffic scene; A generating module, used for generating positioning auxiliary information of the traffic scene according to the front road image; A determination module, used to determine the target road that the vehicle is currently traveling on according to the positioning auxiliary information; A navigation prompt module, used to obtain navigation information and provide navigation prompts according to a preset destination and the target road; Wherein, the generation module is specifically used for: Based on a pre-trained deep neural network model, semantic segmentation is performed on the road ahead image to obtain traffic element information; Extracting positioning auxiliary information for the traffic scene from the traffic element information; The positioning auxiliary information includes the vehicle mapping road surface, road surface area and lane number, and the generation module is further specifically used for: According to the results of semantic segmentation, the road surface area is extracted; According to the order from left to right, the road surface area is divided into a first road surface and a second road surface; Extracting the position of the vehicle from the semantic segmentation image; Determining the association relationship between the vehicle and the first road surface and the second road surface, and determining the vehicle mapping road surface; According to the image semantic segmentation result, preprocessing the semantic segmentation image to extract lane lines; According to the lane line extraction result, the intersection is calculated with the vehicle mapped road surface to determine the number of lane lines on the vehicle mapped road surface; Determining the number of lanes of the road surface mapped by the vehicle based on the number of lane lines; The traffic scene includes: a first road surface and a second road surface arranged on the left and right by an isolation device, the first road surface and the second road surface being roads for traveling in the same direction; The determination module is specifically used for: Determining whether there is a road surface on the right side of the vehicle according to the vehicle mapped road surface and the road surface area surface; If there is a road surface on the right side of the vehicle, determining that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene; If there is no road surface on the right side of the vehicle, determining whether there is a road surface on the left side of the vehicle; If there is a road surface on the left side of the vehicle, determining that the target road on which the vehicle is currently traveling is a second road surface in the traffic scene; The device also includes: A second acquisition module, used for acquiring navigation data of the traffic scene when there is no road surface on the left side of the vehicle; The identification module is used to identify whether the target road on which the vehicle is currently traveling is the first road surface or the second road surface in the traffic scene according to the number of lanes in the positioning auxiliary information and the number of lanes in the navigation data.
4. The positioning navigation device according to claim 3, in, The traffic scene includes: a first road surface and a second road surface arranged up and down by a bridge device, the first road surface and the second road surface are up and down same-direction driving roads; the positioning auxiliary information includes information on whether there is a bridge body; The determination module is specifically used for: If the positioning auxiliary information contains bridge information, determining that the target road on which the vehicle is currently traveling is the first road surface in the traffic scene; If the positioning auxiliary information is no bridge information, it is determined that the target road on which the vehicle is currently traveling is the second road surface in the traffic scene.
5. An electronic device, include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the positioning and navigation method according to any one of claims 1 to 2.
6. A non-transitory computer-readable storage medium storing computer instructions, in, The computer instructions are used to enable the computer to execute the positioning and navigation method according to any one of claims 1 to 2.
7. A computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the positioning and navigation method according to any one of claims 1-2.
Citation Information
Cited By
Systems and methods for localizing a vehicle on a map
US20260146866A1
