Navigation guide line generation method and device, medium and electronic equipment
By performing visual detection and dot replacement methods in AR-HUD, it is generated and rendered the navigation guidance line to solve the problem of beyond or covered with the prohibited area, and improved the accuracy and security of navigation.
Patent Information
- Application Number
- CN202411214792.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-30
- Publication Date
- 2025-01-28
AI Technical Summary
When AR-HUD displays road guide lines, it is easy to cause the guidance line indication range exceeds the driving area or covers the prohibited area, which reduces the accuracy and safety of navigation.
By obtaining the target image and navigation guidance, the visual detection is performed to determine the driving area, determine the search point and reference point corresponding to each navigation guidance point, and replace the navigation guidance point as the target search point located in the feasible driving area, connect to the connection These target search points to generate navigation guidance lines.
Without increasing the calculation cost and time cost, the restrictions on the navigation guidance line in the driving area have been realized, the accuracy and safety of the navigation are improved, and the safety of the driver during driving.
Smart Images
Figure CN119380296A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure belongs to the field of navigation technology, and in particular, relates to a method for generating a navigation guide line, a device for generating a navigation guide line, a computer-readable storage medium, and an electronic device. Background Art
[0002] With the continuous development and breakthrough of AR (Augmented Reality) and image recognition technologies, the in-vehicle AR-HUD (Augmented Reality Head-Up Display) uses the windshield as the projection medium, generates images through digital micromirror elements, and projects these enhanced display information into the driver's field of view, integrating it with traffic conditions to provide accurate navigation, warning prompts and other information, which accurately matches the real-world environment. This technology not only improves driving safety through windshield imaging, but also provides richer interactive information, allowing drivers to keep a continuous focus on the road ahead and avoid potential dangers caused by looking down at traditional instruments or car computers.
[0003] However, when AR-HUD displays road guide lines, it is easy for the road guide line indication range to exceed the drivable area, and the displayed road guide lines even frequently cover prohibited areas, which reduces the accuracy and safety of AR-HUD navigation and poses a great safety hazard to the driver during driving. Summary of the invention
[0004] In order to overcome the problems existing in the related art, the present disclosure provides a method for generating a navigation guide line, a device for generating a navigation guide line, a computer-readable storage medium and an electronic device.
[0005] According to a first aspect of an embodiment of the present disclosure, a method for generating a navigation guide line is provided, the method comprising: Acquire a target image and a plurality of navigation guidance points corresponding to the target image, and perform visual detection processing on the target image to obtain a drivable area; Determine a plurality of search points corresponding to each of the navigation guide points, and determine a reference point column according to each of the search points, wherein the plurality of search points are arranged around the navigation guide point, and the reference point column includes a plurality of reference points arranged around the search point; replacing the navigation guide point with a target search point, wherein the target search point corresponds to the target reference point column, and the target reference point column is a reference point column located within the drivable area; A plurality of the target search points are connected to obtain a navigation guide line, and the navigation guide line is rendered on the target image.
[0006] Optionally, the acquiring a target image and a plurality of navigation guide points corresponding to the target image includes: Acquire a target image and a plurality of initial guiding points corresponding to the target image, wherein the initial guiding points are in a first coordinate system; The initial guide point is processed by perspective mapping to obtain a navigation guide point on a second coordinate system.
[0007] Optionally, the connecting a plurality of the target search points to obtain a navigation guide line comprises: Performing inverse perspective mapping processing on the plurality of target search points to obtain target search points on the first coordinate system; A navigation guide line is obtained by connecting a plurality of target search points on the first coordinate system using a target curve.
[0008] Optionally, performing visual detection processing on the target image to obtain a drivable area includes: The target image is input into a visual detection model so that the visual detection model outputs a mask image of a drivable area.
[0009] Optionally, replacing the navigation guide point with a target search point includes: Acquire the point column coordinates of each reference point in the reference point column, and query the point column pixel value at the point column coordinates in the mask image; When it is determined according to the pixel value of the point sequence that each of the reference points in the reference point sequence is in the drivable area, taking the reference point sequence as the target reference point sequence; The target search point is determined according to the target reference point list, and the navigation guide point is replaced by the target search point.
[0010] Optionally, the determining a plurality of search points corresponding to each of the navigation guide points, and determining a reference point column according to each of the search points, comprises: Determine a plurality of search points on a first circumference corresponding to each of the navigation guide points, wherein the first circumference is a circle with the navigation guide point as the center and a first value as the radius; Determine a reference point sequence on a second circumference corresponding to each of the search points, wherein the second circumference is a circle with the search point as the center and a second value as the radius; The second value is smaller than the first value.
[0011] Optionally, the step of determining a plurality of search points corresponding to each of the navigation guide points, and determining a reference point column according to each of the search points, further includes: When it is determined according to the pixel values of the point sequence that the reference point sequence of all the search points on the first circumference is not in the drivable area, a plurality of search points on a third circumference corresponding to each of the navigation guide points are determined, wherein the third circumference is a circle with the navigation guide point as the center and a third value as the radius, and the third value is greater than the first value; Repeat the steps of determining a corresponding reference point column according to a plurality of search points on the third circumference and judging whether the corresponding reference point column is in the drivable area until the target search point corresponding to each navigation guide point is found.
[0012] According to a second aspect of an embodiment of the present disclosure, there is provided a device for generating a navigation guide line, comprising: A visual detection module is configured to obtain a target image and a plurality of navigation guidance points corresponding to the target image, and perform visual detection processing on the target image to obtain a drivable area; a point sequence determination module, configured to determine a plurality of search points corresponding to each of the navigation guide points, and determine a reference point sequence according to each of the search points, wherein the plurality of search points are arranged around the navigation guide point, and the reference point sequence includes a plurality of reference points arranged around the search point; a point sequence determination module, configured to replace the navigation guide point with a target search point, the target search point corresponding to the target reference point sequence, the target reference point sequence being a reference point sequence located within the drivable area; The navigation generation model is configured to connect a plurality of the target search points to obtain a navigation guide line, and render the navigation guide line on the target image.
[0013] According to a third aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored. When the program instructions are executed by a processor, the steps of the method for generating a navigation guide line provided in any one of the first aspects of the present disclosure are implemented.
[0014] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: execute the executable instructions to implement the steps of any one of the navigation guide line generation methods provided in the first aspect of the present disclosure.
[0015] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects: In the method and device provided by the exemplary embodiment of the present disclosure, a search point is determined according to a navigation guide point, a reference point column is determined according to the search point, a reference point column located in a drivable area is used as a target reference point column, a search point corresponding to the target reference point column is used as a target search point, the navigation guide point is replaced with the target search point, and a plurality of target search points are connected to obtain a navigation guide line. The method for generating a navigation guide line can achieve the effect of constraining the navigation guide line to the drivable area without increasing the calculation cost and time cost, improve the accuracy and safety of vehicle navigation through the navigation guide line, solve the safety problem of the driver during driving, and provide the driver with a better driving experience.
[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0018] Figure 1 The schematic diagram of the interface of AR-HUD displaying road guide lines in the related art is schematically shown; Figure 2 The schematic diagram of the interface for AR-HUD to generate road guide lines in the related art is shown schematically; Figure 3 A schematic diagram of a process of generating a navigation guide line in an exemplary embodiment of the present disclosure is schematically shown; Figure 4 A schematic diagram schematically shows a flow chart of a method for acquiring navigation guidance points in an exemplary embodiment of the present disclosure; Figure 5 Schematically showing an interface diagram of a drivable area and a navigation guide point in an exemplary embodiment of the present disclosure; Figure 6 A schematic diagram schematically shows a flow chart of a method for determining a reference point sequence in an exemplary embodiment of the present disclosure; Figure 7 The following schematically shows an interface diagram for determining a search point and a reference point list in an exemplary embodiment of the present disclosure; Figure 8 A schematic flowchart of a method for replacing a navigation guide point with a target search point in an exemplary embodiment of the present disclosure is schematically shown; Fig. 9 A schematic diagram of a process for determining a target search point in an exemplary embodiment of the present disclosure is schematically shown; Fig.10A schematic diagram of a process of obtaining a navigation guide line in an exemplary embodiment of the present disclosure is schematically shown; Fig.11 A schematic diagram schematically shows the structure of a navigation guide line generation device in an exemplary embodiment of the present disclosure; Fig.12 A schematic diagram schematically shows the structure of another device for generating a navigation guide line in an exemplary embodiment of the present disclosure; Fig.13 A schematic structural diagram of yet another navigation guide line generation device in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION
[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0020] It should be noted that all actions of acquiring signals, information or data in the present disclosure are carried out in compliance with the relevant data protection laws and policies of the country where the device is located and with the authorization given by the owner of the corresponding device.
[0021] With the continuous development and breakthroughs of technologies such as AR and image recognition, the in-vehicle AR-HUD uses the windshield as the projection medium, generates image elements through digital micromirror elements, and projects these enhanced display information into the driver's field of view, integrating it with traffic conditions to provide accurate navigation, early warning prompts and other information, which accurately matches the real-world environment.
[0022] This technology not only improves driving safety through windshield imaging, but also provides richer interactive information, allowing drivers to maintain continuous attention on the road ahead and avoid potential dangers caused by lowering their heads to check traditional instruments or car computers.
[0023] Figure 1 FIG. 1 shows an interface schematic diagram of an AR-HUD displaying a road guide line in a related art, such as Figure 1 As shown, one of the main functions of AR-HUD is to display road guide lines. However, during the display process, the guided road guide lines cover the guardrail area where passage is prohibited, causing the road guide lines to exceed the drivable area, reducing the accuracy and safety of AR-HUD navigation and posing a great safety hazard to the driver during driving.
[0024] Figure 2 FIG. 4 shows an interface diagram of AR-HUD generating road guide lines in the related art. Figure 2 As shown in the figure, the black dots are the point arrays that form the road guide line, the asterisk indicates the location of the vehicle, the circle area indicates the intersection, and the curve indicates the road guide line generated based on the road conditions and the point array. It can be seen that under normal and accurate navigation conditions, the road guide line can guide the vehicle to determine the best driving route according to the current road conditions and improve driving safety.
[0025] In view of the problems existing in the related art, the present invention provides a method for generating a navigation guide line. Figure 3 is a flowchart of a method for generating a navigation guide line according to an exemplary embodiment. Figure 3 As shown, the method may at least include the following steps: Step S310: Acquire a target image and a plurality of navigation guidance points corresponding to the target image, and perform visual detection processing on the target image to obtain a drivable area.
[0026] Step S320. Determine a plurality of search points corresponding to each navigation guide point, and determine a reference point column according to each search point, wherein the plurality of search points are arranged around the navigation guide point, and the reference point column includes a plurality of reference points arranged around the search point.
[0027] Step S330: Replace the navigation guidance point with a target search point, where the target search point corresponds to a target reference point column, and the target reference point column is a reference point column located within the drivable area.
[0028] Step S340: Connect multiple target search points to obtain a navigation guide line, and render the navigation guide line on the target image.
[0029] In an exemplary embodiment of the present disclosure, a search point is determined according to a navigation guide point, a reference point column is determined according to the search point, a reference point column located in a drivable area is used as a target reference point column, a search point corresponding to the target reference point column is used as a target search point, the navigation guide point is replaced with the target search point, and a plurality of target search points are connected to obtain a navigation guide line. Since all reference points in the target reference point column are located in the drivable area, the target search point used to replace the navigation guide point is also located in the drivable area, and the effect of constraining the navigation guide line to the drivable area can be achieved without increasing the calculation cost and time cost, while the navigation guide line maintains the original indication direction, thereby improving the accuracy and safety of vehicle navigation through the navigation guide line, solving the safety problem of the driver during driving, and providing the driver with a better driving experience.
[0030] The following is a detailed description of each step of the method for generating a navigation guide line.
[0031] In step S310, a target image and a plurality of navigation guidance points corresponding to the target image are acquired, and visual detection processing is performed on the target image to obtain a drivable area.
[0032] In an exemplary embodiment of the present disclosure, the target image may be an image currently captured by a camera of the vehicle, or may be an image obtained by other means, and this exemplary embodiment does not specifically limit this.
[0033] In an alternative embodiment, Figure 4 A flow chart of a method for obtaining navigation guidance points is shown. Figure 4 As shown, the method may at least include the following steps: In step S410, a target image and a plurality of initial guiding points corresponding to the target image are acquired, and the initial guiding points are on a first coordinate system.
[0034] The initial guidance point may be an initial navigation guidance point provided by the navigation system and marked on the first coordinate system representing the overhead plane. Generally, the navigation guidance point provided by the navigation system is marked every 10 meters.
[0035] In step S420, perspective mapping is performed on the initial guidance point to obtain a navigation guidance point on the second coordinate system.
[0036] When the vehicle leaves the factory, a conversion matrix between coordinates is calibrated between the first coordinate system and the second coordinate system. The conversion matrix may generally include 8 parameters.
[0037] Therefore, by using the transformation matrix for perspective mapping, the initial guide point on the first coordinate system can be transformed into a navigation guide point on the second coordinate system, so that the initial guide point is perspectively mapped onto the pixel plane, which is the plane where the vehicle camera captures the target image.
[0038] The first coordinate system is provided by the navigation system and represents the coordinate system of the top-view plane, which is the same as the ground plane. The second coordinate system is the coordinate system of the pixel plane. The pixel plane is the plane where the vehicle camera captures the target image.
[0039] In an optional embodiment, the target image is input into a visual detection model so that the visual detection model outputs a mask image of the drivable area.
[0040] After the target image is acquired, the target image may be input into a visual detection model to determine a drivable area in the target image.
[0041] The visual detection model may be YOLOPv2 (a real-time target detection framework) or other models, which is not particularly limited in this exemplary embodiment.
[0042] YOLOPv2 is a new type of neural network that is mainly used for panoramic perception in the field of autonomous driving. It integrates three tasks: traffic object detection, drivable road area segmentation, and lane detection in a single model. While achieving these tasks, YOLOPv2 also achieves the best performance of the current technical level.
[0043] The key features of YOLOPv2 include: Multi-task learning: YOLOPv2 is able to handle multiple tasks simultaneously, which improves the efficiency and accuracy of the autonomous driving system.
[0044] Real-time performance: The network is designed with real-time performance in mind, which is critical for autonomous driving systems.
[0045] Panoramic Perception: By integrating different perception tasks, YOLOPv2 is able to provide a more comprehensive understanding of the surrounding environment.
[0046] When YOLOPv2 is used as a visual detection model, the visual detection model can output a mask image of the drivable area.
[0047] The mask image has the same size as the target image, for example, both are 640×480, and is a matrix M with each pixel value being 0 or 1. Specifically, a pixel value of 1 represents that the corresponding position is a drivable area, and a pixel value of 0 represents that the corresponding position is a non-drivable area. M is output by the visual detection model.
[0048] Figure 5 The schematic diagram of the interface showing the drivable area and navigation guide points is shown in FIG. Figure 5 As shown, the middle part is the drivable road area, and the black dots are multiple navigation guidance points.
[0049] In step S320, multiple search points corresponding to each navigation guide point are determined, and a reference point column is determined according to each search point, the multiple search points are arranged around the navigation guide point, and the reference point column includes multiple reference points arranged around the search point.
[0050] In an exemplary embodiment of the present disclosure, in order to judge whether a plurality of navigation guidance points are within the drivable area, it may be determined whether a plurality of search points and reference point columns corresponding to each navigation guidance point are within the drivable area.
[0051] In an alternative embodiment, Figure 6 A flow chart of a method for determining a reference point sequence is shown, Figure 6 As shown, the method may at least include the following steps: in step S610, multiple search points on a first circle corresponding to each navigation guide point are determined, and the first circle is a circle with the navigation guide point as the center and a first value as the radius.
[0052] Exemplarily, for each navigation guide point, a circle with a gradually increasing radius r is made to obtain an initial first circumference, and eight points with an interval of 45 degrees are taken on the first circumference as eight search points. The unit of the radius r is pixel, and the initial value is 0 pixels. Then, the radius is increased by 10 pixels each time to expand outward to determine the next round of search points. In the process of determining the search points, the interval angle, the number of selections, the increment of the radius, etc. can be selected according to the actual situation and needs, and this exemplary embodiment does not make any special restrictions on this.
[0053] In step S620, a reference point sequence on a second circumference corresponding to each search point is determined, where the second circumference is a circle with the search point as the center and the second value as the radius; The second value is smaller than the first value.
[0054] Furthermore, for each search point on the first circle, with this search point as the center and the radius as =3 pixels to make a circle, obtain a second circumference, and select eight points on the second circumference with an interval of 45 degrees as eight reference points, and the eight reference points constitute a reference point sequence corresponding to the search point. In the process of determining the reference point sequence, the interval angle, the number of selected points, etc. can also be selected according to actual conditions and needs, and this exemplary embodiment does not make any special restrictions on this.
[0055] Figure 7 A schematic diagram of the interface for determining the search point and reference point list is shown, Figure 7 As shown, 710 is a search point, which is 8 in number; 720 is a reference point column, and the reference point column corresponding to each search point is also composed of 8 reference points.
[0056] Thus, it is determined whether all the small areas where the reference point array is located are in the drivable area. Specifically, it is taken as Eight points on a circle with a radius of 45 degrees are used as reference point sequences, and it is determined whether the reference point sequences are all in the drivable area.
[0057] In step S330, the navigation guidance point is replaced with a target search point, the target search point corresponds to a target reference point column, and the target reference point column is a reference point column located within the drivable area.
[0058] In an exemplary embodiment of the present disclosure, after determining multiple search points corresponding to a navigation guidance point and determining a reference point column according to each search point, a target search point is selected according to whether the reference point column is a target reference point column, thereby replacing the navigation guidance point with the target search point.
[0059] In an alternative embodiment, Figure 8A flow chart of a method for replacing a navigation guide point with a target search point is shown, Figure 8 As shown, the method may at least include the following steps: In step S810, the point column coordinates of each reference point in the reference point column are obtained, and the point column pixel value at the point column coordinates is queried in the mask image.
[0060] Specifically, you can get each reference point in the reference point column The pixel coordinates, that is, the point column coordinates , and obtain the point column coordinates on the mask image The corresponding pixel value of the point column .
[0061] In step S820, when it is determined based on the pixel value of the reference point sequence that each reference point of the reference point sequence is in the drivable area, the reference point sequence is used as the target reference point sequence.
[0062] After getting the pixel value of the point column After that, the pixel value of the point column can be determined Is it equal to 1?
[0063] When the pixel value of the point column When equal to 1, it indicates that the reference point in the reference point column In the drivable area until a certain surrounding radius is found The reference points included in the reference point column are all reference points in the drivable area, the iteration is terminated, and the reference point column is determined to be the target reference point column.
[0064] In step S830, a target search point is determined according to the target reference point list, and the navigation guide point is replaced with the target search point.
[0065] When a certain surrounding radius is found When the target reference point column is all reference points in the drivable area, the search point surrounded by the target reference point column is determined as the target search point, and the navigation guide point is replaced by the target search point.
[0066] It is worth noting that for each navigation guidance point, when determining the search point and reference point column, and judging whether to use the search point to replace the corresponding navigation guidance point, the corresponding search point and reference point column can be judged in a clockwise direction, so that when it is determined that the first group of reference point columns are in a drivable area, the navigation guidance point is replaced by the corresponding search point, without the need for subsequent judgment and replacement of the remaining unjudged search points and reference point columns.
[0067] In an alternative embodiment, Fig. 9 A flow chart of a method for determining a target search point is shown, Fig. 9As shown, the method may include at least the following steps: in step S910, when it is determined based on the pixel values of the point columns that the reference point columns of all the search points on the first circle are not in the drivable area, multiple search points on the third circle corresponding to each navigation guide point are determined, and the third circle is a circle with the navigation guide point as the center and a third value as the radius, and the third value is greater than the first value.
[0068] After getting the pixel value of the point column After that, the pixel value of the point column can be determined Is it equal to 1?
[0069] For the reference points in the reference point column of all search points on the first circumference, when the pixel value of the point column When it is equal to 0, it indicates that the reference point in the reference point column It is not in the drivable area, therefore, a third circumference can be obtained by making a circle with the navigation guidance point as the center and the third value as the radius.
[0070] Eight points on the third circle with an interval of 45 degrees are selected as eight search points on the third circle, so as to further find the target search point among the search points on the third circle.
[0071] In step S920, the steps of determining the corresponding reference point sequence according to the multiple search points on the third circumference and judging whether the corresponding reference point sequence is in the drivable area are repeated until the target search point corresponding to each navigation guide point is found.
[0072] After determining multiple search points on the third circle, step S620, step S810-step S830 may be repeated to confirm the target search point and replace the navigation guide point with the target search point, which will not be described in detail herein.
[0073] This ensures that each navigation guidance point can find the corresponding target search point within the drivable area, ensures that the consistency of the final generated navigation guidance line with the original indication direction is not affected, and improves the accuracy of the navigation guidance line.
[0074] It is worth noting that, for each navigation guide point, when determining the search point, determining the reference point column according to each search point, and judging whether to determine the search point as the target search point according to the reference point column, and replacing the corresponding navigation guide point with the target search point, the corresponding search point, reference point column, target reference point column and target search point can be determined and searched in a clockwise direction, so that when the first group of target reference point columns are determined to be in the drivable area, the navigation guide point can be replaced with the corresponding target search point, without the need to perform subsequent judgment and replacement on the remaining unjudged search point reference point column, target reference point column and target search point. In addition, other search orders can be adopted according to actual conditions and needs, or a screening method can be adopted when multiple target search points are found, and this exemplary embodiment does not make special restrictions on this.
[0075] In step S340, a plurality of target search points are connected to obtain a navigation guide line, and the navigation guide line is rendered on the target image.
[0076] In an exemplary embodiment of the present disclosure, after the navigation guide point is replaced with the target search point, a plurality of replaced target search points may be connected to obtain a navigation guide line.
[0077] In an alternative embodiment, Fig.10 A flow chart of a method for obtaining a navigation guide line is shown in FIG. Fig.10 As shown, the method may at least include the following steps: In step S1010, a plurality of target search points are subjected to inverse perspective mapping to obtain target search points on a first coordinate system.
[0078] Since the transformation matrix of the coordinates of the two coordinate systems is calibrated between the first coordinate system and the second coordinate system, the corresponding inverse transformation matrix can be determined based on the transformation matrix, and the inverse transformation matrix can be used to perform inverse perspective mapping processing to transform multiple target search points on the second coordinate system into target search points on the first coordinate system, so that the target search points are mapped back to the bird's-eye view plane.
[0079] In step S1020, a navigation guide line is obtained by connecting a plurality of target search points on the first coordinate system using a target curve.
[0080] Generally, a Bezier spline curve may be used to connect multiple target search points on the first coordinate system to obtain a navigation guide line.
[0081] A Bezier spline is a mathematical curve commonly used in computer graphics that defines the shape of a curve through a series of control points. A Bezier spline is made up of multiple Bezier curves, each with its own set of control points but sharing endpoints with adjacent curves. To ensure the continuity and smoothness of the spline at the joint, the curves are usually required to have the same tangent or curvature at the joint point.
[0082] Generally, a Bezier spline curve may be used to connect multiple target search points on the first coordinate system to obtain a navigation guide line.
[0083] Furthermore, the navigation guide line is rendered on the AR-HUD through OpenGL (Open Graphics Library), so that the entire point column of the navigation guide line is moved to the nearest drivable area.
[0084] It is worth noting that the computing power consumed by this method depends on the ratio of the increment of radius r to the image size. For example, when the image size is 640x480 and the increment of r is 10, the iteration is at most 64 times, which is a negligible amount of computation, so it will not increase the computational cost. Even when the image size is larger than 640x480, the image compression method can be used to process the image size to 640x480 to reduce the burden of computational cost in the case of any image size.
[0085] In an exemplary embodiment of the present disclosure, a search point is determined according to a navigation guide point, a reference point column is determined according to the search point, a reference point column located in a drivable area is used as a target reference point column, a search point corresponding to the target reference point column is used as a target search point, the navigation guide point is replaced with the target search point, and a plurality of target search points are connected to obtain a navigation guide line. The method for generating a navigation guide line can achieve the effect of constraining the navigation guide line to the drivable area without increasing the calculation cost and time cost, improve the accuracy and safety of vehicle navigation through the navigation guide line, solve the safety problem of the driver during driving, and provide the driver with a better driving experience.
[0086] In addition, in an exemplary embodiment of the present disclosure, a device for generating a navigation guide line is also provided. Fig.11 The schematic diagram of the structure of the device for generating the navigation guide line is shown in FIG. Fig.11 As shown, the navigation guide line generation device 1100 may include: a visual detection module 1110, a point sequence determination module 1120, a point sequence judgment module 1130 and a navigation generation model 1140. Among them: The visual detection module 1110 is configured to obtain a target image and a plurality of navigation guidance points corresponding to the target image, and perform visual detection processing on the target image to obtain a drivable area; The point sequence determining module 1120 is configured to determine a plurality of search points corresponding to each of the navigation guide points, and determine a reference point sequence according to each of the search points, wherein the plurality of search points are arranged around the navigation guide point, and the reference point sequence includes a plurality of reference points arranged around the search point; The point sequence determination module 1130 is configured to replace the navigation guidance point with a target search point, wherein the target search point corresponds to the target reference point sequence, and the target reference point sequence is a reference point sequence located within the drivable area; The navigation generation model 1140 is configured to connect a plurality of the target search points to obtain a navigation guide line, and render the navigation guide line on the target image.
[0087] In some embodiments of the present disclosure, the visual detection module 1110 is configured to: Acquire a target image and a plurality of initial guiding points corresponding to the target image, wherein the initial guiding points are in a first coordinate system; The initial guide point is processed by perspective mapping to obtain a navigation guide point on a second coordinate system.
[0088] In some embodiments of the present disclosure, the navigation generation model 1140 is configured to: Performing inverse perspective mapping processing on the plurality of target search points to obtain target search points on the first coordinate system; A navigation guide line is obtained by connecting a plurality of target search points on the first coordinate system using a target curve.
[0089] In some embodiments of the present disclosure, the visual detection module 1110 is configured to: The target image is input into a visual detection model so that the visual detection model outputs a mask image of a drivable area.
[0090] In some embodiments of the present disclosure, the point sequence determination module 1130 is configured to: Acquire the point column coordinates of each reference point in the reference point column, and query the point column pixel value at the point column coordinates in the mask image; When it is determined according to the pixel value of the point sequence that each of the reference points in the reference point sequence is in the drivable area, taking the reference point sequence as the target reference point sequence; The target search point is determined according to the target reference point list, and the navigation guide point is replaced by the target search point.
[0091] In some embodiments of the present disclosure, the point sequence determination module 1120 is configured to: Determine a plurality of search points on a first circumference corresponding to each of the navigation guide points, wherein the first circumference is a circle with the navigation guide point as the center and a first value as the radius; Determine a reference point sequence on a second circumference corresponding to each of the search points, wherein the second circumference is a circle with the search point as the center and a second value as the radius; The second value is smaller than the first value.
[0092] In some embodiments of the present disclosure, the point sequence determination module 1120 is further configured to: When it is determined according to the pixel values of the point sequence that the reference point sequence of all the search points on the first circumference is not in the drivable area, a plurality of search points on a third circumference corresponding to each of the navigation guide points are determined, wherein the third circumference is a circle with the navigation guide point as the center and a third value as the radius, and the third value is greater than the first value; Repeat the steps of determining a corresponding reference point column according to a plurality of search points on the third circumference and judging whether the corresponding reference point column is in the drivable area until the target search point corresponding to each navigation guide point is found.
[0093] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0094] The present disclosure also provides a computer-readable storage medium having computer program instructions stored thereon, and when the program instructions are executed by a processor, the steps of the method for generating a navigation guide line provided by the present disclosure are implemented.
[0095] Fig.12 1 is a block diagram of another apparatus 1200 for generating a navigation guide line according to an exemplary embodiment. For example, the apparatus 1200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.
[0096] Reference Fig.12 , the device 1200 may include one or more of the following components: a processing component 1202 , a memory 1204 , a power component 1206 , a multimedia component 1208 , an audio component 1210 , an input / output interface 1212 , a sensor component 1214 , and a communication component 1216 .
[0097] The processing component 1202 generally controls the overall operation of the device 1200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 1202 may include one or more processors 1220 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 1202 may include one or more modules to facilitate the interaction between the processing component 1202 and other components. For example, the processing component 1202 may include a multimedia module to facilitate the interaction between the multimedia component 1208 and the processing component 1202.
[0098] The memory 1204 is configured to store various types of data to support the operation of the device 1200. Examples of such data include instructions for any application or method operating on the device 1200, contact data, phone book data, messages, pictures, videos, etc. The memory 1204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0099] The power supply component 1206 provides power to the various components of the device 1200. The power supply component 1206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1200.
[0100] The multimedia component 1208 includes a screen that provides an output interface between the device 1200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 1208 includes a front camera and / or a rear camera. When the device 1200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0101] The audio component 1210 is configured to output and / or input audio signals. For example, the audio component 1210 includes a microphone (MIC), and when the device 1200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1204 or sent via the communication component 1216. In some embodiments, the audio component 1210 also includes a speaker for outputting audio signals.
[0102] The input / output interface 1212 provides an interface between the processing component 1202 and the peripheral interface modules, which may be keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0103] The sensor assembly 1214 includes one or more sensors for providing various aspects of the status assessment of the device 1200. For example, the sensor assembly 1214 can detect the open / closed state of the device 1200, the relative positioning of components, such as the display and keypad of the device 1200, the sensor assembly 1214 can also detect the position change of the device 1200 or a component of the device 1200, the presence or absence of user contact with the device 1200, the orientation or acceleration / deceleration of the device 1200, and the temperature change of the device 1200. The sensor assembly 1214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 1214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 1214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0104] The communication component 1216 is configured to facilitate wired or wireless communication between the device 1200 and other devices. The device 1200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 1216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 1216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0105] In an exemplary embodiment, the apparatus 1200 may be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.
[0106] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1204 including instructions, and the instructions can be executed by the processor 1220 of the device 1200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0107] In addition to being an independent electronic device, the above-mentioned device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip, wherein the integrated circuit can be an IC or a collection of multiple ICs; the chip can include but is not limited to the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip, SoC), etc. The above-mentioned integrated circuit or chip can be used to execute executable instructions (or codes) to implement the above-mentioned method for generating a navigation guide line. The executable instructions can be stored in the integrated circuit or chip, or can be obtained from other devices or equipment, for example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instruction may be stored in the memory, and when the executable instruction is executed by the processor, the above-mentioned method for generating a navigation guide line may be implemented; alternatively, the integrated circuit or chip may receive the executable instruction through the interface and transmit it to the processor for execution, so as to implement the above-mentioned method.
[0108] In another exemplary embodiment, a computer program product is further provided. The computer program product includes a computer program executable by a programmable device. The computer program has a code portion for executing the above-mentioned method for generating a navigation guide line when executed by the programmable device.
[0109] Fig.13 FIG. 1 is a block diagram of another apparatus 1300 for generating a navigation guide line according to an exemplary embodiment. For example, the apparatus 1300 may be provided as a server. Fig.13 The device 1300 includes a processing component 1322, which further includes one or more processors, and a memory resource represented by a memory 1332 for storing instructions executable by the processing component 1322, such as an application. The application stored in the memory 1332 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 1322 is configured to execute instructions to perform the above-mentioned method for generating a navigation guide line.
[0110] The device 1300 may also include a power supply component 1326 configured to perform power management of the device 1300, a wired or wireless network interface 1350 configured to connect the device 1300 to a network, and an input / output interface 1358. The device 1300 may operate based on an operating system stored in the memory 1332.
[0111] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0112] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. A method for generating a navigation guide line, characterized in that: include: Acquire a target image and a plurality of navigation guidance points corresponding to the target image, and perform visual detection processing on the target image to obtain a drivable area; Determine a plurality of search points corresponding to each of the navigation guide points, and determine a reference point column according to each of the search points, wherein the plurality of search points are arranged around the navigation guide point, and the reference point column includes a plurality of reference points arranged around the search point; replacing the navigation guide point with a target search point, wherein the target search point corresponds to the target reference point column, and the target reference point column is a reference point column located within the drivable area; A plurality of the target search points are connected to obtain a navigation guide line, and the navigation guide line is rendered on the target image.
2. The method for generating a navigation guide line according to claim 1, characterized in that: The acquiring of a target image and a plurality of navigation guide points corresponding to the target image comprises: Acquire a target image and a plurality of initial guiding points corresponding to the target image, wherein the initial guiding points are in a first coordinate system; The initial guide point is processed by perspective mapping to obtain a navigation guide point on a second coordinate system.
3. The method for generating a navigation guide line according to claim 2, characterized in that: The step of connecting a plurality of the target search points to obtain a navigation guide line comprises: Performing inverse perspective mapping processing on the plurality of target search points to obtain target search points on the first coordinate system; A navigation guide line is obtained by connecting a plurality of target search points on the first coordinate system using a target curve.
4. The method for generating a navigation guide line according to claim 1, characterized in that: The step of performing visual detection processing on the target image to obtain a drivable area includes: The target image is input into a visual detection model so that the visual detection model outputs a mask image of a drivable area.
5. The method for generating a navigation guide line according to claim 4, characterized in that: The step of replacing the navigation guide point with the target search point comprises: Acquire the point column coordinates of each reference point in the reference point column, and query the point column pixel value at the point column coordinates in the mask image; When it is determined according to the pixel value of the point sequence that each of the reference points in the reference point sequence is in the drivable area, taking the reference point sequence as the target reference point sequence; The target search point is determined according to the target reference point list, and the navigation guide point is replaced by the target search point.
6. The method for generating a navigation guide line according to claim 5, characterized in that: The determining of a plurality of search points corresponding to each of the navigation guide points, and determining a reference point column according to each of the search points, comprises: Determine a plurality of search points on a first circumference corresponding to each of the navigation guide points, wherein the first circumference is a circle with the navigation guide point as the center and a first value as the radius; Determine a reference point sequence on a second circumference corresponding to each of the search points, wherein the second circumference is a circle with the search point as the center and a second value as the radius; The second value is smaller than the first value.
7. The method for generating a navigation guide line according to claim 6, characterized in that: The step of determining a plurality of search points corresponding to each of the navigation guide points, and determining a reference point list according to each of the search points, further includes: When it is determined according to the pixel values of the point sequence that the reference point sequence of all the search points on the first circumference is not in the drivable area, a plurality of search points on a third circumference corresponding to each of the navigation guide points are determined, wherein the third circumference is a circle with the navigation guide point as the center and a third value as the radius, and the third value is greater than the first value; Repeat the steps of determining a corresponding reference point column according to a plurality of search points on the third circumference and judging whether the corresponding reference point column is in the drivable area until the target search point corresponding to each navigation guide point is found.
8. A navigation guide line generation device, characterized in that: include: A visual detection module is configured to obtain a target image and a plurality of navigation guidance points corresponding to the target image, and perform visual detection processing on the target image to obtain a drivable area; a point sequence determination module, configured to determine a plurality of search points corresponding to each of the navigation guide points, and determine a reference point sequence according to each of the search points, wherein the plurality of search points are arranged around the navigation guide point, and the reference point sequence includes a plurality of reference points arranged around the search point; a point sequence determination module, configured to replace the navigation guide point with a target search point, the target search point corresponding to the target reference point sequence, the target reference point sequence being a reference point sequence located within the drivable area; The navigation generation model is configured to connect a plurality of the target search points to obtain a navigation guide line, and render the navigation guide line on the target image.
9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: include: a memory having a computer program stored thereon; A processor, configured to execute the computer program in the memory to implement the steps of the method according to any one of claims 1 to 7.