A curve recognition method and system based on multi-scale calculation

Through the combination of multi-scale computing and deep learning, the existing curve recognition method has solved the problem of low credibility in imperfect facilities construction and bad weather, improved the accuracy and robustness of curve recognition, and achieved the correctness of curve recognition Accurate identification of curves.

CN114092910BActive Publication Date: 2024-11-22WUHAN ZHONGHAITING DATA TECH CO LTD
View PDF 5 Cites 0 Cited by

Patent Information

Application Number
CN202111386948.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-11-22
Publication Date
2024-11-22
Estimated Expiration
2041-11-22

AI Technical Summary

Technical Problem

The existing curve identification method fails when the road facilities are not perfect in economically developed areas, and the method that relies on image detection is low in credibility in bad weather, and cannot provide general and effective prior information, and relying solely on curvature operation will be incorrect. Identify tiny terrain fluctuations or ignore large turns.

Method used

Using a multi-scale operation method, the road vector line data is projected into an image, deep learning is used to identify the curve target box, and curvature is calculated through multi-scale plus sliding windows, and the detected target box and identified curve interval are combined for fusion processing. , get the final curve interval.

Benefits of technology

It improves the accuracy and robustness of curve recognition, enhances the generalization of the method, can accurately identify curves under different conditions, reduces the sensitivity to tiny terrain fluctuations, and the ability to identify large turns.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114092910B_ABST
    Figure CN114092910B_ABST
Patent Text Reader

Abstract

The present invention provides a curve recognition method and system based on multi-scale operation, the method comprising: projecting road vector line data into an image based on multiple scales; identifying a curve target frame from the image based on a deep learning method; calculating the curvature of the road vector line data based on a multi-scale plus sliding window method, and identifying the curve interval based on the calculated curvature; fusing the detected curve target frame and the identified curve interval to obtain the identified final curve interval. The present invention combines the method based on deep learning and the method based on curvature operation, respectively using the multi-scale operation method to improve the effect of the curve recognition method.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of curve recognition, and more specifically, to a curve recognition method and system based on multi-scale calculation. Background Art

[0002] Curves often appear on roads such as road ramps and winding mountain roads. Since the turning radius of curved roads is constantly changing, and they are tilted up and down, left and right, drivers need to slow down appropriately and drive with extra caution. The advanced driver assistance system of automobiles provides a curve warning function to assist drivers in obtaining curve information in advance. The curve warning function relies on the correct identification of the curve position.

[0003] Existing methods for identifying curves can be simply divided into two categories according to whether they rely on road paving facilities. One is to identify the starting and ending positions of the curve by receiving signals emitted by road paving equipment. This type of method is highly dependent on the construction of road facilities. In economically underdeveloped areas where road facilities are not well constructed, this type of method will fail. The other type of method is not dependent on the construction of road facilities. There are methods that detect lane lines in the image data of driving records, and then identify curves by fitting the straight lines and curves of the lane lines.

[0004] The disadvantage of this type of method that only detects lane line curves through images is that the image data is a perspective projection, and the straight lines and curves under perspective projection are different from the real road, and the credibility of the image will decrease in rainy days, at night, etc. The data obtained by this method cannot become universal and effective prior information to be shared with other cars. There are also methods that identify curves through curvature calculations of road data in the map. This type of method has a relatively lower cost and a higher recognition rate for curves, and is also the mainstream method currently used by major map suppliers. However, relying solely on curvature calculations will mistakenly identify some tiny fluctuations in the map data as curves, or may fail to recognize some larger turns. Summary of the invention

[0005] In view of the technical problems existing in the prior art, the present invention provides a curve recognition method and system based on multi-scale calculation, which can improve the accuracy of curve recognition.

[0006] According to a first aspect of the present invention, a curve recognition method based on multi-scale operations is provided, comprising: projecting road vector line data into an image based on multiple scales; identifying a curve target frame from the image based on a deep learning method; for the road vector line data, calculating the curvature based on a multi-scale plus sliding window method, and identifying the curve interval based on the calculated curvature; fusing the detected curve target frame and the identified curve interval to obtain the identified final curve interval.

[0007] Based on the above technical solution, the present invention can also make the following improvements.

[0008] Optionally, projecting the road vector line data into an image based on multiple scales includes: evenly dividing the road vector line data into corresponding sequence vector lines according to length distance thresholds of multiple different scales to obtain multiple sequence vector lines; orthogonally projecting each sequence vector line into an image, and recording the projection parameters Para during projection to obtain an image set.

[0009] Optionally, the deep learning-based method is used to identify the curve target frame from the image, including: for each image in the image set, using a preset deep learning detection network to identify the curve target frame therein; according to the projection parameter Para corresponding to each sequence vector line, each identified curve target frame is back-calculated to the coordinate system before projection to obtain a curve target frame set.

[0010] Optionally, for road vector line data, the curvature is calculated based on a multi-scale plus sliding window method, and the curve section is identified based on the calculated curvature, including: for a point P2 on the road vector line, forwardly searching a preset distance d to obtain a point P1 on the vector line, and backwardly searching the same preset distance d to obtain a point P3 on the vector line; according to the positions of the three points, solving the radius of the circle; starting from the starting position of the road vector line, traversing to the end position of the road vector line, obtaining a set of circle radii, and smoothing all circle radii; identifying the curve section based on the calculated circle radius.

[0011] Optionally, the preset distance d is 1m, 5m or 10m.

[0012] Optionally, the identifying of the curve section based on the calculated circle radius includes: if the circle radius |r|>=preset radius threshold, it is a straight section; if |r|<preset radius threshold, it is a curve section.

[0013] Optionally, the detected curve target frame and the identified curve interval are fused to obtain the final identified curve interval, including: determining the intersection interval of the identified curve target frame and the identified curve interval as the curve area; for other intervals, determining the curve interval or the straight interval according to the calculated circle radius.

[0014] According to a second aspect of the present invention, a curve recognition system based on multi-scale operations is provided, comprising: a first recognition module, used to project road vector line data into an image based on multiple scales, and identify a curve target frame from the image based on a deep learning method; a second recognition module, used to calculate the curvature of the road vector line data based on a multi-scale plus sliding window method, and identify the curve section based on the calculated curvature; a fusion processing module, used to perform fusion processing on the detected curve target frame and the identified curve section to obtain the identified final curve section.

[0015] According to a third aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the processor is configured to implement the steps of a curve recognition method based on multi-scale operations when executing a computer management program stored in the memory.

[0016] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium on which a computer management program is stored, and when the computer management program is executed by a processor, the steps of a curve recognition method based on multi-scale operations are implemented.

[0017] The present invention provides a curve recognition method and system based on multi-scale operation, which projects road vector line data into an image based on multiple scales; identifies a curve target frame from the image based on a deep learning method; calculates the curvature of the road vector line data based on a multi-scale plus sliding window method, and identifies the curve interval based on the calculated curvature; performs fusion processing on the detected curve target frame and the identified curve interval to obtain the identified final curve interval. The present invention combines a method based on deep learning and a method based on curvature operation, respectively using a multi-scale operation method, combining a geometric operation method and a deep learning method, cross-extracting and cross-validating different methods, making the best use of their strengths and avoiding their weaknesses, and improving the robustness and generalization of the method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A flow chart of a curve recognition method based on scale calculation provided by the present invention;

[0019] Figure 2 A schematic diagram of a curved road target box identified using a deep learning detection network;

[0020] Figure 3 It is a schematic diagram of calculating the curvature radius based on the vector line morphology points;

[0021] Figure 4 A schematic diagram of the structure of a curve recognition system based on multi-scale calculation provided by the present invention;

[0022] Figure 5 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;

[0023] Figure 6 A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION

[0024] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0025] Embodiment 1

[0026] A curve recognition method based on multi-scale operation, see Figure 1 The curve recognition method includes: projecting road vector line data into an image based on multiple scales; identifying a curve target frame from the image based on a deep learning method; calculating the curvature of the road vector line data based on a multi-scale plus sliding window method, and identifying the curve interval based on the calculated curvature; fusing the detected curve target frame and the identified curve interval to obtain the final identified curve interval.

[0027] It is understandable that based on the drawbacks of various curve recognition methods in the background technology, the present invention proposes a curve recognition method based on multi-scale operation, and its overall technical solution is divided into three parts: the first part is to project the road vector line data into an image at multiple scales, and use a deep learning method to detect the curve target frame; the second part is to perform multi-scale curvature calculations based on the shape information of the road vector line to identify the curve interval; the third part is to comprehensively process the results of curve recognition based on the information of the first and second parts.

[0028] The curves in the road are identified based on the deep learning detection network, and the bending radius of the road section is calculated using geometric operations to identify the curve section. The recognition results of the curve area by the two methods are then merged to obtain the final curve section.

[0029] The present invention combines a method based on deep learning and a method based on curvature calculation, respectively using multi-scale calculation methods, combining geometric calculation methods and deep learning methods, cross-extracting and cross-validating different methods, making the best use of their strengths and avoiding their weaknesses, and improving the robustness and generalization of the method.

[0030] Embodiment 2

[0031] A curve recognition method based on multi-scale operation mainly includes the following steps:

[0032] S1, projecting road vector line data into images based on multiple scales.

[0033] As an embodiment, the projecting of road vector line data into an image based on multiple scales includes: evenly dividing the road vector line data into corresponding sequence vector lines according to length distance thresholds of multiple different scales to obtain multiple sequence vector lines; ortho-projecting each sequence vector line into an image, and recording the projection parameter Para during projection to obtain an image set.

[0034] Specifically, step S1 specifically includes:

[0035] S11: The road vector line data is evenly divided into sequence vector lines according to a certain length l:

[0036] (1) Set the length l to 1 km and evenly divide it into sequence vector lines Q1;

[0037] (2) Set the length l to 5 km and evenly divide it into sequence vector lines Q2;

[0038] (3) Set the length l to 20 km and evenly divide it into sequence vector lines Q3;

[0039] Among them, multi-scale refers to the analysis of curves on road vector line data at various scales, thereby reducing the sensitivity of the method to the fluctuation of micro-data forms and improving the recognition ability of large-scale curve forms. Road vector lines include many common types of geographic vector data such as national two-dimensional maps, conventional digital line maps DLG, and crowd-sourced driving trajectory data.

[0040] S12: orthographically project the sequence vector line sets Q1, Q2, Q3 into image sets l1, l2, and l3 respectively. The image sets l1, l2, and l3 together constitute the image set l, and record the projection parameters Para1, Para2, and Para3 corresponding to the projection of the sequence vector line sets Q1, Q2, Q3.

[0041] It should be noted that three scales are selected for the specific length l. In specific implementation, more different length scales can be used to segment the road vector line data, which will not be explained in detail here.

[0042] S2, identifying a curve target frame from the image based on a deep learning method.

[0043] As an embodiment, the deep learning-based method is used to identify the curve target frame from the image, including: for each image in the image set, using a preset deep learning detection network to identify the curve target frame therein; according to the projection parameter Para corresponding to each sequence vector line, each identified curve target frame is back-calculated to the coordinate system before projection to obtain a curve target frame set.

[0044] Specifically, the step S2 specifically includes:

[0045] S21: On the image set l, manually mark the curve target box to create a dataset;

[0046] S22: Perform deep learning target detection network training, and obtain the network ANet after training convergence;

[0047] S23: Use the ANet network to detect the curved target and obtain the curved target frame;

[0048] S24: The curve target frame detected by S23 is back-calculated to the coordinate system before projection according to the corresponding projection parameters recorded in S12, and recorded as the deep learning curve set Cl.

[0049] Among them, step S24 can be understood as, for each sequence vector line after segmentation, first projecting from the world coordinate system to the image coordinate system, and recording the corresponding projection parameters, and then using the trained deep learning detection network to identify the curved target frame in the image, wherein the schematic diagram of the curved target frame identified by the deep learning detection network is as follows Figure 2 Then, according to the recorded projection parameters, the curve target frame is calculated back to the world coordinate system.

[0050] S3, for road vector line data, the curvature is calculated based on a multi-scale plus sliding window method, and the curve section is identified based on the calculated curvature.

[0051] As an embodiment, for road vector line data, the curvature is calculated based on a multi-scale plus sliding window method, and the curve section is identified based on the calculated curvature, including: for a point P2 on the road vector line, forward searching a preset distance d to obtain a point P1 on the vector line, and backward searching the same preset distance d to obtain a point P3 on the vector line; according to the positions of the three points, solving the radius of the circle; starting from the starting position of the road vector line, traversing to the end position of the road vector line, obtaining a circle radius set, and smoothing all circle radii; based on the calculated circle radius, identifying the curve section.

[0052] It can be understood that the embodiment of the present invention not only uses a deep learning detection network to identify curves, but also uses a geometric algorithm to calculate the shape characteristics of the road, specifically calculates the curvature of the road, and identifies whether the road is a curve based on the curvature.

[0053] The schematic diagram of calculating the curvature radius based on the vector line morphology points can be found in Figure 3 , the specific implementation of step S3 is:

[0054] S31: For a point P2 on the road vector line, search forward a certain distance d to get a point P1 on the vector line, and search backward the same distance d to get a point P3 on the vector line. Based on the positions of the three points P1, P2 and P3, solve the radius of the circle where the road section of P1, P2 and P3 is located:

[0055] (1) Set the search distance d to 1m and solve to the radius r1;

[0056] (2) Set the search distance d to 5 m and solve to the radius r2;

[0057] (3) Set the search distance d to 10m and solve to the radius r3.

[0058] S32: Starting from the starting position of the road vector line, traverse and execute S31 to the end position of the vector line to obtain a circle radius set.

[0059] Here, it should be noted that the value of d in the embodiment of the present invention is not limited to 1m, 5m and 10m. In actual implementation, the value of d can also take different distance scales. When d takes more distance scales, no further explanation is given here.

[0060] S33: Use the idea of ​​sliding window to smooth the circle radius set obtained after completing S32.

[0061] S34: Divide each radius r in the circle radius set into a straight radius and a curve radius according to the following standard, and associate them with the reference point of the circle radius, that is, P2 in S31.

[0062] As an embodiment, the identification of the curve section based on the calculated circle radius includes: if the circle radius |r|>=preset radius threshold, it is a straight section; if |r|<preset radius threshold, it is a curve section. For example, if |r|>=1000m, it is a straight area, and if |r|<1000m, it is a curve area.

[0063] S4, performing fusion processing on the detected curve target frame and the identified curve section to obtain the identified final curve section.

[0064] As an embodiment, the detected curve target frame and the identified curve interval are fused to obtain the final identified curve interval, including: for the intersection interval of the identified curve target frame and the identified curve interval, it is determined as a curve area; for other intervals, the curve interval or the straight interval is determined according to the calculated circle radius.

[0065] Wherein, step S4 specifically includes:

[0066] S41: The intersection of the curves identified by S2 and S3 is confirmed as a curve.

[0067] S42: The difference between S2 and S3, determined according to the following criteria:

[0068] If |r|>=500m, it is finally determined as a straight area; if |r|<500m, it is finally determined as a curve area.

[0069] Embodiment 3

[0070] A curve recognition system based on multi-scale computing, see Figure 4 The curve recognition system includes a first recognition module 401, a second recognition module 402 and a fusion processing module 403, wherein:

[0071] A first recognition module 401 is used to project the road vector line data into an image based on multiple scales, and recognize a curve target frame from the image based on a deep learning method;

[0072] The second recognition module 402 is used to calculate the curvature of the road vector line data based on a multi-scale plus sliding window method, and identify the curve section based on the calculated curvature;

[0073] The fusion processing module 403 is used to perform fusion processing on the detected curve target frame and the identified curve section to obtain the identified final curve section.

[0074] It can be understood that the curve recognition system based on multi-scale operation provided by the present invention corresponds to the curve recognition method based on multi-scale operation provided by the aforementioned embodiments. The relevant technical features of the curve recognition system based on multi-scale operation can refer to the relevant technical features of the curve recognition method based on multi-scale operation, which will not be repeated here.

[0075] Embodiment 4

[0076] See also Figure 5 , Figure 5 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 5 As shown, an embodiment of the present invention provides an electronic device 500, including a memory 510, a processor 520, and a computer program 511 stored in the memory 510 and executable on the processor 520. When the processor 520 executes the computer program 511, the following steps are implemented: projecting road vector line data into an image based on multiple scales; identifying a curve target frame from the image based on a deep learning method; for the road vector line data, calculating the curvature based on a multi-scale plus sliding window method, and identifying the curve interval based on the calculated curvature; fusing the detected curve target frame and the identified curve interval to obtain the identified final curve interval.

[0077] Embodiment 5

[0078] See also Figure 6 , Figure 6 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 6 As shown, this embodiment provides a computer-readable storage medium 600, on which a computer program 611 is stored. When the computer program 611 is executed by a processor, the following steps are implemented: projecting road vector line data into an image based on multiple scales; identifying a curve target frame from the image based on a deep learning method; for the road vector line data, calculating the curvature based on a multi-scale plus sliding window method, and identifying the curve interval based on the calculated curvature; fusing the detected curve target frame and the identified curve interval to obtain the identified final curve interval.

[0079] The embodiment of the present invention provides a method and system for identifying curves based on multi-scale operations. The present invention combines a method based on deep learning and a method based on curvature operations, respectively using multi-scale operations. Compared with the existing curve identification method, it does not rely on road paving facilities, but is based on a relatively complete national two-dimensional map, or a conventional digital line map DLG, public package driving trajectory data, etc., to achieve nationwide curve identification. It also combines traditional geometric operation methods and deep learning methods, cross-extracts and cross-validates different methods, takes advantage of their strengths and avoids their weaknesses, and improves the robustness and generalization of the method.

[0080] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0081] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0082] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0083] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0084] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0085] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0086] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A curve recognition method based on multi-scale calculation, characterized in that: include: Projecting road vector line data into images based on multiple scales; Identifying a curve target frame from the image based on a deep learning method; For road vector line data, the curvature is calculated based on a multi-scale plus sliding window method, and the curve interval is identified based on the calculated curvature; The detected curve target frame and the identified curve section are fused to obtain the final identified curve section.

2. The curve recognition method according to claim 1, characterized in that: The projecting of the road vector line data into an image based on multiple scales includes: The road vector line data is evenly divided into corresponding sequence vector lines according to length distance thresholds of multiple different scales to obtain multiple sequence vector lines; Ortho-project each sequence vector line into an image, record the projection parameter Para during projection, and obtain an image set.

3. The curve recognition method according to claim 2, characterized in that: The method of identifying a curve target frame from the image based on a deep learning method includes: For each image in the image set, a curved road target frame is identified using a preset deep learning detection network; According to the projection parameter Para corresponding to each sequence vector line, each identified curve target frame is back-calculated back to the coordinate system before projection to obtain a curve target frame set.

4. The curve recognition method according to claim 1, characterized in that: For road vector line data, the curvature is calculated based on a multi-scale sliding window method, and the curve interval is identified based on the calculated curvature, including: For a point P2 on the road vector line, search forward for a preset distance d to get a point P1 on the vector line, and search backward for the same preset distance d to get a point P3 on the vector line; According to the positions of the three points, solve the radius of the circle; Starting from the starting position of the road vector line, traverse to the end position of the road vector line, obtain the circle radius set, and smooth all circle radii; Based on the calculated circle radius, the curve section is identified.

5. The curve recognition method according to claim 4, characterized in that: The preset distance d is 1m, 5m or 10m.

6. The curve recognition method according to claim 4, characterized in that: The step of identifying the curve section based on the calculated circle radius includes: If the circle radius |r|>=preset radius threshold, it is a straight section; If |r|<the preset radius threshold, it is a curve section.

7. The curve recognition method according to any one of claims 1 to 6, characterized in that: The detecting curve target frame and the identified curve interval are fused to obtain the identified final curve interval, including: The intersection section of the identified curve target frame and the identified curve section is determined as a curve area; For other sections, the curve section or the straight section is determined according to the calculated circle radius.

8. A curve recognition system based on multi-scale computing, characterized in that: include: A first recognition module is used to project the road vector line data into an image based on multiple scales, and recognize a curve target frame from the image based on a deep learning method; The second recognition module is used to calculate the curvature of the road vector line data based on a multi-scale plus sliding window method, and identify the curve section based on the calculated curvature; The fusion processing module is used to perform fusion processing on the detected curve target frame and the identified curve interval to obtain the final identified curve interval.

9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the processor is used to implement the steps of the curve recognition method based on multi-scale calculation as described in any one of claims 1 to 7 when executing a computer management program stored in the memory.

10. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by the processor, the steps of the curve recognition method based on multi-scale calculation as described in any one of claims 1 to 7 are implemented.

Citation Information

Patent Citations

  • Lane line detection method and device

    CN106991407A

  • Curve type identification method and device, electronic equipment and storage medium

    CN112435336A

  • A lane pavement number color recognition method and a speed limit information recognition method and system

    CN112507801A

  • Traffic light positioning extraction method and system based on point cloud high-precision map

    CN112733640A

  • Road shape recognizing device

    JP2004280138A