Lane position-based speed limit sign recognition and updating method and system and medium

Abstract

The invention relates to a speed limit sign recognition updating method and system based on a lane position and a medium, and the method comprises the steps: U1, carrying out the image collection based on a camera, entering a step U2 and a step U3 if the image collection succeeds, and otherwise, staying at the step U1 for continuous collection; u2, recognizing a front lane line by using the image information, and constructing a lane polygon under a self-vehicle coordinate system; u3, recognizing a front speed limit sign by using the image information, and projecting the speed limit sign to a self-vehicle coordinate system; and U4, based on the polygon information of the lane and the position information of the speed limit signs, carrying out inclusion judgment, and carrying out lane binding on the speed limit signs one by one. According to the method, detection is carried out for a multi-lane scene according to the position of the lane and the position information of the speed limit sign, so that the speed limit information related to the lane is output, the dependence on a navigation map is reduced, and the accuracy and the real-time performance of speed limit sign recognition are improved.

Description

Technical Field

[0001] This invention relates to the field of lane sign recognition technology, and in particular to a method, system, and medium for speed limit sign recognition and updating based on lane location. Background Technology

[0002] Currently, intelligent driving assistance systems typically detect traffic speed limit signs using cameras or rely on speed limit information output from navigation maps, and then send this information to the vehicle controller to provide speed limit reminders. However, in multi-lane road scenarios, because the speed limits for different lanes may differ, existing technologies are prone to misinterpreting traffic signs due to inaccurate lane information matching, affecting the reliability and real-time performance of the system.

[0003] In the prior art, Chinese patent application (application number: 202111270115.9) discloses a method, apparatus, and device for identifying speed limit signs and lane positions using high-precision maps. The method includes: performing target detection on an image of a traffic road to obtain speed limit signs; identifying an image containing the speed limit signs as an image to be identified; performing scene detection on the image to be identified to determine the scene in which the speed limit signs are located; and performing position detection on the image to be identified to obtain the lane position corresponding to the speed limit signs in the scene. However, this solution requires the use of high-precision maps. Firstly, the cost is relatively high; secondly, it depends on the freshness of the map; and finally, even high-precision positioning sometimes results in inaccurate lane positioning, which can lead to deviations when using high-precision maps to determine lane speed limits.

[0004] In the prior art, Chinese patent application (application number: 202180098544.4) discloses a speed limit recognition method and a speed limit recognition device, which enables the controller to perform the following processes: acquiring a captured image of the environment in front of the vehicle from a camera; recognizing speed limit signs from the captured image; determining whether multiple speed limit signs are recognized from the captured image; if multiple speed limit signs are recognized, determining whether the multiple speed limit signs include speed limit signs representing the same speed limit located on the right front and left front of the vehicle; if the multiple speed limit signs include speed limit signs representing the same speed limit located on the right front and left front of the vehicle, recognizing the speed limit represented by the speed limit signs located on the right front and left front of the vehicle as the speed limit of the road on which the vehicle is traveling. However, this solution considers the situation where multiple speed limits exist simultaneously, but it is only applicable when the speed limits on the left and right sides are the same. For multi-lane roads where different lanes have different speed limits, accurate recognition is not possible. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method, system and medium for speed limit sign recognition and updating based on lane position. It is applicable to multi-lane scenarios, and performs inclusion detection based on the lane position and speed limit sign position information to output speed limit information related to the lane, thereby reducing the dependence on navigation maps and improving the accuracy and real-time performance of speed limit sign recognition.

[0006] To achieve the above and other related objectives, the present invention provides the following technical solution: A method for recognizing and updating speed limit signs based on lane location, the method comprising: U1. Image acquisition is performed based on the camera. If the image acquisition is successful, proceed to steps U2 and U3; otherwise, stay in step U1 and continue to acquire images. U2. Using image information, identify the lane lines ahead and construct a lane polygon in the vehicle coordinate system; U3. Using image information, identify the speed limit sign ahead and project the speed limit sign onto the vehicle's coordinate system; U4. Based on the lane polygon information and speed limit sign location information, perform inclusion judgment and bind each speed limit sign to the lane one by one; U5. If there is a valid speed limit for this lane, output it; otherwise, maintain the output from the previous moment.

[0007] Furthermore, in step U2, constructing the lane polygon in the vehicle coordinate system includes: U21. Perform lane line detection on the acquired image and output the result; U22. Process the output lane line information to form the envelope polygon of this lane. After the lane envelope is formed, proceed to step U3.

[0008] Furthermore, in step U3, identifying the speed limit sign ahead and projecting the speed limit sign onto the vehicle coordinate system includes: U31. Detect and output speed limit signs from the acquired images; U32. Classify the speed limit signs and use an improved hierarchical analysis algorithm to filter out the valid speed limit signs. Then project their image coordinates onto the vehicle coordinate system. After the speed limit sign coordinate transformation is completed, proceed to step U4.

[0009] Furthermore, in step U32, the step of using the improved hierarchical analysis algorithm to filter out valid speed limit signs includes: U321. Classify the speed limit signs to obtain the classified speed limit signs, and then normalize them to obtain the normalized classified speed limit sign data information; U322. Based on the normalized and classified speed limit sign data information, construct the speed limit sign judgment matrix W. , Among them, a ij The data information of the j-th speed limit sign of the i-th class after normalization, i=1,2,3,...,n, j=1,2,3,...,n, where n is a positive integer, and feature value extraction is performed to obtain the feature value data information of the speed limit sign; U323. Based on the data information of the feature value of the speed limit sign, a preset threshold is set. If the feature value of the speed limit sign under the same category is less than the preset threshold, it is a valid speed limit sign.

[0010] Furthermore, if the feature value of a speed limit sign in the same category is greater than a preset threshold, it is considered an invalid speed limit sign and will be removed.

[0011] Furthermore, in step U4, the inclusion determination and lane binding of each speed limit sign includes: U41. Based on the lane envelope polygon and the center coordinates of the speed limit sign, perform a point-polygon inclusion determination; U42. If the center point of the speed limit sign is located within this lane, then bind the speed limit sign to the lane; after binding, proceed to U5.

[0012] Furthermore, in step U5, if the current lane is a single lane, then during the driving process, if a speed limit sign is visually detected at this time, the speed limit information of the current lane will be directly output. The forward millimeter-wave radar can identify and output the target vehicle information in front of the vehicle and output it to the computing unit through the CAN bus.

[0013] To achieve the above and other related objectives, the present invention also provides a lane position-based speed limit sign recognition and updating system for implementing the lane position-based speed limit sign recognition and updating method, the system comprising: The image acquisition module is used for image acquisition based on a camera. The speed limit sign detection module is connected to the image acquisition module and is used to identify the speed limit sign ahead using image information and project the speed limit sign onto the vehicle coordinate system. The lane line detection module is connected to the image acquisition module and is used to identify and detect lane lines in the lane using image information, and output lane line detection data. The lane position calculation module is connected to the lane line detection module and is used to calculate the position of the lane based on the lane line detection data. The speed limit sign-lane space mapping module is connected to the lane position calculation module and the speed limit sign detection module. It is used to perform inclusion judgment based on the lane polygon information and speed limit sign position information, and bind each speed limit sign to the lane one by one. This lane speed limit sign filtering module, together with the speed limit sign-lane space mapping module, is used to filter out valid speed limit signs using an improved hierarchical analysis algorithm.

[0014] Furthermore, the system also includes an ADAS generation module connected to the lane speed limit sign filtering module, which outputs the information if valid speed limit information exists in the lane, otherwise maintains the output content from the previous moment.

[0015] To achieve the above and other related objectives, the present invention also provides a computer-readable storage medium storing a computer program programmed or configured to perform any of the lane location-based speed limit sign recognition and update methods described herein. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the method flow of the present invention; Figure 2 This is a flowchart illustrating the improved hierarchical analysis algorithm of the present invention; Figure 3 This is a schematic diagram of the system framework of the present invention. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] Example 1: As Figure 1 As shown, a speed limit sign recognition and update method based on lane location is provided, the method comprising: U1. Image acquisition is performed based on the camera. If the image acquisition is successful, proceed to steps U2 and U3; otherwise, stay in step U1 and continue to acquire images. U2. Using image information, identify the lane lines ahead and construct a lane polygon in the vehicle coordinate system; U3. Using image information, identify the speed limit sign ahead and project the speed limit sign onto the vehicle's coordinate system; U4. Based on the lane polygon information and speed limit sign location information, perform inclusion judgment and bind each speed limit sign to the lane one by one; U5. If there is a valid speed limit for this lane, output it; otherwise, maintain the output from the previous moment.

[0019] In this embodiment, in step U2, constructing the lane polygon in the vehicle coordinate system includes: U21. Perform lane line detection on the acquired image and output the result; U22. Process the output lane line information to form the envelope polygon of this lane. After the lane envelope is formed, proceed to step U3.

[0020] In this embodiment, step U3, identifying the speed limit sign ahead and projecting the speed limit sign onto the vehicle coordinate system, includes: U31. Detect and output speed limit signs from the acquired images; U32. Classify the speed limit signs and use an improved hierarchical analysis algorithm to filter out the valid speed limit signs. Then project their image coordinates onto the vehicle coordinate system. After the speed limit sign coordinate transformation is completed, proceed to step U4.

[0021] In this embodiment, as Figure 2 As shown, in step U32, the step of using the improved hierarchical analysis algorithm to filter out valid speed limit signs includes: U321. Classify the speed limit signs to obtain the classified speed limit signs, and then normalize them to obtain the normalized classified speed limit sign data information; U322. Based on the normalized and classified speed limit sign data information, construct the speed limit sign judgment matrix W. , Among them, a ij The data information of the j-th speed limit sign of the i-th class after normalization, i=1,2,3,...,n, j=1,2,3,...,n, where n is a positive integer, and feature value extraction is performed to obtain the feature value data information of the speed limit sign; U323. Based on the data information of the feature value of the speed limit sign, a preset threshold is set. If the feature value of the speed limit sign under the same category is less than the preset threshold, it is a valid speed limit sign.

[0022] In this embodiment, if the feature value of a speed limit sign under the same category is greater than a preset threshold, it is an invalid speed limit sign and is removed.

[0023] In this embodiment, in step U4, the inclusion determination and lane binding of each speed limit sign includes: U41. Based on the lane envelope polygon and the center coordinates of the speed limit sign, perform point-polygon inclusion determination; U42. If the center point of the speed limit sign is located within this lane, then bind the speed limit sign to the lane; after binding, proceed to U5.

[0024] In this embodiment, in step U5, the lane is a single lane. During the driving process, if a speed limit sign is detected visually, the speed limit information of the lane is directly output. The forward millimeter-wave radar can identify and output the target vehicle information in front of the vehicle and output it to the computing unit through the CAN bus.

[0025] To verify the superiority of this application, three sets of comparative experiments were designed to compare it with the following three typical methods: Method A: Traditional TSR systems only identify the largest and clearest speed limit signs in the image, without considering lane affiliation.

[0026] Method B: A distance threshold-based screening method that considers the speed limit sign closest to the vehicle as valid, ignoring spatial distribution.

[0027] Method C: Lane binding method relying on high-precision map matching, obtaining the current lane speed limit through GPS + map API.

[0028] The test route covered typical scenarios such as urban main roads, highways, and ramp merging areas, with a total distance of approximately 200 kilometers. A total of 1,248 valid samples were collected, of which 43% contained multi-lane interference scenarios.

[0029] Table 1. Comparison of correct binding rates of different methods in multi-lane scenarios.

[0030] Data shows that the present invention has significant advantages in complex lane environments, especially in avoiding misidentification of adjacent lanes. Method A, which completely ignores spatial relationships, has an error binding rate as high as 41.1%; although Method B introduces distance factors, it cannot distinguish between left and right lanes and still has a significant risk of misjudgment; Method C, limited by map coverage and positioning delay, has 36 instances of "unable to judge" in tunnels or areas with signal obstruction.

[0031] In this embodiment, for example, there are three speed limit signs in the image, located in the left lane (100km / h), middle lane (80km / h), and right lane (60km / h). The test vehicle is located in the middle lane. This invention successfully identified and correctly bound the middle lane sign, while method A mistakenly selected 60km / h because the right lane sign was clearer, and method B mistakenly selected 100km / h because the left lane sign was closer. Only this invention and method C yielded the correct results.

[0032] Regarding output stability, the number of speed limit changes during a continuous 10-minute driving period was counted: This invention: average 1.2 times / min Method A: Average 3.8 times / min Method B: Average 2.9 times / min Method C: Average 2.1 times / min (affected by map switching) As can be seen, the present invention significantly reduces output fluctuations through the state-maintaining mechanism, which is beneficial to the smooth operation of longitudinal control systems such as ACC and LCC.

[0033] Example 2: Based on the speed limit sign recognition and update method based on lane position in Example 1, the present invention will be further explained and described below.

[0034] like Figure 1 As shown, a speed limit sign recognition and update method based on lane location is provided, the method comprising: U1. Image acquisition is performed based on the camera. If the image acquisition is successful, proceed to steps U2 and U3; otherwise, stay in step U1 and continue to acquire images. U2. Using image information, identify the lane lines ahead and construct a lane polygon in the vehicle coordinate system; U3. Using image information, identify the speed limit sign ahead and project the speed limit sign onto the vehicle's coordinate system; U4. Based on the lane polygon information and speed limit sign location information, perform inclusion judgment and bind each speed limit sign to the lane one by one; U5. If there is a valid speed limit for this lane, output it; otherwise, maintain the output from the previous moment.

[0035] In this embodiment, as Figure 3 As shown, this invention provides a lane position-based speed limit sign recognition and update system for implementing the lane position-based speed limit sign recognition and update method. The system includes: The image acquisition module is used for image acquisition based on a camera. The speed limit sign detection module is connected to the image acquisition module and is used to identify the speed limit sign ahead using image information and project the speed limit sign onto the vehicle coordinate system. The lane line detection module is connected to the image acquisition module and is used to identify and detect lane lines in the lane using image information, and output lane line detection data. The lane position calculation module is connected to the lane line detection module and is used to calculate the position of the lane based on the lane line detection data. The speed limit sign-lane space mapping module is connected to the lane position calculation module and the speed limit sign detection module. It is used to perform inclusion judgment based on the lane polygon information and speed limit sign position information, and bind each speed limit sign to the lane one by one. This lane speed limit sign filtering module, together with the speed limit sign-lane space mapping module, is used to filter out valid speed limit signs using an improved hierarchical analysis algorithm.

[0036] In this embodiment, the system further includes an ADAS generation module connected to the lane speed limit sign filtering module, which outputs the information if valid speed limit information exists in the lane, and otherwise maintains the output content from the previous moment.

[0037] In this embodiment, during a long-distance test, the vehicle departed from Beijing and traveled south along the G4 Beijing-Hong Kong-Macau Expressway. The journey involved multiple lane changes and speed limit adjustments. When the vehicle changed lanes from the fourth lane to the second lane, the system automatically recognized the "120" sign above the new lane and promptly updated the speed limit suggestion to assist the driver in accelerating legally. Upon entering the Hebei section, due to road maintenance, "lane-specific speed limits" were implemented; the speed limit in the third lane was reduced to 80 km / h, while the speed limit in the other lanes remained at 100 km / h. The system accurately identified this difference, avoiding false alerts.

[0038] Furthermore, this method can also be extended to other traffic sign application scenarios, such as the recognition of lane-specific signs like "No Lane Change" and "Bus Only," further enriching the environmental understanding capabilities of autonomous driving systems.

[0039] In this embodiment, the present invention provides a computer-readable storage medium storing a computer program programmed or configured to perform any of the lane location-based speed limit sign recognition and update methods described herein.

[0040] Any references to memory, storage, database, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0041] In summary, this invention not only addresses multi-lane scenarios by performing inclusion detection based on the lane position and speed limit sign position information to output speed limit information related to the lane, reducing reliance on navigation maps, but also improves the accuracy and real-time performance of speed limit sign recognition.

[0042] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A lane position-based speed limit sign recognition update method, characterized by, The method comprises: U1. Image acquisition based on the camera, if the image acquisition is successful, then enter step U2 and step U3, otherwise stay in step U1 and continue to collect; U2. Using image information, identifying the front lane line, and constructing the lane polygon under the self-vehicle coordinate system; U3. Using image information, identifying the front speed limit sign, and projecting the speed limit sign to the self-vehicle coordinate system; U4. Based on the lane polygon information and the speed limit sign position information, performing inclusion judgment, and binding the speed limit sign lane by lane; U5. If there is valid speed limit information in the lane, output, otherwise, maintain the output content at the last time.

2. The lane position-based speed limit sign recognition updating method of claim 1, wherein, In step U2, the construction of the lane polygon under the self-vehicle coordinate system comprises: U21. Lane line detection output is performed on the collected image; U22. The output lane line information is processed to form the envelope polygon of the lane, and after the lane envelope is formed, step U3 is entered. 3.The lane position-based speed limit sign recognition updating method of claim 1, wherein, In step U3, the identification of the front speed limit sign and the projection of the speed limit sign to the self-vehicle coordinate system comprise: U31. Speed limit sign detection output is performed on the collected image; U32. The speed limit sign is classified, and an improved analytic hierarchy process algorithm is used to screen out valid speed limit signs, and then the image coordinates of the speed limit signs are projected to the self-vehicle coordinate system, and after the speed limit sign coordinate conversion is completed, step U4 is entered.

4. The lane position-based speed limit sign recognition updating method of claim 3, wherein, In step U32, the use of the improved analytic hierarchy process algorithm to screen out valid speed limit signs comprises: U321. The speed limit signs are classified to obtain classified speed limit signs, and normalized processing is performed to obtain data information of the normalized classified speed limit signs; U322. Based on the data information of the normalized classified speed limit signs, a judgment matrix W of the speed limit signs is constructed, ， wherein a ij is the data information of the i-th speed limit sign of the j-th type after normalization processing, i = 1, 2, 3, …, n, j = 1, 2, 3, …, n, n is a positive integer, and the characteristic value extraction is performed to obtain the data information of the characteristic value of the speed limit sign. U323. Based on the data information of the characteristic values of the speed limit signs, a preset threshold is set, and if the characteristic value of the speed limit sign under the same classification is less than the preset threshold, the speed limit sign is a valid speed limit sign. 5.The lane position based speed limit sign recognition updating method of claim 4, wherein: If the characteristic value of the speed limit sign under the same classification is greater than the preset threshold, the speed limit sign is an invalid speed limit sign and is removed. 6.The lane position-based speed limit sign recognition updating method of claim 1, wherein, In step U4, the inclusion judgment and lane binding of the speed limit signs one by one comprise: U41. Point-polygon inclusion judgment is performed based on the lane envelope polygon and the center position coordinates of the speed limit sign; U42. If the center point of the speed limit sign is located in the lane, the speed limit sign is bound to the lane; after the binding is completed, U5 is entered. 7.The lane position-based speed limit sign recognition updating method of claim 1, wherein, In step U5, the lane is a single lane, and during driving, the front lane is a single lane, if a speed limit sign is detected at this time, the speed limit information of the lane is directly output, a forward millimeter wave radar can identify and output vehicle front target vehicle information, and output the vehicle front target vehicle information to the computing unit through the CAN bus.

8. A lane position-based speed limit sign recognition updating system characterized by, The system is used for implementing the lane position-based speed limit sign recognition updating method of any one of claims 1-7, and the system comprises: An image acquisition module, configured to perform image acquisition based on a camera; A speed limit sign detection module, connected with the image acquisition module, configured to use image information to identify a front speed limit sign and project the speed limit sign to a self-vehicle coordinate system; The lane line detection module is connected with the image acquisition module, and is configured to utilize image information to identify and detect lane lines in a lane, and output detection data of the lane lines. The lane position calculation module is connected with the lane line detection module, and is configured to calculate a position of the lane according to the detection data of the lane lines. The speed limit sign-lane space mapping module is connected with the lane position calculation module and the speed limit sign detection module, and is configured to perform a containment judgment according to the lane polygon information and the speed limit sign position information, and bind the speed limit signs to the lane one by one. The current lane speed limit sign screening module is connected with the speed limit sign-lane space mapping module, and is configured to screen out effective speed limit signs by using an improved analytic hierarchy process algorithm.

9. The lane position based speed limit sign recognition updating system of claim 8, wherein, The system further includes an ADAS generation module connected with the current lane speed limit sign screening module, and configured to output, if there is effective speed limit information in the current lane, or maintain the output content at the previous time.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium has stored thereon a computer program programmed or configured to perform the lane position based speed limit sign recognition updating method of any one of claims 1-7.

Images