Vehicle automatic driving navigation method and system based on road sign recognition

Abstract

The invention provides a vehicle automatic driving navigation method and system based on road sign recognition, and relates to the field of automatic driving, and the method comprises the steps: obtaining image data and point cloud data of a road sign, carrying out the alignment processing of the image data and the point cloud data, obtaining a fusion sign feature, and extracting the initial position information; acquiring actual position information of the road sign, and comparing the actual position information with the initial position information to obtain an offset vector feature; performing angle correction processing on the image data according to the visual features and the offset vector features to obtain identification attitude information, and determining a change trend of the identification attitude information; and acquiring a vehicle driving track, adjusting the vehicle driving track according to the change trend and the offset vector features to obtain a predicted driving track, and performing vehicle navigation based on the predicted driving track. According to the invention, real-time precision and dynamic updating of automatic driving of the vehicle are realized.

Description

Technical Field

[0001] The present invention relates to the field of autonomous driving, and in particular to a vehicle autonomous driving navigation method and system based on road sign recognition. Background Art

[0002] As a core pillar of intelligent transportation, autonomous driving technology plays a crucial role in improving road safety and traffic efficiency. Accurate road sign recognition is an integral component of autonomous driving systems, directly impacting the navigation and decision-making capabilities of vehicles using them.

[0003] However, existing road sign recognition methods exhibit significant shortcomings when dealing with complex, dynamic environments. Relying on a static database of preset signs, they struggle to adapt to changes in sign position due to construction, inclement weather, or vandalism. Furthermore, some road sign recognition methods lack sufficient robustness when dealing with physical changes in the signs, such as positional deviation or tilt, which can easily lead to recognition errors and compromise the reliability of autonomous driving systems. The dynamic changes in road sign position have become a core challenge for road sign recognition in the field of autonomous driving.

[0004] Therefore, how to identify more accurate and reliable driving trajectories in complex dynamic environments to improve navigation accuracy has become an important issue that needs to be urgently addressed in the field of autonomous driving. Summary of the Invention

[0005] In view of this, the present invention proposes a vehicle automatic driving navigation method and system based on road sign recognition, so as to identify a more accurate and reliable driving trajectory in a complex dynamic environment and thus improve navigation accuracy.

[0006] The technical solution of the present invention is achieved as follows: According to a first aspect, an embodiment of the present invention provides a vehicle automatic driving navigation method based on road sign recognition, the method comprising: Acquire image data and point cloud data of the road sign at each collection point, align the image data and point cloud data to obtain a fused sign feature for each collection point, and extract preliminary position information of the road sign from the fused sign feature; the time intervals between adjacent collection points are equal; Obtain the actual location information of the road sign, compare the actual location information with the preliminary location information, and obtain the offset vector characteristics of each collection point; Performing angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the identification posture information of each acquisition point and determine the change trend of the identification posture information of different acquisition points within each preset time period; A vehicle driving trajectory is obtained, the vehicle driving trajectory is adjusted according to a change trend and an offset vector feature to obtain a predicted driving trajectory, and vehicle navigation is performed based on the predicted driving trajectory.

[0007] In conjunction with the first aspect, in a first embodiment of the first aspect, acquiring image data and point cloud data of a road sign at each collection point, aligning the image data and the point cloud data to obtain a fused sign feature for each collection point, and extracting preliminary position information of the road sign from the fused sign feature specifically includes: Obtain image data of road signs at each collection point and perform denoising on the image data; Obtain point cloud data of road signs at each collection point; Use timestamps to align image data and point cloud data in the time dimension; Map the time-aligned image data and point cloud data to a preset spatial coordinate system to obtain the fusion identification features of each acquisition point; The fused identification features are corrected and the preliminary position information of the target points in the corrected fused identification features is extracted.

[0008] In combination with the first embodiment of the first aspect, in the second embodiment of the first aspect, obtaining the actual position information of the road sign, comparing the actual position information with the preliminary position information, and obtaining the offset vector feature of each collection point specifically includes: Retrieving actual position information of each target point in the road sign from a preset database; the actual position information is the recorded position information of each target point recorded in the preset database, and the recorded position information specifically includes the horizontal coordinate, vertical coordinate, and depth coordinate of the target point recorded in the preset database; The coordinate information and direction information of the actual position information and the preliminary position information are compared to obtain the offset vector feature of each collection point of the road sign.

[0009] In combination with the first aspect, in a third embodiment of the first aspect, performing angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain identification posture information of each acquisition point, and determining a change trend of the identification posture information of different acquisition points within each preset time period, specifically includes: Acquire the first point feature and the second point feature of the target point and the reference point of the reference object in the image data of each acquisition point respectively, match the first point feature with the second point feature, and obtain the visual feature of each acquisition point of the target point; Determine the first tilt angle of each collection point of the road sign according to the visual features of all target points in each collection point; Determine a second tilt angle of each collection point of the road sign according to the offset vector feature of each collection point; generating a transformation matrix for each acquisition point according to a first tilt angle and a second tilt angle of each acquisition point; Perform angle correction processing on the image data of the corresponding acquisition point according to the transformation matrix of each acquisition point to obtain the identification posture information of each acquisition point; The speed information of the vehicle is obtained, and the changing trend of the identification posture information of different collection points within each preset time period is determined based on the speed information.

[0010] In combination with the third embodiment of the first aspect, in the fourth embodiment of the first aspect, obtaining the vehicle speed information and determining the change trend of the identification posture information of different collection points within each preset time period based on the speed information specifically includes: Obtaining vehicle speed information and determining a preset time period based on the speed information; Determine a first position offset of the identification posture information of adjacent collection points within each preset time period, and a second position offset of the identification posture information of the first collection point and the last collection point within the preset time period; A change trend is obtained according to the first and second position offsets.

[0011] In combination with the third embodiment of the first aspect, in the fifth embodiment of the first aspect, before the step of obtaining the vehicle speed information and determining the change trend of the identification posture information of different collection points within a preset time period based on the speed information, the method further includes: The sign posture information is verified to determine whether the sign posture information is in a standard posture; the standard posture is that the road sign is visually in a vertical state.

[0012] In combination with the first aspect, in a sixth implementation of the first aspect, obtaining the vehicle driving trajectory and adjusting the vehicle driving trajectory according to the change trend and the offset vector feature to obtain the predicted driving trajectory specifically includes: Extract vehicle driving trajectory, change trend and offset vector features within each preset time period; Determine whether the change trend exceeds a preset change range based on the vehicle's driving speed; If it is determined that the preset change range is exceeded, the vehicle driving trajectory corresponding to the preset time period is marked as a key change trajectory; The key change trajectory in the vehicle's driving trajectory is adjusted according to the change trend and offset vector characteristics to obtain the predicted driving trajectory.

[0013] In combination with the sixth embodiment of the first aspect, in the seventh embodiment of the first aspect, adjusting the key change trajectory in the vehicle driving trajectory according to the change trend and the offset vector feature to obtain the predicted driving trajectory specifically includes: Obtain the historical driving trajectory of historical vehicles passing the same road sign; According to the historical driving trajectory, change trend and offset vector characteristics, the key change trajectory is adjusted to obtain the predicted driving trajectory; Determine whether the predicted driving trajectory meets safe driving conditions, and adjust the predicted driving trajectory if it does not meet safe driving conditions.

[0014] According to a second aspect, an embodiment of the present invention provides a vehicle automatic driving navigation system based on road sign recognition, the system comprising: A preliminary processing module is used to obtain image data and point cloud data of the road sign at each collection point, align the image data and point cloud data to obtain a fused sign feature for each collection point, and extract preliminary position information of the road sign from the fused sign feature; the time intervals between adjacent collection points are equal; The offset determination module is used to obtain the actual position information of the road sign, compare the actual position information with the preliminary position information, and obtain the offset vector characteristics of each collection point; A change trend module is used to perform angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point, obtain the identification posture information of each acquisition point, and determine the change trend of the identification posture information of different acquisition points within a preset time period; The trajectory prediction module is used to obtain the vehicle's driving trajectory, adjust the vehicle's driving trajectory according to the change trend and the offset vector characteristics to obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

[0015] The vehicle automatic driving navigation method and system based on road sign recognition of the present invention have the following beneficial effects compared with the prior art: By acquiring image data and point cloud data of road signs at each collection point, aligning the image data and point cloud data, obtaining fused sign features for each collection point, and extracting preliminary position information of the road signs from them, the fused sign features are then compared with the actual position information of the road signs to obtain offset vector features. The image data is then angle-corrected by combining visual features and offset vector features to obtain sign posture information, and the changing trend of sign posture information at different collection points within each preset time period is determined. That is, in a complex dynamic environment, by comparing the deviation between the road sign-related information obtained by the on-board sensor and the actual position information recorded in a preset database, the displaced or tilted road signs are accurately identified. Finally, combined with the motion trajectory prediction algorithm, an accurate and reliable predicted driving trajectory is output. This effectively solves the problem of inaccurate road sign positioning due to dynamic environmental changes, external interference, high-speed driving, and other factors, improves the accuracy and reliability of autonomous driving system navigation, and provides a safer and more reliable navigation reference for autonomous driving vehicles, which is of great significance to improving the overall performance of intelligent transportation systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 This is a flow chart of a method for automatic vehicle navigation based on road sign recognition according to the present invention; Figure 2 This is a second flow chart of the vehicle automatic driving navigation method based on road sign recognition of the present invention; Figure 3 This is a third flow chart of the vehicle automatic driving navigation method based on road sign recognition of the present invention; Figure 4 This is a fourth flow chart of the vehicle automatic driving navigation method based on road sign recognition of the present invention; Figure 5 Schematic diagram of the structure of the vehicle automatic driving navigation system based on road sign recognition of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0019] The vehicle automatic driving navigation method based on road sign recognition provided in this specification is intended to accurately identify displaced or tilted road signs by using the deviation between road sign-related information and actual position information recorded in a preset database in a complex dynamic environment, and combine it with a motion trajectory prediction algorithm to output an accurate and reliable predicted driving trajectory to improve navigation accuracy.

[0020] See also Figure 1 , Figure 1 The flowchart of the vehicle automatic driving navigation method based on road sign recognition of the present invention is as follows: S101. Obtain image data and point cloud data for each road sign at each collection point, align the image data and point cloud data to obtain a fused marker feature for each collection point, and extract preliminary location information for the road sign from the fused marker feature. The preliminary location information includes the horizontal coordinate, vertical coordinate, and depth coordinate of each target point on the road sign. The collection point is the time point corresponding to data collection. Vehicle-mounted sensors are used to acquire the aforementioned image data and point cloud data. The time interval between adjacent collection points is always equal and can be set to 1-3 seconds.

[0021] In this embodiment, target points are also core recognition points of road signs, such as the center point, outline point, and distinguishing point of a road sign, which enable rapid identification. Distinguishing points are points in the area where a road sign is distinguished from other road signs. It will be understood that each road sign has a corresponding number of target points, and different road signs have different target points.

[0022] In this embodiment, the alignment processing includes the alignment of two dimensions, time dimension and space dimension. The fused identification features obtained in this way can integrate the texture information contained in the image data and the depth information contained in the point cloud data, thereby improving the accuracy of the application of the fused identification features, significantly enhancing the stability and accuracy of road sign recognition, effectively coping with road sign recognition in dynamic environments, and providing important support for the construction of intelligent transportation systems.

[0023] S102: Acquire the actual position information of the road sign, compare the actual position information with the preliminary position information, and obtain the offset vector feature of each collection point. The offset vector feature includes the offset amount and offset direction. The offset vector feature can intuitively reflect the offset of the road sign position recorded by the autonomous driving system, providing an accurate reference for subsequent processing.

[0024] By comparing the actual and preliminary position information, the difference between the two is calculated, and the offset is then determined. This offset indicates whether the road sign has experienced a planar offset. The offset vector characteristics of road signs can vary depending on factors such as on-board sensor performance, vehicle trajectory, and driving environment (such as road construction and inclement weather).

[0025] In this embodiment, a determination is made based on the offsets of all target points as to whether the displacement of the road sign in the plane exceeds a preset movement amount. If the displacement exceeds the preset movement amount, it indicates that the preliminary position information of the road sign acquired by the autonomous driving system is deviated. Therefore, the preliminary position information and the offset with the deviated information are associated with the unique identification information (e.g., ID) of the road sign to obtain associated information. The associated information may be presented in a structured format, including information such as unique identification information, preliminary position information, and offset. The associated information may be stored in a preset database. This storage method facilitates subsequent queries and updates, and provides data support for the maintenance and management of road signs.

[0026] S103. Perform angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the identification posture information of each acquisition point, and determine the change trend of the identification posture information of different acquisition points within each preset time period.

[0027] In this embodiment, the image data of each acquisition point is angle-corrected based on the offset vector features of each acquisition point, combined with changes in tilt angle and visual features, to obtain the corrected road sign posture information for each acquisition point. The dynamic changes in the sign posture information during vehicle driving due to factors such as complex road conditions and high-speed driving are then analyzed. To ensure the real-time recognition of road signs by the autonomous driving system, the analysis time window of the above-mentioned sign posture information is limited to a preset time period. This allows the changing trend of the sign posture information to be determined, and the real-time three-dimensional coordinate data of the road signs to be obtained, thereby achieving dynamic position updating and recording of the sign posture information.

[0028] Specifically, step S103 includes: S1031. Obtain first and second point features of the target point and the reference point of the reference object in the image data of each acquisition point, respectively, and match the first and second point features to obtain visual features of each acquisition point of the target point. The reference object may have at least one reference point, which is used to determine subsequent tilt angle changes. For example, the reference object may be a pole on which a road sign is mounted. Accordingly, the reference point of the reference object is the connection point on the pole that connects to the road sign.

[0029] The first and second point features mentioned above include position information of the points in the image data.

[0030] In this embodiment, each target point is a visual feature point used in visual feature processing. By analyzing the relative position of each target point with respect to the reference object, the visual features contained in the image data can be obtained. This visual feature can reflect whether there is a significant change in the visual features of the corresponding target point.

[0031] S1032: Determine a first tilt angle of each collection point of the road sign according to the visual features of all target points in each collection point.

[0032] Since the visual features are obtained based on the first point features of the target point and the second point features of the reference point, after collecting the visual features of all target points, the inclination angle of the road sign at the visual level in the image data collected by the sensor can be determined.

[0033] S1033: Determine a second tilt angle of each collection point of the road sign according to the offset vector feature of each collection point.

[0034] The second angle information can be obtained according to the offset direction included in the offset vector feature. The second angle information can reflect whether there is a large tilt angle change at the corresponding target point.

[0035] S1034: Generate a transformation matrix for each acquisition point according to the first tilt angle and the second tilt angle of each acquisition point.

[0036] The transformation matrix is ​​obtained by combining the two factors of visual feature changes and tilt angle changes. In the subsequent angle correction processing of image data, it can improve the correction effect at the angle level.

[0037] S1035 , performing angle correction processing on the image data of the corresponding collection point according to the transformation matrix of each collection point to obtain identification posture information of each collection point.

[0038] In this embodiment, the image data of each acquisition point is remapped based on the obtained transformation matrix for each acquisition point to perform angle correction processing. The transformation matrix is ​​intended to adjust the pixel distribution of each pixel point in the image data through mathematical transformation to make it as close as possible to the actual standard posture. The standard posture is that the road sign is visually in a vertical state.

[0039] S1036: Verify the marker posture information to determine whether the marker posture information is in a standard posture.

[0040] Taking into account that there may be problems with some parameters in the generated transformation matrix, resulting in poor angle correction processing effect, in this embodiment, posture analysis is also performed on the sign posture information, and the distribution information of each pixel point in the sign posture information is analyzed to confirm whether the effect of the angle correction processing meets expectations, such as checking whether the road sign boundary line is straight and whether the text and symbols are clearly visible.

[0041] For example, when extracting the distribution information of each pixel in the sign's posture information and determining whether it is in the standard posture, the posture parameter calculation tool analyzes the pixel distribution information. If the vertical axis of the road sign is found to be substantially aligned with the standard axis after verification, with only a slight deviation of 0.05 degrees, the angle correction is determined to be successful, and the final angle posture information is generated. This is used to record the sign's status and facilitate long-term tracking and management.

[0042] S1037: Acquire the speed information of the vehicle, and determine the change trend of the identification posture information of different collection points within a preset time period based on the speed information.

[0043] In this embodiment, during the process of analyzing the change trend, the position offset of the identification posture information of different collection points within a preset time period is analyzed and processed to generate a dynamic position. It is understandable that the dynamic position can be recorded.

[0044] More specifically, step S1037 includes: S10371. Obtain vehicle speed information and determine a preset time period based on the speed information.

[0045] The faster the vehicle speed represented by the speed information is, the longer the preset time period is set. Conversely, the shorter the vehicle speed represented by the speed information is, the shorter the preset time period is set.

[0046] S10372: Determine a first position offset of the identification posture information of adjacent collection points within each preset time period, and a second position offset of the identification posture information of the first collection point and the last collection point within the preset time period.

[0047] In this embodiment, the rate of change of the marker posture information can be captured by obtaining the position offset of the marker posture information, and the position offset includes the plane displacement and the angle change. Recording the position offset of the marker posture information of adjacent collection points and the starting and ending collection points can better determine whether the marker posture information has undergone periodic offsets. For example, the marker posture information of a speed limit sign within a preset time period shows periodic offsets, which may be due to road construction or sensor calibration problems. In this way, the autonomous driving system can generate a corresponding trajectory report. This method can provide data support for road management, ensure the accuracy and reliability of sign recognition information, and through full-process optimization from data collection to storage and analysis, it can effectively improve the data processing capabilities under complex road conditions and provide a solid foundation for subsequent decision-making.

[0048] S10373. Obtain a change trend based on the first and second position offsets.

[0049] The changing trends obtained in this way help to capture the short-term trajectory characteristics of road signs and form the basis for trajectory tracking.

[0050] S104 , obtaining a vehicle driving trajectory, adjusting the vehicle driving trajectory according to a change trend and an offset vector feature to obtain a predicted driving trajectory, and performing vehicle navigation based on the predicted driving trajectory.

[0051] In this embodiment, in combination with a trajectory prediction algorithm, the relative motion trajectory of the vehicle and road signs is predicted for the erroneous information in the navigation information output by the autonomous driving system due to the dynamic environment. The vehicle's driving trajectory is then adjusted based on the relative motion trajectory to obtain a predicted driving trajectory. This makes the relative motion of the vehicle's driving trajectory and the road signs show a consistent motion trend, thereby improving the autonomous driving system's ability to adapt to complex dynamic environments.

[0052] The present invention relates to a method for autonomous vehicle navigation based on road sign recognition. The method acquires image data and point cloud data of road signs at each acquisition point, aligns the image data and point cloud data, obtains fused sign features at each acquisition point, and extracts preliminary position information of the road sign from the features. The features are then compared with the actual position information of the road sign to obtain offset vector features. The image data is then angle-corrected using the visual features and offset vector features to obtain sign posture information. The method also determines the changing trend of the sign posture information at different acquisition points within each preset time period. Specifically, in a complex dynamic environment, the method accurately identifies displaced or tilted road signs by comparing the deviation between road sign-related information acquired by on-board sensors and the actual position information recorded in a preset database. Finally, the method combines a motion trajectory prediction algorithm to output an accurate and reliable predicted driving trajectory. This method effectively addresses the issue of inaccurate road sign positioning due to dynamic environmental changes, external interference, high-speed driving, and other factors. It improves the accuracy and reliability of autonomous vehicle navigation, provides a safer and more reliable navigation reference for autonomous vehicles, and is of great significance for improving the overall performance of intelligent transportation systems.

[0053] See also Figure 2 , the method may further comprise the following steps: S2011: Obtain image data of the road sign at each collection point, and perform denoising processing on the image data.

[0054] In this embodiment, real-time image data of road signs is acquired using a vehicle-mounted camera. This captured image data retains environmental noise caused by lighting changes, shadows, rain, fog, and other factors during driving. To remove this environmental noise, pre-established filtering tools, such as median filtering, can be used to denoise the image data, effectively removing isolated noise points and improving image quality.

[0055] S2012: Obtain point cloud data of the road sign at each collection point.

[0056] In this embodiment, a vehicle-mounted laser radar is installed on the vehicle to scan the three-dimensional spatial information of road signs and their surroundings, thereby obtaining high-precision point cloud data containing depth information.

[0057] S2013. Use timestamps to align image data and point cloud data in the time dimension.

[0058] According to the timestamp information in the acquired image data and point cloud data, the two are aligned in the time dimension and the image data and point cloud data at the same acquisition point are ensured to participate in the subsequent spatial multi-dimensional processing.

[0059] In order to ensure that the data captured by different sensors can be subsequently fused, the denoised image data and point cloud data can be standardized before alignment in the time dimension, and the image data and point cloud data can be unified into structured data.

[0060] S2014: Map the time-aligned image data and point cloud data to a preset spatial coordinate system to obtain the fusion identification features of each acquisition point. This step maps the image data and point cloud data to the same spatial coordinate system through coordinate system conversion.

[0061] It can be seen that the fused identification feature is obtained by aligning the image data and point cloud data in time and space dimensions. This information fusion method can integrate the texture information contained in the image data and the depth information contained in the point cloud data, thereby improving the accuracy of the application of the fused identification feature.

[0062] S2015: Correct the fused identification features and extract preliminary position information of the target point in the corrected fused identification features.

[0063] In order to prevent the fused sign features from producing position deviations exceeding expectations, in this embodiment, the fused sign features are also corrected using a preset standard matrix to ensure that the position deviations of the fused sign features are controlled within a reasonable range. The corrected fused sign features can more realistically reflect the actual characteristics of the road signs, avoid cumulative errors in subsequent position calculations, and significantly improve system reliability.

[0064] In this embodiment, a 3D dead reckoning tool can be used to extract preliminary position information for target points of road signs included in the fused sign features. This tool can extract this preliminary position information based on triangulation principles. For example, on a straight road, the center point of a speed limit sign (one of the target points) is calculated to be 50 meters in front of the vehicle and 2.5 meters in height. This position information provides a critical reference for subsequent autonomous driving decisions, helping to improve navigation accuracy and safety.

[0065] S202: Acquire the actual position information of the road sign, compare the actual position information with the preliminary position information, and obtain the offset vector feature of the road sign. For details, refer to step S102.

[0066] S203: Perform angle correction on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the marker posture information of each acquisition point and determine the changing trend of the marker posture information of different acquisition points within each preset time period. For details, refer to step S103.

[0067] S204: Acquire the vehicle's driving trajectory, adjust the vehicle's driving trajectory according to the change trend and the offset vector characteristics to obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory. For details, refer to step S104.

[0068] See also Figure 3 , the method may further comprise the following steps: S301: Obtain image data and point cloud data of the road sign at each collection point, align the image data and point cloud data to obtain a fused sign feature for each collection point, and extract preliminary position information of the road sign from the fused sign feature. For details, refer to step S101.

[0069] S3021. Retrieve the actual location information of each target point in the road sign from a preset database.

[0070] In this embodiment, the actual location information is the recorded location information of each target point recorded in the preset database, wherein the recorded location information specifically includes the horizontal coordinate, vertical coordinate and depth coordinate of the target point recorded in the preset database. The preset database will record the above three coordinate values ​​of the target points corresponding to several road signs. These coordinate values ​​recorded in the preset database represent the actual location coordinates of the road signs.

[0071] S3022: Compare the coordinate information and direction information of the actual position information and the preliminary position information to obtain an offset vector feature of the road sign.

[0072] The calculation method of the offset caused by the plane displacement of the road sign is:

[0073] in, Indicates road sign The offset of the target point; Indicates the The horizontal coordinates of the target points recorded in the preset database; Indicates the The horizontal coordinate of each target point in the preliminary position information; Indicates the The vertical coordinate of each target point recorded in the preset database; Indicates the The vertical coordinate of each target point in the preliminary position information; Indicates the The depth coordinates of each target point recorded in the preset database; Indicates the The depth coordinates of the target points in the preliminary position information. 、 as well as Constitutes the The actual location information of the target point, 、 as well as Constitutes the The preliminary location information of the target point.

[0074] It should be noted that when obtaining the coordinate information of the actual position information and the preliminary position information, the specific values ​​of the coordinate information can be determined with the help of a reference object.

[0075] Then, by comparing the actual position information and the direction information of the preliminary position information, the offset direction is obtained, and the offset vector feature of the road sign is obtained by combining the offset amount and the offset direction.

[0076] S303: Perform angle correction on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the marker posture information of each acquisition point and determine the changing trend of the marker posture information of different acquisition points within each preset time period. For details, refer to step S103.

[0077] S304: Acquire the vehicle's driving trajectory, adjust the vehicle's driving trajectory according to the change trend and the offset vector characteristics to obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory. For details, refer to step S104.

[0078] See also Figure 4 , the method may further comprise the following steps: S401: Obtain image data and point cloud data of the road sign at each collection point, align the image data and point cloud data to obtain a fused sign feature for each collection point, and extract preliminary position information of the road sign from the fused sign feature. For details, refer to step S101.

[0079] S402: Acquire the actual position information of the road sign, compare the actual position information with the preliminary position information, and obtain the offset vector feature of each collection point. For details, refer to step S102.

[0080] S403: Perform angle correction on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the marker posture information of each acquisition point and determine the changing trend of the marker posture information of different acquisition points within each preset time period. For details, refer to step S103.

[0081] S4041. Extract vehicle driving trajectory, change trend, and offset vector features within each preset time period.

[0082] When generating a predicted vehicle trajectory, the vehicle trajectory, its changing trends, and its offset vector features can be collated and summarized by time period. In this embodiment, a preset time period is used as a summary window, and the data within each preset time period is extracted, and the data is then segmented based on the time series. This segmented summary clearly demonstrates the short-term changing trends of the marker location, facilitating subsequent analysis of key change nodes. The summarized data for each time period includes timestamp information to ensure data integrity.

[0083] The vehicle driving trajectory can be obtained through various types of vehicle-mounted sensors, and the offset vector feature corresponding to the preset time period is a set of offset vector features corresponding to all collection points within the preset time period.

[0084] S4042. Determine whether the change trend exceeds a preset change range based on the vehicle's driving speed.

[0085] The vehicle's movement trend can be obtained based on the vehicle's driving speed, and the change trend indicates the trajectory characteristics of the road sign. Both trends will be affected by the dynamic environment, such as road construction or sensor calibration. In this embodiment, an ideal preset change range is first determined based on the vehicle's driving speed, and then it is determined whether the change trend exceeds the preset change range. If it does not exceed the change range, it means that the relative movement trajectory of the vehicle and the road sign is in line with expectations.

[0086] S4043. When it is determined that the preset change range is exceeded, the vehicle driving trajectory corresponding to the preset time period is marked as a key change trajectory.

[0087] This marking method can quickly identify abnormal changes in the corresponding information of road signs, such as coordinate offsets caused by jitter of on-board sensors, thereby improving the accuracy of subsequent predictions.

[0088] S4044. Adjust the key change trajectory in the vehicle's driving trajectory according to the change trend and the offset vector characteristics to obtain a predicted driving trajectory.

[0089] During the trajectory adjustment process, only the portion of the vehicle's relative motion trajectory with road markings that exceeds expectations is adjusted. This improves subsequent recognition efficiency and reduces the amount of trajectory correction data. This setup also analyzes potential errors in the navigation information output by the autonomous driving system and generates more reliable navigation information.

[0090] More specifically, step S4044 includes: S40441. Obtain historical driving trajectories of historical vehicles passing the same road sign.

[0091] The reliability of path planning can be further improved by obtaining the historical driving trajectories of other vehicles driving through the same road sign and adjusting the trajectory.

[0092] It should be noted that the driving trajectory of the current vehicle passing the same road sign at other time points can also be classified as historical trajectory information.

[0093] S40442. Adjust the key change trajectory according to the historical driving trajectory, change trend, and offset vector characteristics to obtain a predicted driving trajectory.

[0094] The trajectory adjustment method is to make local adjustments to key changing trajectories. The path information is recalculated based on the dynamic change data of road signs, namely the change trend and offset vector characteristics. The purpose of the adjustment is to make the relative movement of the vehicle's driving trajectory and the road signs show a consistent movement trend, and obtain an adjusted predicted driving trajectory to ensure that the predicted driving path planned by the autonomous driving system allows the vehicle to avoid areas that may be misjudged. This adjustment method improves the adaptability of the path.

[0095] S40443. Determine whether the predicted driving trajectory meets the safe driving conditions, and adjust the predicted driving trajectory if it does not meet the safe driving conditions.

[0096] To ensure the safety and practical value of the predicted driving trajectory, this embodiment evaluates whether the predicted driving trajectory meets safe navigation conditions. If the deviation from the adjusted predicted driving trajectory still exceeds the safe range, the autonomous driving system initiates a secondary correction process. This secondary correction incorporates additional environmental data to optimize the final path, ensuring that it meets safety standards and providing assurance of path reliability.

[0097] The automated driving system can also back up the predicted trajectory after the secondary adjustment, creating a complete correction record. If an adjustment involves multiple road marking areas, the automated driving system will record the pre- and post-adjustment data for each area. This archiving method facilitates subsequent tracing and analysis, improving data management efficiency.

[0098] In this embodiment, the preliminary position information can also be calibrated in combination with navigation data such as GPS data, so as to effectively cope with dynamic environmental changes.

[0099] The following describes a system provided by an embodiment of the present invention. The system described below and the method described above can refer to each other.

[0100] See also Figure 5 , Figure 5A schematic diagram of the structure of a vehicle automatic driving navigation system based on road sign recognition according to an embodiment of the present invention is shown. The system may include: The preliminary determination module 10 is configured to acquire image data and point cloud data for each road sign at each acquisition point, align the image data and point cloud data to obtain a fused marker feature for each acquisition point, and extract preliminary position information for the road sign from the fused marker feature. The preliminary position information includes the horizontal coordinate, vertical coordinate, and depth coordinate of each target point on the road sign. The acquisition point is the time point corresponding to data acquisition. Vehicle-mounted sensors are used to acquire the aforementioned image data and point cloud data. The time interval between adjacent acquisition points is always equal and can be set to 1-3 seconds.

[0101] In this embodiment, target points are also core recognition points of road signs, such as the center point, outline point, and distinguishing point of a road sign, which enable rapid identification. Distinguishing points are points in the area where a road sign is distinguished from other road signs. It will be understood that each road sign has a corresponding number of target points, and different road signs have different target points.

[0102] In this embodiment, the alignment processing includes the alignment of two dimensions, time dimension and space dimension. The fused identification features obtained in this way can integrate the texture information contained in the image data and the depth information contained in the point cloud data, thereby improving the accuracy of the application of the fused identification features, significantly enhancing the stability and accuracy of road sign recognition, effectively coping with road sign recognition in dynamic environments, and providing important support for the construction of intelligent transportation systems.

[0103] The offset determination module 20 is used to obtain the actual position information of road signs, compare the actual position information with the preliminary position information, and obtain the offset vector feature of each collection point. The offset vector feature includes the offset amount and offset direction. The offset vector feature can intuitively reflect the offset of the road sign position recorded by the autonomous driving system, providing an accurate reference for subsequent processing.

[0104] By comparing the actual and preliminary position information, the difference between the two is calculated, and the offset is then determined. This offset indicates whether the road sign has experienced a planar offset. The offset vector characteristics of road signs can vary depending on factors such as on-board sensor performance, vehicle trajectory, and driving environment (such as road construction and inclement weather).

[0105] In this embodiment, it is determined whether the displacement of the road sign in the plane exceeds the preset movement amount based on the offset of all target points. If it exceeds the preset movement amount, it indicates that there is a deviation in the preliminary position information of the road sign obtained by the automatic driving system. Therefore, the preliminary position information and offset with the unique identification information of the road sign with the deviation will be associated with each other to obtain associated information. The associated information can be presented in a structured format, including unique identification information, preliminary position information, offset and other information. The associated information can be stored in a preset database. This storage method is convenient for subsequent queries and updates, and provides data support for the maintenance and management of road signs.

[0106] The change trend module 30 is used to perform angle correction processing on the image data of the corresponding acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point, obtain the identification posture information of each acquisition point, and determine the change trend of the identification posture information of different acquisition points within each preset time period.

[0107] In this embodiment, the image data of each acquisition point is angle-corrected based on the offset vector features of each acquisition point, combined with changes in tilt angle and visual features, to obtain the corrected road sign posture information for each acquisition point. The dynamic changes in the sign posture information during vehicle driving due to factors such as complex road conditions and high-speed driving are then analyzed. To ensure the real-time recognition of road signs by the autonomous driving system, the analysis time window of the above-mentioned sign posture information is limited to a preset time period. This allows the changing trend of the sign posture information to be determined, and the real-time three-dimensional coordinate data of the road signs to be obtained, thereby achieving dynamic position updating and recording of the sign posture information.

[0108] The trajectory prediction module 40 is used to obtain the vehicle driving trajectory, adjust the vehicle driving trajectory according to the change trend and the offset vector characteristics to obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

[0109] In this embodiment, in combination with a trajectory prediction algorithm, the relative motion trajectory of the vehicle and road signs is predicted for the erroneous information in the navigation information output by the autonomous driving system due to the dynamic environment. The vehicle's driving trajectory is then adjusted based on the relative motion trajectory to obtain a predicted driving trajectory. This makes the relative motion of the vehicle's driving trajectory and the road signs show a consistent motion trend, thereby improving the autonomous driving system's ability to adapt to complex dynamic environments.

[0110] The vehicle automatic driving navigation system based on road sign recognition of the present invention obtains image data and point cloud data of the road sign at each collection point, aligns the image data and point cloud data, obtains a fused sign feature for each collection point, and extracts preliminary position information of the road sign from it. The fused sign feature is then compared with the actual position information of the road sign to obtain an offset vector feature. The image data is then angle-corrected in combination with the visual features and the offset vector features to obtain sign posture information. The changing trend of the sign posture information at different collection points within each preset time period is determined. That is, in a complex dynamic environment, by comparing the deviation between the road sign related information obtained by the on-board sensor and the actual position information recorded in a preset database, the system accurately identifies road signs that have been displaced or tilted. Finally, the system outputs a predicted driving trajectory in combination with a motion trajectory prediction algorithm. This effectively solves the problem of inaccurate road sign positioning due to factors such as dynamic environmental changes, external interference, and high-speed driving, improves the accuracy and reliability of the automatic driving system navigation, and provides a safer and more reliable navigation reference for automatic driving vehicles. This is of great significance for improving the overall performance of the intelligent transportation system.

[0111] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A method for automatic vehicle navigation based on road sign recognition, characterized by: The method comprises: Acquire image data and point cloud data of the road sign at each collection point, align the image data and point cloud data to obtain a fused sign feature for each collection point, and extract preliminary position information of the road sign from the fused sign feature; the time intervals between adjacent collection points are equal; Obtain the actual location information of the road sign, compare the actual location information with the preliminary location information, and obtain the offset vector characteristics of each collection point; Performing angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the identification posture information of each acquisition point and determine the change trend of the identification posture information of different acquisition points within each preset time period; A vehicle driving trajectory is obtained, the vehicle driving trajectory is adjusted according to a change trend and an offset vector feature to obtain a predicted driving trajectory, and vehicle navigation is performed based on the predicted driving trajectory.

2. The method for automatic vehicle navigation based on road sign recognition according to claim 1, wherein: The method of obtaining image data and point cloud data of the road sign at each collection point, aligning the image data and the point cloud data to obtain a fused sign feature of each collection point, and extracting preliminary position information of the road sign from the fused sign feature specifically includes: Obtain image data of road signs at each collection point and perform denoising on the image data; Obtain point cloud data of road signs at each collection point; Use timestamps to align image data and point cloud data in the time dimension; Map the time-aligned image data and point cloud data to a preset spatial coordinate system to obtain the fusion identification features of each acquisition point; The fused identification features are corrected and the preliminary position information of the target points in the corrected fused identification features is extracted.

3. The method for automatic vehicle navigation based on road sign recognition according to claim 2, wherein: The step of obtaining the actual position information of the road sign, comparing the actual position information with the preliminary position information, and obtaining the offset vector feature of each collection point specifically includes: Retrieving actual position information of each target point in the road sign from a preset database; the actual position information is the recorded position information of each target point recorded in the preset database, and the recorded position information specifically includes the horizontal coordinate, vertical coordinate, and depth coordinate of the target point recorded in the preset database; The coordinate information and direction information of the actual position information and the preliminary position information are compared to obtain the offset vector feature of each collection point of the road sign.

4. The method for automatic vehicle navigation based on road sign recognition according to claim 1, wherein: The method of performing angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point to obtain the identification posture information of each acquisition point and determining the change trend of the identification posture information of different acquisition points within each preset time period specifically includes: Acquire the first point feature and the second point feature of the target point and the reference point of the reference object in the image data of each acquisition point respectively, match the first point feature with the second point feature, and obtain the visual feature of each acquisition point of the target point; Determine the first tilt angle of each collection point of the road sign according to the visual features of all target points in each collection point; Determine a second tilt angle of each collection point of the road sign according to the offset vector feature of each collection point; generating a transformation matrix for each acquisition point according to a first tilt angle and a second tilt angle of each acquisition point; Perform angle correction processing on the image data of the corresponding acquisition point according to the transformation matrix of each acquisition point to obtain the identification posture information of each acquisition point; The speed information of the vehicle is obtained, and the changing trend of the identification posture information of different collection points within each preset time period is determined based on the speed information.

5. The method for automatic vehicle navigation based on road sign recognition according to claim 4, wherein: The obtaining of the vehicle's speed information and determining the change trend of the identification posture information of different collection points within each preset time period according to the speed information specifically includes: Obtaining vehicle speed information and determining a preset time period based on the speed information; Determine a first position offset of the identification posture information of adjacent collection points within each preset time period, and a second position offset of the identification posture information of the first collection point and the last collection point within the preset time period; A change trend is obtained according to the first and second position offsets.

6. The method for automatic vehicle navigation based on road sign recognition according to claim 4, wherein: Before the step of obtaining the vehicle's speed information and determining the change trend of the identification posture information of different collection points within a preset time period based on the speed information, the method further includes: The sign posture information is verified to determine whether the sign posture information is in a standard posture; the standard posture is that the road sign is visually in a vertical state.

7. The method for automatic vehicle navigation based on road sign recognition according to claim 1, wherein: The method of obtaining the vehicle driving trajectory and adjusting the vehicle driving trajectory according to the change trend and the offset vector feature to obtain the predicted driving trajectory specifically includes: Extract vehicle driving trajectory, change trend and offset vector features within each preset time period; Determine whether the change trend exceeds a preset change range based on the vehicle's driving speed; If it is determined that the preset change range is exceeded, the vehicle driving trajectory corresponding to the preset time period is marked as a key change trajectory; The key change trajectory in the vehicle's driving trajectory is adjusted according to the change trend and offset vector characteristics to obtain the predicted driving trajectory.

8. The method for automatic vehicle navigation based on road sign recognition according to claim 7, wherein: The step of adjusting the key change trajectory in the vehicle's driving trajectory according to the change trend and the offset vector feature to obtain the predicted driving trajectory specifically includes: Obtain the historical driving trajectory of historical vehicles passing the same road sign; According to the historical driving trajectory, change trend and offset vector characteristics, the key change trajectory is adjusted to obtain the predicted driving trajectory; Determine whether the predicted driving trajectory meets safe driving conditions, and adjust the predicted driving trajectory if it does not meet safe driving conditions.

9. A vehicle automatic driving navigation system based on road sign recognition, characterized in that: include: A preliminary processing module is used to obtain image data and point cloud data of the road sign at each collection point, align the image data and point cloud data to obtain a fused sign feature for each collection point, and extract preliminary position information of the road sign from the fused sign feature; the time intervals between adjacent collection points are equal; The offset determination module is used to obtain the actual position information of the road sign, compare the actual position information with the preliminary position information, and obtain the offset vector characteristics of each collection point; A change trend module is used to perform angle correction processing on the image data of each acquisition point based on the visual features and offset vector features contained in the image data of each acquisition point, obtain the identification posture information of each acquisition point, and determine the change trend of the identification posture information of different acquisition points within a preset time period; The trajectory prediction module is used to obtain the vehicle's driving trajectory, adjust the vehicle's driving trajectory according to the change trend and the offset vector characteristics to obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a stored computer program, wherein, when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute the vehicle automatic driving navigation method based on road sign recognition as described in any one of claims 1 to 8.

Images