A vehicle automatic driving navigation method and system based on road sign recognition

Abstract

The application provides a vehicle automatic driving navigation method and system based on road sign recognition, and relates to the field of automatic driving, which comprises the following steps: acquiring image data and point cloud data of road signs, performing alignment processing on the image data and the point cloud data to obtain fused sign features, and extracting preliminary position information; acquiring actual position information of the road signs, comparing the actual position information with the preliminary position information to obtain offset vector features; performing angle correction processing on the image data according to the visual features and the offset vector features to obtain sign posture information, and determining the change trend of the sign posture information; acquiring a vehicle driving track, adjusting the vehicle driving track according to the change trend and the offset vector features to obtain a predicted driving track, and performing vehicle navigation based on the predicted driving track. The application realizes real-time precision and dynamic updating of vehicle automatic driving.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of automatic driving, and in particular to a vehicle automatic driving navigation method and system based on road sign recognition. BACKGROUND

[0002] As a core pillar in the field of intelligent transportation, automatic driving technology has a key significance in improving road safety and traffic efficiency. Precise recognition of road signs is an indispensable part of automatic driving systems, directly affecting the navigation and decision-making capabilities of vehicles using automatic driving systems.

[0003] However, existing road sign recognition methods have significant shortcomings in dealing with complex dynamic environments. They rely on static sign preset databases during recognition, making it difficult to adapt to changes in sign locations caused by construction, adverse weather, or human damage. In addition, some road sign recognition methods lack sufficient robustness when dealing with physical state changes of signs such as position deviation or inclination, which can easily lead to recognition errors and affect the reliability of automatic driving systems. Dynamic changes in road sign locations have become a core challenge in road sign recognition in the field of automatic driving.

[0004] Therefore, how to recognize more accurate and reliable driving trajectories in complex dynamic environments to improve navigation accuracy has become an important issue to be solved in the field of automatic driving. SUMMARY

[0005] In view of this, the present application proposes a vehicle automatic driving navigation method and system based on road sign recognition to recognize more accurate and reliable driving trajectories in complex dynamic environments to improve navigation accuracy.

[0006] The technical solution of the present application is as follows:

[0007] According to a first aspect, the present application provides a vehicle automatic driving navigation method based on road sign recognition, comprising:

[0008] Obtaining image data and point cloud data of road signs at each collection point, aligning the image data and point cloud data to obtain fusion sign features at each collection point, and extracting preliminary position information of the road signs from the fusion sign features; the time intervals of adjacent collection points are equal;

[0009] Obtaining actual position information of the road signs, comparing the actual position information with the preliminary position information to obtain offset vector features at each collection point;

[0010] According to the visual features and the offset vector features contained in the image data of each collection point, the image data of the corresponding collection point is subjected to angle correction processing to obtain the identification posture information of each collection point, and the change trend of the identification posture information of different collection points in each preset time period is determined.

[0011] The vehicle driving track is obtained, the vehicle driving track is adjusted according to the change trend and the offset vector features to obtain a predicted driving track, and vehicle navigation is performed based on the predicted driving track.

[0012] In combination with the first aspect, in a first implementation manner of the first aspect, the image data and the point cloud data of the road identification of each collection point are obtained, the image data and the point cloud data are subjected to alignment processing to obtain the fusion identification features of each collection point, and the preliminary position information of the road identification is extracted from the fusion identification features, specifically including:

[0013] The image data of the road identification of each collection point is obtained, and the image data is subjected to denoising processing.

[0014] The point cloud data of the road identification of each collection point is obtained.

[0015] The image data and the point cloud data are aligned in the time dimension by using the time stamp.

[0016] The image data and the point cloud data aligned in the time dimension are mapped into a preset spatial coordinate system to obtain the fusion identification features of each collection point.

[0017] The fusion identification features are subjected to correction processing, and the preliminary position information of the target point position in the corrected fusion identification features is extracted.

[0018] In combination with the first implementation manner of the first aspect, in a second implementation manner of the first aspect, the actual position information of the road identification is obtained, the actual position information and the preliminary position information are compared to obtain the offset vector features of each collection point, specifically including:

[0019] The actual position information of each target point position in the road identification is called from a preset database; the actual position information is the recorded position information of each target point position recorded in the preset database, and the recorded position information specifically includes the horizontal coordinate, the vertical coordinate and the depth coordinate of the target point position recorded in the preset database.

[0020] The coordinate information and the direction information of the actual position information and the preliminary position information are compared to obtain the offset vector features of each collection point of the road identification.

[0021] With reference to the first aspect, in a third implementation form of the first aspect, the image data of each collection point is subjected to angle correction processing according to the visual features and the offset vector features contained in the image data of each collection point, to obtain the identification posture information of each collection point, and the change trend of the identification posture information of different collection points in each preset time period is determined, specifically including:

[0022] The first point feature and the second point feature of the target point and the reference point of the reference object in the image data of each collection point are acquired respectively, and the first point feature and the second point feature are matched to obtain the visual features of each collection point of the target point;

[0023] The first inclination angle of each collection point of the road sign is determined according to the visual features of all target points in each collection point;

[0024] The second inclination angle of each collection point of the road sign is determined according to the offset vector features of each collection point;

[0025] The transformation matrix of each collection point is generated according to the first inclination angle and the second inclination angle of each collection point;

[0026] The image data of the corresponding collection point is subjected to angle correction processing according to the transformation matrix of each collection point, to obtain the identification posture information of each collection point;

[0027] The speed information of the vehicle is acquired, and the change trend of the identification posture information of different collection points in each preset time period is determined according to the speed information.

[0028] With reference to the third implementation form of the first aspect, in a fourth implementation form of the first aspect, the speed information of the vehicle is acquired, and the change trend of the identification posture information of different collection points in each preset time period is determined according to the speed information, specifically including:

[0029] The speed information of the vehicle is acquired, and a preset time period is determined according to the speed information;

[0030] The first position offset of the identification posture information of adjacent collection points in each preset time period, and the second position offset of the identification posture information of the first collection point and the last collection point in the preset time period are determined;

[0031] The change trend is obtained according to the first and second position offsets.

[0032] With reference to the third implementation form of the first aspect, in a fifth implementation form of the first aspect, before the step of acquiring the speed information of the vehicle and determining the change trend of the identification posture information of different collection points in the preset time period according to the speed information, the method further includes:

[0033] The identification posture information is checked to determine whether the identification posture information is in a standard posture; the standard posture is that the road identification is in a vertical state in vision.

[0034] In combination with the first aspect, in a sixth implementation manner of the first aspect, the vehicle driving track is acquired, the vehicle driving track is adjusted according to the change trend and the offset vector feature, and a predicted driving track is obtained, specifically including:

[0035] The vehicle driving track, the change trend and the offset vector feature in each preset time period are extracted;

[0036] The change trend is determined according to the vehicle driving speed to determine whether the change trend exceeds a preset change amplitude;

[0037] In a case where it is determined that the preset change amplitude is exceeded, the vehicle driving track corresponding to the preset time period is marked as a key variable track;

[0038] The key variable track in the vehicle driving track is adjusted according to the change trend and the offset vector feature, and a predicted driving track is obtained.

[0039] In combination with the sixth implementation manner of the first aspect, in a seventh implementation manner of the first aspect, the key variable track in the vehicle driving track is adjusted according to the change trend and the offset vector feature, and a predicted driving track is obtained, specifically including:

[0040] The historical driving track of a historical vehicle passing through the same road identification is acquired;

[0041] The key variable track is adjusted according to the historical driving track, the change trend and the offset vector feature, and a predicted driving track is obtained;

[0042] It is determined whether the predicted driving track meets a safe driving condition, and the predicted driving track that does not meet the safe driving condition is adjusted.

[0043] According to the second aspect, an embodiment of the present application provides a vehicle automatic driving navigation system based on road identification recognition, the system including:

[0044] A preliminary processing module is configured to acquire image data and point cloud data of a road identification of each collection point, align the image data and the point cloud data to obtain fusion identification features of each collection point, and extract preliminary position information of the road identification from the fusion identification features; the time intervals of adjacent collection points are equal;

[0045] An offset determination module is configured to acquire actual position information of the road identification, compare the actual position information with the preliminary position information to obtain offset vector features of each collection point;

[0046] a change trend module configured to perform angle correction processing on the image data of each collection point according to the visual features and the offset vector features contained in the image data of each collection point, to obtain identification posture information of each collection point, and to determine the change trend of the identification posture information of different collection points in a preset time period;

[0047] a trajectory prediction module configured to obtain a vehicle driving trajectory, adjust the vehicle driving trajectory according to the change trend and the offset vector features, obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

[0048] The vehicle automatic driving navigation method and system based on road identification recognition of the present application have the following beneficial effects over the prior art:

[0049] By obtaining the image data and point cloud data of the road identification of each collection point, performing alignment processing on the image data and point cloud data, obtaining the fusion identification features of each collection point and extracting the preliminary position information of the road identification therefrom, comparing the preliminary position information with the actual position information of the road identification, obtaining the offset vector features, and then performing angle correction processing on the image data in combination with the visual features and the offset vector features to obtain the identification posture information and determine the change trend of the identification posture information of different collection points in each preset time period, the deviation between the road identification related information obtained by the vehicle-mounted sensor and the actual position information recorded in the preset database is compared in a complex dynamic environment, the road identification that has been displaced or tilted is accurately identified, a precise and reliable predicted driving trajectory is finally output in combination with the motion trajectory prediction algorithm, the problem of inaccurate positioning of the road identification under the influence of factors such as dynamic environment change, external factor interference and high-speed driving is effectively solved, the accuracy and reliability of the navigation of the automatic driving system are improved, a more safe and reliable navigation reference is provided for the automatic driving vehicle, and the overall performance of the intelligent transportation system is of great significance. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0051] Figure 1 Fig. 1 is a flowchart of the vehicle automatic driving navigation method based on road identification recognition of the present application;

[0052] Figure 2 Fig. 2 is another flowchart of the vehicle automatic driving navigation method based on road identification recognition of the present application;

[0053] Figure 3 Figure 3 is a flowchart of a third embodiment of the vehicle automatic driving navigation method based on road sign recognition of the present application;

[0054] Figure 4 Figure 4 is a flowchart of a fourth embodiment of the vehicle automatic driving navigation method based on road sign recognition of the present application;

[0055] Figure 5 Figure 5 is a structural diagram of the vehicle automatic driving navigation system based on road sign recognition of the present application. DETAILED DESCRIPTION

[0056] The technical solutions in the embodiments of the present application will be clearly and completely described in combination with the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0057] The vehicle automatic driving navigation method based on road sign recognition provided in the specification aims to accurately identify the road sign that has been displaced or tilted by the deviation of the road sign related information from the actual position information recorded in the preset database in a complex dynamic environment, and output accurate and reliable predicted driving trajectory in combination with the motion trajectory prediction algorithm, so as to improve the navigation accuracy.

[0058] Please refer to Figure 1 , Figure 1 Figure 1 is a flowchart of the vehicle automatic driving navigation method based on road sign recognition of the present application. The method can include the following steps:

[0059] In S101, image data and point cloud data of the road sign of each collection point are acquired, the image data and the point cloud data are aligned, the fusion sign feature of each collection point is obtained, and the preliminary position information of the road sign is extracted from the fusion sign feature. The preliminary position information includes the horizontal coordinate, the vertical coordinate and the depth coordinate of each target point in the road sign. The collection point is the time node corresponding to the data collection. The vehicle is equipped with a vehicle sensor for acquiring the image data and the point cloud data. The time interval between adjacent collection points is always equal, and the time interval can be set to 1-3s.

[0060] In the embodiment, the target point is also a core recognition point of the road sign, such as a center point, a contour point, a distinguishing point, and the like of the road sign, which can be quickly recognized. The distinguishing point is a point in a region that distinguishes the road sign from other road signs. It can be understood that each type of road sign has a corresponding plurality of target points, and different road signs have different target points.

[0061] In the embodiment, the alignment processing includes alignment in the time dimension and the space dimension. The fusion sign feature obtained in this way can integrate the texture information contained in the image data and the depth information contained in the point cloud data, thereby improving the accuracy when the fusion sign feature is applied, significantly enhancing the stability and accuracy of road sign recognition, effectively dealing with road sign recognition in a dynamic environment, and providing important support for the construction of an intelligent transportation system.

[0062] S102, actual position information of the road sign is obtained, and the actual position information and the preliminary position information are compared to obtain an offset vector feature of each collection point. The offset vector feature includes an offset amount and an offset direction. The offset vector feature can intuitively reflect the offset of the position of the road sign recorded by the autonomous driving system, and provide an accurate reference for subsequent processing.

[0063] By comparing the actual position information and the preliminary position information, the difference between the two types of position information is obtained, and then the offset amount is obtained. The offset amount can reflect whether the road sign is planarly offset. Influenced by the performance of the vehicle-mounted sensor, the vehicle driving trajectory, and the driving environment (such as road construction, adverse weather, and the like), the offset vector feature of the road sign will be different.

[0064] In the embodiment, whether the displacement of the road sign in the plane exceeds a preset movement amount is determined according to the offset amount of all target points. If the preset movement amount is exceeded, it indicates that the preliminary position information of the road sign obtained by the autonomous driving system is deviated. Therefore, the preliminary position information with deviation, the offset amount, and the unique identification information (such as ID) of the road sign are associated to obtain association information. The association information can be presented in a structured format, including unique identification information, preliminary position information, and offset amount, and the like. The association information can be stored in a preset database. This storage method is convenient for subsequent query and update, and at the same time provides data support for the maintenance and management of the road sign.

[0065] S103, the image data of each collection point is angle-corrected according to the visual feature and the offset vector feature contained in the image data of each collection point, to obtain identification posture information of each collection point, and to determine the change trend of the identification posture information of different collection points in each preset time period.

[0066] In the embodiment, for the offset vector feature of each collection point, the image data of the corresponding collection point is subjected to angle correction processing in combination with the change of the inclination angle and the change of the visual feature, to obtain the corrected road sign identification posture information of each collection point, and then the dynamic change of the identification posture information in the vehicle driving process due to the influence of complex road conditions and high-speed driving and other factors is analyzed. In order to ensure the real-time identification of the road sign by the automatic driving system, the analysis time window of the above identification posture information is limited to a preset time period, thereby obtaining the change trend of the identification posture information, obtaining the real-time three-dimensional coordinate data of the road sign, and then realizing the dynamic position update and recording of the identification posture information.

[0067] Specifically, step S103 includes:

[0068] S1031, respectively acquiring the first point feature and the second point feature of the target point and the reference point of the reference object in the image data of each collection point, matching the first point feature and the second point feature, and obtaining the visual feature of each collection point of the target point. Wherein, the reference object can have at least one reference point, and these reference points participate in the subsequent judgment of the change of the inclination angle, for example, the reference object can be a rod body on which the road sign is installed, and the reference point of the reference object is the connection position point of the rod body connected with the road sign.

[0069] The first and second point features above contain the position information of the point in the image data.

[0070] In the embodiment, each target point is a visual feature point used in visual feature processing, and by analyzing the relative position of each target point relative to the reference object, the visual feature contained in the image data can be obtained. The visual feature can reflect whether the corresponding target point has a large visual feature change.

[0071] S1032, determining the first inclination angle of the road sign at each collection point according to the visual feature of all target points in each collection point.

[0072] Since the visual feature is obtained according to the first point feature of the target point and the second point feature of the reference point, after collecting the visual features of all target points, the inclination angle of the road sign in the visual layer in the image data collected by the sensor can be determined.

[0073] S1033, determining the second inclination angle of the road sign at each collection point according to the offset vector feature of each collection point.

[0074] The offset direction contained in the offset vector feature can obtain the second angle information above, and the second angle information can reflect whether the corresponding target point has a large inclination angle change.

[0075] S1034, generating a transformation matrix of each collection point according to the first tilt angle and the second tilt angle of each collection point.

[0076] The transformation matrix is obtained by comprehensively considering the visual feature change and the tilt angle change, and can improve the correction effect in the angle correction process of the image data.

[0077] S1035, performing angle correction processing on the image data of the corresponding collection point according to the transformation matrix of each collection point, to obtain the identification posture information of each collection point.

[0078] In this embodiment, the image data of the corresponding collection point is remapped according to the obtained transformation matrix of each collection point to perform angle correction processing. The transformation matrix aims to adjust the pixel distribution of each pixel point in the image data, that is, each point position, so that it is as close as possible to the actual standard posture. The standard posture is that the road sign is in a vertical state in vision.

[0079] S1036, verifying the identification posture information to determine whether the identification posture information is in a standard posture.

[0080] Considering that some parameters in the generated transformation matrix may have problems, resulting in poor angle correction processing effect, in this embodiment, the identification posture information is also analyzed for posture analysis. The distribution information of each pixel point in the identification posture information is analyzed to determine whether the angle correction processing effect meets the expectation, such as verifying whether the boundary line of the road sign is straight, and whether the text and symbols are clear and visible.

[0081] For example, when the distribution information of each pixel point in the identification posture information is extracted and judged whether it is in a standard posture, the distribution information of the pixel point is analyzed by a posture parameter calculation tool. Assuming that it is found through verification that the vertical axis of the road sign basically coincides with the standard axis, and only has a slight deviation of 0.05 degrees, it is determined that the angle correction is successful, and the final angle posture information is generated and used for subsequent identification state recording, facilitating long-term tracking and management.

[0082] S1037, obtaining speed information of the vehicle, and determining a change trend of the identification posture information of different collection points in a preset time period according to the speed information.

[0083] In this embodiment, the position offset of the identification posture information of different collection points in the preset time period is analyzed and processed during the analysis of the change trend to generate a dynamic position. It can be understood that the dynamic position can be recorded.

[0084] More specifically, step S1037 includes:

[0085] S10371, obtain speed information of the vehicle, and determine a preset time period according to the speed information.

[0086] In the speed information, the faster the speed of the vehicle, the longer the preset time period is set, and vice versa.

[0087] S10372, determine a first position offset of the identification posture information of the adjacent collection points in each preset time period, and a second position offset of the identification posture information of the first collection point and the last collection point in the preset time period.

[0088] In this embodiment, the change rate of the identification posture information can be captured by obtaining the position offset of the identification posture information, which includes the generated plane displacement and the generated angle change. Recording the position offset of the identification posture information of the adjacent collection points and the start and end collection points can better determine whether the identification posture information has periodic offset. For example, the identification posture information of a speed limit sign in a preset time period presents periodic offset, which may be due to road construction or sensor calibration problem, so that the automatic driving system can generate a corresponding trajectory report. This way can provide data support for road management, ensure the accuracy and reliability of the identification information, and through the whole process optimization from data collection to storage analysis, the data processing capacity under complex road conditions can be effectively improved, and a solid foundation is provided for subsequent decision-making.

[0089] S10373, obtain a change trend according to the first and second position offsets.

[0090] The change trend obtained in this way helps to capture the short-term trajectory characteristics of the road sign and forms the basis of trajectory tracking.

[0091] S104, obtain a vehicle driving trajectory, adjust the vehicle driving trajectory according to the change trend and the offset vector feature, obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

[0092] In this embodiment, in combination with the trajectory prediction algorithm, the relative motion trajectory of the vehicle and the road sign is predicted for the error information of the navigation information output by the automatic driving system due to the dynamic environment, and then the vehicle driving trajectory is adjusted according to the relative motion trajectory to obtain a predicted driving trajectory, so that the vehicle driving trajectory and the relative motion of the road sign show consistent motion trend, and the adaptability of the automatic driving system to complex dynamic environment is improved.

[0093] The road sign recognition-based vehicle automatic driving navigation method provided by the application obtains the image data and point cloud data of the road sign of each collection point, aligns the image data and point cloud data, obtains the fusion sign feature of each collection point, extracts the preliminary position information of the road sign from the fusion sign feature, compares the preliminary position information with the actual position information of the road sign, obtains the offset vector feature, performs angle correction on the image data by combining the visual feature and the offset vector feature, obtains the sign posture information, determines the change trend of the sign posture information of different collection points in each preset time period, compares the road sign information obtained by the vehicle-mounted sensor with the actual position information recorded in the preset database in the complex dynamic environment, accurately identifies the road sign that has been displaced or tilted, finally outputs the accurate and reliable predicted driving track by combining the motion trajectory prediction algorithm, effectively solves the problem of inaccurate road sign positioning caused by dynamic environment changes, external factor interference, high-speed driving and other factors, improves the accuracy and reliability of the navigation of the automatic driving system, provides a safer and more reliable navigation reference for the automatic driving vehicle, and has important significance for improving the overall performance of the intelligent transportation system.

[0094] Referring to Figure 2 The method can further include the following steps:

[0095] S2011, image data of a road sign of each collection point is obtained, and the image data is denoised.

[0096] In this embodiment, the real-time image data of the road sign is obtained by the vehicle-mounted camera mounted on the vehicle. The image data collected in this way retains the environmental noise information caused by light changes, shadows, rain and fog weather and the like during the driving of the vehicle. In order to remove these environmental noise information, the image data can be denoised by a pre-established filtering tool such as a median filtering tool, so as to effectively remove the isolated noise points in the image data, thereby improving the image quality of the image data.

[0097] S2012, point cloud data of the road sign of each collection point is obtained.

[0098] In this embodiment, the three-dimensional space information of the road sign and its surroundings is scanned by the vehicle-mounted laser radar mounted on the vehicle, so as to obtain high-precision point cloud data containing depth information.

[0099] S2013, the image data and the point cloud data are aligned in the time dimension by using the time stamp.

[0100] According to the time stamp information in the obtained image data and point cloud data, the alignment in the time dimension of the two is performed and it is ensured that the image data and the point cloud data of the same collection point participate in the subsequent spatial multi-dimensional processing.

[0101] In order to ensure that the data captured by different sensors can realize subsequent data fusion, the denoised image data and the point cloud data can be standardized before being aligned in the time dimension. The image data and the point cloud data are unified into structured data.

[0102] S2014, map the image data and the point cloud data aligned in the time dimension into a preset spatial coordinate system to obtain a fusion identification feature of each collection point. This step maps the image data and the point cloud data into the same spatial coordinate system through coordinate system conversion.

[0103] It can be seen that the fusion identification feature is obtained after the image data and the point cloud data are aligned in the time and spatial dimensions. This information fusion method can integrate the texture information contained in the image data and the depth information contained in the point cloud data, thereby improving the accuracy when the fusion identification feature is applied.

[0104] S2015, correct the fusion identification feature, and extract preliminary position information of a target point of the corrected fusion identification feature.

[0105] In order to avoid the fusion identification feature from generating a position deviation exceeding the expectation, in the embodiment, the fusion identification feature is also corrected by a preset standard matrix, so as to ensure that the position deviation of the fusion identification feature is controlled within a reasonable range. The corrected fusion identification feature can more truly reflect the actual features of the road sign, avoid the cumulative error of subsequent position calculation, and significantly improve the system reliability.

[0106] In the embodiment, the preliminary position information of the target point of the road sign contained in the fusion identification feature can be extracted by a three-dimensional position calculation tool. The three-dimensional position calculation tool can extract the preliminary position information based on the principle of triangulation. Assuming that the preliminary spatial position of the center point of a speed limit sign (which is one of the target points) on a straight road is 50 meters away from the front of the vehicle and 2.5 meters high, such position information provides a key reference for subsequent automatic driving decision-making, which helps to improve the navigation accuracy and safety.

[0107] S202, obtain actual position information of the road sign, compare the actual position information and the preliminary position information, and obtain an offset vector feature of the road sign. For details, refer to step S102.

[0108] S203, according to the visual features contained in each acquisition point image data and the offset vector features, the image data of the corresponding acquisition point is angle corrected, the identification posture information of each acquisition point is obtained, and the change trend of the identification posture information of different acquisition points in each preset time period is determined. For specific content, refer to step S103.

[0109] S204, obtaining the vehicle driving track, adjusting the vehicle driving track according to the change trend and the offset vector features, obtaining the predicted driving track, and performing vehicle navigation based on the predicted driving track. For specific content, refer to step S104.

[0110] Please refer to Figure 3 , the method can further include the following steps:

[0111] S301, obtaining the image data and point cloud data of the road identification of each acquisition point, aligning the image data and point cloud data, obtaining the fusion identification features of each acquisition point, and extracting the preliminary position information of the road identification from the fusion identification features. For specific content, refer to step S101.

[0112] S3021, the actual position information of each target point of the road identification is called from the preset database.

[0113] In this embodiment, the actual position information is the recorded position information of each target point in the preset database, wherein the recorded position information specifically includes the horizontal coordinate, vertical coordinate and depth coordinate of the target point recorded in the preset database. The preset database records the above three coordinate values of the target point corresponding to the road identification, and the coordinate values recorded in the preset database represent the actual position coordinates of the road identification.

[0114] S3022, comparing the coordinate information and direction information of the actual position information and the preliminary position information, and obtaining the offset vector features of the road identification.

[0115] The calculation method of the offset amount of the road identification in the plane displacement is:

[0116]

[0117] Among them, represents the offset amount of the first target point of the road identification; represents the horizontal coordinate recorded in the preset database of the first target point; represents the horizontal coordinate of the first target point in the preliminary position information; represents the offset amount of the first target point of the road identification; represents the horizontal coordinate recorded in the preset database of the first target point; represents the horizontal coordinate of the first target point in the preliminary position information; represents the offset amount of the first target point of the road identification; The ordinates of each target point are recorded in the preset database; Indicates the first The vertical coordinates of each target point in the preliminary location information; Indicates the first The depth coordinates of each target point are recorded in the preset database; Indicates the first The depth coordinates of each target point in the preliminary location information. , as well as This constitutes the first The actual location information of each target point. , as well as This constitutes the first Preliminary location information of each target point.

[0118] It should be noted that when obtaining the coordinate information of the actual location and the preliminary location, reference objects can be used to determine the specific values ​​of the coordinate information.

[0119] Then, by comparing the actual location information with the directional information of the preliminary location information, the offset direction is obtained. The offset vector feature of the road sign is obtained by combining the offset amount and the offset direction.

[0120] S303. Based on the visual features and offset vector features contained in the image data of each acquisition point, perform angle correction processing on the image data of the corresponding acquisition point to obtain the identification posture information of each acquisition point, and determine the changing trend of the identification posture information of different acquisition points within each preset time period. For details, refer to step S103.

[0121] S304. Obtain the vehicle's driving trajectory, adjust the trajectory based on the changing trend and offset vector characteristics to obtain a predicted trajectory, and perform vehicle navigation based on the predicted trajectory. Refer to step S104 for details.

[0122] Please see Figure 4 The method may also include the following steps:

[0123] S401. Acquire image data and point cloud data of road signs at each collection point, perform alignment processing on the image data and point cloud data to obtain the fused signage features of each collection point, and extract the preliminary location information of the road signs from the fused signage features. Refer to step S101 for details.

[0124] S402. Obtain the actual location information of the road signs, compare the actual location information with the preliminary location information, and obtain the offset vector features of each collection point. Refer to step S102 for details.

[0125] S403、According to the visual features contained in each acquisition point image data and the offset vector features, the image data of the corresponding acquisition point is angle corrected to obtain the identification posture information of each acquisition point, and the change trend of the identification posture information of different acquisition points in each preset time period is determined. For specific content, refer to step S103.

[0126] S4041、Extract the vehicle driving trajectory, change trend and offset vector feature in each preset time period.

[0127] In the process of generating the predicted driving trajectory of the vehicle, the vehicle driving trajectory, change trend and offset vector feature can be arranged first, and the above data is summarized in time periods. In this embodiment, the preset time period also serves as the induction time window, and the above data in each preset time period is extracted, and then the data is processed in segments based on time sequence. This segmented induction can clearly present the short-term change trend of the identification position, which is convenient for subsequent analysis of key change nodes. The above data summarized in time periods will contain timestamp information to ensure the integrity of the data.

[0128] Among them, the vehicle driving trajectory can be obtained through various vehicle-mounted sensors, and the corresponding offset vector feature in the preset time period is the set of offset vector features corresponding to all acquisition points in the preset time period.

[0129] S4042、Determine whether the change trend exceeds the preset change amplitude according to the vehicle driving speed.

[0130] According to the vehicle driving speed, the motion trend of the vehicle can be obtained, and the change trend indicates the trajectory characteristics of the road sign. Both trends will be affected by the dynamic environment, for example, due to road construction or sensor calibration. In this embodiment, a desired preset change amplitude is first determined based on the vehicle driving speed, and then it is determined whether the change trend exceeds the preset change amplitude. If the change amplitude is not exceeded, it means that the relative motion trajectory of the vehicle and the road sign is as expected.

[0131] S4043、In the case where it is determined that the preset change amplitude is exceeded, the vehicle driving trajectory corresponding to the preset time period is marked as a key change trajectory.

[0132] This marking method can quickly identify abnormal changes in the corresponding information of the road sign, such as coordinate offset caused by vehicle-mounted sensor jitter, thereby improving the accuracy of subsequent prediction.

[0133] S4044、According to the change trend and the offset vector feature, the key change trajectory in the vehicle driving trajectory is adjusted to obtain the predicted driving trajectory.

[0134] In the adjustment process of the trajectory, only the part of the relative motion trajectory of the vehicle and the road sign that exceeds the expectation is adjusted, which can improve the efficiency of subsequent identification and reduce the amount of trajectory correction data. At the same time, such a setting can also analyze the potential error risk in the navigation information output by the automatic driving system and generate more reliable navigation information.

[0135] More specifically, step S4044 includes:

[0136] S40441, obtaining a historical driving trajectory of the vehicle driving through the same road sign.

[0137] Adjusting the trajectory by obtaining the historical driving trajectory of other vehicles driving through the same road sign can further improve the reliability of path planning.

[0138] It should be noted that the driving trajectory of the current vehicle driving through the same road sign at other time points can also be classified as historical trajectory information.

[0139] S40442, adjusting the key variable trajectory according to the historical driving trajectory, the change trend and the offset vector feature to obtain a predicted driving trajectory.

[0140] The adjustment method of the trajectory is a local adjustment of the key variable trajectory. The path information is recalculated in combination with the dynamic change data of the road sign, i.e. the change trend and the offset vector feature. The purpose of the adjustment is to make the relative motion of the vehicle driving trajectory and the road sign show a consistent motion trend, obtain an adjusted predicted driving trajectory, and ensure that the predicted driving path planned by the automatic driving system can avoid the area that may be misjudged. This adjustment method improves the adaptability of the path.

[0141] S40443, determining whether the predicted driving trajectory meets the safe driving condition, and adjusting the predicted driving trajectory that does not meet the safe driving condition.

[0142] In order to ensure the safety and practical value of the predicted driving trajectory, in this embodiment, whether the predicted driving trajectory meets the safe navigation condition is evaluated. If the deviation value in the adjusted predicted driving trajectory still exceeds the safe range, the automatic driving system will start a secondary correction process. The secondary correction will be optimized in combination with more environmental data to ensure that the final path meets the safety standard, thereby providing a guarantee for the reliability of the path.

[0143] The automatic driving system can also backup the predicted driving trajectory after the secondary adjustment to form a complete correction record. Assuming that a certain adjustment involves multiple road sign areas, the automatic driving system will record the data comparison before and after the adjustment of each area. This archiving method facilitates subsequent tracing and analysis, thereby improving the efficiency of data management.

[0144] In the embodiment, the preliminary position information can be calibrated in combination with navigation data such as GPS data, so as to effectively cope with dynamic environmental changes.

[0145] The system provided by the embodiment of the application is described below, and the system described below can be correspondingly referred to the method described above.

[0146] Please refer to Figure 5 , Figure 5 The structure schematic diagram of the vehicle automatic driving navigation system based on road sign recognition provided by the embodiment of the application is shown, and the system can include:

[0147] The preliminary determination module 10 is configured to acquire image data and point cloud data of the road sign of each collection point, align the image data and the point cloud data, obtain fusion sign features of each collection point, and extract preliminary position information of the road sign from the fusion sign features. The preliminary position information includes horizontal coordinates, vertical coordinates and depth coordinates of each target point of the road sign. The collection point is a time node corresponding to data collection. The vehicle is provided with a vehicle sensor for acquiring the image data and the point cloud data. The time interval between adjacent collection points is always equal, and the time interval can be set to 1-3s.

[0148] In the embodiment, the target point is also a core recognition point of the road sign, such as a center point, a contour point, a distinguishing point and other points that can be quickly recognized. The distinguishing point is a point in the area that distinguishes the road sign from other road signs. It can be understood that each type of road sign has a plurality of corresponding target points, and different road signs have different target points.

[0149] In the embodiment, the alignment includes alignment in time dimension and space dimension. The fusion sign features obtained in this way can integrate the texture information contained in the image data and the depth information contained in the point cloud data, thereby improving the accuracy of the fusion sign features when applied, significantly enhancing the stability and accuracy of the road sign recognition, effectively coping with road sign recognition in dynamic environment, and providing important support for the construction of intelligent transportation system.

[0150] The offset determination module 20 is configured to acquire actual position information of the road sign, compare the actual position information and the preliminary position information, and obtain offset vector features of each collection point. The offset vector features include offset amount and offset direction. The offset vector features can intuitively reflect the offset of the position of the road sign recorded by the automatic driving system, and provide accurate reference for subsequent processing.

[0151] By comparing the actual position information and the preliminary position information, the difference between the two kinds of position information is obtained, and then the offset is obtained, which can reflect whether the road sign exists planar offset. Influenced by the performance of the vehicle-mounted sensor, the driving trajectory of the vehicle and the driving environment (such as road construction, bad weather and other factors), the offset vector characteristics of the road sign will be different.

[0152] In the embodiment, whether the displacement of the road sign in the plane exceeds the preset movement amount is determined according to the offset of all target points. If the preset movement amount is exceeded, it indicates that the preliminary position information of the road sign obtained by the automatic driving system is deviated, and therefore the preliminary position information with deviation, the offset and the unique identification information of the road sign are associated to obtain association information. The association information can be presented in a structured format and contains information such as unique identification information, preliminary position information and offset. The association information can be stored in a preset database. This storage method is convenient for subsequent query and update, and at the same time provides data support for the maintenance and management of the road sign.

[0153] The change trend module 30 is configured to perform angle correction processing on the image data of each collection point according to the visual features and the offset vector features contained in the image data of each collection point, to obtain the identification posture information of each collection point, and to determine the change trend of the identification posture information of different collection points in each preset time period.

[0154] In the embodiment, for the offset vector characteristics of each collection point, the image data of the corresponding collection point is subjected to angle correction processing in combination with the change of the inclination angle and the change of the visual features, to obtain the identification posture information of each collection point of the corrected road sign. Then, the dynamic changes of the identification posture information caused by complex road conditions and high-speed driving during the driving of the vehicle are analyzed. In order to ensure the real-time identification of the road sign by the automatic driving system, the analysis time window of the above identification posture information is limited to a preset time period, thereby obtaining the change trend of the identification posture information, obtaining the real-time three-dimensional coordinate data of the road sign, and then realizing the dynamic position update and record of the identification posture information.

[0155] The trajectory prediction module 40 is configured to obtain the driving trajectory of the vehicle, adjust the driving trajectory of the vehicle according to the change trend and the offset vector characteristics, obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

[0156] In the embodiment, in combination with the trajectory prediction algorithm, the relative motion trajectory of the vehicle and the road sign is predicted for the error information of the navigation information output by the automatic driving system due to the dynamic environment, and the vehicle driving trajectory is adjusted according to the relative motion trajectory to obtain a predicted driving trajectory, so that the vehicle driving trajectory and the relative motion of the road sign show a consistent motion trend, and the adaptability of the automatic driving system to the complex dynamic environment is improved.

[0157] The vehicle automatic driving navigation system based on road sign recognition provided by the application obtains the image data and the point cloud data of the road sign of each collection point, performs alignment processing on the image data and the point cloud data, obtains the fusion identification feature of each collection point, extracts the preliminary position information of the road sign from the fusion identification feature, compares the preliminary position information with the actual position information of the road sign, obtains the offset vector feature, performs angle correction processing on the image data in combination with the visual feature and the offset vector feature, obtains the identification posture information, determines the change trend of the identification posture information of different collection points in each preset time period, compares the road sign related information obtained by the vehicle-mounted sensor with the actual position information recorded in the preset database in the complex dynamic environment, accurately identifies the road sign that has been displaced or tilted, and finally outputs the predicted driving trajectory in combination with the motion trajectory prediction algorithm, so that the problem of inaccurate positioning of the road sign under the influence of factors such as dynamic environment change, external factor interference and high-speed driving is effectively solved, the accuracy and reliability of the navigation of the automatic driving system are improved, a more safe and reliable navigation reference is provided for the automatic driving vehicle, and the overall performance of the intelligent transportation system is improved.

[0158] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the application, and not to limit them; although the application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the application.

Claims

1. A vehicle automatic driving navigation method based on road sign recognition, characterized in that: The method comprises: obtaining image data and point cloud data of the road sign at each collection point, aligning the image data and the point cloud data, obtaining fusion sign features of each collection point, and extracting preliminary position information of the road sign from the fusion sign features; the time intervals of adjacent collection points are equal; obtaining actual position information of the road sign, comparing the actual position information with the preliminary position information, and obtaining offset vector features of each collection point; performing angle correction processing on the image data of the corresponding collection point according to the visual features and the offset vector features contained in the image data of each collection point, obtaining sign posture information of each collection point, and determining the change trend of the sign posture information of different collection points in each preset time period; obtaining a vehicle driving track, adjusting the vehicle driving track according to the change trend and the offset vector features, obtaining a predicted driving track, and performing vehicle navigation based on the predicted driving track; wherein the angle correction processing on the image data of the corresponding collection point according to the visual features and the offset vector features contained in the image data of each collection point, the obtaining of the sign posture information of each collection point, and the determination of the change trend of the sign posture information of different collection points in each preset time period specifically comprise: obtaining first point feature and second point feature of target point and reference point of reference object in the image data of each collection point, respectively, matching the first point feature and the second point feature, and obtaining visual features of each collection point of the target point; determining a first inclination angle of each collection point of the road sign according to the visual features of all target points in each collection point; determining a second inclination angle of each collection point of the road sign according to the offset vector features of each collection point; generating a transformation matrix of each collection point according to the first inclination angle and the second inclination angle of each collection point; performing angle correction processing on the image data of the corresponding collection point according to the transformation matrix of each collection point, and obtaining the sign posture information of each collection point; obtaining speed information of the vehicle, and determining the change trend of the sign posture information of different collection points in each preset time period according to the speed information. 2.The road sign recognition based vehicle automatic driving navigation method of claim 1, wherein: The obtaining of the image data and the point cloud data of the road sign at each collection point, the alignment of the image data and the point cloud data, the obtaining of the fusion sign features of each collection point, and the extraction of the preliminary position information of the road sign from the fusion sign features specifically comprise: obtaining image data of the road sign at each collection point, and performing denoising processing on the image data; obtaining point cloud data of the road sign at each collection point; aligning the image data and the point cloud data in the time dimension by using a time stamp; mapping the image data and the point cloud data aligned in the time dimension into a preset spatial coordinate system to obtain the fusion sign features of each collection point; performing correction processing on the fusion sign features, and extracting preliminary position information of the target point in the corrected fusion sign features. 3.The road sign recognition based vehicle automatic driving navigation method of claim 2, wherein: The obtaining of the actual position information of the road sign, the comparison of the actual position information with the preliminary position information, and the obtaining of the offset vector features of each collection point specifically comprise: The actual position information of each target point in the road sign is obtained from the preset database; the actual position information is the recorded position information of each target point recorded in the preset database, and the recorded position information specifically includes the horizontal coordinate, the vertical coordinate and the depth coordinate of the target point recorded in the preset database; The coordinate information and the direction information of the actual position information and the preliminary position information are compared to obtain the offset vector feature of each collection point of the road sign. 4.The road sign recognition based vehicle automatic driving navigation method of claim 1, wherein: The speed information of the vehicle is obtained, and the change trend of the identification posture information of different collection points in each preset time period is determined according to the speed information, specifically including: The speed information of the vehicle is obtained, and the change trend of the identification posture information of different collection points in each preset time period is determined according to the speed information, specifically including: The speed information of the vehicle is obtained, and the change trend of the identification posture information of different collection points in each preset time period is determined according to the speed information, specifically including: The speed information of the vehicle is obtained, and the change trend of the identification posture information of different collection points in each preset time period is determined according to the speed information, specifically including: 5.The road sign recognition based vehicle automatic driving navigation method of claim 1, wherein: Before the step of obtaining the speed information of the vehicle and determining the change trend of the identification posture information of different collection points in each preset time period, it further includes: The identification posture information is verified to determine whether the identification posture information is in a standard posture; the standard posture is that the road sign is in a vertical state in vision. 6.The road sign recognition based vehicle automatic driving navigation method of claim 1, wherein: The vehicle trajectory is obtained, and the vehicle trajectory is adjusted according to the change trend and the offset vector feature to obtain a predicted trajectory, specifically including: The vehicle trajectory, the change trend and the offset vector feature in each preset time period are extracted; It is determined whether the change trend exceeds a preset change amplitude according to the vehicle speed; In the case where it is determined that the preset change amplitude is exceeded, the vehicle trajectory corresponding to the preset time period is marked as a key variable trajectory; The key variable trajectory in the vehicle trajectory is adjusted according to the change trend and the offset vector feature to obtain a predicted trajectory. 7.The road sign recognition based vehicle automatic driving navigation method of claim 6, wherein: The key variable trajectory in the vehicle trajectory is adjusted according to the change trend and the offset vector feature to obtain a predicted trajectory, specifically including: The historical driving trajectory of the vehicle passing through the same road sign is obtained; The key variable trajectory is adjusted according to the historical driving trajectory, the change trend and the offset vector feature to obtain a predicted trajectory; It is determined whether the predicted trajectory meets the safe driving condition, and the predicted trajectory that does not meet the safe driving condition is adjusted.

8. A vehicle automatic driving navigation system based on road sign recognition, characterized by, A vehicle automatic driving navigation method based on road sign recognition is implemented, including: A preliminary processing module is configured to obtain image data and point cloud data of the road sign of each collection point, align the image data and the point cloud data to obtain fusion identification features of each collection point, and extract preliminary position information of the road sign from the fusion identification features; the time intervals of adjacent collection points are equal; An offset determination module is configured to obtain actual position information of the road sign, compare the actual position information and the preliminary position information, and obtain offset vector features of each collection point; The change trend module is configured to perform angle correction processing on the image data of each collection point according to the visual features and the offset vector features contained in the image data of each collection point, to obtain identification posture information of each collection point, and to determine a change trend of the identification posture information of different collection points within a preset time period. The trajectory prediction module is configured to obtain a vehicle driving trajectory, adjust the vehicle driving trajectory according to the change trend and the offset vector features, obtain a predicted driving trajectory, and perform vehicle navigation based on the predicted driving trajectory.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium comprises a stored computer program, wherein the computer program controls a device in which the computer readable storage medium is located to perform the vehicle automatic driving navigation method based on road identification recognition according to any one of claims 1 to 7 when the computer program is running.

Images