A vehicle azimuth angle acquisition method and device, electronic equipment and storage medium

Abstract

The application discloses a vehicle azimuth angle acquisition method and device, electronic equipment and a storage medium, and relates to the technical field of data processing. In the application, an original road image collected by an image collection device at a set time is acquired, edge detection is performed on the original road image, a corresponding road edge image is obtained, and then, based on road region features corresponding to the original road image and road edge features corresponding to the road edge image, a corresponding reconstructed road region is obtained, so that a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a position of the vehicle at the set time are used to obtain a vehicle azimuth angle. In this way, the accuracy and reliability of the vehicle azimuth angle acquisition are improved, and to some extent, through reconstruction of the road region, the interference of road congestion on the vehicle azimuth angle acquisition is greatly weakened.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a vehicle azimuth angle acquisition method and device, electronic equipment and a storage medium. BACKGROUND

[0002] With the continuous growth of the number of automobiles, great pressure is caused on road traffic, and traffic jams and traffic accidents are prone to occur. In order to alleviate this situation, a driving assistance system can inform a driver of a driving planning route determined according to real-time road conditions, thereby improving the safety of the driver driving the vehicle.

[0003] However, the performance of the driving assistance system depends on accurate vehicle state information, such as attitude angle, speed and acceleration, wherein the attitude angle includes roll angle, pitch angle and azimuth angle, and the control of vehicle stability is crucial.

[0004] Since the vehicle generally does not leave the road and the roll angle and pitch angle are small, in the related art, the roll angle and the pitch angle can be obtained more accurately by means of vehicle dynamics and kinematics models, but it is currently difficult to obtain an accurate azimuth angle in a relatively complex road environment due to the constraint of low cost.

[0005] For example, when acquiring the azimuth angle of the vehicle, the original road image is directly taken as the input of the azimuth angle estimation model, so as to obtain the angle difference between the driving direction of the vehicle and the road direction according to the selected reference in the original road image, and the feature extraction of the convolutional neural network (CNN) and the nonlinear regression of the least square-support vector machine (LS-SVM), and then determine the azimuth angle of the vehicle according to the obtained angle difference and the digital map containing the road direction.

[0006] It can be seen that, by using the above-mentioned vehicle azimuth angle acquisition method, the selected reference may be unreliable or incomplete, thereby resulting in low accuracy of the angle difference between the driving direction of the vehicle and the road direction, and further unable to guarantee the accuracy of the acquired azimuth angle.

[0007] Therefore, by using the above-mentioned method, the accuracy of the vehicle azimuth angle acquisition is low. SUMMARY

[0008] The embodiments of the present application provide a vehicle azimuth angle acquisition method and device, electronic equipment and a storage medium, to improve the accuracy of the vehicle azimuth angle acquisition.

[0009] In a first aspect, the embodiments of the present application provide a vehicle azimuth angle acquisition method, the method comprising:

[0010] acquiring an original road image collected by an image collection device at a set time, and performing edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image comprises an original road region that satisfies a preset azimuth angle estimation reliability condition;

[0011] performing region feature extraction on the original road image to obtain road region features of the original road region, and performing edge feature extraction on the road edge image to obtain road edge features of the original road region;

[0012] reconstructing the original road region based on the road region features and the road edge features to obtain a corresponding reconstructed road region; wherein the reconstructed road region satisfies a preset road region unocclusion condition;

[0013] obtaining a vehicle azimuth angle of the vehicle based on a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a position of the vehicle at the set time.

[0014] In a second aspect, the embodiments of the present application also provide a vehicle azimuth angle acquisition device, the device comprising:

[0015] an acquisition module configured to acquire an original road image collected by an image collection device at a set time, and perform edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image comprises an original road region that satisfies a preset azimuth angle estimation reliability condition;

[0016] an extraction module configured to perform region feature extraction on the original road image to obtain road region features of the original road region, and perform edge feature extraction on the road edge image to obtain road edge features of the original road region;

[0017] a reconstruction module configured to reconstruct the original road region based on the road region features and the road edge features to obtain a corresponding reconstructed road region; wherein the reconstructed road region satisfies a preset road region unocclusion condition;

[0018] a determination module configured to obtain a vehicle azimuth angle of the vehicle based on a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a position of the vehicle at the set time.

[0019] In an optional embodiment, when the extraction module performs region feature extraction on the original road image to obtain road region features of the original road region, the extraction module is specifically configured to:

[0020] The original road image is subjected to multi-scale feature extraction, and sub-road region features of the original road region corresponding to each feature extraction channel are obtained; each feature extraction channel corresponds to a scale feature extraction;

[0021] Based on the obtained sub-road region features, a road region feature of the original road region is obtained.

[0022] In an optional implementation, when the road edge image is subjected to edge feature extraction to obtain a road edge feature of the original road region, the extraction module is specifically configured to:

[0023] The road edge image is subjected to single-scale feature extraction to obtain an edge feature extraction result corresponding to the corresponding feature extraction channel;

[0024] The edge feature extraction result is taken as the road edge feature of the original road region.

[0025] In an optional implementation, when the original road region is reconstructed based on the road region feature and the road edge feature to obtain a corresponding reconstructed road region, the reconstruction module is specifically configured to:

[0026] The road region feature and the road edge feature are subjected to cascade feature fusion to obtain a corresponding region fusion feature;

[0027] The original road region is reconstructed based on the region range of the original road region in the road image and the region fusion feature to obtain a corresponding reconstructed road image;

[0028] From the reconstructed road image, a reconstructed road region corresponding to the original road region is determined.

[0029] In an optional implementation, when the vehicle azimuth angle of the vehicle is obtained based on the vehicle deflection angle corresponding to the reconstructed road region and the road direction information of the position of the vehicle at the set time, the determination module is specifically configured to:

[0030] Based on the device parameter set of the image acquisition device, the reconstructed road region is subjected to inverse perspective transformation to obtain a target road region satisfying a preset region distribution change condition; the target road region has a different observation angle from the reconstructed road region;

[0031] Based on the vehicle deflection angle associated with the region distribution feature of the target road region and the road direction angle contained in the road direction information, a vehicle azimuth angle of the vehicle is obtained.

[0032] In an optional implementation, before the vehicle azimuth angle of the vehicle is obtained based on the vehicle deflection angle associated with the region distribution feature of the target road region and the road direction angle contained in the road direction information, the determination module is further configured to:

[0033] For each sample road image, the following operations are performed respectively:

[0034] A sample road region corresponding to a sample road image is obtained, as well as a corresponding sample azimuth angle and a sample direction angle;

[0035] Based on the sample azimuth angle and the sample direction angle, a sample deflection angle corresponding to the sample road image is determined;

[0036] The region distribution feature of the sample road region is associated with the sample deflection angle.

[0037] In an optional implementation, when the vehicle deflection angle is associated based on the region distribution feature of the target road region, and the road direction angle is obtained from the road direction information, the determining module is specifically configured to:

[0038] From the road direction information, a plurality of node information of a plurality of road nodes respectively satisfying a preset distance interval condition at a position of the vehicle at a set time is obtained, wherein each node information is used to indicate a coordinate position of the corresponding road node in a standard reference coordinate system;

[0039] Based on the obtained coordinate positions respectively included in the plurality of node information, a plurality of road section slopes corresponding to the plurality of road nodes are obtained, and based on the road section slopes, the road direction angle is obtained, wherein each road section slope represents a road section inclination degree between adjacent two road nodes in each road node;

[0040] Based on the vehicle deflection angle and the road direction angle, the vehicle azimuth angle of the vehicle is obtained.

[0041] In a third aspect, an electronic device is provided, which includes a processor and a memory, wherein the memory stores program code, and when the program code is executed by the processor, the processor executes the steps of the vehicle azimuth angle obtaining method of the first aspect.

[0042] In a fourth aspect, a computer readable storage medium is provided, which includes program code, and when the program code is run on an electronic device, the program code is used to make the electronic device execute the steps of the vehicle azimuth angle obtaining method of the first aspect.

[0043] In a fifth aspect, a computer program product is provided, which, when invoked by a computer, makes the computer execute the steps of the vehicle azimuth angle obtaining method of the first aspect.

[0044] The application has the following advantages:

[0045] In the vehicle azimuth angle acquisition method provided in the embodiments of the present application, an original road image collected by an image collection device at a set time is acquired, and edge detection is performed on the original road image to obtain a corresponding road edge image; then, region feature extraction is performed on the original road image to obtain road region features of the original road region, and edge feature extraction is performed on the road edge image to obtain road edge features of the original road region; further, based on the road region features and the road edge features, the original road region is reconstructed to obtain a corresponding reconstructed road region; finally, based on a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a position of the vehicle at the set time, a vehicle azimuth angle of the vehicle is obtained.

[0046] In this way, a highly reliable road region is selected as an estimation reference of the vehicle deflection angle, and the road edge image is introduced through edge detection, and the road region features corresponding to the original road image and the road edge features corresponding to the road edge image are fused to obtain reconstructed region features, that is, a complete road region (reference) is acquired, and then the vehicle azimuth angle of the vehicle is obtained according to the vehicle deflection angle corresponding to the reconstructed road region and the road direction information of the vehicle, which effectively avoids the technical drawbacks in the related art that the accuracy of the angle difference between the vehicle driving direction and the road direction is low due to the unreliable or incomplete reference, and thus the accuracy of the acquired azimuth angle cannot be guaranteed, and therefore the accuracy and reliability of the vehicle azimuth angle acquisition are improved, and to some extent, the interference of road congestion on the vehicle azimuth angle acquisition is greatly weakened through the reconstruction of the road region.

[0047] In addition, other features and advantages of the present application will be described in the subsequent description, and some will become apparent from the description, or will be understood by those skilled in the art through implementation of the present application. The purposes and other advantages of the present application can be achieved and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor based on these drawings. In the drawings:

[0049] Figure 1 An optional schematic diagram of a system architecture applicable to the embodiments of the present application;

[0050] Figure 2 A structural schematic diagram of a vehicle provided by the embodiments of the present application;

[0051] Figure 3 An implementation flowchart of a vehicle azimuth angle acquisition method provided for an embodiment of the present application is shown in FIG. 1;

[0052] Figure 4 A road schematic diagram of a road area congestion provided for an embodiment of the present application is shown in FIG. 2;

[0053] Figure 5 A structure schematic diagram of a multi-scale convolution kernel provided for an embodiment of the present application is shown in FIG. 3;

[0054] Figure 6 A structure schematic diagram of a multi-core CNN road area feature extraction provided for an embodiment of the present application is shown in FIG. 4;

[0055] Figure 7 A structure schematic diagram of a light CNN road edge feature extraction provided for an embodiment of the present application is shown in FIG. 5;

[0056] Figure 8 A decoder structure schematic diagram for road area reconstruction provided for an embodiment of the present application is shown in FIG. 6;

[0057] Figure 9 A method implementation flowchart for determining a vehicle azimuth angle provided for an embodiment of the present application is shown in FIG. 7;

[0058] Figure 10 A structure schematic diagram of a vehicle deflection angle estimation model based on dilated convolution provided for an embodiment of the present application is shown in FIG. 8;

[0059] Figure 11 A relationship schematic diagram among a vehicle deflection angle, a road direction angle, and a vehicle azimuth angle provided for an embodiment of the present application is shown in FIG. 9;

[0060] Figure 12 A logic schematic diagram of a vehicle azimuth angle acquisition provided for an embodiment of the present application is shown in FIG. 10;

[0061] Figure 13 A structure schematic diagram of a vehicle azimuth angle acquisition apparatus provided for an embodiment of the present application is shown in FIG. 11;

[0062] Figure 14 A structure schematic diagram of an electronic device provided for an embodiment of the present application is shown in FIG. 12. DETAILED DESCRIPTION

[0063] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments described in the present application document, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the technical solutions of the present application.

[0064] It should be noted that in the description of the present application, "multiple" is understood as "at least two". The association relationship of the associated objects is described, which means that there can be three relationships, for example, A and / or B can mean that A exists alone, A and B exist together, and B exists alone. A and B are connected, which means that A and B are directly connected and A and B are connected through C. In addition, in the description of the present application, "first", "second", and the like are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can it be understood as indicating or implying order.

[0065] In addition, in the technical solutions of the present application, the collection, transmission, use and the like of data all meet the requirements of relevant national laws and regulations.

[0066] In order to facilitate those skilled in the art to understand, first, some nouns and terms related to the embodiments of the present application are briefly described and explained as follows:

[0067] (1) Electronic Stability Control (ESC): also known as electronic stability program or dynamic stability control, which can improve the stability of the vehicle by detecting and reducing the loss of traction (slip), when ESC detects loss of steering control, it can automatically apply brakes to help the driver "turn" the vehicle to the desired direction of travel, and the brakes will be automatically applied to the wheels, for example, the outer front wheels to deal with excessive steering or the inner rear wheels to prevent understeering; in addition, some ESC systems will also reduce engine power until control is regained.

[0068] (2) Global Navigation Satellite System (GNSS): also known as Global Satellite Navigation System, is an air-based radio navigation positioning system that can provide 3D coordinates, speed and time information for users on the earth's surface or in near-earth space at any location, which includes one or more satellite constellations and the enhanced system required to support a particular work.

[0069] (3) Inertial Measurement Unit (IMU): is a device that measures the attitude angle (or angular velocity) and acceleration of an object. Generally, an IMU contains three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the acceleration signal of the object in the independent three-axis coordinate system of the carrier, while the gyroscope detects the angular velocity signal of the carrier relative to the navigation coordinate system. The angular velocity and acceleration of the object in three-dimensional space are measured, and the attitude of the object is calculated based on this.

[0070] (4) Micro Electro Mechanical System (MEMS): also known as micro-electro-mechanical system, micro-system, micro-mechanical system, etc. It refers to a high-tech device with a size of a few millimeters or even smaller, with internal structures generally in the order of microns or even nanometers, which is an independent intelligent system.

[0071] (5) Visual Odometry (VO): refers to the process of estimating the pose of an object carrying a camera by detecting the image changes caused by motion. Its input is image, video sequence, and its output is camera motion trajectory, i.e. object motion trajectory.

[0072] (6) Backbone: refers to the backbone of the network, which usually has higher bandwidth and higher reliability. In deep learning networks, it is often used to extract feature information for the use of subsequent networks.

[0073] (7) Inertial Navigation System (INS): also known as inertial system or inertial navigation, is a system that uses gyroscopes and accelerometers installed on a carrier to determine the position of the carrier. Through the measurement data of gyroscopes and accelerometers, the motion of the carrier in the inertial reference coordinate system can be determined, and the position of the carrier in the inertial reference coordinate system can also be calculated.

[0074] (8) B-spline algorithm: refers to the connection of the entire curve with a piece-by-piece curve, i.e. using a segmented continuous multi-segment generation.

[0075] (9) Autoregressive Integrated Moving Average Model (ARIMA) algorithm: the essence is to find out the rules of data with trends, seasonality, and business scenario periodicity, and gradually extract the regular information from the data. The final data is left without rules, or called noise, which is ideally white noise.

[0076] Further, based on the above-mentioned noun and related term explanation, the design idea of the embodiments of the present application is briefly introduced as follows:

[0077] The continuous growth of the number of cars has caused great pressure on road traffic, and is prone to cause serious traffic congestion and frequent traffic accidents. Therefore, developing advanced driving assistance systems (such as ESC, lane keeping system, collision warning system, etc.) is one of the effective ways to solve this problem.

[0078] However, the functions of these vehicle auxiliary driving systems depend on accurate vehicle state information, including attitude angle, speed, acceleration and other information. Among them, the attitude angle is very important for vehicle stability control, which includes roll angle, pitch angle and azimuth angle.

[0079] Since the wheels generally do not leave the road, the roll angle and pitch angle are small, and with the help of vehicle dynamics and kinematics model, the existing roll angle estimation method and pitch angle estimation method can obtain relatively accurate estimation results. Compared with the existing azimuth angle estimation method, it is difficult to obtain accurate estimation results in complex urban environment due to the constraints of low cost and high reliability.

[0080] In related technologies, according to the different ways of obtaining the azimuth angle of the vehicle, the existing azimuth angle estimation methods can be roughly divided into four categories: measurement-based method, yaw rate integral-based method, VO-based method and learning-based method.

[0081] 1. Measurement-based method: In the measurement-based method, digital compass and GNSS are the most commonly used sensors. The digital compass directly obtains the azimuth angle by measuring the earth's magnetic field, but this method has two shortcomings: the measurement value of the digital compass is a pseudo-azimuth angle, which is actually relative to the geomagnetic north rather than the true north, and in complex urban environment, the digital compass is easily disturbed by local magnetic field, and the measurement value is unstable and has poor dynamic response; GNSS can provide position, speed and heading angle information. When the vehicle does not have obvious slip motion, the heading angle is approximately equal to the azimuth angle. However, GNSS is easily affected by high-rise buildings, trees and other obstructions on both sides of the road, and the reliability of the heading angle cannot be guaranteed.

[0082] 2. Yaw rate integral-based method: In the yaw rate integral-based method, the vehicle's azimuth angle information at each time is obtained by integrating the yaw rate measured by the IMU and adding the initial azimuth angle. Due to the cost constraints, most existing vehicle-mounted IMUs are manufactured based on MEMS technology, which has very low cost, but the IMU measurement value contains complex noise characteristics, and the existing denoising algorithm cannot effectively suppress this noise. After integration, the adverse effects of noise are further amplified, making it difficult for this method to obtain accurate and reliable estimation values.

[0083] Therefore, most of the existing methods adopt a multi-sensor fusion strategy to solve this problem, and the most common method is to fuse MEMS-IMU and GNSS through a data fusion algorithm (such as Kalman filtering) to play the complementary advantages of the two, however, it is difficult for GNSS and MEMS-IMU combination to obtain accurate heading angle measurement in long-time satellite failure environment.

[0084] 3. VO-based method: In the VO-based method, the pose angle information (including the azimuth angle) of the vehicle is obtained by extracting feature points from adjacent video frames and then performing feature matching. Although many VO methods have achieved relatively good accuracy, their reliability is difficult to guarantee in the conditions of fast vehicle movement, high dynamic scene and low texture area, and for the VO-based method, the premise for providing effective azimuth angle is to detect a sufficient number of feature points, which means that when the vehicle drives to a static feature point with few or low texture road sections, the performance of these methods will be significantly reduced.

[0085] 4. Learning-based method: In the learning-based method, features related to the azimuth angle are mined from the image, and then traditional machine learning or deep learning models are used to classify or regress these features, and finally the azimuth angle estimation is realized with the aid of the digital map. These data-driven methods are not easily disturbed by complex environments and have stronger environmental adaptability compared with the above three methods, however, the existing learning-based azimuth angle estimation method does not consider the influence of the change of driving scene in the input image, such as frequent road congestion, which makes it difficult to achieve high reliable estimation.

[0086] For example, when obtaining the azimuth angle of the vehicle, the original road image is directly taken as the input of the azimuth angle estimation model, so as to obtain the angle difference between the driving direction of the vehicle and the road direction according to the selected reference in the original road image, and combine the feature extraction of CNN and the nonlinear regression of LS-SVM, and then determine the azimuth angle of the vehicle according to the obtained angle difference and the digital map containing the road direction.

[0087] As can be seen, in the related art, the above four azimuth angle estimation methods cannot guarantee that the obtained vehicle azimuth angle has high accuracy, for example, in the learning-based method, the selected reference may be unreliable or incomplete, which leads to low accuracy of the angle difference between the driving direction of the vehicle and the road direction, and further cannot guarantee the accuracy of the obtained azimuth angle.

[0088] Therefore, in order to improve the accuracy and reliability of the vehicle azimuth angle, a highly reliable road area is selected as a reference for vehicle deflection angle estimation, and road area reconstruction and vehicle deflection angle estimation are performed to obtain the corresponding vehicle azimuth angle, which specifically includes: acquiring an original road image collected by an image collection device at a set time, and performing edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image includes: an original road area that meets a preset azimuth angle estimation reliability condition; then, the original road image is subjected to region feature extraction to obtain the road region feature of the original road region, and the road edge image is subjected to edge feature extraction to obtain the road edge feature of the original road region; further, based on the road region feature and the road edge feature, the original road region is reconstructed to obtain a corresponding reconstructed road region; wherein the reconstructed road region meets a preset road region unobstructed condition; finally, based on the vehicle deflection angle corresponding to the reconstructed road region and the road direction information of the position of the vehicle at the set time, the vehicle azimuth angle of the vehicle is obtained.

[0089] It should be noted that based on the above-mentioned vehicle azimuth angle acquisition method, the accuracy and reliability of the vehicle azimuth angle estimation in a complex environment (such as a city road) are obviously improved.

[0090] In particular, the preferred embodiments of the present application are described below in conjunction with the accompanying drawings of the specification, it should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments and features of the embodiments can be combined with each other without conflict.

[0091] Referring to Figure 1 As shown in the figure, a system architecture provided by the embodiment of the present application includes: a target terminal 101 and a server 102. The target terminal 101 and the server 102 can exchange information through a communication network, wherein the communication network can adopt a communication mode including: a wireless communication mode and a wired communication mode.

[0092] For example, the target terminal 101 can access the network through a cellular mobile communication technology and communicate with the server 1202, wherein the cellular mobile communication technology includes, for example, a fifth generation mobile communication (5th Generation Mobile Networks, 5G) technology.

[0093] Optionally, the target terminal 101 can access the network through a short-range wireless communication mode and communicate with the server 102, wherein the short-range wireless communication mode includes, for example, a wireless fidelity (Wireless Fidelity, Wi-Fi) technology.

[0094] The number of communication devices involved in the system architecture described above is not limited by the embodiments of the present application. For example, there can be more target terminals, or no target terminal, or other network devices such as Figure 1 As shown, only the target terminal 101 and the server 102 are taken as examples for description, and the following briefly introduces each device and its respective function.

[0095] The target terminal 101 is a device that can provide voice and / or data connectivity to a user, and can be a device supporting wired and / or wireless connection mode.

[0096] For example, the target terminal 101 includes but is not limited to a mobile phone, a tablet computer, a notebook computer, a palm computer, a mobile Internet device (MID), a wearable device, a virtual reality (VR) device, an augmented reality (AR) device, a wireless terminal device in industrial control, a wireless terminal device in unmanned driving, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, or a wireless terminal device in smart home, etc.

[0097] In addition, the target terminal 101 can be installed with a related client, and the client can be software such as an application (APP), a browser, a short video software, etc., or a webpage, an applet, etc. In the embodiments of the present application, the target terminal 101 can send the original road image to the server 102 to perform subsequent vehicle azimuth angle acquisition through the client related to the vehicle azimuth angle acquisition. It should be noted that Figure 2 As shown, in the embodiments of the present application, the target terminal 101 can be a vehicle, and the vehicle is loaded with an image acquisition device.

[0098] The server 102 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, etc. basic cloud computing services.

[0099] It is worth mentioning that, in the embodiment of the present application, the server 102 is configured to acquire an original road image collected by an image collection device at a set time, and perform edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image comprises: an original road region satisfying a preset azimuth angle estimation reliability condition; then, region feature extraction is performed on the original road image to obtain road region features of the original road region, and edge feature extraction is performed on the road edge image to obtain road edge features of the original road region; further, based on the road region features and the road edge features, the original road region is reconstructed to obtain a corresponding reconstructed road region; wherein the reconstructed road region satisfies a preset road region unocclusion condition; finally, based on a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a location of the vehicle at the set time, a vehicle azimuth angle of the vehicle is obtained.

[0100] The vehicle azimuth angle acquisition method provided by the exemplary embodiments of the present application will be described below in conjunction with the above system architecture and in reference to the accompanying drawings. It should be noted that the above system architecture is only shown for the purpose of facilitating the understanding of the spirit and principles of the present application, and the embodiments of the present application are not limited in this respect.

[0101] Referring to Figure 3 FIG. 3 shows an implementation flowchart of a vehicle azimuth angle acquisition method according to an embodiment of the present application, and the execution subject is taken as an example of a server. The specific implementation process of the method is as follows:

[0102] S301: Acquire an original road image collected by an image collection device at a set time, and perform edge detection on the original road image to obtain a corresponding road edge image.

[0103] It should be noted that, in the embodiment of the present application, no specific limitation is imposed on the color mode of the original road image collected by the image collection device, i.e., it can be a Red Green Blue (RGB) mode, a Cyan Magenta Yellow Black (CMYK) mode, or other color modes, and only the original road image in the RGB mode is taken as an example.

[0104] The original road image comprises: an original road region satisfying a preset azimuth angle estimation reliability condition, and the preset azimuth angle estimation reliability condition represents that the selected reference object in the original road image has high redundancy.

[0105] It is worth noting that selecting highly redundant reference points for estimating vehicle deflection angles is a crucial guarantee for achieving highly reliable estimations. Since vehicles travel on roads, the road area as a reference point is almost never missing. Compared to reference points such as lane lines, traffic lights, and traffic signs, the former has high redundancy, which helps improve the reliability and accuracy of vehicle deflection angle estimation.

[0106] However, in complex environments, refer to Figure 4 As shown, road congestion caused by traffic participants (such as motor vehicles and pedestrians) is unavoidable, resulting in irregular shapes of drivable road areas, which become narrow in some scenarios, i.e., the selected reference is incomplete (obscured). If such road areas are used to estimate vehicle deflection angles, it will lead to large errors. Therefore, if the road areas can be reconstructed, the above problems will be effectively solved.

[0107] S302: Extract regional features from the original road image to obtain the road region features of the original road region, and extract edge features from the road edge image to obtain the road edge features of the original road region.

[0108] In one alternative implementation, when performing step S302, it is considered that the shape of the drivable area of ​​the road changes differently under different levels of road congestion. For example, the road area is larger in the case of slight congestion, while the road area is smaller in the case of severe congestion. That is, the road area has multiple scales. The receptive field of a CNN with a single-scale convolutional kernel (e.g., the common 3×3) has a limited coverage range, making it difficult to effectively extract the road area features.

[0109] Therefore, to solve the above problems, after obtaining the original road image, the server can perform multi-scale feature extraction on the original road image to obtain the sub-road region features of the original road region corresponding to each feature extraction channel. Each feature extraction channel corresponds to a scale feature extraction. Based on the obtained sub-road region features, the road region features of the original road region are obtained. By stacking multi-scale convolution kernels (referred to as multi-kernel), receptive fields of multiple scales are formed, so that multi-kernel CNN can capture richer road region features from the road image.

[0110] For example, see Figure 5 As shown, the server uses convolution kernels of three scales: 1×1, 3×3, and 5×5 to extract features from the original road image at multiple scales. In order to prevent the number of parameters from being too large, a 1×1 convolution is added before the 3×3 and 5×5 convolutions to achieve dimensionality reduction of the data. In addition, pooling layers are not used in the above 3-core CNN.

[0111] It should be noted that when designing the structure of the multi-core CNN, the encoder architecture of a well-known Backbone network can be directly migrated, the single-scale convolution kernel therein is replaced with a multi-core structure, and then the best network structure is determined through model selection, refer to Figure 6 As shown in FIG. 6, which is a structural schematic diagram for road region feature extraction based on a multi-core CNN provided by an embodiment of the present application.

[0112] In an optional implementation, when step S302 is performed, in view of the fact that the edges of each target occupy fewer pixels in the road edge image, the receptive field of the CNN used for extracting road edge features should not be too large, otherwise it is difficult to extract fine road edge features; in addition, the road edge image is usually a single-channel grayscale image, and the information complexity of the former is lower than that of the original road image which is usually a three-channel (such as RGB) image.

[0113] Therefore, considering these two factors, when designing the structure of the road edge feature extraction CNN, it should be considered to use fewer convolution kernels and the number of convolution layers should not be too large to avoid a too large receptive field, so as to meet the lightweight requirement; at the same time, considering that the edges of each target do not change in scale, a single-scale convolution kernel with a fixed size (such as 3x3) can be used for single-scale feature extraction.

[0114] Specifically, as shown in FIG. 7, the server first obtains the road edge image, and then performs single-scale feature extraction on the road edge image to obtain the edge feature extraction result corresponding to the feature extraction channel, so as to take the edge feature extraction result as the road edge feature of the original road region. Figure 7 Figure 7 As shown in FIG. 8, the standard convolution represents the use of a single-scale convolution kernel.

[0115] S303: reconstructing the original road region based on the road region feature and the road edge feature to obtain a corresponding reconstructed road region.

[0116] The reconstructed road region meets a preset road region unocclusion condition, that is, the reconstructed road region is not occluded, or the acquired road region (i.e., the selected reference) is complete in the acquisition angle of the image acquisition device.

[0117] ​In one optional implementation, when executing step S303, after obtaining the road region features and road edge features, in order to decouple the accurate complete road region (i.e., reconstructed road region) from the richer road features (i.e., road region features and road edge features), the server performs cascaded feature fusion on the road region features and road edge features to obtain corresponding region fusion features. Then, based on the regional range of the original road region in the road image and the region fusion features, the original road region is reconstructed to obtain the corresponding reconstructed road image. Thus, the reconstructed road region corresponding to the original road region is determined from the reconstructed road image.

[0118] It should be noted that during the cascaded feature fusion of road region features and road edge features, the width and height of the road region feature map and the road edge feature map remain unchanged, and the number of channels is superimposed. Therefore, it is necessary to ensure that the downsampling times of the multi-core CNN and the lightweight CNN are the same. Next, the decoder network is constructed to upsample the fused region features, restoring the region fused feature map to the size of the road region feature map or the size of the road edge feature map (i.e., the area range of the original road region in the original road image), thereby realizing the complete reconstruction of the road region, that is, obtaining the reconstructed road image, and then determining the reconstructed road region corresponding to the original road region from the reconstructed road image.

[0119] For example, see Figure 8 As shown, in the decoder network, the server performs progressive upsampling through deconvolution operations. The number of upsampling operations is the same as the number of downsampling operations in a multi-core CNN. In addition, an appropriate stride needs to be selected to avoid the loss of spatial location information in the road area due to an excessively large stride.

[0120] It should also be noted that, in order to obtain the ground truth of the reconstructed road image, the server can use an image annotation tool to label the part within the left and right boundaries of the road as the reconstructed road region. In addition, after the training data is prepared, the model selection can be performed on the road region reconstruction module to determine the optimal structure of the neural network involved in the module.

[0121] S304: Based on the vehicle deflection angle corresponding to the reconstructed road area and the road direction information of the vehicle's position at a set time, obtain the vehicle's azimuth angle.

[0122] In one optional implementation, during step S304, after obtaining the reconstructed road area (i.e., after the road area reconstruction is complete), the server can determine the vehicle yaw angle corresponding to the obtained reconstructed road area based on the correspondence between the reconstructed road area and the vehicle yaw angle. Then, combined with the road direction information of the vehicle's position at a set time, the server obtains the vehicle's azimuth angle. (See [link to relevant documentation]). Figure 9 As shown, the specific implementation process of this method is as follows:

[0123] S901: based on the device parameter set of the image acquisition device, inversely perspective transforming the reconstructed road region to obtain a target road region satisfying a preset regional distribution change condition.

[0124] The device parameter set contains related parameters of the inverse perspective transformation, and the target road region is different from the observation angle of the reconstructed road region.

[0125] For example, when the vehicle deflection angle of the vehicle changes, the regional distribution of the reconstructed road region (i.e., the complete road region) in the reconstructed road image does not change significantly due to the perspective projection of the image acquisition device. However, this change is very obvious in the bird's eye view, which helps to obtain an accurate vehicle deflection angle. Therefore, the server first inversely perspective transforms the reconstructed road region, and then extracts the regional distribution features of the target road region from the bird's eye view, and then estimates the vehicle deflection angle according to the obtained regional distribution features.

[0126] It should be noted that in order to obtain the related parameters of the inverse perspective transformation, i.e., the device parameter set, image acquisition device calibration is required. For example, the Zhang Zhengyou chessboard calibration method can be used to obtain the intrinsic parameters f x , f y , c x and c y of the image acquisition device, where f x and f y represent the projections of the focal length of the image acquisition device along the x and y axes, respectively, and c x and c y represent the offsets of the optical axis of the image acquisition device on the x and y axes, respectively. In addition, the extrinsic parameters of the image acquisition device involved in the inverse perspective transformation mainly include the installation height h and the pitch angle Finally, after obtaining the above parameters (intrinsic and extrinsic parameters) of the image acquisition device, the inverse perspective transformation can be realized according to the resolution of the original road image collected by the image acquisition device.

[0127] S902: based on the vehicle deflection angle associated with the regional distribution features of the target road region, and the road direction angle contained in the road direction information, obtaining the vehicle azimuth angle of the vehicle.

[0128] In an optional implementation, before step S902 is performed, the server needs to pre-establish a correspondence between the regional distribution features and the vehicle deflection angles, so as to ensure that, after the regional distribution features of the target road region are obtained, the vehicle azimuth angle of the vehicle is obtained based on the vehicle deflection angle associated with the regional distribution features of the target road region and the road direction angle contained in the road direction information; wherein, the correspondence between the regional distribution features and the vehicle deflection angles can be obtained in the following manner: for each sample road image, the following operations are performed respectively: obtaining a sample road region corresponding to a sample road image, and a corresponding sample azimuth angle and sample direction angle, and then determining a sample deflection angle corresponding to the sample road image based on the sample azimuth angle and the sample direction angle, so as to associate the regional distribution features of the sample road region with the sample deflection angle; it should be noted that the sample deflection angle is the difference between the sample azimuth angle and the sample direction angle.

[0129] For example, considering that the target road region after inverse perspective transformation belongs to a large-scale target, therefore, the server can construct a CNN-based vehicle deflection angle estimation model that needs to have a large receptive field, so as to capture the global features (i.e. regional distribution features) of the target road region from the target road region. Therefore, when constructing the vehicle deflection angle estimation model, dilated convolution can be introduced, that is, 0 elements are introduced into the convolution kernel to achieve the purpose of expanding the receptive field without introducing additional parameters, and at the same time, in order to avoid the checkerboard effect of dilated convolution, the dilated rate of each convolution layer should obey a sawtooth distribution.

[0130] Referring to Figure 10 As shown in the figure, the vehicle deflection angle estimation model based on dilated convolution is essentially a regression model. Since the input of the model is a single-channel grayscale image, which contains a target road region and has a low information complexity, the vehicle deflection angle model uses fewer convolution layers, and the output layer is a fully connected layer containing only one neuron, which maps the regional distribution features extracted by the convolution layer to the vehicle deflection angle. In addition, the best network structure can also be determined through model selection.

[0131] Specifically, when the server performs model selection of the best network structure, it needs to construct a data set of the vehicle deflection angle estimation model, and the input of the data set can be an image after inverse perspective transformation of a complete road region manually labeled, i.e. each sample road image, and the output is the vehicle deflection angle; wherein, since the data set usually does not contain data related to the vehicle deflection angle, in order to obtain the true value of the vehicle deflection angle, other sensing means need to be used.

[0132] For example, first, the heading angle of the vehicle is obtained by a high-precision integrated inertial navigation system, which internally embeds a high-precision INS and GNSS, and can still provide accurate and reliable heading angle measurement values within a period of time when satellite signals fail, and when the slip angle of the vehicle is small, the heading angle of the vehicle is approximately equal to the azimuth angle of the vehicle; then, a digital map needs to be constructed to obtain road direction information, i.e., the included angle between the road direction and true north; finally, the heading angle (i.e., the sample azimuth angle) measured by the high-precision integrated inertial navigation system is subtracted from the road direction information (i.e., the sample direction angle) provided by the digital map, and the true value of the vehicle deflection angle (i.e., the sample deflection angle) can be obtained.

[0133] Therefore, the server trains the vehicle deflection angle estimation model according to each sample road image, the area distribution characteristics of the respective sample road region, and the sample deflection angle, and thus the corresponding relationship between the area distribution characteristics and the sample deflection angle can be established.

[0134] Based on the above method steps, after the server obtains the corresponding relationship between the area distribution characteristics and the vehicle deflection angle, the vehicle azimuth angle of the vehicle can be obtained based on the vehicle deflection angle associated with the area distribution characteristics of the target road region and the road direction angle contained in the road direction information.

[0135] In an optional implementation, after the server obtains the area distribution characteristics of the target road region, if there is no sample road region with the same area distribution characteristics as the target road region in the above corresponding relationship, the server can filter out a sample road region that satisfies a preset area distribution characteristic similarity condition from the sample road region corresponding to each sample road image, and the vehicle deflection angle corresponding to the area distribution characteristics of the obtained sample road region is taken as the vehicle deflection angle associated with the area distribution characteristics of the target road region.

[0136] For example, the above preset area distribution characteristic similarity condition can be that the sample road region with the highest or maximum similarity to the area distribution characteristics of the target road region in the sample road region corresponding to each sample road image.

[0137] In one optional implementation, during step S902, after determining the vehicle deflection angle associated with the regional distribution characteristics of the target road area, the server can obtain the node information of multiple road nodes that meet the preset distance interval conditions relative to the vehicle's position at a set time from the road direction information. Each node information indicates the coordinate position of the corresponding road node in the standard reference coordinate system. Then, based on the coordinate positions contained in the obtained node information, the slope of each road segment corresponding to the multiple road nodes is obtained, and the road direction angle is obtained based on the slope of each road segment. Each road segment slope represents the degree of inclination between two adjacent road nodes. Finally, the vehicle azimuth angle is obtained based on the vehicle deflection angle and the road direction angle.

[0138] For example, see Figure 11 As shown, the relationship between vehicle deflection angle, road direction angle, and vehicle azimuth angle reveals that building a digital map containing road direction angles (i.e., road direction information) is a prerequisite for calculating vehicle azimuth angles. The creation of a digital map generally involves three steps: data acquisition, data processing, and road modeling. First, in the data acquisition stage, the longitude, latitude, and altitude information of the road centerline (i.e., road node information) is collected using a high-precision integrated inertial navigation system. Second, in the data processing stage, coordinate transformation is performed, converting the acquired road centerline position information from the geocentric fixed coordinate system (latitude and longitude) to a Cartesian coordinate system (i.e., the standard reference coordinate system). Simultaneously, considering that the longitude, latitude, and altitude data collected by the integrated inertial navigation system may jump when GNSS fails, a data smoothing algorithm can be used to correct the original data. Finally, for the most crucial road modeling, for convenience, the B-spline algorithm can be used to select road nodes from the processed data. Then, the AKIMA algorithm is used to calculate the node slope of the road nodes, where the formula for calculating the node slope is:

[0139]

[0140] Where i = 1, 2, ..., n, n represents the number of road nodes, λ i Let s represent the slope of the i-th road node. i This represents the slope of a road segment between two adjacent road nodes, and the specific formula for calculating the road segment slope is as follows:

[0141]

[0142] Where i = 1, 2, ..., n-1, n represents the number of road nodes, s j The slope x of the j-th road node represents the node gradient. j and y jrespectively represent the horizontal and vertical coordinates of the jth road node in a planar rectangular coordinate system (i.e., a standard reference coordinate system), and the direction information (i.e., the road direction angle) of the road node can be obtained by an inverse tangent operation according to the slope of the road node, wherein the calculation formula of the road direction angle is specifically as follows:

[0143]

[0144] wherein i = 1, 2,..., n, n represents the number of road nodes, γ i represents the road direction angle corresponding to the ith road node.

[0145] Therefore, the server can obtain the road direction angle γ i and store the obtained road direction angle γ i as a geometric attribute in the digital map.

[0146] In addition, considering that the distance between adjacent road nodes is small, the road direction closest to the target node (the position of the vehicle at a set time) can be obtained by map matching, and the road direction of the road node is taken as the road direction of the current position, and then it is added to the vehicle deflection angle, i.e., the vehicle azimuth angle can be obtained, wherein the calculation formula of the vehicle azimuth angle can be represented as follows:

[0147]

[0148] wherein represents the vehicle azimuth angle of the vehicle at the current position, δ represents the vehicle deflection angle of the vehicle at the current position, and λ represents the road direction angle of the vehicle at the current position.

[0149] For example, assuming that the server obtains the vehicle deflection angle δ = 8.2° and the road direction angle λ = 37.5° when the vehicle is at a set position, it can be known from the above calculation formula of the vehicle azimuth angle that the vehicle azimuth angle of the vehicle at the set position is

[0150] Based on the above vehicle azimuth angle obtaining method steps S301-S304, refer to Figure 12As shown, it is a logic diagram of vehicle azimuth angle acquisition provided by the embodiment of the present application, which specifically proposes a vehicle azimuth angle estimation framework including but not limited to a road region reconstruction module, a vehicle deflection angle estimation module and a vehicle azimuth angle calculation module, the core of which is to estimate the vehicle deflection angle by using a highly reliable complete road region (i.e. a reconstructed road region or a target road region), and for the problem that the road boundary is difficult to accurately depict, the road edge image is introduced through edge detection, and then a road region reconstruction model based on the fusion of road region features and road edge features is proposed, and at the same time, the complete road region is inversely perspective transformed to expand the saliency of its distribution features, thereby greatly weakening the interference of road congestion and improving the accuracy and reliability of vehicle azimuth angle estimation.

[0151] Obviously, by using the above method, the environmental adaptability and reliability of the existing vehicle azimuth angle estimation method based on low-cost means are poor, in the embodiment of the present application, a multi-module cascade model is constructed to mine vehicle deflection angle information from the road region, and a digital map containing road direction information is made to assist in calculating the vehicle azimuth angle, thereby realizing accurate and reliable estimation of the vehicle azimuth angle in a complex environment (such as urban traffic).

[0152] In summary, in the vehicle azimuth angle acquisition method provided by the embodiment of the present application, the original road image collected by the image acquisition device at the set time is acquired, and the original road image is edge detected to obtain the corresponding road edge image; then, the region features of the original road image are extracted to obtain the road region features of the original road region, and the edge features of the road edge image are extracted to obtain the road edge features of the original road region; further, based on the road region features and the road edge features, the original road region is reconstructed to obtain the corresponding reconstructed road region; finally, based on the vehicle deflection angle corresponding to the reconstructed road region and the road direction information of the position of the vehicle at the set time, the vehicle azimuth angle of the vehicle is obtained.

[0153] In this way, a highly reliable road area is selected as an estimation reference of the vehicle deflection angle, and a road edge image is introduced through edge detection, and the road area features corresponding to the original road image and the road edge features corresponding to the road edge image are fused to obtain reconstructed area features, that is, a complete road area (reference) is obtained, and then the vehicle azimuth angle is obtained according to the vehicle deflection angle corresponding to the reconstructed road area and the road direction information of the vehicle, thereby effectively avoiding the technical defects that in the related art, due to the unreliability or incompleteness of the selected reference, the accuracy of the angle difference between the obtained vehicle driving direction and the road direction is low, and thus the accuracy of the obtained azimuth angle cannot be guaranteed, thereby improving the accuracy and reliability of the vehicle azimuth angle acquisition, and to a certain extent, through the reconstruction of the road area, the interference of the road congestion on the vehicle azimuth angle acquisition is greatly weakened.

[0154] Further, based on the same technical concept, the embodiment of the present application provides a vehicle azimuth angle acquisition device for implementing the above method flow of the embodiment of the present application. Referring to Figure 13 As shown in the figure, the vehicle azimuth angle acquisition device comprises an acquisition module 1301, an extraction module 1302, a reconstruction module 1303 and a determination module 1304, wherein:

[0155] The acquisition module 1301 is configured to acquire an original road image collected by an image collection device at a set time, and perform edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image comprises an original road area satisfying a preset azimuth angle estimation reliability condition;

[0156] The extraction module 1302 is configured to perform area feature extraction on the original road image to obtain road area features of the original road area, and perform edge feature extraction on the road edge image to obtain road edge features of the original road area;

[0157] The reconstruction module 1303 is configured to reconstruct the original road area based on the road area features and the road edge features to obtain a corresponding reconstructed road area; wherein the reconstructed road area satisfies a preset road area unobstruction condition;

[0158] The determination module 1304 is configured to obtain a vehicle azimuth angle of the vehicle based on a vehicle deflection angle corresponding to the reconstructed road area and road direction information of a location of the vehicle at the set time.

[0159] In an optional embodiment, when performing area feature extraction on the original road image to obtain road area features of the original road area, the extraction module 1302 is specifically configured to:

[0160] The original road image is subjected to multi-scale feature extraction, and sub-road region features of the original road region corresponding to each feature extraction channel are obtained; each feature extraction channel corresponds to a scale feature extraction;

[0161] Based on the obtained sub-road region features, a road region feature of the original road region is obtained.

[0162] In an optional implementation, when the road edge image is subjected to edge feature extraction to obtain a road edge feature of the original road region, the extraction module 1302 is specifically configured to:

[0163] The road edge image is subjected to single-scale feature extraction to obtain an edge feature extraction result corresponding to the corresponding feature extraction channel;

[0164] The edge feature extraction result is taken as the road edge feature of the original road region.

[0165] In an optional implementation, when the original road region is reconstructed based on the road region feature and the road edge feature to obtain a corresponding reconstructed road region, the reconstruction module 1303 is specifically configured to:

[0166] The road region feature and the road edge feature are subjected to cascade feature fusion to obtain a corresponding region fusion feature;

[0167] The original road region is reconstructed based on the region range of the original road region in the road image and the region fusion feature to obtain a corresponding reconstructed road image;

[0168] From the reconstructed road image, a reconstructed road region corresponding to the original road region is determined.

[0169] In an optional implementation, when the vehicle azimuth angle of the vehicle is obtained based on the vehicle deflection angle corresponding to the reconstructed road region and the road direction information of the position of the vehicle at the set time, the determination module 1304 is specifically configured to:

[0170] The reconstructed road region is subjected to inverse perspective transformation based on the device parameter set of the image acquisition device to obtain a target road region satisfying a preset region distribution change condition; the target road region has a different observation angle from the reconstructed road region;

[0171] Based on the vehicle deflection angle associated with the region distribution feature of the target road region and the road direction angle contained in the road direction information, the vehicle azimuth angle of the vehicle is obtained.

[0172] In an optional implementation, before obtaining the vehicle azimuth angle based on the vehicle deflection angle associated with the regional distribution feature of the target road region and the road direction angle contained in the road direction information, the determining module 1304 is further configured to:

[0173] For each sample road image, the following operations are performed respectively:

[0174] obtain a sample road region corresponding to a sample road image, and a corresponding sample azimuth angle and sample direction angle;

[0175] determine a sample deflection angle corresponding to the sample road image based on the sample azimuth angle and the sample direction angle;

[0176] associate the regional distribution feature of the sample road region with the sample deflection angle.

[0177] In an optional implementation, when obtaining the vehicle azimuth angle based on the vehicle deflection angle associated with the regional distribution feature of the target road region and the road direction angle contained in the road direction information, the determining module 1304 is specifically configured to:

[0178] obtain, from the road direction information, node information of a plurality of road nodes respectively satisfying a preset distance interval condition with the position of the vehicle at the set time; wherein each node information is used to indicate the coordinate position of the corresponding road node in the standard reference coordinate system;

[0179] obtain road section slopes corresponding to the plurality of road nodes based on the obtained coordinate positions respectively contained in the plurality of node information, and obtain the road direction angle based on the road section slopes; wherein each road section slope represents the inclination degree of a road section between two adjacent road nodes in each road node;

[0180] obtain the vehicle azimuth angle based on the vehicle deflection angle and the road direction angle.

[0181] Based on the same technical concept, the embodiments of the present application also provide an electronic device, which can implement the vehicle azimuth angle acquisition method process provided by the above-mentioned embodiments of the present application. In an embodiment, the electronic device can be a server, a terminal device or other electronic device. As shown in Figure 14 the electronic device can include:

[0182] at least one processor 1401 and a memory 1402 connected with the at least one processor 1401, and the specific connection medium between the processor 1401 and the memory 1402 is not limited in the embodiments of the present application, Figure 14 for example, the connection between the processor 1401 and the memory 1402 through the bus 1400. The bus 1400 is connected Figure 14The connection between the other components is shown by a thick line, and the connection mode is only illustrative and is not limited. The bus 1400 can be divided into an address bus, a data bus, a control bus, etc. For convenience of representation, Figure 14 The bus 1400 is shown by a thick line, but it does not mean that there is only one bus or only one type of bus. Alternatively, the processor 1401 can also be called a controller, and the name is not limited.

[0183] In the embodiment of the application, the memory 1402 stores instructions executable by the at least one processor 1401, and the at least one processor 1401 can execute the vehicle azimuth angle acquisition method discussed above by executing the instructions stored in the memory 1402. The processor 1401 can realize Figure 13 The functions of each module in the device shown.

[0184] The processor 1401 is the control center of the device, and can connect each part of the entire control device through various interfaces and lines, and monitor the entire device by running or executing the instructions stored in the memory 1402 and calling the data stored in the memory 1402, thereby processing data and monitoring the entire device.

[0185] In a possible design, the processor 1401 can include one or more processing units, and the processor 1401 can integrate an application processor and a modem processor, where the application processor mainly processes an operating system, a user interface, and an application program, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor can also not be integrated into the processor 1401. In some embodiments, the processor 1401 and the memory 1402 can be implemented on the same chip, and in some embodiments, they can also be implemented on independent chips respectively.

[0186] The processor 1401 can be a general-purpose processor, such as a CPU, a digital signal processor, an application-specific integrated circuit, a field programmable gate array, or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, and can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the vehicle azimuth angle acquisition method disclosed in the embodiments of the application can be directly embodied by a hardware processor for execution, or executed by a combination of hardware and software modules in the processor.

[0187] The memory 1402, as a non-volatile computer readable storage medium, can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The memory 1402 can include at least one type of storage medium, for example, can include flash memory, hard disk, multimedia card, card type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic storage, magnetic disk, optical disk, etc. The memory 1402 is any other medium capable of carrying or storing desired program code in the form of instructions or data structures and capable of being accessed by a computer, but is not limited thereto. The memory 1402 in the embodiments of the present application can also be a circuit or any other device capable of realizing a storage function, used for storing program instructions and / or data.

[0188] By designing and programming the processor 1401, the code corresponding to the vehicle azimuth angle acquisition method introduced in the foregoing embodiments can be fixed in the chip, so that the chip can execute the steps of the vehicle azimuth angle acquisition method of the embodiments shown in the running time. Figure 3 How to design and program the processor 1401 is a technology known to those skilled in the art, and will not be described here.

[0189] Based on the same inventive concept, the embodiments of the present application also provide a storage medium storing computer instructions, when the computer instructions run on a computer, the computer instructions make the computer execute the vehicle azimuth angle acquisition method discussed above.

[0190] In some possible implementations, the present application also provides various aspects of a vehicle azimuth angle acquisition method, which can also be implemented in the form of a program product, including program code, when the program product runs on a device, the program code is used to make the control device execute the steps of the vehicle azimuth angle acquisition method according to various exemplary embodiments of the present application described above in the specification.

[0191] It should be noted that, although several units or sub-units of the apparatus are mentioned in the above detailed description, such division is merely exemplary and not mandatory. Indeed, according to an embodiment of the application, the features and functionalities of two or more units described above can be embodied in one unit. Conversely, the features and functionalities of one unit described above can be further divided into units embodied by several units.

[0192] Moreover, although the operations of the method(s) herein can be described in a particular, sequential order, this order is not meant to be a limitation and is not intended to imply that

[0193] Those of skill in the art would understand that embodiments of the present application can be provided as a method, a system, or a computer program product. Accordingly, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application can take the form of a computer program product on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage devices, and the like) embodying computer readable program code.

[0194] The present application is described in reference to the flowchart illustrations and / or block diagrams of the methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing system or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams.

[0195] The program code can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++, or the like, and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server.

[0196] In situations where the remote computing device is involved, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0197] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0198] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions that are executed on the computer or other programmable apparatus provide steps for implementing the Figure 1 function specified in the flow or flows and / or blocks Figure 1 of the block or blocks.

[0199] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A vehicle azimuth acquisition method, characterized by, The method comprises: obtaining an original road image collected by an image collection device at a set time, and performing edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image comprises an original road region that satisfies a preset azimuth estimation reliability condition; performing region feature extraction on the original road image to obtain a road region feature of the original road region, and performing edge feature extraction on the road edge image to obtain a road edge feature of the original road region; based on the road region feature and the road edge feature, reconstructing the original road region to obtain a corresponding reconstructed road region; wherein the reconstructed road region satisfies a preset road region unocclusion condition; based on a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a position of a vehicle at the set time, obtaining a vehicle azimuth of the vehicle, comprising: based on a device parameter set of the image collection device, performing inverse perspective transformation on the reconstructed road region to obtain a target road region that satisfies a preset region distribution change condition; wherein the target road region has a different observation angle from the reconstructed road region; based on a vehicle deflection angle associated with a region distribution feature of the target road region and a road direction angle included in the road direction information, obtaining a vehicle azimuth of the vehicle.

2. The method of claim 1, wherein, The region feature extraction on the original road image to obtain the road region feature of the original road region comprises: performing multi-scale feature extraction on the original road image to obtain a sub-road region feature of the original road region corresponding to each feature extraction channel respectively; wherein each feature extraction channel corresponds to a scale feature extraction; based on the obtained each sub-road region feature, obtaining the road region feature of the original road region.

3. The method of claim 1, wherein, The edge feature extraction on the road edge image to obtain the road edge feature of the original road region comprises: performing single-scale feature extraction on the road edge image to obtain an edge feature extraction result corresponding to a corresponding feature extraction channel; the edge feature extraction result is taken as the road edge feature of the original road region.

4. The method of claim 1, wherein, The reconstruction of the original road region based on the road region feature and the road edge feature to obtain the corresponding reconstructed road region comprises: performing cascaded feature fusion on the road region feature and the road edge feature to obtain a corresponding region fusion feature; based on a region range of the original road region in the original road image and the region fusion feature, reconstructing the original road region to obtain a corresponding reconstructed road image; from the reconstructed road image, determining the reconstructed road region corresponding to the original road region.

5. The method of claim 1, wherein, Before the vehicle azimuth of the vehicle is obtained based on the vehicle deflection angle associated with the region distribution feature of the target road region and the road direction angle included in the road direction information, the method further comprises: for each sample road image, the following operations are performed respectively: obtaining a sample road region corresponding to a sample road image, and a corresponding sample azimuth and sample direction angle; determine a sample deflection angle corresponding to the one sample road image based on the sample azimuth angle and the sample direction angle; associate the region distribution feature of the sample road region with the sample deflection angle.

6. The method of claim 1, wherein, The vehicle deflection angle associated based on the region distribution feature of the target road region and the road direction angle contained in the road direction information are used to obtain a vehicle azimuth angle of the vehicle, including: acquire, from the road direction information, node information of multiple road nodes each satisfying a preset distance interval condition with a position of the vehicle at the set time; wherein each node information is used to indicate a coordinate position of a corresponding road node in a standard reference coordinate system; obtain road section slopes corresponding to the multiple road nodes based on the coordinate positions contained in the obtained multiple node information, and obtain the road direction angle based on the road section slopes; wherein each road section slope represents a road section inclination degree between two adjacent road nodes in the road nodes; obtain a vehicle azimuth angle of the vehicle based on the vehicle deflection angle and the road direction angle.

7. A vehicle azimuth acquisition device, characterized by comprising: including: an acquisition module, configured to acquire an original road image collected by an image collection device at a set time, and perform edge detection on the original road image to obtain a corresponding road edge image; wherein the original road image includes an original road region satisfying a preset azimuth angle estimation credibility condition; an extraction module, configured to perform region feature extraction on the original road image to obtain road region features of the original road region, and perform edge feature extraction on the road edge image to obtain road edge features of the original road region; a reconstruction module, configured to reconstruct the original road region based on the road region features and the road edge features to obtain a corresponding reconstructed road region; wherein the reconstructed road region satisfies a preset road region unocclusion condition; a determination module, configured to obtain a vehicle azimuth angle of the vehicle based on a vehicle deflection angle corresponding to the reconstructed road region and road direction information of a position of the vehicle at the set time, including: perform inverse perspective transformation on the reconstructed road region based on a device parameter set of the image collection device to obtain a target road region satisfying a preset region distribution change condition; wherein the target road region has a different observation angle from the reconstructed road region; obtain a vehicle azimuth angle of the vehicle based on a vehicle deflection angle associated based on a region distribution feature of the target road region and a road direction angle contained in the road direction information.

8. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the method in any one of claims 1-6.

9. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1-6. The computer program is executed by the processor to implement the steps of the method in any one of claims 1-6.

Images