Image-based heading angle determination method and apparatus, electronic device, and storage medium

Abstract

The present disclosure provides an image-based heading angle determination method, which can include: determining a conversion equation set for representing a relationship between a heading angle of a target vehicle in an image and an imaging coordinate of the target vehicle in an image coordinate system by using camera parameters of an image acquisition device; determining intersection coordinates of a plurality of intersection points between a boundary line of the target vehicle and a vanishing line in the image in the image coordinate system based on the conversion equation set; and substituting the intersection coordinates into the conversion equation set to obtain the heading angle. The present disclosure also provides an image-based heading angle determination device, an electronic device, and a storage medium.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of computer vision perception, and particularly relates to an image-based heading angle determination method and device, electronic equipment and storage medium. BACKGROUND

[0002] The heading angle is the included angle between the center of mass velocity direction of the target vehicle (i.e. the driving direction of the target vehicle) and the center of mass velocity direction of the reference vehicle (i.e. the driving direction of the reference vehicle) in the vehicle coordinate system. In the driving process of the target vehicle, the heading angle needs to be determined to realize the control of the driving direction of the reference vehicle.

[0003] In related technologies, there are the following ways to determine the heading angle. Way one is to detect the wheel or wheel grounding point in the image containing the target vehicle, and based on the ground assumption, the wheel or wheel grounding point is transformed to the vehicle coordinate system through the camera imaging model, and then the heading angle of the target vehicle is calculated. Way two is to regress the heading angle of the target vehicle based on a neural network deep learning method. Way three is a heading angle calculation method based on the target vehicle detection box. According to the principle of plane perspective, the plane perspective view is geometrically operated to obtain the heading angle of the target vehicle. Way four is to fit the wheel grounding point of the target vehicle into a wheel trajectory segment, and calculate the heading angle of the target vehicle according to the trajectory start point and the trajectory end point on the wheel trajectory segment.

[0004] However, the aforementioned way one is not applicable to the case where the wheels of the target vehicle are blocked, and the inverse projection transformation based on the ground assumption will bring certain errors. The implementation of way two requires the cooperation of a neural network, and the training of the neural network requires a large amount of data annotation to support, which is a large amount of work and reduces the calculation speed of the heading angle of the target vehicle; and requires a large amount of computing power of the operation platform, which is poor in real-time performance and difficult to meet the application requirements of actual driving scenarios. Way three needs to preset the aspect ratio value of the target vehicle, and the aspect ratio values of different target vehicles are different, which is difficult to cover various types of target vehicles, so it is difficult to accurately determine the heading angle of the target vehicle. Way four is invalid when the wheels of the target vehicle are blocked, and the method of using historical trajectory calculation is not applicable to the low-speed or static working conditions of the target vehicle. SUMMARY

[0005] In order to solve at least one of the above technical problems, the present disclosure provides an image-based heading angle determination method, device, electronic equipment and storage medium.

[0006] One aspect of the present disclosure provides an image-based heading angle determination method, which can include: determining, using camera parameters of an image acquisition device, a conversion equation set for representing a relationship between a heading angle of a target vehicle in an image and imaging coordinates of the target vehicle in an image coordinate system; determining, based on the conversion equation set, intersection coordinates of a plurality of intersection points between boundary lines of the target vehicle and a vanishing line in the image in the image coordinate system; and substituting the intersection coordinates into the conversion equation set to obtain the heading angle.

[0007] In some embodiments, the determining, using camera parameters of an image acquisition device, a conversion equation set for representing a relationship between a heading angle of a target vehicle in an image and imaging coordinates of the target vehicle in an image coordinate system, includes: constructing a boundary line equation of the boundary line in a vehicle coordinate system, wherein the boundary line equation is used to represent a relationship between the heading angle and position coordinates of the boundary line in the vehicle coordinate system; determining, according to the camera parameters, a conversion matrix for representing a relationship between the image coordinate system and the vehicle coordinate system; and substituting the boundary line equation into the conversion matrix to determine the conversion equation set based on position coordinates of the intersection points in the vehicle coordinate system.

[0008] In some embodiments, the conversion equation set is:

[0009]

[0010]

[0011] wherein u i is a row value of the image coordinate system, v i is a column value of the image coordinate system, m1, m2, m5, m6, m9, m 10 are known camera parameters, and θ i is the heading angle corresponding to the i-th boundary line, and i is 1 or 2.

[0012] In some embodiments, the determining, based on the conversion equation set, intersection coordinates of a plurality of intersection points between boundary lines of the target vehicle and a vanishing line in the image in the image coordinate system, includes: determining, according to the conversion equation set, a vanishing line equation for representing a relationship between a column value and a row value of the vanishing line in the image coordinate system; determining, based on position coordinates of at least two boundary points of the boundary line in the vehicle coordinate system, the boundary line equation; and determining, according to the vanishing line equation and the boundary line equation, the intersection coordinates of the plurality of intersection points between the boundary line and the vanishing line in the image coordinate system.

[0013] In some embodiments, the heading angle is:

[0014]

[0015] wherein, u i is a row value of the i-th intersection point in the image coordinate system, v i is a column value of the i-th intersection point in the image coordinate system, m1, m2, m5, m6, m9, m 10 are all known camera parameters, and θ i is a heading angle corresponding to the i-th boundary line, wherein i is 1 or 2.

[0016] In some embodiments, after the intersection point coordinates are substituted into the conversion equation set to obtain the heading angle, the method further comprises:

[0017] optimizing the heading angle according to the lengths of the boundary lines to obtain a target heading angle.

[0018] In some embodiments, the target heading angle is:

[0019]

[0020] wherein, θ is the target heading angle of the target vehicle, θ th is a preset difference, ||l1|| is the length of a first boundary line among the boundary lines, and ||l2|| is the length of a second boundary line among the boundary lines.

[0021] Another aspect of the present disclosure provides an image-based heading angle determination device, which comprises: a conversion equation set construction module configured to determine a conversion equation set for representing a relationship between a heading angle of a target vehicle in an image and an imaging coordinate of the target vehicle in an image coordinate system by using camera parameters of an image acquisition device; an intersection point coordinate determination module configured to determine intersection point coordinates of a plurality of intersection points between boundary lines of the target vehicle and vanishing lines in the image in the image coordinate system based on the conversion equation set; and a heading angle calculation module configured to substitute the intersection point coordinates into the conversion equation set to obtain the heading angle.

[0022] Still another aspect of the present disclosure provides an electronic device, which can comprise: a memory storing execution instructions; and a processor executing the execution instructions stored in the memory, so that the processor executes the image-based heading angle determination method according to any one of the above embodiments.

[0023] Yet another aspect of the present disclosure provides a readable storage medium, which stores execution instructions, and the execution instructions are executed by a processor to implement the image-based heading angle determination method according to any one of the above embodiments. Attached Figure Description

[0024] The accompanying drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0025] Figure 1 This is a flowchart of a heading angle determination method according to an exemplary embodiment of the present disclosure;

[0026] Figure 2 This is a schematic diagram of a vanishing line according to an exemplary embodiment of this disclosure;

[0027] Figure 3 This is a real-world vanishing line illustration of an exemplary embodiment of this disclosure;

[0028] Figure 4 This is a schematic diagram of the boundary lines of an exemplary embodiment of this disclosure;

[0029] Figure 5 This is a real-world intersection diagram illustrating an exemplary embodiment of this disclosure; and

[0030] Figure 6 This is a schematic diagram of a heading angle determination device according to an exemplary embodiment of this disclosure.

[0031] Reference numerals

[0032] 1000 Image-based heading angle determination device

[0033] 1002 Intersection Coordinate Determination Module

[0034] 1004 Heading Angle Calculation Module

[0035] 1006 Compensation Module

[0036] 1100 bus

[0037] 1200 processor

[0038] 1300 memory

[0039] 1400 Other Circuits Detailed Implementation

[0040] The present disclosure will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure. Furthermore, it should be noted that, for ease of description, only the parts relevant to the present disclosure are shown in the accompanying drawings.

[0041] It should be noted that the embodiments and features of the embodiments in the present disclosure can be combined with each other in the case of no conflict. The technical solutions of the present disclosure will be described in detail below with reference to the drawings and in combination with the embodiments.

[0042] Unless otherwise specified, the example embodiments / instances shown will be understood as providing exemplary features of various details that can implement the technical concepts of the present disclosure in practice. Therefore, unless otherwise specified, the features of various embodiments / instances can be additionally combined, separated, interchanged and / or rearranged without departing from the technical concepts of the present disclosure.

[0043] In the drawings, cross-hatching and / or shading are generally used to indicate that a portion of one feature can be located structurally close to another, even though not directly visible. As such, unless stated otherwise, the presence of cross-hatching or shading is not meant to imply that a specific material, material property, dimension, ratio, etc., is being represented other than that which might be explicitly stated. In addition, for purposes of clarity and / or description, dimensions and / or relative dimensions shown in the figures can have been exaggerated relative to other elements. When example embodiments can be practiced differently, a specific process sequence can be performed in a different order than the described order or sequentially. For example, two consecutively described processes can be performed at substantially the same time or in the reverse order of the described order. Moreover, like reference numerals designate like elements throughout the several views.

[0044] When a component is referred to as being “on” or “over” another component, “connected to” or “coupled to” another component, it can be directly on, directly connected to, or directly coupled to the other component, or intervening components can be present. However, when a component is referred to as being “directly on”, “directly connected to”, or “directly coupled to” another component, there are no intervening components present. For this reason, the term “connected” can refer to a physical connection, an electrical connection, etc., with or without intervening components.

[0045] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. Furthermore, to the extent that the terms "comprise," "have," "contain," and / or "include" and variations thereof are used herein, such terms are intended to be inclusive, in an aspect, it is also noted that the terms "substantial," "approximately," and other similar terms are used herein as terms of approximation and not as terms of degree, such that they are utilized to account for inherent deviations in measurements, calculations, and / or provided values that would be recognized by those of ordinary skill in the art.

[0046] Figure 1 A flowchart of a heading angle determination method for an exemplary embodiment of the present disclosure; Figure 2 A vanishing line diagram for an exemplary embodiment of the present disclosure; Figure 3 A real scene vanishing line display diagram for an exemplary embodiment of the present disclosure; Figure 4 A boundary line diagram for an exemplary embodiment of the present disclosure; Figure 5 A real scene intersection display diagram for an exemplary embodiment of the present disclosure. The following will be described in combination with Figures 1 to 5 The image-based heading angle determination method S100 is described.

[0047] In step S102, a conversion equation set for characterizing the relationship between the heading angle of the target vehicle in the image and the imaging coordinates of the target vehicle in the image coordinate system is determined using the camera parameters of the acquisition device.

[0048] The image is a photographing result of the target vehicle by the acquisition device of the reference vehicle, and includes the target vehicle and its surrounding environment.

[0049] The acquisition device can be a camera, and is a device for acquiring the relative position between the target vehicle and the reference vehicle. The acquisition device has camera parameters, including intrinsic parameters and extrinsic parameters. The intrinsic parameters of the camera represent parameters related to the characteristics of the acquisition device, such as the focal length and pixels of the acquisition device. The extrinsic parameters of the camera determine the position and orientation of the acquisition device in the three-dimensional space, i.e., the rotation matrix and translation matrix through which the coordinate point in the vehicle coordinate system falls into the image coordinate system. The conversion relationship between the vehicle coordinate system and the image coordinate system can be obtained using the camera parameters.

[0050] The conversion equation set is used to represent the conversion relationship between the vehicle coordinate system and the image coordinate system, and includes a first conversion formula used to represent the conversion relationship between the horizontal coordinate of the vehicle coordinate system and the row value of the image coordinate system, and a second conversion formula used to represent the conversion relationship between the vertical coordinate of the vehicle coordinate system and the column value of the image coordinate system.

[0051] Specifically, the first conversion formula in the conversion equation set can be represented as:

[0052]

[0053] The second conversion formula in the conversion equation set can be represented as:

[0054]

[0055] In the formula (1) and the formula (2), v i is the intersection point p i The row value in the image coordinate system, u i is the intersection point p i The column value in the image coordinate system, m1, m2, m5, m6, m9, m 10 are all known camera parameters, and θ i is the heading angle corresponding to the i-th boundary line; wherein i takes values of 1 and 2.

[0056] The vehicle coordinate system is a kind of world coordinate system, and the driving direction (for example, the front direction) of the reference vehicle is taken as the horizontal axis (i.e., the X axis), and the direction perpendicular to the driving direction and pointing to the left side of the driver's seat is taken as the vertical axis (i.e., the Y axis). The origin (i.e., the O point) of the vehicle coordinate system is set at the middle position of the vehicle head, and the collection device is set at the origin.

[0057] The heading angle is the included angle between the driving direction of the target vehicle and the driving direction of the reference vehicle, and the driving direction of the target vehicle can be controlled according to the heading angle, and the heading angle can also be used for the reference vehicle to predict the driving track of the target vehicle, etc.

[0058] In step S104, based on the conversion equation set, the intersection point coordinates of the multiple intersection points between the boundary line of the target vehicle and the vanishing line in the image in the image coordinate system are determined respectively.

[0059] In this context, the boundary lines are the extensions of two mutually perpendicular end segments on the ground plane of the 3D bounding box. The 3D bounding box is a cube circumscribed around the target vehicle, constructed based on its outline. The ground plane is a side of the 3D bounding box that directly contacts the ground supporting the target vehicle. Since the 3D bounding box is a cube, the ground plane is rectangular. Therefore, the ground plane will have two sets of mutually perpendicular line segment pairs, with the two end segments in each pair being parallel to each other. The parallel end segments form the same angle with the direction of travel of the reference vehicle. Therefore, one end segment is selected from each pair, and its extension is used as the boundary line. Based on the above, there are two boundary lines.

[0060] A vanishing line is the line where the ground intersects the sky in an image; it is the horizon shown in the image. For example... Figure 2 As stated above, based on the principle of perspective transformation, in reality, two parallel straight lines L1 and L2 will intersect at a vanishing point VP in an image. Several sets of vanishing points formed by these lines will then be connected to form a vanishing line VL. In reality, line L1 has points A1 and A2, and line L2 has points A3 and A4. The distance between points A1 and A3 is the same as the distance between points A2 and A4. In the image, the distance between points A1' and A3' on line L1 is less than the distance between points A4' and A2'. Figure 3 The image shows a real-world view of parallel railway tracks intersecting at a vanishing point at infinity, and also displays a vanishing line formed by the convergence of several vanishing points in the image.

[0061] By combining equations (1) and (2) in the transformation equation system, we can obtain the equation for the vanishing line:

[0062]

[0063] The intersection point is where the boundary line of the target vehicle intersects the vanishing line in the image. Since it has been demonstrated earlier that there are two boundary lines, two intersection points will be generated for different boundary lines. Based on the target vehicle's localization system, the position coordinates of the two endpoints of each end segment of the 3D bounding box in the vehicle coordinate system are known. This means that there are two points with known position coordinates on the boundary line corresponding to the end segments. Based on the position coordinates of these two endpoints, the slope of the corresponding boundary line can be determined. According to the slope and the position coordinates of any endpoint on that boundary line, the relationship between the line and column values ​​of the intersection point in the image coordinate system can be obtained, namely:

[0064]

[0065] In formula (4), (x n y n ) is located on the boundary line l i endpoint b nPosition coordinates in the vehicle coordinate system; (x 1+n y n+1 ) is located on the boundary line l i endpoint b n+1 The position coordinates in the vehicle coordinate system. Where n takes the value 1, 2, or 3.

[0066] Because in formulas (3) and (4) except for the intersection point p i (v i u i Since both equations are known, we can obtain the intersection point p by combining the two equations. i (v i u i The imaging coordinates in the image coordinate system.

[0067] Step S106: Substitute the coordinates of the intersection point into the transformation equations to obtain the heading angle.

[0068] The first and second transformation equations in the transformation equation system only have an intersection point p. i (v i u i ) and heading angle θ i Three unknowns, at this point the intersection point p i (v i u i Having obtained the specific values ​​through the aforementioned steps, the coordinates of the intersection point can be substituted into the transformation equations to obtain the value of the heading angle.

[0069] The heading angle can be expressed as:

[0070]

[0071] In some implementations, step S102 may involve: constructing the boundary line equation in the vehicle coordinate system; determining a transformation matrix to characterize the relationship between the image coordinate system and the vehicle coordinate system based on camera parameters; substituting the boundary line equation into the transformation matrix and determining a set of transformation equations based on the position coordinates of the intersection point in the vehicle coordinate system.

[0072] The boundary line equation characterizes the relationship between the heading angle and the boundary line's position coordinates in the vehicle coordinate system. The transformation matrix is ​​a matrix equation used to transform between the image coordinate system and the vehicle coordinate system using camera parameters.

[0073] like Figure 4 As shown, a vehicle coordinate system is constructed with the midpoint of the front of the reference vehicle as the origin, with the horizontal axis as the X-axis and the vertical axis as the Y-axis. The horizontal axis points in the direction of travel of the reference vehicle, and the vertical axis is perpendicular to the horizontal axis and points to the left side of the driver's cab of the reference vehicle. The target vehicle is located in the vehicle coordinate system, and the boundary line l corresponding to the target vehicle is shown. iThe included angle θ between the driving direction of the reference vehicle (i.e. vanishing line) i (i.e. course angle).

[0074] The boundary line l i In the vehicle reference system, it can be expressed as: Y = X*tanθ i + b, (6)

[0075] Equation (6) is the boundary line equation, (X, Y) is the position coordinate of any point on the boundary line in the vehicle reference system, and b is a straight line parameter.

[0076] The conversion matrix between the image coordinate system and the vehicle coordinate system is:

[0077]

[0078] In equation (7), s is a known scale factor, M1 is the camera intrinsic parameter, M2 is the camera extrinsic parameter, (u, v) is the imaging coordinate of any point in space in the image coordinate system, and (X, Y, Z) is the position coordinate of any point in space in the vehicle coordinate system.

[0079] Since the boundary line l i is located on the ground, Z = 0. Substituting equation (6) into equation (7), the conversion matrix corresponding to any point (u, v) on the boundary line l i can be expressed as:

[0080]

[0081] Then the conversion matrix corresponding to the vanishing point VP (i.e. intersection point) of all straight lines parallel to the boundary line l i and the straight line can be expressed as:

[0082]

[0083] According to the intrinsic and extrinsic parameters of the actual acquisition device, the total camera parameter can be set as:

[0084]

[0085] Wherein, M is the total camera parameter, m1 to m 12 are all known camera parameters.

[0086] Then, combined with the total camera parameter and the conversion matrix corresponding to the vanishing point VP, the conversion equation corresponding to the intersection point can be calculated, that is, equation (1) and equation (2).

[0087] In some implementations, step S104 is specifically implemented as follows: based on the transformation equation set, determine the vanishing line equation to characterize the relationship between the column and row values ​​of the vanishing line in the image coordinate system; based on the position coordinates of at least two boundary points in the boundary line in the vehicle coordinate system, determine the boundary line equation; based on the vanishing line equation and the boundary line equation, determine the intersection coordinates of multiple intersection points between the boundary line and the vanishing line in the image coordinate system.

[0088] like Figure 5 As shown, the target vehicle has boundary line l1 and boundary line l2. Boundary line l1 has two endpoints b1 and b2 of end line segments, and boundary line l2 has two endpoints b2 and b3 of end line segments, where b2 is the perpendicular intersection point of the two end line segments. Intersection point p1 is the intersection point of boundary line l1 and vanishing line l, and its imaging coordinates in the image coordinate system are (u1, v1); intersection point p2 is the intersection point of boundary line l2 and vanishing line l, and its imaging coordinates in the image coordinate system are (u2, v2).

[0089] Taking boundary line l2 as an example, formula (4) can be expressed as:

[0090]

[0091] Based on the above formula and formula (3), the imaging coordinates (u2, v2) of the intersection point p2 can be obtained.

[0092] Furthermore, substituting the imaging coordinates (u2, v2) of the intersection point p2 into formulas (1) and (2), since the imaging coordinates (u2, v2) of the intersection point p2 are known, the heading angle corresponding to the boundary line l2 can be obtained.

[0093] Similarly, the heading angle corresponding to boundary line l1 can also be obtained.

[0094] In some implementations, after step S106, the method further includes: optimizing the heading angle based on the modulus of each boundary line to obtain the target heading angle.

[0095] Ideally, the two heading angles corresponding to the two boundary lines should be approximately equal, and their sum should be 90°. However, since there will be construction errors in the 3D bounding box of the target vehicle, the heading angle is optimized to accommodate more 3D bounding box construction errors, taking into account the actual situation of the 3D bounding box.

[0096] The target heading angle can be expressed as follows under different errors:

[0097]

[0098] Where θ is the target heading angle of the target vehicle, θ thFor a preset difference value, ||l1|| is the length of the first boundary line l1 in the boundary line, and ||l2|| is the length of the second boundary line l2 in the boundary line.

[0099] In short, when the sum of the two heading angles is approximately 90°, the mean of the two is taken as the target heading angle; if the sum of the two heading angles is greatly different from 90°, the heading angle corresponding to the boundary line with the larger length is taken as the target heading angle.

[0100] The image-based heading angle determination method provided by the present disclosure can quickly calculate the heading angle by using the conversion relationship between the vehicle coordinate system and the image coordinate system and the intersection point of the boundary line in the three-dimensional boundary box of the target vehicle and the vanishing line in the image, avoiding the need for a large amount of sample data in related technologies and the dependence on various sensors, and having high real-time performance. In addition, the construction error of the various three-dimensional boundary boxes is considered, so that the obtained target heading angle is more accurate.

[0101] Figure 6 A schematic diagram of the heading angle determination device of the exemplary embodiment of the present disclosure.

[0102] As Figure 6 shown, another aspect of the present disclosure provides an image-based heading angle determination device 1000, which can include: a conversion equation set construction module 1002 configured to determine a conversion equation set for representing the relationship between the heading angle of a target vehicle in an image and the imaging coordinates of the target vehicle in an image coordinate system by using the camera parameters of the image acquisition device; an intersection point coordinate determination module 1004 configured to determine the intersection point coordinates of a plurality of intersection points between the boundary line of the target vehicle and the vanishing line in the image in the image coordinate system based on the conversion equation set; and a heading angle calculation module 1006 configured to substitute the intersection point coordinates into the conversion equation set to obtain the heading angle.

[0103] Each module in the image-based heading angle determination device 1000 is configured to implement each step of the image-based heading angle determination method, and the execution steps and implementation principles are the same as those of the image-based heading angle determination method. For reference, the foregoing will not be repeated.

[0104] The device can include corresponding modules that perform each or several steps in the above flowchart. Therefore, each or several steps in the above flowchart can be performed by corresponding modules, and the device can include one or more of these modules. The modules can be one or more hardware modules specially configured to perform the corresponding steps, or implemented by a processor configured to perform the corresponding steps, or stored in a computer-readable medium for implementation by a processor, or implemented by some combination.

[0105] The hardware structure can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application of the hardware and overall design constraints. Bus 1100 connects various circuits including one or more processors 1200, memory 1300, and / or hardware modules together. Bus 1100 can also connect various other circuits 1400 such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.

[0106] Bus 1100 can be an Industry Standard Architecture (ISA), Peripheral Component Interconnect (PCI), Extended Industry Standard Architecture (EISA), or other bus architectures. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one connection line is shown in the figure, but it does not mean that there is only one bus or one type of bus.

[0107] The image-based heading angle determination device provided by the present disclosure can quickly calculate the heading angle by using the conversion relationship between the vehicle coordinate system and the image coordinate system, and the intersection of the boundary line in the three-dimensional bounding box of the target vehicle and the vanishing line in the image, avoiding the need for a large amount of sample data in related technologies and the dependence on various sensors, and having high real-time performance. In addition, the construction error of the various three-dimensional bounding boxes is considered, so that the obtained target heading angle is more accurate.

[0108] Any process or method described in flowcharts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for implementing specific logic functions (or steps) in the process, and the scope of preferred embodiments of the present disclosure includes additional implementation in which the functions are performed in different orders, in substantially simultaneous fashion, or in reverse order, according to the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong. The processor performs the various methods and processes described above. For example, the method embodiments in the present disclosure can be implemented as a software program tangibly embodied in a machine-readable medium, such as a memory. In some embodiments, part or all of the software program can be loaded and / or installed via the memory and / or communication interface. When the software program is loaded into the memory and executed by the processor, one or more steps in the above-described methods can be performed. Alternatively, in other embodiments, the processor can be configured to perform one of the above methods by any other appropriate means (e.g., by means of firmware).

[0109] The logic and / or steps represented in the flowcharts and / or otherwise described herein can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions.

[0110] For the purposes of this specification, a "computer-readable storage medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable storage medium can specifically include the following, which are non-exhaustive examples: electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CD-ROM). The computer-readable storage medium can even be paper or another suitable medium upon which the program can be printed, because the program can be electronically captured, for example, by optically scanning the paper or other medium, then electronically converted into a form that can be edited, compiled, or interpreted, or otherwise processed in electronic form into another appropriate format, and stored in the storage.

[0111] It should be understood that portions of the present disclosure can be implemented with hardware, software, or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be implemented with software stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies known in the art or their combinations can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application-specific integrated circuits (ASICs) having appropriate combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0112] Those of ordinary skill in the art of the present technology can understand that all or part of the steps of the above-described embodiments can be completed by programs instructing relevant hardware, and the programs can be stored in a readable storage medium. When executed, the programs include one or a combination of the steps of the method embodiments.

[0113] In addition, in various embodiments of the present disclosure, each function unit can be integrated in one processing module, or each unit can be physically present separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware or in the form of a software function module. When the integrated module is realized in the form of a software function module and sold or used as an independent product, it can also be stored in a readable storage medium. The storage medium can be a read-only memory, a magnetic disk or an optical disk, etc.

[0114] The present disclosure also provides an electronic device, comprising: a memory storing execution instructions; and a processor or other hardware module executing the execution instructions stored by the memory, so that the processor or other hardware module executes the image-based heading angle determination method.

[0115] The present disclosure also provides a readable storage medium, which stores execution instructions for implementing the image-based heading angle determination method when executed by a processor.

[0116] In the description of the present disclosure, the description of the terms "one embodiment / way", "some embodiments / ways", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment / way or example are included in at least one embodiment / way or example of the present disclosure. In the present disclosure, the illustrative description of the above terms is not necessarily the same embodiment / way or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments / ways or examples. In addition, a person skilled in the art can combine and combine the different embodiments / ways or examples described in the present disclosure and the features of the different embodiments / ways or examples without contradiction.

[0117] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

[0118] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. Based on the above disclosure, other changes or modifications can also be made by those skilled in the art, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. A method for determining heading angle based on an image, characterized in that, include: Using the camera parameters of the image acquisition device, a set of transformation equations is determined to characterize the relationship between the heading angle of the target vehicle in the image and the imaging coordinates of the target vehicle in the image coordinate system; Based on the transformation equations, the coordinates of the intersection points between the boundary line of the target vehicle and the vanishing line in the image are determined in the image coordinate system. as well as Substitute the coordinates of the intersection point into the system of transformation equations to obtain the heading angle; The method of determining a set of transformation equations using the camera parameters of the image acquisition device to characterize the relationship between the heading angle of the target vehicle in the image and the imaging coordinates of the target vehicle in the image coordinate system includes: Construct the boundary line equation in the vehicle coordinate system, wherein the boundary line equation is used to characterize the relationship between the heading angle and the position coordinates of the boundary line in the vehicle coordinate system; Based on the camera parameters, determine the transformation matrix used to characterize the relationship between the image coordinate system and the vehicle coordinate system; Substitute the boundary line equation into the transformation matrix, and determine the transformation equation set based on the position coordinates of the intersection point in the vehicle coordinate system.

2. The image-based heading angle determination method according to claim 1, characterized in that, The transformation equations are as follows: ， ， in, The row value of the image coordinate system. These are the column values ​​of the image coordinate system. , , , , , All of these are known camera parameters. Let i be the heading angle corresponding to the i-th boundary line, where i takes the values ​​1 and 2.

3. The image-based heading angle determination method according to claim 2, characterized in that, The step of determining the coordinates of multiple intersection points between the boundary line of the target vehicle and the vanishing line in the image in the image coordinate system based on the transformation equations includes: Based on the set of transformation equations, determine the vanishing line equations that characterize the relationship between the column and row values ​​of the vanishing line in the image coordinate system; The boundary line equation is determined based on the position coordinates of at least two boundary points in the vehicle coordinate system. Based on the vanishing line equation and the boundary line equation, determine the coordinates of the multiple intersection points between the boundary line and the vanishing line in the image coordinate system.

4. The image-based heading angle determination method according to claim 1, characterized in that, The heading angle is: ， in, Let i be the row value of the i-th intersection point in the image coordinate system. The column value of the i-th intersection point in the image coordinate system. , , , , , All of these are known camera parameters. Let be the heading angle corresponding to the i-th boundary line; where i takes the values ​​1 and 2.

5. The image-based heading angle determination method according to claim 4, characterized in that, After substituting the intersection coordinates into the transformation equations to obtain the heading angle, the process includes: Based on the modulus of each of the boundary lines, the heading angle is optimized to obtain the target heading angle.

6. The image-based heading angle determination method according to claim 5, characterized in that, The target heading angle is: ， in, The target heading angle of the target vehicle. As a preset difference, The modulus of the first boundary line in the boundary lines is given. Let be the modulus of the second boundary line in the boundary lines.

7. A heading angle determination device based on an image, characterized in that, include: The transformation equations construction module is used to determine a set of transformation equations that characterize the relationship between the heading angle of the target vehicle in the image and the imaging coordinates of the target vehicle in the image coordinate system, using the camera parameters of the image acquisition device. The intersection point coordinate determination module is used to determine the intersection point coordinates of multiple intersection points between the boundary line of the target vehicle and the vanishing line in the image in the image coordinate system based on the transformation equation set. as well as The heading angle calculation module is used to substitute the coordinates of the intersection point into the transformation equation system to obtain the heading angle; The method of determining a set of transformation equations using the camera parameters of the image acquisition device to characterize the relationship between the heading angle of the target vehicle in the image and the imaging coordinates of the target vehicle in the image coordinate system includes: Construct the boundary line equation in the vehicle coordinate system, wherein the boundary line equation is used to characterize the relationship between the heading angle and the position coordinates of the boundary line in the vehicle coordinate system; Based on the camera parameters, determine the transformation matrix used to characterize the relationship between the image coordinate system and the vehicle coordinate system; Substitute the boundary line equation into the transformation matrix, and determine the transformation equation set based on the position coordinates of the intersection point in the vehicle coordinate system.

8. An electronic device, characterized in that, include: The memory stores execution instructions; as well as A processor that executes execution instructions stored in the memory, causing the processor to perform the image-based heading angle determination method according to any one of claims 1 to 6.

9. A readable storage medium, characterized in that, The readable storage medium stores execution instructions, which, when executed by a processor, are used to implement the image-based heading angle determination method according to any one of claims 1 to 6.

Images