Vehicle orientation angle calculation method, device, equipment and storage medium

Abstract

The application provides a vehicle orientation angle calculation method, device, equipment and storage medium. The method comprises the following steps: obtaining an initial image based on a monocular camera, wherein the initial image comprises a target vehicle; preprocessing the initial image to obtain a preprocessed image, wherein the preprocessed image contains envelope box information, and the envelope box information comprises a whole vehicle envelope box, a vehicle head envelope box, a vehicle tail envelope box and a vehicle wheel envelope box; detecting the number of vehicle wheel envelope boxes contained in the preprocessed image; in response to the number of vehicle wheel envelope boxes being less than 2, calculating an intersection-over-union ratio and obtaining the orientation angle of the target vehicle based on the intersection-over-union ratio; and in response to the number of vehicle wheel envelope boxes being greater than or equal to 2, calculating the orientation angle of the target vehicle based on the envelope box information.

Description

Technical Field

[0001] This application relates to the field of intelligent driving, and more specifically, to a method, apparatus, device, and storage medium for calculating vehicle heading angle. Background Technology

[0002] In recent years, with the continuous development of computer hardware and artificial intelligence technologies, and the continuous iterative upgrades of automotive intelligence, autonomous driving technology has developed rapidly. Autonomous vehicles utilize onboard sensors such as LiDAR, Global Positioning System (GPS), and cameras to perceive information about the vehicle's surrounding environment, including road information, obstacle information, and information about other vehicles and pedestrians. Based on this information, the vehicle controls its safe and efficient operation.

[0003] To improve the safety of autonomous driving, estimating the heading angles of other vehicles (target vehicles) around the vehicle is crucial. Obtaining the heading angles of target vehicles allows for better prediction of their trajectories, thus enabling better planning of the vehicle's own direction. However, during vehicle operation, due to the complexity and variability of road conditions and the varying sizes of target vehicles, the vehicle often collects information from multiple target vehicles, or the target vehicles may be obstructed by obstacles. Therefore, the information about target vehicles is generally incomplete and subject to some obstruction, making it difficult to detect the heading angles of target vehicles, resulting in significant discrepancies between the obtained heading angles and the actual situation.

[0004] Therefore, how to quickly and accurately obtain the orientation angle of the target vehicle to improve the safety of autonomous driving is an urgent problem to be solved. Summary of the Invention

[0005] Some embodiments of this application provide a method, apparatus, device, and storage medium for calculating vehicle heading angles that can at least partially solve the aforementioned problems existing in the prior art.

[0006] According to one aspect of this application, a method for calculating the heading angle of a vehicle is provided. The method may include: acquiring an initial image based on a monocular camera, wherein the initial image includes a target vehicle; preprocessing the initial image to obtain a preprocessed image, wherein the preprocessed image includes envelope information, including a whole vehicle envelope, a front envelope, a rear envelope, and wheel envelopes; detecting the number of wheel envelopes contained in the preprocessed image; in response to the number of wheel envelopes being less than two, calculating an intersection-over-union ratio (IoU) and obtaining the heading angle of the target vehicle based on the IoU; and in response to the number of wheel envelopes being greater than or equal to two, calculating the heading angle of the target vehicle based on the envelope information.

[0007] In one embodiment of this application, calculating the intersection-union ratio (IUGR) and obtaining the orientation angle of the target vehicle based on the IUGR may include: obtaining the IUGR based on the envelope information, wherein the IUGR is the ratio of the front or rear envelope of the vehicle to the overall vehicle envelope; determining the local orientation angle of the target vehicle based on the IUGR and the envelope information; obtaining the incident angle of the target vehicle relative to the monocular camera; and determining the orientation angle of the target vehicle based on the local orientation angle and the incident angle, wherein θ g =θ l +α, θ g Let θ be the orientation angle of the target vehicle. l Let α be the local orientation angle, and α be the incident angle.

[0008] In one embodiment of this application, determining the local orientation angle of the target vehicle based on the intersection-union ratio (IU) and the envelope information may include: classifying the preprocessed image based on the envelope information and obtaining a corresponding classification result; determining the local orientation angle of the target vehicle based on the classification result; wherein the preprocessed image includes the whole vehicle envelope and the front vehicle envelope, and if the right side of the whole vehicle envelope coincides with the right side of the front vehicle envelope, the local orientation angle is determined. Wherein, IOU is the intersection-union ratio; if the whole vehicle envelope coincides with the left end of the front vehicle envelope, the local orientation angle is... The preprocessed image includes the whole vehicle envelope and the rear vehicle envelope. If the right side of the whole vehicle envelope coincides with the right side of the rear vehicle envelope, the local orientation angle... If the entire vehicle envelope coincides with the left end of the rear vehicle envelope, the local orientation angle

[0009] In one embodiment of this application, calculating the orientation angle of the target vehicle based on the envelope information may include: obtaining the pixel coordinates of the center point of the lower boundary of the wheel envelope; converting the pixel coordinates of the center point into world coordinates in the corresponding world coordinate system; and determining the orientation angle of the target vehicle in the world coordinate system based on the world coordinates of the center point, the front envelope, and / or the rear envelope.

[0010] This application also provides a vehicle heading angle calculation device, which may include: an image acquisition module for acquiring an initial image based on a monocular camera, wherein the initial image includes a target vehicle; an image preprocessing module for preprocessing the initial image to obtain a preprocessed image, wherein the preprocessed image includes envelope information, including a whole vehicle envelope, a front envelope, a rear envelope, and wheel envelopes; and an angle calculation module for detecting the number of wheel envelopes contained in the preprocessed image, and, in response to the number of wheel envelopes being less than 2, calculating an intersection-over-union ratio (IoU) and obtaining the heading angle of the target vehicle based on the IoU; and, in response to the number of wheel envelopes being greater than or equal to 2, calculating the heading angle of the target vehicle based on the envelope information.

[0011] In one embodiment of this application, the angle calculation module can be used to: obtain the intersection-union ratio (IU / R) based on the envelope information, wherein the IU / R is the ratio of the front or rear vehicle envelope to the overall vehicle envelope; determine the local orientation angle of the target vehicle based on the IU / R and the envelope information; obtain the incident angle of the target vehicle relative to the monocular camera; and determine the orientation angle of the target vehicle based on the local orientation angle and the incident angle, wherein θ g =θ l +α, θ g Let θ be the orientation angle of the target vehicle. l Let α be the local orientation angle, and α be the incident angle.

[0012] In one embodiment of this application, the angle calculation module can also be used to: classify the preprocessed image based on the envelope information and obtain a corresponding classification result; determine the local orientation angle of the target vehicle based on the classification result; wherein, the preprocessed image includes the whole vehicle envelope and the front vehicle envelope, and if the right side of the whole vehicle envelope coincides with the right side of the front vehicle envelope, the local orientation angle is determined. Wherein, IOU is the intersection-union ratio; if the whole vehicle envelope coincides with the left end of the front vehicle envelope, the local orientation angle is... The preprocessed image includes the whole vehicle envelope and the rear vehicle envelope. If the right side of the whole vehicle envelope coincides with the right side of the rear vehicle envelope, the local orientation angle... If the entire vehicle envelope coincides with the left end of the rear vehicle envelope, the local orientation angle

[0013] In one embodiment of this application, the angle calculation module can also be used to: obtain the pixel coordinates of the center point of the lower boundary of the wheel envelope; convert the pixel coordinates of the center point into world coordinates of the corresponding world coordinate system; and determine the orientation angle of the target vehicle based on the world coordinates of the center point, the front envelope, and / or the rear envelope in the world coordinate system.

[0014] In another aspect, this application provides an electronic device, which may include: a processor adapted to execute a computer program; and a computer-readable storage medium storing a computer program, which, when executed by the processor, implements the vehicle heading angle calculation method described above.

[0015] In another aspect, this application provides a computer-readable storage medium for storing a computer program that causes a computer to execute the vehicle heading angle calculation method described above.

[0016] According to an exemplary embodiment of this application, an initial image acquired by a monocular camera is preprocessed to obtain a preprocessed image. The envelope information in the preprocessed image is used to classify the images based on the number of wheel envelopes contained within them. If the number of wheel envelopes is less than two, the orientation angle of the target vehicle is directly obtained based on the intersection-union ratio (IU). If the number of wheel envelopes is greater than or equal to two, the orientation angle of the target vehicle is calculated based on the envelope information. Acquiring the initial image using a monocular camera is low-cost, and the generated initial image is easy to calibrate and identify. Furthermore, the classification calculation method is simple and accurate, which can improve the efficiency of obtaining the orientation angle of the target vehicle to a certain extent and ensure vehicle driving safety. Attached Figure Description

[0017] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. Wherein:

[0018] Figure 1 This is a flowchart of a vehicle orientation angle calculation method 1000 according to an embodiment of this application;

[0019] Figure 2 A flowchart illustrating the process of obtaining the target vehicle's orientation angle based on the intersection-union ratio according to an exemplary embodiment of this application;

[0020] Figure 3 This is a schematic diagram showing the relative positions of the vehicle envelope and the front envelope according to an exemplary embodiment of this application;

[0021] Figure 4 This is a flowchart illustrating the calculation of the local orientation angle of a target vehicle according to an exemplary embodiment of this application;

[0022] Figure 5 This is a schematic diagram of the target vehicle's facing angle and incident angle according to an exemplary embodiment of this application;

[0023] Figure 6 A flowchart for obtaining the orientation angle of a target vehicle when the number of wheel envelope frames is greater than or equal to two according to an exemplary embodiment of this application;

[0024] Figure 7 This is a block diagram of a vehicle orientation angle calculation device 2000 according to an embodiment of this application;

[0025] Figure 8 This is a schematic diagram of the structure of an electronic device 3000 adapted to implement embodiments of the present disclosure, according to an embodiment of the present disclosure. Detailed Implementation

[0026] To better understand this application, various aspects of this application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are merely illustrative of exemplary embodiments of this application and are not intended to limit the scope of this application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0027] In the accompanying drawings, the size, dimensions, and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are for illustrative purposes only and are not strictly to scale. As used herein, the terms “approximately,” “about,” and similar terms are used to indicate approximation, not degree, and are intended to illustrate inherent deviations in measured or calculated values ​​that will be recognized by one of ordinary skill in the art. Furthermore, the order in which the steps are described in this application does not necessarily indicate the order in which these steps occur in actual operation, unless otherwise expressly defined or deduced from the context.

[0028] It should also be understood that expressions such as "comprising," "including," "having," "containing," and / or "comprising" are open-ended rather than closed-ended expressions in this specification, indicating the presence of the stated features, elements, and / or components, but not excluding the presence of one or more other features, elements, components, and / or combinations thereof. Furthermore, when expressions such as "at least one of..." appear after a list of listed features, they modify the entire list of features, not just individual elements in the list. Additionally, when describing embodiments of this application, the word "may" is used to mean "one or more embodiments of this application." And the term "exemplary" is intended to refer to examples or illustrations.

[0029] Unless otherwise specified, all terms used herein (including engineering and technical terms) shall have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains. It should also be understood that, unless expressly stated herein, terms defined in common dictionaries shall be interpreted as having the meaning consistent with their meaning in the context of the relevant art, and not as having an idealized or overly formalized meaning.

[0030] It should be noted that, where there is no conflict, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0031] Figure 1 This is a flowchart of a vehicle orientation angle calculation method 1000 according to an embodiment of this application. Figure 1 As shown, the vehicle heading angle calculation method 1000 may include:

[0032] Step S100: Acquire an initial image based on a monocular camera, wherein the initial image includes the target vehicle;

[0033] Step S200: Preprocess the initial image to obtain a preprocessed image, wherein the preprocessed image contains bounding box information, including the whole vehicle bounding box, the front bounding box, the rear bounding box, and the wheel bounding box.

[0034] Step S300: Detect the number of wheel envelopes in the preprocessed image. If the number of wheel envelopes is less than 2, calculate the cross-union ratio (CURBR) and obtain the orientation angle of the target vehicle based on the CURBR. If the number of wheel envelopes is greater than or equal to 2, calculate the orientation angle of the target vehicle based on the envelope information.

[0035] The following will detail the specific steps of the above-mentioned vehicle orientation angle calculation method 1000.

[0036] In an exemplary embodiment of this application, in step S100, an initial image is first acquired based on a monocular camera, wherein the initial image includes the target vehicle. An image acquisition device, which is a monocular camera, can be configured on the vehicle. The vehicle can acquire images of its surrounding environment using the monocular camera to obtain the initial image, wherein the initial image contains at least one target vehicle whose orientation angle needs to be calculated. The initial image can be an independent image or a video frame from a video stream.

[0037] According to an exemplary embodiment of this application, an initial image is acquired by a monocular camera. The monocular camera has a simple structure, low cost, and the generated initial image is easy to calibrate and identify.

[0038] In an exemplary embodiment of this application, in step S200, the initial image can be preprocessed to obtain a preprocessed image. The preprocessed image includes bounding box information, which includes a vehicle bounding box, a front bounding box, a rear bounding box, and wheel bounding boxes. For example, the target vehicle contained in the initial image can be labeled, for example, the front, rear, and wheels of the target vehicle can be labeled. Then, a first-order deep learning network can be used to obtain a rectangular bounding box of the target vehicle in the pixel coordinate system, and the initial image can be cropped based on the rectangular bounding box to obtain a first image, wherein the first image contains the target vehicle. Further, a second-order deep learning network is used to process the first image to obtain a preprocessed image, which includes bounding box information. This bounding box information includes at least one of the vehicle bounding box and either the front or rear bounding box, and may also include wheel bounding boxes. The vehicle envelope is defined as the smallest rectangle containing the target vehicle. If the target vehicle is obscured, the vehicle envelope is the smallest rectangle containing the portion of the target vehicle that is not obscured by obstacles. The front rectangle is the rectangle encompassing the frontmost part of the target vehicle up to the headlights. The rear rectangle is the rectangle encompassing the rearmost part of the target vehicle up to the taillights. The wheel rectangle is the smallest rectangle containing the wheels. The front and rear rectangles have the same width as the vehicle envelope, and at least three sides of the front and rear rectangles overlap with the vehicle envelope.

[0039] According to an exemplary embodiment of this application, a preprocessed image is obtained by calibrating the entire vehicle, its front or rear, and its wheels. The preprocessed image includes envelope information. The envelope information allows for a simpler and faster determination of the target vehicle's heading angle calculation method and the corresponding angle calculation in subsequent processes.

[0040] In an exemplary embodiment of this application, after obtaining the preprocessed image, step S300 can be performed to detect the number of wheel envelope frames contained in the preprocessed image, and to select different methods to calculate the heading angle of the target vehicle based on the detection results. When the number of wheel envelope frames is less than 2, the intersection-over-union ratio (IoU) is calculated, and the heading angle of the target vehicle is obtained based on the IoU; when the number of wheel envelope frames is greater than or equal to 2, the heading angle of the target vehicle is calculated based on the envelope frame information.

[0041] Figure 2 This is a flowchart illustrating the process of obtaining the target vehicle's orientation angle based on the intersection-union ratio (IU / R) according to an exemplary embodiment of this application. Figure 2 As shown, obtaining the target vehicle's orientation angle based on the intersection-union ratio may include the following steps:

[0042] Step S310: Obtain the intersection-union ratio based on the envelope information, wherein the intersection-union ratio is the ratio of the front or rear envelope of the vehicle to the whole vehicle envelope;

[0043] Step S320: Determine the local orientation angle of the target vehicle based on the intersection-union ratio and envelope information;

[0044] Step S330: Obtain the angle of incidence of the target vehicle with respect to the monocular camera;

[0045] Step S340: Determine the target vehicle's orientation angle based on the local orientation angle and the incident angle, where θ g =θ l +α, θ g Let θ be the heading angle of the target vehicle. l Let α be the local orientation angle and α be the incident angle.

[0046] For example, the intersection-union ratio (CIRR) can first be obtained based on the envelope information, where the CIRR is the ratio of the front or rear vehicle envelope to the overall vehicle envelope. The width of the front or rear vehicle envelope is the same as the overall vehicle envelope, and one end of the front or rear vehicle envelope completely overlaps with the overall vehicle envelope. Therefore, the CIRR is the ratio of the area of ​​the front or rear vehicle envelope to the area of ​​the overall vehicle envelope, or the ratio of the length of the front or rear vehicle envelope to the length of the overall vehicle envelope. The overall vehicle envelope and the front vehicle envelope are used as an example for illustration. Figure 3 This is a schematic diagram showing the relative positions of the vehicle envelope and the front envelope according to an exemplary embodiment of this application. Figure 3 As shown, the shaded area is the vehicle front envelope, with an area of ​​A. The shaded and blank areas constitute the overall vehicle envelope, with an area of ​​A+B. The vehicle front envelope and the overall vehicle envelope have the same height, and their right edges coincide. Therefore, the intersection-over-union ratio (IoU) between the vehicle front envelope and the overall vehicle envelope is A / (A+B). This application calculates the IoU between the overall vehicle envelope and the vehicle front envelope using an area ratio. Those skilled in the art will understand that other methods can also be used for calculation. Furthermore, the local orientation angle of the target vehicle is determined based on the IoU and envelope information. Figure 4 This is a flowchart illustrating the calculation of a target vehicle's local orientation angle according to an exemplary embodiment of this application. Figure 4 As shown, calculating the local orientation angle of the target vehicle may include the following steps:

[0047] Step S321: Classify the preprocessed image based on the envelope information and obtain the corresponding classification results;

[0048] Step S322: Determine the local orientation angle of the target vehicle based on the classification results; wherein, the preprocessed image includes the whole vehicle envelope and the front vehicle envelope. If the right side of the whole vehicle envelope coincides with the right side of the front vehicle envelope, the local orientation angle is determined. Where IOU stands for Intersection over Union (IOU); if the entire vehicle envelope coincides with the left end of the front vehicle envelope, the local orientation angle is... The preprocessed image includes the overall vehicle envelope and the rear vehicle envelope. If the right side of the overall vehicle envelope coincides with the right side of the rear vehicle envelope, the local orientation angle is... If the left end of the vehicle's overall envelope coincides with the left end of the rear envelope, the local orientation angle...

[0049] For example, the preprocessed image is classified based on the front or rear vehicle envelope contained therein, and corresponding classification results are obtained. The classification results may include a first classification result and a second classification result. The first classification result indicates that the preprocessed image contains both a front vehicle envelope and a full vehicle envelope; the second classification result indicates that the preprocessed image contains both a rear vehicle envelope and a full vehicle envelope. In the first classification result, the relative positions of the front vehicle envelope and the full vehicle envelope are detected. If the right side of the full vehicle envelope coincides with the right side of the front vehicle envelope (e.g., ...), the classification is performed accordingly. Figure 3 ), local orientation angle If the entire vehicle's envelope coincides with the left end of the front vehicle's envelope, the local orientation angle... Where IOU is the intersection-union ratio of the front-end envelope and the overall vehicle envelope. In the second classification result, the relative position of the rear-end envelope and the overall vehicle envelope is detected. If the right side of the overall vehicle envelope coincides with the right side of the rear-end envelope, the local orientation angle is considered. If the left end of the vehicle's overall envelope coincides with the left end of the rear envelope, the local orientation angle...

[0050] Figure 5 This is a schematic diagram showing the target vehicle's facing angle and incident angle according to an exemplary embodiment of this application. Figure 5 As shown, the target vehicle faces at an angle θ. g The incident angle α is the angle between the vehicle's direction of travel and the horizontal direction, and the angle between the line connecting the center of the target vehicle's entire vehicle envelope and the center of the monocular camera's lens and the lens's optical axis. In step S320, the local orientation angle θ of the target vehicle can be obtained. g The incident angle α can be calculated using the focal length f and imaging distance d of a monocular camera, where... The target vehicle's orientation angle is determined based on the local orientation angle and the incident angle, where θ g =θ l +α.

[0051] According to an exemplary embodiment of this application, by detecting the number of wheel envelopes contained in the preprocessed image, the orientation angle of a target vehicle with fewer than two wheel envelopes can be obtained based on the intersection-union ratio. The calculation method is simple and accurate, which can improve the efficiency of obtaining the orientation angle of the target vehicle to a certain extent and ensure the driving safety of the vehicle.

[0052] In an exemplary embodiment of this application, when the number of wheel envelope frames is greater than or equal to 2, the orientation angle of the target vehicle is calculated based on the envelope frame information. Figure 6 This is a flowchart illustrating how to obtain the heading angle of a target vehicle when the number of wheel envelope frames is greater than or equal to two, according to an exemplary embodiment of this application. Figure 6 As shown, when the number of wheel envelope frames is greater than or equal to 2, obtaining the orientation angle of the target vehicle may include the following steps:

[0053] Step S350: Obtain the pixel coordinates of the center point of the lower boundary of the wheel envelope;

[0054] Step S360: Convert the pixel coordinates of the center point to world coordinates in the corresponding world coordinate system;

[0055] Step S370: In the world coordinate system, determine the orientation angle of the target vehicle based on the world coordinates of the center point, the front envelope, and / or the rear envelope.

[0056] For example, based on the wheel envelope marked in the preprocessed image, the pixel coordinates of the center point of the lower edge of the wheel envelope are obtained, and the pixel coordinates of the center point of the lower edge are converted into world coordinates in the corresponding world coordinate system. By connecting the center points of the lower edge of the wheel envelope, the two orientations of the target vehicle can be determined, obtaining the first orientation angle and the second orientation angle. To further determine the orientation of the target vehicle, the orientation angle of the target vehicle can be further determined by the front and / or rear envelopes. Specifically, determining the orientation angle of the target vehicle by the front and / or rear envelopes can be done as in steps S321 and S322, using the front or rear envelopes to determine the estimated value of the orientation angle of the target vehicle, and then comparing the first and second orientation angles with the estimated values ​​of the orientation angles of the target vehicle, respectively. The orientation angle with the smallest difference between the first and second orientation angles and the estimated values ​​of the orientation angles is selected as the final orientation angle of the target vehicle.

[0057] According to an exemplary embodiment of this application, by obtaining the pixel coordinates of the center point of the lower edge of the wheel envelope and converting them into world coordinates, the orientation angle of the target vehicle can be further determined through the envelope information. The calculation method is simple and accurate, which can improve the efficiency of obtaining the orientation angle of the target vehicle to a certain extent and ensure the driving safety of the vehicle.

[0058] This application also provides a vehicle heading angle calculation device 2000. Figure 7 This is a block diagram of a vehicle orientation angle calculation device 2000 according to an embodiment of this application. Figure 7 As shown, the vehicle orientation angle calculation device 2000 includes an image acquisition module 2100, an image preprocessing module 2200, and an angle calculation module 2300.

[0059] In an exemplary embodiment of this application, the image acquisition module 2100 can be used to acquire an initial image based on a monocular camera, wherein the initial image includes a target vehicle. An image acquisition device, which is a monocular camera, can be configured on the vehicle. The vehicle can acquire environmental images around itself using the monocular camera to obtain the initial image, wherein the initial image contains at least one target vehicle whose orientation angle needs to be calculated. The initial image can be an independent image or a video frame from a video stream.

[0060] According to an exemplary embodiment of this application, an initial image is acquired by a monocular camera. The monocular camera has a simple structure, low cost, and the generated initial image is easy to calibrate and identify.

[0061] In an exemplary embodiment of this application, the image preprocessing module 2200 can be used to preprocess an initial image to obtain a preprocessed image. The preprocessed image includes bounding box information, which includes a whole vehicle bounding box, a front bounding box, a rear bounding box, and wheel bounding boxes. For example, the target vehicle contained in the initial image can be labeled, for example, the front, rear, and wheels of the target vehicle can be labeled. Then, a first-order deep learning network can be used to obtain a rectangular bounding box of the target vehicle in the pixel coordinate system, and the initial image can be cropped based on the rectangular bounding box to obtain a first image, wherein the first image contains the target vehicle. Further, a second-order deep learning network is used to process the first image to obtain a preprocessed image, wherein the preprocessed image includes bounding box information. The bounding box information includes at least one of the whole vehicle bounding box and either the front or rear bounding box, and may also include wheel bounding boxes. The vehicle envelope is defined as the smallest rectangle containing the target vehicle. If the target vehicle is obscured, the vehicle envelope is the smallest rectangle containing the portion of the target vehicle that is not obscured by obstacles. The front rectangle is the rectangle encompassing the frontmost part of the target vehicle up to the headlights. The rear rectangle is the rectangle encompassing the rearmost part of the target vehicle up to the taillights. The wheel rectangle is the smallest rectangle containing the wheels. The front and rear rectangles have the same width as the vehicle envelope, and at least three sides of the front and rear rectangles overlap with the vehicle envelope.

[0062] According to an exemplary embodiment of this application, a preprocessed image is obtained by calibrating the entire vehicle, its front or rear, and its wheels. The preprocessed image includes envelope information. The envelope information allows for a simpler and faster determination of the target vehicle's heading angle calculation method and the corresponding angle calculation in subsequent processes.

[0063] In an exemplary embodiment of this application, the angle calculation module 2300 can be used to detect the number of wheel envelope frames contained in the preprocessed image. If the number of wheel envelope frames is less than two, the intersection-over-union ratio (IoU) is calculated, and the orientation angle of the target vehicle is obtained based on the IoU. If the number of wheel envelope frames is greater than or equal to two, the orientation angle of the target vehicle is calculated based on the envelope frame information. Obtaining the orientation angle of the target vehicle based on the IoU may include: obtaining the IoU based on the envelope frame information, wherein the IoU is the ratio of the front or rear vehicle envelope frame to the overall vehicle envelope frame; determining the local orientation angle of the target vehicle based on the IoU and the envelope frame information; obtaining the incident angle of the target vehicle to the monocular camera; and determining the orientation angle of the target vehicle based on the local orientation angle and the incident angle, wherein θ g =θ l +α, θ g Let θ be the heading angle of the target vehicle. l Let α be the local orientation angle and α be the incident angle.

[0064] For example, the intersection-union ratio (IUGR) can first be obtained based on the envelope information, where the IUGR is the ratio of the front or rear vehicle envelope to the overall vehicle envelope. The width of the front or rear vehicle envelope is the same as the overall vehicle envelope, and one end of the front or rear vehicle envelope completely overlaps with the overall vehicle envelope. Therefore, the IUGR is the ratio of the area of ​​the front or rear vehicle envelope to the area of ​​the overall vehicle envelope, or the ratio of the length of the front or rear vehicle envelope to the length of the overall vehicle envelope. This will be illustrated using the overall vehicle envelope and the front vehicle envelope as an example. Figure 3 As shown, the shaded area is the vehicle front envelope with area A, and the shaded and blank areas are the whole vehicle envelope with area A+B. The vehicle front envelope and the whole vehicle envelope have the same height, and their right ends coincide. Therefore, the intersection-union ratio (IUR) between the vehicle front envelope and the whole vehicle envelope is A / (A+B). This application calculates the IUR between the whole vehicle envelope and the vehicle front envelope by the ratio of their areas. Those skilled in the art will know that other methods can also be used for calculation. Further, the local orientation angle of the target vehicle is determined based on the IUR and the envelope information. Calculating the local orientation angle of the target vehicle may include: classifying the preprocessed image based on the envelope information and obtaining the corresponding classification result; determining the local orientation angle of the target vehicle based on the classification result; wherein the preprocessed image includes the whole vehicle envelope and the vehicle front envelope. If the right end of the whole vehicle envelope coincides with the right end of the vehicle front envelope, the local orientation angle... Where IOU stands for Intersection over Union (IOU); if the entire vehicle envelope coincides with the left end of the front vehicle envelope, the local orientation angle is... The preprocessed image includes the overall vehicle envelope and the rear vehicle envelope. If the right side of the overall vehicle envelope coincides with the right side of the rear vehicle envelope, the local orientation angle is... If the left end of the vehicle's overall envelope coincides with the left end of the rear envelope, the local orientation angle...

[0065] For example, the preprocessed image is classified based on the front or rear vehicle envelope contained therein, and corresponding classification results are obtained. The classification results may include a first classification result and a second classification result. The first classification result indicates that the preprocessed image contains both a front vehicle envelope and a full vehicle envelope; the second classification result indicates that the preprocessed image contains both a rear vehicle envelope and a full vehicle envelope. In the first classification result, the relative positions of the front vehicle envelope and the full vehicle envelope are detected. If the right side of the full vehicle envelope coincides with the right side of the front vehicle envelope (e.g., ...), the classification is performed accordingly. Figure 3 ), local orientation angle If the entire vehicle's envelope coincides with the left end of the front vehicle's envelope, the local orientation angle... Where IOU is the intersection-union ratio of the front-end envelope and the overall vehicle envelope. In the second classification result, the relative position of the rear-end envelope and the overall vehicle envelope is detected. If the right side of the overall vehicle envelope coincides with the right side of the rear-end envelope, the local orientation angle is considered. If the left end of the vehicle's overall envelope coincides with the left end of the rear envelope, the local orientation angle...

[0066] like Figure 5 As shown, the target vehicle faces at an angle θ. g Let α be the angle between the vehicle's direction of travel and the horizontal direction, and let α be the angle between the line connecting the center of the target vehicle's entire vehicle envelope and the center of the monocular camera's lens and the lens's optical axis. The local orientation angle θ of the target vehicle is then obtained. g The incident angle α can be calculated using the focal length f and imaging distance d of a monocular camera, where... The target vehicle's orientation angle is determined based on the local orientation angle and the incident angle, where θ g =θ l +α.

[0067] According to an exemplary embodiment of this application, by detecting the number of wheel envelopes contained in the preprocessed image, the orientation angle of a target vehicle with fewer than two wheel envelopes can be obtained based on the intersection-union ratio. The calculation method is simple and accurate, which can improve the efficiency of obtaining the orientation angle of the target vehicle to a certain extent and ensure the driving safety of the vehicle.

[0068] In an exemplary embodiment of this application, when the number of wheel envelope frames is greater than or equal to two, the heading angle of the target vehicle is calculated based on the envelope frame information. When the number of wheel envelope frames is greater than or equal to two, obtaining the heading angle of the target vehicle may include: obtaining the pixel coordinates of the center point of the lower boundary of the wheel envelope frame; converting the pixel coordinates of the center point into world coordinates in the corresponding world coordinate system; and determining the heading angle of the target vehicle in the world coordinate system based on the world coordinates of the center point, the front envelope frame, and / or the rear envelope frame.

[0069] For example, based on the wheel envelope marked in the preprocessed image, the pixel coordinates of the center point of the lower edge of the wheel envelope are obtained, and the pixel coordinates of the center point of the lower edge are converted into world coordinates in the corresponding world coordinate system. By connecting the center points of the lower edge of the wheel envelope, the two orientations of the target vehicle can be determined, obtaining the first orientation angle and the second orientation angle. To further determine the orientation of the target vehicle, the orientation angle of the target vehicle can be further determined by the front and / or rear envelopes. Specifically, the orientation angle of the target vehicle is determined by the front and / or rear envelopes, and an estimated value of the orientation angle of the target vehicle is determined using the front or rear envelopes. Then, the first and second orientation angles are compared with the estimated values ​​of the orientation angles of the target vehicle, and the orientation angle with the smallest difference between the first and second orientation angles and the estimated values ​​is selected as the final orientation angle of the target vehicle.

[0070] According to an exemplary embodiment of this application, by obtaining the pixel coordinates of the center point of the lower edge of the wheel envelope and converting them into world coordinates, the orientation angle of the target vehicle can be further determined through the envelope information. The calculation method is simple and accurate, which can improve the efficiency of obtaining the orientation angle of the target vehicle to a certain extent and ensure the driving safety of the vehicle.

[0071] This application also provides an electronic device and a computer-readable storage medium. Figure 8 This is a schematic diagram of the structure of an electronic device 3000 adapted to implement embodiments of the present disclosure, according to an embodiment of the present disclosure.

[0072] The following is for reference. Figure 8 The diagram illustrates a structural schematic of an electronic device 3000 suitable for implementing embodiments of the present disclosure. Terminal devices in embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 8 The terminal device / server shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0073] like Figure 8 As shown, the electronic device 3000 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 3100, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 3200 or a program loaded from a storage device 3800 into a random access memory (RAM) 3300. The RAM 3300 also stores various programs and data required for the operation of the electronic device 3000. The processing unit 3100, the ROM 3200, and the RAM 3300 are interconnected via a bus 3400. An input / output (I / O) interface 3500 is also connected to the bus 3400.

[0074] Typically, the following devices can be connected to the I / O interface 3500: input devices 3600 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 3700 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 3800 including, for example, magnetic tapes, hard disks, etc.; and communication devices 3900. The communication device 3900 allows the electronic device 3000 to communicate wirelessly or wiredly with other devices to exchange data. Although... Figure 8 An electronic device 3000 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 8 Each box shown can represent a device or multiple devices as needed.

[0075] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 3900, or installed from a storage device 3800, or installed from a ROM 3200. When the computer program is executed by a processing device 3100, it performs the functions defined in the methods of embodiments of this disclosure.

[0076] It should be noted that the computer-readable medium described in the embodiments of this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In the embodiments of this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the embodiments of this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0077] The aforementioned computer-readable medium may be included in the aforementioned electronic device or may exist independently without being assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, enable the electronic device to: acquire an initial image based on a monocular camera, wherein the initial image includes a target vehicle; preprocess the initial image to obtain a preprocessed image, wherein the preprocessed image includes envelope information, including a whole vehicle envelope, a front envelope, a rear envelope, and wheel envelopes; detect the number of wheel envelopes contained in the preprocessed image; if the number of wheel envelopes is less than two, calculate the intersection-over-union ratio (IoU) and obtain the orientation angle of the target vehicle based on the IoU; if the number of wheel envelopes is greater than or equal to two, calculate the orientation angle of the target vehicle based on the envelope information.

[0078] Computer program code for performing the operations of embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0079] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0080] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for calculating a vehicle heading angle, characterized in that, The method includes: An initial image is acquired based on a monocular camera, wherein the initial image includes the target vehicle; The initial image is preprocessed to obtain a preprocessed image, wherein the preprocessed image contains envelope information, including the whole vehicle envelope, the front envelope, the rear envelope, and the wheel envelope. The number of wheel envelopes contained in the preprocessed image is detected. If the number of wheel envelopes is less than 2, the intersection-over-union ratio (IoU) is calculated, and the orientation angle of the target vehicle is obtained based on the IoU. If the number of wheel envelopes is greater than or equal to 2, the orientation angle of the target vehicle is calculated based on the envelope information. Calculating the intersection-to-union ratio and obtaining the orientation angle of the target vehicle based on the intersection-to-union ratio includes: The intersection-union ratio (IUGR) is obtained based on the envelope information, wherein the IUGR is the ratio of the front or rear envelope of the vehicle to the overall vehicle envelope. The local orientation angle of the target vehicle is determined based on the intersection-union ratio and the envelope information; Obtain the angle of incidence of the target vehicle with respect to the monocular camera; The orientation angle of the target vehicle is determined based on the local orientation angle and the incident angle, wherein, , The heading angle of the target vehicle. The local orientation angle is... The incident angle; Determining the local orientation angle of the target vehicle based on the intersection-union ratio and the envelope information includes: The preprocessed image is classified based on the envelope information, and the corresponding classification result is obtained; The local orientation angle of the target vehicle is determined based on the classification result; wherein, the preprocessed image includes the whole vehicle envelope and the front vehicle envelope, and if the right side of the whole vehicle envelope coincides with the right side of the front vehicle envelope, the local orientation angle is determined. Where IOU is the intersection-union ratio; if the whole vehicle envelope coincides with the left end of the front vehicle envelope, the local orientation angle The preprocessed image includes the whole vehicle envelope and the rear vehicle envelope. If the right side of the whole vehicle envelope coincides with the right side of the rear vehicle envelope, the local orientation angle... If the entire vehicle envelope coincides with the left end of the rear vehicle envelope, the local orientation angle ; Calculating the orientation angle of the target vehicle based on the envelope information includes: Obtain the pixel coordinates of the center point of the lower boundary of the wheel envelope; Convert the pixel coordinates of the center point to world coordinates in the corresponding world coordinate system; In the world coordinate system, the orientation angle of the target vehicle is determined based on the world coordinates of the center point, the front envelope, and / or the rear envelope.

2. A vehicle heading angle calculation device, characterized in that, The device includes: An image acquisition module is used to acquire an initial image based on a monocular camera, wherein the initial image includes the target vehicle; An image preprocessing module is used to preprocess the initial image to obtain a preprocessed image, wherein the preprocessed image includes envelope information, and the envelope information includes the whole vehicle envelope, the front envelope, the rear envelope, and the wheel envelope. An angle calculation module is used to detect the number of wheel envelopes contained in the preprocessed image; in response to the number of wheel envelopes being less than 2, it calculates the intersection-union ratio (IUGR) and obtains the orientation angle of the target vehicle based on the IUGR; in response to the number of wheel envelopes being greater than or equal to 2, it calculates the orientation angle of the target vehicle based on the envelope information. The angle calculation module is used for: The intersection-union ratio (IUGR) is obtained based on the envelope information, wherein the IUGR is the ratio of the front or rear envelope of the vehicle to the overall vehicle envelope. The local orientation angle of the target vehicle is determined based on the intersection-union ratio and the envelope information; Obtain the angle of incidence of the target vehicle with respect to the monocular camera; The orientation angle of the target vehicle is determined based on the local orientation angle and the incident angle, wherein, , The heading angle of the target vehicle. The local orientation angle is... The incident angle; The angle calculation module is also used for: The preprocessed image is classified based on the envelope information, and the corresponding classification result is obtained; The local orientation angle of the target vehicle is determined based on the classification result; wherein, the preprocessed image includes the whole vehicle envelope and the front vehicle envelope, and if the right side of the whole vehicle envelope coincides with the right side of the front vehicle envelope, the local orientation angle is determined. Where IOU is the intersection-union ratio; if the whole vehicle envelope coincides with the left end of the front vehicle envelope, the local orientation angle The preprocessed image includes the whole vehicle envelope and the rear vehicle envelope. If the right side of the whole vehicle envelope coincides with the right side of the rear vehicle envelope, the local orientation angle... If the entire vehicle envelope coincides with the left end of the rear vehicle envelope, the local orientation angle ; The angle calculation module is also used for: Obtain the pixel coordinates of the center point of the lower boundary of the wheel envelope; Convert the pixel coordinates of the center point to world coordinates in the corresponding world coordinate system; In the world coordinate system, the orientation angle of the target vehicle is determined based on the world coordinates of the center point, the front envelope, and / or the rear envelope.

3. An electronic device, characterized in that, include: A processor, adapted to execute computer programs; as well as A computer-readable storage medium storing a computer program that, when executed by the processor, implements the vehicle orientation angle calculation method as described in claim 1.

4. A computer-readable storage medium, characterized in that, Used to store a computer program that causes a computer to perform the vehicle heading angle calculation method as described in claim 1.

Images