Method of installation height estimation and method of driver assistance system algorithm update

Abstract

In a method of installation height estimation, a computer device is configured to perform the steps of: obtaining multiple vehicle front images; using a vehicle location and dimension recognition model to obtain image coordinate information, a real vehicle width and a real vehicle height of a target vehicle in each vehicle front image; for each vehicle front image, based on the corresponding image coordinate information, the real vehicle width, the real vehicle height, a center point image coordinate and a horizon image coordinate, using the pinhole camera model to obtain a candidate installation height from a plurality of preset installation heights; and using a regression analysis method to obtain a real installation height from the candidate installation heights.

Description

Technical Field

[0001] This invention relates to a height estimation method, and more particularly to a method for estimating the installation height of an automotive camera module and a method for updating a driver assistance system formula derived from the installation height estimation method. Prior Technology

[0002] Currently, driver assistance systems incorporating vision processing technology are quite mature, and are used in applications such as lane departure detection, pedestrian detection, forward vehicle departure warning, and surround view monitoring. These applications mostly rely on camera calibration technology to determine the relationship between the position of an object in the real world in three-dimensional space and its corresponding position in a two-dimensional image. That is, the position of the object is transformed between the world coordinate system, the camera coordinate system, and finally the image coordinate system.

[0003] The accuracy of the results obtained using camera calibration technology regarding the aforementioned driver assistance systems depends on the initial camera setup. Besides adjusting the camera's intrinsic parameters, the camera's mounting height must also be adjusted. However, even cameras of the same model can have slight differences in their intrinsic parameters. Furthermore, after installation, the camera's mounting height may change due to accidental impacts or road bumps, affecting the accuracy of calculations within the driver assistance system's algorithms. Moreover, changes in camera mounting height are not easily noticeable to the naked eye; measuring them requires tools or assistance from a technician at the vehicle manufacturer, leading to reduced accuracy and a poorer overall user experience. Even if the user notices that the camera's mounting height has been moved and needs correct adjustment, it is practically untimely, time-consuming, and laborious.

[0004] Therefore, how to conveniently and quickly obtain the current installation height of the camera has become one of the issues that related technical fields want to solve. Summary of the Invention

[0005] Therefore, the object of the present invention is to provide a method for estimating installation height and a method for updating the formula of a driving assistance system, which can overcome at least one drawback of the prior art.

[0006] Therefore, the installation height estimation method provided by this invention can be used to dynamically and in real-time estimate the installation height of a camera module installed in a vehicle. This method is executed using a computer device that pre-establishes a vehicle position and size estimation model. This model is used to obtain the image coordinate information of the bounding box corresponding to each vehicle in the image within an image coordinate system defined according to a pinhole camera model, as well as the predicted true width and predicted true height of each vehicle in the image. The installation height estimation method includes the following steps: (A) obtaining multiple images of the front of the vehicle taken by the camera module; (B) for each image of the front of the vehicle, using the vehicle position and size estimation model to obtain a target vehicle in the image of the front of the vehicle. (C) For each front-view image of a target vehicle, based on the predicted true width and predicted true height of the target vehicle, obtain first-direction world coordinate information and second-direction world coordinate information of the target vehicle in a first direction and a second direction in a second direction within a world coordinate system defined by the pinhole camera model; (D) Based on the image coordinate information, first-direction world coordinate information, and second-direction world coordinate information of the target vehicle in all front-view images, obtain a lens focal length of the camera module using the pinhole camera model; (E) For each front-view image, based on the lens focal length, the image coordinate information, the predicted true width and predicted true height of the target vehicle, obtain the first-direction world coordinate information and second-direction world coordinate information of the target vehicle in a first direction and a second direction in a second direction within a world coordinate system defined by the pinhole camera model; (F) Obtain a pitch angle of the camera module when capturing the front view of the vehicle, based on the pitch angle corresponding to the front view of the vehicle, the lens focal length, the horizon image coordinate of the front view of the vehicle, the image coordinate information of the target vehicle, and multiple preset installation heights, respectively corresponding to the preset installation heights and relating to the target vehicle in a third-party world coordinate system; (G) For each third-party world coordinate value of the target vehicle in each front view of the vehicle, based on the pitch angle of the front view of the vehicle, the lens focal length, the horizon image coordinate of the front view of the vehicle, the image coordinate information of the target vehicle, and multiple preset installation heights, obtain multiple third-party world coordinate values ​​corresponding to the preset installation heights and relating to the target vehicle in a third-party world coordinate system; The system uses a pinhole camera model to reproject the target vehicle in the image coordinate system, along with world coordinate information in one direction, world coordinate information in the second direction of the target vehicle, an intrinsic parameter data matrix of the camera module, and an extrinsic parameter matrix of the front image of the vehicle. This reprojection yields reprojected image coordinate information corresponding to the bounding box of the reprojected target vehicle and a reprojection error value, where the reprojection error value is the error between the image coordinate information of the target vehicle and the reprojected image coordinate information. (H) For each front image of the vehicle, the preset installation height corresponding to the third-direction world coordinate value with the smallest reprojection error value is taken as a candidate installation height. (I) A true installation height is obtained from these candidate installation heights using a regression analysis method.

[0007] Therefore, the present invention provides a method for updating a driver assistance system formula, implemented in a vehicle, and includes the following steps: (A) acquiring multiple images of the front of the vehicle using a camera module installed in the vehicle; (B) calculating at least one possible installation height of the camera module based on the correlation between the camera module and a target vehicle in the front image data; (C) obtaining a most probable installation height through statistical regression analysis; and (D) updating and replacing an original installation height data in the driver assistance system formula of the vehicle with the most probable installation height.

[0008] The advantages of this invention are as follows: By combining the images of the front of the vehicle captured by the camera module with a pinhole camera model, the actual installation height of the camera module can be deduced, achieving the effect of obtaining the current actual installation height anytime and anywhere, which is more time-saving and labor-saving than using additional tools or going to the vehicle manufacturer for measurement. Furthermore, this invention can be used to monitor the actual installation height of the camera module in real time, allowing users to know changes in the installation height immediately and adjust the position of the camera module accordingly. Simultaneously, the actual installation height data can be used to update and replace the original installation height in the driver assistance system's formula in real time. Using the latest actual installation height, the required values ​​for various functions of the driver assistance system are calculated, ensuring the accuracy of the driver assistance system and avoiding unnecessary false alarms caused by inaccurate calculations or uploading such false alarm information to the cloud. Simple Explanation of the Diagram

[0009] Other features and effects of the present invention will be clearly presented in the embodiments with reference to the drawings, wherein: Figure 1 is a flowchart illustrating an embodiment of the installation height estimation method of the present invention; and Figure 2 is a schematic diagram illustrating, by way of example, a front view of a vehicle captured by a camera module of this embodiment, and the bounding box of a target vehicle detected in the front view. Implementation

[0010] Before the invention is described in detail, it should be noted that similar elements are represented by the same numbers in the following description.

[0011] First, the pinhole camera model used in this case is a widely used algorithm in computer vision, used to map coordinate points in the three-dimensional world onto a two-dimensional image plane. The image coordinate system defined by the pinhole camera model is a rectangular coordinate system with the upper left corner of the image as the origin and pixels as the unit, also known as the image pixel coordinate system.

[0012] Referring to Figure 1, an embodiment of the installation height estimation method of the present invention is used to estimate the installation height of a camera module installed in a vehicle. This is executed using a computer device and includes steps S11-S19. The camera module is, for example, a dashcam used to capture images of the front exterior of the vehicle. The computer device pre-establishes a Vehicle Location and Dimension Recognition Model (VLDR model) to obtain and collect the image coordinate information of the bounding box corresponding to each vehicle in the image within an image coordinate system defined according to a pinhole camera model, as well as the predicted true width and predicted true height of each vehicle in the image. The frame rate of the camera module can be, but is not limited to, capturing 30 images per second as training image data.

[0013] More specifically, the image coordinate information of each bounding box includes the four vertices of the bounding box: the upper right corner, the upper left corner, the lower left corner, and the lower right corner. Image coordinates. Each vertex Image coordinates can be represented as ,in, This indicates the vertex Image coordinates in a first direction along a first direction. This indicates that the vertex... The second direction image coordinates in a second direction.

[0014] This VLDR model utilizes object detection technology, trained on multiple training image datasets using a Convolutional Neural Network (CNN) architecture and a modified YOLO model loss function. Each training image dataset includes a driving scene and the actual dimensions of each vehicle within that scene. These driving scenes are, for example, captured by a dashcam, and the actual dimensions of each vehicle in each scene are obtained from publicly available information from the respective vehicle manufacturer. Notably, the modified loss function adds a dimension loss function to the original YOLO model loss function. The dimension loss function measures the error between the VLDR model's predicted actual width and height of the vehicle in the driving scene and the actual vehicle dimensions obtained from the vehicle manufacturer.

[0015] The world coordinate system defined by the pinhole camera model can be used to describe the spatial position of the camera (i.e., the photographic module) and the object being measured. The position of the world coordinate system can be set according to the actual situation. For example, in the setting of this paper, the world coordinate system takes the center point of the rear of the vehicle located directly in front of the vehicle as the origin, the X-axis points to the horizontal direction (i.e., the first direction referred to in this paper), the Y-axis points to the vertical direction (i.e., the second direction referred to in this paper), and the Z-axis points to the direction of the optical axis of the photographic module (i.e., the third direction referred to in this paper). Since the principle of the pinhole camera model is common knowledge to those skilled in the art, and its application can be implemented using the readily available OpenCV computer vision library, a detailed explanation is omitted here for the sake of brevity.

[0016] The following will describe each step of this embodiment in detail with reference to FIG1.

[0017] In step S11, the computer device acquires multiple images of the front of the vehicle taken by the camera module.

[0018] In step S12, for each front view image, the computer device uses the vehicle position and size estimation model to obtain the image coordinate information of the bounding box corresponding to a target vehicle in the front view image, the predicted true width of the target vehicle, and the predicted true height of the target vehicle. In this embodiment, the target vehicle in each front view image is the vehicle that is completely located in front of the vehicle within the lane line of the vehicle in the front view image.

[0019] For example, as shown in Figure 2, one of these front images is, for example, front image 2. The computer device uses the vehicle position and size estimation model to obtain the image coordinate information (i.e., vertex coordinates) of the bounding box 21 corresponding to a target vehicle A in front image 2. , , and (Image coordinates) and the predicted true width of the target vehicle A And the predicted true vehicle height of the target vehicle A .

[0020] In step S13, for each target vehicle in the front image, the computer device obtains first-direction world coordinate information and second-direction world coordinate information of the target vehicle in the first direction and the second direction in the second direction in the world coordinate system set herein, based on the predicted true width and predicted true height of the target vehicle.

[0021] More specifically, the first-direction world coordinate information of the target vehicle includes each vertex of the target vehicle's bounding box. The first direction world coordinate value, and each vertex The first direction world coordinate value can be represented as The second-direction world coordinate information of the target vehicle includes each vertex of the target vehicle's bounding box. The second direction world coordinate value, and each vertex The second direction world coordinate value can be represented as .

[0022] Continuing with the previous example, referring to Figure 2, the first-direction world coordinate values ​​of the four vertices included in the first-direction world coordinate information of the target vehicle A are as follows: , , and The second-direction world coordinate values ​​of the four vertices contained in the second-direction world coordinate information of the target vehicle A are as follows: , , and .

[0023] In step S14, the computer device, based on the image coordinate information of the target vehicle in all the front images, the first direction world coordinate information, and the second direction world coordinate information, uses a pinhole camera model to reverse-engineer the lens focal length (Fx, Fy) and pixel optical centers (Cx, Cy) of the camera module in the image coordinate system, i.e., the intrinsic matrix of the camera module. Since obtaining camera intrinsic data using a pinhole camera model is well-known to those skilled in the art, it will not be elaborated upon here for the sake of brevity.

[0024] In step S15, for each front-view image, the computer device obtains the pitch angle of the camera module when capturing the front-view image based on the lens focal length, the center point image coordinates of the front-view image in the image coordinate system, and the horizon image coordinates of the front-view image in the image coordinate system, through the relationship of the pinhole camera model. More specifically, the computer device calculates an extrinsic matrix based on the lens focal length, the second-direction image coordinates of the center point image of the front-view image in the second direction, and the second-direction image coordinates of the horizon image of the front-view image in the second direction, according to the side and angle relationships of the similar triangles corresponding to the actual position of the target vehicle and the image plane where the camera module and the front-view image are located.

[0025] In this embodiment, the position of the horizon in each front view is the vanishing point of the road where the target vehicle is located in that front view (see the road vanishing point D in Figure 2). Furthermore, the pitch angle corresponding to each front view is calculated using the following radian angle conversion formula: in, This indicates the pitch angle corresponding to the front view of the vehicle. This indicates the second-direction image coordinate value of the center point of the vehicle's front image in the second direction. This represents the second-direction image coordinate value of the horizon image coordinate of the vehicle's front view in the second direction. This indicates the focal length of the lens.

[0026] In step S16, since the triangular geometric depth value corresponding to the central straight optical axis of the camera module (i.e., the actual distance of the camera module to the vehicle in front) will produce different calculation results depending on the installation height of different cameras, after obtaining the pitch angle corresponding to the front image of the vehicle, it is also necessary to perform back calculation through multiple preset installation heights. That is, for each target vehicle in the front image, the computer device obtains multiple third-direction world coordinate values ​​corresponding to the preset installation height and related to the target vehicle in the world coordinate system in the third direction, based on the pitch angle corresponding to the front image, the lens focal length, the horizon image coordinates of the front image, the image coordinate information of the target vehicle, and the preset installation height. More specifically, the computer device calculates based on the second-direction image coordinate value of the horizon image coordinates of the front image in the second direction, and the second-direction image coordinate value of the frame line of the bounding box corresponding to the target vehicle near the bottom of the target vehicle in the second direction (i.e., the second-direction image coordinate value of the lower left or lower right corner of the bounding box corresponding to the target vehicle).

[0027] In this embodiment, the third-party global coordinate value corresponding to the target vehicle in each front-view image is calculated using the following formula: in, This represents the third-party coordinates of the target vehicle in the world. This indicates the pitch angle corresponding to the front view of the vehicle. This indicates the focal length of the lens. Indicates any preset installation height. This represents the second-direction image coordinate value of the bounding box corresponding to the target vehicle, which is close to the bottom of the target vehicle, in the second direction. This indicates the second-direction image coordinate value of the horizon image coordinate of the front view of the vehicle in the second direction.

[0028] It is worth mentioning that the third-party global coordinate value for each target vehicle indicates the straight-line distance between the rear of the target vehicle and the camera module.

[0029] It should be added that the preset installation heights in this embodiment are determined based on the possible vehicle height of any vehicle. Since the vehicle height range from a typical sedan to a large truck is roughly between 100 and 300 centimeters, this range is used as the range for the preset installation heights, and each preset installation height is determined in units of 1 centimeter, i.e., 100 centimeters is one preset installation height, 101 centimeters is one preset installation height, 102 centimeters is one preset installation height, and so on. These preset installation heights can be determined with different values ​​depending on different situations, and should not be limited to the determination method of this embodiment.

[0030] In step S17, for each third-direction world coordinate value of the target vehicle in each front-view image, the computer device, based on the first-direction world coordinate information of the target vehicle, the second-direction world coordinate information of the target vehicle, the intrinsic parameter data matrix corresponding to the camera module, and the extrinsic parameter data matrix of the front-view image, uses a pinhole camera model to reproject the target vehicle in the image coordinate system to reconstruct a reprojected image coordinate information corresponding to the bounding box of the reprojected target vehicle and a reprojection error value. More specifically, the reprojection error value is the error between the image coordinate information of the target vehicle and the reprojected image coordinate information; the reprojected image coordinate information includes each vertex of the bounding box of the reprojected target vehicle. The image coordinates, and each vertex Image coordinates can be represented as ,in, This indicates the vertex The first direction image coordinates in the first direction, This indicates the vertex The second direction image coordinates in the second direction.

[0031] In this embodiment, each vertex contained in the reprojected image coordinate information corresponding to each third-party world coordinate value of the target vehicle in each front image is... Image coordinates Obtained through the following formula:

[0032] in, This represents the intrinsic parameter data matrix. This represents the extrinsic data matrix of the vehicle's front image. Represents an arbitrary scale factor. This indicates that the first-direction world coordinate information of the target vehicle contains the information corresponding to that vertex. vertex First direction world coordinates, This indicates that the second-direction world coordinate information of the target vehicle contains the information corresponding to that vertex. vertex Second direction world coordinates, This represents the third-party coordinates of the target vehicle in the world.

[0033] It is worth mentioning that the internal parameter matrix and the external parameter matrix of each front image can be obtained by camera calibration technology, and the calculation method is well known to those with general knowledge in this field. Therefore, for the sake of brevity, it will not be elaborated here.

[0034] It should be added that the reprojection error value of the bounding box of the target vehicle after each reprojection is calculated as follows: First, calculate the value of each vertex of the bounding box of the target vehicle after the reprojection. The image coordinates of the vertex Corresponding vertex The Euclidean distance between the image coordinates is then used to determine the bounding box of the target vehicle after reprojection. The sum of the corresponding Euclidean distances is used as the reprojection error value.

[0035] In step S18, for each front image of the vehicle, the computer device uses the preset installation height corresponding to the third-party global coordinate value with the smallest reprojection error value as a candidate installation height (i.e., the possible installation height of the camera module).

[0036] In step S19, the computer device uses regression analysis to obtain a true installation height (i.e., the most likely installation height) from the candidate installation heights.

[0037] In this embodiment, the computer device fits a regression curve using the following formula based on the reprojection error values ​​corresponding to all candidate installation heights, and takes the candidate installation height corresponding to the lowest point of the regression curve as the true installation height.

[0038] in, A probability function representing the distribution of normal error data. The parameters representing the equation of the error curve, This represents the standard deviation of the normal error data distribution. This represents the reprojection error value corresponding to the i-th candidate installation height.

[0039] It should be noted that the generation of regression curves is not limited to the method disclosed in this embodiment; regression analysis methods such as the least squares method can also be used.

[0040] It's worth noting that, theoretically, the candidate installation height of a single front view image can be directly used as the actual installation height. However, since the aforementioned calculations rely on the predicted true vehicle width and height initially estimated by the VLDR model, and this model may contain errors during estimation, the final candidate installation height may also be biased. Therefore, collecting a large number of candidate installation heights from front views and then performing regression analysis to select the best one can effectively avoid the bias caused by the estimation errors of the VLDR model.

[0041] It should be noted that the calculated actual installation height can be updated in a driver assistance system. The latest actual installation height can be used to update and replace the original installation height data in the driver assistance system's formula, and to calculate the values ​​required for various functions of the driver assistance system (such as the distance to the vehicle in front).

[0042] In summary, by combining the images of the vehicle's front captured by this camera module with a pinhole camera model, the actual installation height of the camera module can be deduced, achieving the effect of obtaining the installation height anytime, anywhere. This is more time-saving and labor-saving than using additional tools or going to the vehicle manufacturer for measurement. Furthermore, this invention can monitor the actual installation height of the camera module in real time, allowing users to immediately know changes in the installation height and adjust the position of the camera module accordingly. This ensures the accuracy of the driver assistance system based on the images acquired by the camera module, while avoiding unnecessary false alarms caused by inaccurate calculations or uploading such false alarm information to the cloud. Therefore, the purpose of this invention is indeed achieved.

[0043] However, the above description is merely an embodiment of the present invention and should not be construed as limiting the scope of the present invention. Any simple equivalent changes and modifications made in accordance with the scope of the patent application and the contents of the patent specification shall still fall within the scope of the patent of the present invention.

[0044] S11~S19: Steps 2: Front camera 21: Bounding Box P1: Top right corner vertex P2: Top left vertex P3: Bottom left vertex P 4: Bottom right corner vertex H: Predicts the actual vehicle height W: Predicts the actual vehicle width D: The vanishing point of the road

Claims

1. A method for estimating the installation height of a camera module installed in a vehicle, executed using a computer device, the computer device having pre-established a vehicle position and size estimation model, the method comprising the following steps: (A) obtaining multiple images of the front of the vehicle taken by the camera module; (B) for each image of the front of the vehicle, using the vehicle position and size estimation model to obtain image coordinate information of a bounding box corresponding to a target vehicle, a predicted true vehicle width, and a predicted true vehicle height, wherein the predicted true vehicle width and the predicted true vehicle height can be converted into a set of world coordinate information corresponding to a first direction and a second direction; (C) based on the image coordinate information of the target vehicle in all the images of the front of the vehicle and the set of world coordinate information, calculating and obtaining an intrinsic parameter data matrix of the camera module; (D) for each image of the front of the vehicle, based on the data obtained in steps (B) to (C) and a pinhole camera model, obtaining an extrinsic parameter data matrix containing a pitch angle data corresponding to the camera module when taking the image of the front of the vehicle. (E) For each target vehicle in the front view, based on the data obtained in steps (B) to (D) and multiple preset installation heights, obtain multiple third-direction world coordinate values ​​corresponding to the preset installation heights and relating to the target vehicle in a third direction; (F) For each target vehicle in the front view, based on the data obtained in steps (B) to (E), reproject the target vehicle in the image coordinate system to obtain reprojected image coordinate information of the bounding box corresponding to the target vehicle and a reprojection error value; (G) For each front view, take the preset installation height corresponding to the third-direction world coordinate value with the smallest reprojection error value as a candidate installation height; and (H) Obtain a true installation height from the candidate installation heights.

2. A method for estimating the installation height of a camera module installed in a vehicle, executed using a computer device, the computer device having a pre-established vehicle position and size estimation model, the method comprising the following steps: (A) obtaining multiple images of the front of the vehicle taken by the camera module; (B) for each image of the front of the vehicle, using the vehicle position and size estimation model to obtain image coordinate information of the bounding box corresponding to a target vehicle in the image of the front of the vehicle, the predicted true width of the target vehicle, and the predicted true height of the target vehicle; (C) for each target vehicle in the image of the front of the vehicle, based on the predicted true width and the predicted true height of the target vehicle, obtaining first-direction world coordinate information of the target vehicle in a first direction and second-direction world coordinate information in a second direction in a world coordinate system defined according to a pinhole camera model; (D) Based on the image coordinate information of the target vehicle in all front-view images, the first-direction world coordinate information, and the second-direction world coordinate information, a lens focal length of the camera module is obtained using a pinhole camera model; (E) For each front-view image, based on the lens focal length, a center point image coordinate of the image center point of the front-view image in the image coordinate system, and a horizon image coordinate of the position of the horizon in the front-view image in the image coordinate system, a pitch angle of the camera module when shooting the front-view image is obtained; (F) For each target vehicle in each front-view image, based on the pitch angle corresponding to the front-view image, the lens focal length, the horizon image coordinate of the front-view image, the image coordinate information of the target vehicle, and multiple preset installation heights, multiple third-direction world coordinate values ​​corresponding to the preset installation heights and relating to the target vehicle in a third-direction world coordinate system are obtained; (G) For each third-direction world coordinate value of the target vehicle in each front-view image, based on the first-direction world coordinate information of the target vehicle, the second-direction world coordinate information of the target vehicle, an intrinsic parameter data matrix of the camera module, and an extrinsic parameter data matrix of the front-view image, the target vehicle is reprojected in the image coordinate system to obtain reprojected image coordinate information corresponding to the bounding box of the reprojected target vehicle and a reprojection error value, wherein the reprojection error value is the error between the image coordinate information of the target vehicle and the reprojected image coordinate information; (H) For each front-view image, the preset installation height corresponding to the third-direction world coordinate value with the smallest reprojection error value is taken as a candidate installation height; and (I) A true installation height is obtained from the candidate installation heights using a regression analysis method.

3. The installation height estimation method as described in claim 1 or 2, wherein the vehicle position and size estimation model is obtained by using target detection technology and training based on a loss function relating the actual size of the vehicle to the predicted actual width and predicted actual height of the target vehicle.

4. The installation height estimation method as described in claim 2, wherein, In step (E), for each front view, its corresponding pitch angle is calculated by the following formula: where represents the pitch angle corresponding to the front view, represents the second direction image coordinate value of the center point image coordinate of the front view in the second direction, represents the second direction image coordinate value of the horizon image coordinate of the front view in the second direction, and represents the focal length of the lens.

5. The installation height estimation method as described in claim 2, wherein, In step (F), for each target vehicle in the front view, its corresponding third-party world coordinate value is calculated by the following formula: where represents the third-party world coordinate value of the target vehicle, represents the pitch angle corresponding to the front view, represents the focal length of the lens, represents any preset installation height, represents the second-direction image coordinate value of the frame line of the target vehicle's bounding box near the bottom of the target vehicle in the second direction, and represents the second-direction image coordinate value of the horizon image coordinate of the front view in the second direction.

6. The installation height estimation method as described in claim 2, wherein, In step (G), for each third-direction world coordinate value of the target vehicle in each front-view image, the corresponding reprojected image coordinate information includes the image coordinates of each vertex of the bounding box of the reprojected target vehicle corresponding to the reprojected image coordinate information, and the image coordinates of each vertex are obtained by the following formula: where represents the intrinsic data matrix, represents the extrinsic data matrix of the front-view image, represents an arbitrary scale factor, represents the first-direction world coordinate value of the vertex corresponding to the vertex contained in the first-direction world coordinate information of the target vehicle, represents the second-direction world coordinate value of the vertex corresponding to the vertex contained in the second-direction world coordinate information of the target vehicle, represents the third-direction world coordinate value of the target vehicle, represents the first-direction image coordinate value of the vertex in the first direction, and represents the second-direction image coordinate value of the vertex in the second direction.

Images