A BSD-based CMS extended field-of-view switching method

Abstract

The application discloses a BSD-based CMS extended field of view switching method, which comprises the following steps: calculating the positions of two end points, which are four meters apart and one meter wide, in the rear in an image; calculating the longitudinal position of the upper edge of the field of view, calculating the position and size of the locked field of view; calculating a new magnification Mnew, judging whether the magnification is compliant through a known CMS driving field of view magnification M; and judging whether to adjust the field of view according to the magnification. The application relates to the technical field of vehicle-mounted display and discloses the BSD-based CMS extended field of view switching method. BSD can give the position of a target vehicle V in an image, and the position changes every frame. Then, BSD can calculate the TTC of the target vehicle. When it is judged that the TTC of the vehicle is less than a certain threshold value, it is indicated that the target vehicle is a "dangerous vehicle", and the "field of view locking" function is switched. Meanwhile, the position of the vehicle is calculated, and when the vehicle is in a blind area, the "extended field of view" is started to escort the journey of the driver.

Description

Technical Field

[0001] This invention relates to the field of automotive display technology, specifically to a BSD-based CMS extended field-of-view switching method. Background Technology

[0002] CMS (Camera Monitor System) is an alternative to traditional rearview mirrors, including side mirrors and interior mirrors, featuring low wind resistance, wide field of view, and high image quality.

[0003] CMS without BSD functionality relies on manual operation by the driver to switch the field of view. BSD detects vehicles on both sides and behind, as well as vehicles in the blind spot, and then issues a "field of view switching" command to the CMS. This is not only efficient, but also reduces the driver's attention during driving and improves safety. However, there are no existing cases of combining BSD and CMS. Therefore, a method that combines CMS and BSD functionality is essential. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a BSD-based method for switching extended views in CMS, solving the problem that existing CMS and BSD do not have a good integration method.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a BSD-based CMS extended view switching method, comprising the following steps: Calculate the positions of two endpoints that are four meters apart and one meter wide in the image, and denote the two endpoints as RP1(x,y) and RP2(x,y) respectively. Calculate the circumscribed rectangle RpRoi (x1,y1,w1,h1) formed by the endpoints RP1 and RP2; Set the current frame, the position of the target vehicle V in the image is Voi, calculate the center coordinates of the current frame Voi Vroi C(x2,y2), the formula for calculating the center coordinates Voi C is VoiCx=Vroix+Vroiw / 2,VroiCy=Vroiy +Vroih / 2; Calculate the lower y1 value of RpRoi. The formula for calculating the y1 value is y1 = RpRoiy + RpRoih. Calculate the longitudinal distance Dv between the y1 value and the center point VroiC of Vroi. The formula for calculating the longitudinal distance Dv is Dv=RpRoiy+RpRoih_VroiCy; Calculate the longitudinal position Pupy at the upper edge of the field of view. The formula for calculating the longitudinal position Pupy is Pupy = VroiCy – Dv. Calculate the position and size of the locked field of view, denoted as LviewRoi (x3,y3,w3,h3); Calculate the new magnification factor as Mnew, and use the known CMS driving field of view magnification factor M to determine whether the magnification factor Mnew is compliant; Determine if the magnification factor is compliant; When the magnification is within acceptable limits, determine the situation of vehicles approaching from behind and activate the extended field of view.

[0006] Preferably, calculating the location and size of the locked field of view includes the following steps: The known CMS driving field of view is denoted as viewRoi (x4,y4,w4,h4); Calculate the position and size of the locked field of view, LviewRoih, using the formula LviewRoih = 2 * Dv; While keeping the aspect ratio unchanged, calculate the position and size of the locked field of view LviewRoiw, and the calculation formula is LviewRoiw=LviewRoih*viewRoiw / viewRoih); LviewRoix=VroiCx–LviewRoiw / 2; LviewRoiy=Vro iCy–Dv.

[0007] Preferably, the formula for calculating the circumscribed rectangle RpRoi is: RpRoix = min(RP1x, RP2x); RpRoiy = min(RP1y, RP2y); RpRoiw = max(RP1x,RP2x) - min(RP1x,RP2x); RpRoih = max(RP1y,RP2y) - min(RP1y,RP2y).

[0008] Preferably, the formula for calculating the magnification is: Mnew=M*viewRoih / LviewRoih.

[0009] Preferably, determining whether the magnification is compliant includes the following steps: Determine that the magnification factor Mnew is greater than or equal to the threshold M; When the magnification factor Mnew is greater than or equal to the M threshold, it is used as a new field of view for display; when the magnification factor Mnew is less than the M threshold, the original field of view viewRoi is used for display.

[0010] Preferably, when the magnification is compliant, determining the situation of oncoming vehicles and activating the extended field of view includes the following steps: Obtain the pixel coordinates of the vehicle detection bounding boxes in the image frame by frame. Take the center point of the bottom edge of the vehicle detection frame as the target point for the transformation, denoted as Vroi P(VroiX+VroiW / 2 ,VroiY+VroiH); The target point is converted into the coordinates P of the target point to the camera on the ground in a 3D coordinate system; Calculate P and the angle θ' referred to by the camera; θ' determines whether the vehicle is in the driver's field of vision; if it is outside the driver's field of vision, the view is switched to an extended field of vision.

[0011] Preferably, determining whether the vehicle is in the driving field of vision by using θ', and switching to an extended field of vision when the vehicle is outside the driving field of vision, includes the following steps: The position of the target vehicle V in the image, Vroi P, is converted into a 3D coordinate system Vpos(x4, y4, z4); Projecting the 3D coordinate system Vpos onto the ground, at which point y = 0, the 3D coordinate system Vpos is represented as Vpos(x4, z4); Calculate the angle θ' between the vehicle approaching from behind and the camera; The calculated field of view θ' is compared with the known driving field of view Fov, where Fov = θ; When θ is greater than θ', the extended field of view is not invoked; when θ is less than θ', the extended field of view is invoked. The extended field of view is denoted as Fov'. The driving field of view angle Fov and the extended field of view Fov' are inherent parameters of the CMS system.

[0012] Preferably, the formula for converting the target point into the coordinates P of the target position to the camera on the ground in the 3D coordinate system is as follows: The formula for converting 3D to 2D is known to be: in, For camera internal parameters, For camera external parameters; let: so, It is deduced that, Here is the transformation formula from the image coordinate system to the world coordinate system; therefore, This is the final conversion formula. Preferably, the formula for calculating the field of view angle θ' between the approaching vehicle and the camera is: Beneficial effects

[0013] The present invention provides a BSD-based CMS extended field-of-view switching method. The beneficial effects are as follows: regarding this BSD-based CMS extended field-of-view switching method, the so-called field-of-view locking means that BSD can calculate the TTC of a target vehicle, where TTC is the time to collision. If TTC is less than the TTC threshold, the target vehicle is regarded as a "dangerous vehicle", and BSD will output the position of the target vehicle V in the image, denoted as Vroi(x,y,w,h). The position of Vroi changes in each frame. Let the display field of view of CMS be M(w,h). When it is determined that the vehicle distance is less than a certain value, the extended field of view will be invoked to safeguard the driver's travel. Brief Description of the Drawings

[0014] Figure 1 is a schematic diagram of the field of view angle of the present invention.

[0015] Figure 2 is a schematic diagram of the identification frame of the present invention. Detailed Description of the Embodiments

[0016] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by a person of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0017] Please refer to Figure 1 _2, the present invention provides a technical solution: a BSD-based CMS extended field-of-view switching method, comprising the following steps: S1. Calculate the positions in the image of two endpoints that are 4 meters apart from the rear and 1 meter in width, and the two endpoints are respectively denoted as RP1(x,y) and RP2(x,y); S2. Calculate the circumscribed rectangle RpRoi(x1,y1,w1,h1) formed by the endpoints RP1 and RP2; S3. Set the position of the target vehicle V in the current frame in the image as Vroi, calculate the center coordinate VroiC(x2,y2) of Vroi in the current frame, and the formula for calculating the center coordinate VroiC is VroiCx=Vroix+Vroiw / 2, VroiCy=Vroiy+Vroih / 2; Specifically, VroiCx is the x-value of the center coordinate VroiC, VroiCy is the y-value of the center coordinate VroiC, Vroi is a rectangle, which is the recognition box formed around the vehicle image after the camera recognizes the vehicle, Vroix represents the starting x-coordinate of the rectangle, Vroiy represents the starting y-coordinate of the rectangle, Vroiw represents the width of the rectangle, and Vroih represents the height of the rectangle. S.4 Calculate the lower y1 value of RpRoi. The formula for calculating the y1 value is y1 = RpRoiy + RpRoih; S.5 Calculate the longitudinal distance Dv between the y1 value and the center point VroiC of Vroi. The formula for calculating the longitudinal distance Dv is Dv=RpRoiy+RpRoih_VroiCy; S.6 Calculate the longitudinal position Pupy of the upper edge of the field of view. The formula for calculating the longitudinal position Pupy is Pupy=VroiCy–Dv. Specifically, if Pupy < 0, then Pupy = 0; S.7 Calculate the position and size of the locked field of view, denoted as LviewRoi(x3,y3,w3,h3); Specifically, LviewRoi is the rectangle, x3 is the starting x value of the rectangle, y3 is the starting y value of the rectangle, w3 is the width of the rectangle, and h3 is the height of the rectangle. S.8 Calculate the new magnification factor as Mnew, and use the known CMS driving field magnification factor M to calculate whether the magnification factor Mnew is compliant; S.9 Determine if the magnification factor is compliant; S.10 When the magnification is compliant, determine the situation of the vehicle approaching from behind and call up the extended field of view.

[0018] Calculating the location and size of the locked field of view involves the following steps: S.7.1 The known CMS driving field of view is denoted as viewRoi(x4,y4,w4,h4); S.7.2 Calculate the position and size of the locked field of view LviewRoih. The formula for calculating LviewRoih is LviewRoih = 2 * Dv; S.7.3 Under the premise of keeping the aspect ratio unchanged, calculate the position and size of the locked field of view LviewRoiw, and the calculation formula is LviewRoiw=LviewRoih*(viewRoiw / viewRoih); Specifically, LviewRoiw is the width of the locked field of view, while LviewRoih is the height of the locked field of view. viewRoiw is the width of the CMS driving field of view, and viewRoih is the height of the CMS driving field of view. S.7.4LviewRoix=VroiCx–LviewRoiw / 2; LviewRoiy=VroiCy–Dv; Specifically, LviewRoix is ​​the starting x-value for locking the field of view, and LviewRoiy is the starting y-value for locking the field of view.

[0019] The formula for calculating the circumscribed rectangle RpRoi is: RpRoix = min(RP1x, RP2x); RpRoiy = min(RP1y, RP2y); RpRoiw = max(RP1x,RP2x) - min(RP1x,RP2x); RpRoih = max(RP1y,RP2y) - min(RP1y,RP2y).

[0020] The formula for calculating the magnification is: Mnew=M*viewRoih / LviewRoih.

[0021] Determining whether a magnification factor is compliant includes the following steps: Determine that the magnification factor Mnew is greater than or equal to the threshold M; Specifically, the M threshold is commonly 0.33, but is not limited to 0.33; When the magnification factor Mnew is greater than or equal to the M threshold, it is used as a new field of view for display; when the magnification factor Mnew is less than the M threshold, the original field of view viewRoi is used for display.

[0022] When the magnification is compliant, determining the situation of oncoming vehicles and activating the extended field of view includes the following steps: S.10.1 Obtain the pixel coordinates of the vehicle detection box in the image frame by frame. S.10.2 Take the center point of the bottom edge of the vehicle detection frame as the target point for conversion, denoted as Vroi P (VroiX+VroiW / 2,VroiY+VroiH); S.10.3 Convert the target point into the coordinates P of the target point to the camera on the ground in the 3D coordinate system; S.10.4 Calculate P and the angle θ' referred to by the camera; S.10.5 determines whether the vehicle is in the driving field of vision by using θ'. If the vehicle is outside the driving field of vision, it switches to the extended field of vision.

[0023] S.10.6 Determining whether the vehicle is in the driving field of vision using θ', and switching to an extended field of vision if the vehicle is outside the driving field of vision, includes the following steps: S.10.7 Convert the 2D position Vroi P of the target vehicle V in the image into a 3D coordinate system Vpos(x4, y4, z4); S.10.8 Project the 3D coordinate system Vpos onto the ground, at which point y = 0, and the 3D coordinate system Vpos is represented as Vpos(x4, z4); S.10.9 Calculate the angle θ' between the vehicle approaching from behind and the camera; S.10.10 The calculated field of view θ' is compared with the known driving field of view Fov, where the known field of view Fov = θ; S.10.11 When θ is greater than θ', the extended field of view is not invoked; when θ is less than θ', the extended field of view is invoked. The extended field of view is denoted as Fov'. The driving field of view angle Fov and the extended field of view Fov' are inherent parameters of the CMS system.

[0024] The formula for converting the target point into the coordinates P of the camera on the ground in the 3D coordinate system is as follows: The formula for converting 3D to 2D is known to be: in, For camera internal parameters, Specifically, Zc represents the camera's magnification, a fixed parameter of the camera, while f represents the focal length, also a fixed parameter of the camera. is a point matrix in a 2D coordinate system, where u and v are points in the 2D coordinate system; in, This is the pixel transformation matrix for the camera. This is the camera's focal length matrix; together, they form the camera's intrinsic parameters. Let Xw, Yw, and Zw be a point matrix in the world coordinate system, where Xw, Yw, and Zw are all points in the world coordinate system. so, It is deduced that, This is the transformation formula from the image coordinate system to the world coordinate system; therefore, This is the final conversion formula. The formula for calculating the field of view angle θ′ between the vehicle approaching from behind and the camera is: Those skilled in the art can connect the components in this case sequentially. The specific connection and operation sequence should refer to the working principle described below. The detailed connection methods are well-known technologies in the field. The working principle and process are mainly described below.

[0025] Embodiment: The so-called field of view locking: BSD can calculate the TTC of the target vehicle, where TTC is the time to collision. If TTC is less than the TTC threshold, the target vehicle is regarded as a "dangerous vehicle", and BSD will output the position of the target vehicle V in the image, which is represented as Vroi(x,y,w,h). The position of Vroi changes in each frame. Let the display field of view of CMS be M(w,h). When it is determined that the vehicle distance is less than a certain value, the extended field of view will be invoked to safeguard the driver's journey.

[0026] It should be noted that in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or sequence between these entities or operations. Moreover, the terms "comprising", "including" or any other variants thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements not only includes those elements, but also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, article or apparatus. Without further limitations, the statement "comprising a defined element" does not exclude the presence of additional identical elements in the process, method, article or apparatus that comprises the defined element.

[0027] Although the embodiments of the present invention have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present invention, and the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A BSD-based CMS extended view switching method, characterized in that, Includes the following steps: Calculate the positions of two endpoints that are four meters apart and one meter wide in the image, and denote the two endpoints as RP1(x,y) and RP2(x,y) respectively. Calculate the circumscribed rectangle RpRoi(x1,y1,w1,h1) formed by the endpoints RP1 and RP2; Set the current frame, the position of the target vehicle V in the image is Voi, calculate the center coordinates of the current frame Voi Vroi Vroi C(x2,y2), the formula for calculating the center coordinates Voi C is Voi Cx = Voi x + Voi w / 2, Voi Cy = Voi y + Voi h / 2; Calculate the lower y1 value of RpRoi. The formula for calculating the y1 value is y1 = RpRoiy + RpRoih. Calculate the longitudinal distance Dv between the y1 value and the center point VroiC of Vroi. The formula for calculating the longitudinal distance Dv is as follows: Dv = RpRoiy + RpRoih - VroiCy; Calculate the longitudinal position Pupy at the top edge of the field of view. The formula for calculating the longitudinal position Pupy is Pupy = Vro iCy – Dv; calculate the position and size of the locked field of view, denoted as LviewRoi(x3,y3,w3,h3); Calculate the new magnification factor as Mnew, using the known CMS driving field of view magnification factor M, to calculate the new magnification factor. Is Mnew compliant? Determine if the magnification factor is compliant; When the magnification is within acceptable limits, determine the situation of vehicles approaching from behind and activate the extended field of view.

2. The BSD-based CMS extended view switching method according to claim 1, characterized in that... Calculating the location and size of the locked field of view includes the following steps: The known CMS driving field of view is denoted as viewRoi(x4,y4,w4,h4); Calculate the position and size of the locked field of view, LviewRoih. The formula for calculating LviewRoih is LviewRoih = 2 * Dv. While keeping the aspect ratio unchanged, calculate the position and size of the locked field of view, LviewRoiw, using the formula LviewRoiw = LviewRoih * (viewRoiw / viewRoih). LviewRoix=VroiCx–LviewRoiw / 2; LviewRoiy=Vro iCy–Dv.

3. The BSD-based CMS extended view switching method according to claim 1, characterized in that... The formula for calculating the circumscribed rectangle RpRoi is: RpRoix = min(RP1x, RP2x); RpRoiy = min(RP1y, RP2y); RpRoiw = max(RP1x,RP2x) - min(RP1x,RP2x); RpRoih = max(RP1y,RP2y) - min(RP1y,RP2y).

4. A BSD-based CMS extended view switching method according to claim 2, characterized in that... The formula for calculating the magnification is: Mnew=M*viewRoih / LviewRoih.

5. A BSD-based CMS extended view switching method according to claim 1, characterized in that... Determining whether a magnification factor is compliant includes the following steps: Determine that the magnification factor Mnew is greater than or equal to the threshold M; When the magnification factor Mnew is greater than or equal to the M threshold, a new field of view is used for display; when the magnification factor Mnew is less than the M threshold, the original field of view viewRoi is used for display.

6. A BSD-based CMS extended view switching method according to claim 1, characterized in that... When the magnification is compliant, determining the situation of oncoming vehicles and calling the extended field of view includes the following steps: obtaining the pixel coordinates of the vehicle detection box in the image frame by frame, taking the center point of the bottom edge of the vehicle detection box as the target point for conversion, denoted as Vroi P(VroiX+VroiW / 2,VroiY+VroiH); converting the target point into the coordinates P of the target point to the camera on the ground in the 3D coordinate system; calculating the angle θ' between P and the camera; determining whether the vehicle is in the driving field of view through θ', and switching to the extended field of view if it is outside the driving field of view.

7. A BSD-based CMS extended view switching method according to claim 6, characterized in that... The process of determining whether a vehicle is in the driver's field of vision by using θ', and switching to an extended field of vision if the vehicle is outside the driver's field of vision, includes the following steps: The position Vroi P of the target vehicle V in the image is converted into a 3D coordinate system Vpos(x4, y4, z4); Project the 3D coordinate system Vpos onto the ground, where y = 0. The 3D coordinate system Vpos is represented as Vpos(x4, z4). Calculate the angle θ' between the vehicle approaching from behind and the camera. The calculated θ' is compared with the known driving field of view angle Fov, where Fov = θ; When θ is greater than θ', the extended field of view is not invoked; when θ is less than θ', the extended field of view is invoked. The extended field of view is denoted as Fov'. The driving field of view angle Fov and the extended field of view Fov' are inherent parameters of the CMS system.

8. A BSD-based CMS extended view switching method according to claim 7, characterized in that... The formula for converting the target point into the coordinates P of the target point to the camera on the ground in a 3D coordinate system is: The formula for converting 3D to 2D is also known as: in, For camera internal parameters, For camera external parameters; let: so, It is deduced that, This is the transformation formula from the image coordinate system to the world coordinate system; therefore, This is the final conversion formula.

9. A BSD-based CMS extended view switching method according to claim 7, characterized in that... The formula for calculating the field of view angle θ' between the vehicle approaching from behind and the camera is:

Images