Display system and method for compensating for blind spots in tractor trailer rearview mirrors

Abstract

The present invention relates to a display system for compensating blind spots caused by the rearview mirror of a tractor trailer, comprising a collection device, a sensing device, a display control device and a display device, the collection device comprises a left front camera, a right front camera, a left rear camera and a right rear camera, the sensing device is an angle sensing sensor installed on the tractor trailer, the angle sensing sensor is used to obtain the angle at which the axis of the trailer box deviates from the axis of the tractor, the display control device is connected to the collection device and the sensing device, and the display device is a display installed in the cab, used to display the image output from the display control device to the driver. The present invention can effectively solve the problem of the trailer blocking the field of view observation by performing screen compensation for the area blocked by the trailer on the rearview mirror during turning through the multi-camera installation, and compared with the general panoramic image spliced ​​with a fisheye lens, the driver can more intuitively sense the location of the dangerous elements, and further shorten the thinking time and reduce accidents.

Description

[Technical field]

[0001] This application claims priority to Chinese invention patent application No. 202110901387.8, entitled “Display system and method for compensating blind spots by tractor-trailer rearview mirror”, and to Chinese invention patent application No. 202210100871.5, entitled “Method and device for automatically adjusting exposure”.

[0002] The present invention relates to the technical field of in-vehicle electronics, and more particularly to a display system and method for compensating for blind spots in the rearview mirror of a tractor trailer. [Background technology]

[0003] Rearview mirrors and electronic rearview mirrors are an important part of ensuring driving safety when a vehicle is traveling. The automobile rearview mirror performance standard GB-15084 stipulates that the observation range of a vehicle rearview mirror is the field of view range when the vehicle is normally stopped, and the field of view observation range on both sides should be extended to 25-45 degrees outside the central axis of the vehicle (caused by different observation eye points), but does not specify the field of view observation range in some specific environments of the vehicle.

[0004] A towing vehicle or winch vehicle is a vehicle in which the two bodies, the tractor and the trailer, are connected by an axial lock or hinge. When the vehicle is backing up or turning, the driver needs to observe the images on both sides and directly behind the trailer, which can help the driver to smoothly back up to a predetermined position or run along a predetermined trajectory. However, if the deviation angle of the tractor is too large, the one-sided rearview mirror or electronic rearview mirror will have a situation where the line of sight is blocked by the trailer due to the turning angle, so the driver can only observe the state of one side of the vehicle, which is easy to cause accidents such as one-sided inside wheel difference accidents and vehicle drift accidents in complex and unfamiliar environments.

[0005] In the prior art, for example, the patent with application number 201811278439.5 collects and acquires image information using fisheye cameras installed at multiple locations on the vehicle body, preprocesses and splices the image information, and presents the spliced ​​image to the driver on the display, allowing the driver to obtain a 360-degree view of the vehicle. The 360-degree view is a look-around view formed by splicing 4-6 cameras, and the cameras need to be fixed in a certain position and should not be deflected, which is suitable for low-speed driving of passenger cars or integrated trucks. Because the trailer of the towing vehicle moves at any time, the look-around is affected by the change of the video collection range, and cannot be spliced ​​into a complete image.

[0006] The fisheye image of the panoramic camera is distorted significantly, and because it is a driving assistance product, there are no legal requirements for the depth of field, subject distance, etc., and it can only be used as an auxiliary product.

[0007] Since the panning camera is mounted so as to look down and illuminate downward, due to the illumination angle of the camera, it can only see the range of the area near the vehicle and cannot see the field of view far from the vehicle.

[0008] The present invention is used as a mandatory product to replace rearview mirrors, and the regulations have clear viewing range requirements, which are not equivalent products due to differences in camera performance tests, usage situations, viewing areas required by regulations, safety requirements, etc. Summary of the Invention [Problem to be solved by the invention]

[0009] The objective of the present invention is to propose a display system and method for compensating for blind spots in the field of view using the rearview mirror of a tractor trailer, which solves the problem of displaying blind spots blocked by one side of the trailer due to the tractor offset when the vehicle turns or reverses, and facilitates the driver to quickly locate the position of the blind spot where danger may occur. In a preferred implementation form, the present invention adds a tilt-shift setting method and an imaging adjustment method based on photosensitivity to the camera of the collection device, which can further improve the performance of the present invention. [Means for solving the problem]

[0010] Specifically, the present invention relates to a display system for compensating for blind spots in a rearview mirror of a tractor trailer, the display system including a collection device, a sensing device, a display control device, and a display device; The collecting device includes a left front camera, a right front camera, a left rear camera, and a right rear camera, the left front camera and the right front camera are respectively installed on the left and right sides outside the cab, and the left rear camera and the right rear camera are respectively installed on the left and right sides of the trailer, The sensing device is an angle sensing sensor installed on a tractor trailer, the angle sensing sensor being used to obtain an angle at which an axis of the trailer deviates from an axis of the tractor; The display control device is connected to the collection device and the sensing device and includes a processor, and the processor is used to obtain an angle at which the axis of the trailer deviates from the axis of the tractor by the sensing device, and to directly transmit the screen obtained by the collection device to the display device, and / or to transmit it to the display device after performing cutting and synthesis processing; The display device is a display installed in the driver's cab, and is connected to the display control device and used to display an image output by the display control device to the driver. A display system for compensating for blind spots in the field of view using the rearview mirror of a tractor-trailer is provided.

[0011] Furthermore, the screen field of view collected by the left front camera and the right front camera complies with the Class II field of view and / or Class IV field of view specified in the labeling, and the screens collected by the left rear camera, the right rear camera and the left front camera, and the right front camera include the trailer body on the side on which the cameras are located.

[0012] Further, the display device includes a left display unit and a right display unit, the left display unit is used to display an image collected by the left front camera or an image synthesized by the left front camera and the left rear camera, and the right display unit is used to display an image collected by the right front camera or an image synthesized by the right front camera and the right rear camera.

[0013] Further, the processor, in cutting out and combining the images, removes the image of the preset width or fixed width on the side of the front camera blocking the trailer in the direction opposite to the direction in which the tractor turns and / or the direction in which the tractor turns, reserves the image of the preset width or fixed width on the side of the rear camera blocking the trailer in the direction opposite to the direction in which the tractor turns, and arranges the reserved image in the place where the image was removed to perform image composition; Furthermore, when the angle sensing sensor detects that the angle between the axis line of the tractor and the trailer exceeds the illumination angle of the front camera of the turning side, the processor adopts the fixed width image cropping and synthesis; The fixed width is the width that the trailer occupies within the screen of the front camera on the turning side.

[0014] Furthermore, the display control device includes a storage unit, in which a display block diagram X to be displayed on a display device and a set of compensation block diagrams Y and Z for compensating a screen are stored, and the processor calculates and judges according to the sensing of the sensing device, and calls up a compensation block diagram Ya that matches a preset width in the set of compensation block diagrams Y stored in advance and a compensation block diagram Za of a fixed width or a preset width in the set of compensation block diagrams Z; The processor simultaneously cuts out a preset width image on the trailer side of the rear camera in the opposite direction to the direction in which the tractor turns, and projects the processed image onto the called compensation block diagram Ya; The processor cuts out a fixed width image on the trailer side of the rear camera in the direction in which the tractor is turning, and projects the processed image onto the called compensation block diagram Za.

[0015] Furthermore, the preset width is set according to an angle between the centerline of the tractor and the trailer; When the angle between the tractor and trailer axis is 0-15 degrees, the preset width is 0; If the angle between the tractor and trailer axis is 15-30 degrees, the preset width is one-third of the screen width; If the angle between the tractor and trailer axis is 30-45 degrees, the preset width is two-thirds of the screen width; If the angle between the tractor and trailer axis is 45 degrees or greater, the preset width is full screen width.

[0016] Preferably, one or more of the left front camera, right front camera, left rear camera and right rear camera adopt a tilt-shift camera, the tilt-shift camera includes a camera body, a screen collection unit and an imaging lens, the imaging lens and the screen collection unit are respectively attached to the front end and rear end of the camera body, the camera body has a first angle θ tilted toward a first direction with respect to a reference direction, the screen collection unit and the imaging lens have a second angle α and a third angle β with respect to a main axis of the camera body, respectively tilted in a direction opposite to the tilt direction of the camera body, the second angle α and the third angle β are different from each other, preferably the second angle α is greater than the third angle β so that the screen collection unit and the plane on which the imaging lens is located intersect to form an acute angle.

[0017] The present invention further provides a display method for compensating for blind spots in a rearview mirror of a tractor trailer, the display method comprising: The processor includes a step 1 of acquiring video images collected by a plurality of cameras in real time; The processor acquires an operating state of the tractor trailer in step 2; The processor includes a step 3 for generating an optimal screen of an unobstructed and minimized blind spot scheme according to an operating state of the tractor trailer; The operating state includes a trailer state and a driving state, the trailer state includes a trailer mode and a trailerless mode, and the driving state includes straight ahead, turning, and reverse; The multiple cameras include a left front camera, a right front camera, a left rear camera and a right rear camera, the left front camera and the right front camera being installed on both the left and right sides outside the driver's cab, respectively, and the left rear camera and the right rear camera being installed on both the left and right sides of the trailer, respectively.

[0018] Further, in step 3, in the trailerless mode, the processor provides a display device with images of a left front camera and a right front camera; In the trailer mode, when the driving state is straight, the processor provides a screen of a left front camera and a right front camera to a display device; In the trailer mode, when the driving state is turning, the processor provides a screen on a display device according to an angle between a center line of a tractor and a trailer; In cutting out and compositing the image in the opposite direction to the direction in which the tractor turns, the image of the preset width on the trailer blocking side of the front camera in the opposite direction to the direction in which the tractor turns is removed, the image of the preset width on the trailer side of the rear camera in the opposite direction to the direction in which the tractor turns is reserved, and the reserved image is placed in the location where the image was removed to perform new image composition; In cutting out and compositing the image in the direction in which the tractor turns, if the angle between the center line of the tractor and the trailer exceeds the illumination angle of the front camera on the turning side, the fixed-width image of the front camera on the turning side in the direction in which the tractor turns is removed, the fixed-width image of the rear camera on the trailer side in the direction in which the tractor turns is reserved, and the reserved image is placed in the location where the image was removed to perform new image composition; The fixed width is the width that the trailer occupies within the screen of the front camera on the tractor's turning side.

[0019] Furthermore, the preset width is set according to an angle between the centerline of the tractor and the trailer; When the angle between the tractor and trailer axis is 0-15 degrees, the preset width is 0; If the angle between the tractor and trailer axis is 15-30 degrees, the preset width is one-third of the screen width; If the angle between the tractor and trailer axis is 30-45 degrees, the preset width is two-thirds of the screen width; If the angle between the tractor and trailer axis is 45 degrees or greater, the preset width is full screen width. Effect of the Invention

[0020] The advantages of the present invention are as follows:

[0021] By installing multiple cameras, the present invention can effectively solve the problem of trailers blocking the field of view by performing screen compensation for the area blocked by the trailer on the rearview mirror while turning. At the same time, compared with the general panoramic screen spliced ​​with a fisheye lens, the driver can more intuitively sense the location of dangerous elements, further shortening the thinking time and effectively reducing accidents.

[0022] In the present invention, the front camera has a Class II field of view which can meet the GB-15084 standard, and has the ability to replace the traditional rearview mirror, and at the same time, it can be coordinated with the rearview camera on the trailer to realize monitoring of the entire vehicle without blind spots.

[0023] In the present invention, the field of view of the backup camera installed on the trailer includes the trailer itself, so that when the screen is adjusted, the driver can intuitively perceive the positional relationship between dangerous units and the trailer. Furthermore, the time it takes the driver to judge the content of the screen can be reduced, thereby enabling a rapid response to dangerous situations.

[0024] By checking the size of the area to be compensated based on the vehicle's turning width, the present invention can realize that the compensation area and the shading area are consistent, and further, the driver can continuously obtain a more intuitive view in the blind spot. [Brief description of the drawings]

[0025] [Figure 1] 1 is a schematic diagram showing the configuration of a display system for compensating for blind spots using a rearview mirror of a tractor trailer provided by an embodiment of the present invention; [Diagram 2]FIG. 2 is a schematic diagram showing the viewing range requirements that the left and right front cameras must meet in a display system for compensating for blind spots in the rearview mirror of a tractor trailer provided by an embodiment of the present invention. [Diagram 3] FIG. 2 is a schematic diagram showing the viewing range requirements that the left and right rear cameras must meet in a display system for compensating for blind spots in the rearview mirror of a tractor trailer provided by an embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram showing the distribution of cameras in a display system for compensating for blind spots in the rearview mirror of a tractor trailer provided by an embodiment of the present invention; [Diagram 5] 2 is a schematic diagram showing the field of view of each camera in a display system for compensating for blind spots in the rearview mirror of a tractor trailer provided by an embodiment of the present invention when the tractor trailer is turning. FIG. [Figure 6] 1 is a schematic diagram showing an uncompensated screen provided by a turning side display device in a display system for compensating for blind spots in the rearview mirror of a tractor trailer provided by an embodiment of the present invention when the tractor trailer is turning; [Figure 7] FIG. 2 is a schematic diagram showing an uncompensated screen provided by a display device on the opposite side and the turning direction when a tractor trailer is turning in a display system for compensating for blind spots using the rearview mirror of a tractor trailer provided by an embodiment of the present invention. [Figure 8] FIG. 2 is a schematic diagram showing a compensated screen provided by the turning side display device in the display system for compensating for blind spots in the field of view of the tractor trailer rearview mirror provided by an embodiment of the present invention when the tractor trailer is in a turning state; [Figure 9] FIG. 2 is a schematic diagram showing the compensated screen provided by the display device on the opposite side and the turning direction when the tractor trailer is turning in a display system for compensating for blind spots in the field of view using the rearview mirror of a tractor trailer provided by an embodiment of the present invention; [Figure 10]1A to 1E are schematic diagrams showing the transition of the inclination installation of each part of the camera in a preferred implementation form, in which views AC are schematic side views, and views D and E are schematic plan views. [Figure 11] 1 is a schematic side view showing the angular relationship between the imaging lens 100, the image collection unit 200, and the camera body when tilt-down photography is adopted. [Figure 12] 1 is a plan view schematic diagram showing the relationship between the angle of the imaging lens 100 with respect to the main axis of the camera body and the displacement offset when tilt-down photography is adopted. FIG. [Figure 13] The image was captured when the camera was slightly tilted downward and toward the outside of the vehicle body, but the imaging lens 100 was installed parallel to the image collection unit 200. [Figure 14] This is the image captured when the camera is attached to a vehicle, the camera body is tilted downward toward the rear of the vehicle, and both the imaging lens 100 and the image collection unit 200 are tilted at different angles in opposite directions relative to the main axis of the camera body (the central axis of the lens barrel). [Figure 15] 2 is a flow chart of a display method for compensating for blind spots by a rearview mirror of a tractor trailer provided by an embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] The technical solutions of the present invention will be described in more detail below with reference to the drawings, and the present invention includes but is not limited to the following examples.

[0027] As shown in FIG. 1, the present invention provides an indication system for compensating for blind spots in the rearview mirror of a tractor trailer, the indication system including a collection device, a sensing device, a display control device and a display device.

[0028] As shown in Figure 2-4, the collecting device includes a left front camera, a right front camera, a left rear camera and a right rear camera. The left front camera and the right front camera are respectively installed on the left and right sides outside the cab of the tractor trailer, the lenses of the left front camera and the right front camera face the rear side of the side where they are located, and are used to collect the real-time screen of the rear side of the left and right sides outside the cab of the tractor trailer, and the left front camera and the right front camera are used to collect the Class II field of view and Class IV field of view that meet the provisions of the GB-15084 standard. A left rear camera and a right rear camera are installed respectively on the left and right sides of one end of the tractor-trailer trailer near the driver's cab, the lenses of the left rear camera and right rear camera face toward the rear of the side on which they are located and are used to collect real-time images of the rear of the side on both sides of the tractor-trailer trailer, the lenses of the left rear camera and right rear camera are close to the mounting height and angle of the left front camera and right front camera, and the lenses of the left rear camera and right rear camera are consistent with the swing angle of the trailer, and the images collected by the lenses need to include the trailer body on the side on which the cameras are located.

[0029] The sensing device is an angle sensing sensor installed on the tractor trailer, and the angle sensing sensor is used to obtain the angle between the tractor and the trailer in real time. In another embodiment, the sensing device is realized by software integrated into the processing device, which monitors the real-time images acquired by the collecting device, and determines the angle between the tractor and the trailer from the images by analyzing the images acquired by the camera installed on the tractor and the camera installed on the trailer through multiple algorithms.

[0030] The display control device is respectively connected to the collection device and the sensing device, and the display control device includes a processor and a memory unit, the processor and the memory unit are connected to each other, and the processor is used to activate the corresponding collection device on the trailer based on the angle data between the tractor and the trailer acquired by the sensing device, and to cut out and synthesize the real-time screen acquired by the collection device, and the memory unit is used to store the screen acquired by the collection device and the screen cut out by the processor.

[0031] As shown in Figure 5-9, the screen collected by the left front camera is screen C, the screen collected by the right front camera is screen D, the screen collected by the left rear camera is screen E, and the screen collected by the right rear camera is screen F. As for the composite screen, when the vehicle turns left or backs up to the left as shown in Figure 5, the screen observed by the driver on the left side is C shown in Figure 6, and the screen observed by the driver on the right side is D shown in Figure 7. Most of the area in the left field of view is filled by the trailer, but there is a large blind spot in the right field of view, so both sides need to perform a certain degree of screen compensation. After compensation, the screen observed by the driver on the left side is C+E shown in Figure 8, and the screen observed by the driver on the right side is D+F shown in Figure 9.

[0032] Specifically, the screen compensation method is as follows: first, the processor determines the width of the right compensation screen according to the angle that the axis of the trailer deviates from the axis of the tractor, then obtains the corresponding width of the screen on the left side close to the trailer side in the right rear camera screen according to the width of the compensation screen, and again cuts out the corresponding width of the screen of the right front camera that is blocked by the left trailer side according to the width of the compensation screen, and finally fills the cut-out position of the right rear camera screen in the right front camera screen to synthesize an image with blind spot compensation. For the left compensation screen, firstly, the processor determines the vehicle's turning angle according to the angle at which the trailer axis deviates from the tractor axis; then, if the turning angle exceeds the left front camera's coverage angle (the coverage angle is the included angle between the left front camera's field of view edge and the tractor's side wall), obtain a certain width of the screen on the left rear camera screen close to the right trailer side according to the width of the compensation screen; again, cut out a certain width of the screen on the right side of the left front camera screen according to the width of the compensation screen; finally, fill the cut-out position on the left front camera screen with the left rear camera screen to synthesize an image with blind spot compensation; if the turning angle does not reach the left front camera's illumination angle, no compensation is performed and only the screen collected by the left front camera is displayed.

[0033] Similarly, when the vehicle turns right or backs up to the right, the images from the front and rear cameras on both sides are synthesized, and the two rear cameras may always be on, or the two rear cameras may be activated after it is determined that the vehicle is turning right or backing up to the right. For the left compensation screen, the processor first determines the width of the compensation screen according to the angle at which the trailer axis deviates from the tractor axis, then obtains the screen of the corresponding width on the right side close to the trailer side on the left rear camera screen according to the width of the compensation screen, and again cuts out the screen of the corresponding width of the part blocked by the right trailer side on the left front camera screen according to the width of the compensation screen, and finally fills the cut-out position on the left front camera screen with the left rear camera screen to synthesize an image with blind spot compensation. For the right compensation screen, firstly, the processor determines the vehicle's turning angle according to the angle at which the trailer axis deviates from the tractor axis; then, if the turning angle exceeds the coverage angle of the right front camera (the coverage angle is the included angle between the viewing edge of the right front camera and the side wall of the tractor), obtains a certain width of the screen on the left side close to the trailer in the right rear camera screen according to the width of the compensation screen; again, cuts out a certain width of the screen on the left side in the right front camera screen according to the width of the compensation screen; finally, fills the cut-out position in the right front camera screen with the right rear camera screen to synthesize an image with blind spot compensation; if the turning angle does not reach the illumination angle of the right front camera, no compensation is performed and only the screen collected by the right front camera is displayed.

[0034] In one embodiment, in compensation of a screen opposite to the direction in which the trailer head turns, a pre-drawn display block diagram X and a set of compensation block diagrams Y for compensating the screen are stored in the memory, the resolution of the display block diagram X corresponds to the resolution provided by the corresponding display device, a compensation block diagram Ya is superimposed in the display block diagram X, the compensation block diagram Ya is an element in the set of compensation block diagrams Y, the width of the compensation block diagram Ya is determined according to the angle at which the trailer axis deviates from the tractor axis, and when the angle at which the trailer axis deviates from the tractor axis reaches a preset range, the compensation block diagram Ya of the corresponding width is automatically called, the compensation block diagram Ya may be rectangular or may have a special shape determined according to the blind spot of a specific tractor trailer or trailer type, the compensation block diagram Y in the left field of view and the compensation block diagram Y in the right field of view adopt a symmetrical design, the compensation block diagram Y in the left field of view is located to the right of the display block diagram X in the left field of view, and the compensation block diagram Y in the right field of view is located to the left of the display block diagram X in the right field of view.

[0035] When the angle at which the trailer axis deviates from the tractor axis reaches a preset range, the compensation block diagram Ya calls up a compensation block diagram Ya of a corresponding size according to the angle at which the trailer axis deviates from the tractor axis. In the display block diagram X of the side that needs compensation, the processor fills the screen of the front camera of that side in the display block diagram X, calls the screen of the rear camera of that side and fills it in the same transition block diagram as the display block diagram X, cuts out the screen corresponding to the position of the compensation block diagram Ya, projects the cut-out screen to the position corresponding to the compensation block diagram Ya in the display block diagram X, covers the screen at the position corresponding to the compensation block diagram Ya in the front camera screen, and generates a screen having a compensation field of view through cutting and synthesis.

[0036] In compensating for the screen of the turning side of the tractor trailer, the memory further stores a set of compensation block diagrams Z for compensating the pre-drawn screen, the compensation block diagram Za is superimposed on the display block diagram X, the compensation block diagram Za is an element in the set of compensation block diagrams Z, the specific width of the compensation block diagram Za may be a fixed width or may be determined according to the angle at which the trailer axis deviates from the tractor axis, and when the angle at which the trailer axis deviates from the tractor axis reaches a preset range, the compensation block diagram Za of the corresponding width is automatically called, the compensation block diagram Za may be rectangular or may have a special shape formulated according to the blind spot of a specific tractor trailer and trailer type, the compensation block diagram Z in the left field of view and the compensation block diagram Z in the right field of view adopt a symmetrical design, the compensation block diagram Z in the left field of view is disposed to the left of the display block diagram X in the left field of view, and the compensation block diagram Z in the right field of view is disposed to the right of the display block diagram X in the right field of view.

[0037] The compensation block diagram Za calls up a corresponding compensation block diagram Za according to the angle at which the trailer axis deviates from the tractor axis, and the size of the compensation block diagram Za may be the width of the trailer in the screen of the front camera, or the corresponding size generated according to the angle at which the trailer axis deviates from the tractor axis. In the display block diagram X of the side that needs compensation, the processor fills the screen of the front camera of that side into the display block diagram X, calls the screen of the rear camera of that side and fills it in the same transition block diagram as the display block diagram X, cuts out the screen corresponding to the position of the compensation block diagram Za, projects the cut-out screen to the position corresponding to the compensation block diagram Za in the display block diagram X, covers the screen at the position corresponding to the compensation block diagram Za in the screen of the front camera, and generates a screen having a compensation field of view through cutting and synthesis.

[0038] The display device is a display installed in the driver's cab, and the display device is connected to the display control device and used to display the composite image output from the display control device to the driver. The display device includes a left display unit and a right display unit, and the left display unit is used to display the screen collected by the left front camera or the screen synthesized by the left front camera and the left rear camera, and the right display unit is used to display the screen collected by the right front camera or the screen synthesized by the right front camera and the right rear camera. The two left and right display units may be integrated into one display and displayed in two separate parts, or may be independent displays. The display device may be connected to the display control device and may be connected to a control bus or other vehicle control units of the vehicle to display various state parameters of the vehicle to the driver. The display of the display device may be integrated into the central control area or the electronic rearview mirror, or may be installed on the A-pillars on both sides of the driver's cab as a display device of the electronic rearview mirror.

[0039] In one embodiment, in the process of the tractor trailer driving straight, the processor obtains the video screens of the left front camera and the right front camera in real time, and directly sends the video screens to the display device without processing, and the display device displays the rear view screens of the left and right sides of the tractor trailer. When the tractor trailer starts to turn and reverse while driving, the sensing device sends the detected angle between the axis lines of the tractor and the trailer to the processor in real time, and the processor cuts out the video screens collected by the front cameras and the rear cameras on both sides with the size corresponding to the angle according to the angle between the axis lines of the tractor and the trailer. In the cutting process, the original video screen and the cut-out video screen are both cached in the storage unit, and the processor finally synthesizes the cut-out video screens, and sends the synthesized video screens on both sides to the corresponding areas of the display device for display. Of course, the driver can manually control the display device to display the vehicle class II mirror display mode provided by the left front camera and the right front camera, and the manual control mode takes precedence over the display screen provided by the present invention.

[0040] Specifically, for the screen compensation on the opposite side to the direction in which the tractor trailer turns, the left display unit and the right display unit are two displays respectively installed on the A-pillars on the left and right sides of the cab, the screen width of each display is 109.5mm, the resolution is 1920*720 pixels, when the angle between the axis line of the tractor and the trailer is 15 degrees, the display width of the compensation screen displayed on the display is 36.5mm, when the angle between the axis line of the tractor and the trailer is 30 degrees, the display width of the compensation screen displayed on the display is 73mm, when the angle between the axis line of the tractor and the trailer is 45 degrees, the display displays the compensation screen completely. For the screen compensation on the side of the tractor trailer turning, when the angle between the axis line of the tractor and the trailer is greater than the degree of the field of view of the front camera, the display displays a screen with a fixed width compensation, the fixed width is the fixed width of the screen occupied by the trailer in the screen collected by the front camera, and the compensation measure of the same angle and width as the opposite side to the direction in which the tractor trailer turns may be adopted.

[0041] Preferably, one or more of the left front camera, right front camera, left rear camera and right rear camera adopt tilt-shift camera. As shown in FIG. 10, each tilt-shift camera includes an imaging lens 100 and a screen collection unit 200, and conventional structures such as camera housing and control chip are not described in detail here. As shown in FIG. 11, the imaging lens 100 and the screen collection unit 200 are both inclined in opposite directions to the main axis of the camera body (the central axis of the lens barrel), and the inclination direction of the imaging lens 100 and the screen collection unit 200 to the main axis of the camera body is opposite to the inclination direction of the main axis of the camera body to the horizontal direction, and the inclination angles are a second angle α and a third angle β, respectively, 10 represents the horizontal direction, and 20 and 30 represent the normal lines of the imaging lens 100 and the screen collection unit 200, respectively. Preferably, the third angle β at which the image collecting unit 200 is tilted is greater than the second angle α at which the imaging lens 100 is tilted, and the second angle α and the third angle β are not the same.

[0042] As shown in the plan views of FIG. 10D and FIG. 10E and FIG. 12, the imaging lens 100 is offset (translated) with respect to the main axis of the camera body, and the offset distance is d, which is approximately equal to tan 8°-tan 15° multiplied by the distance L between the imaging lens 100 and the image collection unit 200, that is, the center of the imaging lens is offset by approximately 8-15 degrees with respect to the main axis of the camera based on the center of the image collection unit, and the offset direction is a direction away from the vehicle body (this offset occurs when the image collection unit 200 translates toward the vehicle body with respect to the main axis of the camera body). At the same time, the imaging lens 100 and the image collecting unit 200 are both turned about their own vertical center line, and the turning direction is a direction away from the vehicle body. More specifically, since the camera is shooting toward the rear of the vehicle body, the imaging lens 100 and the image collecting unit 200 are both turned about their respective vertical center axes toward the outside of the tail of the vehicle body by a certain angle γ, preferably between 8-16 degrees, and more preferably, the angle γ is positively correlated with the distance that the imaging lens 100 is offset from the main axis of the camera body. The turning angles of the imaging lens 100 and the image collecting unit 200 are preferably the same, but a slight difference between the turning angles of the two is not excluded. As can be seen from FIG. 13-14, the image captured by adopting the tilt-shift camera has the least distortion.

[0043] As shown in FIG. 15, the present invention further provides a display method for compensating for blind spots by the rearview mirror of a tractor trailer, the method including the following steps:

[0044] Step 1: The processor acquires the video images collected by multiple cameras in real time. Here, the multiple cameras include a left front camera and a right front camera installed on both the left and right sides outside the cab of the tractor trailer, and a left rear camera and a right rear camera installed on both the left and right sides at one end of the trailer close to the cab. The left front camera and the right front camera are used to collect Class II and Class IV fields of view that meet the requirements of the GB-15084 standard, and the left rear camera and the right rear camera are used to collect fields of view with a horizontal angle of 50 degrees rearward on both sides of the trailer of the tractor trailer, which must include the trailer body on the side where the camera is located.

[0045] Step 2: The processor obtains the working status of the tractor trailer; Here, the operating state of the tractor trailer includes a trailer state and a driving state, the trailer state includes a trailer-with mode and a trailer-without mode, and the driving state includes going straight and turning. The trailer state is detected by a processor connected to a vehicle control bus, and the vehicle control bus detects whether a trailer is attached to the tractor trailer, and the driving state is collected by a sensing device, which may employ an angle sensing sensor installed on the trailer to obtain the angle at which the axis of the trailer deviates from the axis of the tractor through the angle sensing sensor, or may employ a software algorithm loaded by the processor to determine the angle at which the axis of the trailer deviates from the axis of the tractor through the video images in each camera.

[0046] Step 3: The processor generates an optimal screen with no occlusion and minimal blind spot according to the operating state of the tractor trailer.

[0047] In step 3, the processor provides different types of screens on the display device depending on the operating status of the tractor trailer.

[0048] Specifically, when the trailer state is in the no-trailer mode, there is no trailer attached to the tractor-trailer, and the processor no longer monitors the angle at which the trailer axis deviates from the tractor axis; at the same time, because there is no trailer attached, the processor can no longer acquire the image signals collected by the left rear camera and the right rear camera; at this time, the processor directly acquires the images in the left front camera and the right front camera and transmits them to the display device, providing the driver with images corresponding to the field of view of the left and right main exterior mirrors and the left and right wide-angle exterior mirrors.

[0049] When the trailer state is in trailer present mode, a trailer is attached to the tractor-trailer, and at the same time, the processor uses a sensing device to determine the driving state of the tractor-trailer, and when the tractor-trailer is moving straight, or when the sensing device detects that the angle between the trailer's axis and the tractor's axis is less than 15 degrees, the processor determines that the tractor-trailer is moving straight, and when the sensing device detects that the angle between the trailer's axis and the tractor's axis is 15 degrees or more, the processor determines that the tractor-trailer is turning.

[0050] When the processor determines that the tractor trailer is traveling straight, the processor directly captures images from the left front camera and the right front camera and transmits them to a display device, providing the driver with images corresponding to the fields of view of the left and right main exterior mirrors and the left and right wide-angle exterior mirrors.

[0051] When the processor determines that the tractor-trailer is in a turning state, it continues to detect the angle at which the axis of the trailer deviates from the axis of the tractor, and controls the screen displayed on the display device according to the detected angle.

[0052] Specifically, take the case where the video screen of the left display unit is generated during the process of the tractor trailer turning right or reversing right as an example. When it is detected that the angle of the trailer axis off from the tractor axis is 15-30 degrees, the processor first obtains the video screen collected by the left front camera and the left rear camera, and sends the video screen to the storage unit for caching. Then, the processor respectively cuts out the video screen collected by the left front camera and the left rear camera according to the angle of the trailer axis off from the tractor axis, and the left front camera Of the collected video screen, one-third of the width of the video screen that is blocked by the trailer and forms a blind spot on the right side is cropped out, and of the video screen collected by the left rear camera, only one-third of the width of the video screen that is closest to the trailer on the right side is reserved, the cropped and reserved screen is cached in a storage unit, and the video screen reserved by the left rear camera is again added to the position where the video screen was cropped by the left front camera and composited into the same video screen, and finally, the video screen composited by the left front camera and left rear camera is sent to the location where the display system displays the viewing screen on the left side.

[0053] When it is detected that the angle at which the trailer axis deviates from the tractor axis is 30-45 degrees, the processor first obtains the video screens collected by the left front camera and the left rear camera, and sends the video screens to the storage unit for caching; then, the processor cuts out the video screens collected by the left front camera and the left rear camera according to the angle at which the trailer axis deviates from the tractor axis, and cuts out two-thirds of the width of the video screen collected by the left front camera, the right side of which is blocked by the trailer, forming a blind spot; and for the video screen collected by the left rear camera, only two-thirds of the width of the video screen, the right side of which is closest to the trailer, is reserved; the reserved screens are cached in the storage unit after being cut out; and again, the video screen reserved by the left rear camera is added to the position where the video screen was cut out by the left front camera and synthesized into the same video screen; and finally, the video screen synthesized by the left front camera and the left rear camera are sent to the location where the display system displays the viewing screen on the left side.

[0054] If the processor detects that the trailer axis is more than 45 degrees off the tractor axis, it will directly capture the left rear camera image and send it to the display system to display the left viewing image.

[0055] In the process of the tractor trailer turning right or reversing right, there are two situations when the video screen of the right display unit is generated. The first situation adopts a fixed-size compensation screen. First, when the processor determines that the vehicle turning angle exceeds the illumination angle of the right front camera due to the angle of the trailer axis deviating from the tractor axis, the processor obtains the video screen collected by the right front camera and the right rear camera, and sends the video screen to the storage unit for caching; then, the processor reads the video screen collected by the right front camera and the right rear camera and stores it in the storage unit. Then, for the video screen collected by the right front camera, the video screen of the width occupied by the trailer in the right screen is cut out, and for the video screen collected by the right rear camera, only the video screen of the width occupied by the trailer in the right screen is reserved, and the reserved screen after cutting out is cached in the storage unit, and the video screen reserved by the right rear camera is again added to the position where the video screen is cut out by the right front camera to be composited into the same video screen, and finally, the video screen composited by the right front camera and the right rear camera is sent to the place where the display system displays the right viewing screen. If the processor determines that the turning angle of the vehicle does not exceed the illumination angle of the right front camera due to the angle at which the trailer axis deviates from the tractor axis, it does not compensate and directly sends the video screen collected by the right front camera to the place where the display system displays the right viewing screen.

[0056] The second situation adopts a method in which the compensation size corresponds to the turning angle, and the specific compensation method corresponds to the video screen generating method of the left display unit.

[0057] If the tractor-trailer is turning left or backing left, respond to the above conditions.

[0058] In a preferred implementation, each imaging device adopts an automatic exposure adjustment method to adjust exposure conditions when performing video acquisition, and the adjustment step includes: Step 1 of collecting data with a photosensitive sensor to generate a current photosensitive parameter; Step 2: adjusting an exposure mode according to a relationship between the current photosensitive parameter and a first set value; Step 3, in a fixed exposure mode, adjusts a counter state based on a comparison value between the current photosensitive parameter and a second set value; and step 4 of adjusting the counter state and the exposure mode based on the count value and a third set value.

[0059] Specifically, firstly, the current photosensitive parameters determine the mapping range, and the data collected by the photosensitive sensor is used to calculate the current photosensitive average value by the control unit, and then analog-to-digital conversion / digital-to-analog conversion is performed to determine the currently mapped ADC value.

[0060] The first setting value is used to determine whether image acquisition should be performed using the currently mapped ADC value and the auto exposure mode or the fixed exposure mode. The exposure mode adopts the auto exposure mode in the default state, and if the ratio of the currently mapped ADC value to the mapping range is less than the first setting value, the fixed exposure mode is activated, and if the ratio of the currently mapped ADC value to the mapping range is equal to or greater than the first setting value, the auto exposure mode is maintained. Typically, the first setting value is 28%-33%, and since the auto exposure adjusts the brightness based on the intermediate gray scale, if the ratio of the mapped ADC value to the mapping range is less than 33%, the screen display in the auto exposure state becomes dark. In the fixed exposure mode, the exposure time is typically 9ms to 20ms.

[0061] When the fixed exposure mode is activated, the relationship between the ADC value and the second set value is continuously monitored, and when the proportion of the ADC value in the mapping range is equal to or greater than the second set value, the counter count is incremented by 1, and when the proportion of the ADC value in the mapping range is less than the second set value, the counter count is cleared to zero.

[0062] The magnitude of the counted value is compared with a third set value, and if the counted value reaches the third set value, the counter is cleared to zero, the fixed exposure mode is turned off, and the automatic exposure mode is adopted; if the counted value does not reach the third set value, no operation is performed, and the above operation is repeated.

[0063] The vehicle is outdoors and automatically turns on the auto exposure mode when started up, when the vehicle enters a dark environment from a bright ground, the brightness of the light collected by the photosensitive sensor is converted by analog-to-digital conversion and digital-to-analog conversion, the current ADC value is 10% of the full mapping range and is lower than the first set value, the first set value is 28%-33%, which may be 28%, 30% or 33%, the first set value is the ratio of the current ADC value to the mapping range, the control unit sends a command to the screen collection device to turn on the fixed exposure mode and turn off the auto exposure mode, the exposure time is 11ms, and the exposure time may be 9ms-20ms.

[0064] When the vehicle passes through successive tunnels from a bright place, the interval between the tunnels is short, and the tunnels are dark. When the vehicle enters the dark tunnel, the current ADC value collected by the photosensitive sensor is 9% of the mapping range, which is lower than the first set value, so the control unit sends a command to the screen collector to turn on the fixed exposure mode and turn off the automatic exposure mode, and the exposure time is 11ms. When the vehicle leaves the tunnel and enters another tunnel, the current ADC value is still in the fixed exposure on mode, and the current three consecutive ADC values ​​collected by the photosensitive sensor account for 61%, 67% and 33% of the mapping range respectively, where the proportion values ​​of the first two times are higher than 60% of the second set value, and the counter continues to count but has not yet reached the three times defined by the third set value, and the third time is lower than 60% of the second set value, so the counter is cleared to zero and is still in the fixed exposure mode.

[0065] When the vehicle returns from the dark environment to the bright ground, it is currently still in the fixed exposure on mode, and the current ADC value collected by the photosensitive sensor is 61% of the mapping range, which is higher than the second set value of 60%, and the second set value is in the range of 54%-60%, which may be 55%, 58%, 60%, and the counter counts, and when the vehicle returns to the bright ground for a certain period of time, i.e., the count value of the counter reaches the third set value of 3, the third set value may be 2, 3, 4, and the counter is cleared to zero, and the control unit sends a command to the screen collector to turn off the fixed exposure mode and turn on the automatic exposure mode.

[0066] The brightness is high in the middle section between two nearby tunnels, but due to the relationship between the tunnels, the brightness when entering and exiting the tunnel is transient, and the brightness at that point is lower than the brightness outside and higher than the brightness inside the tunnel. In general, the proportion of the mapping range occupied by the ADC value mapped by the collected brightness is 38%-50%. When the second setting value is set to 38%, the proportion of the mapping range occupied by the ADC value mapped by the continuously collected brightness is 28%, 30%, 40%, 61%, 67%, 44%, 30%, and 27%, respectively. At this time, the camera will activate the automatic exposure mode within a short time of exiting the tunnel, and then switch to the fixed exposure mode.

[0067] The vehicle, which has returned to the auto exposure mode, passes through a dark tunnel, and the current ADC value is 33% of the mapping range, which is higher than 30%, so the control unit sends a command to the screen capture device to adopt the auto exposure mode.

[0068] The present invention is not limited to the above specific embodiments, and those skilled in the art can adopt various other specific embodiments to implement the present invention according to the contents disclosed in the examples and drawings. Therefore, any design that adopts the design structure and concept of the present invention and makes some simple modifications or changes shall fall within the protection scope of the present invention.

Claims

1. 1. A display system for compensating for blind spots in a rearview mirror of a tractor trailer, comprising: The display system includes a collection device, a sensing device, a display control device, and a display device; The collecting device includes a left front camera, a right front camera, a left rear camera, and a right rear camera, the left front camera and the right front camera are respectively installed on the left and right sides outside the cab, and the left rear camera and the right rear camera are respectively installed on the left and right sides of the trailer, The sensing device is an angle sensing sensor installed on a tractor trailer, the angle sensing sensor being used to obtain an angle at which an axis of the trailer deviates from an axis of the tractor; The display control device is connected to the collection device and the sensing device and includes a processor, and the processor is used to obtain an angle at which the axis of the trailer deviates from the axis of the tractor by the sensing device, and to directly transmit the screen obtained by the collection device to the display device, and / or to transmit it to the display device after performing a cut-out and synthesis process; A display system for compensating for blind spots in the field of view using the rearview mirror of a tractor trailer, wherein the display device is a display installed in the driver's cab, the display device is connected to the display control device, and is used to display an image output by the display control device to the driver.

2. The display system of claim 1, characterized in that the field of view of the screen collected by the left front camera and the right front camera should conform to the Class II field of view and / or Class IV field of view specified in the standard, and the screen collected by the left rear camera and the right rear camera should include the trailer body on the side where the camera is located.

3. The display system according to claim 1, characterized in that the display device includes a left display unit and a right display unit, the left display unit being used to display an image collected by the left front camera or an image synthesized by the left front camera and the left rear camera, and the right display unit being used to display an image collected by the right front camera or an image synthesized by the right front camera and the right rear camera.

4. The display system of claim 1, characterized in that, in cutting out and compositing the screens, the processor removes a screen of a preset width or a fixed width in the opposite direction of turning and / or on the trailer blocking side of the front camera in the turning direction, reserves a screen of a preset width or a fixed width in the opposite direction of turning and / or on the trailer side of the rear camera in the turning direction, and places the reserved screen in the location where the screen was removed to perform a new screen composition.

5. When the angle sensing sensor detects that the angle between the center axis of the tractor and the trailer exceeds the illumination angle of the front camera of the turning side, the processor adopts the fixed width image cropping and synthesis; The display system according to claim 4 , wherein the fixed width is a width that the trailer occupies within a screen of a front camera on the steering side.

6. The display control device includes a storage unit, in which a display block diagram X to be displayed on a display device and a set of compensation block diagrams Y and Z for compensating a screen are stored; the processor performs calculations and judgments according to sensing by the sensing device, and calls up a compensation block diagram Ya that matches a preset width in the set of compensation block diagrams Y stored in advance and a compensation block diagram Za of a fixed width or a preset width in the set of compensation block diagrams Z; The processor cuts out a preset width of the screen on the trailer side of the rear camera in the opposite direction of the turning direction, and projects the processed screen onto the called compensation block diagram Ya; 2. The display system according to claim 1, wherein the processor cuts out a fixed width image on the trailer side of the rear camera in the turning direction and projects the processed image on the called compensation block diagram Za.

7. The preset width is set according to an angle between the centerline of the tractor and the trailer; When the angle between the tractor and trailer axis is 0-15 degrees, the preset width is 0; If the angle between the tractor and trailer axis is 15-30 degrees, the preset width is one third of the screen width; If the angle between the tractor and trailer axis is 30-45 degrees, the preset width is two-thirds of the screen width; 7. The display system of claim 4 or 6, wherein the preset width is full screen width when the angle between the tractor and trailer axis is 45 degrees or greater.

8. 2. The display system of claim 1, wherein one or more of the left front camera, right front camera, left rear camera and right rear camera adopt a tilt-shift camera, the tilt-shift camera including a camera body, a screen collection unit and an imaging lens, the imaging lens and the screen collection unit are mounted on the front end and rear end of the camera body, respectively, the camera body has a first angle θ tilted toward a first direction with respect to a reference direction, the screen collection unit and the imaging lens have a second angle α and a third angle β with respect to a main axis of the camera body, respectively tilted in a direction opposite to the tilt direction of the camera body, the second angle α and the third angle β are different from each other, and preferably the second angle α is greater than the third angle β, such that the screen collection unit and a plane on which the imaging lens is located intersect to form an acute angle.

9. The processor includes: a step 1 for acquiring video images collected by a plurality of cameras in real time; The processor includes a step 2 of acquiring an operating state of the tractor trailer; The processor includes step 3 for generating an optimal screen of an unobstructed and minimized blind spot scheme according to the operating state of the tractor trailer; The operating state includes a trailer state and a driving state, the trailer state includes a trailer mode and a trailerless mode, and the driving state includes straight ahead, turning, and reverse; The display method for compensating for blind spots in the field of view using the rearview mirror of a tractor trailer based on the display system described in any one of claims 1 to 8, characterized in that the multiple cameras include a left front camera, a right front camera, a left rear camera and a right rear camera, the left front camera and the right front camera being installed on both the left and right sides outside the driver's cab, respectively, and the left rear camera and the right rear camera being installed on both the left and right sides of the trailer, respectively.

10. In step 3, in the trailerless mode, the processor provides a screen to a display device; In the trailer mode, when the driving state is straight, the processor provides a screen of a left front camera and a right front camera to a display device; In the trailer mode, when the driving state is turning or reversing, the processor provides a front camera image on the turning side and a cut-out and composite image on the opposite side of the turning side on the display device according to an angle between the center line of the tractor and the trailer; In cutting out and compositing the screen in the opposite direction of the turn, the screen of the preset width on the trailer blocking side of the front camera in the opposite direction of the turn is removed, the screen of the preset width on the trailer side of the rear camera in the opposite direction of the turn is reserved, and the reserved screen is placed in the place where the screen was removed to perform new screen composition; In cutting out and compositing the screen in the turning direction, if the angle between the axis line of the tractor and the trailer exceeds the irradiation angle of the front camera on the turning side, the fixed width screen on the turning side of the front camera in the turning direction is removed, the fixed width screen on the trailer side of the rear camera in the turning direction is reserved, and the reserved screen is placed in the location where the screen was removed to perform new screen composition; The display method according to claim 9 , wherein the fixed width is a width that the trailer occupies within a screen of a front camera on the turning side.

11. The preset width is set according to an angle between the centerline of the tractor and the trailer; The display method according to claim 10, characterized in that, when the angle between the tractor and trailer axis lines is 0-15 degrees, the preset width is 0, when the angle between the tractor and trailer axis lines is 15-30 degrees, the preset width is 1 / 3 of a screen width, when the angle between the tractor and trailer axis lines is 30-45 degrees, the preset width is 2 / 3 of a screen width, and when the angle between the tractor and trailer axis lines is 45 degrees or more, the preset width is a full screen width.

Images